Using Ansible for Management of a FWD Ubuntu System¶
- Using Ansible for Management of a FWD Ubuntu System
- Configuring the TARGET System
- Configuring the CONTROLLER system
- How the Target System is Configured
- Future Improvements
The VM management
bzr repository at https://proj.goldencode.com/downloads/automation/fwd_mgmt/ provides scripts to automate the configuration of an existing Ubuntu 16.04 machine, to be used as a FWD conversion and/or application server machine.
The scripts are written for an open source systems management tool called Ansible.
To use these scripts, there will be two systems:
TARGETsystem, which will be configured by the Ansible scripts
CONTROLLERsystem, on which the Ansible scripts will be executed
For these instructions, substitute the following placeholders in each command:
<server_hostname>This is the hostname of the target system (VM) on which you are installing.
<admin_username>This is the admin user account on the target system (VM) on which you are installing.
This process is used to create systems that can be used to deploy build, test or production systems. This can be used on physical systems or on virtual machines. The result is designed for and tested on a single server system. There is no requirement for a Graphical Desktop environment (e.g. Gnome or KDE) to be installed.
The following components are installed:
- PostgreSQL Database Cluster
- Imported database instance for the application database (only a single DB instance is supported at this time). This is optional.
- FWD Application Server
- Converted Application (converted binaries and the runtime configuration). Optionally the conversion environment is fully setup and executed.
- FWD Web Client and the Spawner
- Standard Embedded Mode Web Application
This is sufficient to run remote FWD clients (ChUI, Swing GUI, locally installed web clients), ChUI clients and both kinds of GUI web client. If the system does have a graphical desktop environment, then it would be possible to run any of the interactive clients (e.g. Swing clients) there as well.
Ubuntu 16.04 System¶
This process does not install the Ubuntu operating system itself. It is assumed that there are two existing systems (or virtual machines) running Ubuntu 16.04. While it is possible to run the automated installation on a TARGET Ubuntu 16.04 system that has had other software installed and configured, starting with a fresh install will ensure that the process will work. The controller is not sensitive to this same issue.
The application must have an established project configuration. This includes the conversion tool configuration and scripts for conversion as well as the configuration needed for executing the converted result.
The project configuration DOES NOT include the application's source code.
To create a project configuration for an application, start with ChUI Hotel Project as a Template or GUI Hotel Project as a Template. These provide a project configuration that can be customized for your project.
If a system is needed for exploration and prototyping purposes, the two sample projects can be used as is (ChUI Project Configuration, see Hotel ChUI Demo Application or GUI Project Configuration, see Hotel GUI Demo Application).
In short, to create a runtime system you must have an existing project configuration or one of the FWD sample projects. The project configuration must be zipped up without any top level directory included in the zip. This must have the same folder structure as the Hotel demo projects. The zip will be extracted to
/opt/<appname>/ folder in the
In the instructions below, the project configuration zip will be referred to as
By default the process will download the latest FWD code. You may override this to use a specific version. The public versions available can be found in Downloads. In the instructions below, the FWD zip will be referred to as
In the case where you are making edits to the FWD project and need to build this zip for your customized FWD distribution, the zip file must have a root
p2j/ folder. It will be extracted to
/opt/ and a symbolic link
/opt/<appname>/p2j/ will be created, targeting the
There are two ways to deploy an application:
- Put the 4GL source code on the TARGET system and run conversion. This is called
convert_in_placemode. This is useful for development and test systems.
- Run conversion on some other system and gather the converted results (no source code) for deployment to the TARGET system. This is called
deploy_preconverted_resultmode. This is most common for production systems.
These two modes are mutually exclusive: you can either deploy for conversion or deploy a preconverted application.
Convert From 4GL Source on the TARGET System¶
The following inputs are needed when running
Create a zip file that includes all necessary 4GL source code and resources (e.g. icons, bitmaps). In the instructions below, the source code zip will be referred to as
<fwd_app_code_zip>. The zip will be extracted to the
The source must be zipped up without any top level directory included in the zip. The structure of the source tree inside should contain all the directories that would normally be present when running or compiling in OpenEdge. The project configuration will have a PROPATH defined that references these locations.
Create a zip file that includes all necessary 4GL schema files (in
.df form). In the instructions below, the schemata zip will be referred to as
<fwd_app_schemata_zip>. The zip will be extracted to the
The source must be zipped up without any top level directory included in the zip.
This is only needed if you are converting from 4GL source code on the TARGET system.
This is an optional zip file which contains any patches to apply to the 4GL source code. It would also include a patch script named
patches.sh. Without the
patches.sh, nothing will be done. Normally, this would be coded to use the
patch utility to edit source files based on patch files created with
In the instructions below, the patches zip will be referred to as
<fwd_app_patches_zip>. The patches must be zipped up without any top level directory included in the zip. If this exists, it will be extracted to the
/opt/<appname>/abl/ folder and Ansible will execute the
patches.sh script. The
patches.sh script must reside in the top level of the zip, such that it will be found as
/opt/<appname>/abl/patches.sh when it is extracted.
This is only needed if you are converting from 4GL source code on the TARGET system.
If your project uses ADM or ADM2, then it will need to convert with the Possenet version of the ADM/ADM2 framework instead of the version that is included with OpenEdge. There are other use cases where this code may be needed. Please see Possenet for details.
Download the latest Possenet code or use a specific Possenet zip. In the instructions below, the Possenet zip will be referred to as
<possenet_zip>. If specified, the Possenet sources will be extracted to
To enable Possenet sources, add the
fwd_possenet=true Ansible option when invoking the installation script (more details below). This is enough to let the Ansible scripts download the latest Possenet code . If you want to use a specific Possenet zip, set the
Deploy Pre-Converted Results¶
deploy_preconverted_result mode, all inputs will be provided in a single zip file.
In this mode you must create a zip file with the jars and conversion artifacts of a previously converted application. The easiest way to do this is to run
ant deploy.dist AFTER you successfully convert (and compile). It will generate the necessary zip file in
In the instructions below, the binary deployment zip will be referred to as
<fwd_app_deploy_zip>. The zip will be extracted to the
If you manually create the zip file, the source must be zipped up without any top level directory included in the zip. It should include these contents:
build/lib/* adm_windows.json name_map.cache ddl/* data/namespace/* cfg/registry.xml
This zip is used to avoid conversion on the TARGET system. The results are run on the TARGET system using the FWD runtime environment.
A zip file containing the dump (OpenEdge database export) of all ABL tables as exported in
.d files. The data dump files must be zipped up without any top level directory included in the zip. In the instructions below, the dump zip will be referred to as
<fwd_app_data_dump_zip>. The content of this zip will be extracted to
If this file is missing, the data import will be bypassed.
First, on the TARGET system, do the following:
1. Login to the fresh Ubuntu Server 16.04 64-bit system.
Make sure you have admin rights on the account you are using (
<admin_username>). You need to be at a terminal (command prompt). If you are accessing this remotely, it will look something like this:
ssh <server_hostname> -l <admin_username>
2. Install some prequisite software.
sudo apt-get update && sudo apt-get install python
If you are prompted for your "sudo password", just use the same password as you use for login. If you are using an AWS VM, it will probably already be setup for key-based authentication and passwordless sudo so it may not be an issue.
3. Copy the following zip files (as needed and described above) to your home folder on the
- All Scenarios
- Conversion on the TARGET System
- Deployment of Pre-Converted Code
4. Move all these files into
/opt/ and set the proper permissions.
sudo mv *.zip /opt/ && cd /opt/ && sudo chmod 0755 *.zip && cd
Next, on the
CONTROLLER do the following:
1. Setup key-based SSH access.
If the target system is not already setup for key-based SSH access, you must enable it. If the
CONTROLLER hasn't any SSH keys built yet, use
ssh-keygen -t rsa to generate them before running this command:
scp .ssh/id_rsa.pub <target_system_admin_username>@<target_system_hostname>:.ssh/authorized_keys2 && ssh <target_system_admin_username>@<target_system_hostname> chmod 0600 .ssh/authorized_keys2
2. Install some pre-requisite software.
sudo apt-get update && sudo apt-get install python ansible bzr bzrtools python-paramiko
3. Check out the FWD-specific Ansible-driven automation project.
bzr co https://proj.goldencode.com/downloads/automation/fwd_mgmt && cd fwd_mgmt
4. Run the
Use the existing
install.sh script to either deploy (via the
deploy_preconverted_result mode) or install for conversion (via
convert_in_place mode) your application.
The syntax for this script is:
./install.sh -srvhost=<host> -adminuser=<username> -mode=[convert_in_place|deploy_preconverted_result] <ansible_opts>...
<host>must be replaced with the target system's hostname
<username>must be replaced with the target Ubuntu system's admin user account name
-mode=convert_in_placeto install for conversion or
-mode=deploy_preconverted_resultto install an already converted application
<ansible_opts>are specified as
opt1=val1 opt2=val2 ... optn=valn- they will be passed directly to the ansible playbook.
Depending on the mode used, only certain files or Ansible options are required.
Depending on how the
TARGET machine is configured, it might take ~20 minutes for ansible to complete the scripts.
It is recommended to do a reboot to pick up any updated kernels that occur during the installation process. The FWD application servers should autostart.
Common Ansible options and associated files¶
TODO: enhance to work with multiple DBs
TODO: enhance to work with multiple user accounts
Each option must be specified via
ansible_opt=value. If the value contains spaces, enclose it in quotes. Unless specified, all options are mandatory.
fwd_install_root- it represents the location where the FWD project and the Application will be installed. If not specified, defaults to
user_acct_name- an user account which will be created and configured on the target system, under which the FWD client will be executed
user_acct_pw- the password for the user account above
fwd_app_name- the application name
fwd_app_cfg_zip- the converted project configuration files, will be unzipped in i.e.
fwd_db_name- the database name.
fwd_app_data_dump_zip- optional, required only if
fwd_import_db=true. Contains a zip (with no folder structure) with the data dump files for the ABL exported database
fwd_distro_url- optional. If you are using a custom FWD installation, the value will look like
file:///opt/<fwd_distro_zip>. Otherwise, the latest FWD will be downloaded.
pg_mode- optional, defaults to
true. This allows to enable import to Postgresql (when
true) or to H2 (when
db_port- optional, defaults to
5432. Specify the PostgreSQL port.
fwd_import_db- optional, defaults to
false. When set to
true, the import will be performed. If this is not specified or if it is explicitly set to
false, then you will need to run import separately (through the normal process for running a data import for your application).
Ansible options and files for
fwd_possenet- optional, defaults to
true, it specifies that the applications requires the Possenet sources.
fwd_possenet_url- optional. If using a custom Possenet zip, specify its value like
fwd_possenet=true, then the Possenet sources will be extracted to the application's
fwd_app_code_zip- the application's legacy code and resources. Will be extracted to the application's
fwd_app_patches_zip- optional. Contains a script named
patches.shand/or other patch files to be applied to
fwd_app_schemata_zip- a zip containing a single file, with the application's exported schema. This file will be extracted as
fwd_convert_app- optional, defaults to
false. It indicates the application will be converted by the Ansible scripts. If this is not specified or if it is explicitly set to
false, then you will need to run conversion separately (through the normal process for running a FWD conversion for your application). WARNING: for large applications, this may take hours to complete.
Ansible options and files for
fwd_app_deploy_zip- specify a zip file name which contains the application's converted Jars and other conversion-generated files.
How the Target System is Configured¶
For a detailed look, you can check the roles targeted by the 'conversion_reference_system.yml' and 'deploy_reference_system.yml' files from the 'fwd_mgmt' project. Following are some high-level details on what the Ansible scripts will perform on the target machine:
- Installs the necessary packages (FWD dependencies) to run and compile FWD.
- Ensure the Ubuntu installation is up-to-date.
- Adds a 'fwd' account under which the FWD server will run (you can modify it via the
fwd_acct_name=<name>Ansible option for
- Installs and patches ncurses, installs the JDK and will build the FWD project in
- Install a postgresql database, version 9.5, and creates a cluster named
fwdon port 5432 (if not otherwise overridden).
- Creates a user account using the
- Installs the application in
/opt/<app_name>(for convert or deploy, depending on the mode).
- Will configure the
/opt/<app_name>/deploy/server/directory.xmlto work on the target system.
gen_certs=truewas passed to
install.sh, it will re-generate the certificates (without the root CA, this can be re-used).
- Configure the target system to automatically launch the FWD application server and the embedded server. These can be controlled via
sudo service fwd startor
sudo service fwd_embedded_web startfrom an admin OS account.
- Configure the firewall to allow connection to the following ports, from outside the target machine:
- 8443 - to access the embedded application clients
- 7443 - for the AW application's virtual desktop clients
- 7449:7459 - to allow the AW web clients to be accessed from another machine. These port must match the
/etc/hostswill be changed, so that all lines which are ending with the
<server_host>, will be set as
<target-machine-ip> <server_host>. This is required so that the target machine will be able to server the web clients properly.
The following items are WIP for the Ansible configuration:
- Enable support of ACME/Let's Encrypt real certificates, to avoid self-signed certificates.
- Add reverse proxy implementation to allow a single port to be used and multiplexed for multiple users. This will allow access to the machine running the application server from machines outside its network (i.e. internet), by having only one port exposed to the internet.
- Firewall configuration for production usage.
- Custom configuration of the FWD clients and users.
© 2004-2017 Golden Code Development Corporation. ALL RIGHTS RESERVED.