Development Environment Setup¶

• Development Environment Setup
  • Introduction
  • System Requirements
    • Hardware
    • Operating System
    • Network Requirements
      • Build
      • Runtime
  • Choosing the Appropriate JVM
  • Installation Process
    • Install the Java Virtual Machine
    • Install NCURSES Development Headers (Linux Only)
    • Install C Compiler and Native Build Utilities
    • Configure JNI Linking
    • Install libffi
    • Install Apache Ant
    • Install Pluggable Authentication Modules (a.k.a. PAM, Linux only)
    • Install unzip - Linux Only
    • Patch NCURSES - Linux Only
    • Patch TERMINFO - Linux Only
    • Install FWD Source Code
    • Build FWD
    • Install FWD Collation Services for H2 Database
    • Database Server
  • IDE integration
  • Subsequent Setup

Introduction¶

FWD is both a set of automated conversion tools as well as a runtime used to provide an environment and set of ABL-compatible services for the proper execution of a converted application.

This chapter describes how to create a FWD development system. This requires an installation of the FWD project and enough of the related runtime components for the purpose building a development environment. This environment is suitable for converting an ABL application into a Java application. This installation described here is also suitable for the purposes of building, modifying, or enhancing the FWD technology.

With these instructions a user can establish an installation in which they can:

• prepare a Progress 4GL application project for conversion;
• run the analysis and reporting tools to inspect or otherwise process the 4GL source code;
• run the automated conversion tools to translate the 4GL source into an equivalent Java application;
• build/compile the results of conversion such that the application is ready to execute;
• build the FWD code to allow use for conversion or runtime;
• add, remove or otherwise modify, enhance or customize the FWD conversion or runtime code;
• execute and test the converted application using the FWD application server and clients; and
• source level debug the converted application and/or the FWD runtime code.

It is assumed that the FWD client, FWD application server and optionally the database server will be run on the same development system as where the FWD project is modified and built. This installation is intended to be for the convenience of the developer. It is not intended to be a production quality installation.

For production installs or or other non-developer installations, please see the Installation, Configuration and Administration Guide.

The instructions in this chapter are a superset of the Building FWD From Source chapter, the Conversion Environment Setup chapter and some of the Application Server Installation and Client Installation chapters.

System Requirements¶

These are the requirements for a FWD development system.

Hardware¶

Please see the Development Environment System Requirements for details on the hardware requirements.

Operating System¶

Java processes are normally platform neutral. When applications or tools are written in pure Java, it is possible for such processing to be run on any operating system or hardware platform that supports the proper level of J2SE (Java 2 Standard Edition) required. From the software point of view, you can install the client on any operating system as long as there is a JVM implementation for it.

At this time, while the vast majority of the client FWD code is written in pure Java, there is a small amount of platform-specific code (written in C) used for character mode user interface processing, process launching, native memory access and native library calls. That platform-specific code is implemented using the Java Native Interface (JNI) and is often referred to as “native” code. A customized version of that native code is included in the FWD project. This code is currently compiled as a 32-bit or 64-bit shared library for Linux (libp2j.so) or Windows (p2j.dll).

Network Requirements¶

Build¶

By default, the FWD build is written to download its dependencies (3rd party Java libraries) when needed. These will be cached locally and reused. The download is done by Gradle and it typically happens at the following times:

• The first time that FWD is built.
• Whenever the cache is cleared/deleted.
• When a new FWD version requires an updated version or a new dependency.

For this download to occur, Internet access is required. Once downloaded and cached, no further Internet access is needed for building FWD.

Runtime¶

It is assumed that the FWD clients and the FWD application server will run on the development system itself.

It is possible to run the application server and the database server on the same system. In that case, there is typically nothing that must be done to enable the network path.

If the database server is run on a separate system and there is a firewall or router in the network path between the application server and the database server, then you must ensure that the firewall is configured to allow the related communication flows to occur.

Choosing the Appropriate JVM¶

The core requirement for the JVM is that it must be compatible with Java 8.

Each FWD component (conversion, application server, client, user-defined functions running at the database) can use a different JVM. These can run on the same system or different systems. They can even run on a different operating system platform. There is no dependency between JVMs of different components, since they are separate operating system processes.

The following JVM versions have been well tested:

• Oracle JDK 1.8 on Linux
• Oracle JDK 1.8 on Windows
• OpenJDK 1.8 on Linux
• OpenJDK 1.8 on Windows

The specific versions of Oracle JDK 1.8.0_101 and OpenJDK 1.8.0_111 have both been tested and have been found to work well. Generally, it is safe to use any later version of the Java 1.8 platform.

In order to build FWD and/or the converted application, you must have a JDK (Java Development Kit). It is not possible to use the JRE (Java Runtime Environment) since that is a Java distribution which lacks the necessary compiler and tools.

Only a 64-bit JDK should be used on a development system.

Installation Process¶

We will assume that the operating system is already installed and up-to-date.

The Linux instructions included here are written (and tested) on Ubuntu 16.04. If the target Linux system is a different distribution, the equivalent commands must be run.

The Windows instructions can be used for any Windows version from Windows XP through Windows 10.

Install the Java Virtual Machine¶

Please check previous paragraphs for details on how to choose the JDK to install.

sudo apt-get install openjdk-8-jdk openjdk-8-source
cd /usr/lib/jvm
sudo rm default-java
sudo ln -s java-8-openjdk-amd64 default-java

export CLASSPATH=.

mklink /D SomePathWithNoSpaces "The target path\to JDK with\spaces inside" 

mklink /D jdk180 "Program Files\Java\jdk1.8.0_121" 

JAVA_HOME=C:\PROGRA~1\Java\jdk1.8.0_121

Install NCURSES Development Headers (Linux Only)¶

The native code on Linux/UNIX is dependent upon NCURSES. In order to properly compile and link that code, the NCURSES development headers must be installed.

In prior releases of Debian/Ubuntu distributions, the ncurses.h header file was present in a standard gcc installation, but recent versions (e.g. Ubuntu 12.10 and later) no longer provide this by default. Make sure to install the libncurses5-dev package to get the required headers.

Install it using sudo apt-get install libncurses5-dev.

Install C Compiler and Native Build Utilities¶

FWD includes native C code that must be built for many of the build targets. The most commonly used target is all, which builds the native components. Building the native code requires a C compiler and native code build utilities.

Configure JNI Linking¶

The JDK installation must be configured to allow linking JNI modules to the shared library that implements the JVM JNI interfaces.

JNI is a bidirectional interface between C/C++ code and the JVM. It is used by FWD:

• FWD has a native library which is used on the FWD Client to access APIs in the operating system to do things that cannot be done directly in pure Java.
• The FWD Spawner (which is called by the FWD Application Server to launch FWD Client JVM instances) loads a JVM using JNI.

sudo rm /usr/lib/libjvm.so
sudo ln -s /usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so /usr/lib/libjvm.so

Install libffi¶

In order to support the calling of native library calls from Java code, FWD depends on Portable Foreign Function Interface Library (also known as libffi). The FWD JNI library is dependent upon libffi.

The following instructions are valid for a system being used for running the FWD Client as well as for building FWD. To support both use cases, you MUST install the development version of libffi. If you do not need to build FWD, you can follow the libffi install instructions for only executing the FWD client.

In all cases, the bitness of this library must match the OS architecture and also the bitness of the JVM used by the FWD clients.

Install Apache Ant¶

The FWD build process (and usually the execution of the various conversion project build tasks) uses Apache Ant (version 1.9 or later).

Install Pluggable Authentication Modules (a.k.a. PAM, Linux only)¶

Pluggable Authentication Modules (PAM) is a flexible mechanism for authenticating users. It is optional and only available on Linux/UNIX. It is used to perform OS-level authentication when the FWD Server launches a FWD Client, via the spawner tool.

This can be easily installed using:

sudo apt-get install libpam0g-dev

To use it during native task build, add this property to the gradlew build command:

./gradlew ... -Dhave.pam=yes

Install unzip - Linux Only¶

The commonly available unzip utility can be used to extract files from a ZIP archive. This may be needed to extract the source code from the FWD archive or from some of the other downloaded software (in the case of Windows).

Patch NCURSES - Linux Only¶

On Linux/UNIX, building the FWD native library libp2j.so requires linking with a customized version of NCURSES. Even if the console ChUI driver is not going to be used on this system, this step is still necessary.

Please see Patching NCURSES for the details.

Patch TERMINFO - Linux Only¶

This step is only needed if you choose to run the client with the native (console) driver.

Please see Patching TERMINFO for the details.

Install FWD Source Code¶

The FWD project code is available as a ZIP format archive file in the downloads section of this site. This archive should include all of the source code and supporting files needed to build the FWD project on a given system.

The archive filename is normally in the form fwd_<branch_name>_revision_<revno>_<date>.zip. For this example, assume that the fwd_trunk_revision_11145_20170324.zip is in your home directory and that the FWD code is to be installed in a directory named ~/projects (Linux) or c:\projects (Windows). Given these assumptions, installation is as simple as the following commands to extract the zip file:

cd ~/projects
unzip ~/fwd_trunk_revision_11145_20170324.zip

cd c:\projects
jar xf c:\PathToBundle\fwd_trunk_revision_11145_20170324.zip

A directory ~/projects/p2j (Linux) or c:\projects\p2j (Windows) will be created with the source tree of the FWD project. For the remainder of the chapter, the environment variable $FWD (Linux) or %FWD% (Windows) will be a reference to that directory. The "p2j" is a historical name for "Progress 4GL to Java". References to that name are still present in the code. Those references will be removed and/or replaced with references to "fwd", but that work is not yet complete.

This can be "saved" into the environment to simplify subsequent commands:

export FWD=~/projects/p2j

Build FWD¶

Build FWD using the all target:

cd $FWD
./gradlew all

cd %FWD%
gradlew.bat all

This will fetch all the Java dependencies, javac compile all Java source code, create the FWD jar files and compile/link the native code.

By default the build script provides only brief log output. If the build fails for any reason use the --info command line parameter to get more detailed log output of the failure, i.e. ./gradlew --info all.

More details can be found in Building FWD From Source.

Install FWD Collation Services for H2 Database¶

Note:
The content of this page is deprecated. It is temporarily kept just for reference on older installations (installations of FWD with versions older than FWD v4, such as the public downloads mentioned above). Please navigate to H2 String Collation Service Provider Installation for the new version of the resource.
______________

H2 is an in-memory database used implicitly by the FWD runtime for legacy temp-table support and for other internal purposes. This means that even if the permanent database servers being used are not H2, a Java-based text collator used by H2 must be installed on the machine where the FWD Application Server is run. If a permanent database is H2 and is on a different machine, then it must be installed there, too.

To ensure that each H2 database collates its string data in the same manner as Progress, a custom implementation of java.text.spi.CollatorProvider is used. This resource is installed and made available to the FWD runtime using the Java Extension Mechanism. This is necessary in order for J2SE's Locale and related classes to have access to the custom collation services that are needed by FWD.

Make sure that you have built FWD already, so that the p2jspi.jar is available.

sudo chmod 0444 /usr/lib/jvm/default-java/jre/lib/ext/p2jspi.jar

For the temp-table and dirty databases, the server will automatically default to use the en_US_P2J collation.

To set this value for permanent databases or to override the default for the temp-table and dirty databases, requires a configuration entry in the FWD directory. Please see the Directory and Directory Configuration Reference chapters for more information on the FWD directory.

The directory node path is /server/standard/database/{database_name}/p2j/embedded-collation. The default setting should be en_US_P2J, as in:

<node class="container" name="p2j">
    ...
  <node class="string" name="embedded-collation">
    <node-attribute name="value" value="en_US_P2J"/>
  </node>
</node>

Currently, this custom collation service provider only exists for the ISO-8859-1 character set. If you need to support Progress-like collation for a different character set, you most likely will need to provide a new, custom implementation of java.text.spi.CollatorProvider, which requires special preparation/packaging. Please refer to the Internationalization chapter for additional information.

When using H2 for a permanent database (embedded or not), if the core structure DDL generated during M0 is not used to create the tables, then the collation must be set manually before any table is created. Just execute the following command with the database:

set collation en_US_P2J;

The default setting for the collation can be found in the p2j.cfg.xml file, under the cfg/schema/namespace/dialect-specific/collation node. This value is used during M0 to generate the set collation DDL for the H2 dialect.

<schema metadata="standard" >
   <namespace name=”...”>
...
      <dialect-specific name="h2">
         <parameter name="collation" value="en_US_P2J" />
      </dialect-specific>
   </namespace>
</schema>

Currently, only the en_US_P2J collation is supported.

If the p2jspi.jar file is installed properly and the FWD directory has the embedded-collation property set correctly, the FWD runtime environment will initialize the embedded H2 databases to use the en_US_P2J collation upon server startup.

If the JVM cannot find the en_US_P2J collation rules upon server startup, the following error message is issued to the server log:

Error setting P2J-specific collation;  please ensure p2jspi.jar is installed in Java extension directory

This is a fatal error, which indicates that the p2jspi.jar file is not properly installed as a Java Extension for the server's JVM. In this case, double check that you have installed the jar file for the JVM actually being used to launch the FWD server (avoiding a common mistake when multiple JVMs are installed on a development system), and ensure that you have copied the jar to the correct ext subdirectory (since the HotSpot JVM misleadingly has two of them, as discussed above).

Database Server¶

The FWD project can use multiple databases as its backing Relational Database Management System (RDBMS). If you will be working on modifications to the persistence layer of FWD or if you are otherwise going to implement code that is directly using the database, then it may be important to have a local installation of the database in question. Implementing such a scenario is straightforward.

It is important to note that it is easy to use the H2 database (that is built-in to the FWD server) as a database server for your application's imported Progress data. This is less work and has many advantages for a development system. H2 is not recommended for production use, but can be successfully used for testing or development.

The installation and setup of a local database server is not necessary in the following cases:

• The Progress 4GL code does not access any database features.
• The Progress 4GL code's use of database features is limited to temp-table (or work-table) usage.
• The necessary test database(s) are accessible over the network on another system.

If any of the above conditions exist, then this section's instructions can be skipped.

The following chapters can be used to setup a local database installation:

PostgreSQL on Linux
SQLServer on Windows
H2 - any platform

IDE integration¶

To simplify integrating the project into an IDE the build contains the task fetchLibs, which resolves all the Java dependencies and stores them locally.

Subsequent Setup¶

The instructions in this chapter are sufficient for getting a running FWD development environment. It is important to understand the tasks that may be necessary in addition to the steps documented here. This chapter does not document any steps that setup a specific application or database. The following tasks may be necessary to provide an environment that supports a specific application conversion project or imported test database. It is likely this additional work is necessary to constitute a complete test or development environment that allows work on a specific converted application. Please use the referenced documentation to accomplish these tasks.

© 2004-2017 Golden Code Development Corporation. ALL RIGHTS RESERVED.