Project

General

Profile

Installation

This chapter is out of date and is no longer being maintained. It is only being retained for historical purposes. The replacement chapter can be found in Conversion Handbook Introduction.

FWD is both a set of automated conversion tools as well as a runtime used to provide an environment and set of 4GL-compatible services for the proper execution of a converted application.

The first part of this chapter describes how to install the FWD project for the purpose of converting a 4GL application into a Java application. With these instructions a user can establish an installation on which they can:

  • prepare a Progress 4GL application project for conversion;
  • run the analysis and reporting tools to inspect or otherwise process the 4GL source code;
  • run the automated conversion tools to translate the 4GL source into an equivalent Java application; and
  • build/compile the results of conversion such that the application is ready to execute.

The second part of this chapter is an abbreviated description of how to install the FWD runtime environment. This is necessary in order to run and/or test the resulting converted code. It is important to note that this is not intended to be a comprehensive or complete guide to runtime installation. For complete information regarding runtime installation, please see the FWD Runtime Installation, Configuration and Administration Guide.

This chapter does not describe how to install the FWD project for the purposes of modifying or enhancing the FWD technology. Much of the installation is the same, but there are differences that are not addressed here. For details on FWD development environment installation, please see the FWD Developer Guide.

Finding FWD Files

The FWD project code should be available as either a ZIP format archive file or as a sub-directory accessible via the file system.

Either way, the rest of this chapter will assume that the FWD files with specific path and file names, can be found in some manner. Among many options, the commonly available unzip utility can be used to extract files in the case of the ZIP archive. In the case the code is available directly in the file system, this document assumes you can directly copy the specific files referenced using operating system commands or tools such as cp.

Conversion Tools

Requirement Version Notes
Linux (or another UNIX-compatible operating system) n/a Java processes are normally platform neutral. When applications or tools are written in pure Java, it is possible for such processing to be run on any operating system or hardware platform that supports the proper level of the J2SE (Java 2 Standard Edition) required. At this time, while the tools are written in pure Java, it is expected that there may be certain file-system path dependencies, character/file encoding dependencies and/or other dependencies which would require a Linux or UNIX compatible system on which to run the conversion process. In particular, it is likely that the conversion process cannot be run unchanged on a Windows platform.

FWD has been successfully run on a variety of recent Linux distributions from Ubuntu, SuSE, Red Hat and CentOS. There are no specific dependencies on any particular distribution. It is expected that any UNIX-compatible system would work properly.
Java Development Kit (JDK) 6.0 or later The Sun JDK 1.6 on Linux is a well tested option. Various versions from 1.6.0_04-b12 through 1.6.0_16-b01 have been tested and have been found to work well.

Either the 32-bit or 64-bit version may be used. At the time of this writing, there has been no conversion project so large as to require more than 4GB of memory to successfully complete. For this reason, there is no strong requirement for the 64-bit JVM.

The PATH for the user should be set to include the directory containing the command line executables (e.g. java or javac) of this installation. In Linux this would be the bin/ directory. In other words, this should be the default installation for any shell or command prompt from which the FWD technology will be built or used.
It is also recommended to set the user's CLASSPATH (on Unix/Linux) as follows:
export CLASSPATH=.
Apache Ant 1.6 or later Due to the ANTLR dependencies, version 1.5.3 does not work. Please ensure that Ant can be found via the user's PATH.
p2j.jar n/a All of the FWD conversion tools are stored in a single file called p2j.jar. That file in turn is dependent upon other jar files, which have been specified in the p2j.jar file's manifest. This means that as long as the p2j.jar can be found in the Java CLASSPATH and those other jar files exist in the same directory, the conversion tools will work properly.
Each of the dependent jar files is shipped with the FWD project. It is best to use those versions since they are tested and are known to work. These can be found in the p2j/build/lib/ directory of the FWD archive.
Create a directory for all the necessary jar files. Put the
p2j.jar in that directory and remember that location for use as the -classpath parameter when running the conversion tools.
antlr.jar ANTLR 2.7.4 FWD includes multiple lexers, parsers and tree walkers that are built with ANTLR. The resulting code requires supporting classes from this jar file.

This file can be found in the p2j/build/lib/ directory. Place it in the same directory as the p2j.jar.
commons-beanutils-core.jar Apache Jakarta Bean Utils (2004-12-17) Used for schema namespace and configuration processing.

This file can be found in the p2j/build/lib/ directory. Place it in the same directory as the p2j.jar.
commons-collections-2.1.1.jar Apache Jakarta Collections 2.1.1 (2004-12-17) Used for schema namespace and configuration processing.

This file can be found in the p2j/build/lib/ directory. Place it in the same directory as the p2j.jar.
commons-codec-1.3.jar Apache Jakarta Codec 1.3 (2005-09-06) BASE64 support to encode and decode the Progress 4GL RAW data type to and from a string representation.

This file can be found in the p2j/build/lib/ directory. Place it in the same directory as the p2j.jar.
commons-logging-1.0.4.jar Apache Jakarta Logging 1.0.4 (2004-12-17) Used for logging.

This file can be found in the p2j/build/lib/ directory. Place it in the same directory as the p2j.jar.
FWD Conversion Rule-Set XML Files n/a Once the 4GL source code is parsed into Abstract Syntax Trees, the rest of conversion process is executed in a Java-hosted language environment called TRPL (Tree Processing Language). The TRPL programs are called “rule-sets” and are encoded in XML files. These XML files must exist on the target system and the conversion project configuration will contain a reference to the location of these files.
The files can be found in the FWD project under the p2j/rules/ directory path. Copy the entire set of sub-directories of p2j/rules/ to some location on the target system. Each subdirectory will have one or more XML rule-sets.

The contents of the p2j/rules/ directory can be moved to any location. Remember the path to that directory for use in the configuration of the conversion project.

Runtime Installation

From an abstract perspective, there are three major components of the FWD runtime environment to be installed or setup. While these are logically separate environments, physically the components can reside on the same system or up to three separate systems. The components are the client, the application server and the database server.

The following simplified instructions for each component are not intended to be comprehensive. Rather, these instructions are meant to provide a quick way to get enough of the runtime environment installed to allow a converted application to be executed and tested.

Client

Please see the Client Installation chapter for details.

Application Server

Please see the Application Server Installation chapter for details.

Database Server

At this time, the FWD project supports PostgreSQL and H2 as its backing Relational Database Management System (RDBMS). While PostgreSQL is successfully used in production, at this time H2 is intended to be used only in testing or development environments. Additional database types will be supported as customer demand requires. The following instructions will help you get your FWD database server “up and running” in an environment where you can run and test your converted application, but they are not necessarily appropriate for a production FWD installation. When considering a production installation, please consult the FWD Runtime Installation, Configuration and Administration Guide.

To ensure the behavior of the new system matches that of the old as closely as possible, certain legacy features must be supported by the database server in the converted application. These features require special attention when installing the database server.

For instance, a Progress database by default will sort string data differently than a standard PostgreSQL database for the same locale. To make PostgreSQL sort these data in a similar manner to the pre-conversion system, it is necessary to install a custom locale at the operating system level, which the PostgreSQL database cluster can then use to collate its string data.

Likewise, built-in functions used in the where clauses of 4GL data access language statements (e.g., FOR, FIND, etc.) require special consideration. These functions are converted to user-defined functions within the Hibernate Query Language (HQL) queries used by the converted application. This approach presumes support for user-defined functions exists in the backing database. To provide such support, FWD relies upon PL/Java, an open source project which enables Java as a procedure language within a PostgreSQL database.

Overview

At a high level, the steps required to prepare the PostgreSQL database server are:

  1. Install a custom locale into Linux to ensure text data sort properly.
  2. Install PostgreSQL database server software.
  3. Create a FWD-specific database cluster for use with your converted application.
  4. Create a database user account with which your application will access the database.
  5. Install PL/Java for User Defined Function (UDF) support.

For the H2 database server, the required steps are:

  1. Install p2jspi.jar (from p2j/build/lib) into the Java Extension directory for the target JVM to ensure text data sort properly.
  2. Setup H2 to sort the unknown value high
  3. User Defined Function (UDF) support.
  4. Create the database
  5. Using the H2 server

PostgreSQL Task Specifics

The following sections discuss the PostgreSQL requirements at greater length.

Task:

Install the en_US@p2j_basic custom locale into the Linux operating system of the database server.

Details:

The PostgreSQL database cluster and the H2 database used for a converted application must be initialized using a custom locale which defines how FWD databases should sort textual data in indexes and query results. The FWD project provides a custom locale for this purpose: en_US@p2j_basic, which must be installed into the Linux operating system on the database server machine, before the PostgreSQL FWD cluster is created. This locale is intended to mimic, as closely as possible, a Progress database's default collation behavior using the ISO-8859-1 character set. Only this default case is supported at this time, and dynamic collation changes made by individual 4GL language statements currently are not supported by FWD.

The following instructions will install this onto a Linux system (at this time, there is no analog to this procedure for the Windows operating system).

  1. Ensure the Linux package which provides the libc locale database sources is installed on the target machine.
  2. Run localedef --help and review the bottom of the output to determine which directories contain these source files (as well as the path which contains the system's binary locales). The source will often reside in some derivative of i18n, such as /usr/share/i18n. The binaries will often reside in subdirectories of /usr/lib/locale.
  3. Login as the system's superuser.
  4. Change to the directory containing the charmap files (e.g., /usr/share/i18n/charmaps). Locate the archive containing the ISO-8859-1 charmap definition. Decompress it if necessary (it may be gzip'd or otherwise archived).
  5. Copy the custom FWD locale source file from the p2j/locale subdirectory into the system's locales source directory (e.g., /usr/share/i18n/locales).
  6. Run the locale compiler (the following lines should be emitted as a single command on one line):
    localedef -c -f ISO-8859-1 -i en_US@p2j_basic /usr/lib/locale/en_US@p2j_basic/
    

    If the directory for locale binaries is not /usr/lib/locale, replace this portion of the target path accordingly.

Task:

Install PostgreSQL server and client software, version 8.x (8.2 or higher recommended).

Details:

The PostgreSQL database will serve as the back end to your converted application. The simplest way - recommended for test and development environments at a minimum - is to use your platform's package manager to install a binary distribution of the PostgreSQL server. Be sure to install client software at the same time, which you will need for database administration, testing, ad-hoc queries, etc. This document assumes the psql client is installed, but other PostgreSQL client software can be used as well.

If you have a need to compile PostgreSQL with custom settings, you will need to consult the PostgreSQL documentation, as that requirement is beyond the scope of this document.

In either case, stop before you initdb a new cluster.

Task:

Create a FWD-specific database cluster.

Details:

You will create a cluster which is specifically for use with your converted FWD application. It should be separate from any database cluster used for non-FWD applications, because it will be built atop our custom, en_US@p2j_basic locale described above. Thus, it will collate string data like a Progress database, which may not be appropriate for other applications.

If you installed the PostgreSQL server software from a binary package, the package probably created a default database cluster. We will not use this default cluster, since we have to create a cluster which uses the custom, en_US@p2j_basic locale described above. A locale cannot be applied to a PostgreSQL cluster once it has been created, so we must create a new cluster. In this case, consult the packager's documentation (or the postgresql.conf file if a default cluster was installed) to determine the data subdirectory in which PostgreSQL stores its databases.

If you built PostgreSQL yourself, and installed it from scratch, take note of the data directory you specified. You will need it to initialize a cluster.

PostgreSQL provides the initdb utility to create a new database cluster. We need to specify an extra option to this utility, to override the default locale with the en_US@p2j_basic locale we installed previously:

su - postgres
initdb --locale=en_US@p2j_basic -D {data_dir}

where {data_dir} is your data subdirectory.

Task:

Create a database user, other than the postgres user.

Details:

The converted application uses a single database user for all database access required by application business logic. Thus, each user of the application will not have a corresponding database user account. User and group accounts and application security in general are managed at the application server, not at the database.

The database user you create here should have sufficient rights to log into the database and to read and write normal application data, but should not have superuser rights (i.e., should not be able to create/drop databases, grant/revoke privileges, etc.). The postgres user is not re-used for this purpose, because that account does have superuser rights to the database.

PostgreSQL provides the createuser and dropuser utilities to manage database roles and privileges. Please consult the PostgreSQL documentation for instructions on the use of these utilities and/or the DDL statements used to manage user accounts.

Task:

Install Java Runtime Environment (JRE) 6.0.

Details:

The JRE is required to run PL/Java (see below) on the database server. If you are running a 32-bit version of the PostgreSQL server, you should install the 32-bit JRE. Likewise, if you are running 64-bit PostgreSQL, you should install the 64-bit JRE.

FWD has been well tested with Sun Microsystem JVM implementations. Other implementations may work, but this document is written with the assumption that the Sun JVM is in use.

Task:

Install PL/Java, version 1.4 for Progress-compatible, built-in function support in database queries.

Details:

PL/Java enables the use of Java as a procedure language within a PostgreSQL database. This allows the definition of User Defined Functions (UDFs) implemented in Java, for use in SQL statements. This is necessary for FWD to execute queries containing the equivalent of Progress built-in functions. To ensure the converted application behaves identically to the original application, the FWD runtime environment provides compatible Java implementations of Progress built-in functions, such as LOOKUP, NUM-ENTRIES, and so forth. These functions can be used in where clauses for data access language statements like FIND and FOR, and the conversion process will generate Data Manipulation Language (DML) statements which expect these functions to be executed within the context of those queries at the database server. Accordingly, we need a way to execute the compatible Java implementations of these functions at the database server, which requires Java be available as a procedure language within PostgreSQL.

This architecture is much more efficient than executing the functions at the application server. The latter approach would require that we return a superset of the possible results from the database server to the application server, then execute the built-in functions at the application server to perform the final step of filtering. In most cases, this would involve pushing significant amounts of data between database server and application server - an expensive operation - only to throw away all of those results which do not match the criteria specified by the built-in function portions of the query. With PL/Java, this filtering is performed at the database server instead.

There are two phases of enabling FWD's built-in function support. The first, described in this section, is to install the PL/Java software on the database server. This provides the basic support of defining and enabling Java as a procedure language in PostgreSQL. The second phase is to install a FWD-specific jar file into each database which requires Progress-compatible, built-in function support. That step is described in the Data Migration chapter of this handbook.

PL/Java is a third party, open source project (http://pgfoundry.org/projects/pljava/), which is not shipped with the FWD distribution. It is comprised of a Java component, which is platform neutral, and a platform-specific component, which must be binary-compatible with the version of PostgreSQL you are using. The binary distribution of PL/Java 1.4.0 which is available at the time of this writing is compiled for use with the 32-bit version of PostgreSQL 8.3.x. It can be found in the pl subdirectory of the FWD project.

If you are using a different version of PostgreSQL (including the 64-bit version), you will need to build PL/Java on the machine on which PostgreSQL is installed. Building PL/Java from source can be a bit tricky and is a topic beyond the scope of this document. Please refer to the PL/Java project documentation and the FWD Runtime Installation, Configuration and Administration Guide if you have this need.

Once you have downloaded and unpacked the PL/Java binary distribution (or built PL/Java from source), you will need to configure PostgreSQL to use it. PL/Java provides a native library (pljava.so) which must be made accessible to the PostgreSQL server. You can either copy this file alongside PostgreSQL's other libraries in PostgreSQL's lib subdirectory, or you can tell PostgreSQL where to find it. The latter is accomplished by editing the dynamic_library_path setting in the postgresql.conf file. By default, this setting reads:

#dynamic_library_path = '$libdir'

Change it to:

dynamic_library_path = '$libdir:{pljava_home}'

where {pljava_home} is replaced with the location of the pljava.so file on your database server system. Note that this is a path name, not a file name; it is the directory containing the pljava.so file, not the file name itself. While you are editing postgresql.conf, add the following settings as well (customarily located in the CUSTOMIZED OPTIONS section):

custom_variable_classes = 'pljava'
pljava.classpath = '{pljava_home}/pljava.jar'

where {pljava_home} is replaced with the location of the pljava.jar file on your system.

Next, we need to ensure some environment variables are set for the postgres user (the user which launches the database server). This provides the PL/Java software running in the server's context with the information necessary to find and launch the Java Virtual Machine, in order to execute Java UDFs. These environment variables are most easily set by modifying the shell configuration (e.g., .bash_profile if using bash) for the postgres user, which ensures they are available to the postmaster (the main PostgreSQL server process). Add the following lines to the shell configuration file:

JAVA_HOME={location_of_JRE_installation}
CLASSPATH=
LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/i386:$JAVA_HOME/jre/lib/i386/server

{location_of_JRE_installation} should be set to the top-level directory of the Java 6.0 JRE installed in the previous step.

CLASSPATH is nulled out to ensure the JVM launched by the PostgreSQL back end process does not inherit a default CLASSPATH setting for the machine, but rather relies solely upon the classpath set when installing the p2jpl.jar (discussed in the Data Migration chapter).

LD_LIBRARY_PATH is set to allow PL/Java to find the libraries needed to launch the JVM. Note that we specify the HotSpot server compiler (.../jre/lib/i386/@server@) when setting this variable. This is intended to improve performance with the assumption that a JVM, once launched, is long-lived due to connection pooling. Note also the reference to i386, which presumes a 32-bit JVM. Adjust this setting accordingly if using a 64-bit JVM.

On some Linux distributions, there may be a better place to set environment variables used by PostgreSQL. For example, Ubuntu-based PostgreSQL packages create an environment file alongside the standard configuration files specifically for this purpose. Consult your platform-specific documentation for these particulars.

The changes made in this section require a restart of the database server to take effect.

H2 Task Specifics

The following sections discuss the H2 requirements at greater length.

Task:

Install p2jspi.jar (from p2j/build/lib) into the Java Extension directory for the target JVM.

Details:

FWD uses a custom implementation of java.text.spi.CollatorProvider to enable H2 to sort text data the same way Progress would. This resource is shipped with FWD, within the p2jspi.jar file. It must be installed and made available to the FWD runtime using the Java Extension Mechanism. This provides J2SE's Locale and related classes with access to the custom collation services that are needed by FWD. To enable this support, the p2jspi.jar file must be copied from the $P2J/build/lib directory to the Java Extension directory for the target JVM. For a Sun JVM, this directory is located at $JAVA_HOME/jre/lib/ext. Note that administrator rights may be required, depending upon where the JVM is installed.

On the Ubuntu 9.04 distribution, the $JAVA_HOME corresponds to the path /usr/lib/jvm/java-6-sun/. This path must be found if the JAVA_HOME environment variable is not set. Do not make the mistake of putting the p2jspi.jar into the /usr/lib/jvm/java-6-sun/ext/ directory. That is an empty directory which is NOT used for extensions. The correct directory is /usr/lib/jvm/java-6-sun/jre/lib/ext/ in this example.

Task:

Setup H2 to sort the unknown value high

Details:

By default, for an ascending order, H2 sorts the nulls low (i.e. it places the rows with nulls in the sorted column at the beginning of the result set). This contradicts the Progress behavior, which requires to sort the nulls high when an ascending order is used (i.e. place them at the end of the result set). To solve this, H2 provides a system property - h2.sortNullsHigh - which, if set to true, will determine H2 to sort the null value high.

The recommended way of setting this property is by passing it as a parameter to the command used to start the FWD server (if the database is used in embedded mode) or H2 server (if multiple connections are required to the database): -Dh2.sortNullsHigh=true

java -Dh2.sortNullsHigh=true ...

Task:

User Defined Function (UDF) support

Details:

Being written in pure Java, the Java support for user defined functions is available by default in H2. During M0 part of the conversion, FWD will automatically output the DDL necessary to create the aliases for the user-defined functions, when the H2 dialect is used.

After converting a test program from the $P2J_HOME/p2j/testcases/uast/ folder, the file which contains the needed UDF exports is the file which contains the table DDL and can be found at $P2J/testcases/uast/ddl/schema_table_p2j_test_h2.sql. The UDF exports can be found between the SET COLLATION statement and the first CREATE TABLE statement.

If you don't use the core structure DDL to initiate your database, then these DDL statements need to be extracted and executed by hand, before any table is created in the database.

Task:

Install Java Runtime Environment (JRE) 6.0.

Details:

The JRE is required to run the H2 database server. Depending on the machine used to run the H2 database server, the 32-bit or 64-bit JRE should be installed.

FWD has been well tested with Sun Microsystems JVM implementations. Other implementations may work, but this document is written with the assumption that the Sun JVM is in use.

Task:

Create the database

Details:

In H2, a database is created when the first connection is made, if it does not already exist. The connection which creates the database is also responsible for creating a user and setting its password; until changed, all subsequent connections will have to authenticate using that user's name and password.

See the Data import section of the Data migration chapter for details on how the H2 database can be created during schema installation.

Task:

Using the H2 server

Details:

When the H2 database needs to be accessed in TCP mode, a separate H2 server can be started on the local machine or a remote one. A particularity of H2 is that a database can be located anywhere on the partition. The connection URL will contain relative or full paths to a folder where the database's files reside; this location will need to end with the database's name

To start the H2 server, the following command can be used so that the p2j_test database found in $P2J/testcases/uast/dbs/ folder can be accessed:

cd $P2J/testcases/uast/
java -Dh2.sortNullsHigh=true -cp $P2J/build/lib/h2.jar:$P2J/build/lib/p2jpl.jar org.h2.tools.Server -baseDir dbs/ -tcpAllowOthers -tcp

When starting the server, the -baseDir parameter ensures that only databases which reside in the specified folder can be opened; as only the p2j_test database is located there, only one will be available to be accessed. The -tcpAllowOthers parameter ensures that other connections can be established to this server. The -tcp parameter starts the H2 server only in TCP mode, without the web console. If the web console needs to be started, you can add the -web startup parameter. Another useful startup parameter is the -ifExists parameter, which can be used after the database was created; with this parameter, only connections to existing databases can be established (i.e. no new databases can be created).

The following command can be used to shutdown the H2 TCP server. After the TCP server is shut down, the H2 Server can be terminated by using CTRL-C in the terminal window where the H2 Server was started or by terminating the actual process.

java -cp $P2J/build/lib/h2.jar org.h2.tools.Server -tcpShutdown “tcp://localhost”

Summary

By now, you should have a good understanding of how to set up a functional FWD environment on one or more machines. We've discussed where to find the various components necessary to install and configure a client, an application server, and a database server. At this point, you should be ready to perform reporting and further analysis on a 4GL application, and even to convert it to Java source code and build the result. To get started by converting, building, and running a simple example, see the Quick Start chapter. Details on reporting, converting, building, and running more complex programs, and on migrating data from a legacy application, are provided in later chapters.


© 2004-2017 Golden Code Development Corporation. ALL RIGHTS RESERVED.