Project

General

Profile

Wiki » Documentation » Conversion Handbook »

Gathering Inputs

In order to convert a Progress 4GL application, the application source code, schemata and other inputs will be required. These must be gathered so that they can be placed into a well-defined structure in the conversion project.

Source Code

The most important (and obvious) input is the source code for the application. There are two types of source code: programs and includes.

First, there are the programs in the application:

  • External Procedure Files - In a traditional Progress 4GL environment (both in a 4GL client or in an appserver), these are all the external procedure files. External procedures often have an extension of .p or .w, but there is no real limit to the naming convention that may be used.
  • Trigger Procedures - Any source programs which start with a TRIGGER-PROCEDURE which is referenced in the schema is a live program.
  • Classes - Any object oriented classes in use or referenced (these are always .cls files) are actively used.
  • WebSpeed - If WebSpeed is in use, these are the HTML files (perhaps including SpeedScript Embedded 4GL or HTML mapping).

The following factors are important to remember:

  • All programs that are in use must be gathered in the file set.
  • All files that are referenced (e.g. in a RUN statement) in any way are considered "in use".
  • The version that should be provided must be the original source code that has not been run through the Progress 4GL preprocessor (or the WebSpeed E4GL preprocessor). A better result is obtained without providing the source code in preprocessed form since during the conversion, the FWD preprocessor saves hints about its actions that are used downstream to make better conversion decisions.
  • Compiled (r-code) and/or procedure libraries cannot be used. If the code is part of the application, you must include the source files instead of the compiled files.
  • If ADM/ADM2 is in use, the version of these procedures from the Possenet project will be used. Any version of ADM/ADM2 source code released under the Possenet license is legally safe to include.
  • Do you require 3 rd party 4GL code in order to run properly? Those files must be converted or replaced in the resulting system. You will need to check on the licensing for those files if you wish to convert them. If the licensing allows you to create a derivative work of that code (modify/translate the code and distribute/use the result), then you can include those files in this project. Do not include the files if the license does not give you the rights to make a derivative work. In this case you must remove all refererences to that code.

Second, there are include files referenced by all of the programs. The FWD conversion process has a compatible preprocessor that will handle all include file processing and other preprocessor directives, so it is neither necessary nor desirable to provide preprocessed source code. On the other hand, it is crucial to provide a complete set of include files. In a traditional Progress 4GL environment, these are files with a .i extension. Many applications use .f or .frm for include files that are a frame definition or .tt for include files that are temp-table definitions. There is no specific limit on the file name and any file that is included must be gathered.

It is just as important to have all of the source code gathered for conversion as it would be for building a runtime installation of the application in the Progress 4GL. Since this process does a static conversion of the entire application as a batch, into a replacement Java application, everything that is possibly linked into and executed as part of the application must be present at the time the conversion tools are run.

Do NOT forget the following:

  • All files that are referenced in an {include-file.i} directive must be gathered.
  • Include files can be nested arbitrarily deep. This means your search for includes may be an iterative process where you add files and recheck to see if anything else is missing. This continues until everything is found and no failures occur during compilation (in Progress) or during parsing (in FWD).
  • Any procedures in the application, including WebSpeed or TRIGGER-PROCEDURE files can also reference include files.
  • WebSpeed files can be treated as include files in any 4GL code.
  • Include files referenced in the database schema must be gathered either. For example, a field level VIEW-AS clause or VALIDATE expression.
  • If ADM/ADM2 is in use, don't forget any custom templates or application-specific ADM extensions. These are normally application specific files that are not shipped with ADM/ADM2 itself.
  • Calling 3 rd party code often requires include files. The dependencies must be satisfied or removed. If the 3 rd party code is intended to be converted, then you will need to check on the licensing for those files. If the licensing allows you to create a derivative work of that code (modify/translate the code and distribute/use the result), then you can gather those files in this project. Do not gather the files if the license does not give you the rights to make a derivative work. In such a case you must remove the references in the source code that is being converted.

Database Schemata

Due to the complete integration of database access into the core of the Progress 4GL language, it is not possible to even parse 4GL source code (that has any database usage) without having first parsed all database schemata to which that source file has references. Consider that even something such as a DEFINE VARIABLE x LIKE db.table.field clause is a database reference. In real applications, it would be a very rare source file that didn't have database references.

For each schema referenced in the application, that schema must have been exported from the Progress Data Dictionary in the .df format. To create this export file for a Progess database, follow these steps:

  1. Open the Data Dictionary
  2. If necessary, connect to the target database and make it the working database.
  3. Select from the menu: Admin --> Dump Data and Definitions --> Data Definitions (.df file)...
  4. Select all tables to be migrated to the new database (or use F5 and then specify * and ENTER to select all tables).
  5. Then press GO (generally F1).
  6. Select a target .df filename, use code page ISO8859-1, then choose OK.
  7. When the process completes, there should be a .df file that contains the schema for that database.

Repeat this process for each database schema to be exported. If the application uses multiple copies (or instances) of the same database schema, only one .df file is necessary.

The Progress database supports a set of metadata on each database. Metadata is a set of specially named tables (and fields of those tables) that are normally “hidden” in the Data Dictionary, but which are always there. The Progress database manager uses these tables to maintain the state, statistics and structure that is associated with each database. At least, this is the state/statistics/structure that is accessible using queries within the application itself. Applications that use this metadata, will require an additional “standard” metadata schema to be exported into .df format. Applications with no metadata references will not need to provide this .df file.

To create the metadata schema export file, follow these steps:

  1. Open the Data Dictionary
  2. If not already connected to a database, connect to any database and make it the working database. It doesn't matter which database is used, since each one shares the same metadata.
  3. Select from the menu: Admin --> Dump Data and Definitions --> Data Definitions (.df file)...
  4. Press F7 to show hidden tables (all the metadata tables start with an _ (underscore) character and will now appear at the bottom of the table list).
  5. Press F5 and then specify _* and then ENTER to select all metadata tables.
  6. Then press GO (generally F1).
  7. Select a target .df filename (such as standard.df), use code page ISO8859-1, then choose OK.
  8. When the process completes, there should be a .df file that contains the metadata schema.

While the metadata schema does not change often, it can change between Progress 4GL versions. It is important to export the metadata schema from the same version with which you application is designed to work.

For each of these database schemata (not the metadata schema since that is part of every database), the following additional information is needed:

  1. What is the logical database name for this database?
  2. Is this database connected automatically when the application is started (e.g. connected via a command line parameter)?

Proxygen Artifacts

FWD provides an equivalent to the Proxygen facility for generating Java proxy code based on 4GL .xpxg artifacts.

See ProxyGen and Open Client Usage for more details.

REST and SOAP Artifacts

REST and SOAP support can be converted using REST or Web Services Adapter (WSA) artifacts and/or configuration input. This support is for both Classic Appserver (Web Services Adapter/SOAP Adapter, REST Adapter) and PASOE (SOAP Transport, REST Transport).

  • REST .paar files.
  • Web Services (SOAP) .wsm files.
  • WebHandler configuration .properties files.

For more details, see Converting REST and SOAP.

PROPATH

To compile 4GL code, one must specify the PROPATH. Likewise, to execute 4GL code (whether compiled or not), one must specify the PROPATH. The compile-time and runtime PROPATH values are also needed to convert (equivalent of compile-time) and run that same code in FWD.

Compile PROPATH

This can be ignored if the 4GL application is not compiled.

If the 4GL code is compiled, then the following is needed for each separate set of 4GL programs that are compiled together:

  • The exact list of 4GL program names (external procedures, trigger procedures or classes) which are compiled in the set.
  • The PROPATH used for compilation.
  • The platform on which the compilation is done (e.g. Windows or Linux or UNIX).

Many large applications are composed of distinct parts which must be compiled separately. For example, most GUI applications have a GUI client that is compiled on Windows and has a specific PROPATH (that will include things like ADM/ADM2) and they also often have an appserver portion which will be a different set of programs that are only executed on the server side. That server side code is a separate compilation run with a different PROPATH, possibly a different platform (e.g. maybe Linux or UNIX) and different set of program code to be compiled. It is critical that all compilation runs are identified and the details gathered so that the equivalent conversions can be defined in FWD. There will be one FWD conversion run for every separate compilation in OpenEdge.

Runtime PROPATH

The runtime PROPATH for OpenEdge is needed to configure the equivalent FWD runtime environment. In the 4GL, each session that is started will have its own PROPATH, startup program (external procedure filename). Thus, to configure the equivalent in FWD the following is needed for each separate 4GL session that is executed:

  • The 4GL program name (external procedures) used as the startup program for the session.
  • The PROPATH used for the session.
  • The platform on which the session is run (e.g. Windows or Linux or UNIX).

Most large 4GL applications will have many different 4GL sessions, each with a different startup program and PROPATH, often run on different platforms.

Keyword Ignore List

Since each new version of Progress 4GL may (and often does) bring new keywords, older applications can have conflicts between those new keywords and the user-defined symbols in the code. Progress provides a mechanism to disable any new keywords that conflict with older code. That mechanism is called the keyword ignore list. It is a text file that is created by the application programmer, one keyword per line.

Obtain this file if the application depends upon it. The FWD conversion tools will need to be configured to honor that file. This can be done using the keyword-ignore parameter in the global configuration file p2j.cfg.xml. See the Global Configuration section for details.

Data Export

This input is not needed during the code conversion process. But to put an application into testing or into production, it will be necessary to migrate each database instance which is needed to support such installations. To do this, each database instance to be migrated must have its tables exported in the .d (EXPORT) format.

Please see Data Migration for details on how to export the data. The result would be a dump file for each table of each database instance being migrated. The base filename for each export file must match the DUMP-NAME attribute in the .df file ADD TABLE entry for the corresponding table.

What is the encoding of the data export files (.d data dump files)? This must match the encoding specified in the FWD tools for data import.

Internationalization (I18N)

Applications which use the Progress 4GL default settings for internationalization (I18N) can ignore this step. For other applications it is important to have an understanding of the internationalization requirements. I18N can effect everything from conversion processing to runtime support, from the client to the application server and the database server.

See the Internationalization chapter for details on how to encode these values for conversion.

Conversion Inputs Encoding

What is the encoding of the source files for the application (include files, procedures, classes)? This must be the same value for all files and it must match the encoding used by the FWD tools that are used to parse those files.

What is the encoding of the exported schemata? This should match the encoding of the source files because the same encoding type will be used to process the schemata as is used for the source files.

Translation Manager

TBD: Provide details here about what needs to be gathered for Converting a Translated Project.

Runtime Values

Does the application use any of the following?

  • DBCS
  • UNICODE
  • Translation Manager
  • country/locale-specific empty databases
  • country/locale-specific Progress message files
  • code page conversion features
  • customized character processing tables - character attribute tables, case tables, collation tables, code-page conversion tables (convmap.dat)
  • customized word break tables

What does OpenEdge report for the following:

  • code-page(s) used in the Progress database
  • SESSION:CPSTREAM
  • SESSION:CPINTERNAL
  • SESSION:CPTERM
  • SESSION:CPPRINT
  • SESSION:CPCASE
  • SESSION:CPLOG
  • SESSION:CPRCODEIN
  • SESSION:CPRCODEOUT
  • SESSION:CPCOLL
  • SESSION:DATE-FORMAT
  • SESSION:NUMERIC-DECIMAL-POINT
  • SESSION:NUMERIC-FORMAT
  • SESSION:NUMERIC-SEPARATOR

Process Environment

Some of the configuration or state of a Progress 4GL application is derived from the environment or command line used when the Progress runtime is started.

The most important of these values is the PROPATH. This specifies how external procedures and include files are to be found, so it has a huge impact on how the source code is preprocessed and how procedures link to one another (e.g. via the RUN statement). This is especially important when all references to include files or procedures are not relative to the root of the project's source code tree. In such a case, the PROPATH is essential for resolving otherwise ambiguous filenames. For example, the preprocessor may not find all include files if the PROPATH is not right, which could make the 4GL syntactically incorrect or logically incomplete (even if the code does syntactically parse properly). See above for the requirements about the PROPATH.

In addition, the default processing of the Progress environment can be modified using command line options (or a profile or .pf file with options). The list of these options needs to be known. These values can normally be found by reviewing all startup scripts or command lines used to launch the environment. .pf files can be "nested" to an arbitrary level. It is important that the entire "graph" of .pf files is gathered.

In a GUI application, any .ini files should be gathered. These usually contain font definitions, color definitions, as well as defining other options or configuration.

A list of all Windows Progress-specific or application-specific registry keys/values that are set.

All scripts, batch files or other launcher programs should be gathered. Consider that the application may have batch programs, appserver or WebSpeed brokers/agents and multiple kinds of clients, all of which might use different launching mechanisms.

Appserver, WebSpeed and DataServer configuration files (e.g. the ubroker.properties).

Finally, any environment variables that are read from 4GL code should be known and the possible values should be enumerated. This is also often something that can be found by reviewing startup scripts, command lines and user/global profile scripts. These variables and values are not likely to be needed for properly converting the source code, but it is very likely that they will be needed when configuring the runtime environment for executing the converted application.

Terminal Configuration

For character user interface (ChUI) applications, the terminal types used and the terminal configuration have an important effect on how the application will behave. In a graphical user interface (GUI) environment, these can be ignored.

Gather the set of supported protermcap files. These files provide a mapping between each terminal type's features and such things as colors or the keys that will be reported to the application as input.

Make a list of all the supported terminal types that must be certified to work in the converted environment.

Neither of these are needed for running the code conversion or for handling data migration. Instead, these are used to customize the FWD client's runtime environment to match the expected behavior for the supported terminals.

.wrx Files

Any use of OCX controls (a.k.a. Active-X controls) in 4GL code requires a corresponding .wrx file. To convert "live" OCX usage properly, these .wrx files are needed in FWD as well.

Static Resources, Templates and Configuration Files

GUI applications usually have images/bitmaps, icons, fonts that need to be present for the application to run. Likewise most applications have other dependencies such as document templates, configuration or data files. These will be needed at runtime and some of them may be best included at conversion time so that they can be stored/distributed in the converted application .jar file(s). This is especially useful for graphical resources such as images or icons, since these are statically known at compile/conversion time and they should always be present with the application. Distributing them separately is not a good practice.

HLC Source Code

If the application uses HLC (the High Level Call interface) to integrate 4GL code with native (usually C language functions) code, then it is important to review the HLC source code. The HLS source code is compiled into a static object using Progress 4GL header files and calling 4GL services to read and write variables. This resulting compiled object code is linked to create a custom Progress 4GL executable which can dispatch CALL language statements to the linked in HLC functions. Everything runs in the same process.

Such an approach will not work in FWD, however that code can be easily migrated to a shared library and the 4GL CALL statements can be rewritten to invoke the shared library functions. This change has the advantage that it can be automatically converted without losing any functionality. Three key points: the HLC code must be available for review; the HLC/4GL code will have to be modified by hand to use the shared library approach; and the FWD conversion/runtime does not yet have support for generic shared library usage.

Checklist

The following checklist can be used to ensure that everything is available:

Input Type Checklist Item Available
Source Code All Progress 4GL external procedure files which are in use or are referenced.  
Source Code WebSpeed HTML files which are in use or are referenced.  
Source Code TRIGGER-PROCEDURE files referenced by the schema.  
Source Code OO class files referenced in the application.  
Source Code ADM/ADM2 customized templates or application-specific extensions.  
Source Code 3 rd party procedures or classes used in the application AND which are expected to be converted (must be appropriately licensed for this to be legal).  
Source Code All application code is not in r-code or procedure library form.  
Source Code All referenced Progress 4GL include files.  
Source Code WebSpeed HTML files that are used as include files by 4GL source code.  
Source Code Any include files referenced in the database schema.  
Source Code 3 rd party include files used in the application AND which are expected to be converted (must be appropriately licensed for this to be legal).  
Database Schemata An exported .df schema for each AND every database (with unique data definitions) that is accessed by the application.  
Database Schemata The logical database name for each .df file.  
Database Schemata The database metadata schema exported in a .df format. This is necessary if the application has ANY references to metadata tables or fields.  
Database Schemata The list of databases that are connected by default when the application starts.  
REST and SOAP Artifacts TBD  
PROPATH For each compilation set, provide the exact list of 4GL program names (external procedures, trigger procedures or classes) which are compiled in the set.  
PROPATH For each compilation set, provide the PROPATH used for compilation.  
PROPATH For each compilation set, provide the platform on which the compilation is done (e.g. Windows or Linux or UNIX).  
PROPATH For each unique 4GL session executed, provide the 4GL program name (external procedures) used as the startup program for the session.  
PROPATH For each unique 4GL session executed, provide the PROPATH used by the session.  
PROPATH For each unique 4GL session executed, provide the platform on which the session is run (e.g. Windows or Linux or UNIX).  
Keyword Ignore List Keyword ignore list.  
Data Export Data dump files in .d format for each table of each database instance to be supported as a test or production application environment.  
Data Export What is the encoding of the database dump (.d) files? This is the codepage used to read these files.  
Conversion Inputs Encoding (I18N) What is the encoding of the source files and schemata? This is the codepage used to read these files.  
Translation Manager (I18N) TBD  
Runtime Values (I18N) List of I18N features used.  
Runtime Values (I18N) Customized I18N data files.  
Runtime Values (I18N) List of code pages used at runtime.  
Runtime Values (I18N) Date format(s) used.  
Runtime Values (I18N) Numeric format(s) used.  
Process Environment The command line or .pf files in use to startup the Progress session.  
Process Environment All .ini files used by the application (GUI only).  
Process Environment A list of all Windows Progress-specific or application-specific registry keys/values that are set.  
Process Environment Any scripts, batch files or other launching programs for the application. Consider that the application may have batch programs, appserver or WebSpeed brokers/agents and multiple kinds of clients, all of which might use different launching mechanisms.  
Process Environment Appserver, WebSpeed and DataServer configuration files (e.g. the ubroker.properties).  
Process Environment The list of ENVIRONMENT variables read by the application and their possible values.  
Terminal Configuration The set of supported and customized protermcap files.  
Terminal Configuration The list of supported terminal types.  
.wrx Files .wrx file for any OCX that is in use.  
Static Resources, Templates and Configuration Files Images, icons and other resources or critical data/configuration files.  
HLC HLC source code if used.  

Test and Development Environment

The developers working on the conversion must have access to an OpenEdge full 4GL development environment. This is a test/development system where the original application can be run in the 4GL against representative test databases.

At the beginning of the conversion project, source code can be compared to determine if it is broken in the 4GL, the environment can be investigated, 4GL compiles and listings can be generated to understand the code.

Once the code is converted, access is necessary as a baseline to compare against the converted code (to compare the look/feel, behavior and functionality of the 4GL and FWD environments). Accordingly, there are important uses of the comparison environment throughout the entire conversion project, so it is imperative that such an environment is available. That environment must also be as close to a production environment as is possible.

This development environment must be a full 4GL development license which provides the ability to write 4GL code and run it and any 4GL development tools needed. This is often needed to explore how a particular 4GL feature works and ensure that various forms of that feature can be tested.

© 2004-2022 Golden Code Development Corporation. ALL RIGHTS RESERVED.