Project Setup¶

Project Setup

Before the automated conversion tools can be run, a conversion project needs to exist. This chapter provides guidance on the setup and configuration of a conversion project. There is no installation program to automate project setup. It is currently a manual process. Each conversion project is uniquely associated with a specific 4GL application. Once the project setup is complete, the same conversion project can be used to run the conversion tools any number of times to convert that associated 4GL application into a Java result.

Running conversion more than once is useful for the following reasons:

To push 4GL source code changes (modifications, additions, enhancements, fixes or other changes) into the converted Java application.
As features are added or fixed in the FWD conversion processing, older conversions may need to be replaced with the results of a newer run.
To change options that affect the way that code is emitted by the conversion process. This may be in regards to a more compact representation of the source, better refactoring or some other improvement in the resulting source code.

During the work on an application of any significant complexity, it is likely that the conversion tools will need to be run many times. This is part of the design of the tools and is no problem at all.

Even if you intend to only run the conversion tools once, the project setup cannot be avoided. It must be there before the tools will work.

Creating Directories¶

Each conversion project exists inside a specific directory structure. The top of this sub-directory tree will be referred to as the “project root” or “project root directory”. In example commands, it will be referenced as $P2J_HOME (using the UNIX shell scripting convention).

This can be located anywhere on your target system, however it is important to put it in a location which has enough free disk space to allow the conversion to proceed successfully. During the conversion process, there are log files, intermediate form representations and multiple outputs created for each file. Many of these files are significantly larger than the original input source code, such that it may take gigabytes of disk space for the conversion of even a medium sized application, even if the Java source code that is generated is measured only in megabytes.

Create the project root directory, it can have any name that is a valid directory name. All of the FWD tools will be run with this as the current directory. This allows those tools to easily find the project and its configuration, which must reside in specific sub-directories directly under the project root.

Create each of the following directories, as sub-directories of the project root:

Directory	Contains	Required
`cfg/`	project-wide configuration files	yes
`data/`	schema files (in .df format) for each database to be migrated/used by the application	yes
`data/namespace/`	dictionary files and other intermediate forms of the schema files, in a format that can be used during conversion	yes
`pattern/`	custom conversion rules for this application	no
`src/`	application source code (4GL and Java)	yes
`src/`{application_name}	4GL source code to convert	yes
`rpt/`	HTML report output	no

To make it easier to configure and run the conversion tools, it is useful to have easy access to the directory with the FWD jar files and conversion rules. This directory should have been created using instructions from the Installation chapter, in the section regarding Conversion Tools. There are two approaches to this:

Create a sub-directory of the project root named p2j/ that includes both the jar files and the directory tree with the conversion rule XML files. This approach is simple and will work on all operating systems.
Create a symbolic link (named p2j) to the directory that contains the jar files and the conversion rule XML files. A command such as ln -s <path_to_p2j_resources> p2j will work on most Linux or UNIX systems. This approach allows the same FWD installation to be shared by more than one conversion project.

For the remainder of this book, commands and examples will assume that there is the following:

Path/File	Description
`$P2J_HOME/p2j/build/lib/p2j.jar`	Used in `-classpath` parameter with each conversion tool, this jar file contains all the conversion technology as well as references to the other jar files that should be located in `$P2J_HOME/p2j/build/lib/`.
`$P2J_HOME/p2j/rules/`	Contains a set of sub-directories in which all the conversion rule XML files exist.

Place Source Code¶

Once the skeleton directory structure exists, the 4GL source code and schemata must be placed into that structure.

Each of the schema files (in .df format) should be placed in the $P2J_HOME/data/ directory. There must be one .df file for each of the different database schemata in use by the application.

Assuming that a directory named src/my_application/ was created in the Creating Directories section above, all of the 4GL source code (procedures and includes) must be placed inside that directory. This would be considered the top-level or root directory for the 4GL source code. Make sure to retain the proper sub-directory structure that the application expects. For example, if the application has it's source code organized into top level directories such as customers, orders and invoices, that same structure would be maintained as src/my_application/customers/, src/my_application/orders/ and src/my_application/invoices/.

Global Configuration¶

The code conversion process has a global configuration file named p2j.cfg.xml that must exist in the $P2J_HOME/cfg/ directory.

Simple Example¶

This file is the location for project-wide configuration values. Create a file with the following contents:

<?xml version="1.0"?>

<!-- FWD conversion project configuration -->
<cfg>

   <global>
      <!-- conversion tools "internal" values -->
      <parameter name="p2j_rules"        value="${P2J_HOME}/p2j/rules" />
      <parameter name="patpath"          value=".:${P2J_HOME}/pattern:${p2j_rules}/include:${p2j_rules}:" />
      <parameter name="registry"         value="${P2J_HOME}/cfg/registry.xml" />

      <!-- values from the original system on which Progress 4GL ran -->
      <parameter name="opsys"            value="UNIX" />
      <parameter name="propath"          value="${P2J_HOME}:${P2J_HOME}/src/my_application:/original/my_application/propath/here" />
      <parameter name="basepath"         value="./src/my_application" />

      <!-- conversion output -->
      <parameter name="output-root"      value="${P2J_HOME}/src" />
      <parameter name="pkgroot"          value="com.my_domain.my_application" />
      <parameter name="merge-point"      value="/server/default/runtime/default/" />
   </global>

   <schema>
      <namespace
            name="standard" 
            importFile="data/standard.df" 
            xmlFile="data/namespace/standard.dict" />
      <namespace
            name="my_schema" 
            importFile="data/my_schema.df" 
            xmlFile="data/namespace/my_schema.dict" 
         default="true">
         <parameter name="ddl-dialects" value="postgresql,h2" />
         <dialect-specific name="h2">
            <parameter name="collation" value="en_US_FWD" />
         </dialect-specific>
      </namespace>
      <metadata name="standard">
         <table name="_db" />
         <table name="_file" />
         <table name="_field" />
         <table name="_index" />
         <table name="_index-field" />
         <table name="_lock" />
         <table name="_user" />
      </metadata>
   </schema>

</cfg>

The <global> section of cfg/p2j.cfg.xml is a list of <parameter> elements, each of which is a single name and value pair that customizes some aspect of the conversion. These configuration values are read by the class com.goldencode.p2j.cfg.Configuration. All of the conversion tools that need these parameters, use the Configuration class to obtain them.

Configuration Reference¶

The following is the list of all parameter values that can be modified in cfg/p2j.cfg.xml. If a parameter is specified more than once, an error will occur.

The most important values to configure are opsys (which greatly affects the way that projects parse) and propath (which determines how include files and OO classes are found). See below for more details.

Parameter Name	Datatype	Default	Required	Notes
abspath	String	n/a	no	Used during call graph processing to strip any leading absolute path from filenames before comparing them with filenames already in the call tree.
basepath	String	n/a	yes	The general idea of this path is to enable the conversion from a filename that is relative to the project root to a filename that is relative to the root of where the 4GL source code is stored. This allows all the 4GL source code to be contained in a single in a sub-directory of the project root. Normally, this is specified as something like: `./src/my_application` Note that there is no trailing path separator. Used during: • call graph processing • loading directory-specific hints files from the root of where the source code is stored to the specific file being processed • calculating class and package names
batch	boolean	`false`	no	This allows the global (project-wide) specification of the `BATCH-MODE` preprocessor built-in variable value.
broken-schema-validation-warnings	boolean	`true`	no	Validation expressions that exist in the schema (`VALEXP` and `VALMSG`) can be syntactically broken. OpenEdge silently drops these from the schema and from the compiled program. By default, FWD will report a warning for these broken expressions. Setting this parameter to `false` will suppress these broken validation warnings.
callgraph-db-folder	String	`./callgraph/`	no	The absolute or relative path to the directory in which the graph database will be stored. This is only used for callgraph processing.
case-sensitive	boolean	Depending on `opsys` value, `case-sensitive` reflects the case sensitivity of the respective file-system: `no` if `opsys="win32"` and `yes` otherwise (`opsys="unix"`)	no	This specifies if the original system (on which the application ran) had a file-system that differentiated file and path names on a case-sensitive basis (e.g. on Linux or UNIX this is `true`, on Windows this is `false`). At conversion, the preprocessor uses this value to control how it searches the `propath` for include files. This is used during conversion to match external procedures file names, to search for object oriented packages/class files/interface files, to process internal procedure and user-defined function names/signatures and generally to allow full bi-directional mapping of source (4GL) and target (Java) names. This value is written to an output file named `file_sys.xml` whose values are normally moved into the runtime directory for the converted application to ensure that this same data is available at runtime. At runtime, this data is read from the directory and is used to interpret stored name mapping data to provide full bi-directional mapping of source (4GL) and target (Java) names. This is used to duplicate 4GL features such as a `RUN VALUE()` statement that dynamically calculates the external procedure name being executed. It is also used for features such as `PROGRAM-NAME` which inspects the current stack to report on which 4GL program name is at a particular index in the stack.
codenames	String	the value of the `matchlist` parameter	no	Specifies the project-relative path to the XML file that customizes name conversion for the following conversion phases: • Annotations • Business Logic Base Structure • Core Conversion • Frame Generation A common value for this would be `${P2J_HOME}/cfg/codenames.xml`. See the chapter on Name Conversion for more details.
comments	boolean	`true`	no	If specified as `false`, then comments will not be converted (they will be dropped on output). By default, comments are converted.
datanames	String	the value of the `matchlist` parameter	no	Specifies the project-relative path to the XML file that customizes name conversion for the following conversion phases: • Schema Fixups • Schema Annotations • P2O Generation • Database Field Length Statistics • Generate Hibernate Mappings • Generate Data Model Objects A common value for this would be `${P2J_HOME}/cfg/datanames.xml`. See the chapter on Name Conversion for more details.
date-order	String	“MDY”	no	Specifies the order of date components (day, month and year fields) for conversion to and from strings. The order of the 3 date sub-components is defined as a 3 character string using the letters "M", "D" and "Y" once each. The order from index 0 to index 2 represents the left to right ordering of the components. The leftmost date component is defined by the character at index position 0, the middle component is defined by the character at index 1 and the rightmost date component is defined by the character at index 2.
dmo_index_merge	String	n/a	no	Specifies the path of an optional XML file which is used to merge application-specific metadata about DMO interfaces and classes with the contents of the `dmo_index.xml` file generated by conversion. The format of the DMO index merge file specified here is the same as that of the generated `dmo_index.xml` file; however, the contents refer to hand-written DMO interfaces, rather than those generated from the legacy schema(s). For every `interface` XML element in the DMO index merge file, the conversion process will expect to find a corresponding, hand-written Hibernate mapping document (`.hbm.xml`), relative to the path specified by `merge-dmo-root` (see below). More details about the `dmo_index.xml` file can be found in the DMO Index* chapter of the FWD Conversion Reference book.
dynamic-append	boolean	n/a	no	If specified (actually, the value doesn't matter, if this exists at all) each procedure's source directory will be dynamically appended to the `propath` before the procedure is preprocessed/parsed. This duplicates the behavior of some applications where the `propath` is dynamic in this manner. If both `dynamic-append` and `dynamic-prepend` are both specified, it is `dynamic-prepend` that will be honored. This only affects conversion.
dynamic-prepend	boolean	n/a	no	If specified (actually, the value doesn't matter, if this exists at all) each procedure's source directory will be dynamically prepended to the `propath` before the procedure is preprocessed/parsed. This duplicates the behavior of some applications where the `propath` is dynamic in this manner. This only affects conversion.
error-dest	String	`System.err`	No	If specified, any errors generated by the preprocessor will be redirected into the filename specified by this value, e.g. `preproc_errors.log`. This will also redirect preprocessor warnings to the same destination, unless `warn-dest` is also configured.
file-separator	String	Depending on `opsys` value, `file-separator` defaults to the separator of the file-system of the configured OS: `\` if `opsys="win32"` and `/` otherwise (`opsys="unix"`)	no	This specifies the character used to separate the different segments of a single path name on the original system (on which the application ran). On Linux (or UNIX) this is a `/` and on Windows it would be `\`. At conversion, the preprocessor uses this value to control how it processes include filenames. This is used during conversion to match external procedures file names, to process internal procedure and user-defined function names/signatures and generally to allow full bi-directional mapping of source (4GL) and target (Java) names. This value is written to an output file named `file_sys.xml` whose values are normally moved into the runtime directory for the converted application to ensure that this same data is available at runtime. At runtime, this data is read from the directory and is used to interpret stored name mapping data to provide full bi-directional mapping of source (4GL) and target (Java) names. This is used to duplicate 4GL features such as a `RUN VALUE()` statement that dynamically calculates the external procedure name being executed. It is also used for features such as `PROGRAM-NAME` which inspects the current stack to report on which 4GL program name is at a particular index in the stack.
foreign-keys	boolean	`false`	no	This controls whether the conversion will calculate and enforce referential integrity using “natural” joins. A natural join is one where two tables are linked by an exactly matching index specification which is defined as unique in at least one of the tables. Traditionally, Progress code has not had any concept of referential integrity. This means that normal relational database management system (RDBMS) rules may be and probably are broken in most applications. As one example, in Progress the order of deletes between two tables is not important (it can be in any order), because Progress traditionally has not known about nor enforced any relationships between tables. When adding referential integrity where there was none in the past, such problems can be a serious bug. There are known issues with the current implementation of foreign keys such that many converted Progress applications would not work properly if this is enabled. Some of these problems can likely be resolved in the FWD runtime, but they do still exist at the time of this writing. This parameter is used during Data Model Object (DMO) generation, Hibernate mapping document generation and during data import (a step in migrating data into the RDBMS).
graph-node-filter	String	`null`	no	In callgraph mode, the pattern engine will use this filter to determine if each walked node should or should not be visited. The must specify a fully qualified class name (including package) for a class that implements the `GraphNodeFilter` interface. This class must have a default constructor (no parameters) so that instances of the class can be created by the pattern engine.
keyword-ignore	String	n/a	no	This specifies the filename of a text file that contains a list of one or more Progress 4GL keywords which should be ignored for the purposes of lexing the source code from this project. Normally, any text in the source code that matched known Progress 4GL keywords would be recognized as that keyword and treated according to how Progress interprets that keyword. When a keyword is listed in this file (each keyword must be on a separate line), that keyword matching is bypassed and the text is just treated as text, with no special meaning. This allows variable names or other resources to be named the same as a keyword, without causing a problem.
marker	int	`1`	no	This value is a reserved character that is used to insert a mark into the stream of a preprocessed source file (during preprocessing) such that later phases of preprocessing can identify each of the marks and change it's behavior in response. This is an internal value and should rarely need to be changed. The character associated with the integer `1` is something not expected to be seen in normal source code. However, if such a character must be present in the source file, then this parameter can be used to specify a different character for this marking purpose. Note that whatever character is specified, the integer value that is associated with that character is what must be coded in this parameter. That character must not be present in any of the source files being preprocessed. Normally, only values below `0x20` should be considered AND you must avoid characters for `TAB`, `RETURN` or any other character that may reasonably exist in the source.
matchlist	String	n/a	no	Specifies the project-relative path to the default XML file that customizes name conversion when the datanames or codenames is not specified. A common value for this would be `${P2J_HOME}/cfg/matchlist.xml`. See the chapter on Name Conversion for more details.
merge-dmo-root	String	n/a	no*	Specifies the root directory relative to which one or more hand-written, Hibernate mapping documents (`.hbm.xml`) are located. The location relative to this path is determined by the content of the DMO merge index file - specifically the combination of the `package` attribute of the `dmo-index` element and the `name` attribute of each `schema` element -- as follows: `${merge-dmo-root}/` `${dmo-index package}/` `${schema name}/` `.hbm.xml` * This parameter only is required if `dmo_merge_index` is specified.
merge-point	String	n/a	yes	This specifies the location in the runtime directory to which the `file_sys.xml` entries should be merged (copied). Normally, this would be `/server/default/runtime/default/`. During conversion, this is written as an attribute into the `file_sys.xml` but it is otherwise not used or honored in any of the conversion tools.
name_map_merge	String	`null`	no	If specified, this is a absolute or relative filename for an XML file that holds custom name map entries which to be merged with the automatically generated name map that is an output of the conversion process.
number-decimal-sep	String	“.”	no	This overrides the decimal separator used in parsing and rendering decimal numbers. By default the value will be read from the locale or if that fails it will default to the period character. This string should only hold a single character.
number-group-sep	String	“,”	no	This overrides the group separator used in parsing and rendering numbers. By default the value will be read from the locale or if that fails it will default to the comma character. This string should only hold a single character.
oo-skeleton-path	String	`./p2j/skeleton/`	no	Path to the top level directory in which the “fake” `.cls` files are placed to allow the parser to process references to built-in 4GL OO classes (e.g. `Progress.Lang.Object`) and built-in .NET classes (e.g. `System.Object`). See the Object Oriented Classes and References chapter for more details.
opsys	String	`UNIX`	no	This is the value provided to the preprocessor when the OPSYS built-in is used in an expression. If not specified, this value will be dynamically determined based on the Java `os.name` system property:
				os.name	OPSYS Value
				linux	UNIX
				aix	UNIX
				solaris	UNIX
				sunos	UNIX
				hp-ux	UNIX
				freebsd	UNIX
				mac os x	UNIX
				windows xp	WIN32
				windows 2000	WIN32
				windows nt	WIN32
				windows me	WIN32
				windows 98	WIN32
				windows 95	WIN32
				anything else	The unchanged value of the `os.name` property.
				Starting with FWD v4, based on the `opsys` parameter, other parameters are derived if and only if they are not explicitly specified: `file-separator`, `path-separator`, `case-sensitive`, `unix-escapes`, `source-charset` and `winsys`. This approach generates a consistent result that matches the platform chosen in `opsys`. It is recommended to set `opsys` and to leave these other values to default properly (do NOT specify them). Although not recommended, these parameters can be set individually to desired values if needed.
output-root	String	n/a	yes	Specifies the project-relative file-system location into which all generated Java code, Hibernate mapping files and other application resources will be written. Normally, this is a path like `${P2J_HOME}/src` (with no trailing separator). In this example, if the `pkgroot` is specified as `com.my_domain.my_application`, all the generated files would be written in the path `${P2J_HOME}/src/com/my_domain/my_application/`.
registry	String	n/a	yes	This specifies the filename of the XML file that stores a database of unique identifiers for AST nodes. Each AST has one or more nodes (often thousands of nodes), each node has a unique ID. Each source file that is converted to an AST can be found using an AST ID that is unique to that file. This is only used during conversion. Normally, this is specified as `${P2J_HOME}/cfg/registry.xml`. The following parts of the conversion process are dependent upon this value: • schema loading • the conversion driver (this tool automates the complete code conversion run) • the scan driver (this automates the front-end of the conversion process) • the AST generator (primary interface to the preprocessor, lexer and parser) • the pattern engine driver (used to run TRPL programs directly) All of these programs share the behavior of either creation/manipulation of ASTs or the running of TRPL programs.
rootlist	String	n/a	no	This is not required for conversion, but it IS required for call graph processing. This value specifies the file name for the XML file that defines the list of ALL top-level entry points (4GL procedure names) into the application. The call graph processing then analyzes each of these programs and builds a call graph of all reachable code from there. In addition, any file names in the project that are not reachable from one of these top-level entry points is placed in a dead files list.
path-separator	String	Depending on `opsys` value, `path-separator` is selected to match the respective OS file-system: `;` if `opsys="win32"` and `:` otherwise (`opsys="unix"`)	no	This specifies the character used to separate each path in a list of paths on the original system (on which the application ran). On Linux (or UNIX) this is a `:` and on Windows it would be `;`. At conversion, the preprocessor uses this value to control how it parses the `propath` (needed to search for include files). This is used during conversion to match external procedures file names, to process internal procedure and user-defined function names/signatures and generally to allow full bi-directional mapping of source (4GL) and target (Java) names. This value is written to an output file named `file_sys.xml` whose values are normally moved into the runtime directory for the converted application to ensure that this same data is available at runtime. At runtime, this data is read from the directory and is used to interpret stored name mapping data to provide full bi-directional mapping of source (4GL) and target (Java) names. This is used to duplicate 4GL features such as a `RUN VALUE()` statement that dynamically calculates the external procedure name being executed. It is also used for features such as `PROGRAM-NAME` which inspects the current stack to report on which 4GL program name is at a particular index in the stack.
patpath	String	`.:${P2J_HOME}/pattern:${p2j_rules}/include:${p2j_rules}:`	yes	Specifies the list of directories in which to search for named conversion rule-set files. Using this path, conversion rule-sets will be found and loaded for execution. Normally, this should include the project-specific path as well as paths for FWD conversion include files and a path to the root of where the FWD conversion rule-sets reside. Each path in the directory should be separated with the platform-specific path separator used for the system on which the conversion tools are running.
pkgroot	String	n/a	yes	This specifies the “root” package in which all of the converted application will exist as sub-packages. Normally, this is something like `com.my_domain.my_application`. This is used during conversion to create the package name for all of the Java source files that are generated (it is always prefixed on each package name). It also is heavily used to allow full bi-directional mapping of source (4GL) and target (Java) names. This value is written to an output file named `file_sys.xml` whose values are normally moved into the runtime directory for the converted application to ensure that this same data is available at runtime. This is also used to know the location in which the name mapping data (a file named `name_map.xml`) should be placed in the converted application's jar file (so the mapping data can be loaded at runtime). At runtime, this data is read from the directory and is used to find and interpret stored name mapping data to provide full bi-directional mapping of source (4GL) and target (Java) names. This is used to duplicate 4GL features such as a `RUN VALUE()` statement that dynamically calculates the external procedure name being executed. It is also used for features such as `PROGRAM-NAME` which inspects the current stack to report on which 4GL program name is at a particular index in the stack.
preproc-debug	boolean	`false`	No	Used to turn on debugging output for the preprocessor during conversion. This output makes it much easier to diagnose preprocessor problems. It does make the output much more verbose for each file processed.
propath	String	`.:`	no	This is used during conversion and at runtime for the following purposes: • conversion ° During schema loading validation expressions are preprocessed (to resolve include file references) and the preprocessor must be aware of the `propath` in order to find the includes. ° During the scan of the 4GL source code to create Progress ASTs, the `propath` is used to find files specified with relative file names. ° In the preprocessor it is used: · to find the specified procedure file to be preprocessed; · for finding include files, see above; · during preprocessor expression evaluation (which uses the Progress 4GL parser, see below). ° In the parser it is used for 4GL class and package name resolution and recursive loading. ° In name mapping processing it is used to convert filenames (especially absolute filenames) to relative names that can be matched with the legacy Progress 4GL names. ° It is written to the `file_sys.xml` which is whose values are normally moved into the runtime directory for the converted application to ensure that this same data is available at runtime. ° In call-graph processing it is used to find files specified with relative file names. • At runtime, the propath is read from the directory and it is used to provide bi-directional name mapping to convert names into a relative form that will match the legacy names. At conversion the propath should contain the project root directory (first), then the top-level directory where all the 4GL source code can be found and finally, the original system's propath. The first two are used to reverse project-specific paths into relative paths as the application would expect to use them. The original system's propath is needed to artificially duplicate the same search results that the application would have had on that system. Since those paths may not exist on the conversion or target system, the search and name mapping processing is written to handle these paths specially. `${P2J_HOME}:${P2J_HOME}/src/my_application:/my/original/apps/path` At runtime, only the original system's propath is needed.
p2j_rules	String	`${P2J_HOME}/p2j/rules`	no	This is used via variable substitution to simplify other parameter values. It is not used directly by the tools.
source-charset	String	Depending on `opsys` value, `source-charset` is selected to match the respective OS implementation of Progress 4GL: `windows-1252` if `opsys="win32"` and the JVM default charset otherwise (`opsys="unix"`).	no	This sets the global default for reading source files (procedures, classes and include files) and schemata (`.df` files). This is very important to properly handle any locale-specific characters that may be encoded in comments, string literals and other text or symbols. This is available starting with FWD v4.
source-code-version	String	`9.1C`	no	When specified, this allows the conversion to differentiate between v9.x and v10.x processing. The conversion process currently differentiates between these major versions. For example, you can tell the conversion process to treat the source code as a v10 application by specifying `10.2B` as the version. At this time v11 is not checked or recognized.
unix-escapes	boolean	Depending on `opsys` value, `unix-escapes` is selected to match the respective OS implementation of Progress 4GL: `false` if `opsys="win32"` and `true` otherwise (`opsys="unix"`).	no	In the Progress 4GL, the preprocessor translates or interprets input files differently depending on which escape sequences are honored. When run on UNIX, the `\` (backslash) character is silently treated as the full equivalent of the `~` (tilde) character, which is the documented escape character for the Progress 4GL preprocessor. On Windows, the backslash is not supported since it is needed as the file separator in a path. For FWD, this value is read from its configuration instead of determining it based on the current platform. If specified as true, then the backslash will be honored as an escape character.
warn-dest	String	`System.err`	no	If specified, any warnings generated by the preprocessor will be redirected into the filename specified by this value, e.g. `preproc_warnings.log`.
winsys	String	Depending on `opsys` value, `winsys` is selected to match the respective OS implementation of Progress 4GL: `MS-WIN95` if `opsys="win32"` and `TTY` otherwise (`opsys="unix"`)	no	This allows the global (project-wide) specification of the `WINDOW-SYSTEM` preprocessor built-in variable value.
wsmode	String	`WEBSPEED`	no	Specifies the compatibility mode to be used by the E4GL preprocessor during parsing (in each one below there are both short and verbose forms which are equivalent): • `WS` or `WEBSPEED` • `WS_STRICT` or `WEBSPEED_STRICT` • `BD` or `BLUE_DIAMOND`
wsoptions	String	n/a	no	This provides a mechanism to specific a project-wide WebSpeed options string (a set of comma-separated options to control the processing of the E4GL preprocessor and WebSpeed). Any per-file options string that is specified would be honored in preference to this one. See the chapter on encoding hints for more details. This value will change some state of the E4GL preprocessor. It is also used to create the output for footer processing during the E4GL processing.
ws-preproc-syms	boolean	`false`	no	If specified as `true`, the following WebSpeed preprocessor built-in definitions will be initialized and available for use when the preprocessor expands definition references:
				Definition	Expansion
				AMP	&amp~;
				DISPLAY	DISPLAY STREAM WebStream
				END	`.`
				GT	&gt~;
				LT	&lt~;
				OUT	PUT STREAM WebStream UNFORMATTED
				OUT-FILE	PUT STREAM WebStream UNFORMATTED
				OUT-STREAM	PUT STREAM WebStream UNFORMATTED
				OUT-FMT	PUT STREAM WebStream
				OUT-LONG	EXPORT STREAM WebStream
				QUOT	&quot~;
				WEBSTREAM	STREAM WebStream
				If set to `false`, these definitions will not be available.

Substitution variables in the form ${key_name} are permitted, where key_name is the name of a parameter previously defined by this configuration mechanism. The replacement value applied to the substitution variable is the value of the parameter identified by key_name. For example, given the following configuration:

<cfg>
   <global>
      <parameter name="user.home" value="/home/user" />
      <parameter name="subdir"    value="${user.home}/mysubdir" />
   </global>
</cfg>

There will be the following parameter definitions after substitution:

user.home = /home/user

subdir = /home/user/mysubdir

The following restrictions apply to this mechanism:

P2J_HOME is a reserved parameter key. It may not be defined in the configuration file, but it is available for use as a substitution variable. When the conversion tools run, those tools remap any references to ${P2J_HOME} in the configuration to the current directory (which should be the project root directory).
As this is not valid XML syntax, it may only occur inside a single or double quoted string.
The variable name key_name must not include any characters which have special syntax meaning in regular expressions, since regular expressions are used internally for the replacement. Special characters in key_name are not explicitly escaped, which could cause unexpected results.
The order of the parameters is very important. The parameter defining the value for key_name must have been processed before any references to key_name. The file is processed in top to bottom order.

Note that, by convention, ALL paths configured in p2j.cfg.xml must use UNIX style file and path separators, even if the current OS, the legacy one or both, are Windows. The Configuration class will transforms them on-the-fly while applying the substitution mechanism to processed parameters so, when they are requested by code inside FWD, they will be already converted to current OS format. The file-separator and path-separator parameters define the legacy OS file system and they are NOT taken into consideration when parsing any of the parameters, like basepath, propath, p2j_rules etc.

The <schema> section of the p2j.cfg.xml defines details of the schemata that are associated with the application. See the Schema Loading section of this chapter (below) for details on the customization of that section.

Schema Loading¶

The first step in the conversion process is for the FWD conversion engine to parse and understand the schemata of the databases used by your Progress 4GL application. This must be done early on, because the application's source code will contain references to database, table, field, index, trigger and sequence names which are defined in an associated database schema. If these are not known to the conversion engine, the FWD source code parser will raise errors as it encounters database symbols it cannot recognize, when it scans the application's 4GL source code.

FWD conversion gathers its schema information from a schema export (.df) file. Consult your Progress documentation to determine how to create such a file. You will need one schema export file for each database used by your application which has a unique schema. In other words, if you have multiple instances of a database that all share the same schema, then you only need a single schema export file. But if your application code references tables or fields from multiple different schemata, then you must have a schema export file for each of the schemata referenced.

When running an end-to-end conversion of a 4GL software project, you normally will not execute the schema loading step separately. However, when a conversion project is in its early days, it is common to run this step independently to flush out pre-existing errors in the schema.

Schema Loader Configuration¶

The schema loader scans a Progress schema export file and analyzes each database, table, field, index, sequence, and trigger entry. The information extracted from the schema export file is used to populate a schema dictionary, which later can be used during source code parsing, or to dump the scanned information for review.

Some configuration of the schema loader is necessary, so that the program knows which schema export file(s) to process and where to store the output of that processing. This configuration is managed in an XML file, which resides at {project_root_path}/cfg/p2j.cfg.xml.

Within the configuration file, the schema element contains the information needed by the schema loader. Each schema to be analyzed is defined within a namespace element, which is the direct child of the schema element:

<?xml version="1.0"?>

<!-- FWD main configuration -->

<cfg>

  ...

   <schema>
      <namespace
         name="standard" 
         importFile="data/standard.df" 
         xmlFile="data/namespace/standard.dict" />
      <namespace
         name="my_schema" 
         importFile="data/my_schema.df" 
         xmlFile="data/namespace/my_schema.dict" 
         default="true">
         <parameter name="ddl-dialects" value="postgresql,h2" />
         <dialect-specific name="h2">
            <parameter name="collation" value="en_US_FWD" />
         </dialect-specific>
      </namespace>
      <metadata name="standard">
         <table name="_db" />
         <table name="_file" />
         <table name="_field" />
         <table name="_index" />
         <table name="_index-field" />
         <table name="_lock" />
         <table name="_user" />
      </metadata>
   </schema>

</cfg>

The term namespace as used in this configuration file is historical. Each namespace node in the configuration file might better be thought of as describing an individual, database schema used by the application. There are two types of namespace configuration, though they both use the same namespace XML element: metaschema and application schema.

Metaschema Namespace¶

The first namespace entry in the schema element above, with its name attribute set to standard, represents the metadata schema, or metaschema. It is so designated because the metadata element (the last sub-element under the schema element) refers to it (note the metadata element's name attribute, which also is set to standard). There is only one metaschema namespace defined per application. The metaschema is specific to the version of Progress in use with your 4GL application. It defines Progress-specific metadata symbols which may appear in source code (e.g., _file, _field, etc.). The metaschema is analyzed and its symbols are merged with those of each application-specific schema, so that these symbols are available to downstream conversion phases - such as source code parsing - as normal database references.

The importFile attribute tells the loader the path of the Progress schema export file (*.df) to be loaded (imported from the point of view of the loader). The xmlFile attribute represents the path of the intermediate form XML file to be created as the result of the analysis the schema loader performs. Both paths are relative to the project root. Subdirectories referenced in these paths are expected to exist; an error will be raised if they do not.

Application Schema Namespace¶

The second namespace entry in the example above represents schema information for an application database. The name attribute of the namespace node represents the schema name of the associated Progress database, in this case, my_schema.

For each application schema namespace entry, distinct and dialect-specific parameters can be specified. Currently, only the ddl-dialects parameter is used; this parameter specifies a comma-separated list of dialects for which DDL will be generated during the middle phase of the conversion process.

Per-dialect parameters are supported as well; these reside under the dialect-specific node, which specifies the dialect in the name attribute. For the dialect-specific case, only the collation parameter for the h2 dialect is used; by default, this parameter is set to en_US_FWD, which is also the only collation supported by FWD at the time of this writing.

The importFile and xmlFile attributes serve the same purpose as for the metaschema namespace configuration described above. However, in this case, they refer to the schema *.df file and intermediate form XML output file, respectively, for the application schema instead of the metaschema.

The default attribute, if set to true, tells the loader that the application expects the my_schema database to be connected by default, as opposed to being connected dynamically by a program via the 4GL CONNECT statement. Dynamic connections via the CONNECT statement are managed by conversion hints. The default attribute tells the conversion engine that any references to this database found in the source code, be they qualified or unqualified, are known to refer to this schema, without the need for a conversion hint.

Metadata Configuration¶

The use of database metadata in 4GL source code is partially supported by the conversion process and runtime environment at the time of this writing. Database metadata symbols are recognized by the 4GL source code parser and by the reporting engine. Partial runtime support exists for the following metadata tables: _db, _file, _field, _index, _index-field, _lock, and _user. If your application uses any of these tables, they should be specified as table elements under the metadata element, as in the example above. If your application uses a subset of these, note that some tables have dependencies on others. For example, the _field table defines a _file-recid field, which represents a foreign key to the _file table, and thus, a dependency on the _file table. Be sure to analyze the dependencies created by the metaschema tables you use in your application, and include those dependencies as table elements in the metadata configuration.

Running the Schema Loader¶

The following discussion assumes that the user has a shell (command prompt) open and the current directory is in the project root directory ($P2J_HOME).

To run the program:

java -classpath $P2J_HOME/p2j/build/lib/p2j.jar com.goldencode.p2j.schema.SchemaLoader [-q]

where -q optionally turns on quiet mode which reduces, but does not eliminate, the output printed to the console.

Schema Loader Errors¶

It is possible to associate Progress 4GL code with certain schema elements in the form of arbitrary validation expressions for tables and fields, and VIEW-AS clauses for fields.

The schema loader will load and store 4GL validation expressions defined in the schema. However, the parsing of these expressions is deferred until the conversion process parses the business logic procedures from which these tables and fields are referenced. This allows the validation expressions to be parsed within the context of the business logic which uses them. Thus, validation expressions in the schema will not raise errors during schema loading (but may later, during the main parsing phase).

On the other hand, VIEW-AS clauses defined for database fields are parsed inline during schema loading. Under normal circumstances, it is not possible to define an invalid VIEW-AS clause in a Progress schema, so the conversion process assumes all VIEW-AS clauses defined in a schema export file represent valid Progress syntax. If, for some reason (perhaps as a result of manual editing of the schema export file), an invalid VIEW-AS clause is encountered, it will report a parsing error during schema loading. However, this is a non-fatal error, and processing will continue.

Project

General

Profile

FWD

Wiki