Project

General

Profile

Resolving Preprocessing Issues

This chapter provides guidance on how to resolve problems that are occur in running the 4GL compatible preprocessor in FWD. Preprocessing is one of the first steps executed by the ConversionDriver, so problems in preprocessing occur early in the processing of a given source file.

Dealing with issues is an open ended process which is naturally associated with unexpected conditions. The most common issues will be discussed. More importantly, some useful techniques will be described that can be used in general to investigate and resolve preprocessing issues.

Common Issues

Missing Include File

The following output is an example of the error seen during the preprocessing of a file named missing_include_file.p by the ConversionDriver:

missing_include_file.p
Error in missing_include_file.p#1:12 ==> Include file "missing.i" not found
com.goldencode.p2j.preproc.PreprocessorException: Preprocessor done with 1 error(s).
        at com.goldencode.p2j.preproc.Preprocessor.<init>(Preprocessor.java:694)
        at com.goldencode.p2j.uast.AstGenerator.preprocess(AstGenerator.java:1001)
        at com.goldencode.p2j.uast.AstGenerator.prepareDataStream(AstGenerator.java:746)
        at com.goldencode.p2j.uast.AstGenerator.processFile(AstGenerator.java:684)
        at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:203)
        at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:122)
        at com.goldencode.p2j.convert.ConversionDriver.runScanDriver(ConversionDriver.java:373)
        at com.goldencode.p2j.convert.ConversionDriver.front(ConversionDriver.java:270)
        at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:1515)

This tells us that the preprocessor detected an error in missing_include_file.p when it was processing the character 12 of line 1. The preprocessor was directed to include a file named missing.i, which could not be found in the current PROPATH. This error is not fatal, so the preprocessor completes its work and at the end of its processing, it reports the error (this is really more of a warning). The stack trace output after the error is not useful in this case.

Here are the contents of missing_include_file.p:

{missing.i}

The error could not have been detected before character 12 of line 1 since before that time the curly braces containing the include directive was still being parsed.

It is important to note that a given file can have any number of missing includes. Since that doesn't stop preprocessing, there is no limit on the errors that can be reported. The following is a list of potential causes and solutions:

Cause Solution
The include directive is just dead code in the source file which is silently ignored in Progress 4GL, but which is reported in FWD. In this case the referenced file is missing and is not intended to be there. There really isn't a problem here. The error can be ignored or the source file can have the include directive removed.
The include file does not exist in the file system as expected. Put the missing file in place.
The PROPATH is incorrect or missing. Ensure that the static PROPATH value as specified in the propath, dynamic-append, and dynamic-prepend parameters of the p2j.cfg.xml.

The PROPATH as seen for a given file may also be specified or customized in a directory-level or file-level hint. Please see the chapter on Conversion Hints for details on how to use this facility.

Note that any application that dynamically assigns the PROPATH will require use of the dynamic propath modification features of the FWD tools. The above-referenced dynamic-*pend parameters can be used if this is something that should be global to the project. However, there is a more granular solution available using hints, which can be applied to only those exact files to which the hints are appropriate.
The include directive is broken or otherwise malformed. Fix the directive in the source file.

Runtime Preprocessor Usage

The Progress 4GL includes a feature in the RUN statement where the called procedure is dynamically compiled every time the RUN statement executes. Normally, if the called procedure has already been compiled once, the compiled result (r-code) is executed without any further compilation. The idea here is that a single procedure file can have different source code every time it runs by passing custom arguments to the preprocessor as part of the RUN statement. The preprocessor then expands the argument references in the called procedure using those RUN statement arguments. This on-the-fly preprocessed file is then further processed by the Progress compiler and finally it is executed. Each time that RUN statement executes, the resulting code for that procedure is potentially different. This is called “runtime preprocessor usage”.

The FWD preprocessor already handles arguments for included files. Both positional and named arguments are supported and there are no known deviations with the Progress behavior (even though much of the Progress behavior is undocumented and quite non-intuitive). An example of named argument usage is:

{ some_include_file.i &name1 = something &name2 = "something else" }

Here is an example of positional argument usage:

{ some_include_file.i “something 1” “something 2" }

Both examples (and many more) are fully supported in FWD. The only problem is with runtime compilation. It would look something like this:

RUN myprog.p (parameter1, parameter2) preproc_arg1 preproc_arg2, preproc_arg3.

Such a construct requires the called procedure to be compiled dynamically at runtime, which is incompatible with a static Java source to source conversion. This also would require a Progress 4GL environment that has a compiler in order to run since the compiler is used every time such procedures execute.

The code in Progress that uses this would look like this (the references can only be position, the arguments to the RUN statement can't be named):

MESSAGE “{1} {2} {3}”.

The resulting expanded code would look like this:

MESSAGE “preproc_arg1 preproc_arg2 preproc_arg3”.

The preprocessor silently “expands” references to unknown positional or unknown named arguments into the empty string. This is the same as how Progress works. So a procedure file (as opposed to an include file) that references something like:

MESSAGE “{1}”.

will have the result expended to:

MESSAGE “”.

This may be incorrect but it won't cause a problem in parsing. Nor will it cause any problem during the later phases of conversion because the result (in this case) is valid 4GL syntax, even if it is not what the developer intended. FWD does not warn you of such expansions since it is a documented technique that unknown references expand to empty string. In other words, it is used heavily and relied upon heavily, so this isn't an error condition. So you may find such problems late in the processing of the code since the preprocessor will not complain.

Of course, it is also possible that the expanded code is not valid 4GL and in such a case the problem will manifest itself in the parsing step.

Once you determine that you have one or more procedures that rely upon runtime preprocessor arguments, you will have to take one of two possible approaches.

If your objective is only to get the conversion front end to run successfully, then the quickest way to deal with the usage of runtime preprocessor usage is to provide an example set of arguments to the preprocessor via the hints mechanism. In short, you would edit or create a myprog.p.hints file (and locate it in the same directory as myprog.p) that had contents like this:

<?xml version="1.0"?>

<!-- UAST hints -->
<hints>
   <preprocessor>
      <argument pos=”1” value=”&apos;text with embedded quotes&apos;” />
      <argument pos=”2” value=”text with no embedded quotes” />
      <argument pos=”3” value=”preproc_arg3” />
   </preprocessor>
</hints>

Since runtime preprocessor arguments cannot be named, each of these MUST have the pos attribute which specifies the argument position (this is a 1-based index). The preprocessor will pick these up and use them to resolve any positional argument references to {1}, {2} or {3} in the procedure myprog.p.

The hints file is XML so the handling of single and double quotes must be done with care. You must look at the call sites (the RUN statements). If they include embedded quotes (usually by tripling up quotes like “””text with embedded quotes””” then the matching argument will have to have an &quot; or &apos; in place of any tripled quotation marks. In the 4GL, a tripled quotation mark in an argument leaves just 1 quotation mark behind in the result. In the arguments here, only the one escaped character is needed.

This allows the front end to run successfully and there will be a single example AST that represents one possible version of the procedure. That may be sufficient for analysis, reporting or even for conversion in the case that no variability is actually needed (the use of runtime preprocessor arguments was unnecessary in the 4GL). For more details on the syntax of this hint, please see the chapter on Conversion Hints.

Long term, any 4GL code that really does need to use runtime preprocessor arguments will need to be rewritten to avoid such usage. That will allow a static conversion into Java and no conversion hints will be needed.

Configuration

There are several global preprocessor definitions that can be used to modify preprocessor execution. In FWD, three of them can be controlled via the global configuration file or via hints:

BATCH-MODE (defaults to false)

OPSYS (depends on the value of the os.name Java property)

WINDOW-SYSTEM (defaults to TTY)

If the Progress defaults for these differ from those in FWD and this affects the preprocessing of the source code, then you will need to set these values as appropriate to resolve the issue. For details on the global configuration, see the batch, opsys and winsys parameters in the Global Configuration section of the chapter on Project Setup. For directory-level or file-level control, please see the Conversion Hints chapter.

In addition to these definitions, by default the preprocessor honors the \ backslash character as an escape sequence (which is the same as the ~ tilde character in Progress). This is exactly how Progress works on UNIX or Linux systems, but on Windows, Progress does not treat the \ backslash character as an escape sequence. Instead, on Windows the \ backslash is used as the separator for path segments in filenames. Progress 4GL provides a range of different syntax for coding filenames in source files. Some of these methods allow filenames to be directly coded in the source instead of putting the names into a string (enclosing the filename is single or double quotation marks). When such “naked” filenames are directly inserted in source code, the preprocessor will usually fail to operate correctly since those characters will be interpreted as escape sequences by default.

To resolve this, use the unix-escapes parameter in the Global Configuration section of the chapter on Project Setup. For directory-level or file-level control, please see the Conversion Hints chapter.

Input File Syntax Problems

The FWD preprocessor is designed to generate the same outputs as the Progress 4GL preprocessor given the same inputs. Any problems with the syntax of the input files should manifest themselves the same way in both environments. If this is not the case, then there may be a problem with the FWD preprocessor (see next section).

A good approach is to capture and compare the preprocessor output from both Progress and FWD. Please see the section later in this chapter entitled Comparing Progress and FWD Output for details and caveats.

In one case, there can be a message from the FWD preprocessor that will not be seen in Progress. This relates to any use of the preprocessor function DEFINED on a variable name that is NOT defined. Such a usage will result in output such as the following:

line 1:11: unexpected token: UNDEFINED-NAME

In this example, the symbol UNDEFINED-NAME did not exist. If this is expected, then this notification can be ignored. If this symbol is defined in Progress, then there is a real problem to be investigated which may be related to input file syntax or it may be related to configuration, missing include file or some other issue.

Preprocessor Bug or Failure

FWD was implemented independently of the Progress preprocessor. There is no common code and the majority of the details of the Progress preprocessor feature set are undocumented. FWD was still designed to be compatible, based on a wide range of Progress 4GL development experience, test code and running the preprocessor on complex 4GL applications.

As of this writing, the FWD preprocessor is nearly 100% complete. There small number of remaining missing features can be found listed in the book entitled FWD Conversion Reference. It is possible that a problem will occur that is caused by a missing feature in the FWD preprocessor.

Likewise, even for the feature set that is thought to be complete, there may be a broken feature or some particular application code could rely upon a hitherto unknown feature or other undocumented behavior of Progress.

In any case where the Progress and FWD preprocessors are behaving differently (except for the known cases noted in the Comparing Progress and FWD Output below), there are two possible solutions. If the feature is not heavily used in the application, the easiest solution may be to modify the 4GL source to eliminate the dependency. As an alternative, the FWD preprocessor can be fixed or enhanced to provide full compatibility.

Techniques

Reviewing the FWD Preprocessor Output

When the ConversionDriver executes the preprocessor, the output for the preprocessor is captured in a single text file that forms the input to the parsing process. The output file is referred to as the “preprocessor cache” and it is named based on the name of the 4GL procedure file on which it is based.

The input file $P2J_HOME/src/my_app/some/thing.w will cause a preprocessor output cache file to be created with the name $P2J_HOME/src/my_app/some/thing.w.cache. The input file will remain unchanged. The output file is the fully expanded, modified source code as it would be seen by the FWD parser or the Progress compiler.

This file can be reviewed in any text editor.

Please note that the preprocessor output is expected to be syntactically valid Progress 4GL source code. It will be processed by the parser in the next step and it must be correct 4GL or failures will occur during parsing. While the input files (the procedure and all included files) do not need to be valid 4GL source code on their own, the output file must be valid 4GL.

Comparing Progress and FWD Output

The previous section describes how to obtain the output from the FWD preprocessor (when the preprocessor is run from within the ConversionDriver).

To compare that output with that generated by the Progress 4GL runtime, the following 4GL code can be used to compile a given source file and to save the preprocessor output into a known filename:

COMPILE some/thing.w PREPROCESS some/thing.w.cache.4gl.

At that point, one can access the text file named some/thing.w.cache.4gl and that 4GL preprocessor output can be compared to the FWD output visually or using comparison tools like diff.

The output should be identical with the exception of two areas in which the output from Progress and FWD are expected to differ:

Tab Expansion Issue

The preprocessor support for tabs expansion is particularly sensitive to the internal order of actions. If a line of a source file contains both a preprocessor reference {...} of any kind as well as tab characters (0x09), then the result would vary depending on the order of tab expansion and reference resolution.

Consider this example line:

            {11}                      /* comment */

This line has three tabs following the reference {11}. Let's assume the positional argument 11 has the value of '.'. First, the reference is resolved:

            .                         /* comment */

Next, tabs are expanded:

            .                    /* comment */

If the same two actions are executed in reverse order, a different result occurs. First, tabs are expanded:

            {11}                         /* comment */

Next the reference is resolved:

            .                         /* comment */

If the results are compared, there are a different number of spaces that replaced tabs:

            .                    /* comment */
            .                         /* comment */

Currently, the FWD Preprocessor does all processing in one pass, producing the first line. The Progress preprocessor apparently runs in multiple passes, expanding tabs in a pass before resolving references. This produces the second line.

Another known case where the tab expansion issue can be seen is with references to preprocessor variables, if their definitions contain tabs.

Any differences in whitespace caused by this issue should not have any effect on the 4GL code itself. However, any developer comparing the two files must be aware of this difference.

Progress Tilde Bug

The Progress preprocessor has a bug that can be seen when the PREPROCESS option is used in the COMPILE statement. The bug only affects the output saved in the text file. The version of output that is passed to the 4GL compilation step does not show any problems.

The problem is that the Progress preprocessor emits ~ tilde characters (0x7E) on output where they are deleted from the file to be compiled. In other words, the preprocessor removes the tilde characters from the actual output to the compiler, but leaves the tilde characters in the cached output to the file system. Presumably, the Progress preprocessor must save off it's output to the file system before it is done with all expansions or processing. This can be seen by the fact that the cached Progress preprocessor output file fails to compile when the original procedure had no such problem.

An example procedure file:

def var txt as char.
txt = "hel~nlo".
message txt.

The Progress preprocessor cache output:

def var txt as char.
txt = "hel~
lo".
message txt.

The results of running the original procedure file (the message output):

hel

The results of running the preprocessor output file (the message output):

hello

Such cases will result in differences between the output from the Progress preprocessor and the output from the FWD preprocessor. There is an option for the FWD preprocessor called -keeptildes which attempts to duplicate the Progress bug such that files can be comparable. This option is not always sufficient and it is not always correct, so use this with caution. This option is only available when the preprocessor is executed directly (see the section below on Running the Preprocessor Directly). In any case, this option should never be used with something that is expected to be converted since it incorrectly modifies the resulting output.

Using the Preprocessor Hints Output

When trying to understand how the output of the FWD preprocessor corresponds or is derived from its inputs, it is very useful to have a list of the actions the preprocessor took for that source file. The FWD preprocessor provides such a facility. When run are part of the ConversionDriver or when the -hints option is passed to the standalone preprocessor driver (see the section below on Running the Preprocessor Directly), the preprocessor will create an XML file in which it will store a record of its actions. This XML output file is called the “preprocessor hints” file. It will be rewritten every time the preprocessor runs under the above conditions.

When running the preprocessor within the ConversionDriver, given a source file with a name of src/myapp/some/file.p, the preprocessor hints file will be written in src/myapp/some/file.p.pphints.

Consider the following example source procedure basic.p:

&SCOPED-DEFINE nothing better than nothing
&GLOBAL-DEFINE something something else

&if false &then
   /* this can never be included since the expression is false */
&elseif defined(something) &then
   /* welcome to a preprocessed program */
&else
   /* this can never be included since the elseif will always be true */
&endif

{top.i "isn't it?" "second (unused positional argument)"}

This is the top.i referenced include file:

{nested.i &name1="Hello" &name2="World"}
MESSAGE "This is {&something}, {1}".

This is the nested.i include file that is a nested include from top.i:

MESSAGE "{&name1} {&name2}!".

This is the preprocessed cache file created (basic.p.cache):


   /* welcome to a preprocessed program */

MESSAGE "Hello World!".

MESSAGE "This is something else, isn't it?".

This is preprocessor hints file that is written (basic.p.pphints):

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!--Preprocessor hints-->
<hints>
  <preprocessor-output>
    <include end-column="2" end-line="10" file="./top.i" start-column="1" start-line="7">
      <include end-column="2" end-line="8" file="./nested.i" start-column="1" start-line="7">
        <reference column="9" line="7" name="name1" type="argument" value="Hello"/>
        <reference column="9" line="7" name="name2" type="argument" value="World"/>
        <argument name="name2" position="4" used="yes" value="World"/>
        <argument name="name1" position="3" used="yes" value="Hello"/>
        <argument name="{1}" position="2" used="no" value="Hello"/>
        <argument name="{2}" position="1" used="no" value="World"/>
      </include>
      <reference column="9" line="9" name="something" type="global" value="something else"/>
      <reference column="9" line="9" name="{1}" type="argument" value="isn't it?"/>
      <argument name="{1}" position="2" used="yes" value="isn't it?"/>
      <argument name="{2}" position="1" used="no" value="second (unused positional argument)"/>
    </include>
  </preprocessor-output>
</hints>

Everything inside the preprocessor-output element represents the actions that the preprocessor took when processing the basic.p procedure.

The tree structure of the XML is meaningful in that it organizes and encodes the nesting of included files (in include elements). In the above example, the nested.i include file was included by the top.i include file which was included in the main program basic.p.

Any reference elements represent some expansion of a reference to a preprocessor definition or an argument (positional or named).

Any argument elements document the positional or named arguments to the enclosing node.

Each element contains some data about associated action such as the location in the output file or the type reference, or whether an argument is ever referenced.

There are some actions of the preprocessor which are not currently included in the hints output. The example above shows all the basic kinds of output that are possible. But note that expression processing and preprocessor conditionals (e.g. &if) are not shown at all.

Line and Column Numbers in Preprocessor Artifacts

Debugging output, error messages and the preprocessor hints file will contain line and column numbers to identify the associated code location being referenced.

It is believed that the preprocessor hints output line and column numbers are very accurate. But it is important to understand that these numbers refer to the generated file (the .cache file) and do NOT reference the input files.

Preprocessor error output has been found to have line and column numbers that are incorrect, possibly due to the many complex expansions occurring within the preprocessor. It is believed that this code is not 100% correct. Please submit any examples of bugs in this area as standalone testcases where possible, so that Golden Code can fix the problems.

Preprocessor Stack Traces

Usually, when an error occurs, the preprocessor will generate a Java stack trace at the failing location in the code. Although this stack trace displays the line numbers of Java code in the preprocessor, it is still invaluable for understanding the path that the code took through the 4GL code.

Consider the following example (the line numbers and method names may be different in the version of FWD in use):

/projects/src/myapp/some/path/myprog.p:5:1: unexpected token: null
        at com.goldencode.p2j.preproc.TextParser.condtext(TextParser.java:671)
        at com.goldencode.p2j.preproc.TextParser.aif(TextParser.java:416)
        at com.goldencode.p2j.preproc.TextParser.ppstatement(TextParser.java:284)
        at com.goldencode.p2j.preproc.TextParser.textBlock(TextParser.java:207)
        at com.goldencode.p2j.preproc.TextParser.text(TextParser.java:162)
        at com.goldencode.p2j.preproc.Preprocessor.<init>(Preprocessor.java:704)
        at com.goldencode.p2j.uast.AstGenerator.preprocess(AstGenerator.java:1096)
        at com.goldencode.p2j.uast.AstGenerator.prepareDataStream(AstGenerator.java:876)
        at com.goldencode.p2j.uast.AstGenerator.processFile(AstGenerator.java:794)
        at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:203)
        at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:122)
        at com.goldencode.p2j.convert.ConversionDriver.runScanDriver(ConversionDriver.java:380)
        at com.goldencode.p2j.convert.ConversionDriver.front(ConversionDriver.java:277)
        at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:1625)

The TextParser.condtext method is where conditional preprocessor expressions are evaluated. Based on the result of the evaluation, the following block is selected or bypassed.

The level of nesting of the TextParser.aif and the paths downstream of TextParser.condtext will tell the reader how many levels of conditional &IF, &ELSE, &ELSEIF have been processed.

By cross-referencing the preprocessor grammars and Java code, a great deal of the state of the 4GL code can be deduced.

When using a debugger to step through the Java code, these locations are especially useful for setting breakpoints. For example, in the TextParser.condtext method there is a variable in which the expression being evaluated can be seen as a single string.

Running the Preprocessor Directly

There are times when having complete control over a FWD preprocessor run is essential. In such cases, the ConversionDriver must be bypassed and the preprocessor must be invoked directly. The following is an example of the command line that would normally be most useful:

java -classpath $P2J_HOME/p2j/build/lib/p2j.jar com.goldencode.p2j.preproc.Preprocessor -propath src/my_app/:src/my_app/stuff -hints hints_file.xml -out output_file.lst src/my_app/some/path/input_file.p

Note that this command specifies the propath, an output filename for both the preprocessor 4GL cache as well as the preprocessor hints and the input procedure name.

The general syntax is:

java Preprocessor [options] <input_procedure> [4gl_preprocessor_arguments]

The following options are possible:

Option Meaning
-basepath <path> The top-level directory under which all source code for the application can be found. If not specified, it defaults to “.”.
-batch If specified, return yes from the BATCH-MODE built-in function. If not specified, this function will return no.
-casesensitive If specified, treat include file names as case-sensitive/ If not specified, match include file names on a case-insensitive basis.
-debug Causes very verbose logging to the console by the preprocessor. Normally this is only used when debugging a problem in the FWD preprocessor, but it could certainly help debug any kind of unexpected result, even if the preprocessor was working properly.
-err <filename> Specifies the file to which errors errors should be written. The STDERR stream is used by default.

Please note that the STDERR stream is a PrintStream which may inappropriately modify national language characters on output, even if the default language environment is set properly. If you are not using UTF-8 as the default environment, then it is likely that you MUST use this parameter to avoid this problem.
-fileseparator <sep> Specifies the separator string to use between each path component (directories and file names) in a fully-qualified or partially-qualified filename or path. Defaults to File.separator (the platform-specific file separator on the system on which conversion is running) if not specified. On UNIX or Linux this is a / (slash) and on Windows this is a \ (backslash).
-hints <filename> Specifies the filename for the preprocessor hints XML file. The extension does not have to end in .xml.
-keeptildes Preserves the tildes used in escape sequences on output in an attempt to reproduce an output bug of the Progress preprocessor. Do not use this if the output is expected to be run, compiled or converted.
-lexonly Changes the logic of the preprocessor to eliminate expansions and include file processing. The processing is limited to lexing only and this is useful for debugging the preprocessor itself, but it is not normally useful for other purposes.
-marker <integer> Set the byte to be used as a special internal marker. The default value of decimal 1 is likely to be appropriate for most cases. In general, any value in the ranges 1 to 7, 16 to 31 or 127 can be used (since these are ASCII control characters) as long as the input file does not produce this byte. If that happens, however, an exception will be raised.
-opsys <name> If specified, the OPSYS built-in function will return the text given in <name>. If not specified, “UNIX” will be returned.
-out Directs the preprocessor output to a specific file instead of STDOUT.

Please note that the STDOUT stream is a PrintStream which may inappropriately modify national language characters on output, even if the default language environment is set properly. If you are not using UTF-8 as the default environment, then it is likely that you MUST use this parameter to avoid this problem.
-pathseparator <sep> Specifies the separator string to use between each component in a search path. Defaults to File.pathSeparator (the platform-specific path separator on the system on which conversion is running) if not specified. On UNIX or Linux this is a : (colon) and on Windows this is a ; (semicolon).
-propath <string> This is the equivalent of the PROPATH string and is required to allow the preprocessor to find include files.
-pushback <integer> Specifies a different pushback buffer size for the input stream (the default is 8192 and should be sufficient for most runs of the preprocessor).
-unixescapes Honors the backslash character on input as if it was a tilde character (like an escape character).
-warn <filename> Specifies the file to which errors errors should be written. The STDERR stream is used by default.

Please note that the STDERR stream is a PrintStream which may inappropriately modify national language characters on output, even if the default language environment is set properly. If you are not using UTF-8 as the default environment, then it is likely that you MUST use this parameter to avoid this problem.
-windowsescapes Treats the backslash character on input as a normal character (instead of as an escape character).
-winsys <name> If specified, the WINDOW-SYSTEM built-in function will return the text given in <name>. If not specified, “TTY” will be returned.

All options have to precede the source filename.

It is possible to pass arguments to the top level source code being handled. Any arguments would be coded after the source filename as a list of the last parameters on the command line. Coding arguments must follow the usual Progress rules for coding positional or named arguments but the operating system and/or the shell may dictate some specific escape sequences to be used for characters like &, =, (double quotes) or ' (single quotes). For example the following could be added to the end of the command line to provide 2 named arguments to the source file being processed:

"&name1=xyz" “&name2=\”abc 123\””

Using Debugging

Preprocessor debugging output can only be obtained by directly executing the preprocessor (rather than using the ConversionDriver). The following command line will preprocess the sample basic.p (for the source code, please see the Using the Preprocessor Hints Output section above).

java -classpath p2j/build/lib/p2j.jar com.goldencode.p2j.preproc.Preprocessor -propath .: -debug basic.p

The following is the output generated to the console:

Virtual file: { basic.p }

   /* welcome to a preprocessed program */

MESSAGE "Hello World!".

MESSAGE "This is something else, isn't it?".

exiting  scope --- #0:0/{}{}
    ClearStream state:  clearCount = 0; inComment = no; inString = no; keepTildes = no
--- entering scope --- #1:12basic.p{}{}
        ClearStream state:  clearCount = 0; inComment = no; inString = no; keepTildes = no
--- --- entering scope --- #11:58top.i{isn't it? second (unused positional argument)}{}
--- ---     {2=(2,"second (unused positional argument)")
--- ---     {1=(2,"isn't it?")
            ClearStream state:  clearCount = 0; inComment = no; inString = no; keepTildes = no
--- --- --- entering scope --- #1:41nested.i{Hello World}{&name1="Hello" &name2="World"}
--- --- ---     {2=(2,"World")
--- --- ---     {1=(2,"Hello")
--- --- ---     NAME2=(2,"World")
--- --- ---     NAME1=(2,"Hello")
                ClearStream state:  clearCount = 0; inComment = no; inString = no; keepTildes = no
--- --- --- exiting  scope --- #1:41nested.i{Hello World}{&name1="Hello" &name2="World"}
                ClearStream state:  clearCount = 0; inComment = no; inString = no; keepTildes = no
--- --- exiting  scope --- #11:58top.i{isn't it? second (unused positional argument)}{}
            ClearStream state:  clearCount = 0; inComment = no; inString = no; keepTildes = no
--- exiting  scope --- #1:12basic.p{}{}
        ClearStream state:  clearCount = 0; inComment = no; inString = no; keepTildes = no
 scope --- #0:0/{}{}
    OPSYS=(4,"null")
    SEQUENCE=(4,"null")
    BATCH-MODE=(4,"null")
    SOMETHING=(1,"something else")
    WINDOW-SYSTEM=(4,"null")
    LINE-NUMBER=(4,"null")
    FILE-NAME=(4,"null")
=================================
Preprocessor done with errors.
com.goldencode.p2j.preproc.PreprocessorException: Preprocessor done with 7 error(s).
        at com.goldencode.p2j.preproc.Preprocessor.<init>(Preprocessor.java:694)
        at com.goldencode.p2j.preproc.Preprocessor.main(Preprocessor.java:1007)

At the top of this debug output is the resulting preprocessed output. If the -out option was specified, this output would have been redirected to the file specified.

At the bottom of the debug output is a stack trace. Enabling debug causes the preprocessor to be significantly more sensitive to problems. So in this case the preprocessor thinks there were 7 errors when normally we would not have been notified of any problems. This part can largely be ignored.

The portion in the middle is the core of the debug output. The key here is that this output is organized by the level or scope of the preprocessor itself. For example, each level of nesting via include files causes a new scope to be created. At the entry and exit from each scope there is a display of some state variables and an indication of the line number, column number, filename. Note that in the curly braces are the arguments for that scope level. The first set of curly braces display a string of all positional arguments concatenated together. The second set of curly braces displays all named arguments as they were specified.

All scopes with arguments also display a list of the named arguments in a name = value format. Due to an undocumented quirk in the Progress preprocessor, all positional arguments are also named arguments and each named argument can also be referenced by its position.

Finally, after the last scope is displayed, there is a dump of the preprocessor definitions and their current values. In this case, many of the built-in definitions do not have valid values because of a bug in the FWD preprocessor debug output.


© 2004-2017 Golden Code Development Corporation. ALL RIGHTS RESERVED.