Provides specialized Progress 4GL intermediate form representation (abstract
syntax tree or AST) to Java AST conversion services.
Author(s)
|
Greg
Shah
Sergey Yevtushenko
Eric Faulhaber
Nick Saxon
Constantin Asofiei
Stanislav Lomany
Ovidiu Maxiniuc
|
Date
|
June 21, 2013
|
Access
Control
|
CONFIDENTIAL
|
Contents
Pattern Engine Rule Sets
Data Type Mapping
Literals
Naming
Declaring Class Data
Members and Local Variables
Operators and Expression
Evaluation
Variable References
Assignments
Function/Procedure Definitions
Function Calls
Methods and Attributes
Control Flow
Common Language Statements
Transaction Processing and Block
Properties
Accumulator behavior
Database Support
User Interface Support
Unreachable Code Processing
Unreferenced Tables/Fields
Processing
Open Conversion Issues
Planned Optimizations, Refactoring and
Future Development
Pattern Engine Rule Sets
A great deal of the conversion processing is handled from pattern engine rule
sets rather than being encoded in Java classes. This yields a simpler
implementation whenever the conversion task is such that the pattern engine's
tree walking can be used to mask the complexity. There are cases where
Java helpers are needed and in these cases, methods are exposed to the rule sets
from conversion oriented pattern workers.
It is important to carefully review the standard conversion rule sets in the
p2j/rules/convert
directory of the project.
Data Type Mapping
Summary
The following is a summary of the approach to mapping Progress data types to
Java data types.
Progress
Type
|
Parser Token Type
|
Java
Type(s)
|
Initial Value
|
Notes
|
character
|
VAR_CHAR
FIELD_CHAR
|
com.goldencode.p2j.util.character |
""
|
Progress uses a single data
type to handle both single character values and strings.
For situations such as event processing (keystrokes), the Java char
may need to be used in preference. In addition, the data
storage itself could be handled by java.lang.String.
See wrapper considerations
below.
|
int64 |
VAR_INT64 FIELD_INT64 |
com.goldencode.p2j.util.int64
(or long in cases where it is determined that unknown value can never be
assigned or compared to this variable) |
0L |
Starting with version 10.2B, the Progress uses this
type of integer for results and internal/intermediary computations.
The int64 is a 64-bit signed integral value whose maximum value is 9223372036854775807 and
minimum value is -9223372036854775808. Overflows wrap around to the
minimum value and underflows wrap around to the maximum value.
This is an exact match to the Java primitive long, except for the
case where the unknown value is assigned or tested on this variable.
The logical operators have a predictable response to having the
unknown value as one or both operands. This processing is
identical for all data types. See the table below.
See wrapper considerations below. |
integer
|
VAR_INT
FIELD_INT
|
com.goldencode.p2j.util.integer
(or int in cases where it is determined that unknown value can never
be assigned or compared to this variable)
|
0
|
The Progress integer is a
32-bit signed integral value whose maximum value is 2147483647 and
minimum value is -2147483648. 32-but overflows will not occur as Progress works on 64-bit
values even if both operands are 32 bits.
This is an exact match to the Java primitive int, except for the
cases where the unknown value is assigned or tested on this variable or when two operands
are processed, they are automatically widened and the operation is performed on 64-bit.
The logical operators have a predictable response to having the
unknown value as one or both operands. This processing is
identical for all data types. See the table below.
See wrapper considerations
below.
|
decimal
|
VAR_DEC
FIELD_DEC
|
com.goldencode.p2j.util.decimal
(this class will be implemented using double and NaN to represent
the unknown value; BigDecimal will be used as a backing data type in
the case where it is required; a default number of significant
digits right of the decimal point will be 10 but can optionally be
set to a smaller value based on the "decimals" keyword override)
|
0
|
The Progress decimal is a
signed floating point value whose maximum value is 50 significant
digits in size (the mantissa or significand is a maximum of
50). Up to 10 of those digits can be to the right of the
decimal point (the exponent is a maximum of 10). Thus the
largest value is 99999999999999999999999999999999999999999999999999
and the minimum value is
-99999999999999999999999999999999999999999999999999. If all of
the 10 possible digits of precision are used to the right of the
decimal point, then only 40 digits can be used on the left.
Overflows and underflows both cause the same runtime error "Decimal
number is too large (536)".
The logical operators have a predictable response to having the
unknown value as one or both operands. This processing is
identical for all data types. See the table below.
For more details, please see Decimal
Support.
|
logical
|
VAR_LOGICAL
FIELD_LOGICAL
|
com.goldencode.p2j.util.logical
(or boolean in cases where it is determined that unknown value can
never be assigned or compared to this variable) |
false
|
The Progress logical has
multiple "modes" yes/no, true/false and a user defined
value1/value2. However, all of these are the exact equivalent
of the Java boolean type which has only 2 states, true or false,
except for the case where the unknown value is assigned or tested on
this variable.
The logical operators respond
differently to having the unknown value as an operand:
Operator
|
Left
Operand Unknown
|
Right
Operand Unknown |
Both
Operands Unknown
|
EQ
|
false
|
false
|
true
|
NE
|
true
|
true
|
false
|
LT
|
?
|
?
|
false
|
LE
|
?
|
?
|
true
|
GT
|
?
|
?
|
false
|
GE
|
?
|
?
|
true
|
AND (logical type
operands only)
|
false if right
operand is false, ? if right operand is true
|
false if left
operand is false, ? if left operand is true
|
?
|
OR (logical type
operands only) |
true if right
operand is true, ? if right operand is false
|
true if left
operand is true, ? if left operand is false |
?
|
NOT (logical type
operands only) |
n/a
|
?
|
n/a
|
Note that any logical expression that evaluates to the unknown value
is always considered false (e.g. for purposes of control flow in an
IF statement).
See wrapper considerations
below. |
date
|
VAR_DATE
FIELD_DATE
|
com.goldencode.p2j.util.date
|
null
|
A custom wrapper class will be
written to provide the proper semantics of date processing in
Progress. Where needed, an instance of java.util.Calendar
(obtained through Calendar.getInstance() will be used for the
calendar processing however all operators will operate on the day
number directly for speed and simplicity. The custom wrapper
class will provide Progress semantics and will hide the complexities
of dealing with the J2SE Calendar interfaces.
The logical operators have a predictable response to having the
unknown value as one or both operands. This processing is
identical for all data types. See the table below.
Note that there are special
considerations for handling dates within a database where clause.
|
datetime |
VAR_DATETIME FIELD_DATETIME |
com.goldencode.p2j.util.datetime |
null |
A custom wrapper class written to provides the proper
semantics of datetime processing in Progress. Where needed, an instance of
java.util.Calendar (obtained through Calendar.getInstance() will be used for the
calendar processing however all operators will operate on the day number and time
directly for speed and simplicity. The custom wrapper class provides Progress
semantics and will hide the complexities of dealing with; the J2SE Calendar interfaces.
The logical operators have a predictable response to having the unknown value as one
or both operands. This processing is identical for all data types.
See the table below.
Note that there are special
considerations for handling datetimes within a database where clause.
|
datetime-tz |
VAR_DATETIME_TZ FIELD_DATETIME_TZ |
com.goldencode.p2j.util.datetimetz
Note that this is the only wrapper that does not have the same name
with the datatype from Progress. This is because the hyphen found in the Progress
name is not permitted in inside an identifier of the Java language.
|
null |
A custom wrapper class written to provides the proper
semantics of datetime-tz processing in Progress. Where needed, an instance of
java.util.Calendar (obtained through Calendar.getInstance() will be used for the
calendar processing however all operators will operate on the day number, time and
timezone directly for speed and simplicity. The custom wrapper class provides Progress
semantics and will hide the complexities of dealing with; the J2SE Calendar interfaces.
The logical operators have a predictable response to having the unknown value as one
or both operands. This processing is identical for all data types.
See the table below.
Note that there are special
considerations for handling datetime-tz within a database where clause.
|
rowid
|
VAR_ROWID
FIELD_ROWID
|
com.goldencode.p2j.util.integer
|
n/a
|
The persistence runtime
classes transparently map the ID field for the associated database
row to an integer wrapper. |
recid
|
VAR_RECID
FIELD_RECID |
com.goldencode.p2j.util.integer
|
n/a |
The persistence runtime
classes transparently map the ID field for the associated database
row to an integer wrapper. |
raw
|
VAR_RAW
FIELD_RAW
|
com.goldencode.p2j.util.raw |
0 size array
|
In the Java-based runtime,
this is not backed by a pointer to memory. Instead, it is
backed by a byte array. |
memptr
|
VAR_MEMPTR
|
com.goldencode.p2j.util.memptr |
uninitialized array
|
In the Java-based runtime,
this is not backed by a pointer to memory. Instead, it is
backed by a byte array.
|
handle
|
VAR_HANDLE
FIELD_HANDLE
|
com.goldencode.p2j.util.handle
|
no contained object
|
Note that the parser converts
all widget handles into handle types since in Progress these are
equivalent.
This class is really just a container that holds a backing
object. It provides unknown value processing,
comparison/assignment operator support and the valid-handle()
builtin function implementation.
The specific backing object will need to be determined on a case by
case basis. The handle gets dereferenced when used as a
referent for methods and attributes.
|
com-handle |
n/a |
n/a |
n/a |
n/a |
See also (these are rule sets containing the pattern engine logic that handles
variable conversion):
rules/annotations/variable_definitions.rules (name conversion and class name
determination)
rules/convert/variable_definitions.rules (variable definition and
initialization)
rules/convert/variable_references.rules (variable references)
rules/convert/expressions.rules (unwrapping of variables)
rules/convert/literals.rules (initialization constants)
Decimal Support
The Progress decimal is a signed floating point value whose maximum value is 50
significant digits in size (the hh is a maximum of 50). Up to 10 of those
digits can be to the right of the decimal point (the exponent is a maximum of
10). Thus the largest value is
99999999999999999999999999999999999999999999999999 and the minimum value is
-99999999999999999999999999999999999999999999999999. If all of the 10
possible digits of precision are used to the right of the decimal point, then
only 40 digits can be used on the left. Overflows and underflows both cause the
same runtime error "Decimal number is too large (536)".
The Java primitive double can easily handle the smallest values that can be
represented by a Progress decimal. The Java double is based on a 64-bit
IEEE 754 floating point value where the maximum value is 1.7976931348623157 * 10
308
and the minimum value is 4.9 * 10
-324
. The biggest issue is in duplicating Progress behavior with small numbers
near its limit of precision. For example:
0.0000000001 / 2 * 2 = 0.0000000002
In Java this would be calculated as 0.0000000001. The Progress rounding
behavior for very small numbers is not limited to just the operators but rather
it seems that they have an internal floating point representation that is
precise to 11 digits. Every time they convert the "external"
representation into this form, the rounding occurs.
For example:
0.000000000149 + 0.000000000001 = .0000000001
0.00000000005 = .0000000001
0.00000000015 = .0000000002
In this case, the important rounding occurs before the operation rather than
after. The net of this analysis is that all Progress use of decimal
variables must be reviewed to determine if there are any conditions which are
sensitive to the boundary between the 10th and 11th digit of precision (to the
right of the decimal point). This would be rare EXCEPT in the condition
where the result of a calculation is something like 2.999999999999996 which in
Progress would be represented as 3.0 but which is directly represented in Java.
The following approach seems (based on some non-exhaustive testing) to exactly
duplicate the default Progress results for any double d:
Math.round(d * 10000000000L) / 10000000000.D
For any decimal variable specified with the DECIMALS clause (some value x which
is < 10), the proper result can be generated by converting the two constants
(e.g. 10000000000L and 10000000000.D which are both 10
10
) to a version based on 10
x
. This solution handles the rounding (leveraging the higher precision
values just as the Progress "internal" floating point implementation does)
before the truncation occurs. This is quite important.
Interestingly, since the IEEE floating point format is really a data structure
(e.g. the double has 3 fields for sign, exponent and significant) rather
than a primitive, it has forms reserved for Infinity and NaN (not a
number). The NaN in particular can be considered the analog to the
Progress unknown value. In fact, it is silently propagated through
mathematic operators into the result just as Progress does. So if one or
both of the operands are NaN, the result is NaN. With this idea, it may be
feasible to duplicate the Progress behavior without a Double wrapper! Some
J2SE math functions (e.g. Math.round()) do generate results that are not NaN (an
int or long set to 0 in the case of round()) so calls to such methods must be
handled specially. The
NaN comparison
logic is different from the
Progress unknown value comparison
logic, so this would need to be handled. See
wrapper considerations
below.
The logical operators respond differently to having the NaN as an operand:
Operator
|
Left
Operand NaN
|
Right
Operand NaN |
Both
Operands NaN |
==
|
false
|
false |
false
|
!=
|
true
|
true
|
true
|
<
|
false |
false |
false
|
<=
|
false |
false |
false |
>
|
false |
false |
false
|
>=
|
false |
false |
false |
As in all floating point implementations, the system's "epsilon" and rounding
behavior must be matched up. An epsilon is the smallest floating point
value that can be added to another floating point value such that the result can
be differentiated as not equivalent by the native environment's operators.
As with everything else about Progress decimal support, the Progress and Java
differences are substantial.
In Java, the epsilon is defined (by the language specification) as being equal
to 2
-52
or 2.22044604925 X 10
-16
. This is an exceedingly small number, especially in comparison with the
smallest possible Progress decimal number. So the following application
was written to identify the epsilon for Progress:
def var possibleEpsilon as
decimal init 1.0.
def var
epsilon as decimal init
0.0.
def var
mantissaBits as integer init 0.
repeat.
if ((1.0 +
possibleEpsilon) = 1.0) then leave.
epsilon =
possibleEpsilon.
possibleEpsilon =
possibleEpsilon / 2.0.
mantissaBits =
mantissaBits + 1.
display
possibleEpsilon format "99999999999.999999999999999" mantissaBits.
end.
message "Epsilon = " +
string(epsilon).
message "Mantissa Bits = " +
string(mantissaBits).
It turns out that this program will generate an infinite loop. This is
because once the possibleEpsilon is reduced down to 0.0000000001 (the smallest
representable Progress decimal), the result of the
possibleEpsilon / 2.0
sub-expression will always be 0.0000000001 instead of 0.00000000005. Since
a difference of 0.0000000001 can be detected as different from any other decimal
value, another way of looking at this result is that there is no epsilon value
that is "visible" to the programmer. There is certainly an internal
Progress epsilon value but due to the fact that Progress rounds all intermediate
results (after each operator runs), the result is always something that cannot
be smaller than 0.0000000001 unless it is 0.0! So:
0.0000000001 / 2.0 results in 0.0000000001
0.0000000001 / 3.0 results in 0.0
The important result here is that epsilon testing (the "normalization" of
floating point comparisons to detect if something is really different or not) is
not an issue for Progress applications because of the rounding/truncation
behavior. Internally, the Progress system maintains a larger number of
digits of precision (than 10) but whenever accessed from an application (via an
operator, assignment...) it is always truncated to the lesser of 10 digits (or
the value of the decimals option for that variable) and the truncation uses
rounding to determine the least significant digit. In order to ensure
compatible floating point operations in Java, the following rounding algorithm
will be required after every operator (assuming
num
is the double result to be rounded/truncated and the default of 10 digits is
assumed):
Math.round(num * 10000000000L) /
10000000000.D
Given this rounding implementation on the Java side, the normal epsilon testing
is not necessary AND it is reasonably easy to implement the Progress "decimals"
option which allows the programmer to specify an arbitrary number of digits of
precision that is less than the default (and maximum) 10 digits. So as a
more general algorithm (where
x
is the number of digits supported):
Math.round(num *
10x
L) / 10x.D
The main idea here is that result is converted into the
long
primative type which is used to capture those digits that must be maintained and
to drop the other digits, then it is converted back into a
double
. The Math.round() handles the proper rounding behavior for the least
significant digit.
The situation with large decimal numbers is also difficult. The largest
decimal values cannot be represented in a Java double without losing
precision. The Java double has a mantissa which is represented by 53 (52
explicit bits and 1 implicit bit in normalized forms) bits or ~16 decimal
digits. This means that any integer number with 17 or more digits can't be
saved in double without precision loss (the least significant digits are
lost). Arithmetic and comparisons of numbers with > 16 digits will be
incorrect when using a double.
To address this large number issue, decimal operations must be reviewed for any
situation that could overflow this boundary. In these cases, the
BigDecimal class will be used.
The Java double overflows by ignoring any overflow and setting the value to the
maximum value (positive or negative infinity). The Java double underflows
by "wrapping" to (positive or negative) 0. Special "boundary" processing
will need to be written for some generated code due to the fact that Progress
has a significantly smaller precision, a significantly smaller maximum mantissa
and different overflow/underflow behavior.
To Wrap or Not to Wrap?
Where possible, the resulting generated code will be significantly easier to
write, read and maintain when using primatives instead of object wrappers (e.g.
int instead of Integer). There is also a performance advantage to using
primatives since this avoids the overhead of temporary variables and method
calls to handle expression processing. This is an issue for integer, decimal (
note that NaN may be a solution that still allows primative
doubles to be used in this case
) and logical types since in Progress, such variables can be assigned and
compared with the unknown value. The unknown value is a similar thing as
the Java concept of
null
. The main difference is that Progress silently allows unknown value to
"propagate" where Java tends to catastrophically fail at runtime (e.g.
NullPointerException
). In Progress an arithmetic sub-expression will return an unknown value
if either or both operands are the unknown value. Since primitive Java
types cannot be assigned or compared with
null
, wrapper objects or some other solution must be used in that case.
In the following conditions an integer, decimal or logical variable must be
represented by a wrapper class:
- All parameters defined via a "define parameter" language statement
that have an input or input-output type are passed in and thus they may be
passed as the unknown value. An output type parameter doesn't require
wrapper support in and of itself because its origin is known.
- If the variable is shared and is defined external to this procedure
then a wrapper must be used because it must be assumed that it could be
assigned the unknown value or compared against the unknown value either
upstream. Any new shared variables in this procedure do not need to be
wrappered unless some other condition would cause this to occur.
- If the variable is explicitly initialized to the unknown value.
- If the unknown value is ever assigned to that variable through the
assignment operator or the assign language statement.
- If the unknown value is ever used on the other side of any binary
operator (all the arithmatic operators handle operands of unknown value by
returning the unknown value) including multiply, divide, kw_mod, plus,
minus, equals, not_eq, gt, lt, gte, lte, kw_and and kw_or. The
kw_begins, kw_matches and kw_contains binary operators do not operate on
integer, decimal or logical variables so they are not at issue.
- If the variable is ever passed to an external procedure as an
input-output or output parameter (in a run statement), a wrapper will be
used to avoid a complex recursive/multi-file search for downstream
comparisons/assignments against the unknown value. An input type
parameter doesn't require wrapper support in and of itself because its
origin is known. Input-output or output parameters of an external
procedure (run statement) are potentially modified in the called procedure
which means that this procedure could see an unknown value which is
generated externally.
- If the value is ever assigned or compared to the return value of a
function call which can return the unknown value. A subset of the
built-in functions can do this and some user-defined functions can return
the unknown value.
- If the value is ever assigned or compared to the return value of a
method or attribute.
- If the variable is defined as a parameter of a function (function
declaration) that has an input or input-output type. An output type
parameter doesn't require wrapper support in and of itself because its
origin is known. If this function is only called from the current
procedure, then the call sites can be examined to determine if the input can
ever be unknown. If the function is called externally (through the "in
handle" form), then a multi-file review would have to occur which is beyond
the scope of this pass of the conversion.
- If the variable is ever passed in a function call in a position
that matches an input-output or output parameter. Function call parms
that match up with input-output and output types are obviously a problem in
that hidden code may be invoked to assign the unknown value to that variable
before returning. It is possible that a local function could be
analyzed but to limit the complexity of this processing, this will not be
done. Built-in functions and remote functions are not as easy to
analyze. In the case of built-in functions, each one would have to be
reviewed manually as a "black box" since no source is available in these
cases.
- If any variable or field in any expression can be the unknown
value, then any other variable assigned from expression can be silently set
to the unknown value as a result of the expression evaluation. This
last implication causes a requirement to iteratively process expressions
expanding the list of possible nullable variables based on assignments from
other variables/fields that are nullable, until no additional nullable
variables are found. This will take an undefined number of iterations
to finish since each iteration can add to the list of nullable variables and
then a subsequent scan will lead to new nullable variables that are only
nullable because of the last scan's changes... and so on.
See also:
Wrapper Usage
Other Variable-Like Entities
Progress allows many user-defined resources that are not treated as
variables. A non-exhaustive list includes streams, frames, widgets,
queries, buffers, and temp-tables. These resources cannot be used in
expressions nor can they be used as a storage type in the database, however in
Java their representation will be as objects. Each of these resources will
be converted by resource-specific code and this section of the conversion does
not address such resources.
Literals
The following is a summary of the approach to mapping Progress literals to Java
literals.
Progress
Literal
|
Parser Token Type
|
Java
Literal/Object
|
Notes
|
true or yes
|
BOOL_TRUE
|
true
|
This is an exact match. |
false or no
|
BOOL_FALSE
|
false
|
This is an exact match. |
? (unknown value)
|
UNKNOWN_VAL
|
If used as the right side of
an assignment, this is covnerted to a setUnknown() method call on
the BaseDataType object that is the lvalue.
If used as 1 operand of a EQUALS, the other operand will be passed
to CompareOps.isUnknown().
If used as 1 operand of a NOT_EQ, the other operand will be passed
to CompareOps.notUnknown().
In Progress it isn't possible to compare two instances of the
unknown value literal.
Other usage converts to an instance of
com.goldencode.p2j.util.unknown.
|
At first glance, null is an exact
match. However, in order to eliminate the need to place null checks throughout
code and to refactor much of the code, wrappers are being used which
internalize the unknown value.
|
integer literal
|
NUM_LITERAL
|
Java integer literal
|
This is an exact match since
Java integer literals default to int. |
int64 literal |
NUM_LITERAL
with "use64bit" annotation set to true. |
Java long literal |
This is an exact match.
The decision which type of numeric literal is taken dynamically at conversion time:
- If a literal is parsed and fits into 32-bit space, the int type is used;
- Else it it can be represented on 64 bits, the long type is used instead;
- Otherwise, it will be represented as a decimal/ double.
|
decimal literal
|
DEC_LITERAL
|
Java floating point literal |
As long as the value is within
the mantissa limits listed above (16 decimal digits), this is an
exact match since Java floating point literals default to
doubles. Otherwise, a BigDecimal must be used.
|
date literal |
DATE_LITERAL |
com.goldencode.p2j.util.date |
There is no date literal in Java. An instance of the
com.goldencode.p2j.util.date will be created instead using the static method
date.fromLiteral(). |
datetime literal |
DATETIME_LITERAL |
com.goldencode.p2j.util.datetime |
There is no datetime literal in Java. An instance of the
com.goldencode.p2j.util.datetime will be created instead using the static method
datetime.fromLiteral(). |
datetime-tz literal |
DATETIME_TZ_LITERAL |
com.goldencode.p2j.util.datetimetz |
There is no datetime-tz literal in Java. An instance of the
com.goldencode.p2j.util.datetimetz will be created instead using the static method
datetimetz.fromLiteral(). |
string literal |
STRING
|
Java string literal
|
Any string options will be
removed and any enclosing single or double quotes will be
removed. Escaped characters (using the tilde character) will
be converted to native representations and any characters that need
to be escaped will be so escaped:
- '\b' - backspace
- '\t' - horizontal tab
- '\n' - linefeed
- '\f' - formfeed
- '\r' - carriage return
- '\"' - double quote
- '\'' - single quote
- '\\' - backslash
- '\ooo' where ooo is an octal character literal
- '\uxxxx' where xxxx represents 4 hexidecimal digits -
unicode character literal
The final result when output will be output inside enclosing double
quotes but there is no need for those quotes inside the AST form.
|
See also:
LiteralConverter
ExpressionConverterWorker
Naming
Progress identifiers and Java identifiers are similar but there are some
important incompatibilities. Most importantly the following characters are
valid in a Progress name but invalid in a Java name:
#
%
&
- (this is the hyphen character, not the underscore)
While Java does allow Unicode letters to be included in identifier names, the
plan is to limit valid letters to the 7-bit ASCII letters of a - z and A -
Z. Java identifiers will always start with one of these letters and in
subsequent characters a letter, any digit (0 - 9), the underscore (_) and dollar
sign ($) can all be used. All of these Java identifier characters can also
be present in a Progress identifier.
So there is an inherent problem caused by a reduced identifier namespace on the
target platform. The troublesome characters (all of which are used in real
application code) must be converted to something that is unlikely to conflict
with other visible (scopes must be taken into account) names. Conflicts
must be detected and resolved automatically, with overrides being available (via
hints) for manual intervention.
As an example of the conflict, assume that hypens were converted directly to
underscores. The following 2 names "hello-world" and "hello_world" would
both be converted to "hello_world". Thus if both identifiers were in some
accessible scope simultaneously, then the resulting code would not be correct.
One should note that in Progress, the convention is to use the hyphen as a word
delimiter in an compound word identifier. In Java, the convention is to
use a "camelCase" identifier (Java identifiers are case-sensitive) for the same
purpose.
For different types of names, different conversion strategies will be followed
allowing the Java standards to be matched as closely as possible. The
following is a summary of the approachs to be used.
Progress
Identifier
|
Java
Identifier |
Strategy
|
Conflict Resolution
|
directory name
|
package name
|
Package naming will be
somewhat direct (following the Java standard):
com.<customer>.<application>.<directory>
|
TBD
|
file name
|
class name
|
A set of hints will assist in
class naming. Where common sequences are encountered such
characters might be expanded based on hints. For example, so
and po might be expanded to ServiceOrder and PurchaseOrder
respectively. A file ending in -r or -x might be have
ReportBody or Export appended respectively.
|
TBD
|
procedure name
|
method name
|
TBD
|
TBD |
function name
|
method name
|
TBD
|
TBD |
variable name (normal variable
that is assigned or passed as a parameter)
|
variable name
|
Hypens will be removed and the
words will be camelcased. This will result in lowercasing some
letters (including the first char of the identifier) and uppercasing
the first character of subsequent words. Words will be assumed
to be separated at the hyphens.
|
TBD |
variable name (never assigned
or passed as a parameter)
|
constant name
|
All letters will be uppercased
and words will be separated by an underscore (_).
|
TBD |
schema name
|
TBD
|
TBD |
TBD |
See also:
NameConverter
NameConverterWorker
Declaring Class
Data Members and Local Variables
Variables can be generated in a Progress program by any of the following
language constructs:
Language
Statement/Construct
|
Conditions
|
Options
Are Controllable
|
define variable
|
always
|
yes
|
define parameter
|
when TABLE-HANDLE, AS or LIKE
keywords are encountered
|
yes
|
format phrase
|
when following an unrecognized
symbol and the AS or LIKE keywords are encountered
|
yes
|
function parameter
|
when TABLE-HANDLE or AS
keywords are encountered |
no
|
message
|
when SET or UPDATE clause
occurs with a child of SYMBOL and a child of the SYMBOL which is the
AS keyword (as opposed to a child which is an lvalue that has an
optional AS keyword which is bogus)
|
no
|
Reference IDs
In each of the locations above where options can be defined for a variable,
those options are saved as annotations. In addition any AST that
represents a reference to that variable will have all the non-default options
annotated. To allow variable references to be cross-referenced back to the
original AST node that defined the variable, a "temporary index" is maintained
for each variable definition. This index is not project unique (such as
the AST node ID) but rather is only unique within a given file. Each
reference to a variable has the temporary index written to an annotation.
The reason the node IDs are not used for this purpose is because this
cross-reference is best created at parse time and the node IDs don't get created
until after parse time due to limitations of how ANTLR instantiates AST
nodes. Post parse fixups are used to convert these temporary ID
annotations into an annotation containing the real node ID of the defining AST
node.
Shared Variables
Support for shared variables is handled via a shared variable manager.
This runtime class is called
SharedVariableManager
. It is based on the
ScopedSymbolDictionary
and is designed to exist in a per client/session basis (stored in the security
context). This allows all transactions to access these shared variables
but for this access to be done maintaining context specific state. This
approach is a replacement for features that would normally be provided via
thread local variables which can't be used in the transaction server.
Whenever a new external procedure scope is started, if new shared variable
references are added, the corresponding Java source code will add a scope.
Likewise this same Java method will use a
finally {}
clause to delete the scope on exit. Global variables are added to the
global scope and regular shared variables are added to the current (top)
scope. Such additions only occur if the definition is defined with the NEW
keyword. This allows "downstream" methods to access the most current
instance of a shared variable by executing a lookup using a given name and when
the method that defines a new shared (non-global) variable ends, the scope must
be deleted (deleting all references to variables defined in that scope).
Any process that accesses a shared variable gets the shared variable manager out
of the current security context and then does a lookup of the variable name and
assigns the reference to the resulting local instance.
Where possible, shared variable usage will be converted into object references
passed as method parameters or contained in other classes passed as
parameters. In the first pass conversion, this is not planned due to the
extreme use of shared variables (there can be tens or even hundreds of these
defined in a procedure) and the arbitrary depth at which such references can be
used makes it unwieldy to easily convert these to passed parameters. For
this reason, the original semantic must be maintained. The shared variable
manager provides this same semantic at runtime.
Note that this same infrastructure is used to provide support for
shared streams
.
See also:
rules/convert/variable_definitions.rules
rules/convert/input_output.rules
rules/convert/control_flow.rules
rules/annotations/shared_resources.rules
Extents
Progress arrays are very similar in nature to Java arrays. A variable
definition can be an array if it is explicitly specified or if it is not
specified and the variable is defined like another variable or field (a LIKE
clause) that is defined with extent > 0. Either way, there will be a
resulting KW_EXTENT with a numeric constant size and a "long" type annotation
named "extent" on the variable definition root node.
The extent processing is mapped to array variable definitions and array variable
initializers
. This is a fairly direct mapping.
The referencing of array elements is also very similar between Java and Progress
(both use numeric expressions inside postfixed square brackets to index the
element). The following differences exist:
- Progress indexes arrays using a 1-based index. Java uses
0-based indices.
- There are certain language statements (e.g. DISPLAY) that support a
range of indices using a KW_FOR NUM_LITERAL construct inside the square
brackets (after the first numeric expression). This form can't be used
inside expressions but it will have to be handled.
- There are certain language statements (e.g. DISPLAY) that support
referencing all elements when an array variable name is used with no
subscript at all.
Requirement 1 is handled inline during conversion. The Java subscript
operator [] is directly mapped to the Progress extent operator [].
NumberType.subscript() is used to decrement the 1-based subscript and return a
0-based version. For array variables, this method takes a reference to the
BaseDataType[] being used. This allows the subscript() method to handle
bounds checking and generate the proper error.
Please note that using an array index
that is out of bounds during a NO-ERROR (silent error mode) language
statement/assignment will fail with an IndexOutOfBoundsException since we
currently directly map array access into the standard Java subscript operator
[]. While we do have a central method in NumberType.subscript() to handle
the bounds checking and error generation, in silent error mode this will return
an out of bounds value.
Note that any constant (NUM_LITERAL) array subscript in Progress is converted to
the proper 0-based constant in Java (no use of the subscript() method is needed
in that case).
For fields, since all access is via get/set methods, the [] operators are not
used and the array index is passed as the last parameter of the method call (the
only parm for a getter and the 2nd parm for a setter). Bounds checking and
error generation is handled inside the persistence classes in that case.
This same approach is also used in some of the frame field cases in order to
properly dereference the widget in the frame (which is done via getter methods).
Requirements 2-3 are handled by expanding these array references into the proper
list of references. Then all downstream processing is otherwise the same once
the implicit reference is expanded into its explicit list. This expansion
works conditionally in case #2, only working where the start of the range is
specified as a constant (NUM_LITERAL).
No support for array subscripts with
ranges that use a non-constant expression as the start index as in myvar[ 2 + j
FOR 3 ]. Such cases can not be used in an expression, instead they can
only be used as implicit reference to a range of index positions.
Initial Values Logic
All variable definitions are explicitly initialized to some known value (see the
above table
). Either a
null
is assigned or the appropriate constructor is called with the literal
initializer (default or as specified in the Progress source). The
initializers are all rooted at a node of type KW_INIT and these values can come
from another variable or field (LIKE clause) or can be explicitly specified in
the source code. Either way, there will be a resulting list of
initializers which are walked at conversion time to emit the appropriate
constructors or
null
.
In the case of array processing (see
Extents
) the number of explicit extents may exceed the number of initializers
provided. This is a different behavior than Java where one can specify one
or the other but not both. If there are no explicit initializers, then all
elements of the array are initialized to the default value. If any
initializers are speciifed, these are applied to the matching elements in the
array and then all remaining elements without an explicit initializer are
initialized to the last initializer in the list.
Wrapper Usage
See
To Wrap or Not To Wrap
.
Based on the extreme complexity of this task, the effort to definitively
determine whether an integer, decimal or logical variable requires a wrapper is
going to be deferred at this time.
As a first pass, all variables will
always be wrapped with the ability to add an optional hint that allows an
override to a primative type when it is known to be safe.
Note that moving to J2SE 5.0 does provide some small relief by providing a new
language feature for auto-boxing and auto-unboxing. This largely makes the
use of wrappers transparent (it wraps or "boxes" a primitive into a wrapper when
needed and unwraps or "unboxes" a wrapper into a primitive when needed).
This makes it possible to write arithmetic and logical expressions directly
using wrappers.
However, the following limitations (which are also quite
significant
) apply:
- null cannot be
assigned to a primitive and any unboxing or usage of a null wrapper will result in a null pointer exception.
This means that null checks
will have to be embedded throughout expressions which greatly reduces the
value of auto-boxing/auto-unboxing.
- The == operator cannot be used because it does not give consistent
results when comparing two operands where either one or the both of them are
wrappers. This is due to the fact that when this operator is dealing
with objects (versus primatives), it is comparing object references and this
will generally yield false-negatives in most situations.
- Overloaded method signatures can be tricky because method
resolution does change with auto-boxing added. These method resolution
changes are supposed to be compatible but this remains to be seen.
- The combination of auto-boxing and generics means that the ternary
operator will auto-cast the 2 expression results to the least common
denominator result if their types are different.
At this time, we do not plan to implement J2SE 5.0 since the result is not much
better. We can do our null checks and then if everything is non-null, we
can unwrap into temporary primitive variables and then wrap the result back up
and assign them back.
The problem with this solution is that many expressions require significant
refactoring to make this work. For example, in the case where a
null
check fails, all possible assignments that could be forced to the unknown value
must be detected. In addition (possibly more important), any side effects
of the expression must be triggered in a safe manner. For example, if a
function is called as part of an expression, that function may have side effects
that the rest of the procedure relies upon. The conclusion: not only is
the refactoring of the expression highly non-trivial but there are very strong
reasons to actually allow the expression to execute even in the presence of the
unknown value (which is how Progress works today). If
null
is used as a proxy for the unknown value, this aspect of the behavior cannot be
duplicated since Java generates a runtime error for these cases. Since the
J2SE wrappers have no way of encoding the unknown value except setting the
current reference to
null
, this eliminates the J2SE wrappers from being a useful/good solution to this
problem.
The best solution is to implement custom
classes (corresponding to each Progress data type) to handle all of the
following:
- Stores the data in a form
that provides an exact match with the Progress data.
- Stores the state of whether
this instance is really equal to the unknown value.
- Implements all of the
operators and replacements for the Progress built-in functions such that the
100% Progress compatible results are generated. Note that for integer
and logical, this is primarily a matter of handling the unknown value in a
transparent manner. For decimal, there is the added requirement for
the rounding/truncation to be implemented after each operator and function.
All of these custom classes will reside in the com.goldencode.p2j.util package
and they will each have a class name the same as the lowercase data type name
(e.g. class integer for the integer data type). This is unique (does not
conflict with J2SE), is easy to read and looks more like a primative is being
used. The down side is that this is an exception to the normal
Sun-recommended naming rules for classes.
In a perfect world, the exact look of (arithmetic and logical) expressions could
be maintained if Java supported operator overloading. Since that is not
possible, methods (with intuitive names) will be written to substitute for all
operators. Unwrapping will only be needed in those places where the
expression result requires a primative type. So assignments and many
method calls will use the object form returned as an expression result.
However, when expression results are used directly with a Java language
construct (e.g. if/else, array subscripts) or in a J2SE-provided method that has
primatives in the signature, the primatives must be used. In these cases,
the custom wrapper object will be unwrapped.
Note that because each of these classes will internally track whether the
variable is set to the unknown value, all variables will have a valid (non-null)
reference. As long as the conversion code is written with this in mind,
null pointer exceptions will be eliminated (from expressions).
Progress function/procedure parameters have input-output and output forms that
cause the data in the calling procedure to be changed by the called
function/procedure. This is very similar to the C/C++ semantic of a
parameter passed by pointer. On return, the data pointed to by the calling
procedure has been changed. In Java this is easily handled when some kind
of business/container object is passed as a parameter but it is not directly
possible with the normal primatives or their standard J2SE wrappers (e.g.
Integer) which are immutable. While some instances of function/procedure
calls may end up being generated in such a way that a more abstract object is
passed in instead of the primatives, this will require significant refactoring
(of procedural code into object oriented code) and may not always be
possible. For a general purpose solution to this problem, it seems better
to ensure that all of
the custom wrapper classes are designed
as mutable, with an assignment-type interface and setters to allow modification
of the wrapped data in the called method
, where the variable being assigned was defined as an input-output or output
parameter.
Detecting Constants
There is no concept of a constant variable in Progress but some variables may
essentially be "read-only". As an optimization (and a good practice for
Java code) such variables should be converted as "final static". To do
this, it is required that all assignments, run statements and user functions...
are examined to determine if any possible change can ever be made to each
variable. This presents similar issues as the
To Wrap or Not To Wrap
discussion, however it is likely that this is significantly more feasible to
implement.
Format String Support
Variables can have explicit format strings or can implicitly have a format
string based on a LIKE clause (where the database field or variable has an
explicit format string). In these cases, the any of the formatted output
language statements will not use the default data type format string, but
instead will honor the explicit or implicit format string that is specific to
the variable.
These cases are detected and a class member will be used for each unique format
string in a given class. The member is created with the following
approach:
public static final String
FMT_STR_n = "...";
The number
n
will be a generated sequence number to make the string name unique.
At annotation time, a set (the format string is the key and the name of the
class member is the value) of these are accumulated and are used to
cross-reference all format string usage. This allows a single definition
to be reused in multiple places. For this reason, if 2 variables are both
explicitly defined with a format string of "x(40)", then the same FMT_STR_x
constant will be used for both.
The proper hierarchy for determining the correct format string to use is:
- Use an explicit, in-place format string (part of a format phrase on
any language statement that generates formatted output).
- If the variable is being used as a simple variable reference
(rather than an expression or constant):
- Use the explicit format string for that variable, if one was
specified.
- Use the implicit format string that was associated with the
variable via a LIKE clause.
- Use the data type specific default format.
The formatted I/O processing (all putField variants) check on this (and honors
it) for each field that is a simple variable reference.
Operators and
Expression Evaluation
All operators are fully implemented except the highly specialized CONTAINS
operator (see page 960 of the Progress 4GL Reference, in the section on the
record phrase) which handles word index searches.
Each operator is converted to a static Java method call that is implemented in a
data type wrapper class (
integer
,
decimal
,
date
,
logical
,
character
) or in a common operations class such as
MathOps
or
CompareOps
. When an operator is encountered, the left operand (and sometimes the
right operand in the case of date operations) is checked for its type. To
do this, the leftmost descendants are checked until a literal, variable, field
or function is found. All of these nodes have a type that determines the
type of the resulting operation. Depending on this type, the conversion
strategy is picked.
The structure of the Java expression is an exact duplicate of the Progress
source tree structure. This is required to maintain logical correctness in
the result. The only deviations:
- The Progress logical NOT operator has a very low precedence and the
Java NOT operator has a high precedence (the same as unary plus and unary
minus). If the real operator is used, the logical not would have to
have its operand parenthesized to maintain the same result as in
Progress. If this was not handled, then the Java NOT would bind to the
first emitted portion of the operand and this would be evaluated before any
subsequent operator. However, since logical.notOperator() is used
instead, the expression subtree will automatically be paranthesized. This is
the difference between:
- In Progress the equality and inequality operators have the same
precedence level as the comparison operators (>, <, >=, <=) but
in Java, these are split into 2 precedence levels.
The logic for the operator conversion is encoded in the
OperatorConverter
class.
Variable References
User Defined Variables
There are 2 forms of variable reference:
- A simple reference.
- An unwrapped reference.
A simple reference is used when passing a variable to a function or procedure
call. It is also used as the left operand in certain operators and as an
lvalue for an assignment. This is called a simple reference because it is
emitted as the variable's object instance name (converted to the proper Java
form).
An unwrapped reference is how the variable value is accessed in a Java language
contruct. This is usually a control flow construct like "if" or
"switch" where the result of an expression must be unwrapped using
booleanValue() or intValue() respectively. For example, a Progress integer
variable named "i" is represented by an integer variable named "i" in Java and
when used in expressions requiring direct access to the primative value, it is
dynamically unwrapped as "i.intValue()". This works because Progress has
no inline assignment forms inside expressions. In Java there are many
operators that modify and reassign their lvalue (e.g. ++ and --
increment/decrement their lvalue and reassign, the assignment operator itself
can be used inside a Java expression and there are many other modify and
reassign operators such as += or *=).
Without such assignment operators, expressions essentially access variables in a
read-only manner with the exception of user-functions which can have
input-output and output parameters. Such user functions can modify the
original variable's value and this is handled using the assign() method of the
wrappers. The method/attribute support has not been investigated fully at
this time, but there may be implications for these as well.
Since null is not being used to track the unknown value, all wrappers are always
expected to be valid for calling methods and operator replacements. This
eliminates the need to protect variable references using "bracketed"
null
checks. This would have otherwise been necessary at the beginning of any
expression that made a variable reference and possibly in the middle of
expressions to check the return (or the input-output or output) from a function.
Each
operator
is implemented as methods that are "unknown value aware". This way
the operator will return unknown value or other correct results as
needed. For example, many Progress operators (e.g. + or -) silently
return the unknown value if either or both operands are the unknown value.
Global Variables
Progress provides constructs that look like global variables (e.g. opsys,
current-language, progress...). These global variables are termed
"built-in functions" by Progress but they act like read-only variables. In
particular, they take no parameters and even take no parenthesis.
A helper class com.goldencode.p2j.util.EnvironmentOps provides static method
calls ("getters") to provide the values associated with the global
variables. Note that at this time, the return values are hard coded but
eventually they will be backed by lookups from the P2J directory. With
this approach, the values to be passed at runtime can be controlled by the
implementer and are not hard coded to a particular customer environment or
application.
See also:
EnvironmentOps.java
convert/variable_references.rules
Assignments
All assignments (either "assignment" statements or the ASSIGN language
statement) follow these conventions:
- Always have an lvalue as the 1st child.
- Never can be a declaration/definition of a new variable (this is
not possible in Progress).
Since the wrapper approach eliminates the possibility of having a
null
pointer, all assignments can be made by calling the assign() method of the
wrapper. This is required because the standard Progress semantic is to set
the value rather than the Java approach of setting the reference. This
difference can only really be seen in cases where a function or procedure with
input parameters is called. In these cases, the assigning of the value
would "bleed through" into the calling procedure.
In a function or procedure where a variable is an input parameter, it can
simply/safely be assigned the using the Java assignment operator.
The simple form of the ASSIGN statement is that which just is a list of lvalues,
each followed by an assignment operator and an expression of an assignable
type. Such a form is converted by the same rule sets that convert normal
("standalone") assignments. The more complex forms of the ASSIGN statement
which handle the movement of data from input buffers to variables or fields, are
not yet supported.
Function/Procedure Definitions
User-defined functions are differentiated from procedures in the following ways:
- Functions can (actually MUST) return a scalar instance of any of
the basic data types, procedures can only set the return-value "global
variable" which is a character value.
- Functions are limited in the resources they can define.
- Functions can't be referenced before they are defined, though a
FORWARD declaration can be made which is just a signature definition.
Both of these constructs are emitted as Java instance methods (METHOD_DEF) for
the class that maps to this file. The function or procedure name is
converted to a Java method name. Any use of the KW_PRIVATE in the Progress
source converts to a private access specifier, otherwise the default access is
public. Procedures always return void but functions always have a wrapper
data type as a return value.
Both types of definition support input, input-output and output type
parameters. In Java, these parameters are all emitted as a simple variable
definitions (of the form:
classname reference
) that are comma-separated. However, the assignment of these parameters is
differentiated. Input parameters have their
reference
assigned in order to ensure that calling code won't see a change in the
parameter after the function/procedure returns. Input-output and output
parameters get their
value
assigned in order to ensure that the caller sees the change in contained data
after return. All assignments check the type of variable before emitting
and use the "parmtype" annotation to make this determination. The parser
ensures that all parameters have this annotation, even if the KW_INPUT is
omitted (it is optional in some circumstances).
The concept of an input-output or output function parameter is only implemented
for user-defined functions and internal/external procedures. No built-in
Progress functions change the original referenced variable or field passed as a
parameter. The only "returned" data from a built-in function appears to be the
return value itself. There may be environmental/database side effects of
function calls to built-ins. However, no function parameters are ever modified
based on calling a built-in function.
The case of input-output and output parameters is handled using the
mutable wrappers
where the called function/procedure will modify the contained data such that the
caller will see a different value after the call using the same reference.
In this case, the normal assign() method of the wrapper will be called to set
the value. However it is important to note that in the case of an input
parameter, the normal Java "=" assignment operator will be used to modify the
reference rather than the value (referred to by the reference) since any changes
should NOT be visible in the calling code.
Only scalar, table and buffer parameters are allowed. No array parameters
are possible in Progress.
At this time, there is no support for
table or buffer parameters.
The user-defined function definition and procedure definition are converted
directly to a Java method signature rooted at a METHOD_DEF node which is rooted
as a peer with the external procedure's METHOD_DEF subtree. The text of
the METHOD_DEF is the function name (a valid Java function name) and the return
type and access specifiers are defined in "rettype" and "access" annotations
respectively. All parameter definitions (if any exist) are rooted at the
first child which is an LPARENS. The next child is a BLOCK that contains
the method body.
The creation of these nodes in the target tree is different since the structure
of the source tree is different for the two cases. In particular, the
major difference is where the parameter definitions exist. In the function
definition, the parameters are child nodes (PARAMETER) of the FUNCTION and
KW_FUNCT (the function definition itself). In the procedure definition,
the parameters are child nodes (DEFINE_PARAMETER) of the BLOCK that is a child
to the PROCEDURE but is NOT a child of the KW_PROC which is the actual procedure
definition. Procedure definitions rely upon annotations to note how many
parameters there are and then in the conversion step, an LPARENS node is either
created (if "numparms" annotation in the PROCEDURE node > 0) or not. If
it is created, its ID is left behind as an annotation called "parmroot" in the
PROCEDURE node. All DEFINE_PARAMETER statements have a "procrefid"
annotation that is the ID of the enclosing PROCEDURE node (or the AST root node
which is BLOCK and defines the external procedure). This allows the
DEFINE_PARAMETER processing to occur much later than the PROCEDURE definition
itself, yet it can reference its way back to the parent node at which the method
signature must be rooted. The order of the DEFINE_PARAMETER statements
determines the order of the parameters in the signature (just as in
Progress). The function definition doesn't have this problem as it can
emit in-line with the tree walk since the parameters are child nodes of the
definition itself.
Input and input-output parameters never get initialized in a procedure or
function since they always take the value from the caller. Output
parameters are always initialized. DEFINE_PARAMETER allows explicit
initialization using the KW_INIT, if this construct is found on an output
parameter, an in-block (outside the method signature) assignment is made using
the provided data. If no explicit initializer is specified (this is always
the case for function parameters since one cannot specifiy KW_INIT on a
PARAMETER), then output parameters still get initialized in the block
itself. Procedure parameters get initialized at the same location as the
DEFINE_PARAMETER appears in the block. The
default initializer
is the same as for DEFINE_VARIABLE. Function parameters are both more and
less troublesome. All function output parameters are always initialized to
the unknown value, rather than following the rules of normal variable
defaults. Since one can never provide an explicit initializer to a
function parameter, this means that there is only 1 case. However, since
Java cannot initialize parameters as part of the method signature itself, this
initialization must occur in the block that follows the function definition.
This is a problem because the PARAMETER nodes are children of the LPARENS and
its parent KW_FUNCT but the BLOCK in which they must be attached is the right
side sibling of the KW_FUNCT grandparent. In other words, the BLOCK hasn't
been created yet so it cannot be the target for the parameter creation. To
resolve this issue, a BOGUS node is created in place (as a child of the method
signature) all parameters are created as children of this node. They all
get the proper IDs and parenting. Then this node is detached from the tree
and stored in a "depot" (see the
TemplateWorker
) under the name "funcparms". After creation of a matching BLOCK node, if
the BLOCK is a child of FUNCTION (thus a sibling of KW_FUNCT), then the BOGUS
node is retrieved from the depot and attached as the first child of the BLOCK.
See the convert/function_definitions.rules and
convert/procedure_definitions.rules for more details. Please note that the
convert/variable_definitions.rules is also heavily involved with processing
parameters.
Please also see the section on
Control Flow
for details on the RUN and RETURN language statements which are necessary to use
function and procedures.
Function Calls
The following documents the mapping between built-in functions and their Java
equivalents. References to decimal, integer, logical, date, character,
MathOps, and CompareOps refer to classes in package com.goldencode.p2j.util.
Progress
Function
|
Java
Equivalent
|
Category
|
Token Type
|
Nullable
|
Optional Parameters
|
DBCS Issues
|
Supported
|
Notes
|
_CBIT
|
character.testBitAt()
|
bit manipulation
|
FUNC_LOGICAL
|
yes
|
no
|
yes
|
yes
|
logical _cbit(string,
integer)
The first parameter is a string with 0 or more characters.
The second parameter is the bit position to test. The input
string is treated as a multibyte bitfield. The first character of
the string corresponds to bit positions 0-7 (if it is non-DBCS and
presumably in a DBCS character the positions would be 0-15).
In a non-DBCS charset, the second char would correspond to bits
8-15.
The return is always false
if the empty string is input. For any string of > 0 length,
the return is true if the bit in the specified position is 1 and
false if it is 0.
Sample Code:
def var i as int.
def var j as int.
def var b as char.
def var c as char.
do i = -1 to 257.
b = "".
c =
chr(i).
do j = 7
to 0 by -1.
if _cbit(c, j)
then b = b + "1".
else b = b + "0".
end.
message i
" = " b.
end.
|
ABSOLUTE
|
MathOps.abs(integer)
or
MathOps.abs(decimal)
|
math
|
FUNC_POLY |
yes
|
no
|
no
|
yes
|
returns INT or DEC depending
on the type of the input
|
ACCUM
|
Accumulator (abstract base
class); concrete implementations:
- AverageAccumulator
- CountAccumulator
- MaximumAccumulator
- MinimumAccumulator
- TotalAccumulator
|
database
|
FUNC_POLY |
no
|
|
|
yes
|
Resides in the com.goldencode.p2j.util
package.
Each concrete subclass defines the following methods for results
retrieval; each class' implementations of these methods may
return different data types, so these method signatures are named by
convention only, and are not enforced by a common interface:
getResult() to retrieve the cumulative
result
getResult(Resolvable breakGroupKey) to
retrieve the current break group result
|
ADD-INTERVAL |
DateOps.addInterval() |
date/time |
FUNC_POLY |
yes |
no |
no |
yes |
|
ALIAS
|
ConnectionManager.alias() |
database |
FUNC_CHAR
|
yes
|
|
|
yes
|
|
AMBIGUOUS |
RecordBuffer.wasAmbiguous() |
database
|
FUNC_LOGICAL |
no
|
|
|
yes
|
|
ASC
|
character.asc()
|
type conversion
|
FUNC_INT |
yes
|
yes
|
yes
|
yes
|
No source or target codepage
support at this time. Result in a DBCS environment may vary
from the Progress implementation.
|
AVAILABLE
|
RecordBuffer.available()
|
database
|
FUNC_LOGICAL |
no
|
|
|
yes
|
|
CAN-DO
|
character.matchList()
|
security
|
FUNC_LOGICAL |
yes
|
|
|
yes
|
|
CAN-FIND |
RandomAccessQuery.hasFirst()
RandomAccessQuery.hasNext()
RandomAccessQuery.hasPrevious()
RandomAccessQuery.hasLast()
RandomAccessQuery.hasAny()
|
database
|
FUNC_LOGICAL |
no
|
|
|
yes
|
|
CAN-QUERY
|
|
UI
|
FUNC_LOGICAL |
yes
|
|
|
no
|
|
CAN-SET
|
|
UI
|
FUNC_LOGICAL |
yes
|
|
|
no
|
|
CAPS
|
character.toUpperCase()
|
string
|
FUNC_CHAR
|
yes
|
no
|
partial |
yes
|
It can contain DBCS but only
the SBCS chars are uppercased. |
CHR
|
character.chr()
|
type conversion
|
FUNC_CHAR
|
yes
|
yes
|
yes
|
yes
|
No source or target codepage
support at this time. Result in a DBCS environment may vary
from the Progress implementation. |
CODEPAGE-CONVERT
|
|
type conversion
|
FUNC_CHAR
|
yes
|
yes
|
yes
|
no
|
|
COMPARE
|
character.compare()
|
string
|
FUNC_LOGICAL
|
yes
|
yes
|
yes
|
yes
|
No collation table support at
this time. Result in a DBCS environment may vary from the
Progress implementation. |
CONNECTED |
ConnectionManager.connected() |
database |
FUNC_LOGICAL |
no |
no |
|
yes |
|
COUNT-OF
|
P2JQuery.size()
|
database
|
FUNC_INT
|
no
|
no
|
|
yes
|
Implemented by concrete query
classes.
|
CURRENT-CHANGED
|
|
database
|
FUNC_LOGICAL
|
no
|
no
|
|
no
|
|
CURRENT-LANGUAGE
|
EnvironmentOps.getCurrentLanguage()
|
I18N
|
VAR_CHAR
|
no
|
no
|
|
yes
|
|
CURRENT-RESULT-ROW
|
AbstractQuery.currentRow()
|
database
|
FUNC_INT
|
yes
|
no
|
|
yes
|
Implemented by AbstractQuery
and overridden where necessary by concrete query implementations. |
CURRENT-VALUE |
|
database
|
FUNC_INT
|
yes
|
yes
|
|
no
|
|
DATASERVERS |
EnvironmentOps.getDataServerList()
|
database
|
VAR_CHAR
|
no
|
no
|
|
early
|
|
DATE
|
see constructors for date
class
|
date
|
FUNC_DATE
|
yes
|
no
|
no
|
yes
|
|
DATETIME |
see constructors for datetime class |
date/time |
FUNC_DATETIME |
yes |
no |
no |
yes |
|
DATETIME-TZ |
see constructors for datetimetz class |
date/time |
FUNC_DATETIME_TZ |
yes |
no |
no |
yes |
|
DAY |
date.day() which calls
instance method date.getDayNum()
|
date
|
FUNC_INT
|
yes
|
no
|
no
|
yes
|
|
DBCODEPAGE |
|
I18N
|
FUNC_CHAR
|
yes
|
yes
|
|
no
|
|
DBCOLLATION |
|
I18N
|
FUNC_CHAR
|
yes
|
yes
|
|
no
|
|
DBNAME
|
EnvironmentOps.getCurrentDatabaseName()
|
database
|
VAR_CHAR
|
yes
|
no
|
|
yes
|
|
DBPARAM |
|
database
|
FUNC_CHAR |
yes
|
yes
|
|
no
|
|
DBRESTRICTIONS |
ConnectionManager.dbRestrictions()
|
database
|
FUNC_CHAR |
yes
|
yes
|
|
yes
|
|
DBTASKID |
|
database
|
FUNC_INT
|
yes
|
yes
|
|
no
|
|
DBTYPE |
ConnectionManager.dbType() |
database
|
FUNC_CHAR
|
yes
|
yes
|
|
yes
|
|
DBVERSION |
|
database
|
FUNC_CHAR
|
yes
|
yes
|
|
no
|
|
DECIMAL
|
see constructors for decimal
class
|
type conversion
|
FUNC_DEC
|
yes
|
no
|
no
|
yes
|
|
DYNAMIC-FUNCTION |
|
function execution
|
FUNC_POLY |
yes
|
yes
|
|
no
|
|
ENCODE
|
SecurityOps.encode()
|
security
|
FUNC_CHAR
|
yes
|
no
|
|
early
|
The interface is completely
supported and functional, however the calculated result is NOT
compatible with the 4GL version.
|
ENTERED |
GenericWidget.isEntered()
|
UI
|
FUNC_LOGICAL
|
no
|
no
|
|
yes
|
|
ENTRY
|
character.entry()
|
string
|
FUNC_CHAR
|
yes
|
yes
|
yes
|
yes
|
Delimiter processing is
*always* case-sensitive at this time. |
ETIME |
date.elapsed()
|
time
|
FUNC_INT
|
no
|
yes
|
|
yes
|
|
EXP
|
MathOps.pow()
|
math
|
FUNC_DEC
|
yes
|
no
|
|
yes
|
|
EXTENT |
0 if the variable or field is
scalar
OR
array_var.length
|
arrays
|
FUNC_INT
|
no
|
no
|
|
yes
|
This is a direct conversion
without any backing method or class.
|
FILL
|
character.fill() |
string
|
FUNC_CHAR
|
yes
|
no
|
|
yes
|
|
FIRST |
PresortQuery.isFirst() |
loops/transactions |
FUNC_LOGICAL
|
no
|
no
|
|
yes
|
|
FIRST-OF |
PresortQuery.isFirstOfGroup()
|
loops/transactions |
FUNC_LOGICAL |
no
|
no
|
|
yes
|
|
FRAME-COL |
CommonFrame.frameCol()
|
UI
|
FUNC_DEC
|
no
|
yes
|
|
yes
|
|
FRAME-DB
|
|
UI
|
VAR_CHAR
|
no
|
no
|
|
no
|
|
FRAME-DOWN |
CommonFrame.frameDown() |
UI
|
FUNC_INT
|
no
|
yes
|
|
yes
|
|
FRAME-FIELD
|
LogicalTerminal.getFrameField()
|
UI
|
VAR_CHAR |
no
|
no
|
|
yes
|
|
FRAME-FILE
|
|
UI
|
VAR_CHAR |
no
|
no
|
|
no
|
|
FRAME-INDEX |
LogicalTerminal.getFrameIndex()
|
UI
|
VAR_INT
|
no
|
no
|
|
yes
|
|
FRAME-LINE |
CommonFrame.frameLine() |
UI
|
FUNC_INT |
no
|
yes
|
|
yes
|
|
FRAME-NAME
|
|
UI
|
VAR_CHAR
|
no
|
no
|
|
|
|
FRAME-ROW
|
CommonFrame.frameRow() |
UI
|
FUNC_DEC
|
no
|
yes
|
|
yes
|
|
FRAME-VALUE |
LogicalTerminal.getFrameValue()
LogicalTerminal.setFrameValue()
|
UI
|
VAR_CHAR |
no
|
no
|
|
yes
|
|
GATEWAYS
|
EnvironmentOps.getDataServerList()
|
database
|
FUNC_CHAR
|
no
|
no
|
|
early
|
|
GET-BITS
|
|
bit manipulation |
FUNC_INT
|
yes
|
no
|
|
no
|
|
GET-BYTE
|
BinaryData.getByte()
|
raw/memory access
|
FUNC_INT
|
yes
|
no
|
|
yes
|
|
GET-BYTE-ORDER
|
|
raw/memory access |
FUNC_INT
|
no
|
no
|
|
no
|
|
GET-BYTES
|
|
raw/memory access |
FUNC_POLY
|
yes
|
no
|
|
no
|
Returns RAW or MEMPTR.
|
GET-CODEPAGES
|
|
I18N
|
VAR_CHAR
|
no
|
no
|
|
no
|
|
GET-COLLATIONS
|
|
I18N
|
FUNC_CHAR
|
yes
|
no
|
|
no
|
|
GET-DOUBLE
|
|
raw/memory access |
FUNC_DEC
|
yes
|
no
|
|
no
|
|
GET-FLOAT
|
|
raw/memory access |
FUNC_DEC
|
yes
|
no
|
|
no
|
|
GET-LONG
|
|
raw/memory access |
FUNC_INT
|
yes
|
no
|
|
no
|
|
GET-POINTER-VALUE
|
|
raw/memory access |
FUNC_INT |
no
|
no
|
|
no
|
|
GET-SHORT
|
|
raw/memory access |
FUNC_INT
|
yes
|
no
|
|
no
|
|
GET-SIZE
|
memptr.length() |
raw/memory access |
FUNC_INT
|
no
|
no
|
|
yes
|
|
GET-STRING
|
BinaryData.getString() |
raw/memory access |
FUNC_CHAR
|
yes
|
yes
|
|
yes
|
|
GET-UNSIGNED-SHORT
|
|
raw/memory access |
FUNC_INT
|
yes
|
no
|
|
no
|
|
GO-PENDING |
LogicalTerminal.isGoPending()
|
UI
|
VAR_LOGICAL
|
no
|
no
|
|
yes
|
|
IF THEN ELSE
|
condition ? expr1 : expr2
|
ternary
|
FUNC_POLY
|
yes
|
no
|
|
yes
|
Returns same type as the THEN
and ELSE expressions evaluate to.
|
INDEX
|
character.indexOf()
|
string
|
FUNC_INT
|
yes
|
yes
|
|
yes
|
|
INPUT
|
Converts to a field-level
getter in a custom frame interface OR to the
CommonFrame.getScreenValue(widget) method if this is a character
type override case (see notes).
|
UI
|
FUNC_POLY
|
no
|
yes
|
|
yes
|
The INPUT built-in function
normally returns the same type as the single parameter (which must
be an lvalue which is found in a frame that is in scope).
However, this function can be used in an expression that requires a
character data type even in the case where the parameter is NOT of
the character type! This is like an override option and it
must be detected from the SURROUNDING expression (above the
FUNC_POLY node). For example,an direct assignment of the INPUT
function's return type to a character lvalue is honored. This
case works differently than an assignment to a non-character type in
the case that the value in the screen buffer is uninitialized (it
will return the empty string like screen-value instead of returning
the default value of the operand type). This override support
is implemented except for one case: the
usage of INPUT as a parameter to another built-in function (or
presumably a method) which expects a character parameter should also
cause this override. At this time there is no support for this case.
The resulting type (if not already character) will cause the wrong
type to be emitted. Note that this only is an issue for
built-in functions (and methods). If the parent is a user defined FUNC_* then the
result is the same as the operand, no override occurs. But a
built-in function that takes a character type will get the result as
a character instead of the operand type.
See the SCREEN-VALUE attribute which is a
related issue.
|
INTEGER
|
see constructors for integer
class
|
type conversion
|
FUNC_INT |
yes
|
no
|
|
yes
|
|
INTERVAL |
DateOps.interval() |
date/time |
FUNC_INT64 |
yes |
no |
no |
yes |
|
IS-ATTR-SPACE
|
|
UI
|
FUNC_LOGICAL |
yes
|
no
|
|
no
|
|
IS-LEAD-BYTE
|
|
I18N
|
FUNC_LOGICAL |
yes
|
no
|
|
no
|
|
KBLABEL
|
LogicalTerminal.kbLabel() |
UI
|
FUNC_CHAR
|
yes
|
no
|
|
yes
|
|
KEYCODE |
LogicalTerminal.keyCode() |
UI
|
FUNC_INT
|
yes
|
no
|
|
yes
|
|
KEYFUNCTION |
LogicalTerminal.keyFunction()
|
UI
|
FUNC_CHAR
|
yes
|
no
|
|
yes
|
|
KEYLABEL |
LogicalTerminal.keyLabel()
|
UI
|
FUNC_CHAR
|
yes
|
no
|
|
yes
|
|
KEYWORD
|
|
Progress
|
FUNC_CHAR |
yes
|
no
|
|
no
|
|
KEYWORD-ALL |
|
Progress |
FUNC_CHAR |
yes
|
no
|
|
no
|
|
LAST
|
PresortQuery.isLast() |
loops/transactions |
FUNC_LOGICAL |
no
|
no
|
|
yes
|
|
LAST-OF
|
PresortQuery.isLastOfGroup() |
loops/transactions |
FUNC_LOGICAL |
no
|
no
|
|
yes
|
|
LASTKEY
|
KeyReader.lastKey()
|
UI
|
VAR_INT
|
no
|
no
|
|
yes
|
|
LC
|
character.toLowerCase()
|
string
|
FUNC_CHAR
|
yes
|
no
|
partial
|
yes
|
It can contain DBCS but only
the SBCS chars are lowercased. |
LDBNAME
|
ConnectionManager.ldbName() |
database
|
FUNC_CHAR |
yes
|
yes
|
|
yes
|
|
LEFT-TRIM
|
character.leftTrim() |
string
|
FUNC_CHAR
|
yes
|
yes
|
yes
|
yes
|
|
LENGTH
|
character.length()
character.byteLength()
BinaryData.length()
|
raw/memory access
|
FUNC_INT
|
yes
|
yes
|
|
yes
|
No support for "column" based
length at this time.
|
LIBRARY
|
|
R-code library |
FUNC_CHAR
|
yes
|
no
|
|
no
|
|
LINE-COUNTER
|
Stream.getNextLineNum()
|
I/O
|
FUNC_INT
|
no
|
yes
|
|
yes
|
|
LIST-EVENTS
|
|
UI
|
FUNC_CHAR |
yes
|
yes
|
|
no
|
|
LIST-QUERY-ATTRS
|
|
UI
|
FUNC_CHAR |
yes
|
no
|
|
no
|
|
LIST-SET-ATTRS
|
|
UI
|
FUNC_CHAR |
yes
|
no
|
|
no
|
|
LIST-WIDGETS
|
|
UI
|
FUNC_CHAR |
yes
|
yes
|
|
no
|
|
LOCKED
|
RecordBuffer.wasLocked() |
database
|
FUNC_LOGICAL
|
no
|
no
|
|
yes
|
|
LOG
|
MathOps.log()
|
math
|
FUNC_DEC
|
yes
|
yes
|
|
yes
|
|
LOOKUP |
character.lookup()
|
string
|
FUNC_INT
|
yes
|
yes
|
yes
|
yes
|
Delimiter processing is
*always* case-sensitive at this time.
|
MAXIMUM
|
character.maximum()
date.maximum()
logical.maximum()
integer.maximum()
decimal.maximum() |
math
|
FUNC_POLY
|
yes
|
yes
|
|
yes
|
Warning: variable args!
Numeric operands are widened (int to double) if there are a
mixture. All arguments (other than numerics) can only be
compared against others of the same type.
|
MEMBER
|
|
R-code library
|
FUNC_CHAR
|
yes
|
no
|
|
|
|
MESSAGE-LINES
|
LogicalTerminal.getMessageLines()
|
UI
|
VAR_INT
|
no
|
no
|
|
yes
|
|
MINIMUM
|
character.minimum()
date.minimum()
logical.minimum()
integer.minimum()
decimal.minimum() |
math
|
FUNC_POLY |
yes
|
yes
|
|
yes
|
Warning: variable args!
Numeric operands are widened (int to double) if there are a
mixture. All arguments (other than numerics) can only be
compared against others of the same type. |
MONTH |
date.month() which calls
instance method date.getMonthNum()
|
date
|
FUNC_INT
|
yes
|
no
|
|
yes
|
|
MTIME |
datetime.millisecondsSinceMidnight() |
date/time |
FUNC_INT |
yes |
yes |
|
yes |
|
NEW
|
RecordBuffer.isNew() |
database
|
FUNC_LOGICAL
|
no
|
no
|
|
yes
|
|
NEXT-VALUE |
|
database
|
FUNC_INT
|
yes
|
yes
|
|
no
|
|
NOT ENTERED
|
GenericWidget.isNotEntered()
|
UI
|
FUNC_LOGICAL |
no
|
yes
|
|
yes
|
|
NOW |
datetime.now()
datetimetz.now() |
date/time |
FUNC_DATETIME FUNC_DATETIME_TZ |
yes |
yes |
|
yes |
|
NUM-ALIASES
|
ConnectionManager.numAliases()
|
database |
VAR_INT
|
no
|
no
|
|
yes
|
|
NUM-DBS
|
ConnectionManager.numDBs() |
database |
VAR_INT
|
no
|
no
|
|
yes
|
|
NUM-ENTRIES
|
character.numEntries()
character.numEntriesOf()
|
string
|
FUNC_INT
|
yes
|
yes
|
yes
|
yes
|
Delimiter processing is
*always* case-sensitive at this time. |
NUM-RESULTS |
P2JQuery.size()
|
database |
FUNC_INT
|
yes
|
no
|
|
yes
|
Implemented by
concrete query classes. |
OPSYS
|
EnvironmentOps.getOperatingSystem()
|
OS environment |
VAR_CHAR
|
no
|
no
|
|
yes
|
|
OS-DRIVES
|
FileSystemOps.getRootList()
|
OS environment |
VAR_CHAR
|
no
|
no
|
|
yes
|
|
OS-ERROR
|
FileSystemOps.getLastError() |
OS environment |
VAR_INT
|
no
|
no
|
|
yes
|
|
OS-GETENV |
FileSystemOps.getProperty() |
OS environment
|
FUNC_CHAR
|
yes
|
no
|
|
yes
|
|
PAGE-NUMBER |
Stream.getPageNum()
|
I/O
|
FUNC_INT |
no
|
yes
|
|
yes
|
|
PAGE-SIZE |
Stream.getPageSize()
|
I/O
|
FUNC_INT
|
no
|
yes
|
|
yes
|
|
PDBNAME |
ConnectionManager.pdbName() |
database
|
FUNC_CHAR
|
yes
|
yes
|
|
yes
|
|
PROC-HANDLE
|
|
Progress environment |
VAR_INT
|
no
|
no
|
|
no
|
Is this actually polymorphic?
|
PROC-STATUS
|
|
Progress environment |
VAR_INT
|
no
|
no
|
|
no
|
|
PROGRAM-NAME |
EnvironmentOps.getSourceName()
+ a constant (public static final String progressSourceName) in each
generated class.
|
Progress environment |
FUNC_CHAR
|
yes
|
no
|
|
yes
|
Warning: there are 2 flaws in
this implementation.
- Progress returns the same filename as is used on the
RUN statement (so the name can change from invocation to
invocation). Our version has a static string for the base
name of the file (no pathing).
- It is possible that due to refactoring, code that calls
program-name might have dependencies on a source file name that
is not the source file name of the current class. Such cases
might require a class and method specific mapping to a source
file name (some kind of registry database).
|
PROGRESS
|
EnvironmentOps.getRuntimeType()
|
Progress environment |
VAR_CHAR
|
no
|
no
|
|
yes
|
|
PROMSGS |
EnvironmentOps.getMessageSource()
|
Progress environment |
VAR_CHAR |
no
|
no
|
|
yes
|
|
PROPATH |
EnvironmentOps.getSourcePath()
|
Progress environment |
VAR_CHAR |
no
|
no
|
|
yes
|
|
PROVERSION |
EnvironmentOps.getVersion()
|
Progress environment |
VAR_CHAR |
no
|
no
|
|
yes
|
|
QUERY-OFF-END
|
P2JQuery.isOffEnd() |
database
|
FUNC_LOGICAL
|
yes
|
no
|
|
yes
|
Implemented by concrete
query classes.
|
R-INDEX
|
character.lastIndexOf()
|
string
|
FUNC_INT
|
yes
|
yes
|
|
yes
|
|
RANDOM
|
MathOps.random()
|
math
|
FUNC_INT
|
yes
|
no
|
|
yes
|
Warning: it is possible to
force Progress to always generate the same sequence of random
numbers in every session! (This sounds like an awful idea but
perhaps it is being used.)
|
RAW |
|
database
|
FUNC_RAW
|
no
|
yes
|
|
no
|
|
RECID
|
RecordBuffer.recordID |
database |
FUNC_RECID
|
yes
|
no
|
|
yes
|
The type is mapped to integer.
|
RECORD-LENGTH
|
|
database
|
FUNC_INT
|
no
|
no
|
|
no
|
|
REPLACE
|
character.replaceAll()
|
string
|
FUNC_CHAR
|
yes
|
no
|
|
yes
|
|
RETRY
|
TransactionManager.isRetry()
|
loops/transactions
|
VAR_LOGICAL
|
no
|
no
|
|
yes
|
|
RETURN-VALUE
|
ControlFlowOps.getReturnValue()
|
procedure execution
|
VAR_CHAR
|
no
|
no
|
|
yes
|
|
RGB-VALUE
|
|
UI
|
FUNC_INT
|
yes
|
no
|
|
no
|
|
RIGHT-TRIM
|
character.rightTrim()
|
string
|
FUNC_CHAR
|
yes
|
yes
|
|
yes
|
|
ROUND
|
Math.round()
|
math
|
FUNC_DEC
|
yes
|
no
|
|
yes
|
|
ROWID
|
RecordBuffer.recordID |
database |
FUNC_ROWID
|
yes
|
no
|
|
yes
|
The type is mapped to integer. |
SCREEN-LINES
|
LogicalTerminal.getScreenLines()
|
UI
|
VAR_INT
|
no
|
no
|
|
yes
|
|
SDBNAME
|
ConnectionManager.sdbName()
|
database |
FUNC_CHAR
|
yes
|
yes
|
|
yes
|
|
SEARCH
|
FileSystemOps.searchPath()
|
filesystem
|
FUNC_CHAR |
yes
|
no
|
|
yes
|
|
SEEK
|
Stream.getPosition()
|
filesystem
|
FUNC_INT
|
yes
|
yes
|
|
yes
|
|
SETUSERID
|
|
security
|
FUNC_LOGICAL
|
yes
|
yes
|
|
no
|
|
SQRT
|
MathOps.sqrt()
|
math
|
FUNC_DEC
|
yes
|
no
|
|
yes
|
|
STRING
|
character.valueOf() which uses
a wrapper-specific toString()
|
type conversion
|
FUNC_CHAR
|
yes
|
yes
|
|
yes
|
|
SUBSTITUTE
|
character.substitute()
|
string
|
FUNC_CHAR |
yes
|
yes
|
yes
|
yes
|
The variable length argument
list is handled by an argument of type Object[].
|
SUBSTRING
|
character.substring()
|
string
|
FUNC_CHAR
|
yes
|
yes
|
yes
|
yes
|
No support for "raw", "fixed"
or "column" based substrings at this time. |
SUPER |
|
function execution
|
FUNC_POLY
|
yes
|
yes
|
|
no
|
|
TERMINAL
|
LogicalTerminal.getTerminal()
|
UI
|
VAR_CHAR
|
no
|
no
|
|
yes
|
|
TIME
|
date.secondsSinceMidnight()
|
time
|
VAR_INT
|
no
|
no
|
|
yes
|
|
TIMEZONE |
date.getDefaultTimeZoneOffset()
date.getOffsetForSpec()
datetimetz.getTimeZoneOffset() |
date/time |
FUNC_INT |
yes |
yes |
|
yes |
|
TODAY
|
date.today()
|
date
|
VAR_DATE
|
no
|
no
|
|
yes
|
|
TO-ROWID
|
|
database |
FUNC_ROWID
|
yes
|
no
|
|
no
|
|
TRANSACTION
|
TransactionManager.isTransactionActive()
|
loops/transactions
|
VAR_LOGICAL
|
no
|
no
|
|
yes
|
|
TRIM
|
character.trim()
|
string
|
FUNC_CHAR
|
yes
|
yes
|
|
yes
|
|
TRUNCATE
|
MathOps.truncate() |
math
|
FUNC_DEC
|
yes
|
no
|
|
yes
|
|
USERID
|
SecurityOps.getUserId() |
security
|
FUNC_CHAR
|
no
|
yes
|
|
yes
|
|
VALID-EVENT
|
|
UI
|
FUNC_LOGICAL
|
yes
|
yes
|
|
no
|
|
VALID-HANDLE
|
handle.isValid()
|
data types
|
FUNC_LOGICAL |
no
|
no
|
|
yes
|
|
WEEKDAY
|
date.weekday() which calls
instance method date.getWeekDayNum()
|
date
|
FUNC_INT
|
yes
|
no
|
|
yes
|
|
WIDGET-HANDLE
|
handle.fromString()
|
UI
|
FUNC_HANDLE
|
yes
|
no
|
|
yes
|
|
YEAR |
date.year() which calls
instance method date.getYearNum() |
date
|
FUNC_INT
|
yes
|
no
|
|
yes
|
|
See also:
rules/convert/builtin_functions.rules
Methods and Attributes
Progress 4GL provides an object-like construct with its HANDLE data type.
WIDGETs, QUERYs and other resources can be represented by a handle. Using
this handle one can access pre-defined attributes or call methods (invoke
behavior). These attributes and methods are specific to this resource
type, they are built into the environment and cannot be extended by the
user/programmer.
There are 3 possible forms of conversion:
- Special purpose accessors (e.g. SYSTEM HANDLES such as
CURRENT-WINDOW) can be naturally converted to a static method call which
accesses an object instance that is the current value. The ATTRIBUTE
or METHOD node itself can be directly translated to an instance method call
made upon the returned object reference.
- Any singleton resource (e.g. SYSTEM HANDLES such as ERROR-STATUS)
can be easily and naturally converted to a static method call. The
ATTRIBUTE or METHOD node itself can be directly translated to the static
method call. Note that for attributes, the getter methods will need to
be backed by a context-local member.
- Instance resources (e.g. a WIDGET handle) act like instance members
(accessed via a getter method call) or methods. In this case the COLON
patent node should be the peer node at which the method is converted.
Then the HANDLE will need to emit as an object REFERENCE of the proper
class.
The following summarizes the method and attribute support that is implemented:
Progress
Feature
|
Java
Replacement
|
Type
|
Method
or Attribute
|
Supported
|
Notes
|
DCOLOR
|
GenericWidget.getDcolor()
GenericWidget.setDcolor()
|
widget
|
ATTRIBUTE |
Yes
|
|
DESELECT-ROWS
|
GenericWidget.deselectRows()
|
browse
|
METHOD |
Yes
|
|
ENTRY
|
GenericWidget.entry() |
combo-box, selection list
|
METHOD
|
Yes
|
|
ERROR
|
ErrorManager.isError()
|
ERROR-STATUS system handle
|
ATTRIBUTE |
Yes
|
|
FETCH-SELECTED-ROW
|
GenericWidget.fetchSelectedRow() |
browse
|
METHOD |
Yes
|
|
FILE-NAME |
FileSystemOps.initFileInfo()
to set
FileSystemOps.fileInfoGetName() to get |
FILE-INFO system handle |
ATTRIBUTE |
Yes |
|
FILE-SIZE |
FileSystemOps.fileInfoGetSize()
to get |
FILE-INFO system handle |
ATTRIBUTE |
Yes |
|
GET-NUMBER
|
Manager.getErrorNum() |
ERROR-STATUS system handle |
METHOD
|
Yes
|
|
GET-MESSAGE
|
ErrorManager.getErrorText() |
ERROR-STATUS system handle |
METHOD |
Yes
|
|
HANDLE
|
GenericWidget.getHandle()
|
widget
|
ATTRIBUTE |
Yes |
Should this really be
converted into a widget object reference?
|
IS-SELECTED
|
GenericWidget.isSelected() |
combo-box, selection list |
METHOD |
Yes |
|
LIST-ITEM-PAIRS
|
GenericWidget.getListItems()
GenericWidget.setListItems() |
combo-box, selection list |
ATTRIBUTE |
Yes |
|
LOOKUP
|
GenericWidget.lookup() |
combo-box, selection list |
METHOD |
Yes |
|
NEXT-TAB-ITEM
|
GenericWidget.getNextTabItem()
GenericWidget.setNextTabItem() |
widget
|
ATTRIBUTE |
Yes |
|
NUM-MESSAGES
|
ErrorManager.numErrors() |
ERROR-STATUS system handle |
ATTRIBUTE
|
Yes
|
|
NUM-SELECTED-ROWS
|
GenericWidget.getNumSelectedRows()
GenericWidget.setNumSelectedRows() |
browse |
ATTRIBUTE |
Yes |
|
PARENT
|
GenericWidget.getParent()
GenericWidget.setParent() |
frame
|
ATTRIBUTE |
Yes |
|
PERSISTENT
|
false
|
THIS-PROCEDURE system handle
|
ATTRIBUTE
|
Yes
|
Temporary solution to satisfy
the current application project for which we always know the result
will be false. A real implementation will eventually need to
be made.
|
PFCOLOR
|
GenericWidget.getPfColor()
GenericWidget.setPfColor() |
widget
|
ATTRIBUTE |
Yes |
|
PRIVATE-DATA
|
GenericWidget.getPrivateData()
GenericWidget.setPrivateData() |
frame
|
ATTRIBUTE |
Yes |
|
READ-ONLY
|
GenericWidget.isReadOnly()
GenericWidget.setReadOnly() |
widget
|
ATTRIBUTE
|
Yes
|
|
REFRESH
|
GenericWidget.refresh()
|
browse
|
METHOD |
Yes |
|
SCROLLABLE
|
GenericWidget.isScrollable()
GenericWidget.setScrollable() |
widget
|
ATTRIBUTE |
Yes |
|
SCREEN-VALUE
|
CommonFrame.getScreenValue(widget)
or widget setter method from the current frame interface.
|
widget
|
ATTRIBUTE
|
Yes
|
This duality is needed since
the screen-value attribute will always return a character type (the
frame interface getters have the same type as the data being
represented/edited). In the case where the "natural" data type
is not character, this will behave differently when the screen
buffer's version of this data is uninitialized. In particular,
the data will be returned as the empty string rather than as the
default type for the natural data type. So if widget i is an
integer and it has never been initialized (copied into the screen
buffer as a result of a display a screen-value setter call or via UI
editing) then it will return "" instead of 0.
See the INPUT built-in function which
has a similar issue.
|
SELECT-ALL
|
GenericWidget.selectAll()
|
browse
|
METHOD |
Yes |
|
SELECT-FOCUSED-ROW
|
GenericWidget.selectFocusedRow()
|
browse
|
METHOD |
Yes |
|
SELECTED
|
GenericWidget.isSelected()
GenericWidget.setSelected() |
widget
|
ATTRIBUTE |
Yes |
|
SENSITIVE
|
GenericWidget.isSensitive()
GenericWidget.setSensitive() |
widget
|
ATTRIBUTE |
Yes |
|
SET-REPOSITIONED-ROW
|
GenericWidget.setRepositionedRow()
|
browse
|
METHOD |
Yes |
|
TITLE
|
GenericWidget.getTitle()
GenericWidget.setTitle() |
frame
|
ATTRIBUTE |
Yes |
|
VISIBLE
|
GenericWidget.isVisible()
GenericWidget.setVisible() |
widget
|
ATTRIBUTE |
Yes |
|
Instance oriented system handles:
Handle
|
Java
Equivalent
|
CURRENT-WINDOW
|
LogicalTerminal.currentWindow() |
ACTIVE-WINDOW
|
LogicalTerminal.activeWindow() |
THIS-PROCEDURE
|
none at this time |
FOCUS
|
LogicalTerminal.focus()
|
SESSION
|
none at this time
|
SELF |
LogicalTerminal.self() |
See also:
rules/convert/methods_attributes.rules
rules/convert/assignments.rules (setters are implemented here)
rules/convert/variable_references.rules (for handle and system handle
conversion)
Control Flow
Some implementation notes:
- IF/THEN/ELSE is completely supported as Java if/else.
- The IF function converts to the Java ternary operator.
- CASE statements with integral data types properly convert to the
Java switch when using NUM_LITERALs for all WHEN clauses.
- CASE statements that have non-constant integrals (expressions
rather than literals) and/or non-integer (character, date, decimal, logical)
expressions properly convert to an if/else if/else if/else form.
- A LEAVE in a CASE statement breaks out of the closest enclosing
non-DO block. This is handled by using an existing label or
manufacturing a new label (if one doesn't already exist) to identify the
correct block. Such changes are handled in the annotations processing,
and the manufactured label and new label reference should be written into
the source AST. This means that an implicit behavior is made explicit
and no changes to conversion code are needed.
- Simple DO blocks convert to the Java { }.
- Simple REPEAT blocks convert to the Java while (true) { }.
- DO/REPEAT/FOR with a WHILE expression converts to a Java while
(expression) { }.
- DO/REPEAT/FOR with a var = expr1 TO expr2 converts to a Java for
(var = expr1 ; var comparison expr2 ; var += increment) { }.
- Labels are supported (they are attached to the beginning of a
block, exactly as in Java).
- LEAVE
- An unlabeled LEAVE outside of an iterating block acts as a
RETURN statement.
- Within an iterating block (even if inside a nested non-looping
block), the LEAVE and LEAVE label are translated directly to break or
break label respectively.
- NEXT and NEXT label are translated directly to continue or continue
label, except in the case where it is used in a non-iterating block in which
case it is translated into a continue on the nearest enclosing iterating
block. If there is no such enclosing iterating block, it is treated as
a LEAVE.
- STOP is implemented as a "throw new StopConditionException()".
- QUIT is implemented as "throw new QuitConditionException()".
- PAUSE is implemented as ControlFlow.pause(...).
- RETURN when used in a function, returns the result of the
associated expression.
- RETURN when used in a procedure, if it has an expression, saves the
evaluated result into a context-local runtime variable via the
ControlFlowOps.setReturnValue() before emitting a Java return
statement. If there is no expression, then the setReturnValue() is
assigned the empty string. Whenever Progress code references
RETURN-VALUE, this results in a call to ControlFlowOps.getReturnValue()
which returns the current context-local value of this character variable.
- RETURN ERROR is implemented as a special ignore flag in the TransactionManager and "throw
new ErrorConditionException()" where this exception is ignored until it
reaches the caller of the top level block in the current method.
- RETURN NO-APPLY is implemented by calling LogicalTerminal.consumeEvent()
before returning.
- RUN for internal procedures is implemented as a method call to the
referenced instance member in the same class. Any specified parameters
are emitted as method arguments.
- RUN for external procedures is implemented as a method call to the
execute() method of the Java class name generated from the file portion of
the pathname text in the FILENAME child node. First the class name and
a unique Java variable name are generated. Then an instance of the
class is defined and initialized (using a default constructor).
Finally, the method call to the execute() method is emitted in such a manner
as to ensure that any parameters are emitted as children of the method call
node.
- RUN VALUE() is a variation on the procedure execution. Both
external and internal procedures can be called via this mechanism. The
difference is that the procedure name is built dynamically (at runtime).
- The main problem with these statements is that resulting value
of the variable passed to RUN value() statement contains the
Progress-specific name of the internal or external procedure. In other
words, runtime code must provide a way to translate Progress procedure
names into Java class and methods names.
- The following approach is used:
- The value of the passed variable is translated into a Java
class name using data in the directory which was generated at
conversion time.
- If such a Java class name is present then Java class is
instantiated and execute() method is called.
- If no such Java class name is present, then an internal
procedure name is assumed and the Java method name is looked up
using the directory information.
- The instance method is called by ControlFlowOps.invoke()
method via reflection on the instance associated with the caller as
found from the current stack.
- At this time, the implementation of
the name mapping uses an XML data file which contains mapping between
Progress file names/internal procedure names and Java class/method
names. This information is collected during conversion and used by
run-time code. The final implementation will use the directory.
See also:
rules/annotations/annotations.xml
rules/annotations/block_properties.rules
rules/convert/control_flow.rules
Common Language Statements
Assignment Type Language Statements
The following language statements are handled with special processing:
- length (BinaryData.length)
- overlay (character.overlay)
- put-byte (BinaryData.setByte)
- put-string (BinaryData.setString)
- set-size (memptr.length)
- substring (character.replace)
All of these are "assignment type" language statements. That is, language
statements that are structured like an assignment of an expression result to a
function call. This required that the annotations processing rewrite the
source tree to look more like a proper method call with the first parameter
being the object reference. For example, the assigned expression is
inserted as the 2nd child of the KW_SUBSTR or KW_OVERLAY and the ASSIGN node is
removed.
Once rewritten, the convert/language_statements.rules rule set handles a simple
mapping of overlay to the proper instance method mapping.The result is designed
as an instance method because the result must be assigned back to the value of
the variable specified as the 1st child. This matches the method call semantic
exactly.
Process Execution
External command execution is implemented in
com.goldencode.p2j.util.ProcessOps.java. The signature used is as follows:
void launch(String[] cmdlist,
boolean silent, boolean wait)
This handles launching a child process in response to these language statements:
- BTOS
- DOS
- OS-COMMAND
- OS2
- UNIX
- VMS
- INPUT_THRU
- OUTPUT_THRU
- INPUT_OUTPUT_THRU
The COMMAND_TOKENS child is converted into an anonymous string array initializer
and all children are converted into string literals or a string expression (the
value() construct). In this manner, the first parameter of the method call
launch() simultaneously specifies the program to be executed and provides all
command line arguments.
The 2nd and 3rd parameters are two flags which are generated based on optional
(and mutually exclusive) keywords KW_SILENT and KW_NO_WAIT. For this
reason, in Progress there are 3 possible modes which can be encountered:
Options
|
silent
flag
|
wait flag
|
none (the default)
|
false
|
true
|
KW_SILENT
|
true
|
true
|
KW_NO_WAIT
|
false
|
false
|
The "silent and no wait" mode doesn't exist in Progress. All of these
modes are properly supported using these flags. In addition, the
FileSystemOps.lastError (OS-ERROR) value is forced to 0 which is how Progress
handles this value.
In Progress, one can launch an interactive child process with the NO-WAIT
option. However, this seems to invariably disable the terminal and neither
the child process nor the Progress session can properly run. For this
reason, this mode is not supported nor is it expected to be in use in the field.
Command Text Parsing
The lexer and parser have been modified to change how the COMMAND_TOKENS
children are created. In particular, the lexer may deliver tokens broken
into a form that does not match the whitespace delimited words that the author
might have intended. For example: the "word" (used in the shell to
redirect STDERR to the same handle as STDOUT):
&2>1
would be lexed as 4 tokens:
UNKNOWN_TOKEN
NUM_LITERAL
GT
NUM_LITERAL
The lexer has been changed to return "hidden" tokens that represent
whitespace. During COMMAND_TOKENS processing, the parser checks these
hidden tokens and marks the COMMAND_TOKENS children if they need to be merged
back together or not. Annotation processing then handles this merge as
needed. The result is the same set of commands as executed in Progress
4GL.
See Also
Please see the following rulesets for the implementation:
convert/process_launch.rules
convert/expressions.rules (this handles the VALUE() construct)
convert/input_output.rules
These classes back the runtime functions:
com.goldencode.p2j.util.Launcher.java (remote interface definition)
com.goldencode.p2j.util.ProcessOps.java (server side)
com.goldencode.p2j.util.ProcessDaemon.java (real process launcher)
com.goldencode.p2j.util.ProcessStream.java
The majority of the processing of INPUT_THRU, OUTPUT_THRU and INPUT_OUTPUT_THRU
is implemented in
Streams and I/O
however, the process launching and stream connection is handled by the classes
and rulesets noted above.
Limitations
- Remote I/O and Terminal Integration
- The option KW_NO_CONS (no-console) is not supported yet (it may be
Windows-only).
- For some or all of these language statements, the process launched
is really a shell which then executes the command line provided. At
this time, the shell is hard coded to "sh" but needs to be moved into the
directory or provided as a conversion time hint that is passed to a modified
version of launch.
OS-Independent Commands (File System)
The OS independent commands for manipulating the file system are implemented in
com.goldencode.p2j.util.FileSystemOps.java.
The following language statements are supported:
- OS-APPEND
- OS-COPY
- OS-CREATE-DIR
- OS-DELETE
- OS-RENAME
Please see the following rulesets for the implementation:
convert/language_statements.rules
convert/literals.rules (handles FILENAME)
convert/expressions.rules (this handles the VALUE() construct)
Streams and I/O
The P2J project provides significant support for the Progress I/O processing and
streams features.
Summary
The following language statements which allow the creation, opening and closing
of streams are supported:
- DEFINE STREAM
- INPUT FROM
- OUTPUT TO
- INPUT THRU (INPUT THROUGH)
- OUTPUT THRU (OUTPUT THROUGH)
- INPUT-OUTPUT THRU (INPUT-OUTPUT THROUGH)
- INPUT CLOSE
- OUTPUT CLOSE
- INPUT-OUTPUT-CLOSE
The following language statements and functions which provide stream
reading/writing (including formatted reads/writes) are supported:
- READKEY
- IMPORT
- PUT
- EXPORT
- SEEK (statement)
- SEEK() function
- PAGE
- PAGE-SIZE function
- PAGE-NUM function
- LINE-COUNTER function
Stream Types
Stream Type
|
Java Class
|
Input
|
Output
|
Notes
|
File or Device
|
com.goldencode.p2j.util.FileStream
|
Y
|
Y
|
|
Printer
|
com.goldencode.p2j.util.PrinterStream |
N
|
Y
|
On Unix, this will probably be implemented as
a pipe to a child process of "lp". As this type may only be
needed on Windows, it isn't implemented at this time.
|
Directory
|
com.goldencode.p2j.util.DirectoryStream
(not yet implemented)
|
Y
|
N
|
Used for reading directory listings. Not
created at this time since it is not used in the application.
|
Terminal
|
?
|
Y
|
Y
|
How this is to be implemented is TBD.
|
Clipboard
|
?
|
?
|
Y
|
Windows-only, not implemented at this
time. Note that via the CLIPBOARD system handle, reading the
clipboard may be possible, though this has not been investigated.
|
Child Process
|
com.goldencode.p2j.util.PipeStream |
Y
|
Y
|
In Progress the same child process stream can
be created as both input and output (simultaneously), however the
Java class will only handle either input OR output. Thus the
converted code will have 2 streams created in this case and all
operations will need to be properly destined for the correct stream.
|
Basic Design
A DEFINE STREAM corresponds to a local variable of type
com.goldencode.p2j.util.Stream. This class implements an API that provides
the complete Progress semantics for all forms of supported I/O processing.
This is an abstract class and concrete subclasses actually provide the real
implementation for each type of stream. Note that the design is such that
the vast majority of code is implemented in Stream and the subclasses only have
to provide the minimum function to implement the specific resource being
used. This lets the same behavior be obtained from all types of streams
and makes it easy to add more stream types with a small incremental effort.
Generally a stream is implemented as either an input OR an output but not as
both. The sole exception to this is the case of using a named stream in
the INPUT-OUTPUT THROUGH, where the same stream name is used for both the input
and output pipes (the complexity of this is managed inside
com.goldencode.p2j.util.ProcessStream and in the rule-sets that convert the
code..
There are 2 unnamed streams ("unnamed input" and "unnamed output") which are
implicitly used when a named stream is not explicitly present in the source
file. These streams are stored as context local instances that are
accessible throughout a user's session via the UnnamedStreams.input() and
UnnamedStreams.output() methods. Language statements that operate
differently based on whether the unnamed input or output is redirected use the
UnnamedStreams
class to detect these cases and access the stream(s).
See below
.
For each named stream that is defined (and for the 2 unnamed streams), an
instance of a
RemoteStream
is instantiated using a static method in
StreamFactory
. This reference is named with a converted Java name or is accessed
"anonymously" via UnnamedStreams. Each RemoteStream will actually
reference a remote concrete Stream sub-class such as FileStream or ProcessStream
which
operates on the client side of the system
.
A concrete Stream subclass is instantiated and assigned to the named or unnamed
stream instances whenever an INPUT FROM, OUTPUT TO, INPUT-THRU, OUTPUT-THRU or
INPUT-OUTPUT THRU language statement is encountered. The type of requested
resource determines which class is instantiated. This constructor opens
the backing resource on the client. At this point the stream (RemoteStream
actually) reference is valid for use.
When a file is opened for output, any existing version of that file is removed
and a 0 length file is created (unless APPEND is specified). That file is closed
when the next stream is opened or by any explicit INPUT CLOSE, OUTPUT CLOSE or
INPUT-OUTPUT CLOSE.
When a file is opened for input, the read pointer is located at the first byte
of the file.
The STDIO pipes from a child process are fully supported. The common
process launching
infrastructure is used to start the child process but a special ProcessStream
class is used to implement the proper user-driven reading/writing support for
these pipes. All reads are done from a "combined" STDOUT and STDERR, where
output will first be read from STDOUT and then from STDERR. Due to the
"combining" semantic of Progress, a polling method is used to perform I/O
without blocking on one of the pipes (thus ignoring input from the other
pipe). In addition, an extra thread is used to wait for the termination of
the child process. This allows these polling loops to exit when the pipes
are empty and the child process has exited, which eliminates infinite read
blocking.
All of the formatted and unformatted reading and writing is provided by a common
set of methods in Stream:
- readChar
- readField
- readLine
- readBlock
- putField
- putSpace
- putLine
- writeField
- writeBlock
These handle the Progress semantics while actually reading/writing using a
smaller set of very simple workers that are implemented in the subclasses.
The Stream class also implements the buffering of reads/writes, and all the line
number, paging and column support.
Stream processing generates the
EndConditionException
on EOF (ENDKEY) and
ErrorConditionException
for ERROR condition generation. This processing honors the NO-ERROR
construct using the silent error mode in the
ErrorManager
class.
Implicit Close Behavior
Streams are automatically closed streams on exit from the scope in which they
are defined, except for global streams which are closed on end of the session.
The
TransactionManager
registerFinalizable() or registerTopLevelFinalizable() is used to obtain this
support. Global streams are registered with the global flag set
true
. This registraton code is emitted into the client application, just after
the reference is initialized. Shared streams that are not "NEW" are just
imported. For this reason, they have already been registered for implcit
close support in a previous scope. This table summarizes the states in
which registration occurs:
DEFINE
STREAM Type
|
Registration
|
NEW SHARED
|
TransactionManager.registerTopLevelFinalizable(this,
true)
|
NEW GLOBAL SHARED |
TransactionManager.registerFinalizable(this,
true) |
SHARED
|
n/a
|
(not shared at all)
|
TransactionManager.registerTopLevelFinalizable(this,
true) |
At each top level scope (trigger, external or internal procedure), the current
unnamed input and output streams are "remembered" as the defaults for that
scope. Any close that occurs for the unnamed input or output streams which
is not the result of assigning a new stream, causes the unnamed stream to be
restored back to the default for that scope. Since the Progress programmer
can change the unnamed streams but not query them, this feature was needed
otherwise an unnamed stream redirected in a calling scope could be modified
downstream and upon return that stream might have been already closed.
This would lead to bizarre behavior.
Shared Streams
All NEW shared and NEW GLOBAL shared streams are supported using the
SharedVariableManager
. This class has stream-specific methods to add and lookup streams.
Please see
Shared Variables
for more details.
References
rules/convert/base_structure.rules
rules/convert/input_output.rules
rules/convert/process_launch.rules
com.goldencode.p2j.util.Stream.java
com.goldencode.p2j.util.StreamWrapper.java
com.goldencode.p2j.util.FileStream.java
com.goldencode.p2j.util.ProcessStream.java
com.goldencode.p2j.util.ProcessOps.java
Limitations
- Although documented in Progress references, the default code-page
conversion (stream to internal and vice versa) does not appear to actually
be implemented OR perhaps this is locale specific. No default codepage
conversion is implemented in P2J.
- All of the explicit codepage conversion processing that is possible
in Progress is missing.
- The effect of the BINARY option is only accounted for in READKEY
processing and in this case it only modifies how newlines are
returned. If BINARY modifies other processing, it has not been
identified and is thus unimplemented at this time.
- Progress references document that the null character terminates
strings. This is not implemented.
- DBCS support is not provided.
- UNBUFFERED mode is not supported (and is probably not needed).
- MAP/NO-MAP support does not exist.
Remote
I/O and Terminal Integration
Introduction
The Progress 4GL is implemented as a client-centric environment. Even the
Progress database server is really just a (shared memory or network) connection
to a process which takes a limited specification of index fields and the
corresponding values or value ranges and returns records from that index.
All query where clause and sorting is handled on the client side!
In a P2J environment, the "client" is the "thin client". This is the Java
process written to connect to a P2J server and provide a remote (from the
server) user interface. It is run in the context of the user's operating
system login or shell. In other words, the user will be logged in to some
operating system (e.g. Linux) and will obtain a shell or desktop. If this
is a terminal oriented system (CHUI) then that user will have logged in via a
terminal session (via a serial line or a network protocol such as telnet or
ssh). After the login, the user will have some shell or desktop process
running on that system. From there the user either explicitly (with a
command or icon) or implicitly (via a profile or startup script) will launch the
P2J thin client Java process. When this document references the "thin
client" or "client", this is what is meant.
Stream and
File System Remoting
When a Progress 4GL program reads/writes files, interacts with the file system
or launches child processes, it is intrinsicly doing so on the client
system. This may not be evident in a system where the client and the P2J
server are the same physical box. But since all such activities in
Progress 4GL are done in the context of a child process (e.g. on Unix, the
executable "_progres") which is running with the login context of a specific
user, the home directory and file system access rights of that specific
user. In a P2J server environment, the entire server is running in a
process which is a daemon and while it does have an operating system security
context, this is most certainly not the same context as a given user. This
means that every process launch *may* be dependent upon context that is only
available on the client system. The child process may be dependent upon
access to the user's home directory or to data that is in a specific path that
only exists on the client system. Or perhaps the process itself only
exists on the client system and/or has specific environment variables or other
state/resources that are only available there. For example, many Progress
programs launch shell scripts or utilities that depend upon the client's context
for current working directory and other environment variables (e.g. the Unix
mail command). For this reason, the end-points of all I/O operations
(file, file system and process launching) must be executed on the client
system. In other words, they must be "remoted" to the client. The
business logic will still reside on the server. This includes all logic
that accesses and manipulates data, which defines the flow of control for the
application. However, when a read or write or file system search or
process launch occurs, these operations must be remoted to the client but made
to appear as if they were happening on the server. This allows the logic
to remain intact and the minimal amount of processing is pushed to the client.
The Stream class is abstract. It defines the following abstract methods:
- available
- getPos
- setPos
- getLen
- setLen
- writeByte
- writeCh
- write
- readCh
- readLn
- closeIn
- closeOut
- close
The backing file (FileStream) and process streams (ProcessStream) are Stream
subclasses that implement concrete versions of these methods. The
RemoteStream class is used on the server and it redirects all calls to the
abstract methods to instances of FileStream or ProcessStream. This is
handled using the LowLevelStream interface and a StreamDaemon class on the
client side. Each RemoteStream instance represents a single stream on the
client and references that stream via an integer ID. The StreamDaemon
implements the LowLevelStream interface which is called by RemoteStream for
service. Each API has the innteger stream ID passed as its first
parameter. The StreamDaemon looks up the referenced stream and dispatches
the method call to the real stream.
The StreamFactory class provides static factory methods (openFileStream() and
openProcessStream()) to instantiate streams on teh client.. These methods
call a client side export in the StreamDaemon to create the backing FileStream
and ProcessStream resources on the client. It obtains the integer stream
ID in return and intitializes a RemoteStream instance using this ID. The
resulting RemoteStream object is returned to the caller (e.g. business logic)
and all access to the RemoteStream is transparently redirected to the client.
INPUT THROUGH, OUTPUT THROUGH and INPUT-OUTPUT THROUGH all allow one to launch a
child process and access the child's stdout/stderr for reading, or the child's
stdin for writing or both. This is handled by ProcessOps. ProcessOps
is the Progress compatible interface for process launching. It accepts
RemoteStream instances and uses a Launcher interface and the client-side
ProcessDaemon instance on the client to actually process the process launch
(using the J2SE Runtime.exec() method). The resulting child process'
standard I/O is connected to the client-side ProcessStream instances. On
the client side, when the child process is launched, its stdin and/or
stdout/stderr are either connected to a pipe or to the parent process' terminal
depending on the flow direction of the stream definition. This is handled
with the cooperation of ProcessDaemon.launch() and the CHARVA
Toolkit.pseudoTerminalLaunch().
FileSystemOps provides a set of static methods to inspect and interact with the
file system. These are redirected to the client using the FileSystem
interface which is serviced by the FileSystemDaemon class.
As a general rule, the maximum amount of processing that can be done on the
server is done there and only a minimum remote interface is put in place to
provide remote service.
For other process launching (UNIX, OS-COMMAND...), such items don't need to be
read/written as streams on the server, but the launching must occur on the
client anyway. This means that the ProcessOps.launch(String[], boolean,
boolean) method which is used in these cases, is serviced by ProcessDaemon as
above, except the actual process launching mechanism is different (see below).
Non-Stream
Process Launch Remoting
Normal process launching (UNIX, OS-COMMAND...) is done in 3 forms:
- Synchronous (lack of NO-WAIT option)
- interactive
- non-interactive
- Asynchronous (NO-WAIT option)
Some launched applications are synchronous and interactive (the user is allowed
to interact with the child process' UI). In such cases, the child process'
standard I/O must be connected to the terminal (such as "vi"). Such
programs cannot be run with the NO-WAIT option (it just doesn't work properly in
Progress, your terminal gets very "confused"). The result is that the
child process takes over the terminal and Progress is blocked (it is a
synchronous call when NO-WAIT is not available). This means that no output
occurs to the screen except what is being done by the child process.
Java's Runtime.exec() process always creates a child process that whose STDIO is
not connected to a terminal. This means that interactive applications like
"vi" will not properly work when launched by the standard J2SE process launching
mechanism. Since Progress allows child processes to be interactive in the
current terminal, a different process launching mechanism is needed.
To duplicate this processing, the Charva implementation programs NCURSES to
temporarily "shell out" or suspend current processing (see
ThinClient.suspend()). This means that NCURSES resets the terminal
settings to those which were set at NCURSES init. At that point, a
child process is created (using fork). After fork, the parent process will
return and block (unless NO-WAIT was specified) until the child process exits.
If NO-WAIT is specified, the parent process resumes. The child process
copies the command line parameters and then calls the Linux/UNIX execvp() system
call. When the child process is done, it exits. The parent process
will use ThinClient.resume() to re-enable the interactive terminal after it
pauses (or just before it returns from ProcessDaemon.launch() if NO-WAIT was
specified). While the child process runs, all STDIO is directly connected
to the parent process' terminal (the child process is directly connected to the
user's current terminal).
The process launching API exists in the Charva libTerminal.so to handle the
fork() and execvp().
Either way, the worker for ProcessOps static methods is in the client's
ProcessDaemon. The ProcessDaemon uses features of the ThinClient to
properly integrate with the terminal, to suspend/resume CHARVA and to handle the
actual process launching.
Other applications are synchronous and non-interactive child processes.
Some of these may still write to STDOUT/STDERR. Progress attaches the
terminal I/O to the child process. Progress itself can't know what
processes are interactive and which are not. This is true because the fact
of whether a process is or is not interactive is something that cannot be
inspected, it is implicit in the application (or applications if the child
process launches other child proceses) being run. In addition, there is no
provision in the language for the programmer to provide a hint to Progress about
this fact. For this reason, we must conclude that Progress implements the
launch of all synchronous processes in the same way for each process ("shelling
out" the terminal, forking, execing to launch the process).
Finally, one can launch an asynchronous application (by adding the NO-WAIT
option). Such programs are always non-interactive applications.
However, if they do write to the STDOUT/STDERR, the result from an end-user
perspective is that in a NO-WAIT condition, the child process' output is
intermixed (in a non-deterministic manner) with output created by
Progress. For example (press the spacebar quickly when at the first "Press
Space to Continue" prompt):
os-command no-wait "echo
HELLO; sleep 3; echo 'HELLO, AGAIN!'".
def var c as char init
"Hello World!".
display c.
enable.
os-command silent "echo
GOODBYE". |
The NO-WAIT launch in Progress don't seem to be really any different from the
normal synchronous approach in terms of how the launch itself is done. The
only difference seems to be in whether or not Progress waits for the child to
exit. In a NO-WAIT case, the resume() is called immediately after
launching, such that Charva can resume immediately. In the synchronous
case, resume() is not used until after the child process exits. Either
way, this simply means that the decision to wait or not to wait is always made
just before the call the resume().
The remote worker methods handle NO-WAIT (making these methods synchronous -
waiting for the child process to end - or asynchronous) as documented
above. No special proxies or other features are ever returned to the
business logic for these ProcessOps.launch(String[], boolean, boolean)
modes. The server-side call invokes the remote worker which launches the
child process on the client synchronously or asynchronously and then
returns. The server-side call then returns to the business logic.
Note that the ProcessOps class has some "cleaner" classes and extra cleanup
threads to handle the proper close of the pipes and child process resources as
needed.
The NCURSES-related changes are in Charva's libTerminal.so (toolkit.c) and the
corresponding Toolkit.java.
NCURSES references:
http://www.faqs.org/docs/Linux-HOWTO/NCURSES-Programming-HOWTO.html
http://invisible-island.net/ncurses/ncurses.faq.html
http://www.tldp.org/HOWTO/Text-Terminal-HOWTO.html
(especially the bit about termcap/terminfo and the part on psuedo-terminals)
http://www.cs.utk.edu/~shuford/terminal/termcap_news.txt
(search on "newterm(" for a bit of an example on multiple terminals)
http://dickey.his.com/ncurses/ncurses-intro.html
You will especially want to review:
initscr() - default init for single terminal applications, uses newterm() and
other stuff under the covers
newterm() - creates a terminal, used instead of the initscr(), sets the current
terminal to that newly created instance and returns a pointer to it
set_term() - switches the current terminal to the one passed, returns the
previous terminal
delscreen() - deallocates memory for a terminal
SCREEN* - the data structure that holds the terminal "instance"
endwin() - used to reset the tty attributes/modes to the normal interactive
shell (or whatever was set before ncurses started), is used at the end of all
processing and also temporarily to "shell out" to another (non-ncurses) program
def_prog_mode() - temporarily saves off the current terminal settings, called
prior to an endwin() such that the settings can be restored after "shelling out"
Stream
Integration with the UI
In Progress 4GL there is also a concept of the "terminal". The terminal is
the user-interface (UI) with which an end-user will interact. There are a
wide range of language features provided to allow a UI to be created and
maintained in Progress. Progress does differentiate between "regular"
streams (e.g. file system and child process I/O) and the terminal. For
example, there are certain operations that only are allowed on a non-terminal
stream (e.g. PUT, EXPORT, IMPORT, SEEK...). However, in other ways there
is little differentiation between the terminal and streams. In other
words, the terminal can be redirected to a stream and many UI statements that
normally only have interactive meaning become non-interactive and use their
frame-oriented layout and formatting knowledge to read from the stream into a
field list (INSERT/SET/UPDATE/PROMPT-FOR) or write from a field list to the
stream (DISPLAY). Some UI statements don't read or write to the stream,
but do operate on the stream in other ways (DOWN, UP, HIDE, VIEW).
These reads/writes can be made on a replaced unnamed stream (e.g. INPUT FROM
myfile.txt) or from a named stream (e.g. INPUT STREAM my-stream-name FROM
myfile.txt). In Progress, the former method operates on the "unnamed"
stream which by default is the same as the terminal. Thus, normally a UI
language statement would operate upon the terminal by default, but some
conditional processing (e.g. IF/THEN/ELSE) might redirect the unnamed input
and/or output stream(s) such that a single UI statement would operate on either
resource. If that statement happens to be operating on the terminal, then
the statement is interactive. If it is stream based, the statement will
have no interactive (end-user observable) side effects. It is only at
runtime that Progress can determine which "path" will be taken for these UI
statements that can also be used on streams. Since named streams can never
reference the terminal, when a UI statement uses an explicit (named) stream, it
is easy to identify that this is a stream statement and not really a UI
statement. However, when the unnamed stream has been redirected, these
statements become "bimodal". While there are some simple examples to the
contrary, in many cases there is no way to know how that statement will be
serviced (I/O versus UI) by analyzing the source code at conversion time.
Since such a case must inherently be determined at runtime, there must exist a
common interface for these methods which then detects and uses the proper
mode. Consider this example:
def var redir as logical.
def var i as integer.
/* some processing that calculates redir, possibly with UI or
database reads */
if redir then
input from
file.txt.
set i.
message "i =" i. |
Due to the runtime nature of this decision, backing methods for
INSERT/SET/UPDATE/PROMPT-FOR/DISPLAY have been created in
GenericFrame
(actually, there is no support for INSERT support at this time). All of
the frame elements (in the proper order) being read are passed as a
FrameElement
array. If an explicit stream is passed to SET/UPDATE/PROMPT-FOR then it is
expected to be open for input (if not an error occurs). In stream
processing mode, a completely different operational mode is entered which uses
an input reading method to feed the screen buffer from the input stream.
This bypasses the normal user interface processing for enable/disable, standard
view/Z-order/refreshing support. This mode is entered when a stream is
explicitly passed to the API OR if no stream is explicitly passed but the
unnamed input stream is redirected (has been explicitly opened as a stream).
There are 2 forms that can cause this to occur:
- The unnamed input stream is opened as a file or child
process. It defaults to the terminal, but once opened (and as long as
this is not closed) it will reference the non-terminal stream.
- An explicit stream reference can be added to the
INSERT/SET/UPDATE/PROMPT-FOR language statement.
Either way, this forces the input to be removed from the terminal/keyboard and
obtained via the stream. This also eliminates the ENABLE/DISABLE
processing and the WAIT-FOR that is done inside the PROMPT-FOR portion of these
statements. Instead, the order of the fields in the SET/UPDATE/PROMPT-FOR
is the order in which fields are read from the stream. This makes these
statements a kind of analog to the PUT language statement (this writes a list of
fields to a stream). It is important to note that the layout of the
associated *frame* DOES NOT make any difference in how the data is read.
For example if the frame layout itself is ordered differently than the order of
the fields in this language statement OR if the frame has other fields that are
not listed in the language statement, these other fields and ordering are
ignored. It also means that the list of already enabled widgets makes no
difference (only the fields listed explicitly on this statement will be
read
and they will be read in the exact order of this current statement). One
could say that this is really just a reading counterpart to PUT except for these
two points:
- A single source code statement can both handle real UI and reading
from the unnamed stream and this fact is detected and implemented at runtime
only.
- These statements can also
write data, either to the screen (if output is not redirected) or to a named
or unnamed stream (if output is redirected). This additional output
only occurs if the stream in question does
not have NO-ECHO specified (e.g. INPUT FROM myfile.txt NO-ECHO).
The Stream class supports setting the echo mode (it defaults to
true
). Please note that for streams opened via the INPUT-OUTPUT THROUGH
language statement, this value must default to
false
, which is handled by emitting a setter into the business logic in these
cases. Interestingly, even if the input stream is marked as echo, if
output is redirected and that output stream is marked as no-echo then the echo
does not occur. In other words, if both input and output are redirected,
both streams must be marked as echo for the echo to occur. If output is
not redirected, then only the input stream's echo setting is taken into account
since the screen is always considered to be in echo mode.
There is also an unusual behavior in this echo mode. The
SET/UPDATE/PROMPT-FOR language statements are "bimodal" (they handle both
terminal and stream I/O). In stream mode, if echo is off, no output occurs
(to either the terminal or to a file if the unnamed out is redirected to file
before the call occurs). If echo is on, then output is generated AND it is
based on the frame definition/screen-buffer that is in use. However, the
output is done in a "special" text only mode. In other words, the output
values are the exact text that has been parsed out of the input file AND it is
NOT data type specific. Any data in a fill-in field that has a data type
that is non-character is treated as character. In addition, it seems that
any widget type other than fillin (e.g. combo-box or button but strangely enough
also text!) generates no output at all. In all cases (whether the widget
does or doesn't emit), the column headings are done just as normal. A
*normally*
right-aligned field like a decimal fill-in will have its column header right
aligned BUT its data will be the unformatted, left-justified text just as it was
read from the input stream. For example:
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│c[1]
c[2]
c[3]
l l2
dd
i
d│
│────────────────────
────────────────────
────────────────────
──── ──
──────────
─────────
─────────────────│
│Double quoted
text
'single
quoted
fals T 11/22/1999 -9,98787 +2,323243.32
│
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│c[1]
c[2]
c[3]
l l2
dd
i
d│
│────────────────────
────────────────────
────────────────────
──── ──
──────────
─────────
─────────────────│
│
Double quoted text
'single
quoted
Nope t 11/22/1999 -00998787 00002323243.32000│
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
The first output is the echo mode of SET. The second output is using
DISPLAY on the same data that was read. The first 3 columns are character
data and thus show no difference. The l and l2 columns are logicals.
The logical data read from the file differs from the 2 outputs (they are both
using format strings "Yep/Nope" and "t/f" respectively). The dd field is a
date and doesn't show any difference. The i (integer) and d (decimal) fields
show the unformatted, left aligned text read from the file in the echo mode and
the properly formatted, right aligned data in the DISPLAY. The i format
string is -99999999 and the d format string is 99999999999.9999 for both the SET
and DISPLAY.
So both cases use the same frame layout and format strings, but the actual data
is treated as unformatted text in the echo mode.
At this time, this special text mode for
logicals, integers and decimals is not supported.
As an additional unusual behavior of this quirk,
any errors resulting from truncating
the input data cause this truncated data to be written back to the screen buffer
before the error condition is raised (the SET/UPDATE form do not assign that
data back but the data does exist in the screen buffer). If echoing is
active, the echo of that screen buffer will occur even though an error
occurred! At this time, this
screen buffer copy is only supported for character data types, the other types
do not get stored.
INSERT/SET/UPDATE/PROMPT-FOR are reading instructions when used with an input
stream that is not the terminal. Each has a list of lvalues (fields). Data
is "read" from the stream on a whitespace delimited (and for character data,
optionally double quote delimited) basis. It is not possible to control
the delimiter as can be done via IMPORT. The parsed data is converted into
each corresponding field's type and the data is then assigned into that field.
Each field's format string (default, from dictionary, from var def, overridden
in a format phrase) will be honored in that conversion. The format string
is obtained from the associated widget's configuration rather than being passed
via the API itself.
All reading is done in the exact order of the widgets as listed in the frame
element array passed to the SET/UPDATE/PROMPT-FOR APIs. The order of these
same fields in the frame itself is meaningless. So:
form x y z with frame fr.
input from ./text.txt.
set z x y with frame fr.
input close.
The order of the fields in the form statement is the order of the data when
displayed on the screen or when output is redirected to a stream. BUT any
stream input reading honors the order in the SET/UPDATE/PROMPT-FOR. In the
example above, this means that the first double quote or whitespace delimited
entry on the current line in the input stream will be assigned to the "z"
variable, the second to "x" and the third to "y".
It is important to note that the field-level data parsing is identical to that
done for the IMPORT language statement. So field skipping when reading an
embedded hyphen, reading 1 line at a time (per SET/UPDATE/PROMPT-FOR), no
assignment to the screen-buffer for fields read after the end of the line has
been found, double quote processing as a single entry (allows embedded
whitespace or hypens)... all these behaviors are the same. This is exposed by
the Stream.readFieldWorker(). This worker is shared by the IMPORT
implementation, but how the results are used (and the validation that is done)
are quite different after the parsing is done.
In particular, this form of reading is uses the format string as a basis for the
proper length of the data to read from the stream for the given field however
the actual contents of the format string are not used to parse except for the
logical type. In that case, the values specified in the format (e.g.
"Yep/Nope") will be honored. For all other data types, the only affect of
the format string is on the size. An error condition is generated if data
is read that is larger than can fit into the size specified by the format
string.
The conversion from string data (as read from the file) to BaseDataType is
handled by detecting the widget's data type and then instantiating the proper
type using the constructor that using a string as input.
The way the logical formatting is processed is an exception. This
processing trucates the input text (if needed) to the maximum size of either of
the 2 values in the format string (for "Yep/Nope" then maximum size would be
4). Any truncation here is not an error (as it is with the other types),
instead the data value (the string read from the stream) and the format string
are passed to a special logical constructor for processing. Please note
that this means that (as in Progress), there is a quirk where a longer input
(for example, "nopenotatall") could actually match the longer of the 2 format
string values (for example, the
false
value of the "Yep/Nope" format). This also applies to true/false and
yes/no. So a value of "yessiree" would be interpreted as
true
in any case where the longest format value is 3 characters.
For a more interesting example, please see
testcases/uast/io_redirected_by_ui_stmts.p.
Consider these examples:
Example
Program
|
Output
|
def var i as int.
input thru "echo 5".
output to ./test.txt.
prompt-for i.
output close.
message input i.
|
test.txt contains:
i
----------
5
The message line on the terminal contains:
5
|
def var i as int.
input thru "echo 5".
output to ./test.txt.
prompt-for i.
message input i.
output close.
|
test.txt contains:
i
----------
5
5
The message line on the terminal contains nothing.
|
def var i as int.
input thru "echo 5".
prompt-for i.
message input i.
|
The terminal displays:
┌──────────┐
│
i│
│──────────│
│5
│
└──────────┘
The message line on the terminal contains:
5
|
Please note that any statement that generates output (both PROMPT-FOR and
MESSAGE in the examples above) will do so to the redirected stream.
A special case exists when input is NOT redirected BUT output IS
redirected. In this case PROMPT-FOR, SET and UPDATE all will actually
allow interactive editing via the normal terminal (like a temporary switch to
interactive mode) BUT then after the edits are done there is effectively an
additional VIEW of the data to the output stream.
The most common redirected output statement is DISPLAY. DISPLAY writes the
given list of fields/expressions to the stream using the format and layout of
the frame. This is the most common way to generate a report in a file or
for a printer. This can be a redirection of the unnamed stream or the
DISPLAY statement can also reference a named stream.
The format of the stream output will contain explicit (SKIP) and implicit (based
on a list of fields that exceeds the display's width) newline characters exactly
as the terminal would display. The following example illustrates
this:
Example
Program
|
Output
|
def var txt1 as
character.
def var txt2 as
character.
def var
i as integer.
do i = 1 to 6:
txt1 =
txt1 + "Hello World! ".
end.
txt2 = txt1.
display txt1 format
"x(78)" txt2 format "x(78)".
|
┌──────────────────────────────────────────────────────────────────────────────┐
│txt1
│
│txt2
│
│──────────────────────────────────────────────────────────────────────────────│
│Hello World!
Hello World! Hello World! Hello World! Hello World! Hello World!
│
│Hello World!
Hello World! Hello World! Hello World! Hello World! Hello World!
│
└──────────────────────────────────────────────────────────────────────────────┘
|
output to ./test.txt.
def var txt1 as character.
def var txt2 as
character.
def var
i as integer.
do i = 1 to 6:
txt1 =
txt1 + "Hello World! ".
end.
txt2 = txt1.
display txt1 format
"x(78)" txt2 format "x(78)".
|
test.txt contains:
txt1
txt2
------------------------------------------------------------------------------
Hello World! Hello
World! Hello World! Hello World! Hello World! Hello World!
Hello World! Hello
World! Hello World! Hello World! Hello World! Hello World!
|
It is also important to note that the box normally drawn around a frame is not
output to the stream. Instead there is an empty line where the top and
bottom horizontal lines would be and leading spaces where the left vertical line
would be. No output is generated for the right vertical line. If the
frame is defined as a NO-BOX, then none of those spaces or extra lines will be
output.
Column headings do get output using hyphen characters (and a space in between
each column) as the separators. NO-LABELS and SIDE-LABELS are honored just
as they would be in the terminal.
In the case of the DISPLAY statement, it is important to note that it still
copies data to the screen buffer and it is the resulting screen buffer that is
written to the stream.
A close "cousin" of DISPLAY is the VIEW language statement. This does not
copy any new data to the screen buffer, however it does write the contents of
the referenced screen buffer to the named or unnamed stream. This is often
used to output a static (unchanging) report header multiple times. Output
is done using "frames" (screen buffers). If that frame (including the
unnamed frame) is referenced for both file and terminal output, the data placed
in that screenbuffer at the time of output will include all prior data.
For this reason Progress source code often uses a "fake" frame for file I/O so
that that data doesn't appear on the terminal later.
The HIDE statement can also be used in these cases, but there appears to be no
effect of such a statement (it is a NOP except in an obscure case -- see below).
The following examples show the affect of down frames and the DOWN and UP
language statements.
Example
Program
|
Output
|
output to ./test.txt.
def var i as integer.
do i = 1 to 6 with
frame fr:
display i
with 12 down frame fr.
end.
|
test.txt contains:
i
----------
1
2
3
4
5
6
|
output to ./test.txt.
def var i as integer.
do i = 1 to 6 with
frame fr:
display i
with 12 down frame fr.
up 1.
end. |
test.txt contains:
i
----------
1
2
3
4
5
6
|
output to ./test.txt.
def var i as integer.
do i = 1 to 6 with
frame fr:
display i
with 12 down frame fr.
down 1.
end. |
test.txt contains:
i
----------
1
2
3
4
5
6
|
def var i as integer.
do i = 1 to 6 with
frame fr:
display i
with 12 down frame fr.
end.
|
The following is displayed on
the terminal:
┌──────────┐
│
i│
│──────────│
│
1│
│
2│
│
3│
│
4│
│
5│
│
6│
│
│
│
│
│
│
│
│
│
│
│
│
└──────────┘
|
def var i as integer.
do i = 1 to 6 with
frame fr:
display i
with 12 down frame fr.
up 1.
end. |
A series of screens is
displayed on the terminal, with each screen alternating between a
number (1-6) and a blank/empty output. 12 screens are
displayed in all, each in turn. The first looks like:
┌──────────┐
│
i│
│──────────│
│
1│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
│
└──────────┘
|
def var i as int no-undo. do i = 1 to 10: display "Count: " i with 10 down. if i < 6 then down. else up. end.
|
The result on the terminal:
┌──────────────────┐
│
i│
│
──────────│
│Count:
1│
│Count:
10│
│Count:
9│
│Count:
8│
│Count:
7│
│Count:
6│
│
│
│
│
│
│
│
│
└──────────────────┘
|
def var i as int no-undo. output to tmpup1.out. do i = 1 to 10: display "Count: " i with 10 down. if i < 6 then down. else up. end.
|
The result in the stream:
i
----------
Count:
1
Count:
2
Count:
3
Count:
4
Count:
5
Count:
10
|
def var i as integer.
do i = 1 to 6 with
frame fr:
display i
with 12 down frame fr.
down 1.
end. |
The following is displayed on
the terminal:
┌──────────┐
│
i│
│──────────│
│
1│
│
│
│
2│
│
│
│
3│
│
│
│
4│
│
│
│
5│
│
│
│
6│
│
│
└──────────┘
|
Following rules determine the Progress behavior of the DOWN/UP/SCROLL statements
with a down frame, with or without the RETAIN/SCROLL n attributes:
- Up statements with a down frame (see
testcases/uast/downframes/up_statements_with_a_down_frame.p):
- For a down frame, when an up statement lands on an
uninitialized row,the screen buffer will be reseted.
- Up statements which land on an initialized row will restore the
screen buffer for that row.
- If there is an attempt to go up past the first row, then
FRAME-LINE will be set to FRAME-DOWN value and screen buffers will be
reseted for alliterations.
- Down statements with a down frame (see
testcases/uast/downframes/down_statements_with_a_down_frame.p):
- For a down frame, when a down statement lands on a
uninitialized row, the screen buffer will be reseted.
- Down statements which land on a initialized row will restore
the screen buffer for that row.
- If there is an attempt to go down past the FRAME-DOWN row, then
FRAME-LINE will be set to 1 and screen buffers will be reseted for all
iterations.
- Down frame with stream output (see
testcases/uast/downframes/down_frame_with_stream_output.p):
- The up statement will have no effect on the screen buffer. The
down statement will reset the screen buffer and advance to the next row.
- If there is an attempt to go down past the FRAME-DOWN row, then
a new row will be added to the frame.
- FRAME-LINE for streamed frames will always be 0.
- Conditional up/down statements (see
testcases/uast/downframes/up_down_conditional_with_a_down_frame.p):
- if the up/down lands outside the frame, the frame gets cleared
only when the conditional up/down is triggered by a down or display
instruction.
- after a conditional down, the "input frame f0 i" will return
the value specified by the row at frame-line + 1.
- after a conditional up, the "input frame f0 i" will return the
value specified by the row at frame-line - 1
- Up/Down statements with an uninitialized down frame (see
testcases/uast/downframes/up_down_statements_with_a_down_frame_uninitialized.p):
- For uninitialized frames, performing up or down statements have
the same behavior as if the frame would have been initialized.
- Going up over the first row will reset all screen buffers and
land on FRAME-DOWN row.
- Going down over the FRAME-DOWN row will reset all screen
buffers and will land on first row.
- Retain n - down statements with a down frame (see
testcases/uast/downframes/retain_n_down_statements_with_a_down_frame.p): For
a down frame, when a down statement lands on a uninitialized row:
- if retain = 0, screen buffer will be reseted for all rows,
FRAME-LINE is 1
- if retain != 0, then the last RETAIN rows will be set to be the
first RETAIN rows in the same order; FRAME-LINE is (RETAIN + 1). The
screen buffer for the last (FRAME-DOWN - RETAIN) rows will be reset.
- Retain n - up statements with a down frame (see
testcases/uast/downframes/retain_n_up_statements_with_a_down_frame.p): For a
down frame, when a up statement goes past the first row:
- if retain = 0, screen buffer will be reseted for all rows,
FRAME-LINE is FRAME-DOWN
- if retain != 0, then the first RETAIN rows will be set to be
the last rows in the form, in the same order; FRAME-LINE is (FRAME-DOWN
- RETAIN). The screen buffer for the first RETAIN rows will be reset.
- Scroll n - down statements with a down frame (see
testcases/uast/downframes/scroll_n_down_statements_with_a_down_frame.p): For
a down frame, when a down statement lands on a uninitialized row:
- if scroll = 0, screen buffer will be reseted for all rows,
FRAME-LINE is 1.
- if scroll != 0, then the rows are pushed up SCROLL times, the
last SCROLL lines being empty lines. FRAME-LINE will be set to
FRAME-DOWN. The first original SCROLL rows will be lost and
inaccessible.
- Scroll n - up statements with a down frame (see
testcases/uast/downframes/scroll_n_up_statements_with_a_down_frame.p): For a
down frame, when a up statement goes past the first row:
- if scroll = 0, screen buffer will be reseted for all rows,
FRAME-LINE is FRAME-DOWN
- if scroll != 0, then the rows are pushed down SCROLL times, the
first SCROLL lines being empty lines. FRAME-LINE will be set to
FRAME-DOWN - SCROLL.The last original SCROLL rows will be lost and
inaccessible.
- Scroll statement with a down frame (see
testcases/uast/downframes/scroll_statement_with_a_down_frame.p): When
from-current is not used:
- FRAME-LINE remains unchanged
- scroll down will act as if FRAME-LINE is 1 and from-current is
used
- scroll up will act as if FRAME-LINE is FRAME-DOWN and
from-current is used
- SCROLL and RETAIN values have no effect on the SCROLL statement
- Scroll from current with a down frame (see
testcases/uast/downframes/scroll_statement_from_current_with_a_down_frame.p):
When from-current is used:
- FRAME-LINE remains unchanged
- scroll from-current down will push-down the rows starting from
FRAME-LINE. The FRAME-LINE row will be reset.
- scroll from-current up will pull-up the rows starting from
FRAME-LINE. The last row will be reset. The FRAME-LINE row will have the
screen-buffer from FRAME-LINE+1. If FRAME-LINE is FRAME-DOWN then the
last row will be reset.
- SCROLL and RETAIN values have no effect on the SCROLL
statement.
- SCROLL n and RETAIN n should never be used together (see
langerf.pdf page 543).
Following rules determine the Progress behavior of the conditional DOWN
statement with a streamed, down frame (with or without box) (NL = NewLine) (file
testcases/uast/downframes/streamed_paged_frame1.p):
- If the current line is at the end of a page, then: a DOWN 1
statement will write a NL to the next page; a DOWN 2 statement will write a
NL to the current page and a NL to the next page.
- If the current line is at position X having Y more rows in the
current page, then a DOWN Y statement will write Y NL's to the current page
and one more NL to the next page.
- If the current line is at position X having Y more rows in the
current page, then a DOWN (Y+Z) statement will write Y+1 NL's to the current
page and one more NL to the next page.
- The file testcases/uast/downframes/streamed_paged_frame2.p
demonstrates how Progress computes the LINE-COUNTER and PAGE-NUMBER
functions: after a down, if the page is ended, P2J erronetly will update the
line counter to 1 and also increment the page number. Progress seems to do
this when the subsequent statement (down/display/etc) which affects the
stream buffer is executed. NOT IMPLEMENTED YET.
Some notes:
- DISPLAY and VIEW generate a fully formatted output (very close to
the result when viewed on a terminal) where SET/PROMPT-FOR/UPDATE generate
the field output in a "raw" form. In the DISPLAY examples, the
numbers are right aligned where in the PROMPT-FOR examples, the numbers are
left aligned.
- When writing, the UP statement has differing behavior. On the
terminal, it seems to defeat the normal behavior of a DOWN frame. On
the stream, it usually does nothing, however in some cases it can cause a
form of overwriting.
- When writing, DOWN seems to do a similar thing for both terminal
and stream. This behavior extends to the fact that some usage of DOWN
will overwrite the same data rather than generating multiple lines.
- Although this is not shown above, when reading from a stream neither DOWN nor
UP have any apparent effect. Most important, these language statements
do not change the current read position in the stream.
- DISPLAY and VIEW have a special behavior when output is redirected
AND the output stream is paged AND the given frame is either PAGE-TOP or
PAGE-BOTTOM. In this case, the given frame is registered as a header
(PAGE-TOP) or footer (PAGE-BOTTOM) and this will be displayed at the top and
bottom of each page respectively. The only difference is that DISPLAY
will ALSO cause the frame to output immediately (as well as being "called
back" for the header/footer) while VIEW only causes a registration but not
an actual output.
- HIDE only has an effect in the case where output is redirected AND
the output stream is paged AND the given frame is either PAGE-TOP or
PAGE-BOTTOM. In this case the frame is removed from the list of
headers/footers.
- MESSAGE statements output to the stream but in the case where input
is NOT redirected and output IS redirected, they have the curious behavior
of displaying on both the terminal and the redirected output stream.
If input IS redirected (and output IS redirected), then there will be no
interative terminal output, only output to the redirected output stream.
- MESSAGE SET/UPDATE using a redirected input stream operate just
like a SET/UPDATE from a redirected input stream (it is a stream reading
statement).
- ENABLE, PUT SCREEN, BELL, STATUS, PAUSE always operate on the
interactive terminal and have no effect on any redirected streams.
The methods that are the equivalent to the DISPLAY, VIEW, HIDE, DOWN, UP
statements are available in forms that take an explicit Stream reference as a
parameter (see the
CommonFrame
interface and the
GenericFrame
implementation). In the case where any of these are called with an
explicit stream reference the UI runtime code detects that these are outputs to
a stream, such that the terminal is temporarily redirected to the given
stream. Likewise, if an open has occurred on the unnamed output stream,
this causes the terminal to be redirected to this stream (if it isn't already
redirected). The explicit stream reference form removes the redirection
when the statement (e.g. DISPLAY) is complete. The unnamed output stream
terminal redirection remains in place until that unnamed output stream is
closed.
The thin client maximizes the usage of the drawing and layout management of
frames while replacing the back-end output technology. This is done by
implementing a pluggable replacement set of drawing primitives in a class named
OutputPrimitives in CHARVA. The CHARVA Toolkit implements a default
instance of OutputPrimitives which maps all calls to the native Toolkit backing
functions. This means that by default, CHARVA uses NCURSES for all
output. The Toolkit has an interface to switch primitives and the thin
client has a
RedirectedTerminal
class that implements this interface. This class is used for each stream
that is used for redirected output. These drawing primitives duplicate the
buffering, clipping, cursor movement, drawing and rendering (sync) functions
using a stream as the output device.
In addition, the server's UI runtime code is aware of each session's unnamed
streams. This is used to enable use of unnamed streams from the UI
language statements (which can be intermixed between "redirected" UI stmts and
I/O stmts that can't be used on a terminal). See
UnnamedStreams
.
PAGE-TOP (header) and PAGE-BOTTOM (footer) frames are honored and the DISPLAY,
VIEW and HIDE language statements register and deregister these frames with the
associated streams. When the stream starts a new page it forces all
headers in the active header list to render (in order of registration) and
likewise, at page end all footers are rendered. Any partial page at stream
close will be extended to the page end and the footers will be rendered
there. PAGE-TOP and PAGE-BOTTOM frames that are not used on a paged output
stream have no special effect (they are treated as normal frames).
In the Progress 4GL environment, certain frames can exhibit overwrite
behavior. This can occur because of a frame's implicit DOWN 1 behavior OR
because of a programmer's explicit use of DOWN/UP (see examples above).
This appears to be a side-effect of the buffering strategy used by
Progress. Writing seems to occur to a 512 byte buffer (based on my
testcases). This buffer is flushed when Progress needs to write something
that would exceed the 512 byte mark OR when the stream is closed. This can
be seen in the following testcase by changing the iterations from the default 8
to 7. The pause then occurs before the output from cat. Please note
that this example does not exhibit the overwriting behavior (the cause of this
is unknown at this time), but the testcase does show the buffering approach.
def var iterations as int init 8.
message "Redirect?" set redir as logical.
if redir then
output thru cat.
message "Iterations?" update iterations.
def var i as int.
def var c as char format "x(64)" init "1".
do i = 1 to 63:
c = c + "0".
end.
i = 0.
repeat while i < iterations:
i = i + 1.
display c with 1 down no-labels no-box.
end.
pause.
This overwrite behavior is not supported at this time.
Limitations:
- The special text mode echo quirk
for logicals, integers and decimals is not supported. Likewise the
screen buffer copy on error is only supported for character types, not all
types.
- Mixed usage of the PUT language statement and UI statement output
redirection support (e.g. DISPLAY) may not operate properly.
- The overwrite behavior of certain frames
(based on implicit DOWN or explicit DOWN/UP) is not supported as there does
not seem to be a useful reason to actually code an application to do this.
- On Linux, any non-stream associated process launching that has a
PAUSE will suffer from a problem where the PAUSE will cause the output of
the child process to be cleared from the screen (this is not how it works in
Progress). This is due to a defect in NCURSES getch() which is fixed
in NCURSES 5.5.
- Stream width is hard coded to a maximum of 512 columns and 128
lines. A more dynamic/flexible approach should be implemented based on
the (column and line) values in the current frame however this has not been
handled at this time.
Using Frames in Multistream Applications
Progress
applications can use multiple streams simultaneously. An output stream is used
typically with more than one frame and a single frame can be used with more than
one output stream during the application run. This raises questions
about the frames and streams behavior in such a mixed environment.
There are three classes of behavior here:
- a single stream receives output from the multiple frames
- a single frame gives output to the multiple streams
- multiple streams receive output from multiple frames, one frame per
stream and their relations don't change
The first class triggers some processing in the stream every time the stream
gets prepared to get output from another frame. We will call it
frame switching.
The second class triggers some processing in the frame every time the frame gets
prepared to give output to another stream. We will call it
stream switching.
The third class is just a boundary condition where no switching occurs.
For the purpose of this discussion, we won't distinguish the named streams from
the redirected unnamed stream, which is the terminal, whenever appropriate, and
use the term
stream. The term
terminal, conversly, will mean
non-redirected unnamed stream.
The frame switching behavior is defined by the following rules:
- every stream keeps track of the frame it was last used with
- whenever the frame is about to produce some new output, the last
used frame of the stream is compared with the target frame for the
current operation and when these are different, the frame switching takes
place
- PAGE-TOP and PAGE-BOTTOM frames do not participate in the frame
switching when they are used as page elements
- the switching itself causes flushing of the buffers of the remote
terminal associated with the stream and then the streams buffers so that no
partial output remains buffered anywhere
- flushing the terminal is a no-operation.
The stream switching behavior is defined by the following rules:
- every frame keeps track of the stream it was last used with
- VIEW, DISPLAY and DOWN statements check the last used stream of the
frame with the target stream for the operation and when these are different,
the stream switching takes place
- HIDE statement is known not to check for the stream switch
- the switching itself causes flushing of the buffers of the remote
terminal associated with the last used stream and then the streams buffers
so that no partial output remains buffered anywhere
The DOWN and UP statements work differently depending on the stream or terminal
as the target. The following rules are applied, if the target device is a
stream:
- UP statement is a no-operation for streams
- for this very reason, no up/down pending counters are maintained
for streams and those are reset to 0 with every DOWN
- conditional DOWN still sets the down pending flag, which is checked
with VIEW, DISPLAY or unconditional DOWN
- the effect of the conditional DOWN, when it's applied, differs from
a DOWN 1:
- conditional DOWN flushes the current remote terminal's buffers
to the stream and is a no-operation when the buffers are empty
- unconditional DOWN N flushes the current remote terminal's
buffers to the stream and puts a newline on the stream when the
buffers are empty, then adds N - 1 newlines
- conditional DOWN is OK for 1 down frames
- both forms of DOWN clear the current iteration of the frame and
reset pending flag and counters
- implicit conditional DOWN is no different
- frames that were used with a stream at least once, are permanently
flagged
- the above mentioned flag is checked when the frame gets cleared
with DOWN processor for the terminal as target
There are things specific to the terminal as the target device:
- the stream association flag mentioned above triggers a different
frame iteration clearing procedure during DOWN processing:
- the down frame iteration cursor remains unchanged;
- clearing is initiated
- clearing the current iteration causes an implicit PAUSE on the
terminal
- the visibility attribute of the frame is meaningful only for the
terminal use of the frame and is preserved across the stream-oriented
operations.
Transaction Processing
and Block Properties
Summary
Progress 4GL has a transaction processing environment integrated into the base
language. While some explicit control is provided over how transactions
occur, the vast majority of processing is implicit based on the operations,
control flow structures, block properties and sequence of statements/blocks which is encoded by the programmer.
Block Types
In the 4GL, code can be structured into related sections called blocks. In
Progress v9, the following types of blocks are available:
- external procedure
- internal procedure
- function
- trigger
- DO
- REPEAT
- FOR
- EDITING
A block groups code for several purposes:
- flow of control
- transaction processing
- UNDO processing (related to, but different from transactions which by largely a database-only concept)
- scoping related operations (certain 4GL resources such as buffers
or frames have behavior that depends on the flow through or of the
block to which they are associated or "scoped")
Each block has a "header" and a "body". All code that is executed in a
block is contained in the block body. The description of the block itself
and any explicitly controlled options are defined in the block header.
The lines of code in a block's body are executed "top to bottom". All code
in the same block will share the same behavior in terms of transaction
processing. Absent any explicit changes to the flow of control (by
language statements that cause a flow of control change) or the generation of a
condition (such as an ERROR) which will cause the flow of control to change, all
code in a given block is executed if the block body is entered or all the
code is not executed if the block body is not entered.
Some blocks can be made to iteratively execute the code contained in the block
body. Such blocks are known as looping blocks. Progress provides
blocks that can explicitly loop based on programmer controlled expressions in
the block header.
Blocks support transaction processing which is the ability to undo
(reverse) edits to variables or the database that occurred within the scope of
that block, when an abnormal condition is raised or when an explicit UNDO
statement is encountered. Some blocks can have their support for
transactions explicitly coded via the TRANSACTION keyword in the block header.
All blocks have some default behavior when abnornal conditions occur. Some
blocks allow that behavior to be explicitly specified via "ON phrases" in the
block header.
Some blocks provide retry support which is the re-execution of a block with the
same (original) state as the optional action in response to an abnormal
condition.
Some blocks can take parameters and be called explicitly (external
procedures, internal procedures and user-defined functions) or are
otherwise invoked as a callback (triggers). Triggers are special
in that
they are top-level blocks but they can only be invoked indirectly in
response to
an event (usually associated with the user interface but that is not
required)
that has occurred.
In this document, such blocks are called "top-level blocks".
Other blocks
(all forms of DO, REPEAT, FOR and EDITING) are not top-level blocks and
as such
can only be contained inside another block (top-level or otherwise).
In this document, such blocks are called "inner blocks".
Inner blocks are the only kind of block that can be labeled
and that label can then be referenced in language
statements that explicitly change the flow of control such as UNDO
statement, LEAVE statement, NEXT statement and in ON phrases.
Some blocks can have their definitions made nested inside another
block.
The top level blocks (except for triggers) cannot be nested.
Inner blocks can be nested. Do not confuse nesting with
recursion.
The top level blocks can be called recursively (they can call
themselves
directly or indirectly causing multiple instantiations on the call
stack).
But the top level blocks cannot have their definitions made on a
nested
basis. For example, an external procedure cannot be defined
inside another
external procedure.
The following summarizes these behaviors:
Block Type |
Looping |
Default
Transaction Level |
TRANSACTION
Option |
ON Phrases |
Retry
Support |
Parameters |
Callable
(Via Name) |
Labeled |
Nestable |
external procedure |
no |
sub-transaction |
no |
no |
yes |
yes |
yes (filename in a RUN statement) |
no |
no |
internal procedure |
no |
sub-transaction |
no |
no |
yes |
yes |
yes (RUN statement) |
no |
no |
function |
no |
sub-transaction |
no |
no |
yes |
yes |
yes (in an expression) |
no |
no |
trigger |
no |
sub-transaction |
no |
no |
yes |
no |
no |
no |
yes |
DO |
no |
no transaction |
yes |
yes |
yes (depending on options) |
no |
no |
yes |
yes |
DO TO |
yes |
no transaction |
yes |
yes |
yes (depending on options) |
no |
no |
yes |
yes |
DO WHILE |
yes |
no transaction |
yes |
yes |
yes (depending on options) |
no |
no |
yes |
yes |
DO TO WHILE |
yes |
no transaction |
yes |
yes |
yes (depending on options) |
no |
no |
yes |
yes |
REPEAT (all forms) |
yes |
sub-transaction |
yes |
yes |
yes |
no |
no |
yes |
yes |
FOR FIRST |
no (unless there is an additional table defined with EACH
table) |
sub-transaction |
yes |
yes |
yes |
no |
no |
yes |
yes |
FOR LAST |
no (unless there is an additional table defined with EACH
table) |
sub-transaction |
yes |
yes |
yes |
no |
no |
yes |
yes |
FOR EACH |
yes |
sub-transaction |
yes |
yes |
yes |
no |
no |
yes |
yes |
EDITING |
yes |
sub-transaction |
no |
no |
yes |
no |
no |
yes |
yes (editing blocks for other frames can be directly
contained and any kind of editing block can be indirectly contained) |
Conditions
A condition is an unusual event that may occur during the course of
processing. The Progress programmer must be able to specify how each
procedure responds to the range of possible conditions. Progress 4GL
provides a mechanism to specify such behavior at the level of each block.
Progress defines blocks as having very specific "block properties" which define
behavior associated with a block. These properties can be explicitly
defined or are implicit. The beginning and end of each block corresponds
to the beginning and end of a "scope". These scopes can be nested and this
structure allows a Progress programmer to control the granularity to which a
block's properties are applied.
In Progress, all blocks have some form of implicit and/or explicit
condition processing, properties and transaction support.
One of the most central behaviors Progress provides is UNDO. This
is similar to the concept of a rollback of a transaction and allows one
to abort
partial changes to variables (and other non-database resources) when a
failure or problem is encountered. This is provided by default to
reduce the
amount of manual work by the programmer.
The behavior of which scope gets undone in transaction processing rollbacks is
defined by block properties. In addition, these block properties also
define how the block responds to each type of condition AFTER the UNDO
occurs. More specifically, the way that the application changes its flow
of control in response to a condition is something that is defined by block
properties (implicit or explicitly overridden where possible).
The following conditions are possible:
Condition
|
Default
Action in Response
|
ERROR
|
If a transaction is active,
UNDO the closest enclosing block that has the ERROR property (this
will also be a transaction or sub-transaction).
If a transaction is not active, the UNDO will be a no operation.
Then RETRY the same block (or loop) that has the ERROR property.
|
ENDKEY
|
If a transaction is active,
UNDO the closest enclosing block that has the ENDKEY property (this
will also be a transaction or sub-transaction).
If a transaction is not active, the UNDO will be a no operation.
Only blocks with the ENDKEY property will have an interactions
counter. The interactions counter is a simple count of the number
of input-blocking
language statements
have executed in the block. Any nested block that does not have an
interactions counter of its own will have its interactions counted in
the nearest enclosing block's interactions counter. This counter is
used to determine the behavior of the END-ERROR event (see below).
Then LEAVE the same block (or loop) that has the ENDKEY property. |
STOP
|
If a transaction is active,
UNDO the block that defines the transaction (as opposed to a
sub-transaction).
If a transaction is not active, the UNDO will be a no operation.
Then RETRY the "startup procedure". This means that the
*entire* top-level procedure is run again from the top.
Note that if this is generated based on a database disconnect, a
special form of processing will occur where the default STOP
behavior cannot be explicitly overridden until an exit to a scope in
which no access is made to the database that was disconnected.
At this point the normal STOP processing is re-imposed (see Progress
Programming Handbook page 5-24).
|
QUIT
|
If a transaction is active,
COMMIT the block that defines the transaction (as opposed to a
sub-transaction).
If a transaction is not active, the COMMIT will be a no operation.
Exit to the operating system. |
END-ERROR
|
Technically, this really isn't
a separate condition, but rather a key in the UI which generates 2
different conditions based on context. An END condition is
generated if the key press occurs during the first input-blocking
language statement in the block. An ERROR condition is
generated if the key press occurs during a subsequent input-blocking
language statement in the block.
The exception to this processing occurs for blocks that do not have the
ENDKEY property (this can happen for some DO blocks and is always true
for EDITING blocks). In such a case, the interactions counter is not
maintained for the current block but
rather is maintained at the level of the nearest enclosing block which
has the ENDKEY property.
This means that the normal conversion of END-ERROR to ERROR is thus
modified in such blocks. In the EDITING case especially, it is likely
that an ERROR will be generated (because the EDITING block itself
counts as an interaction and thus with a larger interactions counter it
is more likely that an ERROR will be generated). What is especially
different here is that the ERROR will be propagated to the enclosing
block that has the ENDKEY property! So even if the DO or EDITING block
has the ERROR property in this case, if they also do not have the
ENDKEY property and the interactions counter (for the enclosing ENDKEY
block) is < 2 then the ERROR will NOT be processed by the current
block.
See above for ENDKEY and ERROR respectively.
|
Some language statements and all assignments that can generate the ERROR
condition can be executed using the NO-ERROR keyword, which disables the
generation of this condition. When a failure occurs, instead of generating
ERROR, the ERROR-STATUS system handle will have its state updated. Please
see Progress Programming Handbook page 5-6 and Progress Language Reference page
1337 for more details.
The possible actions in response to a condition:
Action
|
Description
|
UNDO
|
Rolls back all database,
variable, temp-table and work-table modifications since the beginning
of the target block. This target block must have opened a
transaction or it must be enclosed within a block that opened a
transaction AND must itself be a sub-transaction. In any other
case, the UNDO is a no operation.
There is no way of disabling UNDO. Any condition that is raised
will always trigger an UNDO (with the exception of the implicit
behavior for QUIT which is to COMMIT instead of UNDO). In
addition, one of the actions below will also be executed. If not
explicitly specified, the implicit action is always a RETRY.
|
LEAVE
|
Breaks (exits) out of the target block. If the target block is a top-level block, the LEAVE will be converted to a RETURN.
|
RETRY
|
The target block
(non-loop or loop) is executed from the top with the same state as
when it last executed.
Since a retry of a block in which the state never changes from
iteration to iteration would yield an infinite loop, Progress
provides a feature to detect when an infinite loop could
occur. This is called Infinite Loop Protection (ILP). RETRY will be converted to a LEAVE, NEXT or RETURN depending on the block type to which the RETRY is targetted.
|
NEXT
|
The next iteration of the loop
is executed. If the associated block is not a loop, then this
is automatically treated as a LEAVE or a RETURN (see below).
Infinite Loop Protection (ILP) protects the NEXT action (when triggered from an ON phrase or from an UNDO statement). In
certain cases, a NEXT will be converted to a LEAVE or RETURN by the infinite loop protection logic depending on the block type to which the RETRY is targetted.
|
RETURN
|
The current procedure,
function or trigger returns to the caller. There are 5 forms:
- Raise an ERROR condition in the caller ("RETURN
ERROR"). In a function, this can be specified but the
error is ignored (not raised).
- Return from a procedure or trigger block and allow
normal processing to continue ("RETURN"). If this executed
from a procedure, then the RETURN-VALUE global character
variable is set to the empty string "" which can be read in the
caller.
- Return the value of one of the basic data types when
returning from a function ("RETURN <expression>").
- Set the RETURN-VALUE global character variable which
can be read in the caller when returning from a procedure
("RETURN <character_expression>").
- Return and disable normal event processing when
returning from an event trigger block ("RETURN NO-APPLY").
This can also be specified in other block types, but the
NO-APPLY will be ignored.
|
In all actions (except RETURN), the block which is the target of the action can
be implicitly determined (if there is no label) OR it can be explicitly
specified using a user-defined label for the block.
One cannot UNDO a specified label which is outside of the scope referenced in
the associated action such as LEAVE or RETRY. In other words, the target
of the action will determine the flow of control for the program. Any
UNDO operation must be scoped inside (more deeply nested than) or equal to the block which is the target
for the other action otherwise a compiler error will result.
Blocks have "properties" that match a condition name. Thus there are
ERROR, ENDKEY, STOP and QUIT properties. The existence of a property
(implicitly or explicitly) in a given block means that that block provides a
predictable sequence of actions in response to the condition (matching that
property) being raised. To explicitly add a property to a block, one uses
the ON ERROR, ON ENDKEY, ON STOP or ON QUIT clauses in the block header.
For each of these explicit overrides, one may also specify "UNDO,
secondary_action". UNDO is required and cannot be avoided. The
secondary action is one of those
listed above. If no secondary action is specified, then RETRY is the action executed
by default.
The secondary action of an ON phrase can have an explicit label defining its
target block. If no label is specified BUT there is a label defined for
the UNDO portion of the ON phrase, then that UNDO label will also define the
target block for the secondary action. If neither portion of the ON phrase
defines a label, then the target block will be determined implicitly.
The logic for this is very complicated. Please
see below for details.
The implicit behavior of each block type is different in regards to condition
handling. Since all condition processing is handled based on whether the
condition has a matching "property" for a given block, if a block doesn't have a
property and that condition is raised, that block will be bypassed and the
processing will occur in an enclosing block. This means that if (for
example) a block has the ERROR property, then in the case where an ERROR
condition is raised, if that block is the innermost block which has the ERROR
property, then that is the level at which implicit UNDO and following implicit
action occurs. The implicit action that occurs is defined differently for
each condition,
see above
. This default action ONLY OCCURS if the block implicitly has the matching
property (e.g. the ENDKEY property when the ENDKEY condition is raised).
Please note that the UNDO action is special since it always occurs first (if it
occurs at all), before the secondary action. UNDO cannot be disabled
however it only occurs when a transaction is active in the current or a
containing scope. Some blocks always have a given properties (implicit or
explicit) and some blocks can only have explicit properties (i.e. DO). If
any property is not associated with a block, then no processing of that
condition will occur in that scope. This means that the stack unwinds past
that block without stopping to do any processing. Only those properties
that are implicitly or explicitly associated with a scope will ever be processed
in that scope.
Each property is independent. For example, if a block has an ERROR
property, that has no bearing on how it processes the ENDKEY.
The following are the
implicit properties that are inherently provided with each
block type:
(*) All of these entries have been found to be different than documented in the
Progress Handbook (see the Block Properties summary table on page 3-3).
The DO block is the only block with no implicit behavior (except in the case
where the TRANSACTION keyword or ON phrases are present). All DO, REPEAT and FOR blocks
can have the same set of explicit properties. If explicitly set, this
definition overrides the default action as
documented above.
The DO block will have sub-transaction support if they contain ANY
ON phrase. The condition does not matter. So an ON QUIT
UNDO, RETRY is enough to force the DO block to have the sub-transaction
property.
Please note that the function block is quite special. In user
defined functions, no user-input blocking statement can be executed.
This means that
PROMPT-FOR/SET/UPDATE/INSERT/CHOOSE/WAIT-FOR/PAUSE and READKEY are all
illegal, there
are very limited choices of editing database fields within a
function.
This combined with the fact that functions don't have the transaction
property
by default means that functions usually are sub-transactions
However,
when a direct assignment (e.g. field = expression) occurs, this WILL
cause the
function block to obtain TRANSACTION level. There is no way to
make a
function block a "NO" transaction (lacking the sub-transaction
or transaction properties). The ERROR and END conditions
both default
to LEAVE and the return value will be the unknown value in these
cases.
This cannot be overridden. Because of this behavior and the fact
that one
cannot specify an ON xxxx phrase with any of the 4 conditions, it is
only possible to get a RETRY at
the function block level by using the UNDO, RETRY. language
statement when infinite loop protection is disabled (e.g. with the
RETRY function).
Infinite Loop Protection (ILP)
Infinite loop protection is a hidden feature of the block processing
that is designed to modify the block behavior as specified by the 4GL
programmer in order to avoid an infinite loop. The idea is that
an action that is specified by a 4GL programmer (RETRY or NEXT) which
could perpetuate an infinite loop, will be silently converted (by the
runtime) to a different action (LEAVE, NEXT or RETURN) in order to
cause a change in the loop state that will avoid an infinite loop.
This silent conversion is done based on runtime state which
is affected by the flow of control of the program. So blocks
and statements that are conditionally executed will change the state of
the runtime and thus modify the conversions that occur. For this
reason, the behavior cannot be duplicated by static code modifications.
In particular, the following behavior is regulated:
- RETRY
- ON condition UNDO [label1] [, RETRY [label2]]
- this is an ON phrase for any of the 4 conditions, which has a secondary action of RETRY
- UNDO, RETRY.
- this is an UNDO language statement with a secondary action of RETRY
- ILP modifies RETRY in all block types.
- NEXT
- ON condition UNDO [label1] [, NEXT [label2]]
- this is an ON phrase for any of the 4 conditions, which has a secondary action of NEXT
- UNDO, NEXT.
- this is an UNDO language statement with a secondary action of NEXT
- ILP does NOT modify NEXT in blocks that are loops which
inherently change the program state in each iteration (DO TO, REPEAT
To, and all forms of FOR EACH). In such cases, the program's data
*generally* changes in each iteration (FOR EACH technically can be an
infinite loop if you edit the index being used to navigate the records,
but this is a rare use case). This means that a NEXT is
"safe" in such cases so ILP does not apply.
- The block to which the RETRY or NEXT is targetted (explicitly using a label or
implicitly calculated by the 4GL runtime/compiler) determines the kind
of conversion that occurs. See below for details on determining the target block for an action.
This conversion will occur unless ILP has been disabled for the
targetted block. ILP becomes disabled based on runtime state
which is maintained implicitly by particular language statements and by
the block processing infrastructure of the runtime.
If ILP is NOT disabled, the following conversions occur:
Block
Type
|
Top Level |
Block/Loop
|
Conversion
|
FOR EACH (all forms - even when there is a WHILE or TO/BY
present)
|
no |
loop
|
RETRY |
converted to NEXT |
NEXT |
no conversion, allowed to execute |
|
DO TO/BY
|
no |
loop
|
RETRY |
converted to NEXT |
NEXT |
no conversion, allowed to execute |
|
REPEAT TO/BY
|
no |
loop
|
RETRY |
converted to NEXT |
NEXT |
no conversion, allowed to execute |
|
FOR FIRST/LAST (all forms - even when there is a WHILE or
TO/BY present) |
no |
block
|
RETRY |
converted to LEAVE |
NEXT |
converted to LEAVE |
|
DO WHILE
|
no |
loop
|
RETRY |
converted to LEAVE |
NEXT |
converted to LEAVE |
|
REPEAT WHILE |
no |
loop
|
RETRY |
converted to LEAVE |
NEXT |
converted to LEAVE |
|
REPEAT
|
no |
loop
|
RETRY |
converted to LEAVE |
NEXT |
converted to LEAVE |
|
DO
|
no |
block
|
RETRY |
converted to LEAVE |
NEXT |
converted to LEAVE |
|
external procedure |
yes |
block |
RETRY |
converted to RETURN |
NEXT |
converted to RETURN |
|
internal procedure
|
yes |
block
|
RETRY |
converted to RETURN |
NEXT |
converted to RETURN |
|
function |
yes |
block |
RETRY |
converted to RETURN |
NEXT |
converted to RETURN |
|
trigger
|
yes |
block
|
RETRY |
converted to RETURN |
NEXT |
converted to RETURN |
|
editing
|
no |
loop
|
Infinite loop protection is always
disabled in editing blocks (since they an only be called from
UPDATE/SET/PROMPT-FOR as part of a user interaction), so no conversions
ever occur in targetted editing blocks.
|
There are 2 cases where this ILP is disabled:
- A language statement which blocks for user input is
executed before the point in the block where the condition is
raised. Please see Language
Statements that Block for User Input for the list. At the
moment that such a statement executes, the infinite loop
protection is disabled. Since editing blocks are
inherently processed as part of user input, infinite loop
protection is meaningless for such loops.
- The RETRY built-in function is executed before the
point in the block where the condition is raised. At the
moment that function executes, the infinite loop protection is
disabled.
Until one of these 2 cases is executed, any CONDITION that generates a
RETRY will result in a NEXT, LEAVE or RETURN. Only when the infinite
loop protection is disabled will RETRY result in a RETRY!
Likewise NEXT may be allowed, but depending on the block type it
may be converted to a LEAVE or RETURN.
All block types have retry support. In other words, it is
possible to retry any block including non-looping blocks and the
top-level blocks (like a built-in function or internal procedure).
But whether the RETRY will be allowed or not is determined by the
ILP processing at runtime.
It is important to note that the NEXT language statement is NOT
affected by ILP. ILP only affects ON phrases and the UNDO
language statement.
A note in the UNDO
language statement states that if "nothing changes during a RETRY of a
block" it will convert the RETRY into a NEXT or LEAVE. As far as
can be determined using application code, the 4GL runtime does not
actually monitor the state of variables or other resources to
detect if "nothing changes". It looks like the disabling of ILP
is taken as a proxy for whether things have changed in the current
iteration.
The
Progress READKEY documentation has a specific note regarding a special
implementation of infinite loop protection. It states that a
count is used to convert UNDO, RETRY into UNDO, NEXT as well as the
conversion of UNDO, NEXT into UNDO, LEAVE. While testing has not found
any special counter behavior for READKEY, there is a "stutter
quirk" (a counter based difference in ILP behavior) that has been found
associated with ON ENDKEY UNDO, RETRY when it is used on certain block
types.
The following code will cause a kind of "stuttering" where RETRY will
sometimes be allowed to occur due to the state of a counter. This
stutter quirk only comes into play when an ENDKEY condition is raised
and is handled by the "ON ENDKEY UNDO, RETRY" phrase. Consider
this code:
def var i as int no-undo.
do i = 1 to 5 on endkey undo, retry:
message i.
apply "endkey".
end.
The results:
1
2
3
3
4
4
5
5
The first 2 retries are converted to a NEXT but the 3rd RETRY is
allowed to go forward as a RETRY (no conversion occurs). Then the
4th RETRY is converted to a NEXT but the 5th RETRY is a RETRY.
This only occurs with the execution of the actions associated
with an ON phrase and ONLY with ENDKEY. If any other condition is
targetted (e.g. ERROR), this will not occur. Likewise, if any
other action is specified (NEXT instead of RETRY), then this behavior
does not occur.
Finally, this stutter mode is only seen in ON ENDKEY UNDO, RETRY phrases for the following block types:
- DO TO
- REPEAT TO
- FOR EACH (all forms)
Determining the Target and Meaning of UNDO, LEAVE, NEXT and RETRY
Each of these actions can explicitly reference a specific labeled block (e.g.
LEAVE inner)
or will implicitly reference a block.
In each of these cases, the action always operates in regards to a
specific block. In these implicit cases (one in which no block label
was specified in the 4GL source code), the 4GL compiler/runtime must
determine the block which is referenced. There are complex rules
by which the UNDO, LEAVE, NEXT and RETRY actions are
"bound" to a specific block as its target. These actions do not
always
operate on the current block or even on an explicitly defined block
target.
This target block resolution is complicated by the block
structure in which the on phrase or language statements are contained.
In particular, the target can reference enclosing
blocks implicitly or explicitly. The block properties of the
enclosing blocks will also affect any implicit target calculation.
There are 3 ways that these actions can be invoked:
- a language statement encountered in the normal flow of control (LEAVE [label] and NEXT [label] only)
- as an action defined in an UNDO language statement (UNDO [label1], [other_action [label2]].), encountered in the normal flow of control
- as an action defined in an ON phrase (ON <condition> UNDO [label1], [other_action [label2]]), in response to a condition being raised inside the block with the ON phrase
The UNDO language statement and ON phrases have the ability to specify
up to 2 labels (one for the UNDO and one for the other action).
There are only 4 types of blocks in Progress which can be labeled: DO,
REPEAT, FOR and EDITING. Thus, only these block types can be
explicitly
targetted
by the UNDO, LEAVE, NEXT or RETRY actions. But when implicit
block targets are calculated by Progress, it is possible for a
non-labeled block to be the target of an action. The non-labeled
block types are special "top-level" blocks (EXTERNAL PROC,
INTERNAL PROC, TRIGGER and USER DEFINED FUNCTION). None of those
non-labeled block types are iterating blocks. This means
that NEXT must be translated into something else (LEAVE in
Progress terms). In Progress, LEAVE on a top-level block is the
same as a RETURN (with no modifiers or returned data). So if
LEAVE or NEXT do implicitly target a top-level block, then the behavior
will logically be converted into a RETURN. RETRY on a top-level
block will be
converted to a RETURN if infinite loop protection is NOT
disabled.
All blocks in Progress
implicitly have the ERROR
property
except for the DO and EDITING (PROMPT-FOR or SET forms) blocks.
The DO only will have the ERROR property
when it has an ON ERROR phrase or if it has the TRANSACTION
keyword.
Note that the ERROR property that exists for DO blocks and for the UPDATE form
of EDITING blocks (and for the special "hidden" block in non-EDITING
UPDATE blocks) is special in that it is not in effect in the following case:
- The block does not also have the ENDKEY property (EDITING blocks
never have the ENDKEY property and DO blocks only have it if it is
explicitly specified in an ON ENDKEY phrase); AND
- The ERROR condition was raised by the END-ERROR key (or an APPLY of that key); AND
- The current interactions counter is less than 2 (see END-ERROR for details).
This is runtime state that will modify how the ERROR property is processed when the ERROR
condition is generated by the END-ERROR key. This means that an ERROR
condition generated by END-ERROR key in an UPDATE block (EDITING or
not) is always forwarded on to the containing block, even though all
other ERROR conditions are handled as an UNDO, RETRY.
Most importantly, for the purposes of resolving unlabeled actions to
blocks with the ERROR property, EDITING blocks (even UPDATE EDITING
blocks) are treated as if they DO NOT have the ERROR property.
Considering this, the following are the possible
implicit block target resolution algorithms:
Code |
Description |
NEAR-ERR |
The nearest enclosing block with the ERROR property.
Example 1
outer:
repeat:
inner:
do on error undo, retry:
<statement>
end.
end.
In example 1, the nearest enclosing block (to the <statement>) with the ERROR properly is inner.
Example 2:
outer:
repeat:
inner:
do:
<statement>
end.
end.
In example 2, the nearest enclosing block (to the <statement>) with the ERROR property is outer. |
CURRENT |
The "current" block. This is the block which directly
encloses the UNDO, NEXT or LEAVE language statement. Or in the
case of an ON phrase, it is the block upon which the ON phrase is
specified.
outer:
repeat:
inner:
do on error undo, RETRY:
<statement>
end.
end.
In this example, for both the <statement> and the RETRY the current block is inner. |
AS-UNDO |
The other action will be targetted at the same block as the
associated UNDO is targetted. If UNDO is targetted (implicitly or
explicitly) to inner then the other action will be targetted to inner. |
Assuming that the
directly containing block is labeled "inner" and that block is
contained inside a block labeled "outer", the following are possible formations of the UNDO language statement and ON phrases:
Feature |
UNDO Target |
Other Action |
Other Action Target |
Static Conversion Possible |
Compiler Error |
UNDO. |
NEAR-ERR |
RETRY (implicit) |
AS-UNDO |
|
n/a |
UNDO inner. |
inner |
RETRY (implicit) |
AS-UNDO |
|
If inner does not have the ERROR property. |
UNDO outer. |
outer |
RETRY (implicit) |
AS-UNDO |
|
If outer does not have the ERROR property. |
UNDO, RETRY. |
NEAR-ERR |
RETRY |
AS-UNDO |
|
|
UNDO, LEAVE. |
NEAR-ERR |
LEAVE |
AS-UNDO |
* |
|
UNDO, NEXT. |
NEAR-ERR |
NEXT |
AS-UNDO |
* |
|
UNDO, RETRY inner. |
NEAR-ERR |
RETRY |
inner |
|
If inner does not have the ERROR property, then UNDO would
choose a different block than inner. Since RETRY is targetted at
inner, this is a problem. UNDO and RETRY must always be targetted
at the same block. |
UNDO, LEAVE inner. |
NEAR-ERR |
LEAVE |
inner |
|
If inner does not have the ERROR property, then UNDO would
choose a
different block (a more enclosing one) than inner. Since LEAVE
must be targetted at a block that is the same as the UNDO target or
which encloses the UNDO target, this is
a problem. |
UNDO, NEXT inner. |
NEAR-ERR |
NEXT |
inner |
|
If inner does not have the ERROR property, then UNDO would choose a
different block (a more enclosing one) than inner. Since NEXT must be
targetted at a block that is the same as the UNDO target or which
encloses the UNDO target, this is
a problem.
In addition, if inner is not an iterating block the compiler fails. |
UNDO, RETRY outer. |
NEAR-ERR |
RETRY |
outer |
|
This is only possible if an inner
block (one that more directly encloses the statement) does not have the
ERROR property. If inner does have such properties, then it
would be implicitly chosen as the UNDO target and that would mismatch
with the outer block as the action target, causing the compiler to
error. |
UNDO, LEAVE outer. |
NEAR-ERR |
LEAVE |
outer |
|
The outer block or one of its nested blocks (which still
encloses this statement) must have the ERROR property, otherwise the
LEAVE will be targetted inside the UNDO target, which is not allowed. |
UNDO, NEXT outer. |
NEAR-ERR |
NEXT |
outer |
|
The outer block or one of its nested blocks (which still encloses this
statement) must have the ERROR property, otherwise the NEXT will be
targetted inside the UNDO target, which is not allowed.
In addition, if outer is not an iterating block the compiler fails. |
UNDO inner, RETRY. |
inner |
RETRY |
AS-UNDO |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO inner, LEAVE. |
inner |
LEAVE |
AS-UNDO |
* |
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO inner, NEXT. |
inner |
NEXT |
AS-UNDO |
* |
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO inner, RETRY inner. |
inner |
RETRY |
inner |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO inner, LEAVE inner. |
inner |
LEAVE |
inner |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO inner, NEXT inner. |
inner |
NEXT |
inner |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO inner, RETRY outer. |
n/a |
n/a |
n/a |
|
Invalid syntax, compiler will always error because RETRY and UNDO must target the same block. |
UNDO inner, LEAVE outer. |
inner |
LEAVE |
outer |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. (outer has no property dependencies) |
UNDO inner, NEXT outer. |
inner |
NEXT |
outer |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. (outer has no property dependencies)
In addition, if outer is not an iterating block the compiler fails. |
UNDO outer, RETRY. |
outer |
RETRY |
AS-UNDO |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO outer, LEAVE. |
outer |
LEAVE |
AS-UNDO |
* |
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO outer, NEXT. |
outer |
NEXT |
AS-UNDO |
* |
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO outer, RETRY inner. |
n/a |
n/a |
n/a |
|
Invalid syntax, compiler will always error because RETRY and UNDO must target the same block. |
UNDO outer, LEAVE inner. |
n/a |
n/a |
n/a |
|
Invalid syntax, compiler will always error because LEAVE must target the same or a more enclosing block as UNDO. |
UNDO outer, NEXT inner. |
n/a |
n/a |
n/a |
|
Invalid syntax, compiler will always error because NEXT must target the same or a more enclosing block as UNDO. |
UNDO outer, RETRY outer. |
outer |
RETRY |
outer |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO outer, LEAVE outer. |
outer |
LEAVE |
outer |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
UNDO outer, NEXT outer. |
outer |
NEXT |
outer |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error.
In addition, if outer is not an iterating block the compiler fails. |
ON <condition> UNDO. |
CURRENT |
RETRY (implicit) |
AS-UNDO |
|
CURRENT must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO inner. |
inner |
RETRY (implicit) |
AS-UNDO |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO outer. |
outer |
RETRY (implicit) |
AS-UNDO |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO, RETRY. |
CURRENT |
RETRY |
AS-UNDO |
|
|
ON <condition> UNDO, LEAVE. |
CURRENT |
LEAVE |
AS-UNDO |
* |
|
ON <condition> UNDO, NEXT. |
CURRENT |
NEXT |
AS-UNDO |
* |
No error even if the implicit target block is not an iterating block. |
ON <condition> UNDO, RETRY inner. |
CURRENT |
RETRY |
inner |
|
CURRENT must be the same block as inner (RETRY and UNDO must always be at the targetted to the same block). |
ON <condition> UNDO, LEAVE inner. |
CURRENT |
LEAVE |
inner |
|
CURRENT must be the same block as inner or enclosed by inner. The LEAVE target block needs no properties in particular. |
ON <condition> UNDO, NEXT inner. |
CURRENT |
NEXT |
inner |
|
CURRENT must be the same block as inner or enclosed by inner. The NEXT target block needs no properties in particular.
However, if inner is not an iterating block the compiler fails. |
ON <condition> UNDO, RETRY outer. |
n/a |
n/a |
n/a |
|
Invalid syntax. The UNDO will target the current block which
is more deeply nested than the RETRY target, which causes the compiler
to fail.
|
ON <condition> UNDO, LEAVE outer. |
CURRENT |
LEAVE |
outer |
|
The LEAVE target block needs no properties in particular. |
ON <condition> UNDO, NEXT outer. |
CURRENT |
NEXT |
outer |
|
The NEXT target block needs no properties in particular.
If outer is not an iterating block the compiler fails. |
ON <condition> UNDO inner, RETRY. |
inner |
RETRY |
AS-UNDO |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO inner, LEAVE. |
inner |
LEAVE |
AS-UNDO |
* |
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO inner, NEXT. |
inner |
NEXT |
AS-UNDO |
* |
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO inner, RETRY inner. |
inner |
RETRY |
inner |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO inner, LEAVE inner. |
inner |
LEAVE |
inner |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO inner, NEXT inner. |
inner |
NEXT |
inner |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error.
If inner is not an iterating block the compiler fails. |
ON <condition> UNDO inner, RETRY outer. |
n/a |
n/a |
n/a |
|
Invalid syntax. The UNDO will target the inner block which is more
deeply nested than the RETRY target, which causes the compiler to fail. |
ON <condition> UNDO inner, LEAVE outer. |
inner |
LEAVE |
outer |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO inner, NEXT outer. |
inner |
NEXT |
outer |
|
inner must have ANY property (not just ERROR) otherwise there is a compiler error.
If outer is not an iterating block the compiler fails. |
ON <condition> UNDO outer, RETRY. |
outer |
RETRY |
AS-UNDO |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO outer, LEAVE. |
outer |
LEAVE |
AS-UNDO |
* |
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO outer, NEXT. |
outer |
NEXT |
AS-UNDO |
* |
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO outer, RETRY inner. |
n/a |
n/a |
n/a |
|
Invalid syntax. The UNDO and RETRY must target the same block, otherwise the compiler fails. |
ON <condition> UNDO outer, LEAVE inner. |
n/a |
n/a |
n/a |
|
Invalid syntax. LEAVE must target the same or a more enclosing block as UNDO is targetting. |
ON <condition> UNDO outer, NEXT inner. |
n/a |
n/a |
n/a |
|
Invalid syntax. NEXT must target the same or a more enclosing block as UNDO is targetting. |
ON <condition> UNDO outer, RETRY outer. |
outer |
RETRY |
outer |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO outer, LEAVE outer. |
outer |
LEAVE |
outer |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error. |
ON <condition> UNDO outer, NEXT outer. |
outer |
NEXT |
outer |
|
outer must have ANY property (not just ERROR) otherwise there is a compiler error.
If outer is not an iterating block the compiler fails. |
In the above table <condition> can be ERROR, ENDKEY, STOP or
QUIT. The particular condition in question does not matter for
the purposes of the behavior documented above.
The "Static Conversion Possible" column denotes the difference between
runtime conversion of actions (infinite loop protection) versus static
(compile time) conversion of actions. A static conversion is one
that will not change based on runtime state, but instead is based on
data known at the time the source code is compiled.
At runtime, a RETRY action will be
converted to a NEXT, LEAVE or RETURN (depending on target block type) if infinite loop protection is NOT
disabled.
If the action is NEXT and was caused by an ON phrase or UNDO
statement AND the target block is a non-iterating inner block (it is
not a top-level block), then the NEXT will be converted to a LEAVE.
If the action is NEXT or LEAVE and the target block matched is a
top level block (external procedure, internal procedure, function or
trigger), then the action will be converted to a RETURN.
Rules for LEAVE statement processing:
- If there is an explicit label, that block is left no matter what properties it may have or not have.
- If there is no label, then the implicit target block must be
found. The nearest enclosing block type of the following will be
the target:
- DO WHILE
- DO TO
- DO TO/WHILE
- REPEAT (all forms)
- FOR EACH (all forms)
- FOR FIRST (all forms)
- FOR LAST (all forms)
- EDITING
- Please note that the simple DO block is the only inner block type that LEAVE will NEVER implicitly target.
- The properties of the blocks have nothing to do with this search process.
- If no inner block is found with the above process, then the LEAVE
is considered to be targetting the nearest top-level block. For
this reason, the LEAVE statically converts to a RETURN.
Rules for NEXT statement processing:
- If there is an explicit label:
- If the block is an iterating block, then the next iteration of the block is invoked.
- If the block is non-iterating:
- If the block is a FOR FIRST/FOR LAST, then the NEXT is statically converted to a LEAVE.
- If the block is a DO, the compiler errors.
- If
there is no label, then the implicit target block must be found. This is a 2 phase process.
- The
nearest enclosing looping block type will be the target:
- DO WHILE
- DO TO
- DO TO/WHILE
- REPEAT (all forms)
- FOR EACH (all forms)
- EDITING
- If one of these blocks is found, then the NEXT iteration of that block will be invoked.
- If there is no enclosing looping block between the NEXT
statement and the nearest enclosing top-level block, then a search will
be made to find the nearest enclosing FOR FIRST or FOR LAST block.
If one of these blocks is found, then the NEXTwill be statically
converted to a LEAVE and targetted at that nearest enclosing FOR
FIRST/LAST block.
- The 2nd phase of the implicit block search process is not
attempted unless the first phase fails to find a match. This
means that a more deeply nested FOR FIRST/LAST block will NOT have
preference to an outer looping block. The looping blocks will
always take precedence over the FOR FIRST/LAST.
- Please note that the simple DO block is the only inner block type that NEXT will NEVER implicitly target.
- The properties of the blocks have nothing to do with this search process.
- If
no inner block is found with the above 2 phase process, then the NEXT is
considered to be targetting the nearest top-level block. For this
reason, the NEXT converts to a RETURN.
Transactions
Progress 4GL transaction processing:
- Defines the start and end of transactions and sub-transactions.
- Implements the proper nesting of sub-transactions matching the
block structuring of the program.
- Rolls back (UNDOes) database, variable, temp-table and work-table
values to a known state upon certain "conditions" occurring.
- Commits all changes (permanently) to databases, variables,
temp-tables and work-tables at the natural successful end of an active
transaction.
- Implements scoping facilities for reou
A transaction begins at the start of any block while the following are true:
- No transaction is currently active.
- The block:
- Has the TRANSACTION keyword which explicitly enables
transaction support.
- The following blocks qualify:
- FOR EACH TRANSACTION
- REPEAT TRANSACTION
- DO TRANSACTION
- Directly contains a language statement that implicitly enables
transaction support.
- The following blocks qualify:
- DO ON ENDKEY
- DO ON ERROR
- FOR EACH
- REPEAT
- procedure
- In particular, at least one of the Language
Statements that Update/Modify the Database or one of the Language
Statements that Read the Database with EXCLUSIVE LOCK must be used
within this block.
- To be a "direct" use, the language statement must not be
contained in a nested block unless that nested block is of a type
that cannot be a sub-transaction. For example, if a simple DO
block (without the ON ENDKEY, ON ERROR or TRANSACTION keywords)
contains one of the above language statements, then this would
define the start of a transaction in the nearest containing block
that can be a transaction. A simple DO block cannot define a
transaction by itself.
A sub-transaction (a nested transaction) begins at the start of any block while
the following are true:
- A transaction is currently active.
- The block:
- Is any form of the following:
- FOR EACH
- REPEAT
- procedure
- Is a form which explicitly enables transaction support:
- Directly contains a language statement that implicitly enables
transaction support.
- The following blocks qualify:
- In particular, at least one of the Language
Statements that Update/Modify the Database or one of the Language
Statements that Read the Database with EXCLUSIVE LOCK must be used
within this block.
- To be a "direct" use, the language statement must not be
contained in a nested block unless that nested block is of a type
that cannot be a sub-transaction. For example, if a simple DO
block (without the ON ENDKEY, ON ERROR or TRANSACTION keywords)
contains one of the above language statements, then this would
define the start of a transaction in the nearest containing block
that can be a transaction. A simple DO block cannot define a
transaction by itself.
Only 1 transaction can ever be active per user context. Other nested
blocks that have transaction properties, become nested sub-transactions.
For more details on which blocks can open a transaction, see page 12-17 of the
Progress Programming Handbook.
Any block that has no direct access to the database cannot open a transaction
(unless it is a block that has the TRANSACTION keyword). Thus the same
block type which would normally open a transaction when database updates or
exclusive database reads are present, will not open a transaction (or in some
cases a sub-transaction) when no database access is contained.
FOR block transaction anomaly:
There is a case where reading a record in a FOR EACH with exclusive lock will
not create a transaction. In particular, if the buffer scope for the FOR
EACH has been expanded beyond the FOR EACH block (there is a free reference
outside that block) AND there is no data editing or other cause to create a
transaction, then the FOR EACH will not create a transaction. Also note
that ALL buffers referenced (in the case where there are > 1 record phrases)
must be scoped outside the FOR block in order for this anomaly to trigger.
If even 1 buffer (referenced in the FOR statement itself) is scoped to
the FOR block itself, then the EXCLUSIVE-LOCK alone is enough to trigger this
behavior. When this case occurs, the FOR block is still treated as a
sub-transaction. See Progress Knowledge Base article KB13974.
Note that the rules defining a sub-transaction are a
super-set
of the rules defining a transaction. Since a given block can be invoked
with a call stack that is different depending on the path taken through the
program, when a particular code block is invoked it is possible that sometimes a
transaction will be active and other times not. This means that the
following cases are possible:
- The start of the block will start either a transaction or a
sub-transaction, depending only on whether a transaction is already active.
- The start of the block will start a sub-transaction if a
transaction is already active, but otherwise a transaction will not be
started.
- The start of the block has no transaction or sub-transaction
implications (notably any DO block without the ON ENDKEY, ON ERROR or
TRANSACTION keywords).
To duplicate this behavior, the determination of whether a block opens a
transaction or a sub-transaction must be handled at runtime. Thus the same
block will process differently depending on runtime conditions.
In this way, procedures are treated just like other block types. Calls to
procedures within an active transaction cause the procedure to open a
sub-transaction. If a procedure would not otherwise open a transaction and
a transaction is not active, then a transaction is not started at the beginning
of a procedure. It is important to note that even though a procedure may
not open a transaction, a nested block may still open a transaction. For
example, a nested FOR EACH that updates the database will have a transaction
scoped to the FOR EACH block rather than to the containing procedure.
The end of any block that defines a transaction or sub-transaction defines the
end of the corresponding transaction/subtransaction. When a transaction
scope ends successfully, all changes made during that scope are committed to the
database. When a sub-transaction scope ends successfully, all changes made
during that scope are effectively added to the "transaction set". This
transaction set can be UNDOne as a set or committed as a set but once the end of
a subtransaction occurs, one loses the ability to UNDO only that portion of the
transaction.
In the event of any condition being raised, an UNDO (rollback) will occur (at
either the transaction or sub-transaction level) and then a change of control
flow to an enclosing scope will occur (to the same scope as the UNDO or to a
scope that encloses the scope that was UNDOne. The implicit or explicit
definition of this behavior is encoded in the source application's block
properties. Each condition has a different set of default behavior in this
regard. For more details on determining the start and end of transactions,
see page 12-8 of the Progress Programming Handbook
This ability to have nested sub-transactions allows blocks to be used to
structure Progress 4GL programs such that the proper level of UNDO and other
actions can be associated with the granularity or lack of granularity which the
programmer desires. This allows any active work (work that has not been
added to the transaction set (defined above) to be rolled back partially (by
UNDOing at the sub-transaction level) or for all work (the full transaction set)
to be backed out (by UNDOing at the transaction level).
The sub-transaction commit point has the other behavior (in the case of a loop),
that the "backup set" of all undoable resources (variables, database fields...)
is updated at this same point. This means that if a variable is defined
external to a scope and the first iteration of a given sub-transaction loop
updates its value and then commits, any this new value will be result of any
UNDO that occurs (whether the variable was modified or not) until the next
sub-transaction commit. This can be described as updating the backup set
at iteration.
Once a transaction is active, database fields, variables, temp-table and
work-table values are saved off at the start of the block defining the
transaction and again at the start of the block defining every
sub-transaction. One can UNDO all of these data structures to a known
state by defining the block properties of each block such that the possible
conditions implicitly UNDO the correct block OR the possible conditions are
explicitly defined to UNDO a specific block. In particular, labeled blocks
can be specified as the target of an UNDO (or other action) by using the "ON
<condition>" clause in a block. If that named condition occurs, then
the specified actions would be invoked on the specified blocks.
UNDO can ONLY EVER be done when a transaction is active. The scope of the
rollback is either explicitly defined by the programmer or is implicit, but in
either case, each condition type matches up with a specific (and potentially
different) rollback scope. A rollback scope can be specified at any block
that defines a transaction or sub-transaction. When a transaction is not
active, UNDO is a null operation.
There is no way to disable UNDO for a block, although individual local variables
and temp-table/work-table fields can be specified as NO-UNDO, which will exclude
them from the transaction rollback. Database field changes are always
rolled back.
Whether a block is a loop or not does not change how transaction processing
works. In fact, even a non-looping block can be retried.
Once the successful end of a transaction block is encountered, Progress commits
all changes (permanently) to databases, variables, temp-tables and
work-tables. At the successful end of a sub-transaction, backup copies of
the database fields (buffers), variables, temp-tables and work-tables are
discarded. A commit is a write of the modified value(s) back to the
database or memory as necessary for the resource type being processed. In
case of the database, temp-tables and work-tables, this includes:
- execution of any validation rules
- execution of an insert (for new records) or an update (current
records) or a delete (current records)
- release of held locks
When the natural flow of control of the procedure flows through an END statement
for a block, this is considered the "successful end" of the block. Please
note that when LEAVE and RETURN are explicitly coded (and not triggered as a
result of a CONDITION) then these are also considered the successful end of the
containing block. This means that all changes are committed in these
cases.
Silent Error Mode (NO-ERROR keyword)
Progress provides the NO-ERROR keyword and a system handle called ERROR-STATUS
to provide the ability to temporarily (for the duration of a single assignment
or language statement) force a silent error mode. In this mode, any
generated errors will not raise a condition but will instead store the error
number and error text in a list of errors. Then the statement or
assignment will silently return.
The ERROR-STATUS handle is used to access this stored data.
Language Statements Which Can Raise Conditions
Statement
|
Condition
|
Notes
|
ERROR
|
STOP
|
QUIT
|
ENDKEY
|
END-ERROR*
|
APPLY |
+
|
+
|
+
|
+
|
+
|
this statement can generate
any event on a procedure including these special events that match
"conditions" (it can also generate events for widgets but this is
conceptually quite different from generating the conditions listed
in this table)
|
ASSIGN |
+
|
-
|
-
|
-
|
-
|
|
BUFFER-COMPARE
|
+
|
-
|
-
|
-
|
-
|
|
BUFFER-COPY
|
+
|
-
|
-
|
-
|
-
|
|
CHOOSE |
+
|
+
|
-
|
+
|
+
|
interactive
|
COMPILE
|
+
|
-
|
-
|
-
|
-
|
provided here for completeness
|
CONNECT
|
+
|
-
|
-
|
-
|
-
|
|
COPY-LOB
|
+
|
-
|
-
|
-
|
-
|
|
CREATE
ALIAS
automation object
CALL
DATABASE
SAX-READER
SERVER-SOCKET
SOCKET
WIDGET-POOL
|
+
|
-
|
-
|
-
|
-
|
|
DDE
|
+
|
-
|
-
|
-
|
-
|
all kinds of DDE statement can
raise ERROR
|
DELETE
OBJECT
PROCEDURE
WIDGET-POOL
|
+
|
-
|
-
|
-
|
-
|
|
DICTIONARY
|
+
|
-
|
-
|
-
|
-
|
"The DICTIONARY statement is
equivalent to RUN dict.p: it runs the Progress procedure called
dict.p. Progress uses the regular search rules to find the
dictionary procedure. The dictionary procedure is part of the
Progress system software."
|
DISCONNECT
|
+
|
-
|
-
|
-
|
-
|
|
DISPLAY |
+
|
-
|
-
|
-
|
-
|
|
EDITING phrase |
+
|
+
|
+
|
+
|
+
|
The event processing inside
the phrase must apply all events explicitly, either using APPLY
statement or by using RETURN ERROR/STOP/QUIT statements. Also, this
phrase is active only if input is from terminal. In general case it
is necessary to check phrase body to determine which events can be
generated.
|
EMPTY TEMP-TABLE |
+
|
-
|
-
|
-
|
-
|
|
ENTRY
|
+
|
-
|
-
|
-
|
-
|
function and statement
|
EXPORT
|
+
|
-
|
-
|
-
|
-
|
|
FIND |
+
|
-
|
-
|
+
|
-
|
|
IMPORT |
+
|
-
|
-
|
+
|
-
|
|
INPUT FROM |
+
|
-
|
-
|
+
|
+
|
|
INSERT |
+
|
+
|
-
|
+
|
+
|
interactive |
LOAD
|
+
|
-
|
-
|
-
|
-
|
|
OUTPUT TO |
+
|
-
|
-
|
-
|
-
|
|
PROCESS EVENTS |
+
|
+
|
-
|
+
|
+
|
behaves like interactive
|
PROMPT-FOR |
+
|
+
|
-
|
+
|
+
|
interactive (equivalent to
ENABLE, WAIT-FOR, DISABLE sequence of statements)
|
PUT-KEY-VALUE
|
+
|
-
|
-
|
-
|
-
|
|
QUIT
|
-
|
-
|
+
|
-
|
-
|
this statement seems the only
way to generate this event (except APPLY)
|
RAW-TRANSFER
|
+
|
-
|
-
|
-
|
-
|
|
READKEY
|
+
|
-
|
-
|
+
|
-
|
Actually, the condition is not
directly raised, instead the
LASTKEY value is set to -1 (on ERROR) or -2 (on ENDKEY). To
raise the condition one then must "APPLY LASTKEY".
|
RELEASE
|
+
|
-
|
-
|
-
|
-
|
all kinds of RELEASE statement
can raise ERROR |
REPOSITION
|
+
|
-
|
-
|
-
|
-
|
|
RUN
|
+
|
+
|
-
|
|
|
all kinds of RUN statement can
raise ERROR and STOP
|
SAVE CACHE |
+
|
-
|
-
|
-
|
-
|
|
SET |
+
|
+
|
-
|
+
|
+
|
interactive (equivalent to
PROMPT-FOR and ASSIGN)
|
STOP
|
-
|
+
|
-
|
-
|
-
|
|
SUBSCRIBE
|
+
|
-
|
-
|
-
|
-
|
|
UNLOAD
|
+
|
-
|
-
|
-
|
-
|
|
UPDATE |
+
|
+
|
-
|
+
|
+
|
interactive
|
USE
|
+
|
-
|
-
|
-
|
-
|
|
VALIDATE
|
+
|
-
|
-
|
-
|
-
|
|
WAIT-FOR |
+
|
+
|
-
|
+
|
+
|
Waits for specified event to
occur. Effectively this does allow other keyboard-driven events to
occur as well.
|
Notes:
- END-ERROR is a key function that behaves as
two separate events (see 5-19 in the Progress Programming Handbook): for the
first input operation (any of the Language Statements
that Block for User Input) of the current block's iteration (if it is a
loop), raises the ENDKEY condition; for subsequent input operations, raise
the ERROR condition. Note that in a WAIT-FOR, the resulting condition
is always ENDKEY. In UPDATE/SET/PROMPT-FOR... (and for non-WAIT-FOR
stmts), it differs based on whether this is the first or a subsequent
interaction in this *iteration* of the block. For example, in a
REPEAT block that has 2 SET statements, on any given iteration of the block
if you press the END-ERROR key while in the 1st SET statement the result is
always ENDKEY condition and if you press the END-ERROR key in the 2nd SET
statement, the result is the ERROR condition.
- Inner loops may have ON ERROR, ON ENDKEY, etc. phrases which refer
to outer loop using appropriate labels.
- The default behavior if the ON ENDKEY phrase is omitted is UNDO,
LEAVE. If ON ENDKEY phrase is partially specified then default is to LEAVE
if other (NEXT, RETRY, or RETURN) action is not explicitly set.
- RETURN ERROR raises ERROR condition in calling procedure.
- The default behavior if the ON ERROR phase is omitted for REPEAT
and FOR EACH blocks - UNDO, RETRY. If Progress detects possible infinite
loop in FOR or iterating DO blocks then UNDO, NEXT is performed instead.
- It is unclear if UNSUBSCRIBE supports NO-ERROR option. Syntax
diagram does not mention it but further description mentions it as a
possible way to avoid raising ERROR.
- From the description it is not clear if the DICTIONARY is an
equivalent to "RUN dict.p" or "RUN dict.p NO-ERROR".
- Quote from the reference: If you use READKEY, it intercepts any
input from the user. Thus no widgets receive input. To pass the input to a
widget, you must use the APPLY statement. READKEY and widget usage are usually
kept separate. READKEY is normally used as part of the editing block
functionality while widget events are part of the wait-for event driven
functionality. As noted above with the APPLY statement it is important
to realize that widgets and events are a very different category of
discussion than these block conditions.
- UNDO statement does not raise conditions by itself but may change
execution flow if followed by LEAVE, NEXT, RETRY or RETURN. Also, in form
UNDO, RETURN ERROR is may raise ERROR condition in the calling procedure
(like RETURN ERROR).
- The default ENDKEY and ERROR processing is defined only for REPEAT
and FOR loops, other loops must specify such processing explicitly using ON
ERROR and ON ENDKEY phrases.
Language
Statements that Block for User Input
The presence of language statements that block for user input modifies or
controls the behavior of the END-ERROR key and of the infinite loop protection (ILP)
for RETRY.
The following language statements do block for user input:
- CHOOSE
- ENABLE
- INSERT
- MESSAGE (only with VIEW-AS ALERT-BOX, UPDATE or SET)
- PROMPT-FOR
- READKEY on terminal (but not when reading from a file)
- SET
- UPDATE
- WAIT-FOR
- PAUSE (waits for "ANY-KEY") - Note that this statement is a
special case in that it blocks for user input, but only sometimes does it disable infinite loop
protection. For examples of this behavior, see testcases
pause_infinite_loop_protection*.p
.
There is a special case for the PAUSE statement. Whether PAUSE does or does not
disable infinite loop protection, is dependent upon the block type:
- PAUSE disables ILP (behaves just like the RETRY built-in function or one of the other interactive statements above)
- In these blocks (these blocks *can* be infinite loops):
- REPEAT
- REPEAT WHILE
- DO WHILE
- For these cases:
- ON <condition> UNDO, RETRY (the same behavior occurs no matter which condition is used)
- ON <condition> UNDO, NEXT (the same behavior occurs no matter which condition is used)
- UNDO, RETRY.
- UNDO, NEXT.
- PAUSE is ignored for purposes of ILP
- In these blocks (all of these blocks *generally* aren't infinite loops):
- DO
- DO TO
- REPEAT TO
- FOR EACH (all forms)
- FOR FIRST (all forms)
- FOR LAST (all forms)
- For these cases:
- ON <condition> UNDO, RETRY
- ERROR, STOP and QUIT all behave this way (PAUSE is ignored)
- ENDKEY is special, it DOES HONOR PAUSE by causing ILP to be disabled!
- UNDO, RETRY.
- Note that NEXT cases don't have ILP protection in the above blocks so ILP only affects RETRY in these cases.
- The source of the PAUSE does not matter. In other words, a
PAUSE will have the same effect or lack of effect no matter whether it
is an explicit PAUSE statement executed by the 4GL source code OR it is
an implicit PAUSE caused by the state of the user interface. The
block behavior is affected equally by both types of PAUSE.
Perhaps surprising is that the following
do not block
for user-input:
- APPLY
- HIDE
- PROCESS-EVENTS
- VIEW
Language Statements
that Update/Modify the Database
Absent the TRANSACTION keyword, only the presence of database updates or
exclusive reads will cause a block to start a transaction. The following
language statements start a transaction, only if they reference a database
table/field (after a record buffer has been created or filled). If
temp-table, work-table or normal variables are used, then a transaction is
not
started. In addition, some of these statements can take a record as an
option (as a shorthand way of referencing all fields in that record). If
this record is a TABLE, this will trigger a transaction.
- ASSIGN
- BUFFER-COPY
- CREATE
- DELETE
- INSERT
- RAW-TRANSFER
- SET
- UPDATE
- Explicit field-level use of the assignment operator on database
records will start a transaction.
Note that the assignment operator can only be used without error on database
records that have SHARE-LOCK or EXCLUSIVE-LOCK. If the record is opened
with NO-LOCK, then an ERROR will be generated. The NO-ERROR keyword can be
used on an assignment to suppress the error generation (the fact that it
occurred can still be discovered via the system handle ERROR-STATUS) but this
doesn't make the update successful. In other words, one cannot
successfully assign to a database record that is NO-LOCK but this still DOES
mark the associated block as a transaction.
Usage of a BROWSE widget (see DEFINE BROWSE) is a "black-box" language
statement which reads from/writes to the database on behalf of the
programmer. Any BROWSE that uses the ENABLE keyword is not read-only (in
Progress terms it is "updateable") and it will update/modify the database. All
transaction support for updateable BROWSE is hidden inside the
"black-box".
For this reason, the transaction
property does not attach to the containing block.
It is not known at this time if statements such as SETUSERID implicitly access
the database on the caller's behalf. There may be other such language
statements that read or update the database explicitly.
Language Statements
that Read the Database with EXCLUSIVE LOCK
Absent the TRANSACTION keyword, only the presence of database updates or
exclusive reads will cause a block to start a transaction. The following
language statements read from the database into a record buffer. A lock
can be exclusive explicitly with the EXCLUSIVE-LOCK keyword.
- FIND
- GET (with an explicit EXCLUSIVE-LOCK OR with no lock qualifiers
when referencing a query with EXCLUSIVE-LOCK)
- FOR (all forms - EACH, FIRST, LAST and unqualified record phrase)
If temp-table or work-table records are read, then a transaction is
not
started. Only exclusive database reads cause a transaction.
Usage of a BROWSE widget (see DEFINE BROWSE) is a "black-box" language
statement which reads from/writes to the database on behalf of the
programmer. BROWSE widgets are read-only when there is no use of the
ENABLE keyword (in Progress terms it is not "updateable") and these can generate
reads with EXCLUSIVE-LOCK (this can be set in the DEFINE BROWSE
statement). If the DEFINE BROWSE does not explicitly use the
EXCLUSIVE-LOCK keyword, then even if the query was opened EXCLUSIVE-LOCK this is
overridden to NO-LOCK (by default) or SHARE-LOCK (if specified). So only a
browse defined explicitly with EXCLUSIVE-LOCK will open the query as
EXCLUSIVE-LOCK. However, since such an approach is only allowed inside a
block which already has transaction support (it must be placed inside a DO
TRANSACTION or REPEAT TRANSACTION), the containing block will already have
transaction support. For this reason, browse usage can be ignored.
The REPOSITION statement fetches a record when the query is associated with a
BROWSE widget. Since the transaction processing associated with a browse
in hidden inside the browse's "black-box", so also is any effect of the
hidden/implicit record reading that occurs with this statement. In other
words, although this statement can have an implicit record read associated,
there is no case where it causes the containing block to become a transaction.
It is not known at this time if
statements such as SETUSERID implicitly access the database on the caller's
behalf. There may be other such language statements that read or update
the database explicitly.
It is likely that such statements handle transactions internally (as a
black-blox rather than associating a transaction with the calling block).
High Level Implementation Options
Many of the features that Progress provides relate to the proper flow of control
in the resulting application. While problems such as data processing
fidelity were relatively easy to centralize or hide inside the
wrapper classes
, features related to flow of control are not easily centralized.
This is due to the fact that the all control flow constructs in Java cannot be
split across two or more methods. For example, one cannot centrally define
catch () blocks that are then used in multiple places. There must be a
catch block in every method in which the matching logic must be triggered.
While one can take the user-defined logic and move that into methods that are
called from many locations, all flow control constructs are built-in language
features and they are essentially monolithic in the Java language.
The Progress feature set is a complex one with much implicit behavior that must
be duplicated. In many cases, this means that there will be more code in
Java where no such code appears in the Progress source. This has support,
maintenance and conversion implications (the greater the difference in source
and target tree structure, the greater the effort to convert). For this
reason, one must consider how to obtain complete fidelity while centralizing and
hiding as much logic as possible in utility classes, leaving the generated
source cleaner and more natural to maintain/support.
The following are the most feasible implementation options to provide complete
fidelity with the Progress language/runtime feature set:
Option
|
Description
|
Pros
|
Cons
|
explicit hard coding in
generated source
|
All control flow, logic and
data processing is explicit in the generated source code. Some
features can be helped by hiding things like undo processing in the
data type wrappers or utility classes like streams (with cleanup
processing). However, the core problem here is that the
generated code still must hook into this processing. This will
take the form of try/catch/finally blocks, instrumentation calls to
push/pop scopes, manufactured labels/bogus outer loops to handle
things like retry and call-backs to utility classes to access or
invoke that logic that can be centralized.
|
- Simple to understand and implement.
- Easiest to debug.
- Good performance (considering what has to be done).
- Most independent of external tools.
|
- Source code is most verbose of all options.
- Since most code is affected by the implicit features,
this extra code will be in many/most locations.
- More effort later to convert any generated code to a
more native Java approach.
|
indirect call mechanism using
a utility class to implement the logic |
In this model, each block that
has special properties would be implemented in 2 pieces:
- An "external" interface that exposes a well-known named
method and interface. This method would use reflection to
obtain a Method object, it would package its parameters as an
Object[] and invoke a special BlockProperties class that in turn
would use the Method and Object[] to indirectly invoke the
private worker. If the return was not void, the Object
would be cast to the proper type and then returned to the real
caller.
- A private worker method that actually contains the
converted logic. This would be invoked via reflection from
the external interface method.
The point here is that the block properties would not be emitted
into the generated source, but would be centralized in the
BlockProperties class. Since the block properties class is
just invoking the private worker as a method call, this method call
can be inside a hand coded try/catch/finally with all special
Progress-compatible logic completely hidden in this central
location.
|
- Strong level of centralization.
- Source code is reasonably clean (but not completely
clean) compared to the hard coded approach above.
|
- External interface methods have to be created for each
public interface, for each internal procedure or function and
for each block. Thus the source code will still look
different from the original and from how one would want to do it
if manually coding.
- Harder to debug than the hard coded approach above.
- DO/REPEAT/FOR blocks would have to be refactored into
methods which makes this extremely complicated (since all data
must be either promoted to instance members, passed as
parameters or encapsulated into another class and passed
down. This is made harder because anything nested inside
eachblock has to have its data-access requirements "rolled up"
into the containing block.
- Using reflection is a performance penalty, especially
in the Mehtod object lookup (although the resulting Method could
be cached in a static member).
|
bytecode instrumentation |
Generate the Java code just as
one would normally write it by hand coding but after javac creates
the resulting class files, rewrite the class file (statically as a
2nd compile step or dynamically at load time via a custom
classloader) to instrument all the same logic and processing that
would have been implemented explicitly in the source as in the hard
coded option above.
|
- Cleanest source code.
- Easiest to convert away from Progress semantics and to
a more natural Java approach.
- Performance is as good as the hard coded approach.
|
- Hardest to debug since there is truly hidden processing
(only in the byte code and not in the source).
- Editing this code will likely require modifying some
form of additional input (a second hints/annotations file or
possibly some form of inline annotations) in order to specify
the block properties etc... that must be applied (since it is
not implicit in the bytecode) and the locations to which to
apply this instrumentation. This makes maintenance harder than
just editing a single source file.
|
At this time, the hard coded source
approach is the simplest and easiest to achieve. As a likely future
optimization, the bytecode instrumentation approach will probably be provided in
the future as an option.
Implementation Details
As mentioned above, Progress 4GL implements a great deal of its transaction and
block behavior in an implicit manner. One of the advantages Progress has
in its interpreter is the embedding of knowledge about each block open and close
and each loop iteration.
All Progress conditions will be mapped to custom Java exceptions. This
will allow the generation of such conditions to unwind the stack and to be
caught/processed/rethrown where needed. The base class will be
com.goldencode.p2j.util.ConditionException which will allow for a common catch
{} when this is useful. Each of the 4 conditions will be mapped to a
subclass:
In the Java implementation, a TransactionManager class provides a central
clearinghouse that tracks block open, close and loop iteration.
Transaction services will be provided by the TransactionManager based on method
calls emitted at the start and end of each block, and when any loop
iterates. These hooks will provide the same level of processing that can
be implemented internally within the Progress 4GL interpreter.
All blocks and loops which have properties will be emitted inside a try/finally
pair. The TransactionManager pushScope() will be called first. Then
the try {} will surround the loop or block itself. The finally {} provides
the ability to hook the TransactionManager popScope() method call. This
ensures that the scope will be properly managed no matter how we exit the block
(via exception, natural block exit or return).
Inside the outer try block, there will be any loop control processing needed and
the opening of the block itself. This block will contain a special do { }
while () loop to implement retry logic. Inside this do while loop is
another try {} block and a set of catch {} blocks to handle condition
processing. The actual block or loop body is emitted inside this try
block.
Hooks and queries into the Transaction Manager allow it to handle much of the
processing automatically, however all Java statements which affect flow control
must be emitted into the source. This is a requirement of the Java
language as the flow control statements are converted into byte code
(instructions) which are inherently local to the current method. The Java
instruction set (all possible byte codes) only provides the ability to handle
local branching except for method calls, method returns and exception processing
that can arbitrarily unwind the stack.
The following is a sample of the emitted code:
TransactionManager.pushScope("label",
TransactionManager.TRANSACTION,
true,
true,
false,
false);
try
{
label:
// if this is a loop, the loop control structure will emit here
{
TransactionManager.blockSetup();
do
{
try
{
// block body is emitted here
}
catch (ErrorConditionException err)
{
TransactionManager.honorError(err);
TransactionManager.rollback("label");
TransactionManager.triggerRetry(null);
}
catch (EndConditionException end)
{
TransactionManager.rollback("label");
break label;
}
catch (RetryUnwindException ru)
{
TransactionManager.honorRetry(ru);
}
}
while (TransactionManager.needsRetry());
if (TransactionManager.isBreakPending())
{
break label;
}
TransactionManager.iterate();
}
}
finally
{
TransactionManager.popScope();
}
|
The TransactionManager provides the following services via static methods that
are context-local to the current user's context:
- Maintains the state of each block, tracks start and end and handles
nesting of blocks. TransactionManager.pushScope() and
TransactionManager.popScope() are used to handle this maintenance.
Note that the popScope() is implemented inside a finally {} to ensure that
it always gets called. TransactionManager.iterate() is used to allow
the Transaction Manager to implement state management for loops.
- UNDO support:
- When variables and record buffers (database, temp-tables and
work-tables) are defined, they register with the TransactionManager as
undoable unless the Progress source definition was NO-UNDO. All
such Java objects must implement the Undoable Interface. This
interface provides the ability to clone (backup) and assign back (undo)
the state of an Undoable object.
- To obtain rollback services, an object must be added to the
undoable list via TransactionManager.register().
- At every transaction and sub-transaction start, any Java
objects registered in the undo list will be cloned and stored in a map
that is unique to the block being opened. The key for each copy
will be the reference to the object being copied. A static method
(TransactionManager.makeBackup()) is called just inside each
block, after any loop control variables are incremented or
initialized. This is required because of how Progress implements
its backup set, which occurs after loop control variable changes are
made. Otherwise, this processing would have been integrated into
TransactionManager.pushScope() and TransactionManager.iterate().
- Rollback support for variables and record buffers via
TransactionManager.triggerRollback() which can be called by user code (a
conversion of the UNDO language statement) or in response to the
condition processing (a catch block).
- At the successful completion of every transaction and
sub-transaction, all registered undoable objects have their changes
committed. In terms of the database, this means that the changes
are added to the Hibernate session. Due to nesting, it is possible
to partially commit changes but then to backout all changes at the
transaction level. This works for all types of undoable objects,
although the database changes are maintained differently because of
Hibernate. At this time only the database fields implement the
special Commitable interface and get a call to commit(boolean transaction)
upon each transaction or sub-transaction. Variables don't require
such support since they naturally commit at the end of each block and
instead they only get rolled back on a failure, otherwise the backup
sets are just garbage collected when they go out of scope. This
processing can be called from either TransactionManager.iterate() or
TransactionManager.popScope() depending on the method of successful exit
from the block.
- To obtain commit services, an object must be added to the
commitable list via TransactionManager.register() or
TransactionManager.registerCommit().
- RETRY state management/infinite loop protection:
- An extra do {} while () loop encloses the block body in the
emitted code.
- The transaction manager associates a BlockDefinition object
with each block and maintains the proper nesting. Each such
definition has retry related state variables.
- The first time through any loop or block, the state is
initialized to disable retry and enable infinite loop protection.
- During any condition processing (a catch block), if retry was
implicitly or explicitly defined in the original Progress source, then
retry would be enabled for that associated block. The
TransactionManager.triggerRetry() is called to set the state (and
possibly throw an exception).
- If a retry is requested for a block that is not the current
block, the RetryUnwindException is thrown to unwind the stack to the
proper block. Each block catches this exception and tests if this
is the target for the retry. If not, the exception is
rethrown. When the proper block is reached, the retry flag is set
and normal processing of the do {} while () loop continues. The
TransactionManager.honorRetry() is called in each catch block to
implement this logic.
- Every iteration of a loop reset the retry state such that it is
as if we are starting fresh.
- The do {} while () loop calls TransactionManager.needsRetry()
to determine whether or not it needs to loop.
- needsRetry() implements infinite loop protection (in the case
where a retry is requested, it is sometimes converted into another
operation).
- Input blocking language statements notify the
TransactionManager such that the current scope's infinite loop
protection can be disabled. Similarly, the replacement for the
RETRY built-in function is backed by a TransactionManager method
isRetry(). Before this method returns its value, the current
scope's infinite loop protection is disabled (see
TransactionManager.disableLoopProtection()).
- RETURN ERROR state/processing:
- Normal ERROR conditions are thrown "inline", wherever the
problem occurs.
- This language statement must bypass normal ERROR condition
processing and instead should be processed in the caller of the current
method, no matter how deeply nested the block is in which this is
thrown.
- To throw this, one calls
TransactionManager.triggerErrorInCaller().
- A special ignore flag is kept to track if this special mode is
being used. It is checked using To throw this, one calls
TransactionManager.honorError() in catch blocks which process the
ErrorConditionException and if this returns silently, then normal error
processing is honored. Otherwise it will rethrow the exception,
propagating it upwards. This results in unwinding the stack to the
top-level block of the method. At this point, the exception is
thrown once more but first the ignore flag is cleared. The caller
will then catch and process this error.
- Provides a hook for any object to get a callback on exit from the
scope in which it is defined. This provides an easy method for
implementing cleanup processing. All objects needing this support must
be registered using TransactionManager.registerFinish() and must implement
the Finalizable interface.
- Provides a hook for any object that requires notification of all
block entrances and block exits. This is provided by the Scopeable
interface and such objects are registered via the
TransactionManager.registerScopeable().
- Provides a session-level batch start and stop notification service.
- At every block start, stop and iteration
the current thread's interrupted
status is checked and if the thread has been previously interrupted, a StopConditionException
will be thrown. See TransactionManager.honorStopCondition().
All blocks will be labeled, with a manufactured label or one that comes from the
original Progress source. This facilitates condition processing, since
processing that was previously implicit is now being made explicit. This
is also important since a simple block in Progress is implemented as a set of
related blocks in Java, in order to always be sure that the flow control
statement references the correct block, it is made explicit.
Silent error processing is provided using the
ErrorManager
class. This class:
- Stores the state of whether errors should or should not be thrown.
- Tracks if an error has been thrown and the list of error text and
numbers.
- Allows access to this data.
- Provides helpers to throw errors or record the error if silent mode
is enabled.
Lower level interfaces are also provided to allow more control within a library,
especially in the case where one must retain control over where to throw the
error due to the inability to silently return from the location at which the
error was encountered. The
ErrorUnwindException
is provided to facilitate this requirement.
All data type wrappers, expression operator implementations and the runtime
backing for built-in functions have been modified to be aware of the
ErrorManager and silent error mode. As a rule, where an RUNTIME errorn
would normally be generated in Progress source, the P2J runtime will use the
ErrorManager to honor the silent error mode. COMPILE errors are not
handled at all since no such errors should occur in the converted
environment. Note that there may be possibilities of runtime issues in
Java that are compile-time issues in Progress. At this time, such cases
will throw Java exceptions such as IllegalArgumentException rather than attempt
to duplicate Progress COMPILE time error strings.
Java has no undo or retry functionality. This means that Progress and Java
are mismatched in these areas. For this reason, the vast majority of the
processing in the Transaction Manager is related to one of these two areas.
See also:
TransactionManager
BlockDefinition
ErrorManager
ErrorUnwindException
Undoable
Commitable
Finalizable
Scopeable
ConditionException
RetryUnwindException
ContextLocal
rules/annotations/block_properties.rules
rules/annotations/leave_fixups.rules
rules/annotations/labels.rules
rules/convert/control_flow.rules
rules/convert/main_method.rules
rules/convert/java_template.tpl
Honoring Stop Conditions
Stop conditions can be generated by certain runtime errors such as a database
disconnection or failures in stream processing. Any runtime code running
on a thread which is servicing converted Progress business logic will translate
any InterruptedException into a StopConditionException. Classes
which provide such a service:
There are unusual cases in the runtime where processing is not occurring on a
thread which is servicing converted Progress business logic. For example,
if a secondary thread is used to cleanup filesystem resource (e.g. close pipes
or files) or if a secondary thread is used to copy data between two pipes.
Note that in these cases, there is no counterpart or proper mapping into a
Progress semantic since such cases don't exist in the Progress runtime.
Generally, such cases silently exit or fail. Classes which provide such
behavior:
- ProcessOps (both the Cleaner
inner class and interruptions that occur during a child process' execution
--> Progress does not see such interruptions)
- StreamConnector (secondary
copying thread)
In addition, the STOP language statement in Progress allows an explicit stop
condition to be raised. In this case the business logic will explicitly
throw a StopConditionException.
Stop condition processing is complicated by the Progress feature that
the user
may press the CTRL-C key combination at any time and this immediately
interrupts
the execution of the Progress program (which is single threaded). In
the
P2J environment this is complicated by the split of UI processing into
a
separate process accessed via the network. The UI client will monitor
for
the CTRL-C key combination and this will cause asynchronous processing
of the
interruption to occur. In particular, if processing is currently
occurring on the client side, the interruption will be converted
to an interruption of that running thread which will be converted to a
StopConditionException in the UI client. This will
then unwind the stack normally until the location in the business logic
that
handles (catches) the stop condition is reached. If the current thread
is
processing on the server (such that the client cannot honor the UI
since it is
not actively running), then the server will be asynchronously notified
and this
will trigger a call to Thread.interrupt(). If the currently processing
thread on the server (for that user's context) is blocked, an
InterruptedException would be generated and this would be caught by the
runtime
as noted above and translated into a StopConditionException. This is a
key
point: J2SE only generates an InterruptedException in specific cases
where a
thread blocks such as Object.wait() or Thread.sleep(). If the thread
is
not blocked, then J2SE will not generate an InterruptedException,
instead it
just sets the
interrupt status
of the thread. The TransactionManager checks the
interrupted status
of the current thread at block start, stop and when a block iterates. Any
previous interruption will be honored by throwing a
StopConditionException. Please
see above
for more details on the TransactionManager.
Accumulator behavior
Contents
- ACCUMULATE statement
- DISPLAY with aggregate phrase
- ACCUM function
- Aggregator types
- Accumulator expressions
- Accumulator usage
- Initial (entry) value for accumulators
- Block-behavior for accumulators
- ACCUM scope computation
- Reset behavior for accumulators with simple blocks
- Reset behavior for accumulators with IF statement
- The "backup" mechanism
- ACCUM with nested RUN/TRIGGER statements
TBD
- for COUNT accumulators, if the expression contains a field reference and the buffer currently
does not reference any records, this 'not-existent' field will be counted. So, accumulation must
somehow be triggered before retrieving the record.
- accumulation "variables" (like a "naked" count variable) can be referenced directly in Progress
without the "accum 1 count" construct. This is a parse-time problem but it also has consequences
that are harder to fix since such vars can be assigned!
ACCUMULATE statement
The syntax of the ACCUMULATE statement is:
ACCUM {expression (aggregate-phrase)}
The following rules about ACCUM stmt must be taken into consideration:
- When converting this statement, it must be taken into consideration
that it may contain more then one accumulation expression and also more
then one aggregators for a certain expression. This knowledge is used
by the annotations/accumulate.rules file to
find the correct expression for a certain accumulator - in the
post-rules section, it will find and set the scope-id for the
expression
to the computed id for the current accumulator.
- When aggregate-phrase contains more then one aggregator, this will
trigger generation of an accumulator instance for each combination of
(expression, aggregator).
- When the accumulation is performed while iterating a FOR EACH block
with a BREAK BY option, the accumulation may be computed for each
BREAK BY group, if the "BY break-group" is specified in the
aggregate-phrase.
DISPLAY with aggregate phrase
The DISPLAY with aggregate phrase affects the accumulators in the following ways:
- on conversion, the accumulator must be scoped at least to the same block as the frame used in the display statement
- after the block is ended, the display statement will display a cumulative accumulation result
- if a sub-only with a BY group is used, then it will display the results for each BY value.
When an accumulator is used with a DISPLAY statement and with a ACCUM
statement, then it must not emit an implicit c'tor; the rule is that,
when the accumulator is used with an ACCUM statement, the ACCUM
statement will be used to emit the c'tor - any DISPLAY with accum
references
will be ignored. The default c'tor (except COUNT and SUB-COUNT, which
always use default c'tor) will be generated only when there are DISPLAY
with accum references and there is no ACCUM statement references, for a
certain accumulator.
Also, the Accumulator.accumulate(BaseDataType) must not be restricted
to a ExternalDataSource, to allow the ACCUM and DISPLAY statements to
work together.
There are a few special cases, when an accumulator is used with a DISPLAY statement:
- if the DISPLAY statement is never executed but the block is
iterated at least once, the frame will display the total or sub-total
values.
- a certain accumulator can be used with different DISPLAY
statements, within the same block. This means that accumulation must be
triggered only for the first DISPLAY call; TBD - this is not implemented yet.
As a DO i = x TO z statement is converted to a for(i = x; i < z;
i++) Java statement, it means that it requires conversion changes; P2J
must know if the DO block was entered and trigger accumulation inside
the DO block.
- the accumulators registered with a DISPLAY statement may never
accumulate; thus, their type must be set on instantiation, and this
type must be identical to the type of the accumulated expression; more,
this type is identical to the type of the widget with which this
accumulator is registered with.
During this document, there may be references to the DISPLAY accum
statement or DISPLAY statement - this will always mean that the author
is
referring to the "DISPLAY with aggregator phrase" statement.
ACCUM function
The ACCUM function is used to retrieve the accumulation result at a
certain time. This does not affect the accumulation result and will
affect only the part where the scope-id is computed: the found block
must enclose all accumulator references.
Aggregator types
The PROGRESS accumulator types are:
Aggregator long name |
Short name |
AVERAGE |
AVG |
COUNT |
- |
TOTAL |
- |
MINIMUM |
MIN |
MAXIMUM |
MAX |
SUB-AVERAGE |
- |
SUB-COUNT |
- |
SUB-TOTAL |
- |
SUB-MINIMUM |
SUB-MIN |
SUB-MAXIMUM |
SUB-MAX |
For all accumulator types it can be specified a "BY break-group"
option. When specifying such an option, the accumulation will be
performed for each
distinct break-group value; these values will be available for
retrieving using the ACCUM function only in the current or an enclosed
child block.
After leaving the block which has the "BREAK BY group" option, the
accumulation values for each break-group are no longer accessible.
The sub-only accumulators will behave the same as normal accumulators, if they have no break-by option attached.
If a "By group" is present and the accumulator is used with a DISPLAY
stmt, the sub-only accumulators will display the accumulation result
for each "BY" value; for normal accumulators, the DISPLAY stmt will
display only a final result.
Accumulator expressions
Accumulator expressions can be cataloged as:
1. simple expressions: a variable/table field name
2. literals - a constant expressions
3. complex expressions
The accumulation performs the same for all expression types: the
expression will be evaluated (if needed) and will be added to the
primary result and sub-results, if needed.
Accumulator usage
In PROGRESS, the accumulator is identified by its expression, being
aware of the operands' order. For example, "i + 0" and "0 + i" will
represent different accumulators.
So, if the expression must be used to identify unique accumulators, the
following rules are used at conversion time to compute the accumulator
name:
- if a DISPLAY statement with a aggregator phrase is encountered, then
the accumulation expression is checked with the global expr-to-name map
(named "accumNames") which
holds the computed java names (for each expression) extracted from the
ACCUM statements and the global expr-to-name map (named "dispNames")
which holds the computed java names for all
accumulators used in DISPLAY with aggregator phrase stmts.
If the expression is found in either the accumNames or dispNames maps,
then the found name is returned. Else, a name like "accumDisp#"(where #
holds for an unique index) is computed
- if an ACCUM statement is encountered, then the dispNames and
accumNames maps will be checked if they contain the accumulator
expression. If the expression is found, then the retrieved name is
used; else, a name like 'accumType#' will be returned, where "type"
represents the short-name for the accumulator (without the "sub-"
prefix).
- if an ACCUM function is used, then the accumNames is first checked
(to see if the accumulator was used with an ACCUM statement) and also
it computes the default label and the type for this expression; after
this, if the dispNames map contains the accum expression,
then it will override the java name.
Initial (entry) value for accumulators
All accumulators start with its value set to PROGRESS unknown (?)
value. This value will be maintained until a BLOCK is entered (the
BLOCK header is executed), where the accumulation value will be set to:
Aggregator |
AVERAGE |
COUNT |
TOTAL |
MINIMUM |
MAXIMUM |
SUB-AVERAGE |
SUB-COUNT |
SUB-TOTAL |
SUB-MINIMUM |
SUB-MAXIMUM |
Value |
0 |
0 |
0 |
? |
? |
0 |
0 |
0 |
? |
? |
More about which blocks reset the accumulators and how this reset is done can be found in sections
Reset behavior for accumulators with simple blocks and
Reset behavior for accumulators with IF stmt.
Block-behavior for accumulators
The block behavior for accumulators affects how the accumulation is
executed and also what value is retrieved when the ACCUM function is
used.
To implement this, the accumulators must accumulate their values into
two distinct data bundles:
- the 'pending' accumulation data: when the accumulation is executed,
the data must be accumulated to all parent blocks; so, the accumulation
is done to all data found in the pending stack.
This pending data is used when the accumulator exists within a certain
block: the parent block will be updated with the pending accumulation
value from the child block.
- the 'reported' accumulation stack (the bundleStack): for each block,
there will be kept a set with all the eventually accumulated data for
each break-group value and also the primary data bundle. When the
accumulation is performed for this block, the primary data bundle and
the break-group data - if needed - are updated
- the sub-only property: the accumulator will be marked as sub-only
when it enters the block; this property is distinct for each block
- the accumulator must have knowledge that it enters a block which
encloses a DISPLAY with aggregate phrase/ACCUM stmt for this
accumulator, which may be executed; so, upon entry of a block which
uses this accumulator, an Accumulator.reset() call is executed - this
will set the "usage" flag to all parent blocks to true and also will
reset the primary data bundle to its entry value.
Using the ACCUM function to retrieve the accumulated result presents a few usage scenarios:
1. Using the ACCUM function right after the ACCUM stmt has been executed:
repeat:
...
accum i (count).
message accum count i.
...
end.
- the retrieved accumulation result will be the total accumulation result for this block (which is kept in the 'bundleStack').
2. Using the ACCUM function right after the a block which accumulated some data is executed:
B1:
repeat:
...
B2:
repeat:
...
accum i (count).
...
end.
message accum count i.
...
end.
- the retrieved accumulation result will be the accumulation result for
block B2. If the body for block B2 did not execute, then the
accumulation result will be the default "entry" value.
The rule extracted from this example is that, when leaving a block
which accumulated some data, it will set the 'reporting' data bundle
(the primary data bundle from dataBundle stack) for the parent block
(block B1)
to be the total accumulated value (from the pendingStack) for block B2.
In other words, block B1 will inherit and report the accumulation
result from block B2, until a new ACCUM stmt is executed or another
block is entered.
Other special cases about the ACCUM function will be presented in the following "reset" and "backup" sections.
The PROGRESS statements which present the above behavior are REPEAT, DO
[TRANSACTION/ON END-KEY UNDO, LEAVE], FOR EACH/LAST/FIRST, OPEN QUERY
... DO WHILE AVAILABLE.
The editing blocks (PROMPT-FOR/UPDATE/SET statements) do not present
special behavior when accumulators are used, so accumulator scope
notifications will be ignored for these blocks.
Also, DO statements (which have no transaction support) do not affect
accumulators in any way.
ACCUM scope computation
A certain accumulator may be scoped to a certain block or may be promoted to an instance field, following those rules:
1. The computed block to which the accumulator will be scoped (and also the accumulator) presents these properties:
- it encloses all references for this accumulator
- the accumulator is not used with a PUT or FORM statement
- the block presents transaction managenment (transaction level annotation (trans_level) is not equal to prog.no_transaction)
- all frames used by a DISPLAY with aggregator phrase statement are enclosed within this block
- it is not the root block
2. In all other cases, it will be promoted to instance field
Also, if the same accumulation expression is used within a procedure,
function or a trigger, after conversion will result in 3 different
accumulators.
The computed accumulator scope is also used by the accumulator
expression - when the accumulator expression is instantiated (field
reference, literal, etc) it must be promoted or instantiated in the
same block as the accumulator.
Reset behavior for accumulators with simple blocks
Accumulators present a special behavior when a block which encloses an
ACCUM statement is not executed or the block is executed and the ACCUM
statement is skipped.
Before continuing, it must be noted that this behavior is presented
with all aggregator types, with all REPEAT, DO [TRANSACTION/ON ...],
FOR EACH/LAST/FIRST, OPEN QUERY ... DO WHILE ... statements and with
both ACCUMULATE statement and DISPLAY with aggregate phrase statement.
For this, the following testcases are used to determine the rules:
1. a block which does not use the ACCUM stmt is not executed
B1:
repeat k = 1 to 2:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat while false: /* no accumulator usage */
...
end.
display accum total i with down column 13 frame f2.
end.
Frames f1 and f2 will display the same value. This testcase shows that
blocks which do not use the ACCUM statement for a certain accumulator
will not affect it in any way. This behavior is also present if B2 has
no ACCUM stmt and is executed. Also, any ACCUM function usage in block
B2 will return the same value as if it was executed in block B1,
as proved in the following testcase:
B1:
repeat k = 1 to 2:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 1: /* no accum stmt usage */
display accum total i with down column 13 frame f2.
end.
display accum total i with down column 26 frame f2.
end.
This translates into: the accumulator must not set the "reported"
bundle for block B1 to be the "pending" bundle computed in block B2,
when B2 is leaved, because block B2 does not have any ACCUM stmts for
this accumulator. Also, when executing a block which does not use the
ACCUM statement, the accumulator instance must ignore any "scopeStart",
"scopeFinished", "iterate", "retry" and "finished" calls.
2. a block which uses the ACCUM stmt is not executed
B1:
repeat k = 1 to 2:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat while false:
accum i (total).
end.
display accum total i with down column 13 frame f2.
end.
In this example, block B2 is not executed, but the block uses the ACCUM stmt; frames f1 and f2 will display different results:
- frame f1 will display the accumulated value for block B1
- frame f2 will display the "reset" value (0 in this case)
This is caused by the fact that, when executing the header for block
B2, the accumulator is notified that it enters a block which may
execute accumulation; so, the accumulator will be reset and also, when
leaving block B2, it will set B1's "reported" bundle to be the
"pending" (currently reset) bundle for block B2.
3. a block which uses the ACCUM stmt is executed, but the ACCUM stmt is skipped
B1:
repeat k = 1 to 2:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
do transaction:
if false then
accum i (total).
end.
display accum total i with down column 13 frame f2.
end.
In this example, block B2 is executed, but the ACCUM stmt is not; frames f1 and f2 will also display different results:
- frame f1 will display the accumulated value for block B1
- frame f2 will display the "reset" value (0 in this case)
This is caused by the fact that, when executing the header for block
B2, the accumulator is notified that it enters a block which may
execute accumulation; so, the accumulator will be reset and also, when
leaving block B2, it will set B1's "reported" bundle to be the
"pending" (currently reset) bundle for block B2.
4. a block scoped to procedure block which does not use the ACCUM stmt
is not executed, with accumulation performed in a previous block
B1:
repeat k = 1 to 2:
accum i (total).
end.
display accum total i with down column 0 frame f1.
B2:
repeat while false: /* no accumulator usage */
...
end.
display accum total i with down column 13 frame f2.
Frames f1 and f2 will display the same value. This testcase shows that
blocks which do not use the ACCUM statement for a certain accumulator
will not affect it in any way. Also, this proves that, regardless of
scoping level, if a block doesn't use the ACCUM statement and is not
executed, the accumulator is not reset. This behavior is also present
if B2 has no ACCUM stmt and is executed.
5. a block scoped to procedure block which uses the ACCUM stmt is not executed, with accumulation performed in a previous block
B1:
repeat k = 1 to 2:
accum i (total).
end.
display accum total i with down column 0 frame f1.
B2:
repeat while false:
accum i (total).
end.
display accum total i with down column 13 frame f2.
In this example, block B2 is not executed, but the block uses the ACCUM
stmt; frames f1 and f2 will also display different results:
- frame f1 will display the accumulated value for block B1
- frame f2 will display the "reset" value (0 in this case)
This is caused by the fact that, when executing the header for block
B2, the accumulator is notified that it enters a block which may
execute accumulation; so, the accumulator will be reset and also, when
leaving block B2, it will set B1's "reported" bundle to be the
"pending" (currently reset) bundle for block B2.
The testcase proves that the accumulator will be reset if a block which
may use execute the ACCUM statement is entered, even if the block is
scoped to procedure block.
6. a block scoped to procedure block which uses the ACCUM stmt is
executed but the ACCUM stmt is skipped, with accumulation performed in
a previous block
B1:
repeat k = 1 to 2:
accum i (total).
end.
display accum total i with down column 0 frame f1.
B2:
repeat:
if false then
accum i (total).
end.
display accum total i with down column 13 frame f2.
In this example, block B2 is executed, but the the ACCUM stmt is skipped; frames f1 and f2 will also display different results:
- frame f1 will display the accumulated value for block B1
- frame f2 will display the "reset" value (0 in this case)
This is caused by the fact that, when executing the header for block
B2, the accumulator is notified that it enters a block which may
execute accumulation; so, the accumulator will be reset and also, when
leaving block B2, it will set B1's "reported" bundle to be the
"pending" (currently reset) bundle for block B2.
The testcase proves that the accumulator will be reset if a block which
may use execute the ACCUM statement is entered, even if the block is
scoped to procedure block.
Reset behavior for accumulators with IF statement
The IF statement impact on the accumulator "reset" behavior is related
to stopping block execution using an "IF condition", when the condition
is false. This will mean that neither the block header or body are
executed.
1. IF statement with the ACCUM statement
B1:
repeat k = 1 to 2:
accum i (total).
display accum total i with down column 0 frame f1.
if false then accum i (total).
display accum total i with down column 13 frame f2.
end.
Although the second accumulation statement does not execute at all, frame f1 and f2 will display the same results.
So, this proves that when an IF statement is executed and it encloses
an ACCUM statement without an intermediate block, the accumulation
value is
not altered.
2. IF false statement with a block which does not execute
B1:
repeat k = 1 to 2:
accum i (total).
display accum total i with down column 0 frame f1.
if false then
B2:
repeat while false:
accum i (total).
end.
display accum total i with down column 13 frame f2.
end.
Although block B2 does not execute at all (header or body), frame f1 and f2 will display different results:
- frame f1 will display the current accumulation value for block B1
- frame f2 will always display the UNKNOWN (?) value
So, this may prove that, when an IF statement is executed and it
encloses an ACCUM statement inside another block, the accumulation
value is set to
unknown; also, when executing the THEN/ELSE branch, the accumulation
value will be set to the "reset" value only if the header for a block
which encloses the ACCUM statement is executed; if no such blocks are
encountered on the THEN/ELSE branch, the accumulation value will
remain set to UNKNOWN. (this assumption is disproved on the following
testcase)
3. IF true statement with a block which does not execute
B1:
repeat k = 1 to 2:
accum i (total).
display accum total i with down column 0 frame f1.
if true then do:
display accum total i with down column 13 frame f2.
B2:
repeat while false:
accum i (total).
end.
end.
display accum total i with down column 26 frame f3.
end.
Block B2 does not execute and frames f1, f2 and f3 will display different results:
- frame f1 will display the current accumulation value for block B1
- frame f2 will display the same data as frame f1
- frame f3 will display the "reset" value for the accumulator (in this case, "0"), because block B2 does not execute.
So, this proves that, when an IF statement is executed and it encloses
an ACCUM statement inside another block, the accumulation value is NOT
set to
unknown - this is postponed until the THEN/ELSE branch has ended and
only if no ACCUM statement have been executed on the THEN/ELSE branch
or no blocks (body or header) which enclose the ACCUM statement are
executed.
4. IF false statement scoped to procedure block with a block which does not execute
B1:
repeat k = 1 to 2:
accum i (total).
end.
display accum total i with down column 0 frame f1.
if false then
B2:
repeat while false:
accum i (total).
end.
display accum total i with down column 13 frame f2.
Block B2 does not execute at all (header or body) but frames f1 and f2 will display the same result.
So, this proves that, when the IF statement is scoped to the procedure
block, it will not set to unknown the accumulator, even if it encloses
a block which uses the accumulator.
Further more, scoping the IF to a DO statement:
B1:
repeat k = 1 to 2:
accum i (total).
end.
display accum total i with down column 0 frame f1.
do:
if false then
B2:
repeat while false:
accum i (total).
end.
end.
display accum total i with down column 13 frame f2.
will result in frames f1 and f2 displaying the same values.
Experimenting with other statements which do not affect the
accumulators (like SET/UPDATE/PROMPT-FOR - the editing blocks)
showed the same behavior. The conclusion is that only IF statements
which are enclosed within a block will set the accumulator to UNKNOWN.
The "backup" mechanism
The backup mechanism is related to cases when a block is entered and,
before any other ACCUM/DISPLAY statement usage, the ACCUM function is
used to retrieve the accumulated result. The following testcases
describe this behavior:
1. display the accumulation result before any accumulation
B1:
repeat k = 1 to 1:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 2:
display accum total i with down column 13 frame f2.
accum i (total).
display accum total i with down column 26 frame f3.
end.
display accum total i with down column 39 frame f4.
end.
In this case, frame f2 will always display the same result as frame f1
(on every B2 iteration) and frame f3 will display the current
accumulation result for block B2; frame f4 will display the final
accumulation result for block B2. This proves that PROGRESS presents
some backup mechanism - used to report the "entry" accumulation value
if no ACCUM stmts were executed in the current block before the ACCUM
function call.
2. accumulation enclosed to the current block, but skipped
B1:
repeat k = 1 to 1:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 2:
display accum total i with down column 13 frame f2.
if false then accum i (total). /*1*/
display accum total i with down column 26 frame f3.
accum i (total). /*2*/
display accum total i with down column 39 frame f4.
end.
display accum total i with down column 52 frame f5.
end.
In this case, frame f2 will not display the same result as frame f1:
- frame f1 will display the "reported" accumulation result for block B1
- frame f2 will display the same data as frame f3
- frame f3 will display:
- when the first iteration is performed, the "reset" value (in this case, 0)
- on subsequent iterations, the current accumulated result for block B2
- frame f4 will display the current accumulated result for block B2
- frame f5 will display the final accumulation result for block B2 (2 in this case)
So, after executing the "if false" statement, the accumulator will no
longer report the "entry" value (the accumulation value for block B1).
This testcase also proves that, even if there are two ACCUM statements
inside block B2, the second accumulation is executed. The difference
between
the two ACCUM statements is that the second one has as parent block B2
and the first one has as parent the IF statement.
Further more, investigating a testcase like:
B1:
repeat k = 1 to 1:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 2:
display accum total i with down column 13 frame f2.
accum i (total). /*1*/
display accum total i with down column 26 frame f3.
if false then accum i (total). /*2*/
display accum total i with down column 39 frame f4.
end.
display accum total i with down column 52 frame f5.
end.
shows this:
- frame f1 will always display the current accumulation value for block B1
- frame f2 will display the "entry" accumulation value (the current accumulated value in block B1)
- frame f3 will always display the current accumulated value for block B2
- frame f4 will display the same values as frame f3
- frame f5 will display the total accumulated value in block B2
The conclusion from this testcase is that the "IF false" statement
which uses the accumulator will reset it only if no other ACCUM
statements
were executed before it. So, the accumulator somehow knows to ignore
the "if false then accum i (total)." statements after the accumulation
has been performed within the current or a child block.
3. accumulation enclosed to the current block, but skipped - case 2
B1:
repeat k = 1 to 1:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 2:
if false then
B3:
repeat while false:
accum i (total).
end.
display accum total i with down column 13 frame f2.
accum i (total).
display accum total i with down column 26 frame f3.
end.
display accum total i with down column 39 frame f4.
end.
In this case, frame f2 will not display the same result as frame f1:
- frame f1 will display the "reported" accumulation result for block B1
- frame f2 will always display UNKNOWN (?) value
- frame f3 will always display UNKNOWN (?) value
- frame f4 will display the accumulated result in block B2 (in this case, value 2)
It seems that, after executing the "IF false" statement and setting the
accumulator to UNKNOWN, the reported value for block B2 will always be
UNKNOWN, even if accumulation is performed
for block B2.
4. accumulation enclosed to the current block, but skipped - case 3
B1:
repeat k = 1 to 1:
accum i (total).
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 2:
B3:
repeat while false:
accum i (total).
end.
display accum total i with down column 13 frame f2.
accum i (total).
display accum total i with down column 26 frame f3.
end.
display accum total i with down column 39 frame f4.
end.
In this case, frame f2 will not display the same result as frame f1:
- frame f1 will display the "reported" accumulation result for block B1
- frame f2 will always display the reset value (0 in this case)
- frame f3 will always display the reset value (0 in this case)
- frame f4 will display the total accumulation result for block B2.
For this testcase, the "reported" result for block B2 will not be set
to UNKNOWN after executing the header for block B3 - it will be reset.
This "reset" value will always be reported by any ACCUM function call,
regardless of any executed ACCUM statement, until another block with an
enclosed ACCUM statement or an IF statement with an enclosed ACCUM
statement is executed.
5. "pending" and "reported" values for a block
B1:
repeat k = 1 to 1:
if false then accum i (total). /*1*/
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 3:
accum i (total). /*2*/
end.
display accum total i with down column 13 frame f2.
accum i (total). /*3*/
display accum total i with down column 26 frame f3.
B3:
repeat j = 1 to 4:
accum i (total). /*3*/
end.
display accum total i with down column 39 frame f4.
end.
In this example, frames f2 and f3 will always display the accumulated result for block B2 and frame f4 will always display the
accumulated result for block B3:
- frame f1 will display 0 on first iteration and the current accumulated result for block B1, on subsequent iterations
- frame f2 will always display the accumulated result for block B2
- frame f3 will always display the same values as frame f2
- frame f4 will always display the accumulated result for block B3
So, when the third ACCUM statement is executed, the "reported" value
for block B1 is not affected - after executing a block which
uses the ACCUM statement, the "reported" value for the parent block
(block B1 in this case) will always be the "pending" (computed) value
for
the executed block, until another block is executed.
More, this affects the reported value for the enclosed block:
B1:
repeat k = 1 to 1:
if false then accum i (total). /*1*/
display accum total i with down column 0 frame f1.
B2:
repeat j = 1 to 3:
message "B2:" + string(accum total i). /*1*/
accum i (total). /*2*/
end.
display accum total i with down column 13 frame f2.
accum i (total). /*3*/
display accum total i with down column 26 frame f3.
B3:
repeat j = 1 to 4:
message "B3:" + string(accum total i). /*2*/
accum i (total). /*4*/
end.
display accum total i with down column 39 frame f4.
end.
The ACCUM function used in the two message statements will behave differently:
- the ACCUM function in case #1 (block B2) will always display the accumulated (pending) value for block B1
- the ACCUM function in case #2 (block B3) will be affected by the fact
that another block (block B2) has been executed - this will display
0 on the first iteration and the accumulated/pending result for block
B3 on subsequent iterations.
ACCUM with nested RUN/TRIGGER statements
1. ACCUM with nested RUN statements (recursion)
def var i as int.
procedure proc1.
def input parameter p as int.
do transaction:
accum i (count).
if p <> 0
then run proc1(p - 1).
end.
put screen row p + 1 string(p) + " " + string(accum count i).
end.
run proc1(10).
For the testcases above, the ACCUM function will always return the same
value. This is because accumulators - which are used inside a
procedure/function -
are alive and can be used only as along as the procedure is executed;
this means that the accumulator is not scoped to the root external
procedure, but is scoped to the internal procedure. In P2J, instead of
scoping the accumulator to the internal procedure, we keep a stack with
the "toplevel" status of each entered block. This way, when recursion
occurs and the ACCUM statement is called, the accumulation will be
performed only until a "toplevel" block is encountered - and so no
other blocks from a previous recurssion will be affected.
2. ACCUM with nested TRIGGERS
def var j as int.
def var i as int.
def var k as int init 0.
form j with frame f0 column 39.
view frame f0.
on F7 of j in frame f0 do:
do transaction:
accum i (count).
end.
put screen row k + 1 column 1 string(k) + " " + string(accum count i).
if k < 10 then do:
k = k + 1.
apply "F7" to j in frame f0.
k = k - 1.
end.
put screen row k + 1 column 13 string(k) + " " + string(accum count i).
end.
apply "F7" to j in frame f0.
In this testcase, applying the same trigger as the one being executed
will not be possible: in PROGRESS, the trigger will not go into
recurssion and
so the 'apply "F7" to j in frame f0.' statement inside the trigger
definition will be ignored.
Database Support
Data Types
The following Progress data types are directly supported as persistable types
using the P2J wrappers. The mappings are made transparent to the
application using custom, Hibernate user types. See the
com.goldencode.p2j.persist
package
documentation
for additional details.
Progress
Type
|
P2J
Wrapper Type
|
character
|
character
|
date
|
date
|
datetime |
datetime |
datetime-tz |
datetimetz
(At the moment of writting this there are some issues persisting
multicolumn values with Hibernate.)
|
decimal
|
decimal
|
integer
recid
|
integer
|
int64 |
int64 |
logical
|
logical
|
raw
|
raw
|
Relational Integrity and Natural Joins
An enhanced level of relational integrity is provided during conversion in that
foreign key constaints are added to the database schema where the associated
relations can be detected unequivocally at conversion. Currently, this
detection is limited to the analysis of instances of the
<record> of <table>
construct within a Progress record phrase. Although this is a limited
limited approach, since this construct is simply short-hand for a more verbose
where clause construct, it is the only mechanism currently implemented, because
it is reliable and easily automated (mostly). This analysis and its effect
on schema conversion and DMO creation is described in the
com.goldencode.p2j.schema
package
documentation
.
The introduction of foreign keys unfortunately increases the complexity of the
runtime environment, because these resources now need to be used and managed
transparently (since the pre-conversion application knows nothing of
them). This support is discussed in teh
com.goldencode.p2j.persist
package
documentation
.
Buffer Scoping
To understand the logic and rules of buffer scoping, please see
Record Scoping
.
The implementation of the scoping calculations is done completely during
annotation processing, though it is so complicated that it takes 3 rule-sets to
properly implement. In addition, the complex arrangement of data
structures required to calculate scopes means that several external classes and
a custom worker were necessary to implement this processing in the most simple
manner.
The result is a set of new BUFFER_SCOPE nodes which are each a child node of the
BLOCK to which they are scoped. Each one has a set of annotations
necessary for downstream processing. These annotations include the class
name and Java name of the buffer, among other important information.
Each record and field reference has a "
refid
" annotation that is the AST ID of the BUFFER_SCOPE node to which the reference
refers. This allows the buffer's name to be dereferenced at conversion
time.
Please see the following for more details:
rules/annotations/record_scoping_prep.rules
rules/annotations/record_scoping.rules
rules/annotations/record_scoping_post.rules
BufferScopeTracker
BufferList
BufferBlockState
BufferScopeWorker
Record Buffer Instantiation
The Java AST will get
RecordBuffer
instantiation or declaration nodes in the following cases:
- There is an explicit language statement (define buffer) which would
cause a memory allocation to occur in Progress.
- There is an explicit language statement (define parameter or
parameter) which would cause a buffer declaration to occur in a function or
internal procedure.
- The first implicit reference to a buffer in the source file will
cause a memory allocation of a buffer of the same name as the record.
This can be found by examining all BUFFER_SCOPE nodes and finding those
references that have both the "implicit"
and "first" annotations as true.
The location of the resulting nodes is determined by the following rules:
- A "define buffer" inside an internal procedure, trigger or function
will emit a RecordBuffer.define() in place (in the main block of the
internal procedure, trigger or function). This allocates a new data
model object. Note that this works since the DEFINE_BUFFER nodes are
moved/sorted to always be rooted as the first children of the BLOCK node in
any internal procedure, trigger or function.
- An explicit DEFINE_PARAMETER inside an internal procedure is
emitted to the method definition signature (it won't appear in a function or
trigger).
- An explicit PARAMETER (that is a buffer definition) inside a
function is emitted directly based on peerid (it won't appear in an internal
procedure or trigger).
- All other buffers are instance members (this avoids the need for
differentially constructing some in the external procedure and some as
instance members via "promotion" due to usage across the entire class or due
to being an explicit shared buffer definition).
Import statements for the generic
com.goldencode.p2j.persist
package are always added, as is an import for the custom package(s) that contain
the application-specific data model objects.
The Java AST nodes that are created as a result of this logic are as follows:
- An imported shared buffer (define shared buffer) is emitted as a
buffer declaration with an initialization based on calling SharedVariableManager.lookupBuffer().
- An exported shared buffer (define new shared buffer) is emitted as
a standard buffer definition using RecordBuffer.define() with a second
method call to SharedVariableManager.addBuffer() to export the buffer.
Note that in the case where the buffer is located as an instance member, the
define and the addBuffer will be located in 2 places. In particular,
the addBuffer() will be done in line.
- Implicit buffers and non-shared explicit buffers (define buffer)
are emitted as a RecordBuffer.define().
- Parameters (define parameter and parameter in a function
definition) are emitted in the corresponding method signature as a
REFERENCE_DEF (classname followed by the instance name).
All BUFFER_SCOPE nodes also generate a notification to RecordBuffer.openScope()
to ensure that the database runtime code marks the end of each block for the
associated record release and record lock release.
Please see:
Transaction
Management
in the
persist
package
rules/convert/buffer_definitions.rules
Buffer and Field References
Buffer/record references are emitted in 3 cases:
- The reference is a child of a KW_BUFFER node. This conforms
to the usage of a buffer reference as a RUN statement parameter or function
call parameter.
- The reference's "bufreftype"
annotation is not NO_REFERENCE and the 3rd level ancestor is not PARAMETER.
- The reference's "bufreftype"
annotation is not NO_REFERENCE and the 3rd level ancestor is not
DEFINE_PARAMETER.
The resulting node is a simple Java REFERENCE node with the buffer's Java name.
Field references are always emitted, but these translate into field-specific
getter and setter calls on the data model object. The getter and setter
method names are determined at annotation time.
The assignments rule-set required changes to modify the normal ASSIGN node
processing when the field reference is a setter. This is required to
ensure that the data model object (which is the first child) will emit as the
referent of the setter method call. It is also required such that the 2nd
operand of the ASSIGN node would emit as the 2nd or 3rd child (which would be
the 1st or 2nd parameter to the setter call).
The variable references rule-set was modified to change how subscripts work for
fields. Since Progress fields can be treated as array values, such
parameters must be properly handled as the first integer index parameter of a
getter or setter method call. This is the reason that the 2nd operand of
the ASSIGN node can be either the 1st or 2nd parameter to the setter method
call. The same logic as normal subscripting applies here too (all values
are converted into 0-based indices from the Progress 1-based approach).
Please see:
rules/convert/assignments.rules
rules/convert/database_references.rules
rules/convert/variable_references.rules
WHERE Clause Processing
The following is the mapping of Progress WHERE clause constructs to HQL
(Hibernate Query Language).
Progress
WHERE
|
HQL
|
Notes
|
LPARENS
|
(
|
|
AND
|
and
|
|
OR
|
or
|
|
EQUALS
|
=
|
|
NOT_EQ
|
!=
|
|
GT
|
>
|
|
GTE
|
>=
|
|
LT
|
<
|
|
LTE
|
<=
|
|
BEGINS
|
like '<criteria>%'
|
The 2nd operand's text is
converted into a Java string using ExpressionConversionWorker
progressToJavaString() and then the '%' is appended. Strings
are enclosed in single quotes. |
MATCHES
|
like '<criteria>'
|
The 2nd operand's text is
converted into a Java string using ExpressionConversionWorker
progressToJavaString() and then special matching characters are
converted into the HQL equivalents using ExpressionConversionWorker
convertToSQLLike(). Strings are enclosed in single quotes.
|
CONTAINS
|
n/a
|
|
NOT
|
not ( )
|
The child node of this
operator is wrapped in () to ensure that the Progress precedence is
maintained, since this is different compared to HQL NOT precedence.
|
BOOL_TRUE
|
true
|
|
BOOL_FALSE
|
false
|
|
NUM_LITERAL
|
text is emitted directly
|
|
DEC_LITERAL
|
text is emitted directly |
|
STRING
|
'text'
|
The text is converted into a
Java string using ExpressionConversionWorker
progressToJavaString(). Strings are enclosed in single quotes. |
DATE_LITERAL
|
'YYYY-MM-DD'
|
The text is turned into an
instance of com.goldencode.p2j.util.date and the toStringSQL()
method is used to generate the correct format. The resulting
string is enclosed in single quotes.
Note that there are special
considerations for handling dates within a database where clause.
|
UNKNOWN_VAL
|
is null
is not null
null
|
If the parent note is EQUALS
or NOT_EQ one of the two first forms are emitted. Otherwise
the 3rd form is emitted.
|
FUNC_RECID and oldtype ==
KW_RECID
|
<javaname_of_buffer>.id
|
This builtin function is
directly converted to a reference to the special "id" field.
|
FUNC_ROWID and oldtype ==
KW_ROWID
|
<javaname_of_buffer>.id |
This builtin function is
directly converted to a reference to the special "id" field. |
FIELD_ in the current buffer
|
<buffer_qualifier>.<property_name>
|
|
LBRACKET
|
[]
|
Only FIELD_ extents are
supported.
Subscripts are supported using [ ] but only if the index is a
NUM_LITERAL. Such an index is decremented by 1 and emitted as
text inside the square brackets.
|
Anything not directly supported above is treated as a substitution
parameter. The HQL string will contain a '?' character at the point in the
string in which the runtime substitution should occur. The "root" node of
such a sub-expression is emitted as a runtime expression in the generated
business logic. This result is passed as a query argument. Hibernate
handles the query substitutions using the result of the evaluated runtime
expression.
Other notes:
- All other operators not shown above (e.g. arithmetic such as PLUS,
MINUS, MULTIPLY, DIVIDE, KW_MOD, UN_MINUS, UN_PLUS) are either supported by
custom, server-side database
functions or as client-side
where expressions.
- VAR_*, FUNC_*, METH_* and ATTR_* nodes are not supported.
- FIELD_* references to a different buffer are not supported, unless
this is a multi-table query and the buffer is a preceding one. This
special case is handled using the FieldReference
class.
- Case sensitivity:
- All text used in comparisons is uppercased in the emitted
source code unless CASE-SENSITIVE processing was specified in the
original source code or in the data dictionary for the given field.
- All field references to a character field that are not
CASE-SENSITIVE are wrapped in the "UPPER()" function.
- Operand positions are not meaningful. This means that
right/left operands require no normalization or other special processing.
- Direct boolean operands are legal for AND, OR, LPARENS, NOT and
even as the root node of the WHERE clause. All of these forms are also
legal in HQL so no normalizations are required here.
- Direct self-references on supported operators are forwarded to the
server in HQL.
- Non-literals as field extents are not supported at this time.
Database Language Statements
Java classes referenced in the table below reside in the
com.goldencode.p2j.persist
package unless otherwise noted.
Statement
|
Java
Equivalent
|
Notes
|
ACCUM
|
Accumulator (abstract base
class); concrete implementations:
- AverageAccumulator
- CountAccumulator
- MaximumAccumulator
- MinimumAccumulator
- TotalAccumulator
|
- reside in the
com.goldencode.p2j.util
package (following the Progress semantic, these classes are not
strictly database related; they can be used standalone
with variables or added
to a query)
- same classes are used to handle both cumulative results
and break group results
|
BUFFER COMPARE
|
RecordBuffer.compare
|
|
BUFFER COPY
|
RecordBuffer.copy
|
|
CLOSE QUERY
|
n/a
|
P2J queries are not explicitly
closed; this statement is dropped
|
CONNECT
|
ConnectionManager.connect
|
|
CREATE
|
RecordBuffer.create
|
No USING support at this time.
In Progress, CREATE does not immediately flush the new record to the
database, since in most cases the default values assigned to a new
record would violate uniqueness or nullability constraints.
Instead, the record is not written to the database until the first
statement is executed, which would update an index for the record's
table.
In our implementation, we defer writing the record to the database
(or more accurately, persisting the record using Hibernate;
actual flushing to the database may be further deferred by
Hibernate), until the earliest of the following occurs:
- all properties participating in ANY uniqueness
constraint are updated (the last property updated in such a set
triggers validation and flushing);
- a query which operates against the record's table is
executed;
- the current record in the record buffer is replaced or
released;
- a transaction or subtransaction commit occurs;
- the end of a converted ASSIGN statement is reached
(RecordBuffer.endBatch() is invoked).
Note that our implementation departs slightly from Progress --
particularly with regard to #1 in the list above -- in that the
setting of a property which does NOT participate in a unique
constraint (but DOES participate in a non-unique index) WILL NOT
trigger a buffer flush. However, we believe the remaining
conditions adequately make up for this departure.
|
CREATE ALIAS
|
ConnectionManager.createAlias
|
|
CREATE BUFFER
|
|
TBD; dynamic buffer
creation is not supported in the first release
|
CREATE DATABASE
|
|
TBD; dynamic database
creation is not supported in the first release |
CREATE QUERY
|
|
TBD; dynamic query
creation is not supported in the first release |
CREATE TEMP-TABLE
|
|
TBD; dynamic temp-table
creation is not supported in the first release |
DEFINE BUFFER
|
RecordBuffer.define
|
see Record
Buffer Instantiation
|
DEFINE QUERY
|
AbstractQueryWrapper (abstract
base class); concrete implementations:
- CompoundQueryWrapper
- PreselectQueryWrapper
- RandomAccessQueryWrapper
|
These wrapper classes are used
to support the DEFINE QUERY semantic of defining a query once and
possibly opening it multiple times. An instance is constructed
once, as an instance variable, corresponding with the DEFINE QUERY
statement. Each corresponding OPEN QUERY is converted as an
instance of a concrete subclass of AbstractQuery, which is set into
the wrapper when opened.
|
DEFINE TEMP-TABLE
DEFINE WORK-TABLE
|
TemporaryBuffer.define |
no distinction is made between
temp-tables and work-tables at conversion; both are converted
to DMO class definitions in the <...>.dmo._temp
package (DMO interfaces) and the <...>.dmo._temp.impl
package (DMO implementation classes) in the converted application
hierarchy
see also Temporary
and Work Table Support
|
DELETE
|
RecordBuffer.delete
|
no VALIDATE support at this
time
|
DELETE ALIAS
|
ConnectionManager.deleteAlias
|
|
DISCONNECT
|
ConnectionManager.disconnect
|
|
FIND (standalone)
|
FindQuery
AdaptiveQuery
|
|
FIND (as preselect retrieval
mechanism)
|
P2JQuery.first
P2JQuery.last
P2JQuery.next
P2JQuery.previous |
|
FOR/DO/REPEAT
|
AbstractQuery (abstract base
class); concrete implementations:
- CompoundQuery
- PreselectQuery
- AdaptiveQuery
- PresortQuery
- RandomAccessQuery
|
Which concrete implementation
is chosen at conversion time depends upon the number of tables
involved and upon the nature of the query.
For an individual table, PreselectQuery is chosen if it is defined
explicitly in Progress using the PRESELECT keyword or if the
Progress construct's explicit sort criteria do not match the index
selected according to index selection rules. PresortQuery is
chosen if client-side sorting is required (i.e., the by clause
contains an expression which cannot be converted to an HQL 'order
by' clause, but must instead convert to a runtime expression).
Both of these types can be applied in a multi-table join situation.
AdaptiveQuery is chosen by default for a single table query, if
other factors do not force a preselect variant. This query
operates in preselect mode until some change to a record forces it
to operate in dynamic mode. As soon as it can switch back to
preselect mode, it will.
RandomAccessQuery is used by an AdaptiveQuery when switching to
dynamic mode. This is the most flexible query type (but also
has the worst performance); it is necessary to support the
Progress "dynamic" record retrieval semantic, where updates to a
record made during the loop can change the results found later in
the loop (an effect [side effect?] of Progress' index
implementation).
CompoundQuery is chosen for nested FOR loop conversions, where the
component loops would convert to AdaptiveQuery instances or to a
mixture of adaptive and preselect query variants.
CompoundQuery is a driver to which Joinable (implemented by
RandomAccessQuery and PreselectQuery) query instances may be added
as components.
|
GET
|
P2JQuery.first
P2JQuery.last
P2JQuery.next
P2JQuery.previous
|
|
RELEASE
|
RecordBuffer.release
|
|
REPOSITION
|
RandomAccessQuery.reposition
|
|
OPEN QUERY
|
AbstractQuery (abstract base
class); concrete implementations:
- CompoundQuery
- PreselectQuery
- RandomAccessQuery
|
in the event of an OPEN QUERY
which references a query previously declared by a DEFINE QUERY, one
of these concrete classes is set into the corresponding concrete
wrapper implementation; otherwise, an instance of one of these
classes is used directly
|
VALIDATE
|
RecordBuffer.validate
|
|
Built-in Function Support
See all entries in
this table
with type "database
essimistic Record Locking Progress relies upon a pessimistic locking strategy
consisting of three record locking modes:
- NO-LOCK (the lack of a lock);
- SHARE-LOCK (other entities may read the record but may not upgrade
to a writable lock);
- EXCLUSIVE-LOCK (other entities may acquire neither a share nor an
exclusive lock on the record).
References to these keywords have been converted to use constants defined as
static variables of the
LockType
class. These constants are used by the persist package internally.
They appear in converted application code as arguments to various query
constructors and initialization methods, and in record retrieval methods (e.g.,
first
,
last
,
next
,
previous
,
current
, and
unique
), to correspond to explicitly specified lock requests in Progress record
phrases.
For information regarding P2J's runtime pessimistic locking strategy, please
refer to the
persist
package documentation
.
Please see:
rules/convert/database_access.rules
Temporary and Work Table
Conversion
Temp-tables and work-tables convert to the same temporary table
implementation; there is no distinction between the two in the P2J
environment, so both are referred to generically as "temp tables" or "temporary
tables" in this document. Temp table structural information is extracted
from DEFINE TEMP-TABLE and DEFINE WORK-TABLE statements in the Progress source
code. Each unique definition is converted into a specialized DMO interface
and implementation class. A Hibernate mapping document is created for
each, and each receives an entry in the runtime
dmo_index.xml
document, which resides at the relative root of the
dmo
sub-package in the converted application's package hierarchy.
A temp table definition is considered unique if its structural organization and
content (i.e., field and index elements) is unique across the application.
Currently, Hibernate contains no direct support for temp tables; however,
it does not preclude that Java objects be mapped to temporary database
tables. Furthermore, Hibernate requires that a Java object be mapped to
only one backing table within a single SessionFactory context. It also
requires that these mappings be known at SessionFactory configuration, which for
practical purposes (it is quite expensive) must occur infrequently and
preferably during server initialization. For these reasons, only the
conversion of statically defined temp tables are supported at this time.
Please see the
persist package
documentation
for more information about runtime temp table support.
Please see:
rules/convert/buffer_definitions.rules
rules/convert/database_access.rules
Client-Side Where
Clause Processing
Certain where clauses in Progress cannot be converted to server-side queries
expressed entirely in HQL. Specifically, if a where clause contains a
nested reference to a field in the record being retrieved, it must be tested
against a client-side expression to determine whether it will be included in the
query result. By "nested", we mean embedded within a subexpression which
cannot be expressed in HQL. For instance, a field reference to the current
buffer as a parameter to to a built-in or user function would qualify for
client-side where clause processing:
def var snippet as character initial "whatever".
for each customer
no-lock,
each invoice
where invoice.cust-num = customer.cust-num
and substring(invoice.category, 10, length(snippet)) = snippet
no-lock:
...
Although this is a trivial example, which could be expressed differently, it
serves to illustrate the point: the
customer.name
reference embedded within the built-in function
substring
causes this code to convert to Java code which executes a client-side where
clause expression.
Progress where clauses which qualify for this type of handling convert, where
possible, to a server-side component and a client-side component. The
server-side component is expressed in HQL and the client side component converts
to an anonymous inner class which extends the abstract class
WhereExpression
. At runtime, the query is executed (at a high level) in two phases:
- a server-side query which returns a preliminary, candidate result;
- a client-side expression which filters the candidate results.
The server-side query is necessary to restrict, to the extent possible, the
amount of data pushed to the client for final evaluation, since this is likely
to be the most expensive component of the process. The greater the level
of restriction performed at the server, the better. That being said, it is
not always possible to generate HQL which accomplishes this goal. In the
worst case scenario, no server-side restriction will be possible at all, and
every record in the table will be retrieved and evaluated on the client.
Server-side restriction is possible only if the a top-level operation in the
where clause expression is a logical AND, and at least some portion of one side
of the expression can be evaluated at the database server. In the above
example, the
invoice.cust-num = customer.cust-num
component of the where clause can be expressed in HQL as
invoice.cust-num = ?
, where
?
is substituted at runtime with each
customer.cust-num
value found in the outer loop.
The client-side portion of this expression is converted to an anonymous, inner
subclass of
WhereExpression
, as the following code segment illustrates:
character snippet = new character("whatever");
WhereExpression whereExpr0 = new WhereExpression()
{
public Object[] getSubstitutions()
{
return new Object[]
{
new IntegerExpression()
{
public integer execute()
{
return snippet.length();
}
},
snippet
};
}
public logical evaluate(BaseDataType[] args)
{
return CompareOps.equals(character.substring(invoice.getCategory(), 10, (integer) args[0]), (character) args[1]);
}
};
During the client-side filtering phase of query execution, the
evaluate()
method defined above is invoked for each
invoice
record retrieved by the server-side phase. The
WhereExpression
class and the various query implementations ensure that the
arguments/expressions returned by
getSubstitutions()
are resolved at the appropriate times during execution of the query loops (this
is important, since it preserves the Progress semantic by ensuring that any side
effects of invoking built-in or user functions -- such as state changes -- are
maintained). The resolved results of these arguments/expressions compose
the
args
parameter passed to
evaluate()
. If
evaluate()
returns
true
, the current
invoice
record is retained as a final query result; if it returns
false
, the record is dropped from the query's final results.
NOTE:
the conditions which trigger client-side where clause processing as
discussed above have been greatly reduced since this section was first
written. Since that time, we have introduced and implemented many
server-side (i.e.,
database
server, not P2J server) functions which minimize the circumstances which require
client-side where clause processing. Since this implementation, the only
where clauses that require client-side where clause processing are those which
integrate user-defined functions, because these may have side effects which
cannot be predicted and must execute on the client. All
built-in
Progress functions which are used in where clauses in the initial project to
which P2J is targeted have now been implemented as server-side functions.
Please refer to the following package documentation for additional details on
this implementation. The information in the following package summaries
take precedence over the information above:
Please also see:
rules/annotations/where_clause_pre_prep.rules
rules/annotations/where_clause_prep2.rules
rules/annotations/where_clause.rules
rules/annotations/where_clause_post.rules
rules/annotations/where_clause_post.rules
rules/convert/database_access.rules
Database Validation Expressions
Validation expressions and messages can be defined for database fields (and
tables, though this is not supported at this time). Field-level validation
expressions are used by default in certain blocking, input, UI language
statements. Application code may provide an explicit override (using the
VALIDATE keyword), in which case the default validation expression is ignored.
Any
Progress code which is valid within the context of the business logic in which
the corresponding field reference is made can be used within a validation
expression. Therefore, although this would be very bad form, it is legal
for a validation expression to contain references to local variables and other
constructs valid only within the enclosing scope of the field reference.
This is problematic for conversion, because it requires that validation
expressions emulate this behavior in Java code. To accomplish this, we
duplicate the conversion expression as a subclass of
Validatable
, implemented as an inner class within
each
business logic class in which the field must be validated.
Although these expressions are defined at the database schema level, they are
evaluated only by the UI.
Field-level validation expressions
defined in the schema are not triggered by simple assignments
. The following UI language statements trigger field-level validation:
- ENABLE
- INSERT
- PROMPT-FOR
- SET
- UPDATE
It is the first occurence of a reference to a validatable field, in association
with one of these statements, which triggers the
Validatable
implementation to be emitted in the target business class. It also
triggers an instance of the
Validatable
implementation class to be created and registered with the UI widget associated
with the validatable field. In addition to the above listed statements,
the FORM statement is given special consideration, because it may contain the
first reference in a frame to a validatable field, even though it does not
itself trigger validation. Surprisingly, the SET and UPDATE options of the
MESSAGE statement
do not
trigger field-level validation.
Please see:
rules/schema/p2o.xml
rules/fixups/schema-validations.rules
rules/annotations/validation-prep.rules
rules/annotations/validation.rules
rules/annotations/validation-post.rules
rules/convert/ui_statements.rules
rules/convert/expression.rules
Methods and Attribute Support
At this time, the following subset of database-related methods and attributes
are supported:
Method/Attribute
|
Java
Equivalent
|
Notes
|
CURRENT-RESULT-ROW
|
P2JQuery.currentRow()
|
|
GET-FIRST()
|
P2JQuery.first()
|
|
GET-LAST()
|
P2JQuery.last()
|
|
GET-NEXT()
|
P2JQuery.next()
|
|
GET-PREV()
|
P2JQuery.previous()
|
|
HANDLE
|
P2JQuery instance reference
|
There is no method equivalent
in Java for this attribute; the "handle" of a P2JQuery object
is the object reference itself. Therefore, the conversion of
Progress code which dereferences the handle attribute from a query
object is simply to emit the Java query object reference.
|
QUERY-OFF-END
|
P2JQuery.isOffEnd()
|
|
QUERY
|
N/A
|
Use of this attribute of
browse is superfluous in Java, in the customer cases encountered
thus far. Therefore, it is simply removed from the tree in a
customer-specific annotation ruleset. If it is determined that
this attribute is universally superfluous, this processing will be
relocated to a common ruleset.
|
Known Issues
- When generating HQL expressions from a Progress where clause where
a character comparison is necessary,
the case sensitivity of a
field or variable which is substituted at runtime for an HQL query
substitution marker is only maintained if that field or variable is a direct
operand of the comparison. For instance, the Progress where clause customer.name
= invoice.cust-name
might convert to an HQL expression upper(?)
= upper(invoice.cust-name)
, if both fields are case-insensitive.
However, the case sensitivity of customer.name
may not be
considered in the HQL generation if the field reference is more deeply
nested. For example, the Progress where clause substring(customer.name,
0, 5) = invoice.cust-name
would convert to the same HQL phrase shown earlier
if invoice.cust-name
is case-insensitive,
even if customer.name
was case-sensitive. This is considered
to be a relatively obscure case; it will be addressed as needed.
- For searches using HQL expressions which have an embedded,
non-literal (i.e., dynamic), subscript reference (e.g.,
where
myBuf.myField[?] = ...
), P2J diverges from Progress in the type of error
raised for a subscript out of bounds condition. Because the query
substitution is handled on the database server using this notation,
Hibernate simply comes back with an empty result set in the case where the
subscript is out of bounds, which will result in an error condition with
text ** <XXXX> record not on file. (138)
when a null DMO
is set as the current record in a record buffer. Progress raises a
stop condition with text ** Array subscript <X> is out of
range. (26)
.
User Interface Support
Basic Design
All static elements of the UI are removed from the business logic and used to
create 2 UI specific classes for each unique frame:
- A custom screen buffer interface with getters, setters and widget
accessors for the associated frame.
- A frame definition that can be used to instantiate and initialize a
frame of the given layout/fields/configuration and its contained widgets.
These classes are used at runtime by the business logic. In particular,
the control flow is defined in the business logic. This means that the
controller of the UI is in the business logic, but the format, layout ... of the
UI itself has been separated into UI specific classes.
The behavior of DISPLAY/UPDATE/SET/PROMPT-FOR is special because these
statements depend upon a passed array of
FrameElement
instances which define the list of frame fields that are to be processed.
Each FrameElement is one of the following subclasses:
- Element - a displayable data value
and the widget it is associated with
- WidgetElement - a widget
with no data (e.g. a button)
- EmbeddedAssignment - a
mechanism to delegate the execution of an assignment (and dynamic evaluation
of the associated expression) during the assignment processing in these
language statements.
For each of the statements the copy to/from the screen-buffer, the widget
enable/disable, the view(), the wait-for() are all hidden in the runtime.
This processing is mapped into primitive operations but the logic is properly
enforced in these higher level APIs such that the Progress semantic is
maintained. Please see
GenericFrame
for more details.
The rules/annotations/frame_scoping.rules is the core rule set that calculates
the frame scoping, widget lists for each frame and many other critical
annotations. The majority of the UI related business logic annotations are
done in rules/annotations/screen_buffer.rules.
The rules/convert/frame_generator.xml is a standalone rule set that is run
between annotations and code conversion. This rule set uses the previously
stored annotations to generate the 2 classes above per frame. The majority
of the UI related business logic code conversion is done in
rules/convert/ui_statements.rules.
UI Language Statements
Statement
|
Java
Equivalent
|
Notes
|
APPLY
|
CommonFrame.apply()
LogicalTerminal.apply()
|
|
ASSIGN (input buffer form) |
Rewritten using the INPUT
function such that this is converted as an explicit assignment
statement that directly reads the field's getter method in the
screen buffer.
|
|
BELL
|
LogicalTerminal.bell()
|
|
CHOOSE
|
CommonFrame.choose() |
|
CLEAR
|
CommonFrame.clear()
CommonFrame.clearAll() |
|
COLOR
|
LogicalTerminal.setColors()
|
|
CREATE WIDGET
|
not converted at this time |
|
CREATE WIDGET-POOL
|
not converted at this time |
|
DEFINE BROWSE
|
new BrowseWidget() |
|
DEFINE BUTTON
|
new ButtonWidget()
|
|
DEFINE FRAME
|
new FrameWidget() |
|
DEFINE RECTANGLE
|
not converted at this time |
|
DELETE OBJECT
|
not converted at this time |
|
DELETE WIDGET
|
not converted at this time
|
|
DISABLE
|
GenericWidget.disable()
CommonFrame.disable()
CommonFrame.disableExcept()
|
|
DISPLAY
|
CommonFrame.display() |
|
DOWN
|
CommonFrame.down() |
|
ENABLE
|
GenericWidget.enable()
CommonFrame.enable()
CommonFrame.enableExcept()
|
|
FORM
|
Converted into a frame
definition class.
|
|
FRAME-VALUE
|
LogicalTerminal.setFrameValue()
|
|
HIDE
|
LogicalTerminal.hideMessage()
LogicalTerminal.hideAll()
CommonFrame.hide()
|
|
INPUT CLEAR
|
LogicalTerminal.inputClear()
|
|
MESSAGE
|
LogicalTerminal.message()
LogicalTerminal.messageBox()
|
|
NEXT-PROMPT
|
CommonFrame.nextPrompt()
|
|
ON
|
LogicalTerminal.registerRunnable()
LogicalTerminal.deregisterRunnable()
LogicalTerminal.remapKey()
|
|
PAUSE
|
LogicalTerminal.pauseBeforeHide()
LogicalTerminal.pause()
|
|
PROCESS-EVENTS
|
LogicalTerminal.processEvents()
|
|
PROMPT-FOR
|
CommonFrame.promptFor()
|
|
PUT-CURSOR |
LogicalTerminal.putCursor()
|
|
PUT-SCREEN |
LogicalTerminal.putScreen() |
|
READKEY
|
KeyReader.readKey()
|
|
SCROLL
|
CommonFrame.scroll()
|
|
SET
|
CommonFrame.set()
|
|
STATUS
|
LogicalTerminal.statusDefault()
LogicalTerminal.statusInputOff()
LogicalTerminal.statusInput()
|
|
TERMINAL
|
LogicalTerminal.setTerminal()
|
|
UNDERLINE
|
LogicalTerminal.underline()
CommonFrame.underline()
|
|
UP
|
CommonFrame.up()
|
|
UPDATE
|
CommonFrame.update()
|
|
VIEW
|
CommonFrame.view()
|
|
WAIT-FOR
|
LogicalTerminal.waitFor()
|
|
See also:
rules/annotations/trigger_prep.rules
rules/annotations/set_update_embedded_assignments.rules
rules/annotations/frame_scoping.rules
rules/annotations/screen_buffers.rules
rules/annotations/validation_prep.rules
rules/annotations/validation.rules
rules/annotations/when_rewriting
rules/annotations/dynamic_ui_indexes
rules/convert/ui_statements.rules
Special UI Block Types/Callbacks
The Progress UI contains 2 special block types (trigger blocks and editing
blocks) and 1 pure callback (validation expressions). All 3 of these
constructs are tightly integrated with language statements that interact with
users.
Triggers are blocks that are executed when defined events occur in the user
interface. Instead of defining triggers like internal procedures, triggers
are defined "in line" with the procedure or function being processed.
Though Progress treats a trigger as a distinct block type which has block
properties (
see above
), it is not well seperated in terms of location in source code.
Triggers take no parameters and are not named. Instead, they are
registered as a set of events and a set of widgets which cause the trigger to be
called. When any of the listed events occurs for any of the listed
widgets, the trigger will be executed. Multiple triggers can be registered
for the same event + widget combinations. The last registration "wins" (is
active). The scoping for such registrations is at the procedure
(internal/external), trigger and function level. When such a new scope is
opened, duplicate trigger registrations will hide registrations from previous
scopes. Likewise, the closing of a scope will remove all triggers
registered in that scope, which will make previously hidden triggers
"re-appear". Within a single procedure, function or trigger (yes, triggers
can be nested in other triggers), duplicate registrations hide previous
registrations and this will not implicitly ever be undone (without the entire
scope exiting and thus removing all triggers registered in that scope).
The explicit REVERT option can be used to deregister a specified trigger and
then the trigger registered most recently (in this scope or a previous scope)
will become the active trigger for that given event + widget combination.
One other interesting feature of trigger registration is that it is strictly
based on the flow of control of the registering code. If the flow of
control never executes the branch of code in which a trigger is defined (e.g. an
ELSE block that is never executed), then that trigger is not registered.
This means that the registration of triggers must be kept strictly in line with
the matching control flow, even if the trigger block itself is refactored into
another location.
Editing blocks are really special purpose key-press processing loops that are
called in-line from the PROMPT-FOR processing (and from UPDATE/SET since they
have an embedded PROMPT-FOR). Such blocks are invoked before any other
event processing associated with a key press. They can override or modify
all event processing associated with the key press (editing blocks take
precedence over all other forms of event processing). The editing
block is peculiar in that it is completely integrated with the surrounding code
(the block structuring of the surrounding code). In particular, this means
that one can execute low-level flow control statements such as NEXT or LEAVE and
these can directly reference enclosing block names. Although it may not be
obvious, such blocks are in fact loops and one can use NEXT to force an
iteration (restart from the top of the block). Such blocks do have block
properties (
see above
) but they are limited and cannot be customized (with the "ON clause").
Validation expressions are not a block type but instead are a direct evaluation
of a logical expression to determine if the content of an associated field is or
is not valid. The expression is evaluated in the context of the defining
program but it called when a field is changes in the UI and the user attempts to
leave the field or screen. If the validation expression returns FALSE, a
second expression is evaluated to obtain message text and this message is
displayed to the user. The user is then placed back into that field (or
just not allowed to leave) and is provided an opportunity to edit the field
again. If the validation expression evaluates TRUE, then the field is
considered valid. If no edits are made to a field but the screen is
exited, all fields that had not yet been validated will be validated. The
first validation expression to fail will display the message and place the user
back into the associated field to edit it. Of particular interest is the
fact that both the validation expression itself and the expression used to
generate the message text are real expressions and can contain the full variety
of function calls, operators, constants and resource references one would expect
in a Progress expression.
In terms of scoping, all 3 constructs can access resources (vars, streams,
buffers and frames) that are defined in the enclosing scope. This
complicates the generation of code immensely since the rules for scoping in Java
are much more strict. A great deal of processing was implemented in
annotations to ensure that all such resources are "promoted" to instance members
in the generated business logic classes. This allows triggers and
validation expressions to be emitted as inner classes while still accessing all
the resources in the proper context. In particular, streams, buffers and
frames were all modified to ensure that *all* instances of such resources have
unique names and thus all instances can always be made into instance
members. With variables, this was not desireable since the variables are
so common and keeping a scoped approach yields a better result. So
variables are analyzed to determine which need promotion. This promotion
logic is in rules/annotations/scope_promotion.rules.
Triggers are implemented as a named inner class (each one gets a generated name)
which extends Runnable. The trigger block itself is emitted inside the
run() method and it is treated as a top-level sub-transaction block in terms of
the
TransactionManager
. All local resources are promoted to instance members in the containing
class so these resources are directly accessible. A instance of the inner
class is constructed (default constructor) and passed to the registration
method, along with a definition of the event + widget combinations that are
valid. These event list + widget list combinations are defined as an
instance of the WaitList class.
Editing blocks (loops actually) are implemented in-line with the original
containing logic (at the location of the UPDATE/SET/PROMPT-FOR statement in the
original control flow). This is required to ensure that the integrated
control flow processing (e.g. NEXT, LEAVE, RETURN...) occur in the proper
context. Java has no way to provide such direct control flow processing
(such as break, continue or return) without the editing block being defined as a
nested block in-line. This requires "inverting" the normal UI
processing. Normally, the server's UI runtime drives the logical terminal
based on well known rules AND most importantly without any return to the calling
code. In this case, the calling code really has control and must drive the
UI processing. This block is treated as a nested loop (e.g. like a REPEAT
inner block) which is a sub-transaction in TransactionManager terms.
The emitted block handles ERROR and retry but END, STOP and QUIT are ignored
(handled only by containing blocks). Special methods of the referenced frame are
called at specific points to ensure that the UI is invoked at the proper times:
- CommonFrame.startEditingMode() - this code is executed once at
entrance to the block. The UI uses this to initialize and prepare for
the "inverted" UI processing.
- CommonFrame.waitForNextKey() - this code is executed at the start
of every iteration. It returns when there is a key available for
processing.
- CommonFrame.continueEditing() - this code is executed at the end of
every iteration. It allows any pending GO events to be processed and
state to be maintained properly. This is critical to the processing of
NEXT and any retry (due to the ERROR condition being raised) as well, since
in such cases any pending GO event must be cleared.
Validation expressions are emitted as inner classes that extend the Validator
class. This is a class that implements the Validatable interface.
Each validation expression (as defined in a format phrase for a variable/field
override or as defined in the data dictionary for a database field) is
registered with the associated widget and is thus called back as needed during
the UI processing. If a field has a default validation expression in the
data dictionary AND there is no override in the business logic, then a factory
method (RecordBuffer.getValidatable()) is used to obtain the Validatable
instance to be registered. Any override for a database field in the
business logic will result in an inner class being generated and registered,
just as for variables.
The validation expression itself almost always will have a reference to the
current field or variable AND this reference must resolve to the latest or
proposed version of the variable/field. To handle this, the new value is
passed as a parameter in the Validatable interface. The emitted inner
class casts this to a local variable of the proper type and name such that the
expression will naturally access the local version in its expression
evaluation. This also handles the special case in Progress where an array
element is referenced in a validation expression without the associated
subscript.
See also:
CommonFrame.java
GenericFrame.java
LogicalTerminal.java
ThinClient.java
WaitList.java
Validatable.java interface
Validator.java
rules/annotations/trigger_prep.rules
rules/annotations/validation_prep.rules
rules/annotations/validation.rules
rules/convert/ui_statements.rules
Key and Condition Processing
The thin client has a separate thread that handles key reading from the
terminal. This reader thread provides typeahead functionality and polls
the keyboard every 100ms. It can be suspended and resumed, which is
necessary while handling redirected input streams. These are cases where
any interactive input to the terminal must be routed to the stream in use (often
a child process, for example a "vi" editor session) and thus the thin client's
key reading cannot be done simultaneously.
The key reader thread fills a queue of input events. This queue is read by
the thin client during editing mode (anything that calls WAIT-FOR) and during an
explicit READKEY. When read during WAIT-FOR, the key events are processed
in the widgets via APPLY. READKEY bypasses all such APPLY processing and
returns the latest key from the top of the queue.
The key reader uses features in the CHARVA Toolkit class (some of which are
backed by native methods that use NCURSES) to read the keyboard in "raw" mode
with the "keypad" features activated. This means that all keystrokes such
as CTRL-C will be read without signals being generated. The "keypad"
support means that NCURSES will convert special ("extended") keys like function
keys or numpad keys into simple single keys rather than passing through the
multiple scancodes that the keyboard actually generates.
CTRL-C is handled specially. It generates a STOP condition AND this works
in both synchronous and asynchronous modes. While processing is occuring
on teh thin client, a STOP condition is raised directly on the client and this
exception unwinds the stack back into the business logic on the server.
When processing is active on the server, the key reader thread will
asynchronously interrupt the thread on the server (via a callback from the key
reader thread to the server) which will raise the STOP condition there.
This happens using the current thread's "interrupted status" and the interrupt()
method in Thread. If server's thread is blocked in some form of
wait, then an InterruptedException, IOInterruptedException or other related
exception will be generated and this will be caught and converted into a
StopConditionException. All such locations have been instrumented with the
proper catch blocks to ensure this happens. If the server thread is not
blocked in a wait, the interrupted status is set and at critical points (start
scope, iterate and stop scope) for each block the TransactionManager checks this
status and throws the StopConditionException is the thread was previously
interrupted.
See:
KeyReader.java
ThinClient.java
Limitations
- ENABLE/DISABLE/SET/UPDATE/PROMPT-FOR/DISPLAY
- The KW_UNL_HID (unless-hidden keyword) is not supported.
- Certain runtime configuration in the FRAME_PHRASE and
FORMAT_PHRASE is not supported: AT COLUMN-OF/ROW-OF/X-OF/Y-OF.
- INSERT - no support exists.
- No "trigger phrase" support. Regular triggers are fully
supported but the "inline" triggers implemented in a variable or widget
definition are not implemented at this time.
- Validation expressions for an array element that is referenced in
the expression via subscript will fail.
Unreachable Code
Processing
The purpose of the Unreachable Code Processing (UCP) is twofold:
- To eliminate code which during conversion may result in generation
of Java code which will be considered unreachable by javac. Such code causes
compilation errors and therefore must be avoided.
- To eliminate code which can never be accessed and thus is not
actually part of the application. This reduces the amount of
conversion that must be done AND more importantly it yields a much better
resulting application since it is smaller and easier to understand.
The reachability of statements depends on flow control statements such as IF,
REPEAT, NEXT, LEAVE and so on. These statements change execution flow and under
some circumstances may change reachability of code.
The approach used to determine reachability of code is simple: all nodes of the
AST tree are visited and analyzed in order to determine if they can affect
reachability of subsequent statements. The first node of the AST is always
reachable.
Table below lists some examples of unreachable code:
Progress
Code
|
Java
Equivalent
|
Notes
|
repeat i = 1 to 10: ... leave. message "Hello!". end.
|
for (i = 1; i <= 10; i+= 1) { ... break; }
|
The code after LEAVE is not
reachable.
|
repeat i = 1 to 10: ... if i > 7 then leave. else next. message "Hello!". end.
|
for (i = 1; i <= 10; i+= 1) { ... if (i > 7) break; else continue; }
|
The code after IF statement is
unreachable.
|
label_xx: repeat: ... case i: when 1 then leave label_xx. when 2 then next label_xx. otherwise return. end. ... end.
|
labelXx: while(true) { ... switch(i) { case 1: break labelXx; case 2: continue labelXx; default: return; } ... }
|
The code after CASE statement
is unreachable.
|
Statements
A significant number of Progress statements may affect reachability of the other
statements. Some of the statements do that directly, because they change
execution flow (e.g. IF, CASE or LEAVE). Other statements, such as the
PROCEDURE, FUNCTION or ON statements do not change execution flow. Instead they
result in marking code as always reachable because its reachability can't be
checked or depends on external conditions.
The processing of individual statements is described below:
- NEXT, LEAVE, RETURN, QUIT,
STOP
These statements stop the execution flow and make remaining code in the AST
subtree unreachable. Note that a reachable LEAVE statement makes
code after the loop which contains this LEAVE statement, reachable. If the
LEAVE statement contains a label then code after the labeled loop will be
marked as reachable.
- PROCEDURE, FUNCTION and ON
statement
The procedure file may be a mixture of the procedure itself and internal
PROCEDURE and FUNCTION statements. The reachability of these pieces of code
must be estimated independently. This is necessary because a particular
PROCEDURE or FUNCTION can be reachable even if it is located in the middle
of unreachable code. The same is true for the ON statement.
- IF
The IF in general case can make other code unreachable only if both
THEN and ELSE clauses are present and both make remaining code unreachable
(for example, end with LEAVE or RETURN statements). Other special case is
the situation when IF statement condition can be easily calculated. In this
case code in the THEN or ELSE clauses of the IF statement can be
unreachable. Also, if code in the reachable part ends with the statements
from listed in #1, then code after IF statement will be unreachable.
- REPEAT, DO, FOR loops
These loops may have easily calculable or implicit (REPEAT loop) condition
which may result to infinite loop or loop which does not allow execution of
code after the loop (there are exceptions, see #7).
- CASE
The CASE statement can make other code unreachable only if it contains THEN
and OTHERWISE clauses and all clauses end with statements listed in #1.
- EDITING and TRIGGERS
phrases
These phrases may (and usually do) contain code which ends with statements
listed in #1. Direct processing of such statements may result in incorrect
marking of large portions of code as unreachable. In order to avoid this,
after processing of these statements the reachability flag is restored to
the state which it had before processing of these statements.
- STOP,
ERROR, QUIT and ENDKEY conditions
These conditions may change execution flow and therefore they must be taken
into account during processing. In particular, some loops (FOR and REPEAT)
have default processing for these conditions. Also, all loops may have
explicitly specified condition handling specified with ON phrases. This
handling may include LEAVE or LEAVE <label> statements and therefore
change reachability of the code after loops.
- Statements which may raise
STOP, ERROR, QUIT and ENDKEY conditions
These statements do not need special handling except that presence of such
statements must be tracked because they may affect processing described in #7.
The expression evaluation mentioned in the #3 and #4 may calculate simple
expressions such as:
- TRUE
- FALSE
- variable = "value"
- "value" = variable
- variable <> "value"
- "value" <> variable
Please note that at this time, the expression evaluation of variables is limited
to variables of type VAR_CHAR being compared (EQUALS or NOT_EQ) with a literal
STRING. Changes to UnreachableCodeWorker will be needed to provide a wider
range of support in the future (variable type and value type information would
have to be stored and used in the variable pool and evaluation respectively).
Values and variables are stored in a customer-specific rule set named
customer_specific_variables.rules. This allows customer specific character
variables to be specified with a list of values that can possibly evaluate to
true
. This allows such simple comparisons to be evaluated statically, yielding a
increase in known unreachable code.
See also:
UnreachableCodeWorker
rules/unreachable/unreachable.xml
Produced Output
The results of unreachable code processing are stored directly in the AST: each
AST node receives annotation "reachable" which contains a boolean value
true
or
false
, representing the reachability of the code. Also, information about each
unreachable node is printed to standard output. This output then can be stored
in the log file and then used for analysis.
Known Limitations
- At present TRIGGER PROCEDURE and SUBSCRIBE statements are not
supported.
Unreferenced Tables/Fields
Processing
The main purpose of the Unreferenced Tables/Fields Processing (UTFP) is to
minimize amount of data which need to be moved to converted system by
eliminating unused tables and fields.
The general approach of UTFP is simple: scan ASTs and collect references to
tables and fields. This approach is complicated with following issues which need
to be handled:
- Fields referenced in the LIKE clause should not be taken into
account.
- An application may create a temporary table as a structural copy of
an existing table. This case should NOT be handled as if original table and
its fields are referenced. The reason is that this temporary table
definition will be statically defined in the target application based on the
structure of the Progress table however that table does not need to exist in
the target database in order for the temporary table to be supported.
- Some statements or variants of statements may reference entire
table (i.e. all fields in table). In some cases such a statement may
explicitly exclude some fields from the list of all fields.
- Tables and fields can be accessed indirectly via system tables such
as _File.
List below contains all statements which may reference all fields in table:
- ASSIGN record [EXCEPT field]
- BUFFER-COMPARE [EXCEPT field] 1,2
- BUFFER-COPY [EXCEPT field] 1,2
- DEFINE BROWSE ... DISPLAY ...
- DEFINE FRAME ... [EXCEPT field]
- DEFINE QUERY ... EXCEPT 1,3
- DISABLE ... ALL [EXCEPT field]
- DISPLAY ... [EXCEPT field]
- ENABLE ... ALL [EXCEPT field]
- EXPORT record [EXCEPT field]
- FORM record [EXCEPT field]
- IMPORT record [EXCEPT field]
- INSERT record [EXCEPT field]
- PROMPT-FOR record [EXCEPT field]
- SET record [EXCEPT field]
- UPDATE record [EXCEPT field]
- DO, FIND, FOR, OPEN QUERY, REPEAT, function CAN-FIND:
- record-phrase EXCEPT [field]
Notes:
- Note if USING
phrase is present then statement does reference only specified fields.
- May contain NO-LOBS phrase which excludes LOBS
from the list of referenced fields.
- Field list after EXCEPT can be empty, this means that
all fields are referenced.
See also:
DatabaseReferenceWorker
rules/dbref/collect_refs.xml
rules/dbref/apply_refs.xml
Known Limitations
- At this time, no processing of runtime preprocessor arguments is
handled. Since tables and fields can be specified via this mechanism,
based on runtime calculated expressions, the range of possible values must
be provided via hints.
- No tracking is done of references on indexes. Since indexes are
built using fields this may add references to the fields as well.
- Perhaps better handling of the system tables is possible.
Open Conversion Issues
- Database
- Unqualified field references (e.g. "display id" instead of
"display customer.id") will reference an already created named buffer
(define buffer ccc for customer) for the same table if there is no
default buffer already created. (ECF:
please doc and remove)
- Raise the STOP condition for out of bounds array subscripts
inside of where clause.
- Implement runtime support for multi-table, mixed retrieval-type
queries (e.g., REPEAT PRESELECT EACH a, FIRST b, LAST c) for:
- PresortQuery
- PreselectQuery (currently, we have an implementation which
uses CompoundQuery in preselect mode to meet this need, but a more
efficient implementation may be possible using subselects)
- Temp-tables don't always get buffer scopes at the external proc
as is done today, they can be scoped to specific nested blocks in
Progress. They probably can participate normally in scoping (today
record_scoping.rules has some code at the top that hard codes temp-table
cases to the external proc) which is an easy fix but needs testing.
- Multiple buffer scopes are opened in the same block for the
same buffer. This doesn't make any sense. See
codes/crewcode.p.
- recid(currentBuffer) in client side where clauses is incorrect?
- Add an extent size query method for extent fields and change
the current implementation in builtin_functions.rules which hard codes
the extent value as a NUM_LITERAL.
- The STOP condition generated by a DB disconnect is a special
form that cannot be caught until the first block that doesn't access the
database in question.
P2JIndexComponent
needs type information, or at
least we need to know when a component is a text component (JPRM watch
point @32647). Currently, this class' getNameCaseAware()
method does not have enough information to wrap a text component in the
rtrim()
SQL function, unless that component also happens to
be case-insensitive. This hurts us with temp table indexes,
because this method is used to create indexes on temp table at runtime,
and these indexes will not be usable by the database, because they will
not match the SQL we submit, which embeds the rtrim()
function to match Progress' behavior.
- Uncommitted transaction updates visibility. This is
implemented partially in that converted code can perform dirty reads
when validating DMO changes. However, a loophole still exists for
new code which does not go through the legacy layer of the persistence
framework. This loophole must be closed. We can probably
extend our session interceptor implementation to handle this.
- Implement secure login to DB server from Hibernate;
currently, userid and password are stored in clear text in the directory
- look into validation of extent fields; confirm correct
behavior
- There are cases where the 4GL will report an ambiguous name,
but P2J does not consider it ambiguous. This probably means that we
have been too aggressive about "promoting" namespaces in the
SchemaDictionary. In addition, there may be suppressed
AmbiguousNameException instances in progress.g and SymbolResolver which
further complicates this problem.
- Our PL/Java-based, server-side function implementation
currently does not take into account the following:
- precision
of decimal arguments and return values (test that PL/Java does deliver
parms with the correct scale when set in DDL, check that our methods
are returning BigInteger values with the right scale based on what is
passed in)
- case-sensitivity of character arguments and return values
- timezone of date arguments and return values
- upper() wrapping for case-insensitive fields (and literals and query subsitution parms)
- is done too aggressively in where clauses/query subst (it
should only be done for the immediate children of a comparison
operator), which can cause things like this where clause example: "...
substringOf(upper(myfield), 1, 5) = 'SOMETHING' ...", which works for
this substring() replacement but for other built-ins it may not
- is ignored for string concat operator and character functions
(we rely upon the fact that all fields, literals and query subst data
are already uppercased)
- can be moved into the P2J runtime where it belongs, leaving the generated source code significantly better
- rtrim() wrapping for character fields is done too aggressively in where clauses/query subst (it should only be done for the immediate
children of a comparison operator), which can cause things like this
where clause example: "... lengthOf(upper(rtrim(myfield))) > num ...", which would be incorrect
- Date out-of-bounds issues. Need to support
database-specific date range checking for dates which are hard-coded
into where clauses. Currently, this check is only done for query
substitution parameters. A very rough out-of-bounds check is done
at conversion time in order to log a warning message, but this is
insufficient in the long term.
- IS NULL performance may have issues
- shared temp-table may leak memory in the UNDO implementation
- Runtime error in server logs: JDBC error dropping a temp-table
during session exit. This is more of an annoyance than a real problem
as there is no persistent data to corrupt in a TT.
- Runtime error in server logs: "Can't operate on a
closed statement". This is a warning that
Hibernate logs sometimes when closing the delegate ScrollingResults
object used by a ProgressiveResults object (converted FOR EACH
loops). Previous fixes may have made this better, but problems
remain. One
of the issues was in ScrollingResults.cleanup. What might cause this
would be a race condition, where two threads call
ScrollingResults.sessionClosing at the same time - but I'm not sure
this can happen; I mention this because, once the
ScrollingResults.closed flag is set to TRUE, it no longer be set back
to FALSE; only explication would be a race condition, so synchronizing
the ScrollingResults.cleanup method on its object instance may be a
solution.
- Progress-compatibility of DB error messages should be fully
provided. This will require mapping info for the legacy DB names
(in the DMO index?) and some changes to error processing in the runtime.
- v10+ schema .df parsing will require updates as there are new options and syntax to support.
- Rollback processing for foreign key association
auto-synchronization.
- More robust locking strategy for records resolved by the
ForeignResolver during foreign key association
auto-synchronization. Currently, these records are locked
exclusively, but only until their association object references are
updated, then the locks are released. This could cause deadlock
problems during subtransaction rollback, if a lock previously held
cannot be reacquired during rollback.
- Disallow cross-database joins (
PreselectQuery
)
during conversion. Currently, PreselectQuery.addComponent()
disallows adding cross-database query components at runtime as a
last-ditch safety check, but by then this is too late. This does
not appear to be an issu, but should be addressed for the long
term.
- PostgreSQL Dependencies
- PL/Java is used to implement server-side built-in
functions. We would need to create similar functionality for
other databases. Some vendors support Java natively for custom
function implementations, but the level of support and the
difficulty of the implementation is still unknown. Other
vendors (e.g., SQL Server) do not support the definition of custom
functions using Java. For these, we would have to create a
PL/Java-like layer, probably using C/JNI. For databases which
do not support custom functions at all, we would have to convert
differently (i.e., using client-side where clause processing instead
of embedding server-side functions in HQL). This is the least
desirable option, both because the performance would be unacceptable
and because the converted output would differ from the
standard. These database implementations are effectively
removed as candidates for a scalable, converted system.
- During schema conversion, indexes are used to implement
non-nullable constraints. Must research whether this is the
case with all databases, or whether a different approach would be
necessary.
- During data import, we issue an "analyze" command after
each table is loaded to generate statistics, allowing subsequent
queries against that table to be faster. This is specific to
PostgreSQL.
- Not necessarily specific to PostgreSQL, but some databases
do not support this needed feature: PostgreSQL allows the use
of expressions when defining table indexes. We leverage this
feature by wrapping text columns in the
rtrim()
SQL
function to match Progress' behavior, and for case-insensitive text
components, we further wrap that expression inside the upper()
SQL function.
- PostgreSQL Limitations
- Collation strategy cannot be made dynamic. Collation
behavior for a database cluster is determined at database
installation, by using a custom locale during
initdb
step. It may not be changed thereafter.
- No support for mixed ASC and DESC indexes. Progress
allows an index to contain multiple fields which sort in different
directions. PostgreSQL (and possibly other databases) do not
by default. This will result in performance penalties
(possibly severe, for large tables) in converted applications which
rely on such mixed direction sorting. Currently, we have no
workaround for this issue for PostgreSQL, which has no syntax for
specifying sort direction for index columns. There may be a
way to leverage custom operators to assist with this, but further
research is required to determine if this in fact has potential for
the development of a solution to the problem. Note:
PostgreSQL v8.3 is supposed to add support for mixed direction
indices, but we will have to change our data import processing to
handle this capability.
- PostgreSQL query planner doesn't take NOT NULL constraint
into account for field
F
when optimizing queries
such as select ... from ... where F is null...
Instead of
returning an empty result set immediately (since it is impossible
for F
to ever
be null based upon the NOT NULL constraint), it does a sequential
table scan (which can be very expensive for a large table, since
every row must be visited to determine there is no such
record). There may be a way to tune the query planner to be
smarter about this. Another alternative is to create a limited
range index (i.e., create index i on table t where id
is null
). This will result in a very fast response to this
type of query, but it has drawbacks: it adds another index to
maintain and is not portable, since not all databases will support a
where clause in a create index
statement.
- Progress index bracketing bug we do not replicate: during
where clause processing, expressions (or portions thereof) which test
whether a field (of any type) is less than, or is less than or equal to,
unknown value (as a constant or as the result of a sub-expression) will
trigger an index bracketing bug. The result of this defect is
that all records which would be matched had that subexpression
evaluated to
true
, are included in that index bracket.
We have determined that this must be an index bracketing defect
rather than an expression processing defect, because the same
sub-expression on a stand-alone basis (i.e., outside of a where clause)
behaves normally. That is, it matches the normal expression
processing rules for comparison with unknown value. An
illustration of this defect is provided with P2J uast testcase where-clause-index-unknown-value-bug.p
.
This defect was found using Progress v9.1C. We do not know
if it is present in other versions. At the time of this writing,
there are no plans to mimic this obviously incorrect behavior, since it
seems unlikely that there are legitimate use cases in production
code which would rely upon it. If any cases are found which
inadvertently rely upon this behavior, it is more likely that the
Progress source code should be fixed.
- Unknown
value embedded within a where clause: currently, we "inline" the
most common unknown value references within a where clause, both static
references (i.e., a direct comparison between a database field and the
?
symbol), and variable references (i.e., a direct comparison between a
database field and a variable set to unknown value at runtime).
"Inlined" here means these comparisons are replaced with the
appropriate {database_field} is [not] null
clause in the
HQL which is submitted to Hibernate. In the static case, the
replacement is done at conversion time; in the variable case, at
runtime (by the HQLPreprocessor). However, several corner cases
remain unaddressed:
- The unknown value reference (static or
variable) is embedded in a non-logical subexpression of a direct field
comparison, such that the evaluation of the subexpression would result
in a comparison of the database field and unknown value (rather than
the more correct
{database_field}
is [not] null
idiom). These cases can be addressed by expanding our
replacement/inlining coverage in both the conversion and in the
persistence runtime. Examples:
where ... some-database-field >= ?
is handled
where ...
some-database-field >= some-var
(where some-var
is set to unknown value at runtime) is handled
where ...
some-database-field >= (? + 5)
(however unlikely) is not handled
where ...
some-database-field >= (some-var + 5)
(where some-var
is set to unknown value at runtime) is not handled
- Unknown value (as
NULL
)
is passed from the database into a server-side (e.g., PL/Java)
function. It is unclear whether this is truly a problem, or
whether the server-side expression processing within a database already
correctly handles this situation. If truly an issue,
this problem potentially could be addressed by wrapping every
such instance with case...when...
handling logic, though
the impact on performance would have to be considered carefully.
I suspect this would add noticeable overhead (and could provide
for some very tortuous HQL), but since embedding the database
field within the PL function presumably already precludes any
index-based optimizations, the incremental hit may be acceptable.
Note that this is only an issue for nullable database columns
(non-mandatory fields in Progress-speak), so we may elect to at least
flag potential problem areas during conversion by recognizing the
condition during where clause conversion and issuing a warning to the
conversion log. Example:
where ...
field-1 = dec(field-2)
(where field-2
is not mandatory and potentially represents unknown value) is not handled; although the implementation of the dec()
function correctly returns unknown value, this leaves us with an
equality comparison between a database column and NULL inside the SQL
database server, when it should really be processing field1 is null
. TODO: test whether such processing in an RDBMS really differs from Progress.
- TODO: currently, when there are multiple buffers defined for the same
table, to fix the rollback processing, the BufferManager keeps together all reversibles related
to the same table belonging to the same database. This ensures
proper reversible processing on rollback.
But, the rollback related part in Commitable can be
reworked to allow multiple buffers for a certain table in a database.
This will eliminate the small overhead of rolling back a buffer
which has nothing to do (rollback was performed by one of its
siblings).
Note: a case which needs to be taken into consideration and checked
is when the foreign-keys are enabled. In this case, if the child is
rolled back first in a Child.Parent relation, there might be some
problems.
- UI development and fixes
- Review all TODO markers in both ui and chui packages.
Prepare a report on any items that are left. We will discuss
the priorities.
- Make a list of all non-condition exceptions generated in the
thin client. Which of these should be translated into proper
Progress ErrorConditionException (use
ErrorManager.recordOrThrowError())?
- Using testcases/uast/on_entry_no_apply.p, the on-entry event is
captured for field2 and returns no-apply. After leaving field1, in
Progress field2 is skipped and field3 is focused. In P2J, field3 never
gets focused, focus remains on field1.
- Using testcases/uast/trig-wait.p, type 3 in the first field (i)
and then press enter. This will cause a pause. If you use
CTRL-C there, it will abort the pause but it won't propagate the stop
condition all the way out (exiting the app) like it does in Progress.
- at_base_field_max_quirk10.p right aligns a number when it
should not.
- at_base_field_max_quirk5.p has a prompt-for that should
truncate the data using format "x(2)" but does not.
- Number editing in P2J seems to disallow backspacing over the
leftmost char but in Progress this is possible.
- Remove frame duplication (detect frames that are functionally
identical and reuse the same definition in all business logic rather
than creating a new one).
- Remove duplicate frame.openScope(), validation expressions
(search on Validation2 and widgetDaysSupply) and frame elements from the
business logic.
- Constant (literal) header expressions can be emitted in-line
(via alternate constructors for HeaderElements) instead of as inner
classes.
- Add ControlSetItem(String) and ControlSetItem(character)
constructors. They can simply use the same object as the value and
label. Investigate using Strings instead of the character for the
label (this can't be done for the value, which must be a BaseDataType).
- Remove \n label packing code if it is unneeded.
- frame_generator.xml does not need the add_format function (it's
results are not ever used).
- Remove duplicated code in frame definitions. For example, there are often multiple outputs of the same setFormat(...) for the same widget
are added to the frame def for @ base fields.
- Create and use (in frame definitions) common case widget
constructors that take things like: label, format string, data type,
legacy name... so that the majority of the setter code in the frame def
can be eliminated (reduced to some constructor parameters).
- redirected_output_buffering.p output differs from 4GL (see SIY).
- CHOOSE issues:
- field mode cursor movement is sometimes broken
- row mode group navigation is sometimes broken
- bell is not always occurring when it should
- cursor drawing is sometimes wrong
- hidden attribute/visible issues
- handle data type cannot be displayed in P2J
- MESSAGE statement error handling deviations:
- apply error in a called user-defined function just displays
"?" instead of propagating the error further (is this a more general
issue?)
- page-number(s) on a closed stream displays an error and
then a 2nd line with "?"
- fields need ErrorManager warning mode
- MESSAGE ... VIEW-AS ALERT-BOX deviations:
- conversion
issues related to TITLE in a ALERT-BOX (when the TITLE is a character
expression, not constant). testcase: alert_box_width2b.p
- an edge case when the max length of the message lines is 20 and a SKIP is used. testcase: alert_box_width2.p
- UI performance issues:
- optimize multiple screen definition updates (which have no
intervening visible change to the user)
- pushScreenDefinition() can be deferred until at least
GenericFrame.openScope()
- by using the state sync approach, these can even be
queued up/batched which is probably the better approach
- alternatively, consider pullScreenDefinition() call made
from the client when a new frame is about to be instantiated
- it acts as an ultimately deferred pushScreenDefinition
- screen definition won't be transferred if not required
by this specific run of the application
- repeatable calls will be eliminated since the client
will maintain a registry
- optimize single value (non-batch) screen definition updates
- dynamic setting of screen definition values can be
"broken out" as separate calls to the client to handle
application of specific values only, instead of using the
pushScreenDefinition() to apply the change
- change batching can be redesigned to batch individual
calls as "orders"
- opportunities:
- the COLOR statement (setColors)
- runtime frame options (dynamic expression use in
frame phrases):
- setDynamicTitle
- setColumn
- setRow
- setDown
- setDcolor
- Drawing should never be done during
pushScreenDefinition! Even if something has changed, it should
never be visible to the user until the next view()... so this is
wasted effort and potentially incorrect visually. Currently
the COLOR stmt relies upon this behavior, but if split off as a
separate client export, this would be avoided.
- eliminate extra drawing and screen buffer processing in
ThinClient.promptFor()
- check on whether the following is still true:
- Toolkit.blankBox is called by containers to clear all
contents every time draw is called, even when this is completely
unnecessary.
- Toolkit.blankBox is called by sub-containers even when
an enclosing container has already called this. For
example, Window calls this, then Container calls this and then
Scrollpane calls this...
- Toolkit.blankBox is called 3-9 times for every end-user
keystroke in LoginClient.
- Toolkit.blankBox is called hundreds of times after the
last keystroke in LoginClient.
- creation of the frame proxy (Proxy.newProxyInstance()) in
GenericFrame.createFrame() is expensive because of the large size of
the CommonFrame interface
- cache/pool instances and reuse them over time?
- use something like CGLIB to do the instrumentation?
- FillIn UP/DOWN key movements shift the cursor to widgets
above/below however there are cases in which the cursor is placed in the
first valid cursor position instead of being placed in the exact column
in which the cursor resided in the previous field. If there is
data in the position being targetted, then the cursor will move
there. Note that there is no issue with cases where the cursor is
supposed to move to the beginning of the field (this works). UP key applied
to a fillin in the top frame row moves the cursor to the first fillin
in this row. Movement from fillin to a non-fillin widget and back with
UP/DOWN keys should set the cursor position in fillin to the same as it was
before we have left it.
- Hiding focused widget hides the widget itself, but the cursor remains
on its position while it should move to the nearest widget.
- Cursor movements are constrained more than in Progress in
number editing. See editing5.p.
- Converted help_precedence3.p testcase fails to compile.
- In the testcase testcases/uast/convert_hide.p HIDE
<multiple widgets> statement is converted incorrectly so the
converted testcase cannot be compiled.
- input_lastkey.p shows that certain terminal types have the
initial lastkey value set to -1 (vt220, vt320) instead of 401 (xterm)
- Using SESSION:DATE-FORMAT to change the date component order
dynamically will not be picked up by existing instances of date (on the
client side) nor by existing instances of DateFormat (client side).
See date_component_order.p.
- FillIn bugs with NumberFormat (see testcases/uast/num_edit.p):
- Deleting or back-spacing over last char in P2J moves cursor
out of the field but in the 4GL it does not.
- Overwriting the last char properly moves the cursor out of
the field BUT subsequent keystrokes should BELL. Instead P2J
insert characters and shift everything left.
- If you are in overwrite mode in one field and then use
CURSOR-RIGHT to move the cursor into the next field, the 4GL DOES
NOT auto-zap but P2J does. So in "99,999", the first character
typed (e.g. '4') after cursoring in will yield a "40,000" in the 4GL
and a "00,004" in P2J.
- Extra space added to the right of the "left user text" in
an uninitialized field with formats "+>>>>>>>",
"$(>>>>>)", "$>>>>>" (and others).
- Pressing '0' in an uninitialized "(>>>)" field
causes the cursor to be re-drawn outside and far to the right of the
field BUT subsequent key presses enter data properly into the field
and reset the cursor to the right position. Another example of
this same problem: in a ">>,>>>" field, enter '0',
'1', '9', '1'. This will result in 191 in the field BUT there
will be a temporary misplacement of the cursor between the '0' and
'1'.
- "-999" allows interactive editing of the '-' sign character
when not-negative.
- A field with a 0 value and a format with an optional sign
character (e.g. "-999") will display with '-' if the user explicitly
presses '-' to make the value negative. In other words, a
negative 0 can be displayed as "-000".
- Fields with formats that have a sign character (e.g.
"-999") don't allow using arrow (e.g. CURSOR-LEFT) keys to move the
cursor into the sign character but can in the 4GL.
- Can't enter data on left of decimal point if not in insert
mode for decimal formats: "zzz.99", "-zzz.", "+zzzz.999", "(zzz.9)",
"zz,zzz.999".
- Initial cursor placement in format "zzzzz+" should be on
the right (the + char) instead of on the left.
- Initial cursor placement and subsequent cursor draws for
"$(zzzzz)" format all draw 1 char too far right.
- Cursor placement is too far right in format "-zzz,zzz,zzz"
after activation and insertion of a single "0" character (in the
rightmost position).
- integer fill-in with a value of "1017" and format of
"zzzzzzz", delete on the first digit results in "17" in P2J but "017"
in 4GL
- integer fill-in with a value of "1000" and format of "zzzzzzz", insert
mode off, disengage auto-zap (cursor left/cursor right, for example),
move cursor to digit "1" and type "0"; P2J compresses all 0's into a
single "0" while 4GL just replaces "1" with "0" for a result of "0000" (similar root cause as above with 1017)
- integer fill-in with a value of "0" and format of "zzzzzzz", delete
on "0" results in "0" in P2J but clears the widget completely in 4GL
- integer fill-in with a value of "1" and format of "zzzzzzz",
insert mode off, attempts to enter mode digits must fail while P2J
behaves as if insert mode is active
- Decimal cursor placement is incorrect when all digits to
the left of the decimal point are zero suppressed.
- The sign and/or left user text appears 1 char too far left
with "+>>>>>>>", "$>>>>>",
"$(>>>>>)". The cursor position is also wrong
in this case.
- A failure during auto-complete of a date field should display
00 in the day or month (depending on the field being completed) and the
current year in the year portion. In P2J we draw blanks
instead. Note that only uninitialized fields are affected by
auto-complete.
- Eliminate cursor flashing during drawing.
- Name skip/space widgets skipX or spaceY instead of using exprZ.
- Expression widget numbering should use a per-frame counter
instead of one that is per-file. For example, a given frame should
not have its first expression widget named expr17 because there happened
to be other frames defined in the same file.
- Use a single alignment approach for widgets. The format
phrase usage of AT, TO and COLON should emit to the same thing as
LEFT-ALIGN, RIGHT-ALIGN and COLON-ALIGN. The widget interface for
this should be setAlignment(int) where there are
GenericWidget.LEFT_ALIGN, GenericWidget.RIGHT_ALIGN and
GenericWidget.COLON_ALIGN constants passed as a parameter. The
setTo(), setAt() and setColon() would be dropped.
- If a combo-box is defined without a size phrase, its width is
determined by the rendering size of the format string.
testcases/uast/combo_box/combo_box7.p shows this case. Note that
the BaseDataType.calcFormatLength() and BaseDataType.formatLength() can
be used to calculate the rendered size of a format string. This is
the size of the variable portion of the combo-box.
- The "thumb" button on a scrollbar positions differently.
In particular, the algorithm that determines when it moves and where it
is to be positioned is slightly different. A user that is not
comparing the two implementations side-by-side may not notice this, but
in P2J the thumb button will move at different times and may be
positioned differently at any given time.
- Determine the nature of some unexpected Progress behavior when
displaying uninitialized field level widgets in the
testcases/uast/uselect2.p
testcase. When using a 2 DOWN
frame, the selection
list is shown as uninitialized for the iteration 1 and 3, and as
initialized for the iterations 2 and 4. This is expected. However,
starting with the iteration 5, all subsequent iterations show the
selection list as initialized. This does not happen when using 3
down
frame, for instance.
- The double buffering output driver has solved many subtle
issues in PUT SCREEN handling, but not all of them. There is a class of
situations where it fails, backed by these testcases: put_screen_flush*.p. These are the observed facts and their interpretation:
- in Progress, there is a delay between some frame output and
copying that output to the master buffer, which is used to optimize the
subsequent output;
- this delay affects the way the PUT SCREEN output gets cleared naturally with the frame output;
- this delay also affects the way the final frame picture looks like with the mixture of PUT SCREENs;
- if no PUT SCREENs are involved, this delay does not cause any
difference in the output besides the degree of the output optimization,
which can be measured as the number of characters in the produced
diff;
- the reason for the delay is that Progress saves the frame
output to the master buffer only when a frame switch occurs: any
operation that performs output to a different frame B, comparing with
the previous such operation with a frame A, saves the output of the
frame A at this moment;
- if the frame B from the case above is an overlay frame being
displayed on top of the frame A, then the existance of the delay
directly affects the calculated diff for the frame B, since this
calculation could have been done either with the previous contents of
the master buffer or the most current ones;
- the testcases clearly demonstrate that the diff calculation
happens before the update to the master buffer; that's why the frame B
looks differently when displayed for the first time after displaying A
and a PUT SCREEN, and the second time, when the master buffer has been
updated yet;
- the right way to fix this issue in P2J is to separate the regular syncing of the output in sync() method of the DoubleBufferedTerminal.java from copying that output to the master buffer, which happens simultaneously in today's implementation. sync() should produce the diff based on the existing contents of the master buffer and leave it unmodified. A new method, say, flush(),
should be added, which updates the master buffer. This method should be
called from the view() method whenever the latter detects the frame
switch;
- a fix as described has been attempted but had to be backed
out due to massive regressions in PUT SCREEN output and probably
beyond. Separating sync() from the master buffer update is a very
sensitive operation with the current implementation of the frame
output. Implementor of the fix should be ready to face it.
- Browse
issues. You can view reproductions using the browse_issues.p testcase.
The reproductions theirselfes are placed in the code of this testcase
and each item below references a reproduction from it.
- In
4GL you cannot change the READ-ONLY attribute while the browse has
focus (message "You cannot change the READ-ONLY attribute for BROWSE
{browse_name} while that browser has focus. (4517)" will be displayed).
In P2J, if you will change read-only mode while the browse isn't
focused, you will get full-screen graphical artifacts. (see
reproductions c9, c10 from browse_issues.p)
- In 4GL, if you
will close the query while the editable record has invalid value (does
not conform browse VALIDATE statement), then "No query record is
available. (4114)" and "Warning: Progress cannot get the row you've
just begun editing with a Share-Lock. (4316)" messages will be
displayed, the active record will remain on the clean browse area.
Further actions in this state may lead to a non-intuitive behavior.
(c11)
- In 4GL, if you will close the query while the
editable record has invalid value (does not conform index), then the
empty browse area will be displayed. Further actions in this state will
lead to the normal browse "no query opened" state. (c12)
- In,
4GL BROWSE:REFRESH() function, 4GL will roll back the old value of the
edited field, but you will leave the field, the new value will be
displayed again. Depending on the validness of this value, it will be
committed or not (i.e. you may have invalid values in a browse, but on
the next refresh, they will go away). (c13)
- Validation option into DEF BROWSE .. ENABLE .. VALIDATE doesn't work in P2J. (c14)
- Added records shouldn't be displayed into browse after BROWSE:REFRESH(). (c15)
- Browse shouldn't appear on repositions or on BROWSE:REFRESH() if it is currently hidden. (c16)
- After the backing query of an editable browse has been reopened, you cannot edit browse cells anymore. (c17)
- Browse receives focus too early. (b1)
- final runtime development:
- Client side support for:
- DirectoryService API (note NumberFormat needs to use the
proper getGroupSeparator() instead of using direct access to
NumberType.GROUP_SEP...)
- Logging (in ThinClient, ErrorManager...). Provide a
custom remote log class with simple methods to handle the remote
entries on the server.
- Directory API needs improvements. For example, one would
think that adding a single string node into the directory could be
easily done with a single API call. If you look in the API, you
will find addNodeString(). This does not do what you might think
(even after reading the javadoc). So the docs are confusing and
the simple API is missing. Something similar to the
MinStartupDirectory.addString() method is what would be useful.
All basic datatypes should have such simple helpers.
- Default P2J directory exports should not be directory driven
otherwise simple/stupid admin mistakes can kill the server. Shift
the directory to a RemoteObject approach.
- The security manager must provide warnings before certificates
actually expire so that production installations have adequate time to
handle this issue. This warning MUST NOT be only at startup as a
server may run for months crossing the expiration boundry with no
warning... The lead time should be customizable via a directory
entry.
- Review the state of output and input-output parameters for
procedures/functions when the block aborts due to an error. It is
possible that in Progress the assignments never occur but in ours the
assignments are done "as we go". We may need to handle this on
exit as a special kind of undo OR we can use temporaries and do a batch
assign at the end (this may have negative consequences for database
validation...).
- Constructs like opsys = "VMS" are not being recognized as
dead code.
- Performance tune stream processing, especially redirected
terminal mode (report generation can be slower in P2J by a noticeable
amount of time).
- The build.xml is broken (it always compiles all frame
classes, even when they have not changed).
- Logging improvements:
- Review all project files for:
- Open logging todos (e.g. // TODO: log this) and implement
the proper logging.
- In any location which already logs, make sure that any
exceptions are not just concatinated via toString() or "+" operators
but instead are passed directly to the Logger.log() method call.
- For any logging calls that are expensive (that do string
concatination, data lookups...) should be wrappered in an if
(logger.isLoggable(LEVEL)) to decrease the overhead when logging is
disabled.
- Add trace level logging to the TransactionManager in key
locations.
- Shift ECF log usage to the standardized approach.
- Add socket endpoint info (IP, port) into
LogHelper.describeContext().
- Invent a strategy to obtain a certificate alias for a remote
server using a reverse lookup of hostname/IP address and port/service
name. This is required if we want to support requester-side
validation of a remote server's certificate in server-to-server
connections (i.e., virtual sessions). See
[Router]SessionManager.connectVirtual(InetSocketAdress
address, String certAlias, SessionListener notify)
in the net
package. Currently, this method disables this check if a null certAlias
is provided, which is the common case. But there currently is no
mechanism for the caller of this method to easily determine what to pass
for a remote server's alias. Currently, the directory contains a default/runtime/port_services
path to enable lookups of port number by service name (since Java offers
no standard way to do this). We would probably want to merge this
into a larger table that took hostnames/IP addresses and certificate
aliases into account as well.
- Rename SecurityManager to SecurityMgr to avoid the name
collisions with the java.lang version. (NOTE [ECF]: I
disagree; I don't like class names containing abbreviations --
also note it violates our coding standards. The collision is
easily resolved with an explicit import statement for the P2J
SecurityManager. You will rarely (if ever) need to access both in
the same source file. In fact, over 10+ years, I have never
had to use java.lang.SecurityManager in normal Java development).
- Move the following into com/goldencode from the
com/goldencode/p2j location:
- cfg (at least the bootstrap config stuff)
- the generic (non-Progress) util/ stuff
- directory/
- net/
- security/
- the non-Progress stuff in main/
- As part of this work:
- make sure that any dependencies on Progress-specific
stuff are removed if otherwise a class would be independent
- remove dependencies between packages where possible
(especially dependencies on the UI package like ThinClient and
LogicalTerminal (example: the use of LogicalTerminal.message()
and ThinClient.message() from the util/ErrorManager)
- The SSL handshaking procedure of the protocol sends no client
certificate to the server even when one is specified.
- Is any usage of a custom locale needed for purposes of
non-database "character" type sorting and comparisons?
- RETURN ERROR should not work in a function (it should be like a
RETURN ?). Only procedures and triggers honor RETURN ERROR.
- unknown value in a CASE statement doesn't work as in 4GL
- each file has 2 entries in the registry.xml (with ./ prefix and
without it), eliminate one (and fix whatever code is dependent upon it)
- p2j/testcases/uast/ieee_754_representation_issue.p shows a
deviation where the inexact representation of Java's double causes two
(different but close) double constants in the source program to be
determined as the same number when they really aren't (this may only
occur with numbers that are dependent upon the 11th digit to the right
of the decimal point when there are 6 digits on the left --> thus
causing the total number of representable significant digits to be too
large for double to handle)
- Add BaseDataType constructors that allow a string name (which
if non-null) would force a SharedVariableManager registration inside the
constructor.
- frame_hiding02.p demonstrates TM behavior (probably in infinite
loop protection) that is incorrect. In particular, in Progress we
see 0 1, 2, 3 on the first row of output but in P2J, there is a 0 1 and
then the loop iterations end prematurely.
- helloworld.p has an NPE when run because of a conversion error
in array var initialization.
- string_perf.p has an unreachable issue (failing code commented
out).
- progress.g has a parsing issue when encountering a format
string without quotes and a date format has more than 2 characters in
the 1st or 2nd date component like 99-9999-99.
- scope promotion issue: update c1 c2 = c1 + "text". will create
an EmbeddedAssignmentExpr which cannot resolve var c1
- Write directory merge tool.
- See name_map.xml created by
rules/annotations/name_map_helpers.rules and
rules/annotations/collect_names.rules. This is a file with a
merge tag that stores the attachment path in the main
directory. One good result of this approach is that the
directory is always maintained as a proper directory and any other
merge files can be integrated one by one or in a batch.
- An alternate approach advocated by NVS was to create a
master "template" directory that defines includes. This is
more of a pull concept. It would require that all of the
dependencies are resolvable at merge time, otherwise the master file
cannot be turned into a proper directory.
- The XML schema for the directory is unnecessarily
complicated. While it makes sense to use child elements to
multi-valued attributes, for the primitive node types, it is complete
overkill. A single string node can be specified as <node
class="string" name="mynode" value="stuff" />.
- Retry processing quirks: when validation of a new
temp-table record fails due to the buffer being flushed in preparation
for a query (loosely meaning FIND, FOR, etc.), the 4GL displays more
validation error messages than one would expect, if outside a
transaction. We do not emulate this behavior in P2J. Note
that the number of messages varies depending upon the existence of
previous message and pause statements in the procedure.
See testcases uast/flush-by-query[1-4].p. When inside a
transaction, we behave the same as the 4GL (a single error message).
See testcases uast/flush-by-query[5-9].p.
- Build administrative tools.
- provide editing support for the server, security... configuration values in the directory that cannot be edited today
- finish the runtime/console mgmt features
- add a directory editor to the admin GUI
Planned Optimizations, Refactoring
and Future Development
- Dead code removal:
- Unreferenced variables removal is simplistic at this time and
there are cases which can be removed but are not:
- new shared vars that are never used downstream (requires
call-graph based processing)
- we simply check for any node that has a refid to the var
def, but there are cases that are not real source code references so
the vars stay around (but are really dead anyway)
- Unreferenced streams, frames, buffers... all can be removed
too.
- conditional expressions that always evaluate true or false
- remove concatenation of empty strings
- use of GUI-specific stuff in a CHUI app
- use of persistence in an app that never uses run persistent
- Convert "for each table: delete table. end." into a simpler
form (a single method call to something like
RecordBuffer.deleteAll(table). Some other forms would be possible
too, taking a where clause... to allow deletion of all records matching
is given criteria.
- Convert "for each table: i = i + 1. end." to something like
RecordBuffer.count(table). Some other forms would be
possible too, taking a where clause... to allow a count of all records
matching is given criteria.
- IF/ELSE unreachable processing needs to promote the THEN/ELSE
child nodes into the enclosing block. This can be single
statements (e.g. even "blocks" like REPEAT) or complex groups (any case
of a simple DO block). In the complex group case, all children of
the simple DO block are reparented into the enclosing block at the index
position of the statement node (STATEMENT/KW_IF...). Today such
things are put in an unnecessary DO block.
- Deeper analysis of IF/CASE and other condition expressions. For
example, a logical var that is initialized or assigned false and then
never otherwise changed (or passed as an output or input-output parm) is
effectively a constant. This can be used to detect a larger amount
of dead code.
- An example of expressions that can be cleaned up:
- this code:
- for each itemloc where itemloc.item = cur-item and
itemloc.site = (if "" <> "" then "" else itemloc.site)
- can be converted into this:
- for each itemloc where itemloc.item = cur-item
- these cleanups are not limited to where clauses, but one
should note that any expression rewriting must ensure that the index
selection that would normally be driven by a more complex where
clause is not broken (see ECF or GES on this)
- unreachable processing should be rewritten to run in a loop
(and/or using ascent rules) to detect and "rollup" dead code. The
issue is that if all contents of a particular IF block turn out to be
unreachable BUT the IF *is* reachable, then the IF can be removed.
This is handled by a fixed number of multiple sequential unreachable
cleanup runs in annotations today but would be better handled by a loop
until no more removals can be found in a given pass.
- variable definitions made in unreachable code actually still
exist and can be used in Progress code (note that this is tougher in
cases where there is a message statement or format_phrase var definition
because there are other side effects)
- dead code can have an effect on frame formatting (e.g. format
string processing and width calcs for @ base fields) because the removed
code, though dead in Java, is still processed by the Progress compiler
--> the order of processing can change formatting (see
testcases/uast/deadcode_vs_frame1.p)
- frame which is completely scoped to the dead code, can be
safely removed along with the dead code (requires taking into account
frame scoping before dead code removal)
- LEAVE/NEXT rewriting in annotations should be moved into
fixups. This will allow simplification of the code handling this
as well as improvements in unreachable processing (which happens after
fixups and before annotations). For example, there is a case where
an unlabeled next that isn't in a loop acts as a return, but today
unreachable code processing doesn't detect this (see
testcases/uast/next_as_return.p).
- Detect cases where a handle is not shared AND is not passed as
a parameter AND is only assigned a single type of object. In this
case, replace the usage with the object directly and remove the handle
var. The only other problem is to ensure that the valid-handle()
method is not being used on that handle.
- Expression rewriting:
- call i.increment() and i.decrement() instance members for
expressions like i = i + 1
- convert x <> ? to !x.isUnknown() and convert x = ? to
x.isUnknown()
- constant expressions that can be evaluated at conversion time
- constant array indices can be decremented by 1 instead of
written as an expression
- 3 + 1 * 2 would be converted as the literal 8
- "Hello " + "World" would be converted as "Hello World"
- Convert > 2 chained string concatenations from the
character.concatinate() "operator" to chained StringBuffer.append().
- if the same string literal is used in multiple places (in the
same scope), implement a String variable and then reference
- Remove unnecessary wrapping/unwrapping for expressions that where
the primitive results could be directly used instead.
- When a CompareOps method is used directly in a control flow
statement such as IF, rework the method name into one which returns a
boolean to avoid the need to wrap in CompareOps and then unwrap in
calling code.
- Allow fields and wrappers (vars) to be compared directly from
primative Java types (e.g. int). This can easily be done with
character and String but there is probably value in overloading the
equals() and compareTo() methods... CompareOps and other changes may be
needed too.
- Allow fields and wrappers (vars) to be assigned directly from
primative Java types (e.g. int).
- These improvements may all the BaseDataType class to have the
knowledge of the logical class removed if the unknown value testing does
not need to be there anymore.
- Wrapping and unwrapping could be implemented in a more simple
manner by always marking the literals for wrapping rather than marking the
parent. Perhaps other approaches might also simplify this. Right
now this mechanism is too hidden, obscure and complicated.
- Provide alternate signatures for LogicalTerminal.message() that
allows BaseDataTypes to be passed instead of an AccessorWrapper. Then
change conversion for var refs to eliminate the "newAccessorWrapper(var)"
and replace it with a simple "var".
- ControlFlowOps.setReturnValue("") can probably be done from inside
the TransactionManager at every top-level scope open. Then this code
can be removed from the business logic.
- Progress 4GL if/else if/else if/else does not emit as a Java
if/else if/else if/else but instead as an if/else { if/else { if/else {] }}
which is sub-optimal from a readability perspective.
- Convert Progress 4GL if/else do: if/else end. into a Java if/else
if/else construct which is easier to read.
- Functions should be analyzed to detect if they can ever trigger
conditions to be raised such that they would need the TransactionManager
infrastructure. If not they can be emitted as much simpler methods.
- repeatEmit is probably no longer needed in literals.rules.
- The parser should implement format_string as KW_FORMAT^ STRING
instead of KW_FORMAT^ expr. This was done when processing
initially because of hitting something that didn't parse right. At this
point, the parsing of these obscure areas has been improved greatly such
that this is no longer an issue. The project must be reviewed to understand
if this can be done. Then all the rule sets would have to be modified
to handle the missing EXPRESSION node.
- Format string constants that are not referenced in the business
logic should not exist.
- The extra block if not an "integrated_simple_do" is not always
needed in control_flow.rules. This results in an unnecessary set of {
} output in many blocks in the Java code.
- Eliminate undo properties from vars that don't need it.
- Hide undo registration (possibly inside the wrapper constructors).
- Emit an accessor call to the buffer (which returns a Resolveable or
a instead of generating new FieldReference(buffer, "field").
- Long lived user/process contexts will probably require us to
deregister global resources (e.g. Streams) that are closed rather than leave
the reference around in the finalizables list. Or we can use a weak
hash map.
- Reset the counter for "condition" and "editingLabel" in
<init-rules> for annotations rule sets so that each file will have a 0
based set of indexes.
- Remove unused/unreferenced frames, streams, buffers, queries,
variables...
- Replace all usage of @inheritdoc with explicit coding of the
javadoc code, even though it duplicates text.
- Rename frame definition callback setup() instead of setUp().
Move the batch(false) call out of this setup method and back into the
GenericFrame class which calls this method (there is no reason to emit this
when it will always be needed anyway).
- The chui package's FillIn formatting classes (DateFormat...) should
maximize the use of the BaseDataType formatting support instead of
duplicating it.
- Constant variable detection and rewriting as final statics.
AdaptiveQuery
should be more restrictive in its
decision to invalidated a preselected result set in the case of record
deletes and insertions. Specifically, these cases should only
invalidate the result set if the deleted/inserted record would actually
affect result set traversal.
- Query substitution parameter inlining: currently, inlining
of parameters is based upon database dialect. We probably should
base this on directory configuration, using reasonable defaults.
- Smarter implementation for record counting loops and record
deleting loops. Note: need to consider pessimistic locking
implications here.
- Avoid
emitting unnecessary, top-level instance variables for function
parameters, which are later hidden by block-level, local variables of
the same name, and avoid initializing the top-level variables in the
init()
method of the inner subclass of Block
which is emitted inside the method which represents the converted
function, so long as those top-level instance variables are never
needed. There is some scenario where these are
needed (not sure what that is at the time of writing), but in other cases, they are declared
and initialized, but never subsequently read. The testcase uast/query-test-for-scrolling.p
, for instance, has a function updateRecords(input valOld as int, input valNew as int)
. The valOld
parameter generates both an instance variable in the top-level class QueryTestForScrolling
, as well as an instance variable in the anonymous inner subclass of Block
within the updateRecords()
method. In the init()
method of that inner class, QueryTestForScrolling.this.valOld
is initialized to the value of the anonymous inner class' valOld
variable, but then the top-level valOld
is never accessed. Both the top-level class' valOld
instance variable, and the initialization of it in the inner class' init()
method (and in fact the entire init()
method, since this is all that is done there) could be eliminated as a code cleanup optimization.
- Macro-pattern matching.
- large amounts of code have been "cut-n-pasted" all over the
place
- e.g. select output destination in report generation is ~200
lines of this
- some of these pieces of cut-n-paste code may have been
slightly modified (sometimes in incompatible ways and sometimes in a
manner that may be handled by method parameters in the target
system)
- other places in the code may be highly similar or even the
same, simply by the fact that the same logic is written separately in
multiple places
- the optimal result is to create one or more methods than can be
reused from many locations
- with the current expression engine/pattern engine support, one
can easily write a small ruleset that matches on very complex patterns
BUT to do this one must know what the pattern looks like first
- how might we search throughout the project for such candidates
without knowing the pattern in advance?
- make the full list of live ASTs
- start at the root of the first AST and generate a ruleset
that matches this entire tree
- search all ASTs for a match, save results, reduce the
pattern by walking down the tree and iteratively generating a
ruleset that matches this entire sub-tree and searching all ASTs...
- issues to solve:
- how to build the expressions and how to submit them to
the pattern engine?
- do we make a first pass to build all possible
expressions (a potentially huge list of expressions!) which then
are tested in a second pass? OR do we have some more automated
driver that iteratively processes things (this would require a
pattern engine that can be run against different profiles in the
same JVM)
- do we take some kind of "template" approach where we
pass an AST (rooting the subtree to match) into the pattern
engine instead of a ruleset
- real matches are likely even though an exact match with
the tree is not there, to handle this some form of fuzzy
matching (setting a heuristic that a 90% match is still valid)
OR allowing the pruning of likely variable stuff out of the tree
and still allowing a match in the target when additional nodes
exist that don't exist in the original
- how do we rank matches?
- probably based on the % of nodes that match
- some measure of the deviation caused by nodes that
exist in the target or source but not both
- the number of matches across the list of ASTs
- how do we know when to stop creating a match pattern
for a given input
- matching a single node is likely to be a waste of
time
- even matching small subtrees of 2-3 nodes is likely
to be a waste of time
- perhaps a more general rule can be made that
patterns only found WITHIN language statements are too small
to be matched upon (e.g. matching on all DEFINE VARIABLE
subtrees is useless as we already know that these subtrees
appear everywhere and by themselves they don't constitute a
useful match)
- SO: we are really trying to match common patterns made by
groups of:
- top-level parser/Progress language constructs like
language statements, assignments, loops...
- expressions that are of high complexity (how to
determine this?) might also be good candidates for replacement
as a method
- "if ___ then x = y + z. else x = y + z + a." can be rewritten as "x
= y + z. if ___ then x += a."
- Emit primitives instead of wrappers based on hints or heuristics.
- Heuristic based optimizations will be written to unwrap
variables once into temporary primitives and reuse these until
reassignment or modification of the original wrapper instance. For
example, if there is > 1 read of the same variable without that
variable being modified, the unwrapping would occur once and the same
temporary primitive would be used multiple times. This makes the
code more readable and improves performance. Issues to resolve
include:
- how to name the temp variables?
- how to detect when it is safe to reuse an already unwrapped
variable and when something could be modified?
- How might we define beans for highly coupled/related shared
variables such that all of these related members are stored and retrieved in
the shared variable manager in one call?
- Improve refactoring by enhancing the call graph with the (currently
lost) knowledge of loops/recursion and duplicate calls. This knowledge
may allow the aggregation of multiple procedures into a common class.
- Implement call graph mode for the pattern engine. This will
allow much more advanced analysis of resource usage (across scope
boundaries), undoability, dead code analysis etc...
- Remove debug code in the Progress source. (How do we identify
such things?)
- Fix bug in date.neutralMillisToCalendar(). It does take the
daylight savings time offset into account in that method. The problem
is that the DST offset is queried from a GregorianCalendar object that is
set to the current (today's) date/time. This means that if the current
TZ is in DST right now, the DST offset will be positive for all dates!
That is why any date in the DST range is fine but those outside are
off. I need to think on a solution for this as it is not as easy as
setting the target date/time into the calendar first, since anything very
close to the DST transition point may still calculate incorrectly. The
following testcase shows the issue:
- date
firstP2JDate = new date(2,1,2006); //EST
- date
secondP2JDate = new date(7,4,2006); //EDT
- Date
firstJavaDate = firstP2JDate.dateValue(TimeZone.getDefault());
- Date
secondJavaDate = secondP2JDate.dateValue(TimeZone.getDefault());
- DateFormat
format = new SimpleDateFormat("MM/dd/yyyy 'at' HH:mm:ss z");
-
System.out.println("P2J First: " + firstP2JDate.toString());
-
System.out.println("Java First: " + format.format(firstJavaDate));
-
System.out.println("P2J Second: " + secondP2JDate.toString());
-
System.out.println("Java Second: " + format.format(secondJavaDate));
- Unimplemented Progress features:
- database
- dynamic database support
- CREATE BUFFER
- CREATE DATABASE
- CREATE QUERY
- CREATE TEMP-TABLE
- methods/attributes for using query handles and
other dynamic database resources
- All the backing support (compatibility logic) for these
features is implemented, but some of the conversion
processing needed to convert queries needs to be moved into the
runtime.
- database triggers
- DISABLE
TRIGGERS
- TRIGGER
PROCEDURE
- sequences
- CURRENT-VALUE()
- CURRENT-VALUE
statement
- NEXT-VALUE()
- temp-table handle as parameters
- CONTAINS
operator
- misc built-in functions
- CURRENT-CHANGED
- DBPARAM
- DBTASKID
- DBVERSION
- RECORD-LENGTH
- TO-ROWID
- internationalization is partial
- DBCODEPAGE()
- DBCOLLATION()
- stored procedures
- RUN
STORED-PROCEDURE
- CLOSE
STORED-PROCEDURE
- two-phase commit
- schema meta-data access (e.g. _file)
- base language
- Windows-specific features
- registry and environment support
- GET-KEY-VALUE
- LOAD
- PUT-KEY-VALUE
- UNLOAD
- USE
- system dialogs
- SYSTEM-DIALOG COLOR
- SYSTEM-DIALOG FONT
- SYSTEM-DIALOG
GET-FILE
- SYSTEM-DIALOG
PRINTER-SETUP
- SYSTEM-DIALOG HELP
- "well known" named stream I/O destinations
- Active-X / COM support
- COM-HANDLE
data type
- CREATE
<com_object>
- RELEASE
OBJECT
- DDE support
- DDE ADVISE
- DDE EXECUTE
- DDE GET
- DDE INITIATE
- DDE REQUEST
- DDE SEND
- DDE
TERMINATE
- Finish BigDecimal support for decimal literals and for all decimal math functions.
- user-defined functions with IN
<procedure_handle> or IN
SUPER.
- shared library support
- appserver
- CREATE SERVER
- RUN ON SERVER (invoking remote procedures on
an appserver)
- TRANSACTION-MODE AUTOMATIC
- persistent procedures, super procedures
- DELETE PROCEDURE
- RUN PERSISTENT (no persistent procedure support is
provided)
- RUN SUPER
- named events support is handled in the conversion and is
stubbed out in the runtime, but the runtime needs to be finished
(see NamedEventManager.java)
- PUBLISH
- SUBSCRIBE
- UNSUBSCRIBE
- internationalization and sorting
- CURRENT-LANGUAGE
- CODEPAGE-CONVERT()
- IS-LEAD-BYTE()
- GET-CODEPAGES()
- GET-COLLATIONS()
- 22% of built-in functions:
- DYNAMIC-FUNCTION()
- SETUSERID()
- SUPER()
- PROC-STATUS()
- PROC-HANDLE()
- persistent triggers
- ON PERSISTENT (all other forms of triggers are already
supported)
- misc
- PROMSGS
- PROPATH assignment is supported in the runtime but not
in conversion
- SETUSERID
- raw/memptr support
- GET-BITS()
- GET-BYTE-ORDER()
- GET-BYTES()
- GET-DOUBLE()
- GET-FLOAT()
- GET-LONG()
- GET-POINTER-VALUE()
- GET-SHORT()
- GET-UNSIGNED-SHORT()
- PUT-BUTS
- PUT-BYTES
- PUT-DOUBLE
- PUT-FLOAT
- PUT-LONG
- PUT-SHORT
- PUT-UNSIGNED-SHORT
- RAW()
- RAW
- SET-BYTE-ORDER
- SET-POINTER-VALUE
- XML
- CREATE X-DOCUMENT
- CREATE X-NODEREF
- sockets
- CREATE SERVER-SOCKET
- CREATE SOCKET
- VIEW with a list of widgets, what seems to be an
undocumented language feature
- user interface
- finish editor widget support, phase 3: all
methods/attributes, all options, all word wrapping and editing
features
- frame-index is only implemented with CHOOSE, but in reality
it should be maintained by any data input statement
- remaining CHUI support
- remaining GUI support
- base frame implementation
- drawing/layout for GUI widgets
- new widgets
- dynamic widget/frame support
- CREATE WIDGET/OBJECT
- CREATE BROWSE
- CREATE WIDGET-POOL
- DELETE WIDGET/OBJECT
- DELETE WIDGET-POOL
- remaining methods/attributes (75%)
- miscellaneous built-in functions
- CAN-QUERY
- CAN-SET
- FRAME-DB
- FRAME-FILE
- FRAME-NAME
- IS-ATTR-SPACE
- LIST-EVENTS
- LIST-QUERY-ATTRS
- LIST-SET-ATTRS
- LIST-WIDGETS
- LOAD-PICTURE
- RGB-VALUE
- VALID-EVENT
- language statements
- DEFINE IMAGE
- DEFINE MENU
- DEFINE RECTANGLE
- DEFINE SUB-MENU
- INSERT
- Buttons don't honor the default attribute in CHUI. A
number of experiments show that this attribute is ignored in the
Progress character mode. Documentation is not clear regard to this
attribute, but it mentions that presence of default button in the
frame disables handling of the RETURN key by fill-in widgets and
this does not happen in character mode. In GUI mode this will
need to be added.
- accumulation "variables" (like a "naked" count variable)
can be referenced directly in Progress without the "accum 1 count"
construct. This is a parse-time problem but it also has
consequences that are harder to fix since such vars can be assigned!
- validation expressions, screen-value and the INPUT builtin
function all allow variables/fields to be treated bimodally (as
their normal type and as a character type, depending on context),
better detection logic is needed during conversion, including having
a database of signatures for builtin functions, methods...
- aggregation phrases in a display statement are supported,
but the implementation is limited to service the simpler use cases
we have encountered to date; specifically, the following cases
will break the current implementation:
- currently, there is a simplifying assumption that all
active aggregate phrases for a frame are grouped together in the
same display statement; thus, the "same" aggregate phrase
appearing in multiple display statements will not be processed
correctly, whether:
- in the same loop, or
- across different loops
- the "when" option will disrupt accumulation by
producing a null frame element (Progress accumulates the row
data, regardless of whether "when" masks display of the
data); because we drive accumulation from within the frame
element, this implementation breaks down in this situation
- a related problem is that the "when" option is not
honored currently, if it appears after an aggregate phrase
in a display statement
- internationalization
- code page conversions
- collation
- "translation manager" features
- string options processing
- usage of Java resource bundles where possible
- Deviations from Progress behavior:
- parser
- An empty frame phrase is legal as in "repeat with j = 1 to
5:". This is the equivalent to "repeat j = 1 to 5:". The
problem is that the down_clause rule isn't conditional, it matches
as soon as there is an expression in LA1.
- The file testaces/uast/two_frames_same_name.p defines two
frames: one in the procedure file and one in a local procedure.
Although both frames have the same name, P2J creates 2 fields with
the same name in the generated class. Also, there is only one frame
definition for both generated fields.
- user interface
- Applying ENTRY event to other widget in trigger attached to EDIT widget
may not have visual feedback in 4GL - cursor remains inside EDIT widget (see
event_behavior_edit.p testcase) while in P2J cursor is moved to activated
widget. This is definitely a bug in 4GL while P2J handles this case correctly
and this results to visible difference in appearance. Note that input focus is
correctly moved so next key typed by the user is forwarded to correct widget.
- PUT SCREEN can "bleed through" a frame in Progress because
certain fields (e.g. fill-ins) only seem to output to some character
positions and blank positions don't get output as spaces. This
means that a prior put screen can bleed through in Progress.
In the P2J implementation, this does not occur.
- In Progress DOWN frames behave differently when CHOOSE is
invoked for a variable that is in a DOWN frame when CHOOSE is in ROW
mode. In this case no intermediate pauses are performed and all
values are scrolled up so only the last value is visible in the
frame (it occupies the top data row in the frame).
- FRAME-INDEX may return incorrect values (in comparison with
Progress) if CHOOSE is invoked for regular (extent = 1) variables or
for a combination of array and regular variables. This
happens because information about array indexes associated with a
particular variable instance is lost during conversion. Since
this function is intended for use with arrays, real applications
should not have any problems with that.
- FillIn widget
- when editing a decimal number, after entering the last
(rightmost) digit to the right of the decimal point) using the
backspace key (or left arrow) will jump 2 characters to the left
(in Progress) instead of 1 (in P2J and as most users would
expect)
- An explicit PAUSE statement will ignore the HELP key (by
default F2 or any key that has been remapped to the HELP
function). It does not trigger help nor does it clear the
pause. Some tests show that implicit pauses (like those caused
by a block exiting/iterating with a frame scoped in that block) may
not provide this same "feature".
- A string override on a date base field will display with
separators in Progress but not in P2J. "hello" @ mydate will
display as "he/lo/ ". This doesn't seem very useful so
it has not been duplicated.
- A string override in a logical base field will change the
value of the field in a way that can be edited. "ye" @ mybool
format "yes/no" will display AND edit as "yep". This is
unusual because normally edited data is not changed when an override
occurs which has mismatched types.
- LEAVE and ENTRY events show inconsistent behavior if there
are triggers on either of them that return NO-APPLY. The reaction
may even depend on the source of the event. Cursor movement keys,
tabs and back tabs and APPLY ENTRY all may produce varying results.
The most often, the cursor left and right keys behave in the most
logical way: the focused widget remains in focus and does not lose
its highlight. If this is the case, repeatable application of those
keys produces predictable and repeatable results. The other times,
the focused widget loses its highlight *and* the state of the
widgets changes invisibly, so that the repeatable application of the
keys produces ever changing results! In P2J this behavior is not
reproduced and the focused widget always retains focus.
- ON ... REVERT is implemented with deviations. In Progress,
- triggers having widget list (even though they may have
ANYWHERE option as well) can't be reverted entirely down to the
no trigger at all; the first defined trigger is "sticky" and
REVERT operation has no effect on it. See the
testcases/ui/revert1.p
testcase.
- global ANYWHERE triggers seem to have a single item
representing them instead of a normal stacking. No matter how
many ON registrations were made, a single ON ... REVERT
deregisters the trigger entirely.See the
testcases/ui/revert2.p
testcase.
- the findings above have to be further tested to
determine the applicable scoping (within the current block or
globally).
- READKEY in editing block may be responsible for firing
triggers on the fly for some high level events. CHOOSE event for
buttons is known to be such an event. The list of events in this
category and details of how it works need to be investigated. See the testcases/uast/readkey_trigger*.p testcases.
- Some error messages are slightly different:
- Database lock conflicts don't show the TTY of the owning
user.
- Database field and table names in error messages are the
new ones.
- Stream access failures don't show the source code name of
the stream.
- Some runtime generated stream exceptions (e.g. "Attempt to
read from closed stream ____. (1386)" may appear as unspecified
IOExceptions today where this translates into error text that may
say something like "IOException during read. (-1)".
- Condition processing:
- CTRL-C while the server is in a tight DO loop doesn't honor
the stop until after the loop exits. If the loop is very long
(or infinite), the STOP will never be honored.
- The program-name function may show different output than in
Progress due to different decompositions of function in both the
generated code and differing code paths in the runtime.
- regular stream I/O
- Although documented in Progress references, the default
code-page conversion (stream to internal and vice versa) does not
appear to actually be implemented OR perhaps this is locale
specific. No default codepage conversion is implemented in
P2J.
- All of the explicit codepage conversion processing that is
possible in Progress is missing.
- The effect of the BINARY option is only accounted for in
READKEY processing and in this case it only modifies how newlines
are returned. If BINARY modifies other processing, it has not
been identified and is thus unimplemented at this time.
- Progress references document that the null character terminates
strings. This is not implemented.
- DBCS support is not provided.
- UNBUFFERED mode is not supported (and is probably not
needed).
- MAP/NO-MAP support does not exist.
- redirected I/O
- The special
text mode echo quirk for logicals, integers and decimals is not
supported. Likewise the screen buffer copy on error is only
supported for character types, not all types.
- Mixed usage of the PUT language statement and UI statement
output redirection support (e.g. DISPLAY) may not operate properly.
- The overwrite
behavior of certain frames (based on implicit DOWN or explicit
DOWN/UP) is not supported as there does not seem to be a useful
reason to actually code an application to do this.
- Stream width is hard coded to a maximum of 512 columns and
128 lines. A more dynamic/flexible approach should be
implemented based on the (column and line) values in the current
frame however this has not been handled at this time.
- Using an array index that is out of bounds during a NO-ERROR
(silent error mode) language statement/assignment will fail with an
IndexOutOfBoundsException since we currently directly map array access
into the standard Java subscript operator []. While we do have a
central method in NumberType.subscript() to handle the bounds checking
and error generation, in silent error mode this will return an out of
bounds value.
- IF/THEN/ELSE that are detected as having an unreachable branch
(the THEN or ELSE) will have that branch removed. This is
logically correct, however in the case where the IF expression has side
effects (e.g. a user defined function that changes some program state),
then this code would not be executed in the rewritten generated code.
- Any NO-ERROR expression that causes an error to be raised, does
not immediately abort in P2J as it would in Progress. Assignment
processing is protected such that no assignment is made in this case,
BUT any side effects of the expression that might be sequenced after the
error will occur in P2J when they would not in Progress. For
example, use of user defined functions can have side-effects.
Since in P2J, a user defined function in an expression would still
execute after the error occurred, an unexpected behavior could be
caused.
- NUM_LITERAL of 999999999999 (12 digits) can be encoded in a
progress source file but an error does not occur until runtime. To avoid
a javac problem (compile time), we detect this case during conversion
and throw an exception.
- DEC_LITERAL mantissas (the significand or the digits to the
left of the decimal point) can be up to 50 digits in Progress, the
current implementation of P2J only supports up to 16. Detect and
use a BigInteger implemention instead. This may have to be done as
a configuration value for the entire server/client (use use of the
decimal class). It also has implications for code emit since the
Java double can't be used to represent the DEC_LITERAL.
- Handle data type's default format string is larger because the
32-bit integer hashcode that is used to back the handle's representation
is too large to fit in the normal Progress default.
- No support for array subscripts with ranges that use a
non-constant expression as the start index as in myvar[ 2 + j FOR 3
]. Such cases can not be used in an expression, instead they can
only be used as implicit reference to a range of index positions.
- No conversion of Progress strings to Java strings is done at
runtime. All strings passed are expected to be Java strings.
This behavior may need changing in the case where a Progress-compatible
string (one that has Progress-specific escape sequences) is built at
runtime but needs to be used as a Java string.
- Based on testing a wide range of Progress features (escape
sequences in string literals, reading from files/processes, built-in
functions...), embedded null characters in a character var can only be
generated using the get-string() builtin function on either a raw or
memptr var. Even in this case, the var must contain null bytes AND
the get-string() call must specify an explicit length that causes the
copied data to include a range that contains the null byte. This
processing is fully implemented in P2J. Once you have a character
var with a null character embedded, the support for that data varies
depending upon the part of Progress that is in use. All character
related operators (especially the comparison ones), built-in functions
and language statements need to be reviewed for their compatibility with
such embedded null byte cases. The following are already handled:
+ (concatination operator), asc(), chr(), string(), substring() and =
(assignment operator). Note that chr() cannot return a null byte,
and both substring() and the assignment operator are completely
insensitive to null bytes.
- frame-index, lastkey and other variable-like builtin functions
can silently ignore assignment usage in business logic (the errors occur
and no change occurs to the value) --> this is a form of dead code
- The following features aren't useful in converted Java code
(and are not currently converted at all):
- DICTIONARY
- COMPILE
- KEYWORD
- KEYWORD-ALL
- LIBRARY
- MEMBER
- PROMSGS
(function and statement)
- SAVE-CACHE
- SHOW-STATS
- Complete support for database methods and attributes. The
first release supports only a handful of database-related methods and
attributes.
- Support "preselect fetch" FIND statement where clauses. These
statements can have their own where clauses which further refine a find
among the preselected results. Currently, these are not handled.
- Support a multi-table AdaptiveQuery. Currently, multi-table
joins which cannot be converted to PreselectQuery convert instead to
CompoundQuery. The latter is inefficient because the join is performed
at the database client and it therefore requires an order of magnitude more
database-level queries for each join than would a multi-table AdaptiveQuery
implementation. A multi-table AdaptiveQuery would perform the join on
the server when in preselect mode, but would fall back to using
CompoundQuery when converting to dynamic mode.
- Future Development
- AST ID stability.
- Better regression testing harness.
- Naming
- Add hints support for naming overrides.
- Cross-namespace (vars, streams....) conflicts.
- Minimize the use of disambiguating text in the names (like adding Frame to every frame name).
- Remove fileMatch in all annotations rules (we always persist
now so this is just gorp).
- Add an option to allow get/setProperty() in FileSystemOps to be
backed by the directory instead of by Java system properties.
- Update the lexer and parser for v10.1C changes (it is up to date with 10.1B).
Copyright (c)
2005-2010, Golden Code Development Corporation.
ALL RIGHTS RESERVED. Use is subject to license terms.