FWD » Core Development » Base Language

• VAR INT a.
• VAR INT a = 1.
• VAR INT a, b.
• VAR INT a, b = 1.

The goal of this task is to enhance the parsing and conversion process to support the VAR statement.

Before we move ahead, please create ABLUnit testcases that:

• Explore the different syntax options.
• Prove (or disprove) all of the above "facts" from the documentation.
• Explore the range of initialization expression options for both scalar and array cases.

These should be added to whatever Variable Definition Tests currently exist. I expect that we will need to be more thorough than those tests. Since this statement has scoping and program/block structure limitations, the tests should be written as .p and .cls files which are then used by ABLUnit tests. This approach will also allow us to use them for parser testing (see below).

Save any cases you write that are syntax failures in OE. We will need to put those (as .p or .cls files) into the testcases project in tests/conversion/parser/ in an "invalid" directory similar to how we do it for the Lexer Testcases. These failure cases cannot be ABLUnit code. Please note that we did not document how the invalid testcases are run/checked so you'll have to look inside the tests/conversion/lexer/ and grep through the .xml files for "invalid" to trace it out. You can also read through #6860 for the discussion of invalid tests.

We have #6861 for writing the parser testcases which is the analog to #6860. The details will be documented in Parser Testcases. I'm not saying you will work on #6861 right now. I'm just letting you know that we will be doing this and I want your work to be saved in a way that allows both the invalid cases and valid cases to be used as the parser tests for the VAR statement.

For VAR, I am wondering where the expression should be evaluated? I thought of the following:

• Using a lambda
• Evaluating the expression directly at the start of the block where it is defined and assign it.
• Rewriting the VAR as DEFINE VARIABLE ... AS ... NO-UNDO and then assign ... = ... .. (this was discussed and it is not appropriate - see #9488-241)

I think the semantics of when initializers are evaluated will be the same as we implement for DEF VAR so I would expect to follow that pattern. If you find some requirement that contradicts this, let's discuss it here.

I looked a lot more into the lexer and preprocessor tests, but I can't figure out how to generate the OE baselines.

For the lexer, I have to delete the baselines folder and simply run generate_baselines_script.sh. But even after I run the oe_test_plan.xml on the customer machine I require a baseline. I noted that the command used for the lexer uses driver.p which only runs COMPILE, so there was no way for me to obtain an actual baseline for the OE.

In the case of the preprocessor, there are no expected errors.

My goal is to create the parser tests which will use the same framework as the preprocessor, but will output the parsing errors. The preprocessor tests are actually what I needed.

I ran the harness against the failing tests and found errors such as:

19521 Public, package-protected, package-private, protected or private qualifiers may only be used on elements in a class or interface. (19521)
14593 The STATIC, SERIALIZABLE and FINAL qualifiers are only valid in a class file. (14593)
18396 The NON-SERIALIZABLE qualifier is only valid in a class file. (18396)
20006 The VAR statement must precede any executable statement within block contexts where it is allowed. (20006)
13659 The USING statement must come before all other compilable statements. (13659)
16743 BLOCK-LEVEL ON ERROR statement must come before all executable or definitional statements. (16743)
14145 ROUTINE-LEVEL ON ERROR statement must come before all executable or definitional statements. (14145)
20005 The non-full-keyword i is not allowed in the VAR statement. (20005)
12874 The CLASS statement must come before all other compilable statements other than USING. (12874)

Currently, there is no support for those errors.

That is OK, remember that we don't have syntax checking support in FWD until #3882. So long as the tests exist and "work" in OE, we can move on.

Code Review Task Branch 9767a Revision 16564

Overall, this is quite close. Here are changes I'd like to see:

1. Don't inline custom type matching in var_stmt. With slight changes, the var_type can be used directly. Add a boolean limited parameter that changes the behavior of var_type when it is false:

• Bypass specialDataTypeKeywordAbbreviations(1) in the init action.
• Using a semantic predicate to disallow matching to KW_WID_HAND (assumes that the VAR stmt also disallows this, please check it).
• Move the exit action logic to a new method createLocalVariable().

Something like this:

var_type [ String varName, boolean nativeTypes, boolean limited ] returns [ int vartype ]
   :
      {
         vartype = -1;
         if (!limited)
         {
            specialDataTypeKeywordAbbreviations(1);
         }
      }   
      ( 
         options { generateAmbigWarnings = false; }
         :
           KW_CHAR                              { vartype = VAR_CHAR; }
         | KW_COM_HNDL                          { vartype = VAR_COM_HANDLE; }
         | KW_DATE                              { vartype = VAR_DATE; }
         | KW_DATETIME                          { vartype = VAR_DATETIME; }
         | KW_DATE_TZ                           { vartype = VAR_DATETIME_TZ; }                            
         | KW_DEC                               { vartype = VAR_DEC; }
         | KW_HANDLE                            { vartype = VAR_HANDLE; }
         | { !limited }?
           KW_WID_HAND)                         { vartype = VAR_HANDLE; }
         | KW_INT                               { vartype = VAR_INT; }
         | KW_INT64                             { vartype = VAR_INT64; }             
...
      {
         if (!limited)
         {
            createLocalVariable(varName, vartype, #nam);
         }
      }

The new method:

   private void createLocalVariable(String varName, int vartype, Aast nam)
   {
      if (varName != null)
      {
         String cls = (#nam == null) ? null : (String) #nam.getAnnotation("qualified");

         // here is where we enable the variable lookup for all subsequent
         // code in this scope, note that Progress only seems to ever have
         // 2 levels of scope: the external proc and any internal proc/
         // trigger defs, since you can only define shared or global shared
         // vars in the external proc scope, we should *not* need to
         // explicitly add vars to the global scope since adding to the
         // current scope should be equivalent
         Variable var = sym.addLocalVariable(varName, vartype, cls);

         if (#nam != null && #nam.isAnnotation("dotnet-array"))
         {
            var.setDotNetArray(true);
         }
         if (#nam != null && #nam.isAnnotation("generic-type-parameter"))
         {
            var.setGenericType((String) #nam.getAnnotation("generic-type-parameter"));
         }
         if (#nam != null && #nam.isAnnotation("generic-type-is-primitive"))
         {
            var.setGenericPrimitive((boolean) #nam.getAnnotation("generic-type-is-primitive"));
         }
      }
   }   

varType will also have to be returned so that it is available in the caller.

This leaves the logic in a shared location / reduces duplication. The exit action logic in particular is good to share, the createLocalVariable() would be called from var_single_decl.

2. Changes to var_single_decl:

a. It is not clear why boolean isHandle, boolean isObject are parameters. I think they can be dropped.

b. The code to set symbol / any_symbol_at_all / any_non_reserved_symbol should be moved into a new rule symbolic_variable_name instead of being duplicated:

symbolic_variable_name [boolean inClass] returns [String varname]
   :
      (
           { !inClass }?
           s:symbol
           { varname = #s.getText(); }

           // inside a class it is OK to use reserved keywords
         | { inClass && !inMethodDef }?
           res:any_symbol_at_all
           { varname = #res.getText(); }

         | { inClass && inMethodDef }?
           non_res:any_non_reserved_symbol
           { varname = #non_res.getText(); }
      )
   ;

It would be called from both var_single_decl and def_var_stmt.

c. Add a new token called VAR_DEF_LIST and in the init action use astFactory.makeASTRoot(currentAST, #[VAR_DEF_LIST, ""]); to create an artificial root node. All the variable instances will be grouped under that node. It will make processing cleaner and less ambiguous later.

d. The extent initialization case probably should be designed as a separate rule with a structure like initializer. We can't reuse that rule because it only allows constants to be used for array init. I think we need to root the list of expressions at the LBRACKET.

3. Don't create an artificial KW_NO_UNDO node. We can either set the no-undo annotation directly or we can just have the downstream rules know to process the var stmt instances as no-undo.

4. Instead of the manual processing of annotations at the end of var_stmt, I think we should use postDefineVar().

I've worked on the changes from the review and the only problem I am facing right now is resolving the qualified name of an expression when adding the local variable.

var_single_decl was modified in the following way:

var_single_decl [ int ftype, int size, String className ]
   returns [String varname = null]
   :
      {
         astFactory.makeASTRoot(currentAST, #[VAR_DEF_LIST, ""]);
         boolean inClass = (sym.getCurrentClassDef() != null);
      }
      (
         varname = nvar:symbolic_variable_name[inClass]
      )
      (
         options { generateAmbigWarnings = false; }
         :
         eq:EQUALS^
         (
              { LA(1) == LBRACKET }?
              array_initializer
            | expr
         )
      )?
      {
         createLocalVariable(varname, ftype, #nvar);
      }
   ;

at com.goldencode.p2j.uast.Variable.<init>(Variable.java:326)
at com.goldencode.p2j.uast.SymbolResolver.addLocalVariable(SymbolResolver.java:3333)
at com.goldencode.p2j.uast.ProgressParser.createLocalVariable(ProgressParser.java:4993)
at com.goldencode.p2j.uast.ProgressParser.var_single_decl(ProgressParser.java:47116)
at com.goldencode.p2j.uast.ProgressParser.var_stmt(ProgressParser.java:38482)
at com.goldencode.p2j.uast.ProgressParser.stmt_list(ProgressParser.java:28112)

The nvar_AST.parent.parent looks like this for a basic example:

= [EQUALS]:<id_unavailable> @78:34
      (original-token=["=",<2974>,line=78,col=34])
    [VAR_DEF_LIST]:<id_unavailable> @0:0
      myobj1 [SYMBOL]:<id_unavailable> @78:27
            (original-token=["myobj1",<2994>,line=78,col=27])
   expression [EXPRESSION]:<id_unavailable> @0:0
      new [FUNC_CLASS]:<id_unavailable> @78:36
            (builtin-cls=false, returnsunknown=false, indirect-dotnet=false, found-in-cls=support.myclass, is-class=true, access-mode=2135, tempidx-file=./abl/support/myclass.cls, original-token=["new",<745>,line=78,col=36], found-in-source-file=./abl/support/myclass.cls, qualified=support.myclass, source-file=./abl/support/myclass.cls, is-interface=false, signature=myclass(), is-enum=false, static=false, is-java=false, dotnet-cls=false, tempidx=0, oldtype=745, builtin=true)
         support.myclass [CLASS_NAME]:<id_unavailable> @78:40
               (builtin-cls=false, indirect-dotnet=false, is-class=true, original-token=["support.myclass",<6>,line=78,col=40], qualified=support.myclass, source-file=./abl/support/myclass.cls, is-interface=false, is-enum=false, is-java=false, dotnet-cls=false, oldtype=2994)
         ( [LPARENS]:<id_unavailable> @78:55
               (original-token=["(",<2979>,line=78,col=55])

To me, it looks like the node placement is wrong.

Yes, the use of ^ in eq:EQUALS^ overrides the makeASTRoot() and forces the EQUALS to be the root node.

I should have thought about this before. Create a rule and move the calling code from var_stmt (and the makeASTRoot() from var_single_decl) into it:

var_list_decl [ int ftype, int size, String className, java.util.List<String> varnames ]
   :
      {
         String var1;
         astFactory.makeASTRoot(currentAST, #[VAR_DEF_LIST, ""]);
      }
      var1 = var_single_decl[ftype, size, className]
      { varnames.add(var1); }

      // additional variable declarations separated by commas
      (
         options { generateAmbigWarnings = false; }
         :
         comma
         {
            String var2;
         }
         var2 = var_single_decl[ftype, size, className]
         { varnames.add(var2); }
      )*
   ;

var_list_decl then gets invoked unconditionally in var_stmt.

In regard to the classname issue, cls should be null if the type isn't a class. If it is a class, then find out why it is not being passed through properly.

Also, I should note that the EQUALS node will need to have its type rewritten as ASSIGN. Look at existing grammar rules and search on ASSIGN to see what I mean.

Greg Shah wrote:

Also, I should note that the EQUALS node will need to have its type rewritten as ASSIGN. Look at existing grammar rules and search on ASSIGN to see what I mean.

I understand, will make the changes.

The current problem I am working on is the definition of var char[3] c1 = [1,2,3], this generates the following ast:

    <ast col="0" id="21474836482" line="0" text="statement" type="STATEMENT">
      <ast col="1" id="21474836483" line="1" text="var" type="VAR_STMT">
        <annotation datatype="java.lang.String" key="name" value="c1"/>
        <annotation datatype="java.lang.Long" key="extent" value="3"/>
        <annotation datatype="java.lang.Long" key="type" value="386"/>
        <annotation datatype="java.lang.Long" key="tempidx" value="299"/>
        <ast col="5" id="21474836485" line="1" text="char" type="KW_CHAR"/>
        <ast col="9" id="21474836486" line="1" text="[" type="KW_EXTENT">
          <ast col="10" id="21474836487" line="1" text="3" type="NUM_LITERAL">
            <annotation datatype="java.lang.Boolean" key="use64bit" value="false"/>
            <annotation datatype="java.lang.String" key="use64bit-byrule" value="annotations/cleanup.rules:470"/>
            <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
            <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
          </ast>
        </ast>
        <ast col="0" id="21474836490" line="0" text="" type="VAR_DEF_LIST">
          <ast col="16" id="21474836491" line="1" text="=" type="ASSIGN">
            <ast col="13" id="21474836494" line="1" text="c1" type="SYMBOL"/>
            <ast col="18" id="21474836495" line="1" text="[" type="LBRACKET">
              <ast col="0" id="21474836496" line="0" text="expression" type="EXPRESSION">
                <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/expressions.rules:1486"/>
                <annotation datatype="java.lang.Long" key="peerid" value="201863462961"/>
                <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
                <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
                <ast col="19" id="21474836497" line="1" text="1" type="NUM_LITERAL">
                  <annotation datatype="java.lang.Boolean" key="use64bit" value="false"/>
                  <annotation datatype="java.lang.String" key="use64bit-byrule" value="annotations/cleanup.rules:470"/>
                  <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
                  <annotation datatype="java.lang.Boolean" key="is-literal" value="true"/>
                  <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
                </ast>
              </ast>
            </ast>
          </ast>
        </ast>
      </ast>
    </ast>

The only close structure is c1[1] = 1, but now I have an array on the right side. I am trying to find a way to parse this and generate something like:
TypeFactory.characterExtent(3, "1", "2", "3").

You should be trying for something like this:

...
            <ast col="18" id="21474836495" line="1" text="[" type="LBRACKET">
              <ast col="0" id="21474836496" line="0" text="expression" type="EXPRESSION">
                <ast col="19" id="21474836497" line="1" text="1" type="NUM_LITERAL">
                </ast>
              </ast>
              <ast col="0" id="21474836498" line="0" text="expression" type="EXPRESSION">
                <ast col="19" id="21474836499" line="1" text="2" type="NUM_LITERAL">
                </ast>
              </ast>
              <ast col="0" id="21474836500" line="0" text="expression" type="EXPRESSION">
                <ast col="19" id="21474836501" line="1" text="3" type="NUM_LITERAL">
                </ast>
              </ast>
            </ast>
...

I've removed the annotations for clarity. If you need help, check in your current changes and I can look at the grammar.

Greg Shah wrote:

You should be trying for something like this:

[...]

I've removed the annotations for clarity. If you need help, check in your current changes and I can look at the grammar.

It looks the same:

            <ast col="18" id="21474836495" line="1" text="[" type="LBRACKET">
              <ast col="0" id="21474836496" line="0" text="expression" type="EXPRESSION">
                <ast col="19" id="21474836497" line="1" text="1" type="NUM_LITERAL">

        <ast col="0" id="21474836490" line="0" text="" type="VAR_DEF_LIST">
          <ast col="13" id="21474836491" line="1" text="c1" type="SYMBOL">
            <annotation datatype="java.lang.String" key="name" value="c1"/>
            <annotation datatype="java.lang.Boolean" key="undoable" value="false"/>
            <annotation datatype="java.lang.String" key="classname-byrule" value="annotations/variable_definitions.rules:455"/>
            <annotation datatype="java.lang.Long" key="extent" value="3"/>
            <annotation datatype="java.lang.Long" key="type" value="386"/>
            <annotation datatype="java.lang.Long" key="tempidx" value="299"/>
            <annotation datatype="java.lang.String" key="javaname" value="c1"/>
            <annotation datatype="java.lang.String" key="classname" value="character"/>
            <annotation datatype="java.lang.String" key="javaname-byrule" value="annotations/variable_definitions.rules:446"/>
          </ast>
        </ast>

Ignore my last comment, I forgot a patch.

I was getting too complicated and missed a lot of details, I should've just piggy bagged the DEFINE_VARIABLE implementation instead of just creating a whole new way of converting the VAR_STMT.

• Addressed the review from #9767-11.
• Added two small conversion changes that allow for the following example to be converted: var char[3] c1.

I am gathering my thoughts and revisiting the DEFINE_VARIABLE to check what I should use from the rest of my implementation.

var char[3] c1 = ["1", "2", "3"]. also works after a few small adjustments.

Right now, I am approaching var char[3] c1, c2., the generated ast ignores anything defined before c2 and this is an issue:

    <ast col="0" id="21474836482" line="0" text="statement" type="STATEMENT">
      <ast col="1" id="21474836483" line="1" text="var" type="VAR_STMT">
        <annotation datatype="java.lang.String" key="classname-byrule" value="annotations/variable_definitions.rules:319"/>
        <annotation datatype="java.lang.Boolean" key="vardef" value="true"/>
        <annotation datatype="java.lang.Long" key="peerid" value="201863462976"/>
        <annotation datatype="java.lang.Long" key="type" value="386"/>
        <annotation datatype="java.lang.String" key="javaname" value="c2"/>
        <annotation datatype="java.lang.String" key="vardef-byrule" value="fixups/post_parse_fixups.xml:341"/>
        <annotation datatype="java.lang.Boolean" key="promote" value="true"/>
        <annotation datatype="java.lang.String" key="name" value="c2"/>
        <annotation datatype="java.lang.Long" key="extent" value="3"/>
        <annotation datatype="java.lang.String" key="classname" value="character"/>
        <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/variable_definitions.rules:1318"/>
        <annotation datatype="java.lang.String" key="javaname-byrule" value="include/common-progress.rules:12229"/>
        <annotation datatype="java.lang.String" key="promote-byrule" value="annotations/scope_promotion.rules:306"/>
        <ast col="5" id="21474836485" line="1" text="char" type="KW_CHAR"/>
        <ast col="9" id="21474836486" line="1" text="[" type="KW_EXTENT">
          <ast col="10" id="21474836487" line="1" text="3" type="NUM_LITERAL">
            <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
            <annotation datatype="java.lang.Boolean" key="use64bit" value="false"/>
            <annotation datatype="java.lang.String" key="use64bit-byrule" value="annotations/cleanup.rules:470"/>
            <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
          </ast>
        </ast>
        <ast col="0" id="21474836490" line="0" text="" type="VAR_DEF_LIST">
          <ast col="13" id="21474836491" line="1" text="c1" type="SYMBOL">
            <annotation datatype="java.lang.String" key="name" value="c1"/>
            <annotation datatype="java.lang.Boolean" key="undoable" value="false"/>
            <annotation datatype="java.lang.Long" key="extent" value="3"/>
            <annotation datatype="java.lang.Long" key="type" value="386"/>
            <annotation datatype="java.lang.Long" key="tempidx" value="299"/>
          </ast>
          <ast col="17" id="21474836494" line="1" text="c2" type="SYMBOL">
            <annotation datatype="java.lang.String" key="name" value="c2"/>
            <annotation datatype="java.lang.Boolean" key="undoable" value="false"/>
            <annotation datatype="java.lang.Long" key="extent" value="3"/>
            <annotation datatype="java.lang.Long" key="type" value="386"/>
            <annotation datatype="java.lang.Long" key="tempidx" value="300"/>
          </ast>
        </ast>
      </ast>

Slight issues with the current way we annotate variables, for a DEFINE VARIABLE we'll just annotate the DEFINE_VARIABLE ast. The VAR_STMT requires the SYMBOL to be annotated, especially in the scenarios were we define multiple variables.

In Variable.annotateOptions() we have expectations in regards to the structure of the definition, so I plan to create separate methods in this scenario instead of getting confused by something that was built specifically for DEFINE_VARIABLE.

Greg, is the option of adding a new token like VAR_DEF_ITEM that will replace SYMBOL in VAR_DEF_LIST? I have the vague impression that I am getting into complex rules and writing them only to have some case mess it later and this change will simplify the changes.

I'd prefer not to do that.

Please show the output that is being generated incorrectly along with the 4GL and the AST.

Greg Shah wrote:

I'd prefer not to do that.

Please show the output that is being generated incorrectly along with the 4GL and the AST.

I already started working on it and things were much easier, I'll handle the work with this at the moment and change back what I need at the end.

The main issue I have now is var char c1 = "1". ,this is mostly related to convert/literals.rules and when should a literal be or not be suppressed. I am comparing the VAR_STMT ast with the ast for define variable c2 as character initial "2".

The var_stmt ast:

<ast col="0" id="21474836482" line="0" text="statement" type="STATEMENT">
      <ast col="1" id="21474836483" line="5" text="var" type="VAR_STMT">
        <ast col="5" id="21474836489" line="5" text="char" type="KW_CHAR"/>
        <ast col="0" id="21474836491" line="0" text="" type="VAR_DEF_LIST">
          <ast col="13" id="21474836492" line="5" text="=" type="ASSIGN">
            <annotation datatype="java.lang.Long" key="peerid" value="201863462961"/>
            <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/assignments.rules:500"/>
            <ast col="10" id="21474836495" line="5" text="c1" type="VAR_DEF_ITEM">
              <annotation datatype="java.lang.Boolean" key="vardef" value="true"/>
              <annotation datatype="java.lang.Boolean" key="promote" value="true"/>
              <annotation datatype="java.lang.Long" key="type" value="387"/>
              <annotation datatype="java.lang.String" key="promote-byrule" value="annotations/scope_promotion.rules:306"/>
              <annotation datatype="java.lang.String" key="javaname" value="c1"/>
              <annotation datatype="java.lang.String" key="classname" value="character"/>
              <annotation datatype="java.lang.String" key="name" value="c1"/>
              <annotation datatype="java.lang.Boolean" key="undoable" value="false"/>
              <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/variable_definitions.rules:1531"/>
              <annotation datatype="java.lang.String" key="vardef-byrule" value="fixups/post_parse_fixups.xml:341"/>
              <annotation datatype="java.lang.String" key="javaname-byrule" value="include/common-progress.rules:12229"/>
              <annotation datatype="java.lang.String" key="classname-byrule" value="annotations/variable_definitions.rules:319"/>
              <annotation datatype="java.lang.Long" key="peerid" value="201863462974"/>
            </ast>
            <ast col="0" id="21474836496" line="0" text="expression" type="EXPRESSION">
              <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
              <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
              <ast col="15" id="21474836497" line="5" text="&quot;1&quot;" type="STRING">
                <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
                <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
                <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/literals.rules:1223"/>
                <annotation datatype="java.lang.Long" key="peerid" value="201863462976"/>
                <annotation datatype="java.lang.Boolean" key="is-literal" value="true"/>
              </ast>
            </ast>
          </ast>
        </ast>
      </ast>
    </ast>

and for the c2 variable:

    <ast col="0" id="21474836502" line="0" text="statement" type="STATEMENT">
      <ast col="1" id="21474836503" line="9" text="define" type="DEFINE_VARIABLE">
        <annotation datatype="java.lang.Boolean" key="vardef" value="true"/>
        <annotation datatype="java.lang.Boolean" key="promote" value="true"/>
        <annotation datatype="java.lang.Long" key="type" value="387"/>
        <annotation datatype="java.lang.String" key="promote-byrule" value="annotations/scope_promotion.rules:306"/>
        <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
        <annotation datatype="java.lang.String" key="javaname" value="c2"/>
        <annotation datatype="java.lang.String" key="classname" value="character"/>
        <annotation datatype="java.lang.String" key="name" value="c2"/>
        <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/variable_definitions.rules:790"/>
        <annotation datatype="java.lang.String" key="vardef-byrule" value="fixups/post_parse_fixups.xml:341"/>
        <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
        <annotation datatype="java.lang.String" key="javaname-byrule" value="include/common-progress.rules:12229"/>
        <annotation datatype="java.lang.String" key="classname-byrule" value="annotations/variable_definitions.rules:319"/>
        <annotation datatype="java.util.ArrayList" key="initial">
          <listitem datatype="java.lang.String" value="&quot;2&quot;"/>
        </annotation>
        <annotation datatype="java.lang.Long" key="peerid" value="201863462987"/>
        <ast col="17" id="21474836507" line="9" text="c2" type="SYMBOL"/>
        <ast col="20" id="21474836509" line="9" text="as" type="KW_AS">
          <ast col="23" id="21474836511" line="9" text="character" type="KW_CHAR"/>
        </ast>
        <ast col="33" id="21474836513" line="9" text="initial" type="KW_INIT">
          <annotation datatype="java.lang.Long" key="peerid" value="201863462989"/>
          <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/variable_definitions.rules:1531"/>
          <ast col="41" id="21474836515" line="9" text="&quot;2&quot;" type="STRING">
            <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
            <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
            <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/literals.rules:1223"/>
            <annotation datatype="java.lang.Long" key="peerid" value="201863462990"/>
            <annotation datatype="java.lang.Boolean" key="is-literal" value="true"/>
          </ast>
        </ast>
      </ast>
    </ast>

The generated code looks like this:

   @LegacySignature(type = Type.VARIABLE, name = "c1", dataType = "CHARACTER")
   character c1 = TypeFactory.character();

   @LegacySignature(type = Type.VARIABLE, name = "c2", dataType = "CHARACTER")
   character c2 = UndoableFactory.character("2");

   /**
    * External procedure (converted to Java from the 4GL source code
    * in varexample.p).
    */
   @LegacySignature(type = Type.MAIN, name = "varexample.p")
   public void execute()
   {
      externalProcedure(Varexample.this, new Block((Body) () -> 
      {
         new character("1").assign();
         message(c2);         
      }));
   }

The main issue I have now is var char c1 = "1". ,this is mostly related to convert/literals.rules and when should a literal be or not be suppressed. I am comparing the VAR_STMT ast with the ast for define variable c2 as character initial "2".

I don't think this is a literals issue. The problem is that the expression node is not emitting as a peer in the right place (as an initializer).

Greg Shah wrote:

The main issue I have now is var char c1 = "1". ,this is mostly related to convert/literals.rules and when should a literal be or not be suppressed. I am comparing the VAR_STMT ast with the ast for define variable c2 as character initial "2".

I don't think this is a literals issue. The problem is that the expression node is not emitting as a peer in the right place (as an initializer).

Giving EXPRESSION a peerid will just make it generate in expression.rules instead.

Dănuț Filimon wrote:

Greg Shah wrote:

The main issue I have now is var char c1 = "1". ,this is mostly related to convert/literals.rules and when should a literal be or not be suppressed. I am comparing the VAR_STMT ast with the ast for define variable c2 as character initial "2".

I don't think this is a literals issue. The problem is that the expression node is not emitting as a peer in the right place (as an initializer).

Giving EXPRESSION a peerid will just make it generate in expression.rules instead.

Good, that is exactly what we want. The expression should have its peer set properly (so that it emits as an initializer method parameter. From there no extra work is needed, everything emits naturally.

The new character("1") is still generated in the BLOCK, I'd expect to define it in the variable like this character c1 = TypeFactory.character("1");, but the constructor is empty.

Alright, so I got to into a point where the literal generates in the block instead of the variable definition.

The .jast

                    <ast col="0" id="201863462955" line="0" text="block" type="BLOCK">
                      <annotation datatype="java.lang.String" key="peerid-byrule" value="include/common-progress.rules:12127"/>
                      <annotation datatype="java.lang.Long" key="peerid" value="21474836481"/>
                      <annotation datatype="java.lang.String" key="byrule" value="include/common-progress.rules:12127"/>
                      <ast col="0" id="201863462974" line="0" text="" type="EXPRESSION">
                        <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/expressions.rules:1483"/>
                        <annotation datatype="java.lang.Long" key="peerid" value="21474836496"/>
                        <annotation datatype="java.lang.String" key="byrule" value="convert/expressions.rules:1483"/>
                        <ast col="0" id="201863462975" line="0" text="1" type="STRING">
                          <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/literals.rules:1226"/>
                          <annotation datatype="java.lang.Long" key="peerid" value="21474836497"/>
                          <annotation datatype="java.lang.String" key="byrule" value="convert/literals.rules:1226"/>
                        </ast>
                      </ast>

and this is what I added to expression.rules

         <rule>parent.type == prog.assign and sibling(prog.var_def_item)
            <action>createPeerAst(java.expression, "", closestPeerId)</action>
         </rule>

In the end, I can't find the difference and only think that expressions.rules is not the right place.

         <rule>parent.type == prog.assign and sibling(prog.var_def_item)
            <action>createPeerAst(java.expression, "", closestPeerId)</action>
         </rule>

The core issue here is that you are using closestPeerId which is going to be in the block instead of the variable definition itself. When you create the variable definition, you need to store the id of initializer. That id should be used instead of the closestPeerId.

I would also write the condition differently. Instead of parent.type prog.assign and sibling(prog.var_def_item), I would write something like parent.parent.type prog.var_def_list and parent.type prog.assign and childIndex 1.

Greg Shah wrote:

[...]

The core issue here is that you are using closestPeerId which is going to be in the block instead of the variable definition itself. When you create the variable definition, you need to store the id of initializer. That id should be used instead of the closestPeerId.

I would also write the condition differently. Instead of parent.type prog.assign and sibling(prog.var_def_item), I would write something like parent.parent.type prog.var_def_list and parent.type prog.assign and childIndex 1.

Yes, this was it. Thank you!

var datetime dt4 = "05-05-2002 07:15:03".

def var dt5 as datetime initial "05-05-2002 07:15:03".

The initializer used by DEF VAR

initializer [int vtype] returns [int count = 1]   
   :
      KW_INIT^
      (
           initializer_constant[vtype]
         | lb:LBRACKET! { hide(lb); }
           initializer_constant[vtype] (comma initializer_constant[vtype] { count++; })*
           rb:RBRACKET! { hide(rb); }
      )
   ;

Something similar has to be done for the VAR_STMT in var_single_decl:

         eq:EQUALS^ { #eq.setType(ASSIGN); }
         (
              { LA(1) == LBRACKET }?
              array_initializer
            | expr
         )

In one of the examples I am currently working with:

var int[3] x = [1,2,3]

          <ast col="14" id="21474836497" line="88" text="=" type="ASSIGN">
            <ast col="12" id="21474836501" line="88" text="x" type="SYMBOL">
              <annotation datatype="java.lang.String" key="javaname" value="x"/>
              <annotation datatype="java.lang.String" key="promote-byrule" value="annotations/scope_promotion.rules:306"/>
              <annotation datatype="java.lang.String" key="classname-byrule" value="annotations/variable_definitions.rules:319"/>
              <annotation datatype="java.lang.String" key="vardef-byrule" value="fixups/post_parse_fixups.xml:341"/>
              <annotation datatype="java.lang.String" key="javaname-byrule" value="include/common-progress.rules:12229"/>
              <annotation datatype="java.lang.Long" key="extent" value="3"/>
              <annotation datatype="java.lang.Boolean" key="promote" value="true"/>
              <annotation datatype="java.lang.Boolean" key="undoable" value="false"/>
              <annotation datatype="java.lang.Long" key="peerid" value="219043332160"/>
              <annotation datatype="java.lang.String" key="classname" value="integer"/>
              <annotation datatype="java.lang.String" key="name" value="x"/>
              <annotation datatype="java.lang.Long" key="type" value="395"/>
              <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/variable_definitions.rules:1492"/>
              <annotation datatype="java.lang.Boolean" key="vardef" value="true"/>
              <ast col="0" id="21474836502" line="0" text="[" type="KW_EXTENT">
                <ast col="0" id="21474836503" line="0" text="3" type="NUM_LITERAL">
                  <annotation datatype="java.lang.Boolean" key="use64bit" value="false"/>
                  <annotation datatype="java.lang.String" key="use64bit-byrule" value="annotations/cleanup.rules:470"/>
                  <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
                  <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
                </ast>
              </ast>
            </ast>
            <ast col="17" id="21474836504" line="88" text="1" type="NUM_LITERAL">
              <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/literals.rules:943"/>
              <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
              <annotation datatype="java.lang.Boolean" key="use64bit" value="false"/>
              <annotation datatype="java.lang.String" key="use64bit-byrule" value="annotations/cleanup.rules:470"/>
              <annotation datatype="java.lang.Long" key="peerid" value="219043332163"/>
              <annotation datatype="java.lang.Boolean" key="is-literal" value="true"/>
              <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
            </ast>
            <ast col="20" id="21474836507" line="88" text="2" type="NUM_LITERAL">
              <annotation datatype="java.lang.String" key="peerid-byrule" value="convert/literals.rules:943"/>
              <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
              <annotation datatype="java.lang.Boolean" key="use64bit" value="false"/>
              <annotation datatype="java.lang.String" key="use64bit-byrule" value="annotations/cleanup.rules:470"/>
              <annotation datatype="java.lang.Long" key="peerid" value="219043332164"/>
              <annotation datatype="java.lang.Boolean" key="is-literal" value="true"/>
              <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
            </ast>
            <ast col="23" id="21474836510" line="88" text="3" type="NUM_LITERAL">
<annotation datatype="java.lang.String" key="peerid-byrule" value="convert/literals.rules:943"/>
              <annotation datatype="java.lang.String" key="support_level-byrule" value="gaps/gap_analysis_marking.xml:435"/>
              <annotation datatype="java.lang.Boolean" key="use64bit" value="false"/>
              <annotation datatype="java.lang.String" key="use64bit-byrule" value="annotations/cleanup.rules:470"/>
              <annotation datatype="java.lang.Long" key="peerid" value="219043332165"/>
              <annotation datatype="java.lang.Boolean" key="is-literal" value="true"/>
              <annotation datatype="java.lang.Long" key="support_level" value="16400"/>
            </ast>

Because ASSIGN is used without proper marking of the content array, the variable converts in the following way:

   @LegacySignature(type = Type.VARIABLE, extent = 3, name = "x", dataType = "INTEGER")
   integer[] x = TypeFactory.integer(3);
   @LegacySignature(type = Type.MAIN, name = "var-stmt1.p")
   public void execute()
   {
      externalProcedure(VarStmt1.this, new Block((Body) () -> 
      {
         (long) 1;
         2;
         3;
      }));
   }

The literals used for initializing the var for a DEF_VAR are usually marked contained by a KW_INIT. The current solution I am working on is basically the same, a VAR_INIT that will hold the literals used for initialization.

The NUM_LITERAL instances need to be under an LBRACKET node, even with a KW_INIT.

Greg Shah wrote:

The NUM_LITERAL instances need to be under an LBRACKET node, even with a KW_INIT.

I was confused about that because there is no LBRACKET node for KW_INIT.

You're right, we hide that in the existing initializer.

Committed 9767a/16566. Expanded on the VAR statement support with chained definitions and initializers (including arrays).

Rebased 9767a to latest turnk/16648, the branch is now at revision 16651.