FWD » Core Development » Base Language

• Setup a hotel GUI project or any dataset project, with incremental conversion.
• Do a full conversion with this file: p-test-inc.pp-test-inc.p

  DEFINE TEMP-TABLE tt1 NO-UNDO
      FIELD f1 AS CHARACTER.
  
  CREATE tt1.
  tt1.f1 = "1".
  

• Change the file to have this content: p-test-inc.pp-test-inc.p

  DEFINE TEMP-TABLE tt1 NO-UNDO
      FIELD f1 AS CHARACTER
      FIELD f2 AS CHARACTER.
  
  CREATE tt1.
  tt1.f1 = "1".
  tt1.f2 = "2".
  

• Run incremental conversion

You will notice this error:

     [java] ./abl/p-test-inc.p
     [java] Elapsed job time:  00:00:01.161
     [java] EXPRESSION EXECUTION ERROR:
     [java] ---------------------------
     [java] javaname = p2o.javaPropertyName(getNoteString("schemaname"), false)
     [java]                ^  { No java name found for legacy name:  tt1.f2 }
     [java] ---------------------------
     [java] ERROR:
     [java] com.goldencode.p2j.pattern.TreeWalkException: ERROR!  Active Rule:
     [java] -----------------------
     [java]       RULE REPORT      
     [java] -----------------------
     [java] Rule Type :   WALK
     [java] Source AST:  [ tt1.f2 ] BLOCK/ASSIGNMENT/ASSIGN/FIELD_CHAR/ @7:1 {21474836529}
     [java] Copy AST  :  [ tt1.f2 ] BLOCK/ASSIGNMENT/ASSIGN/FIELD_CHAR/ @7:1 {21474836529}
     [java] Condition :  javaname = p2o.javaPropertyName(getNoteString("schemaname"), false)
     [java] Loop      :  false
     [java] --- END RULE REPORT ---
     [java] 
     [java] 
     [java] 
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1104)
     [java]     at com.goldencode.p2j.convert.TransformDriver.processTrees(TransformDriver.java:610)
     [java]     at com.goldencode.p2j.convert.ConversionDriver.back(ConversionDriver.java:583)
     [java]     at com.goldencode.p2j.convert.TransformDriver.executeJob(TransformDriver.java:1058)
     [java]     at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:1237)
     [java] Caused by: com.goldencode.expr.ExpressionException: Expression execution error @1:16 [FIELD_CHAR id=21474836529]
     [java]     at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:275)
     [java]     at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:210)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.apply(PatternEngine.java:1698)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1584)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1517)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1067)
     [java]     ... 4 more
     [java] Caused by: com.goldencode.expr.ExpressionException: Expression execution error @1:16
     [java]     at com.goldencode.expr.Expression.execute(Expression.java:495)
     [java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:500)
     [java]     at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:751)
     [java]     at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:717)
     [java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:537)
     [java]     at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:751)
     [java]     at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:717)
     [java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:537)
     [java]     at com.goldencode.p2j.pattern.RuleContainer.apply(RuleContainer.java:597)
     [java]     at com.goldencode.p2j.pattern.RuleSet.apply(RuleSet.java:98)
     [java]     at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:262)
     [java]     ... 9 more
     [java] Caused by: com.goldencode.p2j.schema.SchemaException: No java name found for legacy name:  tt1.f2
     [java]     at com.goldencode.p2j.schema.P2OLookup.javaName(P2OLookup.java:798)
     [java]     at com.goldencode.p2j.schema.P2OLookup.javaPropertyName(P2OLookup.java:861)
     [java]     at com.goldencode.p2j.schema.P2OAccessWorker$Library.javaPropertyName(P2OAccessWorker.java:468)
     [java]     at com.goldencode.expr.CE4979.execute(Unknown Source)
     [java]     at com.goldencode.expr.Expression.execute(Expression.java:398)
     [java]     ... 19 more
     [java] 

• on full conversion, the DMO associated with tt1 is Tt1_1.java, but on the next incremental conversion, the DMO class is Tt1_2.java. This does not get generated on the incremental conversion, and the failure point is on compilation stage.

+Danut
+Constantin

Danut, the 6083a has a fix for this issue? I see that Constantin reported it on #10536-15.

Teodor Gorghe wrote:

Danut, the 6083a has a fix for this issue? I see that Constantin reported it on #10536-15.

My understanding is that 6083a does fix this.

Teodor: please find the change from 6083a which can be applied to 9457c, to resolve this.

The change that fixes the incremental conversion is dependent on the changes from the 6083a branch. I think pulling all the changes from 6083a is the easiest way to solve it.

The root cause for this regression is the change to not process any .schema files if there are not changes in .df files, for incremental conversion. So, unless any .df file has changed, no .schema files for temp-tables will be processed.

I can still reproduce #10613-1 with the latest 6083a. As far as I can remember, I did not fix this in the branch because it was also an issue in trunk.

Constantin Asofiei wrote:

The root cause for this regression is the change to not process any .schema files if there are not changes in .df files, for incremental conversion. So, unless any .df file has changed, no .schema files for temp-tables will be processed.

• Pre-processing P2O Generation (temp-table schema files)
• P2O Generation (temp-table schema files)
• P2O Post-processing (temp-table schema files)

Dănuț Filimon wrote:

I can still reproduce #10613-1 with the latest 6083a. As far as I can remember, I did not fix this in the branch because it was also an issue in trunk.

Constantin Asofiei wrote:

The root cause for this regression is the change to not process any .schema files if there are not changes in .df files, for incremental conversion. So, unless any .df file has changed, no .schema files for temp-tables will be processed.

This is not correct. Even if a .df file is not modified, the temp-tables a modified file will still be processed as the source file was changed. You'll have the following steps:

• Pre-processing P2O Generation (temp-table schema files)
• P2O Generation (temp-table schema files)
• P2O Post-processing (temp-table schema files)

The something else is happening in trunk: the .dict has the changes I made to the file's temp-table before running incremental conversion, but the .schema doesn't. Also, this code will not generate the DMO .java code for temp-tables:

      if (!cmode.omitSchemaBrew)
      {
         FileList dmos = generateDMOs(cmode.skipDBSchema);
         if (dmos.size() > 0)
         {
            brew(dmos, "Data Model Objects (DMOs)");
         }

         FileList pojos = generatePOJOs(cmode.skipDBSchema);
         if (pojos.size() > 0)
         {
            brew(pojos, "Plain 0ld Java 0bjects (P0J0s)");
         }
      }

The .dict file is created again when TransformDriver.runScanDriver() is called (ends up in AstGenerator.parse()), but the same call also cleans up the cvt sources (dict, p2o, schema) when the mode.schema is set to true. This is a regression introduced in trunk/16124.

Dănuț Filimon wrote:

The .dict file is created again when TransformDriver.runScanDriver() is called (ends up in AstGenerator.parse()), but the same call also cleans up the cvt sources (dict, p2o, schema) when the mode.schema is set to true. This is a regression introduced in trunk/16124.

I actually think this was a bug that is revealed by trunk/16124 changes. This can be reproduced by running a full conversion, than running incremental conversion where the schema = false (using modes). Since the changes disable the schema steps when none are detected, then this issue appears a lot more.

As for the fix...there are no persistent schema changes, but we modified temp-tables in the source file, which makes it so that there are schema changes that need to be reflected into the schema and p2o files.

This is the patch I came up with:

=== modified file 'src/com/goldencode/p2j/convert/TransformDriver.java'
--- old/src/com/goldencode/p2j/convert/TransformDriver.java    2025-08-19 11:05:12 +0000
+++ new/src/com/goldencode/p2j/convert/TransformDriver.java    2025-10-07 10:46:06 +0000
@@ -302,37 +302,34 @@
                    asts,
                    debug);

-      if (mode.schema)
+      // clean up previous run - build schema list and delete those files
+      FileList delDicts = createDatabaseList(".schema", false);
+      deleteFiles(delDicts.listFilenames());
+      
+      // build dictionary list
+      dicts = createDatabaseList(".dict", omitSchemaMetadata());
+      
+      // fixup dictionary files
+      processTrees("Schema Fixups (data dictionary)",
+                   "schema/fixups",
+                   datanames,
+                   true,
+                   dicts,
+                   debug);
+      
+      // build schema list
+      sdicts = convertSourceNamesToDictNames(".dict");
+      
+      // fixup source file schema files (if any exist for these specific
+      // source files)
+      if (sdicts.size() > 0)
       {
-         // clean up previous run - build schema list and delete those files
-         FileList delDicts = createDatabaseList(".schema", false);
-         deleteFiles(delDicts.listFilenames());
-         
-         // build dictionary list
-         dicts = createDatabaseList(".dict", omitSchemaMetadata());
-         
-         // fixup dictionary files
-         processTrees("Schema Fixups (data dictionary)",
+         processTrees("Schema Fixups (Progress source file schemas)",
                       "schema/fixups",
                       datanames,
                       true,
-                      dicts,
+                      sdicts,
                       debug);
-         
-         // build schema list
-         sdicts = convertSourceNamesToDictNames(".dict");
-         
-         // fixup source file schema files (if any exist for these specific
-         // source files)
-         if (sdicts.size() > 0)
-         {
-            processTrees("Schema Fixups (Progress source file schemas)",
-                         "schema/fixups",
-                         datanames,
-                         true,
-                         sdicts,
-                         debug);
-         }
       }

       if (mode.callgraph)
@@ -448,13 +445,11 @@
          exts.add(".pphints");
       }

-      if (schema)
-      {
-         exts.add(".dict");
-         exts.add(".schema");
-         exts.add(".schema.original");
-         exts.add(".p2o");
-      }
+      // Even if the schema was not changed, those files will still be created for modified/new files.
+      exts.add(".dict");
+      exts.add(".schema");
+      exts.add(".schema.original");
+      exts.add(".p2o");

       // we never delete ".schema.hints" or ".hints" as those are both
       // hand-coded configuration files

I tested #10613-1 and the issue is gone. There are no persistent schema changes in this scenario, but the temp-table is modified and this requires the dict, schema and p2o files to be created/updated. The changes above where conditioned by mode.schema, so I added Greg as a watcher for this input on this. I want to figure out another fix to leave the modes intact, but I need to do some brainstorming.

I am considering removing:

               if(schemasChanged.isEmpty())
               {
                  mode.schema = false;
                  if (mode instanceof CvtRunMode)
                  {
                     ((CvtRunMode) mode).skipDBSchema = true;
                     ((CvtRunMode) mode).omitSchemaBrew = true;
                  }
               }

Created 10613a from the latest trunk/16209.

Code Review Task Branch 10613a Revision 16210

Overall the logic seems correct.

schemasChanged == null || !schemasChanged.isEmpty() should be encapsulated into its own protected method hasSchemaChanges() in the TransformDriver. Make sure to add a comment that this will be true for full conversion or if there are any detected permanent schema changes in incremental mode. Then it can be called from all the locations instead of inlining the logic.

Code Review Task Branch 10613a Revision 16211

The changes are good.

The changes are pretty simple, so the test plan includes running ChUI and a customer conversion/incremental.

Dănuț Filimon wrote:

The changes are pretty simple, so the test plan includes running ChUI and a customer conversion/incremental.

Testing passed, 10613a can be merged.