FWD » Core Development » Conversion Tools

The .cache and .p.ast files do exist, but they are in ./cvt/ instead of ./abl/ which is probably the issue.

The error is the same one mentioned in #10784-1. Octavian found a workaround for this, but I will see if I can fix it.

I've got ./abl/fwd-embedded-driver.p which is normalized to ./deploy/server/abl/fwd-embedded-driver.p. My initial idea was that the normalizeFilename home parameter should be the directory for the given filename, so I made this patch:

=== modified file 'src/com/goldencode/p2j/cfg/Configuration.java'
--- old/src/com/goldencode/p2j/cfg/Configuration.java    2025-10-08 07:50:27 +0000
+++ new/src/com/goldencode/p2j/cfg/Configuration.java    2025-11-25 08:25:45 +0000
@@ -1589,8 +1589,19 @@
     */
    public static String normalizeFilename(String home, String filename)
    {
-      File   abnormal = new File(filename); 
+      String prefix = "." + File.separator;
+      File abnormal;
       String absolute;
+      if (filename.startsWith(prefix))
+      {
+         // remove the dot
+         filename = filename.substring(1);
+         abnormal = new File(home + filename);
+      }
+      else
+      {
+         abnormal = new File(filename);
+      }

       try
       {

Won't this be a problem when we reach the if (absolute.startsWith(home)) code later in that method? Why not have the ReportDriver configure a relative path that is a "fixup" which "gets you" to the project home directory. This could be calculated or configured in the ReportDriver and only if fixup is not null would be add it.

Also, assuming home is always the same value, can't home = new File(home).getCanonicalPath(); be calculated at startup and used as a constant? This is just a possible optimization.

Greg Shah wrote:

Won't this be a problem when we reach the if (absolute.startsWith(home)) code later in that method? Why not have the ReportDriver configure a relative path that is a "fixup" which "gets you" to the project home directory. This could be calculated or configured in the ReportDriver and only if fixup is not null would be add it.

No, it will just remove home from the path. We already have home which is the relative path that we need, so I don't see a use for another path.

For #10919-10: It can be calculated once and stored.

No, it will just remove home from the path. We already have home which is the relative path that we need, so I don't see a use for another path.

What is home set to for the ReportDriver?

Greg Shah wrote:

No, it will just remove home from the path. We already have home which is the relative path that we need, so I don't see a use for another path.

What is home set to for the ReportDriver?

When running the conversion or report server, it is /home/ddf/gcd/branches/hotel_gui (project root).

Above, I was saying ReportDriver but that was a mistake. I meant ReportWebServer.

If the files are being read by the ReportWebServer using a relative path AND the working directory for the ReportWebServer is in <project_home>/deploy/server/, then the files won't be found even if you fix the ./abl/ to ./cvt/.

I fixed this in the report.sh script:

=== modified file 'deploy/server/report.sh'
--- old/deploy/server/report.sh    2024-11-06 15:12:06 +0000
+++ new/deploy/server/report.sh    2025-11-26 08:41:44 +0000
@@ -3,25 +3,29 @@
 port=$1
 logfile=$2

+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" 
+cd ../..
+
 [ ."$port". == ".." ]    && port=9443
 [ ."$logfile". == ".." ] && logfile="report_server.log" 

 # Find FWD libraries
-fwd_lib=$([ -n "$FWD_LIB" ] && echo $FWD_LIB || echo "../../p2j")
+fwd_lib=$([ -n "$FWD_LIB" ] && echo $FWD_LIB || echo "./p2j")
 if [ ! -e "$fwd_lib" ]; then
    echo "ERROR: Either FWD_LIB must be set to a valid directory or ./p2j directory must exist so as to locate p2j.jar" 
    exit 1
 fi
 p2j_jar=$([ -e "${fwd_lib}/build/lib/p2j.jar" ] && echo ${fwd_lib}/build/lib/p2j.jar || echo ${fwd_lib}/lib/p2j.jar)
-p2j_jar=$([ -e "$p2j_jar" ] && echo $p2j_jar || echo ../../p2j/lib/p2j.jar)
+p2j_jar=$([ -e "$p2j_jar" ] && echo $p2j_jar || echo ./p2j/lib/p2j.jar)

 # defaults
 cpath="${p2j_jar}:.:" 
-dtxt="-Xnoagent -DP2J_HOME=../.. -Djava.compiler=NONE" 
+dtxt="-Xnoagent -DP2J_HOME=. -Djava.compiler=NONE" 
+debug="-Xdebug -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=2100,server=y,suspend=y" 

-java $dtxt -classpath $cpath com.goldencode.p2j.report.server.ReportWebServer \
+java $dtxt $debug -classpath $cpath com.goldencode.p2j.report.server.ReportWebServer \
                              -port $port \
-                             -ks ./embedded-private-key.store \
+                             -ks ${SCRIPT_DIR}/embedded-private-key.store \
                              -kspw 87ViGTQVp\<07zPw\(3i2A4%uCHYx3mIgSt\>Tk \
                              -kepw h8jNMO\>Vbn0\`cL\)b95dVn\$w03CePIB6eItL3 \
                              1> $logfile 2>&1

I'm OK with the idea. Please add comments to make it clear that the ReportWebServer MUST be executed with the project home as the working directory.

This change would have to be migrated to every conversion project.

Roger: Please review.

Roger, this is the final patch:

=== modified file 'deploy/server/report.sh'
--- old/deploy/server/report.sh    2024-11-06 15:12:06 +0000
+++ new/deploy/server/report.sh    2025-11-26 12:32:08 +0000
@@ -3,25 +3,31 @@
 port=$1
 logfile=$2

+# Save the current directory.
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" 
+
+# Change the current working directory to the project home directory.
+cd ../..
+
 [ ."$port". == ".." ]    && port=9443
 [ ."$logfile". == ".." ] && logfile="report_server.log" 

 # Find FWD libraries
-fwd_lib=$([ -n "$FWD_LIB" ] && echo $FWD_LIB || echo "../../p2j")
+fwd_lib=$([ -n "$FWD_LIB" ] && echo $FWD_LIB || echo "./p2j")
 if [ ! -e "$fwd_lib" ]; then
    echo "ERROR: Either FWD_LIB must be set to a valid directory or ./p2j directory must exist so as to locate p2j.jar" 
    exit 1
 fi
 p2j_jar=$([ -e "${fwd_lib}/build/lib/p2j.jar" ] && echo ${fwd_lib}/build/lib/p2j.jar || echo ${fwd_lib}/lib/p2j.jar)
-p2j_jar=$([ -e "$p2j_jar" ] && echo $p2j_jar || echo ../../p2j/lib/p2j.jar)
+p2j_jar=$([ -e "$p2j_jar" ] && echo $p2j_jar || echo ./p2j/lib/p2j.jar)

 # defaults
 cpath="${p2j_jar}:.:" 
-dtxt="-Xnoagent -DP2J_HOME=../.. -Djava.compiler=NONE" 
+dtxt="-Xnoagent -DP2J_HOME=. -Djava.compiler=NONE" 

 java $dtxt -classpath $cpath com.goldencode.p2j.report.server.ReportWebServer \
                              -port $port \
-                             -ks ./embedded-private-key.store \
+                             -ks ${SCRIPT_DIR}/embedded-private-key.store \
                              -kspw 87ViGTQVp\<07zPw\(3i2A4%uCHYx3mIgSt\>Tk \
                              -kepw h8jNMO\>Vbn0\`cL\)b95dVn\$w03CePIB6eItL3 \
                              1> $logfile 2>&1

I also committed this comment to 10919a/16293.

=== modified file 'src/com/goldencode/p2j/report/server/ReportWebServer.java'
--- old/src/com/goldencode/p2j/report/server/ReportWebServer.java    2024-07-22 14:19:39 +0000
+++ new/src/com/goldencode/p2j/report/server/ReportWebServer.java    2025-11-26 12:33:43 +0000
@@ -154,6 +154,8 @@

    /**
     * Command line interface to launch web server.
+    * <p>
+    * The web server must be executed with the project home as the working directory.
     * 
     * @param   args
     *          Required arguments.

So you start the web server as: ./deploy/server/report.sh?

Roger Borrello wrote:

So you start the web server as: ./deploy/server/report.sh?

No, like this: ~/gcd/branches/hotel_gui/deploy/server$ ./report.sh

Dănuț Filimon wrote:

Roger Borrello wrote:

So you start the web server as: ./deploy/server/report.sh?

No, like this: ~/gcd/branches/hotel_gui/deploy/server$ ./report.sh

OK. I had crossed the comment from the Java code with the comment from the changing of the directory in the script. This should be fine, especially because that's how Docker environment is running reports server already.

Dănuț Filimon wrote:

Thanks Roger, is there any testing I can do aside from running the report server for Hotel?

Without actually having a 10919a FWD Docker image, not much. Getting one build would involve the container project, and:

./build_fwd.sh -b 10919a -l projects/10919a --latest --jdk_version=jdk17

Then building Hotel GUI Reports with that FWD Docker image and running it:

./docker/build_docker.sh --jdk_version=jdk17 -r . -t 10919a_latest -l --reports
./docker/run_docker.sh --reports

but that is going from memory.

Dănuț Filimon wrote:

Thanks Roger, is there any testing I can do aside from running the report server for Hotel?

This change isn't specific to Docker. You can test the report server from any Hotel GUI install. Make sure to use ant report_server as the target so that you have both reports and call graph. Then you should be able to open the source view from any details report. And you should be able to test a custom search, which is also dependent upon being able to access AST files.

Committed hotel_gui/419, hotel/186. Fixed reports using the wrong location for AST resource.

ETF does not have a report.sh.

ChUI, testcases, customer 1, 2, and 3 need small adjustments. But I'd also need the customer to accept the change before I commit it to their repository branch.

Committed xfer testcases/1817.

I am merging 10919a now so that the trunk revision can be referenced.

Branch 10919a was merged into trunk as rev. 16308 and archived.

Only a customer and ChUI scripts are left to be updated.

I've tested the fix, although it works, it is VERY SLOW. Even use in a small customer project (500KLOC) takes 10+ seconds to load the source/AST view. Even in Hotel GUI, it takes 3-5 seconds at its fastest and 30 seconds for some loads. This is not right.

Please look into this.

I tried to find the revision which introduced this performance issue using VisualVM. I am now switching to YourKit to investigate the root cause.

Committed 10919b/16339. Noticed there were a lot of methods that only do read only operations and also call commit(). I made this change and managed to improve performance a bit.

It is much faster now.

The changes only target the ReportApi, so I will take two customer applications and test them.

There seems to be a bug in a customer report conversion:

     [java] ERROR!  Active Rule:
     [java] -----------------------
     [java]       RULE REPORT      
     [java] -----------------------
     [java] Rule Type :   POST
     [java] Source AST:  [ block ] BLOCK/ @0:0 {6472515715073}
     [java] Copy AST  :  [ block ] BLOCK/ @0:0 {6472515715073}
     [java] Condition :  rw.postprocessFile()
     [java] Loop      :  false
     [java] --- END RULE REPORT ---
     [java] 
     [java] 
     [java] 
     [java] com.goldencode.p2j.pattern.TreeWalkException: ERROR!  Active Rule:
     [java] -----------------------
     [java]       RULE REPORT      
     [java] -----------------------
     [java] Rule Type :   POST
     [java] Source AST:  [ block ] BLOCK/ @0:0 {6472515715073}
     [java] Copy AST  :  [ block ] BLOCK/ @0:0 {6472515715073}
     [java] Condition :  rw.postprocessFile()
     [java] Loop      :  false
     [java] --- END RULE REPORT ---
     [java] 
     [java] 
     [java] 
     [java]     at com.goldencode.p2j.pattern.PatternEngine.handleRunThrowable(PatternEngine.java:1938)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1070)
     [java]     at com.goldencode.p2j.report.ReportDriver.patternEngine(ReportDriver.java:181)
     [java]     at com.goldencode.p2j.report.ReportDriver.oldSyntax(ReportDriver.java:578)
     [java]     at com.goldencode.p2j.report.ReportDriver.main(ReportDriver.java:320)
     [java] Caused by: java.lang.NullPointerException: Cannot invoke "com.goldencode.p2j.schema.NameNode.getType()" because "fromNode" is null
     [java]     at com.goldencode.p2j.schema.SchemaDictionary.addEntries(SchemaDictionary.java:5154)
     [java]     at com.goldencode.p2j.schema.SchemaDictionary.addFieldEntries(SchemaDictionary.java:2953)
     [java]     at com.goldencode.p2j.uast.ClassDefinition.<init>(ClassDefinition.java:689)
     [java]     at com.goldencode.p2j.uast.SymbolResolver.loadClassDefinition(SymbolResolver.java:2461)
     [java]     at com.goldencode.p2j.uast.SymbolResolver.loadClassDefinition(SymbolResolver.java:2411)
     [java]     at com.goldencode.p2j.uast.SymbolResolver.loadClassDefinition(SymbolResolver.java:2393)
     [java]     at com.goldencode.p2j.uast.SymbolResolver.loadConvertedClass(SymbolResolver.java:2374)
     [java]     at com.goldencode.p2j.uast.SymbolResolver.resolveClassDefinition(SymbolResolver.java:1347)
     [java]     at com.goldencode.p2j.schema.P2OAccessWorker.visitAst(P2OAccessWorker.java:264)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1540)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1507)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1065)
     [java]     ... 3 more
     [java] 

ChUI also failed the conversion with Cannot find class/interface ... in PROPATH.

Regardless, the customer mentioned in #10919-40 allowed me to start the report server and test, the load time was pretty smooth and fast. I also noticed the script is missing the changes from #10919-24.