Project

General

Profile

FWD Source Code Analytics

Added by Atul Dalvi almost 7 years ago

How can we use FWD only for Source Code Analytics without much complicated setup ?


Replies (155)

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

As suggested, I have copied contents of Hotel GUI project archive again in your C:\FWD\Project folder and then copied attached "prepare_template.cmd" file to C:\FWD\Project.

After running this cmd again I am getting below result. Is that OK ?
Is there any specific result that we can expect after running this script ?


C:\FWD\Project>prepare_template.cmd -a
Enter your application name (only letters or digits, first character a letter):
TALK3
1. Enter conversion configurations for your project
Edit the regex which matches all your include files (enter for default '*.[fhi]'
): .[fhi]
Edit the regex which matches all your program files (enter for default '
.[pPwW]
'): *.[pPwW]
Enter the PROPATH from your progress.ini file (enter for default '.:'): .;
2. Enter database configuration for your project
Enter the legacy database name(s), separated by a comma: mercator,procs,stddb,ta
lkcore,talkmig,wldb
1 file(s) copied.
Could Not Find C:\FWD\Project\data\hotel.df
The system cannot find the file specified.
The filename, directory name, or volume label syntax is incorrect.

C:\FWD\Project>


RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

Now I am running the Front End Conversion. Its is showing build failed with no errors.
How can we identify the errors ?

Is it possible to take the output in text file to very the result ?

[java] .\abl\group\prg\br\b_qlty1.w
[java] WARNING: Duplicate function definition fcalc1 with different return
types: character and date
[java] .\abl\group\prg\br\b_qlty2.w
[java] .\abl\group\prg\br\b_qltytr.w
[java] .\abl\group\prg\br\b_verrblad.w
[java] EXPRESSION EXECUTION ERROR:
[java] ---------------------------
[java] inSuperRef =
[java] #(com.goldencode.ast.Aast) hmver.get("in_super"))
[java] ---------------------------
[java] ERROR:
[java] com.goldencode.p2j.pattern.TreeWalkException: ERROR! Active Rule:
[java] -----------------------
[java] RULE REPORT
[java] -----------------------
[java] Rule Type : POST
[java] Source AST: [ block ] BLOCK/ @0:0 {5050881540097}
[java] Copy AST : [ block ] BLOCK/ @0:0 {5050881540097}
[java] Condition : inSuperRef =
[java] #(com.goldencode.ast.Aast) hmver.get("in_super"))
[java] Loop : false
[java] --- END RULE REPORT ---
[java]
[java]
[java]
[java] at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.ja
va:1068)
[java] at com.goldencode.p2j.convert.ConversionDriver.processTrees(Conv
ersionDriver.java:1128)
[java] at com.goldencode.p2j.convert.ConversionDriver.front(ConversionD
river.java:382)
[java] at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDr
iver.java:2005)
[java] Caused by: com.goldencode.expr.ExpressionException: Expression execu
tion error @2:52
[java] at com.goldencode.expr.Expression.execute(Expression.java:484)
[java] at com.goldencode.p2j.pattern.Rule.apply(Rule.java:491)
[java] at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:738)
[java]     at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:707)
[java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:503)
[java] at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:738)
[java]     at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:707)
[java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:503)
[java] at com.goldencode.p2j.pattern.RuleContainer.apply(RuleContainer.
java:583)
[java] at com.goldencode.p2j.pattern.RuleSet.apply(RuleSet.java:98)
[java] at com.goldencode.p2j.pattern.PatternEngine.apply(PatternEngine.
java:1650)
[java] at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEn
gine.java:1529)
[java] at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEn
gine.java:1477)
[java] at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.ja
va:1032)
[java] ... 3 more
[java] Caused by: java.lang.NullPointerException
[java] at com.goldencode.expr.CE365.execute(Unknown Source)
[java] at com.goldencode.expr.Expression.execute(Expression.java:391)
[java] ... 16 more
[java] .\abl\group\prg\br\b_vislie.w
[java] Elapsed job time: 00:01:12.106

BUILD FAILED
C:\FWD\Project\build.xml:252: Java returned: 1

Total time: 34 minutes 44 seconds

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

I am running for only 1 program using below command

java -classpath C:/FWD/Project/p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 C:/FWD/Project/abl/ExtractGeneratedFile.p > log.txt 2>&1

but getting log errors as below -

------------------------------------------------------------------------------
Scanning Progress Source (preprocessor, lexer, parser, persist ASTs)
------------------------------------------------------------------------------

C:\FWD\Project\abl\ExtractGeneratedFile.p

------------------------------------------------------------------------------
Elapsed job time: 00:00:04

------------------------------------------------------------------------------
Post-Parse Fixups
------------------------------------------------------------------------------

.\abl\ExtractGeneratedFile.p
Elapsed job time: 00:00:01.625

------------------------------------------------------------------------------
Gap Analysis Marking
------------------------------------------------------------------------------

.\abl\ExtractGeneratedFile.p
Elapsed job time: 00:00:01.843

------------------------------------------------------------------------------
Schema Fixups (data dictionary)
------------------------------------------------------------------------------

.\data\namespace\merc.dict
Elapsed job time: 00:00:00.594
ERROR:
com.goldencode.p2j.pattern.TreeWalkException: ERROR! Active Rule:
-----------------------
RULE REPORT
-----------------------
Rule Type : WALK
Source AST: [ merc ] DATABASE/ @0:0 {8589934593}
Copy AST : [ merc ] DATABASE/ @0:0 {8589934593}
Condition : tableAst.putAnnotation("srcfile", metaSrcFile)
Loop : false
--- END RULE REPORT --

t com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1068)
at com.goldencode.p2j.convert.ConversionDriver.processTrees(ConversionDriver.java:1128)
at com.goldencode.p2j.convert.ConversionDriver.front(ConversionDriver.java:406)
at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:2005)
Caused by: java.lang.NullPointerException
at com.goldencode.expr.CE2295.execute(Unknown Source)
at com.goldencode.expr.Expression.execute(Expression.java:373)
at com.goldencode.p2j.pattern.Rule.apply(Rule.java:491)
at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:738)
at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:707)
at com.goldencode.p2j.pattern.Rule.apply(Rule.java:503)
at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:738)
at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:707)
at com.goldencode.p2j.pattern.Rule.apply(Rule.java:530)
at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:738)
at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:707)
at com.goldencode.p2j.pattern.Rule.apply(Rule.java:530)
at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:738)
at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:707)
at com.goldencode.p2j.pattern.Rule.apply(Rule.java:530)
at com.goldencode.p2j.pattern.RuleContainer.apply(RuleContainer.java:583)
at com.goldencode.p2j.pattern.RuleSet.apply(RuleSet.java:98)
at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:261)
at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:209)
at com.goldencode.p2j.pattern.PatternEngine.apply(PatternEngine.java:1631)
at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1529)
at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1477)
at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1032)
... 3 more

Not getting what is the issue exactly.

RE: FWD Source Code Analytics - Added by Constantin Asofiei over 6 years ago

Hi Atul,

First, running ant conver.front, ant rpt or ant report_server (any task which ends up doing front conversion) will generate a log with the name cvt_<yyyyMMdd_HHmmss>.log - so you can check this file.

For your last error: what is the content of the data/ folder? Does it contain the standard.df schema? Also, do you have an entry like this in cfg/p2j.cfg.xml?

      <metadata name="standard">
         <table name="_db" />
         <table name="_file" />
         <table name="_field" />
         <table name="_index" />
         <table name="_index-field" />
         <table name="_user" />
      </metadata>

Your command to convert a single file is OK; you can also use a relative name, like ./abl/ExtractGeneratedFile.p instead of giving the full path.

RE: FWD Source Code Analytics - Added by Greg Shah over 6 years ago

Make sure you are running any of the parsing/conversion commands (the manual ConversionDriver or the ant build targets like convert.front) with your current directory set to your project dir (C:\FWD\Project\).

Please use relative paths for these commands, based on your location in that directory.

RE: FWD Source Code Analytics - Added by Greg Shah over 6 years ago

In regard to this issue:

...
     [java] inSuperRef =
     [java]                   #(com.goldencode.ast.Aast) hmver.get("in_super"))
...

This is happening during "post-parse-fixups", which is a later stage in the parsing process. It is likely that there are actual parsing failures prior to this (that probably cause this condition).

Please look at the log file that Constantin mentioned and look for any warnings or errors in the section titled "Scanning Progress Source (preprocessor, lexer, parser, persist ASTs)". In particular, I would expect a parsing problem in .\abl\group\prg\br\b_vislie.w or .\abl\group\prg\br\b_verrblad.w. Post the errors here unless it is obvious what to do.

RE: FWD Source Code Analytics - Added by Constantin Asofiei over 6 years ago

Atul Dalvi wrote:

After running this cmd again I am getting below result. Is that OK ?

Yes, that is OK. The "missing hotel.df" is probably because you ran the script more than once and the script wants to delete the hotel-related setup again (and the files are no longer there). This doesn't affect the configuration done by the script, so is OK.

Is there any specific result that we can expect after running this script ?

For now, there isn't one. But thanks for the suggestion, we will make sure to improve it so a specific failed/success message is logged.

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

I modified my cfg/p2j.cfg.xml file as attached and running the command from C:\FWD\Project path for parsing single program only.

java -classpath C:/FWD/Project/p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 C:/FWD/Project/abl/DCORP/Talk/appserver/reporting/ExtractGeneratedFile.p > log.txt 2>&1

But looking at the log file, I am not able to understand the log errors or issues.

Attached log file as well.

p2j.cfg.xml Magnifier (4.94 KB)

log.txt Magnifier (1.33 MB)

RE: FWD Source Code Analytics - Added by Constantin Asofiei over 6 years ago

Atul Dalvi wrote:

I modified my cfg/p2j.cfg.xml file as attached and running the command from C:\FWD\Project path for parsing single program only.

There should be a <metadata name="standard"> node, not one for each table schema - please make sure you have this node and try again.

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

I have 5 database that's why I added those entries in the config file. If I remove those and add only "standard", it is giving me errors as below, its not doing anything -

------------------------------------------------------------------------------
P2J Conversion Driver
------------------------------------------------------------------------------

------------------------------------------------------------------------------
SchemaLoader
------------------------------------------------------------------------------

ERROR:
com.goldencode.p2j.schema.SchemaException: Error importing schema data: standard
at com.goldencode.p2j.schema.SchemaLoader.importSchema(SchemaLoader.java:393)
at com.goldencode.p2j.schema.SchemaLoader.importAll(SchemaLoader.java:284)
at com.goldencode.p2j.convert.ConversionDriver.runSchemaLoader(ConversionDriver.java:459)
at com.goldencode.p2j.convert.ConversionDriver.front(ConversionDriver.java:372)
at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:2005)
Caused by: java.lang.NullPointerException
at com.goldencode.p2j.schema.SchemaLoader.importSchema(SchemaLoader.java:351)
... 4 more

Also I have added propath entry in the config file as below. Is that correct ? -

<parameter name="propath" value="./abl/:./abl/DCORP:./abl/DCORP/Talk:./abl/DCORP/Talk/group/:./abl/DCORP/Talk/group/prg/inc/:./abl/DCORP/Talk/group/prg/:./abl/CORP/Talk/appserver/reporting/" />

Sometimes in the log error I am getting below error, even after that file is available in the folder -

Command -

C:\FWD\Project>java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 ./abl/a.p > log.txt 2>&1

Log error -

------------------------------------------------------------------------------
Scanning Progress Source (preprocessor, lexer, parser, persist ASTs)
------------------------------------------------------------------------------

.\abl\a.p
Failure in file '.\abl\a.p':
com.goldencode.ast.AstException: Error processing .\abl\a.p
at com.goldencode.p2j.uast.AstGenerator.processFile(AstGenerator.java:962)
at com.goldencode.p2j.uast.ScanDriver.lambda$scan$0(ScanDriver.java:375)
at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:410)
at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:248)
at com.goldencode.p2j.convert.ConversionDriver.runScanDriver(ConversionDriver.java:496)
at com.goldencode.p2j.convert.ConversionDriver.front(ConversionDriver.java:377)
at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:2005)
Caused by: java.io.FileNotFoundException: File '.\abl\a.p' not found in path: .\abl\;.\abl\DCORP;.\abl\DCORP\Talk;.\abl\DCORP\Talk\group\;.\abl\DCORP\Talk\group\prg\inc\;.\abl\DCORP\Talk\group\prg\;.\abl\CORP\Talk\appserver\reporting\;
at com.goldencode.p2j.util.Utils.findFileInPath(Utils.java:929)
at com.goldencode.p2j.uast.AstGenerator.prepareDataStream(AstGenerator.java:1020)
at com.goldencode.p2j.uast.AstGenerator.prepareLexer(AstGenerator.java:1556)
at com.goldencode.p2j.uast.AstGenerator.parse(AstGenerator.java:1483)
at com.goldencode.p2j.uast.AstGenerator.processFile(AstGenerator.java:957)
... 6 more

Config file attached.

PLs suggest.

p2j.cfg.xml Magnifier (5.31 KB)

RE: FWD Source Code Analytics - Added by Eric Faulhaber over 6 years ago

Atul Dalvi wrote:

I have 5 database that's why I added those entries in the config file. If I remove those and add only "standard", it is giving me errors as below, its not doing anything -

There should be one <namespace> element per database schema (i.e., *.df file) in your project, plus one for the "standard" (i.e., metadata) schema, standard.df. These identify to FWD the schema definitions to be loaded into the FWD schema dictionary.

There should be one (and only one) <metadata> element, even if you have 5 databases. This identifies to FWD which of the schema definitions represents metadata only. This metadata schema is applied across all the other 5 schemata. So, for example, if your application references mercator._file, FWD will be able to identify such metadata references correctly.

Based on the configuration you posted, I believe the <schema> portion of your p2j.cfg.xml file should look like this:

   <schema>
      <namespace
         name="standard" 
         importFile="data/standard.df" 
         xmlFile="data/namespace/standard.dict" />
      <namespace
         name="mercator" 
         importFile="data/mercator.df" 
         xmlFile="data/namespace/mercator.dict" 
         default="true" >
         <parameter name="ddl-dialects" value="h2,postgresql" />
         <dialect-specific name="h2">
            <parameter name="collation" value="en_US_P2J" />
         </dialect-specific>
      </namespace>
      <namespace
         name="procs" 
         importFile="data/procs.df" 
         xmlFile="data/namespace/procs.dict" 
         default="true" >
         <parameter name="ddl-dialects" value="h2,postgresql" />
            <dialect-specific name="h2">
               <parameter name="collation" value="en_US_P2J" />
            </dialect-specific>
         </namespace>
      <namespace
         name="stddb" 
         importFile="data/stddb.df" 
         xmlFile="data/namespace/stddb.dict" 
         default="true" >
         <parameter name="ddl-dialects" value="h2,postgresql" />
         <dialect-specific name="h2">
            <parameter name="collation" value="en_US_P2J" />
         </dialect-specific>
      </namespace>
      <namespace
         name="talkcore" 
         importFile="data/talkcore.df" 
         xmlFile="data/namespace/talkcore.dict" 
         default="true" >
         <parameter name="ddl-dialects" value="h2,postgresql" />
         <dialect-specific name="h2">
            <parameter name="collation" value="en_US_P2J" />
         </dialect-specific>
      </namespace>
      <namespace
         name="talkmig" 
         importFile="data/talkmig.df" 
         xmlFile="data/namespace/talkmig.dict" 
         default="true" >
         <parameter name="ddl-dialects" value="h2,postgresql" />
         <dialect-specific name="h2">
            <parameter name="collation" value="en_US_P2J" />
         </dialect-specific>
      </namespace>
      <namespace
         name="wldb" 
         importFile="data/wldb.df" 
         xmlFile="data/namespace/wldb.dict" 
         default="true" >
         <parameter name="ddl-dialects" value="h2,postgresql" />
         <dialect-specific name="h2">
            <parameter name="collation" value="en_US_P2J" />
         </dialect-specific>
      </namespace>
      <metadata name="standard">
         <table name="_db" />
         <table name="_file" />
         <table name="_field" />
         <table name="_index" />
         <table name="_index-field" />
         <table name="_user" />
      </metadata>
   </schema>

Please ensure you have exported the metadata tables' schema from the Data Dictionary into a file called standard.df and that this file exists in your project at .\data\standard.df.

Let me know if the SchemaLoader phase completes successfully with these configuration changes.

RE: FWD Source Code Analytics - Added by Greg Shah over 6 years ago

<parameter name="propath" value="./abl/:./abl/DCORP:./abl/DCORP/Talk:./abl/DCORP/Talk/group/:./abl/DCORP/Talk/group/prg/inc/:./abl/DCORP/Talk/group/prg/:./abl/CORP/Talk/appserver/reporting/" />

Sometimes in the log error I am getting below error, even after that file is available in the folder -

Command -

C:\FWD\Project>java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 ./abl/a.p > log.txt 2>&1

Log error -

------------------------------------------------------------------------------
Scanning Progress Source (preprocessor, lexer, parser, persist ASTs)
------------------------------------------------------------------------------

.\abl\a.p
Failure in file '.\abl\a.p':
com.goldencode.ast.AstException: Error processing .\abl\a.p
at com.goldencode.p2j.uast.AstGenerator.processFile(AstGenerator.java:962)
at com.goldencode.p2j.uast.ScanDriver.lambda$scan$0(ScanDriver.java:375)
at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:410)
at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:248)
at com.goldencode.p2j.convert.ConversionDriver.runScanDriver(ConversionDriver.java:496)
at com.goldencode.p2j.convert.ConversionDriver.front(ConversionDriver.java:377)
at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:2005)
Caused by: java.io.FileNotFoundException: File '.\abl\a.p' not found in path: .\abl\;.\abl\DCORP;.\abl\DCORP\Talk;.\abl\DCORP\Talk\group\;.\abl\DCORP\Talk\group\prg\inc\;.\abl\DCORP\Talk\group\prg\;.\abl\CORP\Talk\appserver\reporting\;
at com.goldencode.p2j.util.Utils.findFileInPath(Utils.java:929)

Assuming that .\abl\a.p does exist, I suspect that the problem may be that FWD assumes that the propath will contain an entry for the project directory itself. This is usually done as ${P2J_HOME}:.

Try using this propath entry:

<parameter name="propath" value="${P2J_HOME}:./abl/:./abl/DCORP:./abl/DCORP/Talk:./abl/DCORP/Talk/group/:./abl/DCORP/Talk/group/prg/inc/:./abl/DCORP/Talk/group/prg/:./abl/CORP/Talk/appserver/reporting/" />

Let us know if this resolves the failure.

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

As suggested, I modified p2j.cfg.xml for the <schema> portion and the propath and now there is no error in the log file for a single .p file command. I have attached both log file and config file.

java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 ./abl/a.p > log.txt 2>&1

Also showing no error for the SchemaLoader phase

java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.schema.SchemaLoader [-q]

But looking at the log files for both the command, I am not able to identify whether the command has completed successfully or not.

Also now I am planning to run Parsing for whole source code with ant convert.front command.

Is there any possibility to run this command for few folders only ?

With convert.front.list command, it is possible for only files. Just like that I am looking for folders list.

log.txt Magnifier (1.33 MB)

p2j.cfg.xml Magnifier (3.99 KB)

RE: FWD Source Code Analytics - Added by Hynek Cihlar over 6 years ago

Atul Dalvi wrote:

Hi,

As suggested, I modified p2j.cfg.xml for the <schema> portion and the propath and now there is no error in the log file for a single .p file command. I have attached both log file and config file.

java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 ./abl/a.p > log.txt 2>&1

Also showing no error for the SchemaLoader phase

java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.schema.SchemaLoader [-q]

But looking at the log files for both the command, I am not able to identify whether the command has completed successfully or not.

From the log you attached the command finished with success.

Also now I am planning to run Parsing for whole source code with ant convert.front command.

Is there any possibility to run this command for few folders only ?

Currently it is not possible to specify multiple directories on the command line. The only option is to use the conversion file list in the convert.front.list ant target. Assuming your 4GL sources are located in a common root directory you can use dir command to generate the list: dir /b /s *.p *.w > file-cvt-list.txt.

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

OK thanks.

Few more doubts:

1. Can we change the default H2 database and How ?

2. If there is any change in the program after generating reports, we have to re-generate the AST again, execute calculate & generate commands again ?

RE: FWD Source Code Analytics - Added by Hynek Cihlar over 6 years ago

Atul Dalvi wrote:

OK thanks.

Few more doubts:

1. Can we change the default H2 database and How ?

At the moment FWD supports H2 and PostgreSQL databases. We've never had any issues with the H2 db backend for FWD reports. Do you have any specific (deployment) requirements where H2 would not be a good fit?

2. If there is any change in the program after generating reports, we have to re-generate the AST again, execute calculate & generate commands again ?

As far as I know yes. But I will check with my colleagues who are more familiar with the internals of FWD Code Analytics.

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

OK np.

One more thing I noticed here. Every where we are just mentioning about .p, .w and .cls programs. What about include (.i) programs ?
AST and reports will not execute for include files ?
In above post also u have mentioned about "dir /b /s *.p *.w > file-cvt-list.txt" not about ".i" extension.

RE: FWD Source Code Analytics - Added by Greg Shah over 6 years ago

1. Can we change the default H2 database and How ?

For the use of reporting, we only have used H2. Some technical work would be required to test other databases, if there were issues we would need to do some work for enabling those. At a minimum, we don't currently provide a way to specify the database URL. The idea here is that H2 is embedded inside our reporting server.

For converted applications it is different. We support H2, PostgreSQL and SQLServer. It is also reasonable to add other databases over time.

2. If there is any change in the program after generating reports, we have to re-generate the AST again, execute calculate & generate commands again ?

Correct. Most people setup a scheduled job (once a day) to check out the latest code and run ant rpt which does both the parsing and the report generation. Then they start the report server.

Every where we are just mentioning about .p, .w and .cls programs. What about include (.i) programs ?
AST and reports will not execute for include files ?

Any include files that are referenced by the external procedures or classes will be utilized by our preprocessor. This is a necessary step to turn the original external procedures and classes into valid 4GL source code. Any preprocessor usage in those procedures/classes means that the original source file is not actually compilable 4GL. After the preprocessor runs, we save the results in a <sourcefilename>.cache file (e.g. ./abl/a.p.cache. This is the fully expanded 4GL code, with inclusion of any preprocessor references, conditionally preprocessed code and all referenced include files.

For this reason, you don't have to tell us about the include files. Your procedures/classes have the linkages already encoded. The only thing needed is to ensure that the propath is correct so that the preprocessor finds all the include files that are referenced.

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Thanks Greg.

Now I am executing parsing for a big program but I am getting error as below. Does it related to the JVM memory. What will the fix for this ?

  1. There is insufficient memory for the Java Runtime Environment to continue.
  2. Native memory allocation (mmap) failed to map 421527552 bytes for Failed to commit area from 0x00000000a4300000 to 0x00000000bd500000 of length 421527552.
  3. An error report file with more information is saved as:
  4. C:\FWD\Project\hs_err_pid1640.log
    Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000a4300000, 421527552, 0) failed; error='The paging file is too small for this operation to complete' (DOS error/errno=1455)

RE: FWD Source Code Analytics - Added by Greg Shah over 6 years ago

Questions:

  • The JVM is 64-bit, I assume the Windows platform on which you are running is also 64-bit. What version of Windows are you running?
  • How much memory does the system or VM have?
  • What command are you running to parse?

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

The system details is below -

Windows server 2012 R2 - 64 bit

System Boot Time: 10/10/2017, 12:56:44
System Manufacturer: VMware, Inc.
System Model: VMware Virtual Platform
System Type: x64-based PC
System Directory: C:\Windows\system32
System Locale: nl-be;Dutch (Belgium)
Total Physical Memory: 8 192 MB
Available Physical Memory: 4 650 MB
Virtual Memory: Max Size: 10 304 MB
Virtual Memory: Available: 1 489 MB
Virtual Memory: In Use: 8 815 MB

And I am using below command for a single file.

java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 ./abl/DCORP/Talk/appserver/reporting/ExtractGeneratedFile.p > log.txt 2>&1

RE: FWD Source Code Analytics - Added by Hynek Cihlar over 6 years ago

Atul Dalvi wrote:

java -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 ./abl/DCORP/Talk/appserver/reporting/ExtractGeneratedFile.p > log.txt 2>&1

Since you don't specify the amount of heap size the JVM can utilize it will default to 1GB in your case. This is not enough to run the reporting engine. Please change the command to
java -Xmx4G -classpath ./p2j/build/lib/p2j.jar com.goldencode.p2j.convert.ConversionDriver f2 ./abl/DCORP/Talk/appserver/reporting/ExtractGeneratedFile.p > log.txt 2>&1

You may increase the size a bit higher if needed since your physical memory is 8G. I wouldn't recommend to go beyond 6G in your case though.

RE: FWD Source Code Analytics - Added by Greg Shah over 6 years ago

One thing that is strange here is that this is the ConversionDriver, which should easily run in smaller heaps. Even a large program, when parsed by itself would parse in 256MB I think.

RE: FWD Source Code Analytics - Added by Hynek Cihlar over 6 years ago

Greg is correct, ConversionDriver with the f2 mode should not need that much heap. Atul, what version of Java are you using? Please paste the result of the command java -version.

RE: FWD Source Code Analytics - Added by Atul Dalvi over 6 years ago

Hi,

I am using Java version as below -

C:\Users\H035009>java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

(26-50/155)