FWD » Core Development » Conversion Tools

07/06/2026

09:05 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

The issue was solved, latest 10804b rev is *16627*.
I'll run another overnight conversion, it shouldn't hang at th... Paula Păstrăguș

07:58 AM Feature #7178: move the buffer scoping calculations to parse time

I integrated the remaining tests from #7178-25 and #7178-27. Dănuț Filimon

07/03/2026

01:56 PM Feature #10804: create replacement AST implementation for use with ANTLR 4

I have some good news to share! One of the larger applications successfully passed the first phase of the conversion,... Paula Păstrăguș

07/02/2026

09:35 AM Feature #9638: resolve OO dependencies during parsing using annotations in already converted classes

During today's meeting, we decided to keep the changes in the @oo@ package since the @skeleton@ project is difficult ... Octavian Adrian Gavril

07:48 AM Feature #7178: move the buffer scoping calculations to parse time

> I can still integrate tests from #4105, #4121, #4207, #4208, #4231 (double check), #4440, #4492, #4782, #4791, #665... Greg Shah

06:41 AM Feature #7178: move the buffer scoping calculations to parse time

Second round.
I did not find any tests for #4123, #4124, #4147, #4160, #4162, #4170, #4171, #4174, #4181, #4182, #... Dănuț Filimon

07/01/2026

09:47 AM Feature #7178: move the buffer scoping calculations to parse time

> Greg, I think there's a good reason to go into the related tasks for all those tasks and check if we can find more ... Greg Shah

09:41 AM Feature #7178: move the buffer scoping calculations to parse time

Amazing!
I did not find any tests for #1521, #1868, #1896, #1969, #2683, #3024, #3253, #3255, #3310, #3471, #3490,... Dănuț Filimon

05:20 AM Feature #9638: resolve OO dependencies during parsing using annotations in already converted classes

It looks like the test case above isn't compiling in OE and requires a @SUPER()@ call to be correct. Consequently, I ... Octavian Adrian Gavril

06/30/2026

10:23 AM Feature #7178: move the buffer scoping calculations to parse time

In the latest standard workstation, I've added a tool called @bzr-issues@ which can be used to build a list of all ta... Greg Shah

06/29/2026

03:15 PM Feature #1757: update ANTLR to latest version

A wiki page with guidance on how we migrate from ANTLR2 to ANTLR4 is in [[p2j:ANTLR Migration Guide]]. Greg Shah

06/24/2026

10:06 AM Feature #6320: sharded schemata

Yes, using a per-module namespace would be a way to solve the issue I noted in #6320-5. Greg Shah

10:03 AM Feature #6320: sharded schemata

Greg Shah wrote:
> > This means that apps/modules need to be able to be converted in pieces and executed at runtime ... Octavian Adrian Gavril

07:08 AM Feature #7178: move the buffer scoping calculations to parse time

I've wrapped up the FOR EACH WHERE/BY, dataset with relation-fields, lots of unqualified fields/tables tests.
I'm ... Dănuț Filimon

06/23/2026

03:53 AM Feature #9638: resolve OO dependencies during parsing using annotations in already converted classes

Constantin, have you had a chance to look at the approach mentioned above? Let me know what you think. Octavian Adrian Gavril

06/22/2026

07:11 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

I've committed the build changes as rev *16622*. @./gradlew all@ should work now. Paula Păstrăguș

06/19/2026

06:39 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

10804b was rebased, new rev is *16621*. Paula Păstrăguș

06:08 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Next, I'll rabase the branch, setup new big application, redo the conversion, and solve the remaining parser issues. ... Paula Păstrăguș

05:04 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Paula Păstrăguș wrote:
> One testcase is still failing: @AdaptiveQuery.initialize@ now receives @WHOLE-INDEX@ as a p... Paula Păstrăguș

06/18/2026

08:10 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

*Update*: LATEST Hotel GUI performance comparison between ANTLR2 and ANTLR4. (Please note that all conversions were e... Paula Păstrăguș

04:38 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

For Hotel GUI, the total time needed to generate the tokens with the ANTLR2 based lexer and convert them to ANTLR4 co... Paula Păstrăguș

08:02 AM Feature #7178: move the buffer scoping calculations to parse time

The tests were developed on the examples from #7178-2.
This is a recap of the tests that were completed or have a ... Dănuț Filimon

06/17/2026

09:24 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

The approach seems to work! 🥳 I was able to get the same AST for @standard.df@. However, there are still a few remain... Paula Păstrăguș

06/16/2026

06:43 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

I am moving to an alternative approach for the ANTLR4 lexer performance issue (based on the idea mentioned by Greg in... Paula Păstrăguș

06/15/2026

02:59 PM Feature #10804: create replacement AST implementation for use with ANTLR 4

I added temporary lexer profiling in order to measure the token generation cost per token type, aggregated across all... Paula Păstrăguș

12:26 PM Feature #11540: rework conversion back-end to add optional support for output to C#

This is long awaited feature. The conversion architecture was built with thin in mind. When I first read about it, I ... Ovidiu Maxiniuc

09:41 AM Feature #11540: rework conversion back-end to add optional support for output to C#

There are some customers that prefer C# to Java.
The core of both languages have nearly identical syntax and C# ... Greg Shah

09:21 AM Feature #11540 (New): rework conversion back-end to add optional support for output to C#

Greg Shah

06/12/2026

08:42 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Before moving forward, another project conversion progressed further than in previous attempts, it reached the *serve... Paula Păstrăguș

08:35 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Another slowdown source is the @evaluateExpr@ method: under ANTLR2, Hotel GUI spends *5.3 seconds* in it, whereas und... Paula Păstrăguș

06/11/2026

08:16 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

10804b is at rev *16584* now (here you can find today's parser improvements). Paula Păstrăguș

08:03 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

I also attached a log file in my previous post. It includes the total time ANTLR4 spends in prediction mode for each ... Paula Păstrăguș

08:00 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Hmm, for Hotel GUI, @createShadowNodes@ takes around @0.2@ seconds in total, which seems reasonable from my POV. (we ... Paula Păstrăguș

07:51 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

In this 2X difference for the front end, 40% comes from the ANTLR4 code and 60% from something else. The other major... Greg Shah

07:45 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

*Update*: Hotel GUI performance comparison between ANTLR2 and ANTLR4.
|_. Metric |_. ANTLR2 |_. ANTLR4 |
| Total ... Paula Păstrăguș

06/10/2026

02:35 PM Feature #11528: implement COMPILE stmt

Although this is a 4GL language statement, possibly best considered part of Base Language, from our perspective it ha... Greg Shah

02:22 PM Feature #11528 (New): implement COMPILE stmt

Greg Shah

10:12 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

10804b is at rev *16583* now (the latest). Paula Păstrăguș

10:04 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Greg, it is not 3x slower anymore, it is only around 2x now, and I think I can still push the boundaries:) Paula Păstrăguș

10:00 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

*Update*: the current Hotel GUI parse time is *49 seconds*. I think I can still improve this further through parser c... Paula Păstrăguș

09:20 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Right, I remember having this error from the very beginning, basically two years ago:-)
Anyway, I think a fresh ch... Paula Păstrăguș

09:16 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

> It is not valid, a dot is missing here:
Hmmm. Something is wrong here with the project. You are working with H... Greg Shah

07:11 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Greg Shah wrote:
> ANTLRv2 and ANTLRv4 both have their loops and optional alternatives set to be greedy by default. ... Paula Păstrăguș

07:06 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Paula Păstrăguș wrote:
> Greg Shah wrote:
> > > This suggests that we may also need some special tricks for cases w... Paula Păstrăguș

07:06 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Greg Shah wrote:
> > This suggests that we may also need some special tricks for cases where a @no viable alternativ... Paula Păstrăguș

06:54 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Paula Păstrăguș wrote:
> Another important thing I noticed is that the ANTLR4 prediction algorithm does not seem to ... Greg Shah

06:49 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

> One bottleneck I identified is the @toolbar.p@ file. It throws an NPE caused by a @no viable alternative@, which is... Greg Shah

06:23 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Greg, is a @()*@ construct in ANTLR2 greedy or non-greedy by default? I only found three @options@ blocks with @greed... Paula Păstrăguș

06:11 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

Another important thing I noticed is that the ANTLR4 prediction algorithm does not seem to provide a reliable way to ... Paula Păstrăguș

05:44 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

*Update*: Hotel GUI parse time has now dropped to *1 minute and 8 seconds*, with a total conversion time of *5 minute... Paula Păstrăguș

06/09/2026

02:31 PM Feature #10804: create replacement AST implementation for use with ANTLR 4

> Greg, I remember you suggested something related to this at some point, but I do not recall the exact details. Coul... Greg Shah

10:22 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

The lexer doesn't appear to be the bottleneck. Using @tokens.fill()@ and summing the elapsed times across all Hotel_G... Paula Păstrăguș

09:42 AM Feature #10804: create replacement AST implementation for use with ANTLR 4

I think it is time to document some findings and updates here.
At this point, I think I have stabilized the Progre... Paula Păstrăguș

12:48 PM Support #6409: generate "railroad style" syntax diagrams for the progress parser

h1. Analysis of Approach
h2. Objective
Provide high-quality, readable "railroad" syntax diagrams for the full r... Greg Shah