This crossed my way a couple a weeks ago and now I am feeling guilty that I did not make a post in time. But maybe it is not too late:

• The "DSL" part of the title is the one that triggered my thoughts on Groovy and Kotlin. Both are JVM languages that are close to Java in terms of paradigm (imperative, OO, etc. / but maybe a bit more oriented to scripting?). The point is that both got remarkable in terms of defining DSLs in.
• The syntax for higher order functions and affinity for functional programming makes a statement on their suitability to support DSL definition.
• A proof of concept for this is Gradle building tool (which FWD actually uses), that accepts both Groovy and Kotlin syntax. The FWD Gradle script uses a Groovy syntax, but at first sight, one can simply state that Gradle seems to be defining its building tasks in its own language - a DSL.
• I can agree that Scala's functional paradigm is more suitable to pattern matching (that fits parsers), but does it actually fit TRPL? I suppose the conversion rules can be interpreted as patterns that trigger actions in the conversion routine.

I wanted to bring this up because #1747 was posted 12 years ago (when kotlin was just born). I can't say the same on Groovy.
Let me know of your thoughts. IMHO, I have little experience with both Scala and Groovy/Kotlin, so from technical POV I would enjoy seeing TRPL supporting any of these :)

At one point (15? years ago now) I tested the performance of Groovy vs Java and found that Groovy was several orders of magnitude slower. I think that was at the same time that I was evaluating whether we could use Groovy as the target language for the converted code. At least at the time they did not have the ability to override the assignment operator so some of the most important operator override requirements were not met.

For TRPL, we have the requirement of overriding various operators but we also have other complex requirements. It is a good point about the pattern matching possibilities in Scala, which we may be able to leverage. My sense is also that Scala has a better performance profile than most other non-Java JVM-hosted languages.

To be clear, we can consider any of the JVM-hosted languages (not just Scala). I haven't looked at Kotlin but it has a good reputation. Whatever choice we make must be a well supported language. A big advantage of using a well supported language is to get get full source-level debugging for TRPL.

Greg Shah wrote:

Whatever choice we make must be a well supported language.

Java is still kinda supported, right?

Vladimir Tsichevski wrote:

Greg Shah wrote:

Whatever choice we make must be a well supported language.

Java is still kinda supported, right?

Java is fully supported as a language. But I'm not sure how that relates to this task. The idea of this task was to explore how an already-existing JVM hosted language could potentially replace TRPL. To do that we would need some language features that don't exist in Java, but some of which do exist in some of the JVM hosted languages like Scala.

Anyway, we aren't working on this task yet.

Where is the replacement code for the pattern engine, SR etc... the actual Scala version of TRPL itself?

Please have your Claude session write a document of its transformation rules (how each TRPL feature converts into Scala).

The language features are documented here:

• Writing TRPL Rule-Sets
• Writing TRPL Expressions

If something doesn't have a natural replacement, has limitations in its replacement or if wasn't yet considered, it is OK to say so. Post the documentation here (in Redmine textile format, of course) as a note.

I will note that if and when we use Scala, we will certainly be using curly braces and forcing indentation mode off in the compiler. I don't care that the Scale community has lost their collective minds with indentation mode. We will not use it.

We will also maintain our blank lines in between rules and logically separate code for readability.

I would like to put my changes somewhere, in a branch, both for translator and for the PatternEngine + Scala rules.
Should I create 1747a, with the translator in that branch?

But so far, the only missing natural replacement was about null safety. There is no such null in TRPL. I will think more about that for Scala rules.
For now, I have converted every expression with ?., which is the Scala operator to add null checks and to yield null when one operand in the chain is null.

I have added myself in the watchers list.

Greg Shah wrote:

I will note that if and when we use Scala, we will certainly be using curly braces and forcing indentation mode off in the compiler. I don't care that the Scale community has lost their collective minds with indentation mode. We will not use it.

We will also maintain our blank lines in between rules and logically separate code for readability.

Got it.

Yes, go ahead and creatd 1747a and put all your changes there.

Please create a document that describes the (TRPL to Scala) mappings implemented, how the engine was reworked, what is working/not working, any limitations or problems that are known. Post it here in Redmine textile format.

I have created 1747a, which contains the migrated rules to Scala, alongside with the translator and runtime changes to leverage them.
At current state, hotel_gui + 2 more big project converts and runs without any issue.

The setup for this is not being changes, so to use this, you need to point this branch to the project working tree, build FWD normally and just it for runtime/conversion.

AI generated description, describing TRPL to Scala migration process and the work done.¶

TRPL rules in Scala¶

The conversion rules are now Scala: each TRPL file rules/<dir>/<file>.<ext> is a class in rules/<dir>/<file>.<ext>.scala, compiled into p2j.jar (1747a, r16809). Until the branch is merged, each TRPL file stays next to its translation, for reference and as the translator's input.

TRPL to Scala¶

Each x.? marks a value TRPL null-checks. The nullSafe macro, applied to every rule block, makes the enclosing comparison or and/or operand evaluate to false on null, as TRPL does. A null anywhere else throws, as in TRPL. Comparisons, numeric widening and casts follow the TRPL expression compiler.

Generated code layout¶

• 3-space indentation; lines up to 110 columns.
• Braces on their own lines; if (cond) and else if; a class's parent on its own extends line.
• Long expressions wrap at &&/|| chains (operator at line end), method chains (break before ., dots aligned) and argument lists (aligned after ().
• Blank lines of the TRPL file are kept (at most one in a row); TRPL comments become // comments.

One exception: a DSL call taking a block keeps its brace on the call's line (walkRules {, loop(cond) {, funcLibrary() {, = nullSafe {). Scala 3 starts a new statement at a brace on the next line, silently where the call needs no argument.

The rules compile with -no-indent -old-syntax, so indentation syntax and if ... then cannot creep back.

Engine changes¶

• RuleProvider (new): ConfigLoader asks the provider (TrplProvider, found through ServiceLoader in p2j.jar) for a rule file before reading any XML.
• A Scala class builds the DOM its XML would parse into. Each rule list and each function body is a <code> element, which becomes a TrplCode rule list element. All other declarations still go through ConfigLoader unchanged.
• RuleContainer: added addElement, registerVariable with an Initializer, and a per-clone code binding. NamedFunction: added registerElement and the Initializer variants. BaseRuleContainer applies global elements as RuleListElement.
• PatternEngine: a failure of finish() no longer hides the original rule-set failure.
• Speed-ups: AnnotatedAst (no interning of annotation keys), XmlFilePlugin (unsynchronized buffered AST reads), SymbolResolver/Variable (resetVariables sets the scope only when an initializer needs it).
• Scala runtime: src/com/goldencode/p2j/pattern/Trpl*.scala, built into p2j.jar.

Loading¶

• The Scala class loads, unless the rule name resolves to an XML file without a .scala file next to it. So a project's own rules, or any override placed on patpath, still win.
• Conversion looks only at patpath. The server, and jar mode, also look at p2j.jar resources, as ConfigLoader does.
• On the server, the runtime profiles (runtime/postprocess_*) load from p2j.jar. The Scala library comes through fwdConvertServer, so it is on p2j.jar's class path and in the server distribution.
• -Dtrpl.code=off loads every rule file from its TRPL. -Dtrpl.verbose=true logs each rule file loaded from Scala. -Dtrpl.debug=true reports each failing rule block.

Status¶

• All 53 loadable profiles translate (261 files, 99.8% of expressions in Scala) and load through Scala.
• hotel_gui converts to byte-identical output with TRPL and with Scala (also after the brace reformatting).
• loadCheck loads all 53 profiles through Scala; re-translating rules/ reproduces the committed Scala exactly.

• No Scala equivalent of rules.tracing. Stack traces point into the Scala, and the engine's error reports don't name the active rule.
• Still TRPL in rules/: templates (.tpl), report definitions (.rpt), 4 profiles FWD cannot load outside a project (schema/generate_ddl, schema/generate_seq_ddl, schema/reindex, reports/rpt_nullable_vars), and 7 .rules files no profile reaches.
• Scope binding: TRPL binds a function's free variables per compiled expression, the Scala per unit. This only matters when two callers bind the same name to different variables. hotel shows no difference; other projects are not verified on 1747a yet.
• 18 rule blocks are over HotSpot's 8,000-byte limit. Run conversions with -XX:-DontCompileHugeMethods, or those blocks stay interpreted.
• --offline builds need the Scala compiler already in the Gradle cache.
• 77 code lines of the generated rules are still over 110 columns (mostly string literals that cannot be broken).
• The hand-written runtime (Trpl*.scala) still uses Scala 3 indentation syntax; only the generated rules follow the GCD style.

Changing rules¶

Edit the TRPL in rules/, then run:

./gradlew -p translator translate     # summary must say "0 profiles failed" 
translator/update-rules.sh            # updates only existing rules/*.scala (-n: dry run)
./gradlew jar

Direct edits to rules/*.scala are overwritten by the next update-rules.sh.

Build note: copyResources never deletes files. Rule XML that was copied into build/classes by an earlier build stays there and in p2j.jar, where the server loads it instead of the Scala. After an update from a build that still packaged the XML rules, purge build/classes/rules or do a clean build.