Project

General

Profile

Writing Thread-Safe TRPL

Introduction

Conversion no longer processes one source file at a time. Since #3211 was merged in trunk revision 16723, the phases driven by the PatternEngine convert several files concurrently: the same rule set is applied to several trees at the same moment, in an order that differs from run to run and from machine to machine.

This does not change how a rule is written for the tree in front of it. It changes what a rule may assume about everything outside that tree.

These mistakes are quiet. They do not raise an exception where the mistake is; they change a generated name or the contents of an emitted file, and they surface as a diff in src/ or ddl/ on a later run or on someone else's machine. That is why this page is worth reading before writing a rule set rather than after one misbehaves.

Read Writing TRPL Rule-Sets first. This page covers only what concurrency adds to it.

Shared or Per Thread

Every rule on this page follows from one question: is the state this rule touches private to the file being converted, or shared with every other file?

State Scope
A variable declared in a rule-set profile (.rules) Per thread.
A variable declared in a pipeline profile (.xml) Per thread. Each thread receives its own copy and never sees what the others collected.
A variable declared multiThreadAccess="true" One instance, shared by every thread.
A collection created by createString2StringMap and its siblings One instance, backed by the conversion database in cvtdb/ and shared across threads, across phases and across runs.
A pattern worker instance Shared. Per-file state inside a worker belongs in a ContextLocal.
The source and copy trees Per thread, by construction. Nothing on this page constrains ordinary work on the current tree.

The multiThreadAccess Attribute

A global variable declared in a pipeline profile is, by default, cloned for each conversion thread. That is the right behavior for the common case, where the variable collects something about the file currently being converted. It is the wrong behavior for a variable that is supposed to accumulate a fact about the whole project, because each thread then fills in its own copy and no thread sees the whole.

The multiThreadAccess attribute selects the second behavior:

<variable name="convertedClasses" type="java.util.Map" multiThreadAccess="true" />

What the Attribute Does, and What It Leaves to You

With the attribute set, TRPL guarantees two things. The variable is not cloned, so every thread resolves the same instance rather than its own copy of it. And access to the variable's value is serialized - getValue and setValue take the variable's monitor, and the value field is volatile - so reading or assigning that reference is safe.

Everything past the dereference is yours. Mutation of the contents of the referenced object is not protected unless you explicitly do so. For example, once a rule has a collection in hand, put, get, containsKey and iteration all reach the object directly, outside any monitor TRPL holds. A shared java.util.HashMap is therefore still a data race, and a sequence of two individually safe accesses is still a race: a rule which checks a key and then writes it can interleave with another thread between the two. See Updating Shared State Atomically below.

Global init-rules run single-threaded, so a shared structure assembled there and only read afterwards needs no synchronization at all. Reach for the atomic helpers when the structure genuinely has to grow while the workers run.

Both directions are bugs and neither announces itself:

  • Marking per-file state multiThreadAccess="true" lets one file's job overwrite another's, so a rule reads a value belonging to a different source file.
  • Leaving project-wide state unmarked silently loses most of what is collected, because each thread reports only what it saw itself.

When a rule set declares a global collection without the attribute, the engine says so on the console:

TRPL global variable 'frameXref' (java.util.Map) is not declared multiThreadAccess="true";  each conversion thread will collect into its own copy of it and will not see the data collected by the others

That message is not an error. It is the engine listing every global whose scope is worth a moment's thought, and it is the fastest review of a new rule set available: run the phase, read the lines naming your variables, and confirm each one is in the column you intended.

Updating Shared State Atomically

Marking a collection shared is only half the job. Two threads reaching the same shared map at the same time will interleave, so the familiar TRPL idiom of checking a key and then writing it is unsafe: both threads can find the key absent, and both can then write.

<!-- unsafe against a shared collection -->
<rule>not convertedClasses.containsKey(path)
   <action>convertedClasses.put(path, execLib("computeClassName"))</action>
</rule>

CommonAstSupport provides helpers that perform the whole read-decide-write sequence while holding the collection's monitor. Use them instead of writing the pair yourself:

<action>result = ensureEntry(convertedClasses, path, "computeClassName", false)</action>

ensureEntry returns the existing value if the key is present, and otherwise calls the named TRPL function, stores what it returns and returns that. recordFunctionReturnType and generateMenuClassName do the same for their own read-decide-write patterns. When an existing helper does not fit, add one next to them rather than reaching for a lock in the rules: the atomic step belongs in one place, where it can be read and
reviewed.

Determinism

A rule that produces a generated name, a line of DDL, a WSDL operation or any other emitted content must produce it in a fixed order. A HashMap or HashSet does not guarantee one, and under concurrency the order it happens to produce changes between runs.

Use a TreeMap, or sort the key list before iterating. This is easy to overlook because nothing about it looks concurrent: the rule is correct in isolation, and the damage appears only when two conversions of the same project are compared and the same content is emitted in a different order.

Cross-File Ordering

A rule may not read, during a phase, data which another file's job writes during that same phase. Single-threaded conversion happens to process files in an order where the dependency is usually satisfied, so such a rule appears to work; concurrently the two jobs overlap and the reader gets nothing, or gets a partially written value.

There are three ways to do this correctly, in order of preference.

Move the Work to Global Post-Rules

Global post-rules run on the main thread once every worker has finished, so everything the phase collected is complete and no other thread is running. This is where the naming work computes its results (DMO, frame and conflicting file names), and it is the right answer whenever the processing needs the whole project rather than a particular other file.

Compute It in an Earlier Pass

fixups/stable_naming.xml runs ahead of the phases which consume names and registers its results in the conversion database. Later phases read them back. Do not recompute a name in a later phase; read the registered one.

Make the Job Wait

When one file genuinely depends on another file, the dependency is expressed by making the job wait.

The engine turns each target artifact into an AstProcessorJob and puts it on AstProcessorThread.queue; the workers pull from that queue in no defined order. A job which must not run yet takes itself off the queue entirely:

  • park(parentPath) records this job under the path it is waiting for and returns true, at which point run() returns without processing anything and without counting down the latch.
  • When the file it waited for finishes, that file's own job calls release(sourcePath), which puts every waiter back on the queue.

A parked job costs nothing while it waits. It occupies no worker, burns no cycles asking whether it may proceed, and does not re-read its AST. The check and the parking happen under the same monitor the parent's job takes to mark itself completed, so the parent cannot finish in the gap between the two.

Today this is used for one dependency: a class is not processed until the class it inherits from has completed. The parent is the class named by KW_INHERITS, read from that node's source-file annotation, and classes inherited from the skeleton are skipped. Waiting is enabled only in the profiles listed in ORDERED_PROFILES, currently annotations/annotations_prep — where annotations/naming registers each class' converted Java name and reads back the names of the classes it imports — and annotations/annotations, which consumes the result.

To add a dependency of your own: add your profile to ORDERED_PROFILES, and derive the path of the file you depend on from an annotation on the tree, as the inherits case does.

One rule constrains this: only wait for a file this run will actually process. A job parked on a file which is not part of the run is never released, nothing else can finish the phase, and the run ends at the stall timeout. Incremental conversion is where this bites — the file you depend on may simply not have changed.

If none of the three fits, the phase is the wrong place for the work. Move it to a pass which runs before the readers.

Pattern Workers and Java Support Code

A pattern worker is a single instance serving every thread, so any mutable field on it is shared whether or not that was intended. Per-file state belongs in a ContextLocal or in the per-thread WorkArea the worker already owns. Static data is not a problem in itself: immutable data, or data which is genuinely project-wide and is accessed through proper synchronization, is fine. What is a defect is per-file state kept in a field, and shared state mutated without synchronization.

Runtime Conversion

Dynamic queries, dynamic temp-tables and validation expressions are converted while the server is running, by the same rule sets which run during full conversion. The arrangement is completely different, and a rule set which runs in both regimes has to be correct in both.

How It Works

ConversionPool is initialized once at server startup. It holds a stack of fully configured PatternEngine instances per ConversionProfile: annotations/annotations,
convert/base_structure, convert/core_conversion, schema/java_dmo, the runtime/postprocess_* profiles and the rest; because these objects are expensive to build and are
wanted many times.

ConversionPool.runTask(profile, asts) pops an engine off that profile's stack, runs the task on the calling thread, and pushes the engine back. There is no worker pool here: no
AstProcessorThread, no AstProcessorJob, no -numThreads. One engine belongs to one thread for the whole duration of one task.

What That Changes

Because an engine is never used by two threads at once, the conversion-time machinery is switched off when Configuration.isRuntimeConfig() is true:

  • variables are not cloned per thread;
  • BaseRuleContainer makes no thread-specific copies of rules, functions or includes;
  • multiThreadAccess is ignored and treated as false.

So the first rule is that a shared rule set must not depend on multiThreadAccess semantics for its correctness. What the attribute buys during full conversion is not there at runtime, and what it costs is not there either.

Before Committing a New Rule Set

  1. Run the phase and read the multiThreadAccess lines it prints. Confirm every global of yours is in the column you meant.
  2. For each shared collection you write to, confirm the update is a single helper call and not a check followed by a write.
  3. For each collection whose iteration produces emitted content, confirm it is ordered.
  4. For each value your rules read, confirm it was written by an earlier phase and not by another file's job in this one.
  5. Confirm no worker field holds per-file state.

Verifying

Correctness here is not "the symptom is gone", it is "the output is unchanged". Convert the project twice, once with -numThreads=1 and once with the default, and diff both src/ and ddl/. They must be identical.

A single clean multi-threaded run proves very little, because these are races: repeat at more than one thread count before concluding. A test corpus that exercises a dependency in a chain, rather than in a single pair of files, is far more likely to expose an ordering weakness.

Useful References