TRPL

We have a mechanism for implementing templates in TRPL v1, using the TemplateWorker (TW). The idea is that the TW can initialize an AST subtree (a "snippet" of a tree) using an XML syntax that looks like this:

   <ast-root name="implicit_legacy_ctor_with_super" terse="true" ast_class="com.goldencode.p2j.uast.ProgressAst">
      <ast id="0" text="constructor" type="CONSTRUCTOR">
         <ast id="0" text="constructor" type="KW_CONSTRUC">
            <annotation datatype="java.lang.String"  key="name" value="${name}"/>
            <annotation datatype="java.lang.Long"    key="return_type" value="${rettype}"/>
            <annotation datatype="java.lang.String"  key="signature" value="${signature}"/>
            <annotation datatype="java.lang.String"  key="javaname" value="${javaname}"/>
            <annotation datatype="java.lang.String"  key="classname" value="void"/>
            <annotation datatype="java.lang.String"  key="oo-data-store" value="METHOD"/>
            <annotation datatype="java.lang.Long"    key="access-mode" value="${accessmode}"/>
            <annotation datatype="java.lang.Boolean" key="static" value="false"/>
            <ast id="0" text="public" type="KW_PUBLIC"/>
            <ast id="0" text="(" type="LPARENS">
            </ast>
         </ast>
         <ast id="0" text="block" type="BLOCK">
            <ast id="0" text="statement" type="STATEMENT">
               <ast id="0" text="super" type="KW_SUPER">
                  <annotation datatype="java.lang.String" key="javaname" value="${superjavaname}"/>
                  <ast id="0" text="(" type="LPARENS">
                  </ast>
               </ast>
            </ast>
         </ast>
      </ast>
   </ast-root>

These are stored in an XML file and read in by the TW. Each template has a name and can be used with TW helper functions to do things like grafting a new instance of that subtree into an existing AST at a specific location. That grafting process allows substitution of runtime calculated values into the instantiated template so that the content of the subtree can be customized before the graft occurs.

This works really well and it is super powerful, especially when the subtrees are non-trivial. Notice how we can encode token types, text, arbitrary numbers of annotations and the tree structure itself. Even trivial subtrees are often better handled using templates rather than long sequences of explicit creation steps like this:

            <action>ref1 = lref.get(1)</action>
            <action>ref1 = createProgressAst(prog.parameter, "parameter", ref1)</action>
            <action>execLib("doProcedureParameterType", prog.kw_in_out, ref1)</action>
            <action>ref1 = createProgressAst(prog.expression, "expression", ref1)</action>
            <action>ref1 = createProgressAst(prog.var_logical, "Cancel", ref1)</action>
            <action>ref1.putAnnotation("oldtype", #(long) prog.symbol)</action>
            <action>ref1.putAnnotation("refid", ref.id)</action>

            <action>ref1 = lref.get(1)</action>
            <action>ref1 = createProgressAst(prog.parameter, "parameter", ref1)</action>
            <action>execLib("doProcedureParameterType", prog.kw_in_out, ref1)</action>
            <action>ref1 = createProgressAst(prog.expression, "expression", ref1)</action>
            <action>ref1 = createProgressAst(prog.var_char, "NewValue", ref1)</action>
            <action>ref1.putAnnotation("oldtype", #(long) prog.symbol)</action>
            <action>ref1.putAnnotation("refid", ref2.id)</action>

            <!-- Update new label -->
            <action>ref3 = lref.get(0)</action>
            <action>ref1 = createProgressAst(prog.assignment, "assignment", ref3)</action>
            <action>ref1 = createProgressAst(prog.assign, "=", ref1)</action>
            <action>ref1 = createProgressAst(prog.colon, ":", ref1)</action>
            <action>ref1 = createProgressAst(prog.var_handle, thName, ref1)</action>
            <action>ref1.putAnnotation("refid", #(long) thRefid.get(thName.toUpperCase()))</action>
            <action>ref1.putAnnotation("oldtype", #(long) prog.symbol)</action>
            <action>ref3 = ref1.parent</action>
            <action>ref1 = createProgressAst(prog.attr_char, "NEW-LABEL", ref3)</action>
            <action>ref1.putAnnotation("oldtype", #(long) prog.kw_new_lbl)</action>
            <action>ref1.putAnnotation("direct-property-access", true)</action>
            <action>ref1.putAnnotation("property-access-type", "setter")</action>
            <action>ref3 = ref1.parent</action>
            <action>ref3 = ref3.parent</action>
            <action>ref1 = createProgressAst(prog.expression, "expression", ref3)</action>
            <action>ref1 = createProgressAst(prog.var_char, "NewValue", ref1)</action>
            <action>ref1.putAnnotation("oldtype", #(long) prog.symbol)</action>
            <action>ref1.putAnnotation("refid", ref2.id)</action>

The template version is much more readible and the use of it looks like this:

            <action>
               tpl.graftAt("implicit_legacy_ctor_with_super", null, copy, 0, 
                           "name", legacyname,
                           "javaname", defctor,
                           "signature", sprintf("%s()", legacyname.toLowerCase()),
                           "rettype", sprintf("%s", prog.oo_meth_void),
                           "accessmode", sprintf("%s", prog.kw_public),
                           "superjavaname", sqname)
            </action>

However, it could be made better in 2 ways.

1. The subtrees could be defined in the same code that uses it, using first-class language syntax rather than storing these in a separate XML file.
2. In addition to using this as a template, it seems like this would be a powerful way to define the criteria for a pattern match.

Things I want to ensure we handle:

• The structure, possibly using some kind of Lisp-like syntax (A (B (D) (E)) (C (F) (G))) which would represent a tree like

  A
  |------|
  B      C
  |--|   |--|
  D  E   F  G
  

• for each node, some combination of:
  • symbolic token type
    • potentially with some specification like a list of potentially matching types (for patterns), a range of matches (for patterns)
  • token text
    • for patterns, this might be a regex
  • 0 or more annotations
    • presence of annotations of a given name and/or specific values
  • labels for specific nodes
    • Each label would be a local variable that references the associated AST node.
    • These are optional, but when present will bind the nodes to those local variables, allowing direct reference to specific nodes in the subtree.
    • This same facility would be used to identify which node is marked as this. The idea is that for patterns, the this node would be the anchor from which the pattern is matched.

The actual syntax will have to be designed.