FWD » Core Development » Database

• Naming of the DMO interface should express the temp-table structure. For example: ICD3 for a temp-table like:

  DEFINE TEMP-TABLE tt_sales NO-UNDO
    FIELD order_id AS INTEGER
    FIELD description AS CHAR
    FIELD sales_figures AS DECIMAL EXTENT 3.

  This way, we don't care about the fields name and we can use this interface for other interfaces with the same structure.
• Then, in the business logic layer, we will define an inner interface that will describe the temp-table exactly how it's defined in the original code.
  If we have start.p:

  DEFINE TEMP-TABLE Person NO-UNDO
    FIELD first-name AS CHAR
    FIELD last-name AS CHAR.

  
  Start.java will look like:

  public class Start {
      Person.Buf person = TemporaryBuffer.define(Person.Buf.class, "person", "person", false, false);
  
      [...]
  
      // Inner interface for the Person temp-table
      public interface Person 
      extends CC // where CC is the interface for a temp-table with two character fields
     {
          String getFirstName();
          void setFirstName(String firstName);
          String getLastName();
          void setLastName(String lastName);
      }
  }

• All of the specific clauses like FORMAT, LABEL or INIT can be annotated to the inner interface methods:

  public interface Person 
      extends CC
     {
          @Property([...] format = "x(8)")
          String getFirstName();
          [...]
     }

With this kind of approach, deterministic naming of DMOs interfaces is guaranteed. But we still need to handle new shared/shared/undo/no-undo. However, this implementation would create an inner interface in the source file even if an identical interface already exists in another file. The usage of include files is a good example to see the memory/performance impact:
start1.p:

{tables.i}

{tables.i}

DEFINE TEMP-TABLE Product NO-UNDO
  FIELD product-id AS INTEGER
  FIELD product-name AS CHARACTER
  FIELD price AS DECIMAL
  FIELD in-stock AS LOGICAL.

DEFINE TEMP-TABLE tt_sales NO-UNDO
  FIELD order_id AS INTEGER
  FIELD description AS CHAR
  FIELD sales_figures AS DECIMAL EXTENT 3.

Start1.java and Start2.java will have the same content:

public class Start1 
{
    TtSales.Buf ttSales = TemporaryBuffer.define(TtSales.Buf.class, "ttSales", "ttSales", false, false);
    Product.Buf product = TemporaryBuffer.define(Product.Buf.class, "product", "product", false, false);

    [...]

    public interface TtSales 
   {
        int getOrderId();
        void setOrderId(int orderId);

        String getDescription();
        void setDescription(String description);

        java.math.BigDecimal[] getSalesFigures();
        void setSalesFigures(java.math.BigDecimal[] salesFigures);
    }

    public interface Product 
    extends ICDL
    {
        int getProductId();
        void setProductId(int productId);

        String getProductName();
        void setProductName(String productName);

        java.math.BigDecimal getPrice();
        void setPrice(java.math.BigDecimal price);

        boolean isInStock();
        void setInStock(boolean inStock);
    }
}

Ovidiu suggested to use default to delegate from the get / set to the super-interface cryptic methods (get1, set1).
An idea was also resolve this an run-time when the ASM generated DMO is going to map these methods.

I am going to propose you a radical change in the way we interact with the buffers. This is a new paradigm, maybe closer to what 4GL does. Although it is a bit (or more) different than whet we have today, this has the chance to increase the performance for buffer access at the expense of the change in the syntax. Note that this is only a prototype, I did not put it to the test and the notes below are just theoretical and, due to my going on vacation, the ideas are not fully developed.

Assume we have the following FWD classes:

TempTablesTempTables

public class TempTables
{
   private static Map<String, TempTables> dmos = new HashMap<>();

   private final String legacyName;
   private final String tableSignature;

   private final ArrayList<FieldMetadata> fieldMetadata = new ArrayList<>(); // maybe <DmoMeta> instead?

   private static final AtomicInteger multiplexGenerator = new AtomicInteger(0);
   private final int multiplex;

   private TempTables(String legacyName, String tableSignature)
   {
      this.legacyName = legacyName;
      this.tableSignature = tableSignature;
      this.multiplex = multiplexGenerator.incrementAndGet();
   }

   public static synchronized TempTables define(String legacyName, 
                                                String tableSignature, 
                                                boolean shared, 
                                                boolean isNew)
   {
      TempTables dmo = dmos.computeIfAbsent(tableSignature, s -> new TempTables(legacyName, s));

      // store [legacyName] -> [dmo] mapping as well 

      return dmo;
   }

   public FieldMetadata field(int ord, String legacyName, Class<? extends BaseDataType> fieldType)
   {
      return new FieldMetadata(this, ord, legacyName, fieldType);
   }

   public TempTables done()
   throws RuntimeException
   {
      // TODO: verify the structure of this table and eventually throw an initialization exception 
      return this;
   }

   public RB defaultBuffer()
   {
      // I think we can drop the proxy and this method altogether
      return null;
   }

   public RB defineBuffer(String variableName, String legacyName)
   {
      // I think we can drop the proxy and this method altogether
      return null;
   }
}

public static class FieldMetadata
{
   private final TempTables parentTable;
   private final String legacyName;
   private final Class<? extends BaseDataType> type;
   private final int ord;
   private boolean done = false;

   public FieldMetadata(TempTables dmo, int ord, String legacyName, Class<? extends BaseDataType> type)
   {
      this.parentTable = dmo;
      this.legacyName  = legacyName;
      this.type        = type;
      this.ord         = ord;
   }

   public FieldMetadata format(String format)
   {
      checkRO();

      // TODO: add format to this 
      return this;
   }

   public FieldMetadata initial(String value)
   {
      checkRO();

      // set the initial value
      return this;
   }

   // ... other setters for field metadata

   public FieldMetadata caseSensitive()
   {
      checkRO();

      // make it case-sensitive
      return this;
   }

   public TempTables done()
   {
      checkRO();

      parentTable.fieldMetadata.set(ord, this); // or create DmoInfo
      done = true; // makes it R/O
      return parentTable;
   }

   private void checkRO()
   {
      if (done)
      {
         throw new RuntimeException("Field " + legacyName + " is no longer configurable");
      }
   }
}

RBRB

public class RB
extends BufferImpl
implements DataModelObject
{
   public void create()
   {
      // TODO: implement me
   }

   public void release()
   {
      // TODO: implement me
   }

   public void setCharacter(int ord, String val)
   {
      // TODO: implement me
   }

   public void setCharacter(int ord, character val)
   {
      setCharacter(ord, val.isUnknown() ? null : val.toJavaType());
   }

   public void setInt(int ord, Integer val)
   {
      // TODO: implement me
   }

   public void setInt(int ord, integer val)
   {
      setInt(ord, val.isUnknown() ? null : val.intValue());
   }

   public void setDate(int ord, date val)
   {
      // TODO: implement me
   }

   public character getCharacter(int ord)
   {
      return character.UNKNOWN;
   }

   public integer getInt(int ord)
   {
      return integer.UNKNOWN;
   }

   public date getDate(int ord)
   {
      return new date();
   }

   @Override
   public RecordBuffer buffer()
   {
      // I think we can drop the proxy and this method altogether
      return null;
   }

   // other similar methods
}

and a converted procedure:

Procedure001Procedure001

public class Procedure001
{
   /** PK AS RECID - if really needed, maybe in a super class / interface */
   private static final int RECID = 0;
   /** FIELD f1 AS CHARACTER */
   private static final int F1 = 1;
   /** FIELD f2 AS INTEGER */
   private static final int F2 = 2;
   /** FIELD f3 AS DATE */
   private static final int F3 = 3;

   /**
    * DEFINE TEMP-TABLE tt1
    *    FIELD f1 AS CHARACTER FORMAT "X(20)" CASE-SENSITIVE
    *    FIELD f2 AS INTEGER FORMAT ">>>9" INITIAL 7
    *    FIELD f3 AS DATE INIT NOW.
    */
   private static final TempTables tt_tt1 = TempTables.define("tt1", "cid", false, false)
         .field(F1, "f1", character.class).format("X(20)").caseSensitive().done()
         .field(F2, "f2", integer.class).format(">>>9").initial("7").done()
         .field(F3, "f3", date.class).initial("now").done()
         .done();

   private final RB tt1 = tt_tt1.defaultBuffer();
   private final RB btt1 = tt_tt1.defineBuffer("btt1", "btt1");

   /** External procedure (converted to Java from the 4GL source code in Procedure001.p). */
   @LegacySignature(type = InternalEntry.Type.MAIN, name = "Procedure001.p")
   public void execute()
   {
      RecordBuffer.openScope(tt1);

      tt1.create();
      tt1.setCharacter(F1, "1");
      tt1.setInt(F2, 1);
      tt1.setDate(F3, date.today());
      tt1.release();

      printData(tt1);
   }

   private void printData(RB tt1)
   {
      AdaptiveQuery query0 = new AdaptiveQuery();
      forEach(query0, "loopLabel0", new Block((Init) () ->
      {
         query0.initialize(tt1, "tt1.f2 = 1", null, "tt2.f2 asc");
      },
      (Body) () ->
      {
         message(new Object[]{
               (character) new FieldReference(tt1, "f1").getValue(),
               (integer) new FieldReference(tt1, "f2").getValue(),
               (date) new FieldReference(tt1, "f3").getValue(),
               tt1.getCharacter(F1),
               tt1.getInt(F2),
               tt1.getDate(F3),
         });
      }));
   }
}

• no more DMO interfaces!
• the accessors of buffer.getFieldK([n]), buffer.setFieldK(val[, k]) are replaced by tt1.getType(<fieldKOrd>[, n]), tt1.setType(<fieldKOrd>, value[, n]) respectively;
• the named accessors are replaced by full-indexed (order) variants. The field names are resolved at conversion time (much like a compiler does) and hardcoded in the source file;
• for dereferenciation operator (::), a special method can be implemented using a fast String based switch;
• the DmoMeta is constructed lazily, as a table is statically loaded (at first use).

• this allow a better integration with existing low-level Record class;
• if a record buffer is passed as parameter or shared between shared tables, there is no need for proxies. The temp-table signature can be easily checked at runtime and all accesses are index based, removing the need for name mapping. The field order (with respective types) must be the same, regardless of their names;
• no need for reflection when accessing a generic field, a (static) metadata with necessary information can be constructed with the table definition;
• further optimisation: there is no need to copy data when the buffer is passed as parameter (maybe implement some kind of copy on write mechanism).

• in case of multiple tables accessed in same source file, the constants for each table must be somehow uniquely defined. This might cause a bit of confusion. The table name will probably be used (ex: TT__F1 and TT2_F1 int constants with different values). We can do this only if name collision are encountered.

This would certainly solve the concern I had in #9976 and much more. It seems to me that the performance of this proposed approach could be a significant improvement. I think we should go ahead and prototype it.

The app-by-app conversion process customizes DMO naming by delegating all of the TRPL processing to a dedicated Java class. This class uses a central database to manage a parallel conversion system. The approach maintains a suffix-based naming convention for DMO interfaces.

• For unregistered table names: The system first checks if a table with the same name already exists in the database. If it does, the system retrieves the last stored suffix, increments it, and then registers the new temp-table name, the new suffix, and its unique hash and database signature in the central database.
• For existing table names: If a record with an identical hash and database signature is found in the database, the system append the existing suffix to the current temp-table name. This ensures consistent naming across conversion runs and prevents duplicate DMOs.

The system uses AstKey.hash and db_signature annotation to store the temp-table names and suffixes in database.

• are there limits on Java file/class that might be problematic. I am aware of limits on the number of lines a method can have. For fields I searched online - it seems 65,536 is a hard limit.

Also, adding/deleting a column will severely affect ordering. Maybe we can compute the "static final" offsets at run-time. Something like:

/** PK AS RECID - if really needed, maybe in a super class / interface */
   private static final int RECID = new FieldMetadata<>("recid", integer.class);
   /** FIELD f1 AS CHARACTER */
   private static final FieldMetadata<character> F1 = new FieldMetadata<>("f1", character.class, "X(20)").caseSensitive();
   /** FIELD f2 AS INTEGER */
   private static final FieldMetadata<integer> F2 = new FieldMetadata("f2", integer.class, ">>>9").initial("7");
   /** FIELD f3 AS DATE */
   private static final FieldMetadata<date> F3 = new FieldMetadata("f3", date.class).initial("now");

   /**
    * DEFINE TEMP-TABLE tt1
    *    FIELD f1 AS CHARACTER FORMAT "X(20)" CASE-SENSITIVE
    *    FIELD f2 AS INTEGER FORMAT ">>>9" INITIAL 7
    *    FIELD f3 AS DATE INIT NOW.
    */
   private static final TempTables tt_tt1 = TempTables.define("tt1", "cid", false, false)
         .field(F1)
         .field(F2)
         .field(F3);

   @LegacySignature(type = InternalEntry.Type.MAIN, name = "Procedure001.p")
   public void execute()
   {
      RecordBuffer.openScope(tt1);

      tt1.create();
      tt1.set(F1, "1");
      tt1.set(F2, 1);
      tt1.set(F3, date.today());
      tt1.release();

      printData(tt1);
   }

   private void printData(RB tt1)
   {
      AdaptiveQuery query0 = new AdaptiveQuery();
      forEach(query0, "loopLabel0", new Block((Init) () ->
      {
         query0.initialize(tt1, "tt1.f2 = 1", null, "tt2.f2 asc");
      },
      (Body) () ->
      {
         message(new Object[]{
               (character) new FieldReference(tt1, "f1").getValue(),
               (integer) new FieldReference(tt1, "f2").getValue(),
               (date) new FieldReference(tt1, "f3").getValue(),
               tt1.get(F1),
               tt1.get(F2),
               tt1.get(F3),
         });
      }));
   }

This way we can simply abstract away the ordering of the fields. The metadata will be assigned with offsets when the temp-table will be built. If you need to add a new field, you simply create a new FieldMetadata and attach it to the temp-table; there is no need to re-number all other offsets. Also, with generics I suppose the converted code can disregard the get<Type> syntax:

<T> T get();
<T> void set(FieldMetadata<T> metadata)

Good additions, Alex!