FWD » Core Development » Database

In FWD, we create different DMO interfaces even if the schema matches, but the resource options, like XML-NODE-NAME or LABEL are different at the temp-table. But, in OE these are not part of the check if the argument matches the parameter buffer.

• oo.BarTest
  

  class oo.BarTest:
     def temp-table tt1 xml-node-name "x2" field f1 as int label "B".
  
     method public void btest(buffer btt1 for tt1):
        def var h as handle.
        h = buffer btt1:handle:table-handle.
        message btt1.f1 h:xml-node-name.  // 1 x1
     end.
  end.
  
  

• oo.FooTest
  

  class oo.FooTest
  inherits oo.BarTest:
     def temp-table tt1 xml-node-name "x1" field f1 as int label "A".
  
     method public void mtest():
        create tt1.
        tt1.f1 = 1.
        btest(buffer tt1).
     end.
  end.
  

• testbuf.p
  

  def var f1 as oo.FooTest.
  f1 = new oo.footest().
  f1:mtest().
  

For 9457, the attempt to resolve this problem is to emit the call like this in FooTest:

   private Tt1_4_1.Buf tt1 = TemporaryBuffer.define(FooTest.class, Tt1_4_1.Buf.class, "tt1", "tt1", false);

btest(RecordBuffer.bufferProxy(tt1, Tt1_2_1.Buf.class));

   private Tt1_2_1.Buf tt1 = TemporaryBuffer.define(BarTest.class, Tt1_2_1.Buf.class, "tt1", "tt1", false);

   public void btest(final Tt1_2_1.Buf _btt1)

Note how the DMO interfaces are different.

• revno: 15879 - A buffer can be passed to a method call, which has a different DMO iface, because of resource-level annotations like XML-NODE-NAME. Emit a RecordBuffer.bufferProxy for this case, which will create a proxy. TODO: implement 'bufferProxy.

• at this point I'm not sure if this will work (although testing DYNAMIC-INVOKE will show)
• my first impression is that this dynamic call adds more overhead than just creating a proxy

I've tested DYNAMIC-INVOKE(.., BUFFER tt1) and runtime gets this BUFFER parameter type not allowed for dynamic parameters ... 15296 - so you can't pass a BUFFER argument to a dynamic call.

So I think we are stuck with the proxy.

Eduard, I've tried something like this to create the proxy:

   public static <T> T bufferProxy(Buffer ref, Class<T> type)
   {
      return (T) TemporaryBuffer.makeMutable((Class) type, (Object) ref);
      // return (T) createProxy(type, ref);
//      String varName = ((BufferImpl) ref).buffer().getDMOAlias();
//      String legacyName = ((BufferImpl) ref).buffer().getLegacyName();
//      return defineAlias(type, ref, varName, legacyName);
   }

• FooTest.cls
  

  class oo.FooTest
  inherits oo.BarTest:
     def temp-table tt1 xml-node-name "x1" field f1 as int label "A".
     def temp-table tt2 field f1 as int.
  
     method public void mtest():
        create tt1.
        tt1.f1 = 1.
        create tt2.
        tt2.f1 = 1.
        ctest(buffer tt2).
        dynamic-invoke(this-object, "btest", buffer tt2) no-error.
        message error-status:error error-status:get-message(1).
        btest(buffer tt1).
        dynamic-invoke(this-object, "btest", buffer tt1) no-error.
        message error-status:error error-status:get-message(1).
  
        mcalc(buffer tt1).
     end.
  
     method private void mcalc(buffer btt1 for tt1).
     end.
  end.
  
  

• BarTest.cls
  

  class oo.BarTest:
     def temp-table tt1 xml-node-name "x2" field f1 as int label "B".
     def temp-table tt2 field f1 as int.
  
     method public void btest(buffer btt1 for tt1):
        def var h as handle.
        def var i as int.
        i = btt1.f1.
        h = buffer btt1:handle:table-handle.
        message i btt1.f1 h:xml-node-name.
     end.
  
     method public void ctest(buffer btt2 for tt2):
        def var h as handle.
        def var i as int.
        i = btt2.f1.
        h = buffer btt2:handle:table-handle.
        message i btt2.f1 h:xml-node-name.
     end.
  
  end.
  
  

• testbuf.p
  

  def var f1 as oo.FooTest.
  f1 = new oo.footest().
  f1:mtest().
  
  

In oo.BarTest, we need to make sure that the DMO created via RecordBuffer.defineAlias passes the calls (via the proxy) to actual buffer argument. For testing, we need to check direct field access, field references, table copy (i.e. pass the temp-table tt1 as an input argument to another method call in BarTest, or copy-temp-table method).

Eduard Soltan wrote:

Question: Since we established in #9763 that xml-node-name is a property specific to the buffer or default buffer in case it is defined in temp-table statement. Maybe we shouldn't generate 2 different dmo classes in case we have just xml-node-name different?

That would mean that (all) the options defined at static temp-tables would need to be duplicated in each program/class defining that temp-table; I don't want to remove the options from the temp-table DMO at this time.

I think we have discussed about an idea similar to Eduard's, some time ago.

Currently, the DMO of a shared temp-table will be generated from a single source file. Logically, it should be the one declaring the new definition for that temp table. But there can be multiple such files, so ultimately, only one will prevail.

That would not be a problem if the definitions of TT would be identical. Since usually that is implemented using includes, the majority of cases will work as expected.

However, we encountered testcases where the TT definition is inlined, and maybe during the code lifetime, the definitions of the [new] shared TTs diverge. That is not a problem for OE, but the specific attributes are stored separately, and at runtime, they depend on the procedure the temp-table was initialised (probably its default buffer - in the light of new discoveries).

So I think that would be the right implementation: to remove these attributes (like xml-node-name) from DMO, keeping there only immutable the schema information, and add them in a customisation block at the begin of the external procedures which declare the new shared tables and simple TT. But not to those declared as shared, as we have seen that these procedures inherit / honour the attribute values from the procedures where the TT was instantiated.

Ovidiu Maxiniuc wrote:

So I think that would be the right implementation: to remove these attributes (like xml-node-name) from DMO, keeping there only immutable the schema information, and add them in a customisation block at the begin of the external procedures which declare the new shared tables and simple TT. But not to those declared as shared, as we have seen that these procedures inherit / honour the attribute values from the procedures where the TT was instantiated.

Ovidiu, this is not about shared temp-tables. If we go down this path, then all temp-tables will be affected. Regardless of what options they define (xml-node-name, label, namespace-uri, etc).

Yes, I am aware, as you quoted me, "[...] which declare the new shared tables and simple TT." I was merely expressed that the shared, not new TTs can/should be excepted from this since their attributes are ignored.

What I am thinking is a construct like:

   Tt1_2_1.Buf tt1 = TemporaryBuffer.define(Tt1_2_1.Buf.class, "tt1", "tt1", false, false)
                                    .setXmlDataType("abc")
                                    .setXmlNodeType("TEXT")
                                    .build();

   Tt1_2_1.Buf tt1 = TemporaryBuffer.define(Tt1_2_1.Buf.class, "tt1", "tt1", false, false);

Ovidiu Maxiniuc wrote:

Yes, I am aware, as you quoted me, "[...] which declare the new shared tables and simple TT." I was merely expressed that the shared, not new TTs can/should be excepted from this since their attributes are ignored.

What I am thinking is a construct like:
[...]instead of[...]

I agree; but I'd like to solve this with the proxy, if possible; before committing to the conversion and runtime changes for this (as these are not that simple, and although this will make the number of DMOs smaller, there will be lots of code duplication to set these options in each place where the temp-table is defined).

Constantin Asofiei wrote:

I agree; but I'd like to solve this with the proxy, if possible; before committing to the conversion and runtime changes for this (as these are not that simple, and although this will make the number of DMOs smaller, there will be lots of code duplication to set these options in each place where the temp-table is defined).

I think in this case the solution would be to get the Proxy of the buffer, and check if it can be casted to the current proxy. And if an error is raised, then to used the buffer.getDMOProxy.

=== modified file 'src/com/goldencode/p2j/persist/FieldReference.java'
--- old/src/com/goldencode/p2j/persist/FieldReference.java    2025-01-30 12:08:25 +0000
+++ new/src/com/goldencode/p2j/persist/FieldReference.java    2025-05-06 09:12:49 +0000
@@ -656,6 +656,17 @@
          dmo = buffer.getDMOProxy();
       }

+      DataModelObject newDmo = buffer.getDMOProxy();
+      
+      try
+      {
+         newDmo.getClass().cast(dmo);
+      }
+      catch (ClassCastException exe)
+      {
+         dmo = newDmo;
+      }
+      
       this.dmo = dmo;
       this.proxy = dmo;
       this.property = property;

=== modified file 'src/com/goldencode/p2j/persist/RecordBuffer.java'
--- old/src/com/goldencode/p2j/persist/RecordBuffer.java    2025-04-23 11:54:58 +0000
+++ new/src/com/goldencode/p2j/persist/RecordBuffer.java    2025-05-06 09:15:42 +0000
@@ -2383,9 +2383,8 @@
    * @return   A proxy to access the given reference as the given type.
    */
    public static <T extends Buffer> T bufferProxy(Buffer ref, Class<T> type)
-   {
-      UnimplementedFeature.missing("Buffer argument when arg and param have different DMO interfaces.");
-      return null;
+   {      
+      return (T) TemporaryBuffer.makeMutable((Class) type, (Object) ref);
    }

    /**

@@ -13797,6 +13796,18 @@

                try
                {
+                  method.getDeclaringClass().cast(buffer);                  
+               }
+               catch (ClassCastException exe)
+               {
+                  String methodName = method.getName();
+                  
+                  Class<?> methodClass = buffer.getClass();
+                  method = methodClass.getMethod(methodName);
+               }
+                             
+               try
+               {
                   invokeReturn = Utils.invoke(method, buffer, args);
                }
                finally

Does this solve cases when the field is in an expression, copy-temp-table and others? Because getDMOProxy(); is used in lots of places, not just FieldReference. Maybe intercept getDMOProxy in the proxy and return something 'better'?

Constantin Asofiei wrote:

Does this solve cases when the field is in an expression, copy-temp-table and others? Because getDMOProxy(); is used in lots of places, not just FieldReference. Maybe intercept getDMOProxy in the proxy and return something 'better'?

I do intercept the call in RB.ArgumentBuffer.invoke, but still the changes are not sufficient for statements like buffer-copy, it covers just direct access. But a more general fix that would work with buffer-copy, temp-table-copy.

=== modified file 'src/com/goldencode/p2j/persist/RecordBuffer.java'
--- old/src/com/goldencode/p2j/persist/RecordBuffer.java    2025-04-23 11:54:58 +0000
+++ new/src/com/goldencode/p2j/persist/RecordBuffer.java    2025-05-06 11:08:37 +0000
@@ -2383,9 +2383,8 @@
    * @return   A proxy to access the given reference as the given type.
    */
    public static <T extends Buffer> T bufferProxy(Buffer ref, Class<T> type)
-   {
-      UnimplementedFeature.missing("Buffer argument when arg and param have different DMO interfaces.");
-      return null;
+   {      
+      return (T) TemporaryBuffer.makeMutable((Class) type, (Object) ref);
    }

    /**

@@ -13794,13 +13793,22 @@

                boolean overridden = buffer.overrideName(bufferName);
                Object invokeReturn;
-               
+               BufferImpl bufferCopy = null;
+               
+               RecordBuffer buf = ((BufferImpl) buffer).buffer();
+               
+               bufferCopy = buffer;
+               buffer = (BufferImpl) buf.getDMOProxy();
+               Class<?> methodClass = buffer.getClass();
+               method = methodClass.getMethod(method.getName(), method.getParameterTypes());
+                             
                try
                {
                   invokeReturn = Utils.invoke(method, buffer, args);
                }
                finally
                {
+                  buffer = bufferCopy;
                   if (overridden)
                   {
                      buffer.restoreName();

Basically we will call every time the DMOProxy of the buffer, instead of the proxy that we passes as argument to RB.defineAlias. I am thinking of the way on how to limit the change of proxies only to the cases when we actually need to do this.

I added my changes to 9937a rev. 15888.

When we have an ArgumentBuffer proxy, any direct field access (btt1.getF1), any method like invocation (copy-temp-table, buffer-copy) will go through RB.ArgumentBuffer.invoke method.

In invoke method if the top-level proxy class is not compatible with buffer.getDMOProxy of the buffer the proxy reference, then I will call Utils.invoke with the buffer.getDMOProxy. (this solves the problem the majority of cases like direct access, buffer-copy, copy-temp-table, etc).

I also added the change in FieldReference, because for FieldReference(btt1, "f1").getValue() it skips entirely the call to RB.ArgumentBuffer.invoke and calls the Utils.invoke with ArgumentBuffer proxy as a parameter and a ClassCastException will be thrown by reflectasm.MethodAccess.invoke. I haven't figured a way to bypass it yet.

The proxy is a dynamic class created by ProxyAssembler with the following structure:

public final class [ProxyClassName] extends BufferImpl implements [Interface List]
{
  ...
}

[Interface List] are usually BufferReference.class, TempTableRecord.class and dmo interfaces (TT_1_1) for example.

Now lets suppose a buffer tt1 with 2 levels of proxies. (Proxy1 with signature class Proxy1 extends BufferImpl implements Tt1_1, BufferReference, TempTableRecord and Proxy2 with signature class Proxy2 extends BufferImpl implements Tt1_1, BufferReference, TempTableRecord).

With RecordBuffer.bufferProxy we are adding another level of proxy, Proxy3 with signature class Proxy3 extends BufferImpl implements Tt2_1, BufferReference, TempTableRecord. And the problem here is that Proxy3 could not be casted to the Proxy2 type.

Maybe a better solution would be to create the Proxy3 with the following signature class Proxy3 extends Proxy2 implements Tt2_1, BufferReference, TempTableRecord?

I don't think is OK to have methods from both Tt2_1 and TT1_1 DMO interfaces.

I don't think Proxy3 should be returned by the handler created by bufferProxy, for some specific methods (definition? buffer? something else?).

Constantin Asofiei wrote:

I don't think is OK to have methods from both Tt2_1 and TT1_1 DMO interfaces.

Then we could keep only the methods from TT1_1. The setter and getters should be identical, only the attributes will differs.

I checked and a single method will be created by the ProxyAssembler for Proxy3 (getF1 and setF1), or any other proxy. But in any way I don't think we can send/receive as parameter a buffer of a temp-table which has a different structure in 4GL. (besides different buffer attributes like xml-node-name, etc.).

Example of the Proxy3.getF1 method that will be created.

public Integer getF1(int a, int b) {
    try {
        InvocationHandler handler = this.h;

        Method method = ma[0];

        Object[] args = new Object[2];
        args[0] = Integer.valueOf(a);
        args[1] = Integer.valueOf(b);

        Object result = handler.invoke(this, method, args);

        return ((Integer) result).intValue();
    } catch (Throwable t) {
        throw new RuntimeException(t);
    }
}

Could you take just a quick look at 9937a, rev. 15889?

Eduard Soltan wrote:

Could you take just a quick look at 9937a, rev. 15889?

The only problem that we are facing is the cast that happens in Utils.invoke in a call to mdata.access.invoke (a call to a third party library which we don't control over) because it can not cast Proxy3 to Proxy2.

But making Proxy3 extend the Proxy2 will solve this problem, and any other actions will not be required. Since Tt2_1 and Tt1_1 are practically the same (except for buffer parameters), Tt2_11 will not introduce any other methods.

Eduard, the approach is good and it should work; your reasoning is correct, the temp-tables need to be schema-compatible in 4GL (and this includes field names).

Are the tests in #9937-5 passing, plus what we discussed in previous notes (copy-temp-table, buffer-copy, table parameters, etc)?

Constantin Asofiei wrote:

Are the tests in #9937-5 passing, plus what we discussed in previous notes (copy-temp-table, buffer-copy, table parameters, etc)?

Yes, the tests do pass. I tried getter, setter, table-buffer parameters, FieldParameters and copy-temp-table and buffer-copy.

class oo.BarTest:
   def temp-table tt1 xml-node-name "x2" field f1 as int 
                                         label "B".
   def temp-table tt2 field f1 as int.

   method public void btest(buffer btt1 for tt1):
      def var h as handle.
      def var i as int.
      i = btt1.f1.

      btt1.f1 = 5.

      h = buffer btt1:handle:table-handle.
      buffer tt2:buffer-copy(buffer btt1:handle).
      message tt2.f1 tt2.f1 h:xml-node-name.

      release tt2.

      for each tt2:
          delete tt2.
      end.

      temp-table tt2:copy-temp-table(temp-table btt1:handle).
      message available(tt2).
      find first tt2.

      message "tt2 " tt2.f1 tt2.f1.

      create tt2.
      tt2.f1 = 5.
      buffer btt1:buffer-copy(buffer tt2:handle).

      message btt1.f1 btt1.f1.
   end.

   method public void ctest(buffer btt2 for tt2):
      def var h as handle.
      def var i as int.
      i = btt2.f1.
      h = buffer btt2:handle:table-handle.
      message i btt2.f1 h:xml-node-name.
   end.
end.

Just to double-check, please do the same in ctest, but with the btt2 for tt2 buffer (which will not use the bufferProxy call).

Constantin Asofiei wrote:

Just to double-check, please do the same in ctest, but with the btt2 for tt2 buffer (which will not use the bufferProxy call).

This also works as expected.

Alexandru, can you take a look, too, at 9937a? This touches the ArgumentProxy which IIRC was added by you.

I see that the argument buffer is now sub-classing the proxy class. Can you summarize why is this required or point me out to the right note/example?
From my POV, the argument buffer is inheriting both BufferImpl and the DMO specific interface in trunk, so it should be enough. Why would the super-class have more than the BufferImpl and the DMO specific interface? What is the benefit from extending the class which implements BufferImpl and DMO interface instead of inheriting these directly?

I mean, makeArgumentBuffer is introspecting the proxy to pick up its interfaces including the DMO specific interface and reuse it for the new argument proxy.
With your changes, the interfaces become redundant, right?

Please look at the test case from #9937-5.

As you can see we have pretty similar temp-table definitions in

FooTest: def temp-table tt1 xml-node-name "x1" field f1 as int label "A".

and

BarTest: def temp-table tt1 xml-node-name "x2" field f1 as int label "B".

Because they have different buffer attributes (xml-node-name and label) 2 different DMOs are generated in FWD (Tt1_1_1 and Tt2_1_1). But, in OE these are not part of the check if the argument matches the parameter buffer.

In order to pass as a parameter the Tt2_1_1 buffer to BarTest.btest which only accepts T1_1_1, we should wrap the tt1 buffer (Tt2_1_1) in another level of proxy which will implement Tt1_1_1 interface.

Issue¶

Lets examine that will happen on a direct access of a field of that proxy (btt1.getF1).

btt1 is a 3 level proxy of type Tt1_1_1 with the following structure.

ProxyLevel3 (class ProxyLevel3 extends BufferImpl implements Tt1_1_1)
   ProxyLevel2 (class ProxyLevel2 extends BufferImpl implements Tt2_1_1)
      ProxyLevel1 (class ProxyLevel3 extends BufferImpl implements Tt1_1_1)

And the following stacktrace with the explanations:

ProxyLevel2 (class ProxyLevel2 extends BufferImpl implements Tt2_1_1) -> here Class cast excepetion will happen 
ReferenceBuffer.invoke -> method that is trying to invoke is Tt1_1_1.getF1 on instance ProxyLevel2
ProxyLevel3.getF1 (class ProxyLevel3 extends BufferImpl implements Tt1_1_1)

At ProxyLevel2 the program will stop because it can not cast ProxyLevel2 into a Tt1_1_1.

Solution¶

Now with my approach when ProxyLevel3 extends ProxyLevel2, in ProxyLevel3 it will invoke the method from super class (ProxyLevel2.getF1 method) and from there it can invoke Tt1_1_1.getF1() as we do in trunk.

handler.invoke -> method that is trying to invoke is Tt1_1_1.getF1 on instance ProxyLevel1
ProxyLevel1 (class ProxyLevel1 extends BufferImpl implements Tt2_1_1)
ReferenceBuffer.invoke -> method that is trying to invoke is Tt1_1_1.getF1 on instance ProxyLevel1
ProxyLevel2 (class ProxyLevel2 extends BufferImpl implements Tt2_1_1)
ReferenceBuffer.invoke -> method that is trying to invoke is ProxyLevel2.getF1 on instance ProxyLevel2
ProxyLevel3.getF1 (class ProxyLevel3 extends ProxyLevel2 implements Tt1_1_1)

Clear. This seems to be a case where we need to "disguise" a DMO as another DMO just to satisfy Java's strong type checks. The point is that they have the same schema, so 4GL allows makes them compatible with each other. But in Java, same schema but different specifications triggers different DMOs.

• RecordBuffer.bufferProxy is a new static method added with your changes. I don't see where this is called. Can this be removed?
• The change of BufferImpl.class into proxy.getClass() is quite clever IMHO. You end up with a new class that is both Tt1_1_1 (as it implements it) and Tt2_1_1 (as it extends it). For a get/set, the assembler will generate one for Tt1_1_1 (as it inherits it) but it will override the one from the parent class. However, when dynamically invoking the method, because the proxied instance is not of the type Tt1_1_1, reflection will resolute the method properly to Tt2_1_1 (the original proxy type).
  • IMHO, there should be more documentation on ArgumentBuffer for this case. The point is that you end up with new classes than implement more than 1 DMO and this should be clear.

It is like disguising a DMO as another DMO. As they have the same schema, the override is complete, so there is no leak of methods from one DMO to another. However, how does 4GL behave when it comes to incompatible DMOs (caller has broader/narrower DMO than callee). If this is statically impossible, then I think I am fully OK with the changes.

Alexandru Lungu wrote:

If this is statically impossible, then I think I am fully OK with the changes.

From my testing it is impossible in 4gl.

Alexandru Lungu wrote:

Review of 9937a:

• RecordBuffer.bufferProxy is a new static method added with your changes. I don't see where this is called. Can this be removed?

RecordBuffer.bufferProxy method will be called from converted code, the conversion part of the change is in 9457c.

I found a regression with 9937a changes related to extend fields.

For extend fields in ProxyAssembler there are some additional methods that are being created in the new proxy class using plugin.assemble(classWriter);. Besides the methods that are being reflected from super class and interfaces.

The problem that appears when we try to create a class that extends another proxy, is that with plugin.assemble(classWriter); we duplicate the these methods that are infered from super proxy class.

A recreate of the problem

Bar.cls

class oo.BarTest:
   def temp-table tt1 field f1 as int EXTENT 3.

   method public void btest(buffer btt1 for tt1):
   end.
end.

Foo.cls

class oo.FooTest
inherits oo.BarTest:
   def temp-table tt1 field f1 as int EXTENT 3.

   method public void mtest():
      btest(buffer tt1).
   end.
end.

I committed my solution to this regression on 9937a, rev. 15890.

Changed defineAlias and makeArgumentBuffer to create a proxy without plugin. Committed on #9937, rev. 15892.

Tested on unit tests of a large GUI app, and it went fine.

Constantin Asofiei wrote:

Please continue testing.

Runtime for Chui regression testing and unit tests of a large GUI app went fine. Should I perform some other tests?

Eduard Soltan wrote:

Constantin Asofiei wrote:

Please continue testing.

Runtime for Chui regression testing and unit tests of a large GUI app went fine. Should I perform some other tests?

Please run ETF.

Asked Danut to run ETF, we should have the results by tomorrow.

Eduard Soltan wrote:

Asked Danut to run ETF, we should have the results by tomorrow.

ETF tests passed.

Could 9937a be merge to trunk, or additional tests should be done?

Testing of 2 customers applications went successful.