FWD » Core Development » Base Language

OO classes and enums support serialization in the 4GL.

This includes:

• SERIALIZABLE/NON-SERIALIZABLE/SERIALIZE-NAME <name> syntax
  • for classes (CLASS statement) we must handle SERIALIZABLE (NON-SERIALIZABLE and SERIALIZE-NAME cannot be used in class defs)
  • enums are implicitly serializable (no syntax, it just is implemented)
  • all three keywords/clauses can be used with DEFINE DATASET, DEFINE TEMP-TABLE, DEFINE PROPERTY and DEFINE VARIABLE
  • This is a conversion thing and but also will need some runtime support.
  • All parent classes of a serializable class must also be serializable in the 4GL. This has class hierarchy/inheritance implications. It is a compile error if this is not the case.
  • It also may require generation of additional code to handle the actual serialization processing for private members.
• Runtime changes to properly flatten/restore serializable objects for I/O with:
  • Existing built-in OO classes which honor serializaton must have it added (e.g. the various *Error classes).
  • binary streams (check text output too), Progress.IO.BinarySerializer (this can read private members too?)
  • JSON
    • may be in many places, for example the handle based WRITE-JSON() method but also the OO builtin class Progress.IO.JsonSerializer
    • may only reads public members?
    • find the places where there is implicit JSON serialization support
  • appserver
    • the docs suggest this is only for 4GL clients, check this
    • this matters for both exception support (throwing/catching exceptions) and parameter passing
  • other usage? XML?

Some notes and questions about the standard Java Serializable vs the 4GL implementation:

• The implicit/built-in serialization support in Java can possibly be used for the binary form of 4GL serialization. 4GL tests will be needed to check if this can work. My guess is that it probably can work except if we need binary compatibility. I think we probably should have binary compatibility here because customers may have existing serialized data that must be read.
• I do think we probably should implement the Serializable marker interface even if we end up having a custom binary serialization.
  , The JSON serialization will need something custom.
• For members that cannot or should not be serialized, there is the transient keyword in Java. Is NON-SERIALIZABLE the 4GL equivalent? This only matters for DEFINE DATASET, DEFINE TEMP-TABLE, DEFINE PROPERTY and DEFINE VARIABLE since NON-SERIALIZABLE cannot be used in a CLASS statement.
• In Java it is allowed to have a parent class that is not serializable, but in such a case the parent class must have a default constructor. In the 4GL the parent classes must be serializable. This is probably not a real issue since the 4GL code must already have dealt with this already. The only implication here is that the built-in OO 4GL class hierarchy will need to have serialization added where it exists in the 4GL.

Thoughts from Constantin:

• the runtime can't use 'implements Serializable' as progress.lang.object IS serializable. In OE one needs to mark each and every class as 'serializable'. So we will need some kind of annotation to mark serializable classes.
• we will need to go through all skeleton classes and manually mark them (plus all their transient fields, although I don't know if there is this concept in OE, beside SERIALIZE-HIDDEN at temp-tables)
• some types will need special consideration, as they will need to be recreated, like memptr and object (this needs to be registered with ObjectOps)
• temp-tables - do these get serialized? Are there types which don't get serialized? Do we call getter/setter in property serialization?
• we should build a generic serialization technique to not rely on serializing each and every field, as with read/writeExternal
• types of serialization: JSON, file, etc. What compatibility do we provide?

It seems easy on a first glance, but the deeper I dig, it gets more complex. And here it shows that OE can have custom serializers: https://programinprogress.com/object-serialization-in-progress-openedge/

I assume the OE serializer is walking the object graph somehow. But what happens if the same legacy OO instance is referenced in a certain field, in this object graph?

There are SERIALIZABLE and NON-SERIALIZABLE options at the property and variable definition.

In 6129a/13857 I've added LegacyResource annotation for the legacy classes. I have not touched the property/var options.

Marian: We have a new customer project that needs this support urgently. Implementing this properly will require testcases. How quickly can your team write testcases?

Greg Shah wrote:

Marian: We have a new customer project that needs this support urgently. Implementing this properly will require testcases. How quickly can your team write testcases?

We just need to setup an application server and start writing some testcases, just to be sure this is only for 4GL client -> AppSrv, has nothing to do with rest/soap tests that we've done before. It should probably be a matter of days, what is the OE version that we are testing against?

Marian Edu wrote:

just to be sure this is only for 4GL client -> AppSrv

Yes, just 4GL client -> AppSrv at this time, as what I need is the transport of an object to the appserver and back.

Please start with something simple, and in the end include all datatypes (including memptr/handle - how is handle serialized?), dataset, temp-table (what about PLO table fields? handle fields?), arrays, INPUT, OUTPUT, INPUT-OUTPUT modes, etc.

More complex tests should include nested objects (i.e. a class field or temp-table field is a object), circular references - what if the same 'this' reference is kept in another field somewhere?

A separate test suite should be for p2j.oo builtin classes, and exceptions (AppError, SysError).

Marian Edu wrote:

what is the OE version that we are testing against?

11.7.12

• temp-table with data-types, including object fields, avoid handle type.
• class instance fields of different data-types, including object
• input, output, input-output modes
• object arrays in a parameter or temp-table field

Please provide this first, and expand after this to a more complete test suite.

Constantin Asofiei wrote:

• object arrays in a parameter or temp-table field

Here I mean object arrays at the procedure parameter, temp-table field or class instance field.

Constantin Asofiei wrote:

Marian Edu wrote:

what is the OE version that we are testing against?

11.7.12

That might be a problem for us, we have 12.2 installed and that doesn't include the 'classic appsrv' anymore... I'll check if we still have access to 11.7.

We also need to provide Java serialization support for OO exceptions in order to implement unit-testing engine. See discussion around #6237-197.

Marian's team is (or will be) working on testcases.

However, I think you need the support now so you should look to provide some minimum level of support for the exception classes. Some of that can be done with manual edits to the built-in classes. Some initial provisional conversion support will also likely be needed so that application exceptions can be used in unit tests. Ask questions here and we will help you.

Greg Shah wrote:

Marian's team is (or will be) working on testcases.

However, I think you need the support now so you should look to provide some minimum level of support for the exception classes. Some of that can be done with manual edits to the built-in classes. Some initial provisional conversion support will also likely be needed so that application exceptions can be used in unit tests. Ask questions here and we will help you.

This is how it started in 4GL as well, when an error was thrown on the server side that was serialised as an error object on the client side (at some point)... I mean the exact error object instance not just an SysError.

Greg Shah wrote:

Marian's team is (or will be) working on testcases.

However, I think you need the support now so you should look to provide some minimum level of support for the exception classes. Some of that can be done with manual edits to the built-in classes. Some initial provisional conversion support will also likely be needed so that application exceptions can be used in unit tests. Ask questions here and we will help you.

I am working on it now, and I'd like to commit the resulting change, so we will not do the same thing in parallel.

Vladimir, what branch are you working on? There is some early support for OO serialization in 6129b, we need to work on top of that. If needed, the changes can be brought to 3812c...

Constantin Asofiei wrote:

Vladimir, what branch are you working on? There is some early support for OO serialization in 6129b, we need to work on top of that.

No special branch was created, I post the changes to #6237.

If needed, the changes can be brought to 3812c...

It would be great!

Constantin Asofiei wrote:

Vladimir, what branch are you working on? There is some early support for OO serialization in 6129b, we need to work on top of that.

This branch is well behind the latest 3821c. Is it possible to update/rebase it?

I wonder if there is a way to test OO serialization from 4gl, so I could write unit test?

Greg Shah wrote:

• enums are implicitly serializable (no syntax, it just is implemented)

The LegacyEnum Java class is not serializable. Or, do you mean something else?

Vladimir Tsichevski wrote:

Greg Shah wrote:

• enums are implicitly serializable (no syntax, it just is implemented)

The LegacyEnum Java class is not serializable. Or, do you mean something else?

I mean that in the 4GL, their enums are serializable without any additional 4GL syntax (there is no use of the SERIALIZABLE keyword but the enums are still serializable). We need to add this to our enum implementation.

Vladimir, I'll try to bring the changes from 6129b to 3821c tomorrow. The files in question are, but some of them contain other changes:

added:
  src/com/goldencode/p2j/util/LegacySerializable.java
  src/com/goldencode/p2j/util/LegacyObject.java
modified:
  rules/convert/base_structure.xml
  rules/convert/control_flow.rules
  src/com/goldencode/p2j/persist/TableWrapper.java
  src/com/goldencode/p2j/util/Agent.java
  src/com/goldencode/p2j/util/AppServerHelper.java
  src/com/goldencode/p2j/util/ObjectOps.java
  src/com/goldencode/p2j/util/object.java

I think, I found an error in skeleton: class DataAdminError has wrong class names in constructors: OperationError, must be DataAdminError:

class OpenEdge.DataAdmin.Error.DataAdminError
inherits Progress.Lang.AppError:
   def public property HTTPErrorNum as int get. set.

   def public property InnerError as Progress.Lang.Error get. set.

   constructor public OperationError(input v as char).
   end.

   constructor public OperationError(input v as char, err as Progress.Lang.Error).
   end.

end.

I think, it is save to fix it and commit the fix to skeleton?

Vladimir Tsichevski wrote:

I think, I found an error in skeleton: class DataAdminError has wrong class names in constructors: OperationError, must be DataAdminError:

BTW, the conversion complains a little while it process the file, but in the end generates wrong (but compilable) Java code.

Vladimir Tsichevski wrote:

I think, I found an error in skeleton: class DataAdminError has wrong class names in constructors: OperationError, must be DataAdminError:

[...]

I think, it is save to fix it and commit the fix to skeleton?

I have no commit access to the repo, can anyone help me committing the following change, please?

=== modified file 'oo4gl/OpenEdge/DataAdmin/Error/DataAdminError.cls'
--- oo4gl/OpenEdge/DataAdmin/Error/DataAdminError.cls    2019-05-07 15:57:06 +0000
+++ oo4gl/OpenEdge/DataAdmin/Error/DataAdminError.cls    2022-11-16 18:27:53 +0000
@@ -4,10 +4,10 @@

    def public property InnerError as Progress.Lang.Error get. set.

-   constructor public OperationError(input v as char).
+   constructor public DataAdminError(input v as char).
    end.

-   constructor public OperationError(input v as char, err as Progress.Lang.Error).
+   constructor public DataAdminError(input v as char, err as Progress.Lang.Error).
    end.

 end.

Are you using ~/secure/code/p2j_repo/skeleton? You should have full commit access to that repo. It is OK to commit your change.

Greg Shah wrote:

Are you using ~/secure/code/p2j_repo/skeleton? You should have full commit access to that repo. It is OK to commit your change.

Done, rev. 106.

Previously, I used the repo xfer.goldencode.com/opt/fwd/skeleton/, and when I tried to commit, I was asked for my password at xfer first, then for the password of Constantin (???).

If a repo exists on both devsrv01 and on xfer, then the devsrv01 version is authoritative and the xfer version is a copy that is sync'd when needed. FWD, the Hotel projects, the skeletons all fall into this category.

If a repo only exists on either devsrv01 or xfer but not both, then that repo is by definition authoritative. The testcases project is in this category.

Greg Shah wrote:

If a repo exists on both devsrv01 and on xfer, then the devsrv01 version is authoritative and the xfer version is a copy that is sync'd when needed. FWD, the Hotel projects, the skeletons all fall into this category.

If a repo only exists on either devsrv01 or xfer but not both, then that repo is by definition authoritative. The testcases project is in this category.

Thank you, Greg.

And here it shows that OE can have custom serializers: https://programinprogress.com/object-serialization-in-progress-openedge/

This site doesn't exist now. Do you recall what technique was used? I assume you mean they have something like Externalizable. My review of the 4GL docs didn't highlight any features that provide this.

Greg Shah wrote:

And here it shows that OE can have custom serializers: https://programinprogress.com/object-serialization-in-progress-openedge/

This site doesn't exist now. Do you recall what technique was used? I assume you mean they have something like Externalizable. My review of the 4GL docs didn't highlight any features that provide this.

I don't recall what that site had, and wayback machine doesn't have it. But I'm sure it was not about Externalizable, maybe it was some example using reflection to walk the object graph.

And here it shows that OE can have custom serializers: https://programinprogress.com/object-serialization-in-progress-openedge/

This site doesn't exist now. Do you recall what technique was used? I assume you mean they have something like Externalizable. My review of the 4GL docs didn't highlight any features that provide this.

I don't recall what that site had, and wayback machine doesn't have it. But I'm sure it was not about Externalizable, maybe it was some example using reflection to walk the object graph.

We are implementing reflection separately. For this serialization task, I only want to consider features in which the runtime implements serialization or provides some mechanism for the programmer to get a call back that implements serialization.

Marian: Do you know of any such feature (like Java's Externalizable) in the 4GL?

Greg Shah wrote:

Marian: Do you know of any such feature (like Java's Externalizable) in the 4GL?

The keyword is SERIALIZABLE and can be used on class objects and their properties (non-static) including temp-table and datasets. Initially there were no serializers provided as the use case was to only pass objects between client and application server (first requirement was to get the error objects across the wire). Since then there are now two serializers provided in Progress.IO package - binary and json, most probably the binary one is what they are using internally to pass objects between client/appsrv but that's a black box.

More info about this here [[https://docs.progress.com/bundle/openedge-oo-abl-develop-applications-117/page/Serializing-an-instance-to-file.html]] and about passing objects between client and appsrv -[[https://docs.progress.com/bundle/openedge-oo-abl-develop-applications-117/page/Passing-object-reference-parameters.html]].

I would say some form of reflection needs to be used here - aka, the object has no control of how it's data is being serialised, other than defining what is/isn't serialisable.

the object has no control of how it's data is being serialised, other than defining what is/isn't serialisable

Perfect! That reduces our scope since there is no custom serialization logic/callback.

We will need testcases to be quite complete. What is the status of the tests?

Are the tests serialization tests completed?

I don't know. Please check the testcases project.

Found some tests in testcases/tests/appsrv/4gl_rpc/serialization, but they are pretty limited. I am going to extend that with more tests.

Some findings on 4gl seralization:

JSON wire format¶

Not a flat property map — every object (root or nested) is wrapped in an envelope keyed by its fully-qualified class name, with its own object id:

{"prods:version":1,"prods:objId":1,"myapp.Person":{
  "Name":"Bob","Age":42,
  "HomeAddress":{"prods:objId":2,"myapp.Address":{"City":"Springfield"}}
}}

prods:version appears once, at the document root only. Shared/circular references: the first time an instance is seen it's fully embedded with a fresh objId; every later encounter of the same instance anywhere in the graph collapses to a bare back-reference {"prods:objId":2} — this is what makes circular references terminate instead of recursing forever.

Temp-tables / datasets in JSON¶

- A class-level TEMP-TABLE member appears under a reserved "prods:tempTables" key, one entry per table, each a plain array of flat row objects (no identity envelope — rows aren't objects).
- A DATASET member gets one extra nesting level, under "prods:datasets", keyed by the dataset's own name, then by table name.
- prods:version stays 1 even with temp-tables/datasets present.
- DEFINE BUFFER (an alias, not a table definition) never produces its own separate entry.

Visibility rules differ between the two serializers¶

- BinarySerializer: includes a member regardless of PUBLIC/PROTECTED/PRIVATE. Visibility is never a factor; an explicit NON-SERIALIZABLE still excludes it.
- JsonSerializer: visibility sets the default — PUBLIC defaults to included, PROTECTED/PRIVATE default to excluded. An explicit SERIALIZABLE/NON-SERIALIZABLE option always overrides that default, regardless of visibility:

CLASS Foo SERIALIZABLE:
  DEFINE PUBLIC PROPERTY A AS CHARACTER NO-UNDO GET. SET.                  /* JSON: included (default) */
  DEFINE PRIVATE PROPERTY B AS CHARACTER NO-UNDO GET. SET.                 /* JSON: excluded (default) */
  DEFINE PRIVATE SERIALIZABLE PROPERTY C AS CHARACTER NO-UNDO GET. SET.    /* JSON: included (explicit wins) */
  DEFINE PUBLIC NON-SERIALIZABLE PROPERTY D AS CHARACTER NO-UNDO GET. SET. /* JSON: excluded (explicit wins) */
END CLASS.

All four round-trip fine through BinarySerializer (only D is ever excluded there, due to the explicit option).

Class-level SERIALIZABLE is NOT inherited¶

Each class must redeclare it on its own class statement:

CLASS Person SERIALIZABLE: ... END CLASS.
CLASS Manager INHERITS Person: END CLASS.              /* does NOT repeat it */
CLASS Employee INHERITS Person SERIALIZABLE: END CLASS. /* DOES repeat it */

Serialization order & structure across inheritance¶

- PUBLIC members from every level of a hierarchy flatten into one shared object body, in base-to-derived declaration order (grandparent's members first, then parent's, then the class's own) — the opposite of "most-derived first."
- PRIVATE members behave completely differently: each ancestor class's own private members are grouped into a nested sub-object keyed by that ancestor's own fully-qualified class name — unconditionally, even with no name collision. Confirmed reasoning: private is the only visibility where two genuinely unrelated members can share a name at different levels (private isn't visible to subclasses, so there's no language-level conflict), so the serializer preserves that encapsulation boundary structurally rather than merging them.

Example — two unrelated PRIVATE SERIALIZABLE variables named Counter, one in a base class, one in a subclass:

"myapp.Derived": {
  "myapp.Base": { "Counter": 1 },
  "Counter": 2
}

• include the field types - base BDTs, streams, handles, queries, frames, etc - everything that can be defined as a class field
• datasets - nested tables, relations, etc, before-tables and the before-image fields (row-state, error-string, etc)
• temp-tables - what if you define an instance buffer for a static temp-table?
• explicit buffer definitions at the class
• super-class fields, sub-class fields
• SERIALIZABLE - where it was set, etc
• what about static fields - are these ever serializable?
• properties vs variables - I understand getters are always called
• etc

I think this wrap up everything tested.

Class-level SERIALIZABLE¶

- A class marked SERIALIZABLE reports itself serializable; one without does not
- GetClass():IsSerializable() matches the class-level declaration
- A subclass does NOT inherit SERIALIZABLE — must redeclare it itself
- Serializing a subclass that didn't redeclare SERIALIZABLE throws an error
- Casting to a SERIALIZABLE superclass type doesn't change actual serializability
- An ENUM class is serializable without needing the keyword at all

Member visibility¶

- Binary format includes PUBLIC/PROTECTED/PRIVATE properties and variables, always included in serialization
- JSON format: PUBLIC defaults to included, PROTECTED/PRIVATE default to excluded, unless an explicit option overrides that default

Member-level SERIALIZABLE / NON-SERIALIZABLE¶

- Explicit SERIALIZABLE on a property is honored and restores correctly
- NON-SERIALIZABLE excludes a property from output, binary and JSON
- PRIVATE/PROTECTED property or variable is included in binary serialization, but excluded from JSON serialization.
- Explicit SERIALIZABLE on a PRIVATE/PROTECTED member overrides its default JSON exclusion
- NON-SERIALIZABLE explicitly overrides an otherwise-included member, binary and JSON

SERIALIZE-NAME¶

- Renames a property's/variable's JSON key.
- Not possible as CLASS definition attribute.
- Combined with hidden/excluded members in the same object
- An overridden property's own SERIALIZE-NAME (different from the base) wins over the base's SERIALIZE-NAME

JSON serialization format¶

{
  "prods:version":1,
  "prods:objId":1,
  "objectName": 
  {
    "Name":"Bob",
    "Age":42,
    "HomeAddress": { "prods:objId":2, "nestedObjectName" : {"City":"Springfield"} }
  }
}

"prods:version":1 - I don't know what this means, but every serialized object has it embedded (and it is always 1). I tried serialization of newly created object, change the value of a member afterwards and serialized again, but still the same version. My research online or AI interrogation hasn't yield any fruit.

"prods:objId":1 - this the id of the object that is being serialized. If we have a data member as an OO object, it will follow the same global format, only the id will increment:

{"prods:objId":2,"fwdtest.serialization.Address":{"City":"Springfield","zip":"12345"}}

And if the same nested fwdtest.serialization.Address object will appear anywhere in the main serialization object body we will be shown as a reference: {"prods:objId":2 }.

objectName - is the fully qualified class name

Object graph / shared references¶

- Two properties/variables pointing at the same instance preserve shared identity, binary or JSON
- Shared identity preserved when the same instance is both a scalar property and an array element
- Distinct (non-shared) array elements restore independently
- An array with one set and one unset element preserves the unset element's state

Data types¶

- BDT (int, char, etc)
- INT64, DATE, DATETIME, DATETIME-TZ, ROWID, RECID, RAW
- MEMPTR property
- HANDLE property
- ENUM-typed property, both formats

Inheritance and member ordering¶

- PUBLIC members of SUPERRCLASSES go into the main body of JSON serialized object format.
- PRIVATE/PROTECTED members of SUPERCLASS go into a separate object of the SUPERCLASS nested into the main object body

{
  "prods:version": 1,
  "prods:objId": 1,
  "fwdtest.serialization.GroupOrderDerived": {
    "fwdtest.serialization.GroupOrderBase": {
      "Romeo": "5-Romeo",
      "Tango": "6-Tango",
    },
    "Alpha": "1-Alpha",
    "Quebec": "4-Quebec" 
  }
}

TEMP-TABLE / DATASET / BUFFER¶

- A SERIALIZABLE TEMP-TABLE member restores through JSON
- A BUFFER aliasing an already-serializable temp-table is not separately serialized (could not set SERIALIZABLE)
- QUERY also could not set SERIALIZABLE, and not serialized
- A SERIALIZABLE DATASET wrapping a temp-table restores through JSON
- The temp-table's/dataset's key position comes after all regular properties regardless of declaration position (prods:temptables or prods:dataset)
- temp-table buffer don't get flushed at serialization.
- if temp-table member is part of a dataset member, only the dataset member is serialized.
- didn't tested temp-table or dataset internal serialization heavily (just simple examples), my assumption was that the support is more or less complete in FWD (JsonImport.java/JsonExport.java).

Property getter and setter¶

- A property must have both GET and SET to be included in JSON serialization at all — a GET-only or SET-only property is omitted entirely, regardless of visibility or an explicit SERIALIZABLE option.
- Binary serialization has no such restriction — accessor completeness doesn't matter.
- When GET and SET carry different access modifiers, JSON inclusion follows the property's own top-level declared access, not either accessor's individual modifier.
- Binary serialization is unaffected by access modifiers in any combination.
- Neither serializer ever invokes a property's custom GET/SET body during serialize/deserialize.
- JSON/BINARY: this causes real data loss for custom-bodied properties (override of a default GET or SET) — the key appears, but the value is always the type's bare default, never the true state.
- Properties with plain (get. or set.) gets serialized in both format.

STATIC members¶

- STATIC and SERIALIZABLE cannot be combined on a member declaration at all
- STATIC properties and STATIC variables are excluded from serialized output entirely

Eduard, this is really excellent work!

All of the testcases you used are written as ABLUnit tests and included in testcases?

The meaning of the following is not clear to me:

Combined with hidden/excluded members in the same object

restores through JSON

temp-table's/dataset's key position comes after all regular properties regardless of declaration position

Please provide a more formal explanation of these:

"prods:version":1,
"prods:objId":1,
"objectName":

Please provide details on the serialization format of each BDT (that includes handles, int64 and all the rest).

Eduard, for properties: the DEFINE PROPERTY, GETTER and SETTER each can have their own access modifiers (private, public, etc). Please test combinations with this, if not already.

• the JSON structure of the temp-table/dataset is what WRITE-JSON emits.
• buffers are being flushed before serialization (i.e. a NEW/CHANGED record is flushed).

Another edge case: force a getter, setter, or buffer flush to raise an error condition during serialization - what happens?

Plus, OBJECT fields, when a member (anywhere in the 'reference tree') back-references an already serialized object (root or other field) - when does the recursion stop? How does it know that it was already de-serialized and you just need to set the reference? You mentioned EXTENT is already covered - we need to go further and check how builtin 4GL collection classes get serialized.

Constantin Asofiei wrote:

You mentioned EXTENT is already covered - we need to go further and check how builtin 4GL collection classes get serialized.

Builtin and ADE classes we can have checked in another iteration of this.

Greg Shah wrote:

All of the testcases you used are written as ABLUnit tests and included in testcases?

Written in ABLUnit, but not yet included in testcases project.

The meaning of the following is not clear to me:

Combined with hidden/excluded members in the same object

Sorry just a bad choice of words, I meant to say a class with nested PUBLIC/PROTECTED/PUBLIC members alongside with different combination of SERIALIZABLE/NON-SERIALIZABLE

restores through JSON

It means that an OO object was serialized into JSON format and deserialized into another OO object and compared to the original object (this is how I designed the tests).

temp-table's/dataset's key position comes after all regular properties regardless of declaration position

Regardless of definition order in OO schema, temp-tables/datasets will come last in the resulted JSON output under it's own key. (prods:temptables or/and prods:datasets).

Constantin Asofiei wrote:

Eduard, for properties: the DEFINE PROPERTY, GETTER and SETTER each can have their own access modifiers (private, public, etc). Please test combinations with this, if not already.

Tested different combinations, and updated Property getter and setter of note #4658-50.

Also, confirm that:

• the JSON structure of the temp-table/dataset is what WRITE-JSON emits.

Yes, structure is the same.

• buffers are being flushed before serialization (i.e. a NEW/CHANGED record is flushed).

Buffer does not get flushed before serialization.

Another edge case: force a getter, setter, or buffer flush to raise an error condition during serialization - what happens?

I was wrong about getter or setter being called at serialization, so I couldn't raise any error at serialization. But spend some time documenting the PROPERTY behavior, updated Property getter and setter with the conclusion.

Eduard Soltan wrote:

• buffers are being flushed before serialization (i.e. a NEW/CHANGED record is flushed).

Buffer does not get flushed before serialization.

This is concerning. For normal JSON serialization of dataset/temp-table, we flush all buffers (so all changes are committed to the temp-table!). So if you have a FWD DMO in NEW or CHANGED state, that gets flushed... see AbstractTempTable.flushBuffers usage. Please double-check this, it wouldn't make sense to serialize an object and leave uncommitted data pending.

I was wrong about getter or setter being called at serialization, so I couldn't raise any error at serialization. But spend some time documenting the PROPERTY behavior, updated Property getter and setter with the conclusion.

Simplest way: add an explicit get/set and use a MESSAGE statement; if is executed, then the get/set is called.

Constantin Asofiei wrote:

Simplest way: add an explicit get/set and use a MESSAGE statement; if is executed, then the get/set is called.

I added MESSAGE/THROW statements, and could confirm that get/set are not being called.

Something else for deserialization: is an explicit no-arg constructor mandatory? In case there is only arg constructors defined? If one is mandatory (default or explicit), what happens if an ERROR condition is thrown in the constructor.

Constantin Asofiei wrote:

This is concerning. For normal JSON serialization of dataset/temp-table, we flush all buffers (so all changes are committed to the temp-table!). So if you have a FWD DMO in NEW or CHANGED state, that gets flushed... see AbstractTempTable.flushBuffers usage. Please double-check this, it wouldn't make sense to serialize an object and leave uncommitted data pending.

Yes, indeed the serialization force a record flush. I also force a index validation exception through serialization, which is process through normal 4gl error handling mechanisms.

Please provide details on the serialization format of each BDT (that includes handles, int64 and all the rest).

Something else for deserialization: is an explicit no-arg constructor mandatory? In case there is only arg constructors defined? If one is mandatory (default or explicit), what happens if an ERROR condition is thrown in the constructor.

Deserialization does not go through constructor, it does not matter if I don't have a default constructor.

Plus, OBJECT fields, when a member (anywhere in the 'reference tree') back-references an already serialized object (root or other field) - when does the recursion stop? How does it know that it was already de-serialized and you just need to set the reference? You mentioned EXTENT is already covered - we need to go further and check how builtin 4GL collection classes get serialized.

I have tested a case when a class has a field that reference itself. {"prods:objId":1, "SelfReferencing":{"Label":"Root","Self":{"prods:objId":1}}}. Deserialization of this does not a infinite loop.

I am not sure how the it is done in OE, but in FWD a went with this option: construct the object for a given objId and register it in an objId → instance map before populating any of its own fields. When the reader later hits a nested {"prods:objId":N} pointing back to an instance still being populated (an ancestor, or itself), it looks up N in the map and gets back that same in-progress instance, then assigns it by reference.

Is there any equivalent to Java's Externalizable in which the 4GL developer can override the serialization/de-serialization?

Is there any way for the 4GL developer to specify, change or override the format used in serialization/de-serialization?

Greg Shah wrote:

Is there any equivalent to Java's Externalizable in which the 4GL developer can override the serialization/de-serialization?

Is there any way for the 4GL developer to specify, change or override the format used in serialization/de-serialization?

Did not find any evidence of that.