Feature #6584: implement schema hint which generates a POJO in addition to a DMO interface - Database - Golden Code Redmine

Feature #6584

implement schema hint which generates a POJO in addition to a DMO interface

Added by Eric Faulhaber almost 2 years ago. Updated almost 2 years ago.

Status:

Review

Priority:

Normal

Assignee:

Ovidiu Maxiniuc

Target version:

Start date:

Due date:

% Done:

100%

billable:

vendor_id:

GCD

om20220709_6584a.diff.zip (8.51 KB) Ovidiu Maxiniuc, 07/08/2022 08:00 PM

History

#1 Updated by Eric Faulhaber almost 2 years ago

The idea is to generate a POJO equivalent of a DMO interface during schema conversion, with no binary dependencies on FWD or on a converted application, for use in scenarios where having such dependencies is not possible or is undesirable (e.g., on the remote side of RESTful APIs).

Requirements:

define a schema (i.e., table) hint which tells schema conversion to generate the POJO equivalent of a DMO interface;
- table hint specifies package (or an alias representing the package) for the POJO (presumably separate from any package generated by application conversion);
- do we default to a pojo subdirectory of the DMO interface's package?
no dependencies on FWD or application conversion artifacts;
uses only Java primitives and J2SE classes (e.g., String, BigDecimal) -- same data type mappings used between Record and BaseRecord in FWD;
implements Serializable (Externalizable? more work...), as the POJO must be able to be sent over the network;
FWD runtime support to marshall/unmarshall into associated DMO implementation class, without reflection (would only be used within the FWD server, as this implies a dependency on FWD; e.g., on the server side of RESTful APIs).

These POJOs would be compiled and jarred separately from application code (again, to break any dependency on application code). Although this is more about deployment, it is included here for completeness.

#2 Updated by Ovidiu Maxiniuc almost 2 years ago

Question: is it necessary for these POJOs that the private data to be independent fields?

I think, if this is not a requirement, we could store the private data in an array, the same way we do for FWD's BaseRecord: the data array contains only Java primitives (auto-wrapped). Same semantics. The interface is maintained but we get a both:

a performance boost if we use System.arraycopy() or maybe even sharing the data array if the invoker chooses so (at his own risk);
simplified code for constructing the new objects: instead of N assignments from POJO/DMOImpl to DMOImpl/POJO, a simple System.arraycopy() will do the work.

#3 Updated by Greg Shah almost 2 years ago

I know of no limit in this area. An array should be fine.

#4 Updated by Eric Faulhaber almost 2 years ago

Ovidiu Maxiniuc wrote:

Question: is it necessary for these POJOs that the private data to be independent fields?

I don't think this necessarily is a requirement, but doing otherwise may be "unnatural" to developers. Consider, if we use the same data array structure underneath, that the implementation of the getters and setters will be hard-coded to the positions of "fields" in the array, and this code will be visible to developers (yes, we do this today with the DMO implementation classes, but those generally are not used directly by external code, and there is no source code). Also, this would preclude the use of field-level reflection; I cannot predict how important this might be to any given developer.

These things probably are not show-stoppers, but users might find them odd at best and restrictive at worst. Even so, the performance aspect of data copy may be worth it.

I think, if this is not a requirement, we could store the private data in an array, the same way we do for FWD's BaseRecord: the data array contains only Java primitives (auto-wrapped). Same semantics. The interface is maintained but we get a both:

a performance boost if we use System.arraycopy() or maybe even sharing the data array if the invoker chooses so (at his own risk);

simplified code for constructing the new objects: instead of N assignments from POJO/DMOImpl to DMOImpl/POJO, a simple System.arraycopy() will do the work.

I do like this part; I also was thinking along these lines, but I haven't convinced myself yet. I'm not sure we want to go as far as sharing the data array directly. You note, "if the invoker chooses so...". How do you envision this working/looking?

Note that arraycopy() will work nicely for everything immutable (most types), but what about LOBs?

#6 Updated by Ovidiu Maxiniuc almost 2 years ago

Eric Faulhaber wrote:

I don't think this necessarily is a requirement, but doing otherwise may be "unnatural" to developers.

Not if they are aware that this is generated code.

Consider, if we use the same data array structure underneath, that the implementation of the getters and setters will be hard-coded to the positions of "fields" in the array, and this code will be visible to developers (yes, we do this today with the DMO implementation classes, but those generally are not used directly by external code, and there is no source code).

This is nothing else than converting identifiers to memory addresses. Any compiler does this. In stead of accessing fields from the whole heap, FWD limits the addresses of the 'variables' into the data array. Any programmer knows this and should not be something unexpected.

Also, this would preclude the use of field-level reflection; I cannot predict how important this might be to any given developer.

This is true if they want to access the private fields directly. They have this visibility level for a reason. If reflection is used to invoke the accessors (public getters and setters we generate), there should be no problem.

[...]

I do like this part; I also was thinking along these lines, but I haven't convinced myself yet. I'm not sure we want to go as far as sharing the data array directly. You note, "if the invoker chooses so...". How do you envision this working/looking?

In C/C++ there is a nice keyword, const. A function/method returning such object is sure that the receiver cannot alter it (immutable externally). This was my thinking. So, if the caller obtains a record from FWD and his only need is to serialize it (REST/JSON) but not alter it in any way, it should use the shared version of the POJO. The POJO object will also have the advantage to be 'entangled' with the original DMO, any change in DMO will immediately be reflected in POJO. However, the other way around might not fully work because of the internal maintenance the FWD does: a direct change in data will not be saved to FWD persistence because FWD will not be aware of it.
"if the invoker chooses so..." means that we may have two methods (or a parameter) which will return a mutable or immutable POJO object.

Note that arraycopy() will work nicely for everything immutable (most types), but what about LOBs?

Yes, I did think about them. Probably the LOB container will have to be duplicated each time, although it is probably safe - I think - to use shared data, at first.

#7 Updated by Eric Faulhaber almost 2 years ago

Assignee set to Ovidiu Maxiniuc

All this sounds good. How long do you think it will take to implement?

#8 Updated by Ovidiu Maxiniuc almost 2 years ago

A basic initial solution could be ready in 2-3 days.

#9 Updated by Constantin Asofiei almost 2 years ago

Ovidiu, on FWD side, I'm using the declared fields to determine the getter/setter, in PojoSerializer. If you are removing the individual fields associated with the getter/setter, there will be no way to determine via the reflection the POJO properties.

I understand that this is a performance/compatibility concern with the FWD Record implementation, but the goal of the POJO is to be completely decoupled from FWD, even in implementation details. What happens if at some point, for some reason, we change the FWD BaseRecord implementation?

#10 Updated by Ovidiu Maxiniuc almost 2 years ago

Constantin Asofiei wrote:

Ovidiu, on FWD side, I'm using the declared fields to determine the getter/setter, in PojoSerializer. If you are removing the individual fields associated with the getter/setter, there will be no way to determine via the reflection the POJO properties.

Using the (private) declared fields to work with the POJO object implies you have a strong coupling with the objects and their implementation. It seems to me that PojoSerializer assumes the objects are beans instead, although neither these objects have the constraints of property names having the same name with the field storing it. The POJO (and beans, as well) should be accessed only via their public interface.

I understand that this is a performance/compatibility concern with the FWD Record implementation, but the goal of the POJO is to be completely decoupled from FWD, even in implementation details. What happens if at some point, for some reason, we change the FWD BaseRecord implementation?

If we change the BaseRecord implementation, we must take into consideration the POJO<->DMO converter and adjust accordingly.

I will take into consideration your note when starting to implement this feature.

#11 Updated by Constantin Asofiei almost 2 years ago

Eric/Ovidiu: on the client-side, jackson is used to serialize to JSON via ObjectMapper. This requires class fields and getter/setter (more a bean-style). Also, it's a pain to debug a Record instance in FWD (determining which value in the data array belongs to which property...), and adding this pain to the client-side, too, I don't see a reason for it.

On FWD side, the methods which convert this POJO to a Record and back can just call the getter/setter from the POJO.

#12 Updated by Eric Faulhaber almost 2 years ago

Constantin Asofiei wrote:

Eric/Ovidiu: on the client-side, jackson is used to serialize to JSON via ObjectMapper. This requires class fields and getter/setter (more a bean-style). Also, it's a pain to debug a Record instance in FWD (determining which value in the data array belongs to which property...), and adding this pain to the client-side, too, I don't see a reason for it.

On FWD side, the methods which convert this POJO to a Record and back can just call the getter/setter from the POJO.

I do expect that Jackson will be a pretty common library for this kind of work. Constantin, are you saying Jackson does not have a simple mode which uses the public getters/setters, instead of the internal fields, to do its work? If so, this to me is a pretty convincing argument to implement the POJO as a traditional bean, having recently been forced to write a custom Jackson deserializer for DMOs, instead of being able to use the default, easy mode. My case was more about the BDT wrappers, but if the internal structure of the data storage breaks the common Jackson [de-]serialization use case, that would be a problem, IMO.

#13 Updated by Constantin Asofiei almost 2 years ago

Eric Faulhaber wrote:

I do expect that Jackson will be a pretty common library for this kind of work. Constantin, are you saying Jackson does not have a simple mode which uses the public getters/setters, instead of the internal fields, to do its work?

AFAIK Jackson does introspection for fields and from that it generates the JSON (via getter/setter etc).

#14 Updated by Eric Faulhaber almost 2 years ago

OK, let's go with the bean approach for the POJOs. They will just work with Jackson and they will not confuse developers of client/remote projects. For the tables marked by hint to generate POJOs, we can add default methods to the DMO interfaces to handle the POJO serialization/deserialization.

#15 Updated by Ovidiu Maxiniuc almost 2 years ago

Status changed from New to WIP
% Done changed from 0 to 30

I started the implementation using the bean approach. For the moment, the POJOs are always created - I will add the configuration flags at the end. As I write this note, the conversion is able to:

create the POJO classes in a pojo package sibling to dmo;

create the properties as private fields (all ABL types, including blob, clob, handle, raw, recid, etc) using the types from Record class. With the exception of clob, which is assumed to be a java.sql.Clob instead of com.goldencode.p2j.util.clob. In fact this is the only FWD type present in Record 's data; The type mapping is the following:

typeMapping.put("blob",        "byte[]")
typeMapping.put("character",   "String")
typeMapping.put("clob",        "Clob")
typeMapping.put("comhandle",   "String")
typeMapping.put("date",        "Date")
typeMapping.put("datetime",    "Timestamp")
typeMapping.put("datetimetz",  "OffsetDateTime")
typeMapping.put("decimal",     "BigDecimal")
typeMapping.put("int64",       "Long")
typeMapping.put("integer",     "Integer")
typeMapping.put("handle",      "Long")
typeMapping.put("logical",     "Boolean")
typeMapping.put("object",      "Long")
typeMapping.put("raw",         "byte[]")
typeMapping.put("recid",       "Integer")
typeMapping.put("rowid",       "Long")

create simple and indexed getters for all these fields;
added only necessary import statements for used data type;
all the above are accompanied by javadocs.

Here is an example of how the book permanent table looks as a POJO:POJO:

package com.goldencode.testcases.pojo.fwd;

import java.io.*;
import java.sql.*;
import java.math.*;

/**
 * Plain Old Java Object class corresponding with table {@code fwd.book} (previously {@code
 * fwd.Book}).
 */
public class Book
implements Serializable
{
   /** book-id */
   private Integer bookId;

   /** book-title */
   private String bookTitle;

   /** publisher */
   private String publisher;

   /** isbn */
   private String isbn;

   /** on-hand-qty */
   private Integer onHandQty;

   /** cost */
   private BigDecimal cost;

   /** pub-date */
   private Date pubDate;

   /** author-id */
   private Integer authorId;

   /** sold-qty */
   private Integer soldQty;

   /** price */
   private BigDecimal price;

   /** opt-ions */
   private String[] optIons;

   /** f-6 */
   private Boolean f6;

   /**
    * Getter: bookId
    * 
    * @return  bookId
    */
   public Integer getBookId()
   {
      return bookId;
   }

   /**
    * Getter: bookTitle
    * 
    * @return  bookTitle
    */
   public String getBookTitle()
   {
      return bookTitle;
   }

   /**
    * Getter: publisher
    * 
    * @return  publisher
    */
   public String getPublisher()
   {
      return publisher;
   }

   /**
    * Getter: isbn
    * 
    * @return  isbn
    */
   public String getIsbn()
   {
      return isbn;
   }

   /**
    * Getter: onHandQty
    * 
    * @return  onHandQty
    */
   public Integer getOnHandQty()
   {
      return onHandQty;
   }

   /**
    * Getter: cost
    * 
    * @return  cost
    */
   public BigDecimal getCost()
   {
      return cost;
   }

   /**
    * Getter: pubDate
    * 
    * @return  pubDate
    */
   public Date getPubDate()
   {
      return pubDate;
   }

   /**
    * Getter: authorId
    * 
    * @return  authorId
    */
   public Integer getAuthorId()
   {
      return authorId;
   }

   /**
    * Getter: soldQty
    * 
    * @return  soldQty
    */
   public Integer getSoldQty()
   {
      return soldQty;
   }

   /**
    * Getter: price
    * 
    * @return  price
    */
   public BigDecimal getPrice()
   {
      return price;
   }

   /**
    * Getter: Optional features
    * 
    * @return  optIons
    */
   public String[] getOptIons()
   {
      return optIons;
   }

   /**
    * Indexed getter: Optional features
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 10}).
    * 
    * @return  optIons
    */
   public String getOptIons(int index)
   {
      return optIons[index];
   }

   /**
    * Getter: f6
    * 
    * @return  f6
    */
   public Boolean isF6()
   {
      return f6;
   }
}

The generated code is, of course, fully compilable.

#16 Updated by Greg Shah almost 2 years ago

That looks really good!

Perhaps we should consider generating these for every DMO. Is there a reason to limit it to those that are marked? It seems like a useful feature in general for the long term.

#17 Updated by Ovidiu Maxiniuc almost 2 years ago

% Done changed from 30 to 60

I added the setters and performed more class polishing, like the initialization of extent fields as Java arrays of the specified type.

I have a few questions now:

now all scalar fields (including equivalent of ABL int, for example) are initialized to null (the equivalent of ? in ABL) which is not semantically equivalent to neither Java nor ABL. Is it necessary to initialize the fields to their default ABL value? (Example: int/Integer to 0, character/String to "");
how do we handle the initial values. Should I make the best effort to initialize the properties to respective values (including the dynamic ones like now and today)? This might require pieces of FWD runtime.
some of the data types used for the fields are immutable (Integer, String, OffsetDateTime, BigDecimal), others are not (Timestamp, Date, byte[]). What should be the semantics of assignment (for scalar values) in case of the latter? Do we simply use the assignment and allow them to be altered externally? I am thinking we should make a copy of these, instead;
how do we implement the assignment of extent/array fields? just assign the whole array (which may corrupt the POJO by assigning an array of the wrong size). I am thinking the right thing to do is to make the array/extent properties final and to copy each element on the respective position on bulk assignment. Then we need to decide how to proceed if the size is not a match.

Here is an example of how the the dame table looks now as a POJO:POJO:

package com.goldencode.testcases.pojo.fwd;

import java.io.*;
import java.sql.*;
import java.math.*;

/**
 * Plain Old Java Object class corresponding with table {@code fwd.book} (previously {@code
 * fwd.Book}).
 */
public class Book
implements Serializable
{
   /** Field 'book-id': Book ID */
   private Integer bookId = null;

   /** Field 'book-title': Title */
   private String bookTitle = null;

   /** Field 'publisher': Publisher */
   private String publisher = null;

   /** Field 'isbn': ISBN */
   private String isbn = null;

   /** Field 'on-hand-qty': On Hand */
   private Integer onHandQty = null;

   /** Field 'cost': Cost */
   private BigDecimal cost = null;

   /** Field 'pub-date': Pub Date */
   private Date pubDate = null;

   /** Field 'author-id': Author ID */
   private Integer authorId = null;

   /** Field 'sold-qty': Sold */
   private Integer soldQty = null;

   /** Field 'price': Price */
   private BigDecimal price = null;

   /** Field 'f-6': f6 */
   private Boolean f6 = null;

   /** Field 'opt-ions': Optional features */
   private String[] optIons = new String[10];

   /**
    * Getter: book-id
    * <p>
    * Book ID
    * 
    * @return  bookId
    */
   public Integer getBookId()
   {
      return bookId;
   }

   /**
    * Setter: book-id
    * <p>
    * Book ID
    * 
    * @param   bookId
    *          bookId
    */
   public void setBookId(Integer bookId)
   {
      this.bookId = bookId;
   }

   /**
    * Getter: book-title
    * <p>
    * Title
    * 
    * @return  bookTitle
    */
   public String getBookTitle()
   {
      return bookTitle;
   }

   /**
    * Setter: book-title
    * <p>
    * Title
    * 
    * @param   bookTitle
    *          bookTitle
    */
   public void setBookTitle(String bookTitle)
   {
      this.bookTitle = bookTitle;
   }

   /**
    * Getter: publisher
    * <p>
    * Publisher
    * 
    * @return  publisher
    */
   public String getPublisher()
   {
      return publisher;
   }

   /**
    * Setter: publisher
    * <p>
    * Publisher
    * 
    * @param   publisher
    *          publisher
    */
   public void setPublisher(String publisher)
   {
      this.publisher = publisher;
   }

   /**
    * Getter: isbn
    * <p>
    * ISBN
    * 
    * @return  isbn
    */
   public String getIsbn()
   {
      return isbn;
   }

   /**
    * Setter: isbn
    * <p>
    * ISBN
    * 
    * @param   isbn
    *          isbn
    */
   public void setIsbn(String isbn)
   {
      this.isbn = isbn;
   }

   /**
    * Getter: on-hand-qty
    * <p>
    * On Hand
    * 
    * @return  onHandQty
    */
   public Integer getOnHandQty()
   {
      return onHandQty;
   }

   /**
    * Setter: on-hand-qty
    * <p>
    * On Hand
    * 
    * @param   onHandQty
    *          onHandQty
    */
   public void setOnHandQty(Integer onHandQty)
   {
      this.onHandQty = onHandQty;
   }

   /**
    * Getter: cost
    * <p>
    * Cost
    * 
    * @return  cost
    */
   public BigDecimal getCost()
   {
      return cost;
   }

   /**
    * Setter: cost
    * <p>
    * Cost
    * 
    * @param   cost
    *          cost
    */
   public void setCost(BigDecimal cost)
   {
      this.cost = cost;
   }

   /**
    * Getter: pub-date
    * <p>
    * Pub Date
    * 
    * @return  pubDate
    */
   public Date getPubDate()
   {
      return pubDate;
   }

   /**
    * Setter: pub-date
    * <p>
    * Pub Date
    * 
    * @param   pubDate
    *          pubDate
    */
   public void setPubDate(Date pubDate)
   {
      this.pubDate = pubDate;
   }

   /**
    * Getter: author-id
    * <p>
    * Author ID
    * 
    * @return  authorId
    */
   public Integer getAuthorId()
   {
      return authorId;
   }

   /**
    * Setter: author-id
    * <p>
    * Author ID
    * 
    * @param   authorId
    *          authorId
    */
   public void setAuthorId(Integer authorId)
   {
      this.authorId = authorId;
   }

   /**
    * Getter: sold-qty
    * <p>
    * Sold
    * 
    * @return  soldQty
    */
   public Integer getSoldQty()
   {
      return soldQty;
   }

   /**
    * Setter: sold-qty
    * <p>
    * Sold
    * 
    * @param   soldQty
    *          soldQty
    */
   public void setSoldQty(Integer soldQty)
   {
      this.soldQty = soldQty;
   }

   /**
    * Getter: price
    * <p>
    * Price
    * 
    * @return  price
    */
   public BigDecimal getPrice()
   {
      return price;
   }

   /**
    * Setter: price
    * <p>
    * Price
    * 
    * @param   price
    *          price
    */
   public void setPrice(BigDecimal price)
   {
      this.price = price;
   }

   /**
    * Getter: f-6
    * <p>
    * f6
    * 
    * @return  f6
    */
   public Boolean isF6()
   {
      return f6;
   }

   /**
    * Setter: f-6
    * <p>
    * f6
    * 
    * @param   f6
    *          f6
    */
   public void setF6(Boolean f6)
   {
      this.f6 = f6;
   }

   /**
    * Getter: opt-ions
    * <p>
    * Optional features
    * 
    * @return  optIons
    */
   public String[] getOptIons()
   {
      return optIons;
   }

   /**
    * Indexed getter: opt-ions
    * <p>
    * Optional features
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 10}).
    * 
    * @return  optIons
    */
   public String getOptIons(int index)
   {
      return optIons[index];
   }

   /**
    * Setter: opt-ions
    * <p>
    * Optional features
    * 
    * @param   optIons
    *          optIons
    */
   public void setOptIons(String[] optIons)
   {
      this.optIons = optIons;
   }

   /**
    * Indexed setter: opt-ions
    * <p>
    * Optional features
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 10}).
    * @param   optIons
    *          optIons
    */
   public void setOptIons(int index, String optIons)
   {
      this.optIons[index] = optIons;
   }
}

The generated code is, again, fully compilable.

At this moment, the POJOs are always created. I was thinking of a switch in p2j.cfg.xml for turning their generation on or off (as opposed to individual hints) and maybe specifying the path where they are generated, although the current package seems fair.

#18 Updated by Ovidiu Maxiniuc almost 2 years ago

File om20220709_6584a.diff.zip added

Added the change-set.

#19 Updated by Greg Shah almost 2 years ago

I was thinking of a switch in p2j.cfg.xml for turning their generation on or off (as opposed to individual hints)

This seems reasonable and simple enough.

maybe specifying the path where they are generated, although the current package seems fair.

I agree the default name is good. Go ahead and add the override just to be complete.

#20 Updated by Ovidiu Maxiniuc almost 2 years ago

I have implemented the marshalling methods between POJO and DMO. Here are the important highlights:

the Record has two new methods:
```
   public Object asPOJO();
   public static DataModelObject fromPOJO(Object pojo, Class<? extends DataModelObject> dmoType);
```
They are here for the moment but they can eventually be moved in a separate (static) class which has access to BaseRecord.data;
the copy process in both directions uses direct copy from/to data array to private members of the POJO. Because the location is really unaware of the structure of the POJO object, the reflection is used to discover its fields. Based on the fact that these fields match the fields names from DMO, the index in dmo is used to get/set the information from/to data array. In case of extents, the information is copied in bulk, using the System.arrayCopy;
the asPOJO() does not require the POJO class because it is read from DMO annotation. The annotation is created only if the POJO class was generated. This also assures the flexibility of storing the POJO in any package, as long as the right path is written in @Table annotation;
fromPOJO method need the DMO interface, for the moment. The POJO class has no information about the DMO. I intend to add a reverse lookup support in DmoMetadataManager and avoid the extra parameter;
the fromPOJO() method returns now a DMO object, to match the 2nd parameter (the DMO interface), but internally, it is a local Record variable. Maybe it should be returned as such and the cast from the end dropped.

#21 Updated by Eric Faulhaber almost 2 years ago

Ovidiu Maxiniuc wrote:

I added the setters and performed more class polishing, like the initialization of extent fields as Java arrays of the specified type.
I have a few questions now:

now all scalar fields (including equivalent of ABL int, for example) are initialized to null (the equivalent of ? in ABL) which is not semantically equivalent to neither Java nor ABL. Is it necessary to initialize the fields to their default ABL value? (Example: int/Integer to 0, character/String to "");

how do we handle the initial values. Should I make the best effort to initialize the properties to respective values (including the dynamic ones like now and today)? This might require pieces of FWD runtime.

I think it is OK to make the POJOs "dumb", leaving it up to the developer to initialize the properties for their purposes when they instantiate a new POJO directly. POJOs instantiated automatically from DMOs will have the values of the DMO, so this is taken care of for us in this case.

You and Constantin have written more REST client code than I have. What do you both think?

some of the data types used for the fields are immutable (Integer, String, OffsetDateTime, BigDecimal), others are not (Timestamp, Date, byte[]). What should be the semantics of assignment (for scalar values) in case of the latter? Do we simply use the assignment and allow them to be altered externally? I am thinking we should make a copy of these, instead;

I am inclined to keep it simple, as you currently have it implemented, and let the developer be responsible for data safety, but I am open to other opinions.

how do we implement the assignment of extent/array fields? just assign the whole array (which may corrupt the POJO by assigning an array of the wrong size). I am thinking the right thing to do is to make the array/extent properties final and to copy each element on the respective position on bulk assignment. Then we need to decide how to proceed if the size is not a match.

Again, I think we leave it the way you have it, but detect a problem when unmarshalling into a DMO on the server side, as the DMO has more knowledge of valid data, mandatory data, etc.

Here is an example of how the the dame table looks now as a {{collapse(POJO:)
[...]
}}

The generated code is, again, fully compilable.

At this moment, the POJOs are always created. I was thinking of a switch in p2j.cfg.xml for turning their generation on or off (as opposed to individual hints) and maybe specifying the path where they are generated, although the current package seems fair.

I think it will be a tiny (often 0) percentage of the overall schema for which a customer will need these POJOs, so I'd rather not generate them for all tables. I'd like to stay with the hints idea. I also like the package path. The one concern I have is that the simple POJO name is the same as the simple DMO name. Won't this be messy in the server-side code, as one of them will need to be fully qualified?

#22 Updated by Eric Faulhaber almost 2 years ago

Ovidiu Maxiniuc wrote:

I have implemented the marshalling methods between POJO and DMO. Here are the important highlights:

the Record has two new methods:
[...]They are here for the moment but they can eventually be moved in a separate (static) class which has access to BaseRecord.data;

the copy process in both directions uses direct copy from/to data array to private members of the POJO. Because the location is really unaware of the structure of the POJO object, the reflection is used to discover its fields. Based on the fact that these fields match the fields names from DMO, the index in dmo is used to get/set the information from/to data array. In case of extents, the information is copied in bulk, using the System.arrayCopy;

the asPOJO() does not require the POJO class because it is read from DMO annotation. The annotation is created only if the POJO class was generated. This also assures the flexibility of storing the POJO in any package, as long as the right path is written in @Table annotation;

fromPOJO method need the DMO interface, for the moment. The POJO class has no information about the DMO. I intend to add a reverse lookup support in DmoMetadataManager and avoid the extra parameter;

the fromPOJO() method returns now a DMO object, to match the 2nd parameter (the DMO interface), but internally, it is a local Record variable. Maybe it should be returned as such and the cast from the end dropped.

I thought we had discussed not using reflection and instead having default implementations of the marshalling methods directly generated into the DMO interface. At conversion time, we have full awareness of the mapping between POJO getters/setters and how fields are organized in the DMO's BaseRecord.data array. This approach is perhaps not as hidden away and elegant, but seems like it would avoid extra infrastructure. Is there a reason you moved away from this design? I am not saying rewrite what you've done, I just want to understand your choice.

One other question, about the data type mappings: I don't understand how java.sql.Clob can work within a POJO, as it requires a database connection and a transaction. This won't exist on the client side, as far as I understand.

#23 Updated by Eric Faulhaber almost 2 years ago

Eric Faulhaber wrote:

I thought we had discussed not using reflection and instead having default implementations of the marshalling methods directly generated into the DMO interface. At conversion time, we have full awareness of the mapping between POJO getters/setters and how fields are organized in the DMO's BaseRecord.data array. This approach is perhaps not as hidden away and elegant, but seems like it would avoid extra infrastructure. Is there a reason you moved away from this design? I am not saying rewrite what you've done, I just want to understand your choice.

To partially answer my own question, avoiding the duplication of error handling code in every DMO interface for POJO unmarshalling is one reason to do this in a central place.

#24 Updated by Greg Shah almost 2 years ago

Even if we want to allow for only a subset of the DMOs to get POJOs, we should make it easy to generate POJOs for all DMOs. The global flag is useful for that. Otherwise, someone will have to generate hints for every DMO, which is just a huge churn for what will likely be a common case.

It seems to me that if you are planning to use POJOs, you are trying to move development into Java and will want there to be a pure Java representation of all your tables. In my opinion, picking out a subset now and then being forced to rerun conversion to get more later is not the approach most people would choose in that case.

#25 Updated by Ovidiu Maxiniuc almost 2 years ago

Support for converting to/from POJOs was committed as r14064.

There are no hints (for the moment). The only way to enable POJO generation is to add

<parameter name="generate-pojos" value="true" />

in the desired namespace in p2j.cfg.xml.

To configure the path where they are generated use the global

<parameter name="pojoroot"                value="com.goldencode.testcases.pojoSpecialPackage" />

Here is one simple example of how the two new APIs can be used:

         foo.create();
         foo.setOptIons1(new character("one"));
         foo.setOptIons10(new character("ten"));
         foo.setPrices(0, new decimal(1.00));
         foo.setPrices(9, new decimal(10.0));

         // toPOJO() method
         Object pojo = ((BufferImpl) foo).buffer().getCurrentRecord().toPOJO();

         // Record.fromPOJO() method
         DataModelObject dmo = Record.fromPOJO(pojo1);

#26 Updated by Constantin Asofiei almost 2 years ago

Ovidiu, please post the POJO for a table which has extent fields, blob, clob, etc (all data types).

#27 Updated by Ovidiu Maxiniuc almost 2 years ago

Constantin, here it is. First the .df content: "tm-all" table "tm-all" table

ADD TABLE "tm-all" 
  AREA "Schema Area" 
  DUMP-NAME "tm-all" 

ADD FIELD "f-1" OF "tm-all" AS character 
  FORMAT "X(8)" 
  INITIAL "1111111111" 
  POSITION 2
  MAX-WIDTH 16
  ORDER 10

ADD FIELD "f-2" OF "tm-all" AS date 
  FORMAT "99/99/9999" 
  INITIAL "2/22/22" 
  POSITION 3
  MAX-WIDTH 4
  ORDER 20

ADD FIELD "f-3" OF "tm-all" AS decimal 
  FORMAT "$*,**9.99+" 
  INITIAL "333.333" 
  POSITION 4
  MAX-WIDTH 17
  DECIMALS 2
  ORDER 30

ADD FIELD "f-4" OF "tm-all" AS integer 
  FORMAT "+>99" 
  INITIAL "4" 
  POSITION 5
  MAX-WIDTH 4
  ORDER 40

ADD FIELD "f-5" OF "tm-all" AS int64 
  FORMAT "+>9999" 
  INITIAL "55555" 
  POSITION 6
  MAX-WIDTH 8
  ORDER 50

ADD FIELD "f-6" OF "tm-all" AS logical 
  FORMAT "male/female" 
  INITIAL "no" 
  POSITION 7
  MAX-WIDTH 1
  ORDER 60

ADD FIELD "f-7" OF "tm-all" AS datetime 
  FORMAT "99-99-9999 HH:MM:SS.SSS AM" 
  INITIAL "7/7/77 07:07:07.777" 
  POSITION 8
  MAX-WIDTH 8
  ORDER 70

ADD FIELD "f-8" OF "tm-all" AS datetime-tz 
  FORMAT "99/99/9999 HH:MM:SS.SSS AM +HH:MM" 
  INITIAL "8/8/88 8:8:8.888+8:0" 
  POSITION 9
  MAX-WIDTH 12
  ORDER 80

ADD FIELD "f-9" OF "tm-all" AS blob 
  FORMAT "x(8)" 
  INITIAL ?
  POSITION 10
  LOB-AREA "Schema Area" 
  LOB-BYTES 104857600
  LOB-SIZE 100M
  ORDER 90

ADD FIELD "f-10" OF "tm-all" AS clob 
  FORMAT "x(8)" 
  INITIAL ?
  POSITION 11
  LOB-AREA "Schema Area" 
  LOB-BYTES 104857600
  LOB-SIZE 100M
  CLOB-CODEPAGE "iso8859-1" 
  CLOB-COLLATION "basic" 
  CLOB-TYPE 1
  ORDER 100

ADD FIELD "f-11" OF "tm-all" AS raw 
  FORMAT "x(8)" 
  INITIAL "" 
  POSITION 12
  MAX-WIDTH 16
  ORDER 110

ADD FIELD "f-12" OF "tm-all" AS recid 
  FORMAT ">>>>>>9" 
  INITIAL "121212" 
  POSITION 13
  MAX-WIDTH 8
  ORDER 120

ADD FIELD "e-1" OF "tm-all" AS character 
  FORMAT "X(8)" 
  INITIAL "1111111111" 
  POSITION 2
  EXTENT 5
  MAX-WIDTH 16
  ORDER 15

ADD FIELD "e-2" OF "tm-all" AS date 
  FORMAT "99/99/9999" 
  INITIAL "2/22/22" 
  EXTENT 5
  POSITION 3
  MAX-WIDTH 4
  ORDER 25

ADD FIELD "e-3" OF "tm-all" AS decimal 
  FORMAT "$*,**9.99+" 
  INITIAL "333.333" 
  EXTENT 3
  POSITION 4
  MAX-WIDTH 17
  DECIMALS 2
  ORDER 35

ADD FIELD "e-4" OF "tm-all" AS integer 
  FORMAT "+>99" 
  INITIAL "4" 
  EXTENT 3
  POSITION 5
  MAX-WIDTH 4
  ORDER 45

ADD INDEX "multi-index" ON "tm-all" 
  AREA "Schema Area" 
  UNIQUE
  PRIMARY
  INDEX-FIELD "f-1" ASCENDING 
  INDEX-FIELD "f-2" DESCENDING 
  INDEX-FIELD "f-3" ASCENDING 
  INDEX-FIELD "f-4" DESCENDING 
  INDEX-FIELD "f-5" ASCENDING 
  INDEX-FIELD "f-6" DESCENDING 
  INDEX-FIELD "f-7" ASCENDING 
  INDEX-FIELD "f-8" DESCENDING 
  INDEX-FIELD "f-12" DESCENDING

To force it to extreme, I used a partial denormalization using

<?xml version="1.0"?>

<!-- UAST hints for p2j_test schema -->
<hints>
   <schema>
      <table name="tm-all">
         <custom-extent>
            <field name="e-1"/>
            <field name="e-2"/>
            <!-- <field name="e-3"/> this extent field remains normalized -->
            <!-- <field name="e-4"/> this extent field also remains normalized -->
         </custom-extent>
      </table>
   </schema>
</hints>

Here is the generated DMO interface: [TmAll] DMO interface [TmAll] DMO interface

package com.goldencode.testcases.dmo.fwd;

import com.goldencode.p2j.persist.annotation.*;
import com.goldencode.p2j.persist.*;
import com.goldencode.p2j.persist.annotation.Trigger;
import com.goldencode.p2j.util.*;

/** Data Model Object corresponding with table {@code fwd.tm_all} (previously {@code fwd.tm-all}). */
@Table(name = "tm_all", legacy = "tm-all", pojoClass = com.goldencode.testcases.pojo3.fwd.TmAll.class)
@Indices(
{
   @Index(name = "tm_all_multi_index", legacy = "multi-index", primary = true, unique = true, components = 
   {
      @IndexComponent(name = "f1", legacy = "f-1"),
      @IndexComponent(name = "f2", legacy = "f-2", descending = true),
      @IndexComponent(name = "f3", legacy = "f-3"),
      @IndexComponent(name = "f4", legacy = "f-4", descending = true),
      @IndexComponent(name = "f5", legacy = "f-5"),
      @IndexComponent(name = "f6", legacy = "f-6", descending = true),
      @IndexComponent(name = "f7", legacy = "f-7"),
      @IndexComponent(name = "f8", legacy = "f-8", descending = true),
      @IndexComponent(name = "f12", legacy = "f-12", descending = true)
   })
})
public interface TmAll
extends DataModelObject
{
   /**
    * Getter: f-1
    * <p>
    * f1
    * 
    * @return  f1
    */
   @Property(id = 1, name = "f1", column = "f_1", legacy = "f-1", format = "X(8)", initial = "1111111111", width = 16, order = 10, position = 2)
   public character getF1();

   /**
    * Setter: f-1
    * <p>
    * f1
    * 
    * @param   f1
    *          f1
    */
   public void setF1(Text f1);

   /**
    * Getter: e-1
    * <p>
    * e11
    * 
    * @return  e11
    */
   @Property(id = 2, name = "e11", column = "e_11", legacy = "e-1", format = "X(8)", desc = "", initial = "1111111111", label = "", width = 16, order = 15, position = 2, index = 1, extent = 5, original = "e1")
   public character getE11();

   /**
    * Setter: e-1
    * <p>
    * e11
    * 
    * @param   e11
    *          e11
    */
   public void setE11(Text e11);

   /**
    * Getter: e-1
    * <p>
    * e12
    * 
    * @return  e12
    */
   @Property(id = 2, name = "e12", column = "e_12", legacy = "e-1", format = "X(8)", desc = "", initial = "1111111111", label = "", width = 16, order = 16, position = 2, index = 2, extent = 5, original = "e1")
   public character getE12();

   /**
    * Setter: e-1
    * <p>
    * e12
    * 
    * @param   e12
    *          e12
    */
   public void setE12(Text e12);

   /**
    * Getter: e-1
    * <p>
    * e13
    * 
    * @return  e13
    */
   @Property(id = 2, name = "e13", column = "e_13", legacy = "e-1", format = "X(8)", desc = "", initial = "1111111111", label = "", width = 16, order = 17, position = 2, index = 3, extent = 5, original = "e1")
   public character getE13();

   /**
    * Setter: e-1
    * <p>
    * e13
    * 
    * @param   e13
    *          e13
    */
   public void setE13(Text e13);

   /**
    * Getter: e-1
    * <p>
    * e14
    * 
    * @return  e14
    */
   @Property(id = 2, name = "e14", column = "e_14", legacy = "e-1", format = "X(8)", desc = "", initial = "1111111111", label = "", width = 16, order = 18, position = 2, index = 4, extent = 5, original = "e1")
   public character getE14();

   /**
    * Setter: e-1
    * <p>
    * e14
    * 
    * @param   e14
    *          e14
    */
   public void setE14(Text e14);

   /**
    * Getter: e-1
    * <p>
    * e15
    * 
    * @return  e15
    */
   @Property(id = 2, name = "e15", column = "e_15", legacy = "e-1", format = "X(8)", desc = "", initial = "1111111111", label = "", width = 16, order = 19, position = 2, index = 5, extent = 5, original = "e1")
   public character getE15();

   /**
    * Setter: e-1
    * <p>
    * e15
    * 
    * @param   e15
    *          e15
    */
   public void setE15(Text e15);

   /**
    * Getter: e-1
    * <p>
    * Indexed getter
    * 
    * @return  element
    */
   public character getE1(int index);

   /**
    * Setter: e-1
    * <p>
    * Indexed setter
    * 
    * @param   element
    *          e15
    */
   public void setE1(int index, Text element);

   /**
    * Getter: e-1
    * <p>
    * Indexed getter
    * 
    * @return  element
    */
   public character getE1(NumberType index);

   /**
    * Setter: e-1
    * <p>
    * Indexed setter
    * 
    * @param   element
    *          e15
    */
   public void setE1(NumberType index, Text element);

   /**
    * Set all elements of the target list property to the given scalar value. Modifies the
    * contents of: e1
    * 
    * @param   element
    *          Scalar value to which to set all elements of: element
    */
   public void setE1(Text element);

   /**
    * Set as many elements of the target list property as are provided in the given array of
    * scalar values. Modifies the contents of: e1
    * <p>
    * If the given array is larger than the target list, any additional elements in the given
    * array are ignored. If the target array is smaller than the target list, only those elements
    * whose array indices match are updated.
    * 
    * @param   elements
    *          Array of scalar values to which to set the corresponding elements of: element
    */
   public void setE1(Text[] elements);

   /**
    * Getter: e-1
    * <p>
    * Bulk getter
    * 
    * @return  element
    */
   public character[] getE1();

   /**
    * Getter: f-2
    * <p>
    * f2
    * 
    * @return  f2
    */
   @Property(id = 3, name = "f2", column = "f_2", legacy = "f-2", format = "99/99/9999", initial = "2/22/22", width = 4, order = 24, position = 3)
   public date getF2();

   /**
    * Setter: f-2
    * <p>
    * f2
    * 
    * @param   f2
    *          f2
    */
   public void setF2(date f2);

   /**
    * Getter: e-2
    * <p>
    * e21
    * 
    * @return  e21
    */
   @Property(id = 4, name = "e21", column = "e_21", legacy = "e-2", format = "99/99/9999", desc = "", initial = "2/22/22", label = "", width = 4, order = 29, position = 3, index = 1, extent = 5, original = "e2")
   public date getE21();

   /**
    * Setter: e-2
    * <p>
    * e21
    * 
    * @param   e21
    *          e21
    */
   public void setE21(date e21);

   /**
    * Getter: e-2
    * <p>
    * e22
    * 
    * @return  e22
    */
   @Property(id = 4, name = "e22", column = "e_22", legacy = "e-2", format = "99/99/9999", desc = "", initial = "2/22/22", label = "", width = 4, order = 30, position = 3, index = 2, extent = 5, original = "e2")
   public date getE22();

   /**
    * Setter: e-2
    * <p>
    * e22
    * 
    * @param   e22
    *          e22
    */
   public void setE22(date e22);

   /**
    * Getter: e-2
    * <p>
    * e23
    * 
    * @return  e23
    */
   @Property(id = 4, name = "e23", column = "e_23", legacy = "e-2", format = "99/99/9999", desc = "", initial = "2/22/22", label = "", width = 4, order = 31, position = 3, index = 3, extent = 5, original = "e2")
   public date getE23();

   /**
    * Setter: e-2
    * <p>
    * e23
    * 
    * @param   e23
    *          e23
    */
   public void setE23(date e23);

   /**
    * Getter: e-2
    * <p>
    * e24
    * 
    * @return  e24
    */
   @Property(id = 4, name = "e24", column = "e_24", legacy = "e-2", format = "99/99/9999", desc = "", initial = "2/22/22", label = "", width = 4, order = 32, position = 3, index = 4, extent = 5, original = "e2")
   public date getE24();

   /**
    * Setter: e-2
    * <p>
    * e24
    * 
    * @param   e24
    *          e24
    */
   public void setE24(date e24);

   /**
    * Getter: e-2
    * <p>
    * e25
    * 
    * @return  e25
    */
   @Property(id = 4, name = "e25", column = "e_25", legacy = "e-2", format = "99/99/9999", desc = "", initial = "2/22/22", label = "", width = 4, order = 33, position = 3, index = 5, extent = 5, original = "e2")
   public date getE25();

   /**
    * Setter: e-2
    * <p>
    * e25
    * 
    * @param   e25
    *          e25
    */
   public void setE25(date e25);

   /**
    * Getter: e-2
    * <p>
    * Indexed getter
    * 
    * @return  element
    */
   public date getE2(int index);

   /**
    * Setter: e-2
    * <p>
    * Indexed setter
    * 
    * @param   element
    *          e25
    */
   public void setE2(int index, date element);

   /**
    * Getter: e-2
    * <p>
    * Indexed getter
    * 
    * @return  element
    */
   public date getE2(NumberType index);

   /**
    * Setter: e-2
    * <p>
    * Indexed setter
    * 
    * @param   element
    *          e25
    */
   public void setE2(NumberType index, date element);

   /**
    * Set all elements of the target list property to the given scalar value. Modifies the
    * contents of: e2
    * 
    * @param   element
    *          Scalar value to which to set all elements of: element
    */
   public void setE2(date element);

   /**
    * Set as many elements of the target list property as are provided in the given array of
    * scalar values. Modifies the contents of: e2
    * <p>
    * If the given array is larger than the target list, any additional elements in the given
    * array are ignored. If the target array is smaller than the target list, only those elements
    * whose array indices match are updated.
    * 
    * @param   elements
    *          Array of scalar values to which to set the corresponding elements of: element
    */
   public void setE2(date[] elements);

   /**
    * Getter: e-2
    * <p>
    * Bulk getter
    * 
    * @return  element
    */
   public date[] getE2();

   /**
    * Getter: f-3
    * <p>
    * f3
    * 
    * @return  f3
    */
   @Property(id = 5, name = "f3", column = "f_3", legacy = "f-3", format = "$*,**9.99+", initial = "333.333", width = 17, order = 38, position = 4, scale = 2)
   public decimal getF3();

   /**
    * Setter: f-3
    * <p>
    * f3
    * 
    * @param   f3
    *          f3
    */
   public void setF3(NumberType f3);

   /**
    * Getter: e-3
    * <p>
    * e3
    * 
    * @return  e3
    */
   public decimal[] getE3();

   /**
    * Indexed getter: e-3
    * <p>
    * e3
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * 
    * @return  e3
    */
   @Property(id = 6, name = "e3", column = "e_3", legacy = "e-3", format = "$*,**9.99+", initial = "333.333", width = 17, order = 43, position = 4, extent = 3, scale = 2)
   public decimal getE3(int index);

   /**
    * Indexed getter: e-3
    * <p>
    * e3
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * 
    * @return  e3
    */
   public decimal getE3(NumberType index);

   /**
    * Indexed setter: e-3
    * <p>
    * e3
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * @param   element
    *          e3
    */
   public void setE3(int index, NumberType element);

   /**
    * Indexed setter: e-3
    * <p>
    * e3
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * @param   element
    *          e3
    */
   public void setE3(NumberType index, NumberType element);

   /**
    * Set all elements of the target list property to the given scalar value. Modifies the
    * contents of: e3
    * 
    * @param   element
    *          Scalar value to which to set all elements of: e3
    */
   public void setE3(NumberType element);

   /**
    * Set as many elements of the target list property as are provided in the given array of
    * scalar values. Modifies the contents of: e3
    * <p>
    * If the given array is larger than the target list, any additional elements in the given
    * array are ignored. If the target array is smaller than the target list, only those elements
    * whose array indices match are updated.
    * 
    * @param   elements
    *          Array of scalar values to which to set the corresponding elements of: e3
    */
   public void setE3(NumberType[] elements);

   /**
    * Getter: f-4
    * <p>
    * f4
    * 
    * @return  f4
    */
   @Property(id = 7, name = "f4", column = "f_4", legacy = "f-4", format = "+>99", initial = "4", width = 4, order = 48, position = 5)
   public integer getF4();

   /**
    * Setter: f-4
    * <p>
    * f4
    * 
    * @param   f4
    *          f4
    */
   public void setF4(NumberType f4);

   /**
    * Getter: e-4
    * <p>
    * e4
    * 
    * @return  e4
    */
   public integer[] getE4();

   /**
    * Indexed getter: e-4
    * <p>
    * e4
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * 
    * @return  e4
    */
   @Property(id = 8, name = "e4", column = "e_4", legacy = "e-4", format = "+>99", initial = "4", width = 4, order = 53, position = 5, extent = 3)
   public integer getE4(int index);

   /**
    * Indexed getter: e-4
    * <p>
    * e4
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * 
    * @return  e4
    */
   public integer getE4(NumberType index);

   /**
    * Indexed setter: e-4
    * <p>
    * e4
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * @param   element
    *          e4
    */
   public void setE4(int index, NumberType element);

   /**
    * Indexed setter: e-4
    * <p>
    * e4
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * @param   element
    *          e4
    */
   public void setE4(NumberType index, NumberType element);

   /**
    * Set all elements of the target list property to the given scalar value. Modifies the
    * contents of: e4
    * 
    * @param   element
    *          Scalar value to which to set all elements of: e4
    */
   public void setE4(NumberType element);

   /**
    * Set as many elements of the target list property as are provided in the given array of
    * scalar values. Modifies the contents of: e4
    * <p>
    * If the given array is larger than the target list, any additional elements in the given
    * array are ignored. If the target array is smaller than the target list, only those elements
    * whose array indices match are updated.
    * 
    * @param   elements
    *          Array of scalar values to which to set the corresponding elements of: e4
    */
   public void setE4(NumberType[] elements);

   /**
    * Getter: f-5
    * <p>
    * f5
    * 
    * @return  f5
    */
   @Property(id = 9, name = "f5", column = "f_5", legacy = "f-5", format = "+>9999", initial = "55555", width = 8, order = 58, position = 6)
   public int64 getF5();

   /**
    * Setter: f-5
    * <p>
    * f5
    * 
    * @param   f5
    *          f5
    */
   public void setF5(NumberType f5);

   /**
    * Getter: f-6
    * <p>
    * f6
    * 
    * @return  f6
    */
   @Property(id = 10, name = "f6", column = "f_6", legacy = "f-6", format = "male/female", initial = "false", width = 1, order = 68, position = 7)
   public logical isF6();

   /**
    * Setter: f-6
    * <p>
    * f6
    * 
    * @param   f6
    *          f6
    */
   public void setF6(logical f6);

   /**
    * Getter: f-7
    * <p>
    * f7
    * 
    * @return  f7
    */
   @Property(id = 11, name = "f7", column = "f_7", legacy = "f-7", format = "99-99-9999 HH:MM:SS.SSS AM", initial = "7/7/77 07:07:07.777", width = 8, order = 78, position = 8)
   public datetime getF7();

   /**
    * Setter: f-7
    * <p>
    * f7
    * 
    * @param   f7
    *          f7
    */
   public void setF7(date f7);

   /**
    * Getter: f-8
    * <p>
    * f8
    * 
    * @return  f8
    */
   @Property(id = 12, name = "f8", column = "f_8", legacy = "f-8", format = "99/99/9999 HH:MM:SS.SSS AM +HH:MM", initial = "8/8/88 8:8:8.888+8:0", width = 12, order = 88, position = 9)
   public datetimetz getF8();

   /**
    * Setter: f-8
    * <p>
    * f8
    * 
    * @param   f8
    *          f8
    */
   public void setF8(date f8);

   /**
    * Getter: f-9
    * <p>
    * f9
    * 
    * @return  f9
    */
   @Property(id = 13, name = "f9", column = "f_9", legacy = "f-9", format = "x(8)", initialNull = true, order = 98, position = 10)
   public blob getF9();

   /**
    * Setter: f-9
    * <p>
    * f9
    * 
    * @param   f9
    *          f9
    */
   public void setF9(blob f9);

   /**
    * Getter: f-10
    * <p>
    * f10
    * 
    * @return  f10
    */
   @Property(id = 14, name = "f10", column = "f_10", legacy = "f-10", format = "x(8)", initialNull = true, order = 108, position = 11, codePage = "iso8859-1")
   public clob getF10();

   /**
    * Setter: f-10
    * <p>
    * f10
    * 
    * @param   f10
    *          f10
    */
   public void setF10(clob f10);

   /**
    * Getter: f-11
    * <p>
    * f11
    * 
    * @return  f11
    */
   @Property(id = 15, name = "f11", column = "f_11", legacy = "f-11", format = "x(8)", width = 16, order = 118, position = 12)
   public raw getF11();

   /**
    * Setter: f-11
    * <p>
    * f11
    * 
    * @param   f11
    *          f11
    */
   public void setF11(BinaryData f11);

   /**
    * Getter: f-12
    * <p>
    * f12
    * 
    * @return  f12
    */
   @Property(id = 16, name = "f12", column = "f_12", legacy = "f-12", format = ">>>>>>9", initial = "121212", width = 8, order = 128, position = 13)
   public recid getF12();

   /**
    * Setter: f-12
    * <p>
    * f12
    * 
    * @param   f12
    *          f12
    */
   public void setF12(NumberType f12);

   /**
    * DMO interface for use by converted business logic.
    */
   public interface Buf
   extends TmAll, Buffer
   {
   }
}

Here is the generated POJO class: [TmAll] POJO class [TmAll] POJO class

package com.goldencode.testcases.pojo3.fwd;

import java.io.*;
import java.sql.*;
import java.time.*;
import java.math.*;

/**
 * Plain Old Java Object class corresponding with table {@code fwd.tm_all} (previously {@code
 * fwd.tm-all}).
 */
public class TmAll
implements Serializable
{
   /** Field 'f-1': f1 */
   private String f1 = null;

   /** Field 'e-1': */
   private String e11 = null;

   /** Field 'e-1': */
   private String e12 = null;

   /** Field 'e-1': */
   private String e13 = null;

   /** Field 'e-1': */
   private String e14 = null;

   /** Field 'e-1': */
   private String e15 = null;

   /** Field 'f-2': f2 */
   private Date f2 = null;

   /** Field 'e-2': */
   private Date e21 = null;

   /** Field 'e-2': */
   private Date e22 = null;

   /** Field 'e-2': */
   private Date e23 = null;

   /** Field 'e-2': */
   private Date e24 = null;

   /** Field 'e-2': */
   private Date e25 = null;

   /** Field 'f-3': f3 */
   private BigDecimal f3 = null;

   /** Field 'e-3': e3 */
   private BigDecimal[] e3 = new BigDecimal[3];

   /** Field 'f-4': f4 */
   private Integer f4 = null;

   /** Field 'e-4': e4 */
   private Integer[] e4 = new Integer[3];

   /** Field 'f-5': f5 */
   private Long f5 = null;

   /** Field 'f-6': f6 */
   private Boolean f6 = null;

   /** Field 'f-7': f7 */
   private Timestamp f7 = null;

   /** Field 'f-8': f8 */
   private OffsetDateTime f8 = null;

   /** Field 'f-9': f9 */
   private byte[] f9 = null;

   /** Field 'f-10': f10 */
   private Clob f10 = null;

   /** Field 'f-11': f11 */
   private byte[] f11 = null;

   /** Field 'f-12': f12 */
   private Integer f12 = null;

   /**
    * Getter: f-1
    * <p>
    * f1
    * 
    * @return  f1
    */
   public String getF1()
   {
      return f1;
   }

   /**
    * Setter: f-1
    * <p>
    * f1
    * 
    * @param   f1
    *          f1
    */
   public void setF1(String f1)
   {
      this.f1 = f1;
   }

   /**
    * Getter: e-1
    * <p>
    * 
    * @return  e11
    */
   public String getE11()
   {
      return e11;
   }

   /**
    * Setter: e-1
    * <p>
    * 
    * @param   e11
    *          e11
    */
   public void setE11(String e11)
   {
      this.e11 = e11;
   }

   /**
    * Getter: e-1
    * <p>
    * 
    * @return  e12
    */
   public String getE12()
   {
      return e12;
   }

   /**
    * Setter: e-1
    * <p>
    * 
    * @param   e12
    *          e12
    */
   public void setE12(String e12)
   {
      this.e12 = e12;
   }

   /**
    * Getter: e-1
    * <p>
    * 
    * @return  e13
    */
   public String getE13()
   {
      return e13;
   }

   /**
    * Setter: e-1
    * <p>
    * 
    * @param   e13
    *          e13
    */
   public void setE13(String e13)
   {
      this.e13 = e13;
   }

   /**
    * Getter: e-1
    * <p>
    * 
    * @return  e14
    */
   public String getE14()
   {
      return e14;
   }

   /**
    * Setter: e-1
    * <p>
    * 
    * @param   e14
    *          e14
    */
   public void setE14(String e14)
   {
      this.e14 = e14;
   }

   /**
    * Getter: e-1
    * <p>
    * 
    * @return  e15
    */
   public String getE15()
   {
      return e15;
   }

   /**
    * Setter: e-1
    * <p>
    * 
    * @param   e15
    *          e15
    */
   public void setE15(String e15)
   {
      this.e15 = e15;
   }

   /**
    * Getter: f-2
    * <p>
    * f2
    * 
    * @return  f2
    */
   public Date getF2()
   {
      return f2;
   }

   /**
    * Setter: f-2
    * <p>
    * f2
    * 
    * @param   f2
    *          f2
    */
   public void setF2(Date f2)
   {
      this.f2 = f2;
   }

   /**
    * Getter: e-2
    * <p>
    * 
    * @return  e21
    */
   public Date getE21()
   {
      return e21;
   }

   /**
    * Setter: e-2
    * <p>
    * 
    * @param   e21
    *          e21
    */
   public void setE21(Date e21)
   {
      this.e21 = e21;
   }

   /**
    * Getter: e-2
    * <p>
    * 
    * @return  e22
    */
   public Date getE22()
   {
      return e22;
   }

   /**
    * Setter: e-2
    * <p>
    * 
    * @param   e22
    *          e22
    */
   public void setE22(Date e22)
   {
      this.e22 = e22;
   }

   /**
    * Getter: e-2
    * <p>
    * 
    * @return  e23
    */
   public Date getE23()
   {
      return e23;
   }

   /**
    * Setter: e-2
    * <p>
    * 
    * @param   e23
    *          e23
    */
   public void setE23(Date e23)
   {
      this.e23 = e23;
   }

   /**
    * Getter: e-2
    * <p>
    * 
    * @return  e24
    */
   public Date getE24()
   {
      return e24;
   }

   /**
    * Setter: e-2
    * <p>
    * 
    * @param   e24
    *          e24
    */
   public void setE24(Date e24)
   {
      this.e24 = e24;
   }

   /**
    * Getter: e-2
    * <p>
    * 
    * @return  e25
    */
   public Date getE25()
   {
      return e25;
   }

   /**
    * Setter: e-2
    * <p>
    * 
    * @param   e25
    *          e25
    */
   public void setE25(Date e25)
   {
      this.e25 = e25;
   }

   /**
    * Getter: f-3
    * <p>
    * f3
    * 
    * @return  f3
    */
   public BigDecimal getF3()
   {
      return f3;
   }

   /**
    * Setter: f-3
    * <p>
    * f3
    * 
    * @param   f3
    *          f3
    */
   public void setF3(BigDecimal f3)
   {
      this.f3 = f3;
   }

   /**
    * Getter: e-3
    * <p>
    * e3
    * 
    * @return  e3
    */
   public BigDecimal[] getE3()
   {
      return e3;
   }

   /**
    * Indexed getter: e-3
    * <p>
    * e3
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * 
    * @return  e3
    */
   public BigDecimal getE3(int index)
   {
      return e3[index];
   }

   /**
    * Setter: e-3
    * <p>
    * e3
    * 
    * @param   e3
    *          e3
    */
   public void setE3(BigDecimal[] e3)
   {
      this.e3 = e3;
   }

   /**
    * Indexed setter: e-3
    * <p>
    * e3
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * @param   e3
    *          e3
    */
   public void setE3(int index, BigDecimal e3)
   {
      this.e3[index] = e3;
   }

   /**
    * Getter: f-4
    * <p>
    * f4
    * 
    * @return  f4
    */
   public Integer getF4()
   {
      return f4;
   }

   /**
    * Setter: f-4
    * <p>
    * f4
    * 
    * @param   f4
    *          f4
    */
   public void setF4(Integer f4)
   {
      this.f4 = f4;
   }

   /**
    * Getter: e-4
    * <p>
    * e4
    * 
    * @return  e4
    */
   public Integer[] getE4()
   {
      return e4;
   }

   /**
    * Indexed getter: e-4
    * <p>
    * e4
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * 
    * @return  e4
    */
   public Integer getE4(int index)
   {
      return e4[index];
   }

   /**
    * Setter: e-4
    * <p>
    * e4
    * 
    * @param   e4
    *          e4
    */
   public void setE4(Integer[] e4)
   {
      this.e4 = e4;
   }

   /**
    * Indexed setter: e-4
    * <p>
    * e4
    * 
    * @param   index
    *          Index of element (where {@code 0 <= index < 3}).
    * @param   e4
    *          e4
    */
   public void setE4(int index, Integer e4)
   {
      this.e4[index] = e4;
   }

   /**
    * Getter: f-5
    * <p>
    * f5
    * 
    * @return  f5
    */
   public Long getF5()
   {
      return f5;
   }

   /**
    * Setter: f-5
    * <p>
    * f5
    * 
    * @param   f5
    *          f5
    */
   public void setF5(Long f5)
   {
      this.f5 = f5;
   }

   /**
    * Getter: f-6
    * <p>
    * f6
    * 
    * @return  f6
    */
   public Boolean isF6()
   {
      return f6;
   }

   /**
    * Setter: f-6
    * <p>
    * f6
    * 
    * @param   f6
    *          f6
    */
   public void setF6(Boolean f6)
   {
      this.f6 = f6;
   }

   /**
    * Getter: f-7
    * <p>
    * f7
    * 
    * @return  f7
    */
   public Timestamp getF7()
   {
      return f7;
   }

   /**
    * Setter: f-7
    * <p>
    * f7
    * 
    * @param   f7
    *          f7
    */
   public void setF7(Timestamp f7)
   {
      this.f7 = f7;
   }

   /**
    * Getter: f-8
    * <p>
    * f8
    * 
    * @return  f8
    */
   public OffsetDateTime getF8()
   {
      return f8;
   }

   /**
    * Setter: f-8
    * <p>
    * f8
    * 
    * @param   f8
    *          f8
    */
   public void setF8(OffsetDateTime f8)
   {
      this.f8 = f8;
   }

   /**
    * Getter: f-9
    * <p>
    * f9
    * 
    * @return  f9
    */
   public byte[] getF9()
   {
      return f9;
   }

   /**
    * Setter: f-9
    * <p>
    * f9
    * 
    * @param   f9
    *          f9
    */
   public void setF9(byte[] f9)
   {
      this.f9 = f9;
   }

   /**
    * Getter: f-10
    * <p>
    * f10
    * 
    * @return  f10
    */
   public Clob getF10()
   {
      return f10;
   }

   /**
    * Setter: f-10
    * <p>
    * f10
    * 
    * @param   f10
    *          f10
    */
   public void setF10(Clob f10)
   {
      this.f10 = f10;
   }

   /**
    * Getter: f-11
    * <p>
    * f11
    * 
    * @return  f11
    */
   public byte[] getF11()
   {
      return f11;
   }

   /**
    * Setter: f-11
    * <p>
    * f11
    * 
    * @param   f11
    *          f11
    */
   public void setF11(byte[] f11)
   {
      this.f11 = f11;
   }

   /**
    * Getter: f-12
    * <p>
    * f12
    * 
    * @return  f12
    */
   public Integer getF12()
   {
      return f12;
   }

   /**
    * Setter: f-12
    * <p>
    * f12
    * 
    * @param   f12
    *          f12
    */
   public void setF12(Integer f12)
   {
      this.f12 = f12;
   }
}

For completeness, the in-memory DMO class looks like this: [TmAll__Impl__] DMO implementation class

package com.goldencode.testcases.dmo.fwd;

import com.goldencode.p2j.persist.Persistable;
import com.goldencode.p2j.persist.Record;
import com.goldencode.p2j.persist.orm.RecordMeta;
import com.goldencode.p2j.util.BinaryData;
import com.goldencode.p2j.util.NumberType;
import com.goldencode.p2j.util.Text;
import com.goldencode.p2j.util.blob;
import com.goldencode.p2j.util.character;
import com.goldencode.p2j.util.clob;
import com.goldencode.p2j.util.date;
import com.goldencode.p2j.util.datetime;
import com.goldencode.p2j.util.datetimetz;
import com.goldencode.p2j.util.decimal;
import com.goldencode.p2j.util.int64;
import com.goldencode.p2j.util.integer;
import com.goldencode.p2j.util.logical;
import com.goldencode.p2j.util.raw;
import com.goldencode.p2j.util.recid;
import java.math.BigDecimal;
import java.sql.Clob;
import java.sql.Date;
import java.sql.OffsetDateTime;
import java.sql.Timestamp;

public final class TmAll__Impl__ extends Record implements Persistable, TmAll {
   private static final RecordMeta _metadata = RecordMeta.get(TmAll.class);

   protected final RecordMeta _recordMeta() {
      return _metadata;
   }

   public TmAll__Impl__() {
   }

   public character getF1() {
      return this._getCharacter(0);
   }

   public void setF1(Text var1) {
      this._setCharacter(0, var1);
   }

   public character getE11() {
      return this._getCharacter(1);
   }

   public void setE11(Text var1) {
      this._setCharacter(1, var1);
   }

   public character getE1(NumberType var1) {
      return this._getCharacter(1 + var1.intValue());
   }

   public void setE1(NumberType var1, Text var2) {
      this._setCharacter(1 + var1.intValue(), var2);
   }

   public character[] getE1() {
      return this._getCharacter(5, 1);
   }

   public void setE1(Text[] var1) {
      this._setCharacter(var1, 0, 1);
   }

   public void setE1(Text var1) {
      this._setCharacter(var1, 0, 1);
   }

   public character getE1(int var1) {
      return this._getCharacter(1 + var1);
   }

   public void setE1(int var1, Text var2) {
      this._setCharacter(1 + var1, var2);
   }

   public character getE12() {
      return this._getCharacter(2);
   }

   public void setE12(Text var1) {
      this._setCharacter(2, var1);
   }

   public character getE13() {
      return this._getCharacter(3);
   }

   public void setE13(Text var1) {
      this._setCharacter(3, var1);
   }

   public character getE14() {
      return this._getCharacter(4);
   }

   public void setE14(Text var1) {
      this._setCharacter(4, var1);
   }

   public character getE15() {
      return this._getCharacter(5);
   }

   public void setE15(Text var1) {
      this._setCharacter(5, var1);
   }

   public date getF2() {
      return this._getDate(6);
   }

   public void setF2(date var1) {
      this._setDate(6, var1);
   }

   public date getE21() {
      return this._getDate(7);
   }

   public void setE21(date var1) {
      this._setDate(7, var1);
   }

   public date getE2(NumberType var1) {
      return this._getDate(7 + var1.intValue());
   }

   public void setE2(NumberType var1, date var2) {
      this._setDate(7 + var1.intValue(), var2);
   }

   public date[] getE2() {
      return this._getDate(5, 7);
   }

   public void setE2(date[] var1) {
      this._setDate(var1, 0, 7);
   }

   public void setE2(date var1) {
      this._setDate(var1, 0, 7);
   }

   public date getE2(int var1) {
      return this._getDate(7 + var1);
   }

   public void setE2(int var1, date var2) {
      this._setDate(7 + var1, var2);
   }

   public date getE22() {
      return this._getDate(8);
   }

   public void setE22(date var1) {
      this._setDate(8, var1);
   }

   public date getE23() {
      return this._getDate(9);
   }

   public void setE23(date var1) {
      this._setDate(9, var1);
   }

   public date getE24() {
      return this._getDate(10);
   }

   public void setE24(date var1) {
      this._setDate(10, var1);
   }

   public date getE25() {
      return this._getDate(11);
   }

   public void setE25(date var1) {
      this._setDate(11, var1);
   }

   public decimal getF3() {
      return this._getDecimal(12);
   }

   public void setF3(NumberType var1) {
      this._setDecimal(12, var1);
   }

   public integer getF4() {
      return this._getInteger(13);
   }

   public void setF4(NumberType var1) {
      this._setInteger(13, var1);
   }

   public int64 getF5() {
      return this._getInt64(14);
   }

   public void setF5(NumberType var1) {
      this._setInt64(14, var1);
   }

   public logical isF6() {
      return this._isLogical(15);
   }

   public void setF6(logical var1) {
      this._setLogical(15, var1);
   }

   public datetime getF7() {
      return this._getDatetime(16);
   }

   public void setF7(date var1) {
      this._setDatetime(16, var1);
   }

   public datetimetz getF8() {
      return this._getDatetimetz(17);
   }

   public void setF8(date var1) {
      this._setDatetimetz(17, var1);
   }

   public blob getF9() {
      return this._getBlob(18);
   }

   public void setF9(blob var1) {
      this._setBlob(18, var1);
   }

   public clob getF10() {
      return this._getClob(19);
   }

   public void setF10(clob var1) {
      this._setClob(19, var1);
   }

   public raw getF11() {
      return this._getRaw(20);
   }

   public void setF11(BinaryData var1) {
      this._setRaw(20, var1);
   }

   public recid getF12() {
      return this._getRecid(21);
   }

   public void setF12(NumberType var1) {
      this._setRecid(21, var1);
   }

   public decimal getE3(int var1) {
      return this._getDecimal(22 + var1);
   }

   public void setE3(int var1, NumberType var2) {
      this._setDecimal(22 + var1, var2);
   }

   public decimal getE3(NumberType var1) {
      return this._getDecimal(22 + var1.intValue());
   }

   public void setE3(NumberType var1, NumberType var2) {
      this._setDecimal(22 + var1.intValue(), var2);
   }

   public decimal[] getE3() {
      return this._getDecimal(3, 22);
   }

   public void setE3(NumberType[] var1) {
      this._setDecimal(var1, 3, 22);
   }

   public void setE3(NumberType var1) {
      this._setDecimal(var1, 3, 22);
   }

   public integer getE4(int var1) {
      return this._getInteger(25 + var1);
   }

   public void setE4(int var1, NumberType var2) {
      this._setInteger(25 + var1, var2);
   }

   public integer getE4(NumberType var1) {
      return this._getInteger(25 + var1.intValue());
   }

   public void setE4(NumberType var1, NumberType var2) {
      this._setInteger(25 + var1.intValue(), var2);
   }

   public integer[] getE4() {
      return this._getInteger(3, 25);
   }

   public void setE4(NumberType[] var1) {
      this._setInteger(var1, 3, 25);
   }

   public void setE4(NumberType var1) {
      this._setInteger(var1, 3, 25);
   }

   public Object toPOJO() {
      com.goldencode.testcases.pojo3.fwd.TmAll var1 = new com.goldencode.testcases.pojo3.fwd.TmAll();
      var1.setF1((String)super.data[0]);
      var1.setE11((String)super.data[1]);
      var1.setE12((String)super.data[2]);
      var1.setE13((String)super.data[3]);
      var1.setE14((String)super.data[4]);
      var1.setE15((String)super.data[5]);
      var1.setF2((Date)super.data[6]);
      var1.setE21((Date)super.data[7]);
      var1.setE22((Date)super.data[8]);
      var1.setE23((Date)super.data[9]);
      var1.setE24((Date)super.data[10]);
      var1.setE25((Date)super.data[11]);
      var1.setF3((BigDecimal)super.data[12]);
      var1.setF4((Integer)super.data[13]);
      var1.setF5((Long)super.data[14]);
      var1.setF6((Boolean)super.data[15]);
      var1.setF7((Timestamp)super.data[16]);
      var1.setF8((OffsetDateTime)super.data[17]);
      var1.setF9((byte[])super.data[18]);
      var1.setF10((Clob)super.data[19]);
      var1.setF11((byte[])super.data[20]);
      var1.setF12((Integer)super.data[21]);
      System.arraycopy(super.data, 22, var1.getE3(), 0, 3);
      System.arraycopy(super.data, 25, var1.getE4(), 0, 3);
      return var1;
   }

   public void populateFromPOJO(Object var1) {
      com.goldencode.testcases.pojo3.fwd.TmAll var2 = (com.goldencode.testcases.pojo3.fwd.TmAll)var1;
      super.data[0] = var2.getF1();
      super.data[1] = var2.getE11();
      super.data[2] = var2.getE12();
      super.data[3] = var2.getE13();
      super.data[4] = var2.getE14();
      super.data[5] = var2.getE15();
      super.data[6] = var2.getF2();
      super.data[7] = var2.getE21();
      super.data[8] = var2.getE22();
      super.data[9] = var2.getE23();
      super.data[10] = var2.getE24();
      super.data[11] = var2.getE25();
      super.data[12] = var2.getF3();
      super.data[13] = var2.getF4();
      super.data[14] = var2.getF5();
      super.data[15] = var2.isF6();
      super.data[16] = var2.getF7();
      super.data[17] = var2.getF8();
      super.data[18] = var2.getF9();
      super.data[19] = var2.getF10();
      super.data[20] = var2.getF11();
      super.data[21] = var2.getF12();
      System.arraycopy(var2.getE3(), 0, super.data, 22, 3);
      System.arraycopy(var2.getE4(), 0, super.data, 25, 3);
   }
}

#28 Updated by Constantin Asofiei almost 2 years ago

I don't think is OK to be using the SQL types:

date/datetime/datetime-tz should all map to java.util.GregorianCalendar
clob should map to java.lang.String - I don't even think you can transport this via the network (i.e. serialized).

#29 Updated by Ovidiu Maxiniuc almost 2 years ago

% Done changed from 60 to 90

This was my initial solution, where I tried to keep the same data types from FWD Record (see #6584-15).
I will add some conversion wrappers from java.sql.* types to GregorianCalendar and from Clob to java.lang.String.
Are there other concerns about the types or generally about the implementation itself?

#30 Updated by Eric Faulhaber almost 2 years ago

Ovidiu Maxiniuc wrote:

Are there other concerns about the types or generally about the implementation itself?

Did you post the runtime changes? There seem at least to be changes to the Record class, and I assume some changes in DmoClass or elsewhere to generate the to/from POJO methods in the DMO implementation class. How are errors handled during unmarshalling a POJO, if the POJO user has made mistakes (e.g., set a null array or an array of the wrong size for an extent field, set a null for a mandatory field, etc.)?

#31 Updated by Eric Faulhaber almost 2 years ago

Ovidiu, I'm running a customer application (unrelated to this task). I launched a program which tries to register a temp-table DMO and generate its implementation class. We have a regression.regression.

java.lang.IllegalArgumentException: Failed to create DMO implementation for (interface xxx.dmo._temp.Tt_2_1)
        at com.goldencode.p2j.persist.orm.DmoMetadataManager.registerDmo(DmoMetadataManager.java:245)
        at com.goldencode.p2j.persist.RecordBuffer.<init>(RecordBuffer.java:1862)
        at com.goldencode.p2j.persist.TemporaryBuffer.<init>(TemporaryBuffer.java:837)
        at com.goldencode.p2j.persist.TemporaryBuffer.makeBuffer(TemporaryBuffer.java:3199)
        at com.goldencode.p2j.persist.TemporaryBuffer.makeBuffer(TemporaryBuffer.java:3159)
        at com.goldencode.p2j.persist.TemporaryBuffer.define(TemporaryBuffer.java:955)
        at com.goldencode.p2j.persist.TemporaryBuffer.define(TemporaryBuffer.java:916)
        ...
Caused by: java.lang.NullPointerException
        at com.goldencode.p2j.persist.orm.DmoClass.<init>(DmoClass.java:328)
        at com.goldencode.p2j.persist.orm.DmoClass.assembleImplementation(DmoClass.java:419)
        at com.goldencode.p2j.persist.orm.DmoMetadataManager.registerDmo(DmoMetadataManager.java:241)
        at com.goldencode.p2j.persist.RecordBuffer.<init>(RecordBuffer.java:1862)
        at com.goldencode.p2j.persist.TemporaryBuffer.<init>(TemporaryBuffer.java:837)
        at com.goldencode.p2j.persist.TemporaryBuffer.makeBuffer(TemporaryBuffer.java:3199)
        at com.goldencode.p2j.persist.TemporaryBuffer.makeBuffer(TemporaryBuffer.java:3159)
        at com.goldencode.p2j.persist.TemporaryBuffer.define(TemporaryBuffer.java:955)
        at com.goldencode.p2j.persist.TemporaryBuffer.define(TemporaryBuffer.java:916)
        ...

The issue is these lines in DmoClass:

      Table annotation = dmoIface.getAnnotation(Table.class);
      Class<?> pojoClass = annotation.pojoClass();

Notice in the stack trace, it is trying to register the DMO xxx.dmo._temp.Tt_2_1. This interface has no Table annotation, but its superclass xxx.dmo._temp.Tt_2 does. So, we hit NPE when trying to invoke annotation.pojoClass().

Unless we see a use case for temp-table POJOs, I think we need to exclude temp-tables from this feature.

#32 Updated by Eric Faulhaber almost 2 years ago

Eric Faulhaber wrote:

Ovidiu Maxiniuc wrote:

Are there other concerns about the types or generally about the implementation itself?

Did you post the runtime changes?

You can ignore that question; I missed the entry where you wrote you committed the changes. I didn't realize it, but clearly I have them already ;)

#33 Updated by Eric Faulhaber almost 2 years ago

Eric Faulhaber wrote:

Ovidiu, I'm running a customer application (unrelated to this task). I launched a program which tries to register a temp-table DMO and generate its implementation class. We have a {{collapse(regression.)
[...]
}}

The issue is these lines in DmoClass:

[...]

Notice in the stack trace, it is trying to register the DMO xxx.dmo._temp.Tt_2_1. This interface has no Table annotation, but its superclass xxx.dmo._temp.Tt_2 does. So, we hit NPE when trying to invoke annotation.pojoClass().

Unless we see a use case for temp-table POJOs, I think we need to exclude temp-tables from this feature.

My temporary workaround is a null check, but I think this needs a permanent fix that does not generate unnecessary POJOs for temp-tables:

=== modified file 'src/com/goldencode/p2j/persist/orm/DmoClass.java'
--- src/com/goldencode/p2j/persist/orm/DmoClass.java    2022-07-14 01:33:49 +0000
+++ src/com/goldencode/p2j/persist/orm/DmoClass.java    2022-07-14 19:09:29 +0000
@@ -325,11 +325,14 @@
       }

       Table annotation = dmoIface.getAnnotation(Table.class);
-      Class<?> pojoClass = annotation.pojoClass();
-      if (pojoClass != Object.class)
+      if (annotation != null)
       {
-         create_toPOJO_method(pojoClass, propMeta);
-         create_populateFromPOJO_method(pojoClass, propMeta);
+         Class<?> pojoClass = annotation.pojoClass();
+         if (pojoClass != Object.class)
+         {
+            create_toPOJO_method(pojoClass, propMeta);
+            create_populateFromPOJO_method(pojoClass, propMeta);
+         }
       }
    }

#34 Updated by Ovidiu Maxiniuc almost 2 years ago

I already committed the same fix (r14069).

Notice that, for the moment, the POJOs are not supported for temp database. That is mainly because it is impossible (to my knowledge) to configure the generate-pojos in the p2j.cfg.xml for this database.

#35 Updated by Ovidiu Maxiniuc almost 2 years ago

Constantin Asofiei wrote:

I don't think is OK to be using the SQL types:

date/datetime/datetime-tz should all map to java.util.GregorianCalendar

clob should map to java.lang.String - I don't even think you can transport this via the network (i.e. serialized).

I have committed r14071. It will generate POJOs using the above types. The toPOJO() method will apply the required conversion when creating the POJOs. The reverse implementation is underway.

#36 Updated by Ovidiu Maxiniuc almost 2 years ago

% Done changed from 90 to 100

I committed the (final ?) version of the POJO/DMO marshalling solution. Revision is 14075.

To summarize:

the POJOs are created only if the generate-pojos parameter for respective schema was set to "true" (or "yes") in the desired namespace in p2j.cfg.xml. Generation of POJOs for temp-tables is not (yet) possible simply because this cannot be configured;
the destination package for POJOs can be configured using the global parameter pojoroot of p2j.cfg.xml. If not specified, a pojo package sibling to dmo will be used;
the data types used by POJOs are J2SE classes. The POJOs have no dependencies on FWD code. The full mapping can be found in rules/schema/java_pojo.xml at lines 100-116. All date related use GregorianCalendar instances and the clob is mapped to its String content. The static methods for these conversion are found at the end of Record class ("to...");
to obtain a POJO from a DMO object, the toPOJO() method of Record is used. If the POJO class was defined, an object of that type will be returned. On any error null is returned. The method is abstract in Record but it is implemented on-the-fly in the DMO implementation class. A hard constraint for this is that the conversion method needs access to protected data from BaseRecord. The assembler will use the record's recordMeta to generate the method's code without using reflection at all. The required conversions to POJO types are injected as required;
to obtain a DMO record from a POJO, the Record.fromPOJO() static method must be used. If the class of the parameter was registered as a POJO type when a DMO interface was registered with DmoMetadataManager then an object of this specific type will be created and populated with data from the POJO object sent as parameter. If the object's type is unknown or in case of other errors, null is return.
To populate the new DMO with data, the fromPOJO() invokes populateFromPOJO() for the newly created object. At this moment, the DMO object is aware of the type of the POJO and knows what getter methods to access. This method is also abstract in Record and assembled at runtime, when the DMO is registered. Again, there is no reflection used and, if required, static conversion methods from Record class are injected;
both to and from assembled methods will attempt to use System.arraycopy when possible for normalized extent fields, when possible. If a type conversion is required or the field was denormalized, this is no longer possible;
the POJO classes implement the Serializable interface as a marker but it can be easily removed or changed to Externalizable. This case requires additional work for generating the read/write methods.

#37 Updated by Ovidiu Maxiniuc almost 2 years ago

Constantin,
I know you worked with JSON probably more than any of us. Do you have an API which serialize a POJOs and DMOs as JSON strings? I would like to run some extensive testing here and I think JSON output is a fast way to see differences between objects (using meld eventually) with a minimum of additional code.
Thank you!

#38 Updated by Constantin Asofiei almost 2 years ago

In 3821c/14082 I've added support for Java enums and, if there is no FWD serializer registered for the given type, default to jackson. From your example, a fully initialized tm-all pojo looks like this:

{
    "response": {
        "response": {
            "f1": "abcdef",
            "e1": [
                "a",
                "b",
                "c",
                "d" 
            ],
            "f2": 1658133922731,
            "e2": [
                1689669922731,
                1658169922731
            ],
            "f3": 1234.567,
            "e3": [],
            "f4": 987654,
            "e4": [
                1,
                2,
                3,
                4
            ],
            "f5": 123456,
            "f6": false,
            "f7": 1658133922731,
            "f8": 1658133922731,
            "f9": "enhjdmJubQ==",
            "f10": "asdfgh",
            "f11": "NTY3ODkw",
            "f12": 987654
        }
    }
}

The only difference I see from FWD and jackson is that GregorianCalendar is not serialized as ISO-8601, but as its timestamp. I don't know what happens with the timezone, I didn't test this.

Ovidiu, regarding your question about JSON serialization:

for DMOs, FWD has the RecordSerializer - this serializes the DMO as a Json object, with each property serialized using its FWD serializer (BaseDataTypeSerializer)
for types which don't have a serializer registered in FWD, this defaults to jackson - the code is in ObjectSerializer - this uses ObjectMapper to read and write the JSON; keep in mind that the entire object is serialized using jackson, FWD serializers will not be used.
- ObjectMapper.readValue(jsonString, type); to create an instance
- ObjectMapper.valueToTree(object); to create a JSON node structure.

You can use ObjectMapper standalone to experiment.

#39 Updated by Ovidiu Maxiniuc almost 2 years ago

The documentation can be found at:
https://proj.goldencode.com/projects/timco/wiki/POJOs_-_Tutorial#section-8

I could not find a better place to store it, at this moment.

I also committed revision 14089. It primarily contains a new method associate() in RB which allows a DMO 'rogue' object created using Record.fromPOJO to be taken into the FWD's Session management and be saved. And a few fixes.

Constantin,
I realized that POJO and DMO are not fully compatible when serialized as POJOs. Generally we cannot serialize a DMO and deserialize it as POJO because each has different naming preferences: the DMO will use the legacy name while the POJO will use the property names for the element/attributes. More than that, the serialization of DMO-s does not handle well the denormalized fields (because all properties are mapped to same legacy name).

The best/only solution here is to convert the DMOs to POJOs and then serialize, following at the other end to do the operations is reverse order. From my tests, I could not find any issue in conversion between DMOs and POJOs (both ways).

#40 Updated by Greg Shah almost 2 years ago

I took a quick look at the docs. It looks good. Please move it to here:

Using POJOs for Database Entities

Eric: Please review.

Generally we cannot serialize a DMO and deserialize it as POJO because each has different naming preferences: the DMO will use the legacy name while the POJO will use the property names for the element/attributes.

This is a reasonable restriction. Let's just make sure the limitation is clearly documented.

Is there anything else needed for this task? Do we need a more specific example or some other edits to the customer-specific guide?

#41 Updated by Greg Shah almost 2 years ago

I made some minor edits to the doc. Please review.

#42 Updated by Ovidiu Maxiniuc almost 2 years ago

Status changed from WIP to Review

Greg Shah wrote:

I made some minor edits to the doc. Please review.

Thank you for your contribution. I moved the page to new location, removed the link for its old parent document, but kept a redirection message instead of deleting the whole document.

#43 Updated by Ovidiu Maxiniuc almost 2 years ago

Greg Shah wrote:

This is a reasonable restriction. Let's just make sure the limitation is clearly documented.

I added a final section Note on JSON Compatibility between DMOs and POJOs at the end of the document. However, I am positive we can switch the whole protocol to use properties as field names and intermediary switch to data types (for date, datetime and datetime-tz), but we will have to use a separate method than the one we use for 'native' JSON/XML serialization which has to keep compatibility with ABL.

Is there anything else needed for this task? Do we need a more specific example or some other edits to the customer-specific guide?

If there are specific cases in customer code which pose difficulties I am glad to help, but this will probably need to happen in a private task.

Also available in: Atom PDF

Project

General

Profile

FWD » Core Development » Database

Issues

Custom queries

Feature #6584

implement schema hint which generates a POJO in addition to a DMO interface

History

#1 Updated by Eric Faulhaber almost 2 years ago

#2 Updated by Ovidiu Maxiniuc almost 2 years ago

#3 Updated by Greg Shah almost 2 years ago

#4 Updated by Eric Faulhaber almost 2 years ago

#6 Updated by Ovidiu Maxiniuc almost 2 years ago

#7 Updated by Eric Faulhaber almost 2 years ago

#8 Updated by Ovidiu Maxiniuc almost 2 years ago

#9 Updated by Constantin Asofiei almost 2 years ago

#10 Updated by Ovidiu Maxiniuc almost 2 years ago

#11 Updated by Constantin Asofiei almost 2 years ago

#12 Updated by Eric Faulhaber almost 2 years ago

#13 Updated by Constantin Asofiei almost 2 years ago

#14 Updated by Eric Faulhaber almost 2 years ago

#15 Updated by Ovidiu Maxiniuc almost 2 years ago

#16 Updated by Greg Shah almost 2 years ago

#17 Updated by Ovidiu Maxiniuc almost 2 years ago

#18 Updated by Ovidiu Maxiniuc almost 2 years ago

#19 Updated by Greg Shah almost 2 years ago

#20 Updated by Ovidiu Maxiniuc almost 2 years ago

#21 Updated by Eric Faulhaber almost 2 years ago

#22 Updated by Eric Faulhaber almost 2 years ago

#23 Updated by Eric Faulhaber almost 2 years ago

#24 Updated by Greg Shah almost 2 years ago

#25 Updated by Ovidiu Maxiniuc almost 2 years ago

#26 Updated by Constantin Asofiei almost 2 years ago

#27 Updated by Ovidiu Maxiniuc almost 2 years ago

#28 Updated by Constantin Asofiei almost 2 years ago

#29 Updated by Ovidiu Maxiniuc almost 2 years ago

#30 Updated by Eric Faulhaber almost 2 years ago

#31 Updated by Eric Faulhaber almost 2 years ago

#32 Updated by Eric Faulhaber almost 2 years ago

#33 Updated by Eric Faulhaber almost 2 years ago

#34 Updated by Ovidiu Maxiniuc almost 2 years ago

#35 Updated by Ovidiu Maxiniuc almost 2 years ago

#36 Updated by Ovidiu Maxiniuc almost 2 years ago

#37 Updated by Ovidiu Maxiniuc almost 2 years ago

#38 Updated by Constantin Asofiei almost 2 years ago

#39 Updated by Ovidiu Maxiniuc almost 2 years ago

#40 Updated by Greg Shah almost 2 years ago

#41 Updated by Greg Shah almost 2 years ago

#42 Updated by Ovidiu Maxiniuc almost 2 years ago

#43 Updated by Ovidiu Maxiniuc almost 2 years ago