DmoMeta.java

/*
** Module   : DmoMeta.java
** Abstract : Stores information (both legacy and SQL) on a DMO.
**
** Copyright (c) 2019-2025, Golden Code Development Corporation.
**
** -#- -I- --Date-- ---------------------------------------Description---------------------------------------
** 001 OM  20191015 First revision.
** 002 ECF 20200916 Added schema name.
**     OM  20200919 Improved access to dmo metadata.
**     CA  20200922 Added propsByGetter.
**     OM  20209024 Fixed getter names in getPropsByGetterMap(). Small optimization.
**     OM  20200924 Index components carry multiple information to avoid map lookups for them.
**     CA  20200927 Use IdentityHashMap instead of plain map when the key is a Class.
**     OM  20201001 Improved DMO manipulation performance by caching slow Property annotation access.
**     OM  20201002 Use DmoMeta cached information instead of map lookups.
**     OM  20201012 Moved here delegated methods from manager.
**     AIL 20201125 Added DMO signature computation in constructor.
**         20201126 Added unique constraints to DMO signature.
**         20201203 Added helper methods.
**         20201210 Changes signature from string to specialized class.
**     OM  20201120 Added NO-UNDO attribute support.
**     OM  20201202 Added getPrimaryIndex() method.
**     IAS 20201210 Added support for word tables.
**     IAS 20210219 Word tables support for custom extents
**     IAS 29210401 Added 'isVST' and 'isMeta' methods
**     CA  20210508 Added a signature for the DMO schema, used by shared buffer/temp-table match.
**     ECF 20210511 Normalize field name (right-trim and lowercase) before processing in byLegacyName.
**     ECF 20210910 Minor cleanup.
**     ECF 20211202 Manage a shared map of P2JField descriptors which are needed to define dynamic temp-tables
**                  modeled after the DMO described by this DMO metadata, rather than recreating the
**                  descriptors for every request to build such a dynamic temp-table.
**     OM  20211020 Added _DATASOURCE_ROWID property.
**     OM  20220301 Fixed initialization of fields with [INITIAL ?] option.
**     OM  20220303 Better implementation of initial value of fields.
**     OM  20220707 Shared meta-knowledge about fields and properties of a record was collected in
**                  DmoMeta to avoid duplicating their collection and storage.
**     OM  20220727 FieldId and PropertyId are different for denormalized extent fields. Renamed some fields
**                  to a better suited name.
**     TJD 20220504 Upgrade do Java 11 minor changes.
**     CA  20221114 Added API to get the denormalized properties for a DMO extent property name.
**     ECF 20221207 Changed heuristic to extract schema name from full DMO interface name.
**     OM  20221220 Added getFieldCount() method.
**     OM  20220706 The objects returned by getDatabaseIndexes() carry the legacy names, too.
**     OM  20220914 Use MAX-WIDTH to generate custom sized varchar columns.
**     IAS 20221018 Fixed processing of 'now' and 'today' INITIAL attribute value.
**     SVL 20230110 P2JIndex.components() provides direct access to the components array in order to improve
**                  performance.
**     SVL 20230113 Improved performance by replacing some "for-each" loops with indexed "for" loops.
**     HC  20230118 Eliminated some of the uses of String.toUpperCase and/or String.toLowerCase
**                  for performance.
**     CA  20221114 Added API to get the denormalized properties for a DMO extent property name.
**     ECF 20221207 Changed heuristic to extract schema name from full DMO interface name.
** 003 AL2 20230120 Compute if the underlying DMO requires extra computed columns.
** 004 AL2 20230317 Extents and date-time extra column are not computed columns; it is safe to exclude them.
                    Avoid lazy initializing hasComputedColumns to avoid concurrency issues.
** 005 GBB 20230512 Logging methods replaced by CentralLogger/ConversionStatus.
** 006 AL2 20230512 Use CaseInsensitiveHashMap instead of IdentityHashMap for byLegacyName (#7336).
**     AL2 20230516 Cache the number of datetime-tz fields.  
**     AL2 20230517 Cache the number of SQL fields instead, independently from the field type. 
** 007 SVL 20230516 Support for more granular configuration of dirty sharing functionality.
** 008 DDF 20230627 Replaced static variables of DirtyShareSupport with static method calls.
** 009 AD  20231123 Added array of properties ordered by their original property id.
**     AD  20231127 Changed array creation in computeExistingFields() to have exact capacity from the start.
** 010 AL2 20231211 Check if the DMO has non-expanded extents.
** 011 OM  20231218 Made [propsByLegacy] immutable and initialized it in c'tor.
** 012 CA  20240318 INITIAL value for a DATETIME(-TZ) field will always use 'mdy' as date format.
** 013 OM  20240328 Fixed serialize name in getTableDefinition().
** 014 OM  20240510 Added auto detection for date format, depending on the context and a quick heuristic
**                  analysis of the text.
** 015 OM  20240524 Added initialString attribute. Refined algorithm from H014. It also caused regressions.
** 016 TJD 20240123 Java 17 compatibility updates
** 017 SP  20240531 Fix for getWordTables() to set extentTableName only for normalized fields.
** 018 AD  20240313 Changed constructor access modifier to public.
** 019 AD  20240321 Created static getter for the properties list set in the constructor.
** 020 AP  20240926 Remove the processing of Relation annotation in order to rely only on runtime query
**                  processing.
** 021 TJD 20240408 Renamed isEnabledCrossSection to isEnabledCrossSession
** 022 DDF 20240205 Added an additional parameter to addIndexComponent which represents 
**                  the non-null property value of the component.
** 023 OM  20250326 Replaced in isVST() with getMetaTable() method, for better flexibility and performance.
** 024 AS  20250529 Changed the access modifier for getExistingFieldsMap() from private to public.
**                  Return an immutable map in getExistingFieldsMap.
*/

/*
** This program is free software: you can redistribute it and/or modify
** it under the terms of the GNU Affero General Public License as
** published by the Free Software Foundation, either version 3 of the
** License, or (at your option) any later version.
**
** This program is distributed in the hope that it will be useful,
** but WITHOUT ANY WARRANTY; without even the implied warranty of
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
** GNU Affero General Public License for more details.
**
** You may find a copy of the GNU Affero GPL version 3 at the following
** location: https://www.gnu.org/licenses/agpl-3.0.en.html
**
** Additional terms under GNU Affero GPL version 3 section 7:
**
**   Under Section 7 of the GNU Affero GPL version 3, the following additional
**   terms apply to the works covered under the License.  These additional terms
**   are non-permissive additional terms allowed under Section 7 of the GNU
**   Affero GPL version 3 and may not be removed by you.
**
**   0. Attribution Requirement.
**
**     You must preserve all legal notices or author attributions in the covered
**     work or Appropriate Legal Notices displayed by works containing the covered
**     work.  You may not remove from the covered work any author or developer
**     credit already included within the covered work.
**
**   1. No License To Use Trademarks.
**
**     This license does not grant any license or rights to use the trademarks
**     Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
**     of Golden Code Development Corporation. You are not authorized to use the
**     name Golden Code, FWD, or the names of any author or contributor, for
**     publicity purposes without written authorization.
**
**   2. No Misrepresentation of Affiliation.
**
**     You may not represent yourself as Golden Code Development Corporation or FWD.
**
**     You may not represent yourself for publicity purposes as associated with
**     Golden Code Development Corporation, FWD, or any author or contributor to
**     the covered work, without written authorization.
**
**   3. No Misrepresentation of Source or Origin.
**
**     You may not represent the covered work as solely your work.  All modified
**     versions of the covered work must be marked in a reasonable way to make it
**     clear that the modified work is not originating from Golden Code Development
**     Corporation or FWD.  All modified versions must contain the notices of
**     attribution required in this license.
*/

package com.goldencode.p2j.persist.orm;

import com.goldencode.p2j.convert.*;
import com.goldencode.p2j.persist.*;
import com.goldencode.p2j.persist.Record;
import com.goldencode.p2j.persist.annotation.*;
import com.goldencode.p2j.persist.annotation.Table;
import com.goldencode.p2j.persist.dirty.*;
import com.goldencode.p2j.persist.meta.*;
import com.goldencode.p2j.persist.orm.types.TypeManager;
import com.goldencode.p2j.util.*;
import com.goldencode.p2j.util.logging.*;
import com.goldencode.util.CaseInsensitiveHashMap;
import com.google.common.collect.*;
import org.apache.commons.lang3.tuple.*;
import java.lang.reflect.*;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
import java.util.logging.*;
import java.util.stream.*;

/**
 * An immutable structure that contains the annotation information on a table and its fields.
 * The data is accessed directly from the annotation objects in order to avoid duplicating the
 * information locally.
 */
public class DmoMeta
{
   /** Logger */
   private static final CentralLogger log = CentralLogger.get(DmoMeta.class.getName());
   
   /** The id generator. It guarantees that the {@code id} of each {@code DmoMeta} is unique. */
   private static final AtomicInteger idSeed = new AtomicInteger(0);
   
   /** The annotation for the table. */
   protected final Table table;
   
   /** The record's runtime-meta data. */
   protected RecordMeta recordMeta = null;
   
   /** The set of annotations for each declared property, mapped by its FQL property name. */
   protected Map<String, Property> propsByName = new LinkedHashMap<>();
   
   /**
    * The annotation to for the properties corresponding to custom extents. If a property was denormalized, it
    * will not be part of this list.
    */
   protected Map<String, List<Property>> customExtents = new LinkedHashMap<>();
   
   /** The set of annotations for each declared property, mapped by its legacy name. */
   protected final Map<String, Property> propsByLegacy;
   
   /** The list of indexes of this table. */
   protected List<Index> indexes = new ArrayList<>();
   
   /** The interface of this DMO. */
   protected final Class<? extends DataModelObject> iface;
   
   /** The interface that will be implemented by {@code DmoClass} es. */
   protected final Class<? extends DataModelObject> dmoImplInterface;
   
   /** The implementation class of this DMO. */
   protected Class<? extends Record> implClass = null;
   
   /** The map of DMO property names to field descriptors for this DMO metadata. */
   private volatile Map<String, P2JField> existingFieldsMap = null;
   
   /** The list of field descriptors for this DMO properties, ordered by their id */
   private volatile List<Property> existingFieldPropertiesList = null;
   
   /** The cached list of original fields / properties (lazy initialized in {@code getFields}). */
   private List<Property> originalProperties = null;
   
   /** The cached list of indexes (lazy initialized in {@code getDatabaseIndexes()}). */
   private List<P2JIndex> databaseIndexes = null;
   
   /** The set of properties mapped by their setters. */
   private Map<Method, PropertyMeta> propsBySetter = null;
   
   /** The set of properties mapped by their getters. */
   private Map<Method, PropertyMeta> propsByGetter = null;
   
   /** The cached list of sets of properties participating in UNIQUES indexes. */
   private List<Set<String>> uniqueConstraints = null;
   
   /** The cached list of lists of whole property instances participating in UNIQUES indexes. */
   private ArrayList<List<Property>> uniqueProps = null;
   
   /** The cached list of mandatory fields. */
   private List<Property> mandatoryFields = null;
   
   /** The set of indexed, character columns.  The value is if they are case-sensitive or not. */
   private Map<String, Boolean> computedIndexedProperties = null;
   
   /** 
    * Map of all foreign relation descriptors, indexed by foreign class.
    */
   private final Map<Class<? extends DataModelObject>, RelationInfo> relations = new ConcurrentHashMap<>();
   
   /** Map of all foreign relation descriptors, indexed by local class */
   private final Map<Class<?>, RelationInfo> inverseRelations = new ConcurrentHashMap<>();
   
   /** Flags DMOs of temporary tables. */
   public final boolean tempTable;
   
   /** Flags DMOs of multi-tenant private tables (include both with-default and no-default cases). */
   public final boolean multiTenant;
   
   /** Flags DMOs of multi-tenant private tables with default. Only valid if {@code multiTenant == true}*/
   public final boolean multiTenantWithDefault;
   
   /** Schema name associated with this DMO */
   public final String schema;
   
   /** Caches the {@code name} attribute (SQL table name) from {@link Table} annotation. */
   public final String sqlTable;
   
   /** Caches the interned {@code legacy} attribute from {@link Table} annotation. */
   public final String legacyTable;
   
   /** Caches the interned normalized {@code legacy} attribute from {@link Table} annotation. */
   public final String normalizedLegacyTable;
   
   /** Caches the {@code afterTable} attribute from {@link Table} annotation. */
   public final String afterTable;
   
   /** Caches the {@code beforeTable} attribute from {@link Table} annotation. */
   public final String beforeTable;
   
   /** Caches the {@code category} attribute from {@link Table} annotation. Unused at this time. */
   public final String category;
   
   /**
    * The {@link MetadataManager.SystemTable}, in case this object hold information on a meta table,
    * otherwise {@code null}.
    */
   public final MetadataManager.SystemTable metaTable;
   
   /** Caches the {@code dirtyRead} attribute from {@link Table} annotation. */
   public final boolean dirtyRead;

   /** Caches the {@code dirtyIntraRead} attribute from {@link Table} annotation. */
   public final boolean dirtyIntraRead;
   
   /** Caches the {@code hasValidation} attribute from {@link Table} annotation. */
   public final boolean hasValidation;
   
   /** Caches the {@code isLikeSequential} attribute from {@link Table} annotation. */
   public final boolean isLikeSequential;
   
   /** Caches the {@code serializeName} attribute from {@link Table} annotation. */
   public final String serializeName;
   
   /** Caches the {@code xmlNodeName} attribute from {@link Table} annotation. */
   public final String xmlNodeName;
   
   /** Caches the {@code namespaceUri} attribute from {@link Table} annotation. */
   public final String namespaceUri;
   
   /** Caches the {@code namespacePrefix} attribute from {@link Table} annotation. */
   public final String namespacePrefix;
   
   /** A signature containing information about the DMO structure and field constraints. */
   private final DmoSignature signature;
   
   /** Caches the {@code NO-UNDO} attribute from {@link Table} annotation (TEMP-TABLE specific). */
   public final boolean noUndo;
   
   /** The set of properties and their metadata, indexed by their in-memory positions. */
   private Pair<Property, PropertyMeta>[] properties = null;
   
   /** Number of physical database fields. This may be different than the number of properties. */
   private final int sqlFieldCount;
   
   /** The unique id of this DMO. */
   private final int id = idSeed.incrementAndGet();
   
   /**
    * Collects all properties of the record whose fields are {@code Progress.Lang.Object}-typed. Only makes
    * sense for temp tables. When empty, the OO support (ref-counting) is disabled.
    */
   private Set<String> ooFields = null;
   
   /** {@code UnmodifiableMap} of property names to getter methods for simple POJO fields only. */
   private Map<String, Method> pojoGetterMap = null;
   
   /** {@code UnmodifiableMap}  of property names to setter methods for simple POJO fields only. */
   private Map<String, Method> pojoSetterMap = null;
   
   /** Map of property names to getter methods for legacy fields only */
   private Map<String, Method> legacyGetterMap = null;
   
   /** Map of property names to setter methods for legacy fields only */
   private Map<String, Method> legacySetterMap = null;
   
   /** Map of property names to extents for DMO array properties */
   private Map<String, Integer> extentMap = null;
   
   /** Check if this DMO is going to have computed columns in the underlying database representation */
   private final Boolean[] hasComputedColumns = new Boolean[2];
   
   /** {@code true} if this DMO has normalized extents */
   private boolean nonExpandedExtents;
   
   /** The map used in type-widening operations. */
   private static final Map<Class<? extends BaseDataType>, Class<? extends BaseDataType>> toWider = 
         new IdentityHashMap<>();
   
   static
   {
//      toWider.put(blob.class      , blob.class);
      toWider.put(character.class , Text.class);
//      toWider.put(clob.class      , clob.class);
//      toWider.put(comhandle.class , comhandle.class);
      toWider.put(datetimetz.class, date.class);
      toWider.put(datetime.class  , date.class);
//      toWider.put(date.class      , date.class);
      toWider.put(decimal.class   , NumberType.class);
//      toWider.put(handle.class    , handle.class);
      toWider.put(int64.class     , NumberType.class);
      toWider.put(integer.class   , NumberType.class);
//      toWider.put(logical.class   , logical.class);
//      toWider.put(object.class    , object.class);
      toWider.put(raw.class       , BinaryData.class);
      toWider.put(recid.class     , NumberType.class);
//      toWider.put(rowid.class     , rowid.class);
   }
   
   /**
    * The constructor initializes the data structure.
    *
    * @param   dmoImplInterface
    *          The interface that will be implemented by {@code DmoClass} es.
    * @param   iface
    *          The interface that contains the field declarations and the DMO annotations. It is a
    *          parent of {@code dmoImplInterface} or {@code dmoImplInterface} itself. Its (getter)
    *          methods are iterated in order to extract the annotations for the fields.
    * @param   table
    *          The {@code Table} annotation of {@code iface}. This is the same value from
    *          {@code iface.getAnnotation(Table.class)}.
    */
   public DmoMeta(Class<? extends DataModelObject> dmoImplInterface,
           Class<? extends DataModelObject> iface,
           Table table)
   {
      this.table = table;
      this.sqlTable = table.name().intern();
      this.legacyTable = table.legacy().intern();
      this.normalizedLegacyTable = this.legacyTable.toLowerCase().intern();
      this.dirtyRead = table.dirtyRead();
      this.dirtyIntraRead = table.dirtyIntraRead();
      this.afterTable = table.afterTable();
      this.beforeTable = table.beforeTable();
      this.category = table.category();
      this.hasValidation = table.hasValidation();
      this.isLikeSequential = table.isLikeSequential();
      this.serializeName = table.serializeName();
      this.xmlNodeName = table.xmlNodeName();
      this.namespaceUri = table.namespaceUri();
      this.namespacePrefix = table.namespacePrefix();
      this.noUndo = table.noUndo();
      this.metaTable = MetadataManager.byLegacyName(this.legacyTable);
      
      String mtType = table.tenantType();
      this.multiTenant = mtType.equals("multi-tenant") || mtType.equals("multi-tenant, with-default");
      this.multiTenantWithDefault = mtType.equals("multi-tenant, with-default");
      
      this.iface = iface;
      this.dmoImplInterface = dmoImplInterface;
      this.tempTable = Temporary.class.isAssignableFrom(iface);
      this.schema = getSchemaByInterface(iface, tempTable);
      
      // iterate [Indices] annotation and save its content 
      Indices indices = iface.getAnnotation(Indices.class);
      if (indices != null)
      {
         indexes = Arrays.asList(indices.value());
      }
      
      List<Property> properties = getPropertiesList(iface, tempTable);
      
      // sort properties according to their [id] field. This way, the [fields] [LinkedHashMap]
      // will iterate them in the right order (this is because the [getMethods()] returned
      // the list of methods in a chaotic order) 
      properties.sort(Comparator.comparingInt(property -> property.propId));
      
      // now put them in the [fields] member, mapped by name
      // at the same time populate the immutable [propsByLegacy] map
      int fieldsCount = 0;
      Map<String, Property> tmpMap = new CaseInsensitiveHashMap<>(true);
      for (Property property : properties)
      {
         propsByName.put(property.name, property);
         if (property.original != null)
         {
            customExtents.computeIfAbsent(property.original, n -> new ArrayList<>()).add(property);
         }
         if (property.propId > 0) // skipping reserved properties
         {
            tmpMap.put(property.legacyLower, property);
         }
         
         boolean normalized = property.extent > 0 && property.index == 0;
         if (!normalized)
         {
            Class<?> fwdType = property._fwdType;
            if (BaseDataType.class.isAssignableFrom(fwdType))
            {
               fieldsCount += TypeManager.getDataHandler((Class<? extends BaseDataType>) fwdType).propertySize();
            }
            else
            {
               // is this a good fall-back?
               fieldsCount++;
            }
         }
         else
         {
            this.nonExpandedExtents = true;
         }
      }
      sqlFieldCount = fieldsCount;
      this.propsByLegacy = Collections.unmodifiableMap(tmpMap);
      
      // build a String signature suitable for identifying if two DMOs are compatible in certain 
      // operations (for example copy-temp-table).
      signature = DMOSignatureHelper.buildSignature(this);
      
      // the eventual relations cannot be created at this moment, they will be processed at a
      // later time, when the respective DMO interfaces will be loaded.
   }

   /**
    * Static getter of properties for a DMO interface.
    * @param   iface
    *          The class representing a DMO interface.
    * @param   tempTable
    *          True if this DMO represents a temp table. False otherwise.
    *          
    * @return  All the properties of the given DMO interface, plus reserved properties if needed.
    */
   public static List<Property> getPropertiesList(Class<? extends DataModelObject> iface, boolean tempTable)
   {
      // iterate all methods to identity the fields of the table. They are the getters which
      // are annotated with [Property]
      Method[] allMethods = iface.getMethods();
      List<Property> properties = new ArrayList<>(allMethods.length / 2 + 5);
      
      for (Method method : allMethods)
      {
         com.goldencode.p2j.persist.annotation.Property ann = method.getAnnotation(
               com.goldencode.p2j.persist.annotation.Property.class);
         if (ann != null && ann.id() > 0)
         {
            properties.add(new Property(ann, method));
         }
      }
      
      // add reserved properties:
      properties.add(ReservedProperty.ID); // for all tables
      
      if (tempTable)
      {
         // for all temp-tables:
         properties.add(ReservedProperty._MULTIPLEX);
         
         // these are available even if there is no before-table option for this temp-table.
         properties.add(ReservedProperty._ERRORFLAG);
         properties.add(ReservedProperty._ORIGINROWID);
         properties.add(ReservedProperty._DATASOURCEROWID);
         properties.add(ReservedProperty._ERRORSTRING);
         properties.add(ReservedProperty._PEERROWID);
         properties.add(ReservedProperty._ROWSTATE);
      }
      return properties;
   }
   
   /**
    * Return the schema name associated with the given DMO interface.
    *
    * @param   dmoIface
    *          A DMO interface; if a temporary DMO, the temp table schema name is returned.
    * @param   temporary
    *          {@code true} if {@code dmoIface} represents a legacy temp-table, else {@code false}.
    *
    * @return  Schema name for the given interface.
    */
   private static String getSchemaByInterface(Class<? extends DataModelObject> dmoIface, boolean temporary)
   {
      if (temporary)
      {
         return DatabaseManager.TEMP_TABLE_SCHEMA;
      }
      
      String schema = null;
      
      // heuristically attempt to find schema using the package name
      String[] nameTokens = dmoIface.getName().split("\\.");
      int len = nameTokens.length;
      if (len > 1)
      {
         schema = nameTokens[len - 2];
      }
      
      return schema;
   }
   
   /**
    * Obtain the interface that contains the annotations.
    * 
    * @return  the interface that contains the annotations.
    */
   public Class<? extends DataModelObject> getAnnotatedInterface()
   {
      return iface;
   }
   
   /**
    * Obtain the class that implements the DMO interface.
    * 
    * @return  the class that implements the DMO interface.
    */
   public Class<? extends Record> getImplementationClass()
   {
      return implClass;
   }
   
   /**
    * Obtain the implemented DMO interface.
    * 
    * @return  the implemented DMO interface.
    */
   public Class<? extends DataModelObject> getDmoImplInterface()
   {
      return dmoImplInterface;
   }
   
   /**
    * Obtain the schema name for the DMO represented by this object.
    *
    * @return  Schema name.
    */
   public String getSchema()
   {
      return schema;
   }
   
   /**
    * Obtain the unique {@code id} of this instance. The id is unique among all databases. The {@code id} is
    * usually not the same for multiple runs because the {@code id}s are assigned in the order the tables are
    * processed.
    * 
    * @return  the unique id of this instance.
    */
   public int getId()
   {
      return id;
   }
   
   /**
    * Get the {@code RecordMeta} holding the runtime meta information of this table.
    * 
    * @return  the {@code RecordMeta} holding the runtime meta information of this table.
    */
   public RecordMeta getRecordMeta()
   {
      return recordMeta;
   }
   
   /**
    * Obtain the name of the SQL table for this DMO. For temporary tables representing legacy
    * temp-tables, this name represents the base name. When created, a suffix which is unique
    * at runtime is added to this base name.
    *
    * @return  the name of the SQL table for this DMO.
    */
   public String getSqlTableName()
   {
      return sqlTable;
   }
   
   /**
    * Test whether the DMO belongs to a temp table.
    * 
    * @return  {@code true} if the DMO belongs to a temp table.
    */
   public boolean isTempTable()
   {
      return tempTable;
   }
   
   /**
    * Test whether the table of this DMO was marked as "dirty read", that is, if the DMO should
    * share its dirty data.
    * 
    * @return  {@code true} if the table needs to be tracked by the dirty share manager and was 
    *          hinted at conversion time with {@code dirty-read} XML attribute.
    */
   public boolean isDirtyRead()
   {
      return DirtyShareSupport.isEnabledCrossSession() && 
             (DirtyShareSupport.isForceCrossSession() || dirtyRead) ||
             DirtyShareSupport.isEnabledIntraSession() && 
             (DirtyShareSupport.isForceIntraSession() || dirtyIntraRead);
   }
   
   /**
    * Obtain meta information about a field specified by its property name.
    * 
    * @param   propName
    *          The field's property name.
    *
    * @return  Meta information about {@code propName} field if it exists, or {@code null} otherwise.
    */
   public Property getFieldInfo(String propName)
   {
      return propsByName.get(propName);
   }
   
   /**
    * Get the list of denormalized properties associated with this extent DMO property.
    * 
    * @param    propName
    *           The field's property name.
    *           
    * @return   The list of {@link #customExtents denormalized} properties, or <code>null</code> if this is
    *           not a denormalized or not a extent property.
    */
   public List<Property> getCustomExtentFieldInfo(String propName)
   {
      return customExtents.get(propName);
   }
   
   /**
    * Get the number of properties in this DMO.
    * 
    * @return   The size of {@link #propsByName}.
    */
   public int getFieldSize()
   {
      return propsByName.size();
   }
   
   /**
    * Looks up a {@code Property} by its legacy field name.
    * 
    * @param   legacyField
    *          The name of the legacy field.
    *
    * @return  The {@code Property} whose legacy name was passed as parameter.
    */
   public Property byLegacyName(String legacyField)
   {
      return propsByLegacy.get(legacyField);
   }
   
   /**
    * Obtain the list of properties for this DMO.
    * 
    * @param   extra
    *          If {@code true} the surrogate/hidden  properties ({@code id}, {@code _multiplex})
     *         are also added, otherwise only the original fields are returned.
    * 
    * @return  An iterator for list of properties for this DMO.
    */
   public Iterator<Property> getFields(boolean extra)
   {
      if (extra)
      {
         return propsByName.values().iterator();
      }
      
      if (originalProperties == null)
      {
         ArrayList<Property> buildList = new ArrayList<>(propsByName.size());
         Set<Map.Entry<String, Property>> entries = propsByName.entrySet();
         for (Map.Entry<String, Property> entry : entries)
         {
            Property prop = entry.getValue();
            if (prop.propId >= 0)
            {
               buildList.add(prop);
            }
         }
         
         buildList.trimToSize();
         originalProperties = Collections.unmodifiableList(buildList);
      }
      
      return originalProperties.iterator();
   }
   
   /**
    * Obtain the number of fields in the table.
    * 
    * @param   extra
    *          If true, the "hidden" fields are taken into account. Use {@code false} to get the number of
    *          fields in the original table definition.
    * 
    * @return  the number of fields in the table, as described above.
    */
   public int getFieldCount(boolean extra)
   {
      if (extra)
      {
         return propsByName.size();
      }
      
      getFields(false); // triggers setup of [originalProperties]
      return originalProperties.size();
   }
   
   /**
    * Get the descriptor of a field corresponding with the given DMO property name.
    * 
    * @param   name
    *          DMO property name for the desired field descriptor.
    * 
    * @return  Field descriptor.
    */
   public P2JField getExistingField(String name)
   {
      return getExistingFieldsMap().get(name);
   }
   
   /**
    * Get the descriptor of a field corresponding with the given DMO field id.
    * 
    * @param   fieldId
    *          DMO property field id for the desired field descriptor.
    * 
    * @return  Field descriptor.
    */
   public Property getExistingFieldProperty(int fieldId)
   {
      return getExistingFieldsList().get(fieldId - 1);
   }
   
   /**
    * Get the field descriptors for the table associated with this metadata object.
    *
    * @return  Collection of the fields of the target table.
    */
   public Collection<P2JField> getExistingFields()
   {
      return getExistingFieldsMap().values();
   }
   
   /**
    * Determine whether the given DMO property represents the leading component of any index on this table.
    *
    * @param   property
    *          DMO property to be tested.
    *
    * @return  {@code true} if {@code property} is a leading index component; else {@code false}.
    */
   public boolean isLeadingIndexComponent(String property)
   {
      Iterator<P2JIndex> ixdIt = getDatabaseIndexes();
      
      if (ixdIt == null)
      {
         return false;
      }
      
      while (ixdIt.hasNext())
      {
         ArrayList<P2JIndexComponent> comps = ixdIt.next().components();
         for (int i = 0; i < comps.size(); i++)
         {
            P2JIndexComponent leadingProp = comps.get(i);
            if (property.equals(leadingProp.getPropertyName()))
            {
               return true;
            }
         }
      }
      
      return false;
   }
   
   /**
    * Return the number of indices for this table (declared in DMO annotations). Depending on the parameter
    * the number of unique or non-unique indices will be returned.
    * 
    * @param   unique
    *          Use {@code true} to get the number of unique indices and {@code false} to get the number of
    *          non-unique indices. To obtain the number of all indices, the called must add the two results.
    * 
    * @return  the number of indices, unique or non-unique.
    */
   public int getIndexCount(boolean unique)
   {
      return unique ? recordMeta.getUniqueIndexNames().length : recordMeta.getNonuniqueIndexNames().length;
   }
   
   /**
    * Get word tables' names map by the field name
    * 
    * @return  Word tables' names map by the field name.
    */
   public Map<String, WordIndexData> getWordTables()
   {
      Map<String, WordIndexData> map = new HashMap<>();
      for(Index idx: indexes)
      {
         if (!idx.word())
         {
            continue;
         }
         IndexComponent indexComponent = idx.components()[0];
         String fieldName = indexComponent.name();
         String wordtablename = idx.wordtablename();
         String extentTableName = null;
         Property property = propsByName.get(fieldName);
         if (property != null)
         {
            int extent = property.extent;
            extentTableName = (extent == 0 || property.denormalized)? null: sqlTable + "__" + extent;
            fieldName = property.column; 
         }
         else if (indexComponent.extent() > 0) 
         {
            String legacy = indexComponent.legacy();
            property = propsByName.values().stream().filter(
                  f -> legacy.equals(f.legacy)).findFirst().orElse(null);
         }
         if (property == null)
         {
            throw new IllegalStateException("Unknown field: " + fieldName);
         }
         map.put(fieldName, new WordIndexData(wordtablename, extentTableName, property.caseSensitive));
      }
      
      return map;
   }
   
   /**
    * Get word tables' names map by the index name
    * 
    * @return  Word tables' names map by the index name.
    */
   public Map<String, String> getWordTablesByIndexName()
   {
      return indexes.stream()
             .filter(Index::word)
             .collect(Collectors.toMap(Index::name, Index::wordtablename));
   }
   /**
    * Get an iterator on all indexes of the DMO. The resulting iterator returns {@code P2JIndex}
    * objects, each of which defines the columns participating in an index.
    *
    * @return  Iterator on index definition objects.
    */
   public Iterator<P2JIndex> getDatabaseIndexes()
   {
      if (databaseIndexes == null)
      {
         ArrayList<P2JIndex> buildList = new ArrayList<>();
         for (Index index : indexes)
         {
            P2JIndex p2jIndex = new P2JIndex(
                  sqlTable, index.name(), index.unique(), index.primary(), 
                  index.word(), index.wordtablename());
            p2jIndex.setLegacyName(index.legacy());
            for (IndexComponent comp : index.components())
            {
               String compName = comp.name();
               String legacy;
               String column;
               int extent = 0;
               Property property = propsByName.get(compName);
               if (property == null)
               {
                  legacy = comp.legacy();
                  extent = comp.extent();
                  if (extent == 0)
                  {
                     throw new IllegalStateException("Unknown field: " + compName + "/" + legacy);
                  }
                  List<Property> props = propsByName.values().stream().filter(
                           p -> legacy.equals(p.legacy)).collect(Collectors.toList());
                  property = props.get(0);
                  column = comp.name();
               }
               else
               {
                  legacy = property.legacy;
                  column = property.column;
               }
               boolean ignoreCase = property._isCharacter && !property.caseSensitive;
               boolean abbr = false;                                          // TODO: add support
               p2jIndex.addComponent(legacy,
                                     extent,
                                     compName,
                                     column,
                                     comp.descending(),
                                     property._isCharacter,
                                     ignoreCase,
                                     abbr,
                                     property.mandatory);
            }
            buildList.add(p2jIndex);
         }
         
         buildList.trimToSize();
         this.databaseIndexes = Collections.unmodifiableList(buildList);
      }
      
      return databaseIndexes.iterator();
   }
   
   /**
    * Get the primary index. If a PRIMARY index is found it is returned. If no index was declared PRIMARY and
    * {@code implicit == false} the method returns {@code false}. If {@code implicit == true}, the first
    * declared index is returned. If the table does not define any fields, {@code null} is returned.
    *
    * @param   implicit
    *          If {@code true}, if there is no explicit primary index, the first index is returned as being
    *          the <i>implicit</i> primary index. Otherwise, in the event there is no primary index declared,
    *          the method returns {@code null}.
    *
    * @return  The PRIMARY index, the primary IMPLICIT index or {@code null} as described above.
    */
   public P2JIndex getPrimaryIndex(boolean implicit)
   {
      Iterator<P2JIndex> it = getDatabaseIndexes();
      if (!it.hasNext())
      {
         return null; // no indexes defined, whatsoever
      }
      
      while (it.hasNext())
      {
         P2JIndex index = it.next();
         if (index.isPrimary())
         {
            return index; 
         }
      }
      
      return implicit ? this.databaseIndexes.get(0) : null;
   }
   
   /**
    * Get the list of mandatory fields.
    * 
    * @return   The {@link #mandatoryFields}.
    */
   public List<Property> getMandatoryFields()
   {
      if (mandatoryFields == null)
      {
         List<Property> mfl = new ArrayList<>();
         Iterator<Property> iter = getFields(false);
         while (iter.hasNext())
         {
            Property field = iter.next();
            if (field.mandatory)
            {
               mfl.add(field);
            }
         }
         
         this.mandatoryFields = mfl.isEmpty() ? Collections.emptyList() : Collections.unmodifiableList(mfl);
      }
      
      return mandatoryFields;
   }
   
   /**
    * For a given DMO property which may represent an indexed text column, indicate whether that
    * column is set to ignore case.
    * 
    * @param   property
    *          DMO property name.
    * 
    * @return  {@code true} to ignore case, {@code false} to preserve case sensitivity,
    *          {@code null} if the property does not represent an indexed text column.
    */
   public Boolean isIndexedIgnoreCase(String property)
   {
      if (computedIndexedProperties == null)
      {
         Iterator<P2JIndex> iter = getDatabaseIndexes();
         Map<String, Boolean> cip = new HashMap<>();
         
         while (iter.hasNext())
         {
            P2JIndex index = iter.next();
            if (index.isWord())
            {
               continue; // TODO: WORD indices are not yet supported by FWD
            }

            ArrayList<P2JIndexComponent> comps = index.components();
            for (int i = 0; i < comps.size(); i++)
            {
               P2JIndexComponent indexComp = comps.get(i);
               Property field = propsByName.get(indexComp.getPropertyName());
               
               if (field._isCharacter)
               {
                  cip.put(field.name, !field.caseSensitive);
               }
            }
         }
         
         computedIndexedProperties = cip.isEmpty() ? Collections.emptyMap() : Collections.unmodifiableMap(cip);
      }
      
      return computedIndexedProperties.get(property);
   }
   
   /**
    * Obtain the the sets of unique constraints for each DMO. That is, the set of index components
    * for each unique index for this DMO. Uses lazy initialization, the list for each DMO is
    * computed at first access.
    *
    * @return  list of sets of properties participating in UNIQUES indexes.
    */
   public List<Set<String>> getUniqueConstraints()
   {
      if (uniqueConstraints == null)
      {
         List<Set<String>> ucl = new ArrayList<>();
         Iterator<P2JIndex> it = getDatabaseIndexes();
         it.forEachRemaining(idx -> {
            if (idx.isUnique())
            {
               Set<String> idxComps = new HashSet<>();
               ArrayList<P2JIndexComponent> comps = idx.components();
               for (int i = 0; i < comps.size(); i++)
               {
                  P2JIndexComponent comp = comps.get(i);
                  idxComps.add(comp.getPropertyName());
               }
               idxComps = Collections.unmodifiableSet(idxComps);
               ucl.add(idxComps);
            }
         });
         uniqueConstraints = ucl.isEmpty() ? Collections.emptyList() : Collections.unmodifiableList(ucl);
      }
      
      return uniqueConstraints;
   }
   
   /**
    * Obtain the lists of unique constraints for each DMO. That is, the list of index components
    * for each unique index for this DMO. Uses lazy initialization, the list for each DMO is
    * computed at first access.
    *
    * @return  list of lists of whole property instances participating in UNIQUES indexes.
    */
   public ArrayList<List<Property>> getUniqueConstraintsAsProps()
   {
      if (uniqueProps == null)
      {
         uniqueProps = new ArrayList<>();
         List<Set<String>> uniqueIdxs = getUniqueConstraints();
         for (Set<String> idx : uniqueIdxs)
         {
            List<Property> props = new ArrayList<>();
            for (String prop : idx)
            {
               props.add(propsByName.get(prop));
            }
            uniqueProps.add(props);
         }
      }
      
      return uniqueProps;
   }
   
   /**
    * Get the {@code Property} that correspond to the k-th entry in {@code data} table of the class
    * implementing this DMO.
    * 
    * @param   k
    *          The index of the property.
    *
    * @return  the k-th Property.
    */
   public Property getProperty(int k)
   {
      return properties[k].getLeft();
   }
   
   /**
    * Get {@code PropertyMeta} that correspond to the k-th entry in {@code data} table of the class
    * implementing this DMO.
    *
    * @param   k
    *          The index of the property.
    *
    * @return  the k-th Property.
    */
   public PropertyMeta getMeta(int k)
   {
      return properties[k].getRight();
   }
   
   /**
    * Get the relation descriptor for the relation from this object's DMO class to
    * {@code foreign}.
    *
    * @param   foreign
    *          DMO interface for the foreign table.
    *
    * @return  Relation descriptor or {@code null} if this DMO has no foreign key relation with
    *          {@code foreign}.
    */
   public RelationInfo getRelation(Class<?> foreign)
   {
      return relations.get(foreign);
   }
   
   /**
    * Get a map of converted DMO property names corresponding to simple, bean getter methods.
    *
    * @return  Map as described above.
    */
   public Map<String, Method> getPojoGetterMap()
   {
      if (pojoGetterMap == null)
      {
         pojoGetterMap = PropertyHelper.pojoGettersByProperty(iface);
      }
      return pojoGetterMap;
   }
   
   /**
    * Get a map of converted DMO property names corresponding to simple, bean setter methods.
    *
    * @return  Map as described above.
    */
   public Map<String, Method> getPojoSetterMap()
   {
      if (pojoSetterMap == null)
      {
         pojoSetterMap = PropertyHelper.pojoSettersByProperty(iface);
      }
      
      return pojoSetterMap;
   }
   
   /**
    * Obtain the set of fields which hold {@code Progress.Lang.Object} of the record. This only makes sense
    * for temp-tables. When empty, the OO support (ref-counting) is disabled.
    *
    * @return  An unmodifiable set of {@code Progress.Lang.Object} fields of the table.
    */
   public Set<String> getOoFields()
   {
      if (ooFields == null)
      {
         // populate the [ooFields]
         if (tempTable)
         {
            Set<String> coll = new HashSet<>();
            for (Map.Entry<String, Method> pair : getPojoGetterMap().entrySet())
            {
               if (pair.getValue().getReturnType() == object.class)
               {
                  coll.add(pair.getKey());
               }
            }
            ooFields = Collections.unmodifiableSet(coll);
         }
         else
         {
            ooFields = Collections.emptySet();
         }
      }
      
      return ooFields;
   }
   
   /**
    * Get a map of converted DMO property names to corresponding, legacy, getter methods. Legacy methods
    * include simple bean getters for scalar fields, and an indexed getter which takes a simple {@code int}
    * parameter as the subscript value for extent fields. Even if an extent field has been denormalized, the
    * latter is provided for backward compatibility, as some runtime code may not have been converted to use
    * the denormalized methods directly.
    *
    * @return  Map as described above.
    */
   public Map<String, Method> getLegacyGetterMap()
   {
      if (legacyGetterMap == null)
      {
         legacyGetterMap = PropertyHelper.legacyGettersByProperty(iface);
      }
      
      return legacyGetterMap;
   }
   
   /**
    * Get a map of converted DMO property names to corresponding, legacy, setter methods. Legacy methods
    * include simple bean setters for scalar fields, and an indexed setter which takes a simple {@code int}
    * parameter as the subscript value for extent fields. Even if an extent field has been denormalized, the
    * latter is provided for backward compatibility, as some runtime code may not have been converted to use
    * the denormalized methods directly.
    *
    * @return  Map as described above.
    */
   public Map<String, Method> getLegacySetterMap()
   {
      if (legacySetterMap == null)
      {
         legacySetterMap = PropertyHelper.legacySettersByProperty(iface);
      }
      
      return legacySetterMap;
   }
   
   /**
    * Get the map of DMO property names to extents for the DMO type associated with this buffer,
    * initializing it first if necessary.
    *
    * @return  Map of DMO array properties to extents.
    */
   public Map<String, Integer> getExtentMap()
   {
      if (extentMap == null)
      {
         extentMap = PropertyHelper.extentsByProperty(iface);
      }
      
      return extentMap;
   }
   
   /**
    * Use the passed {@code implClass} to extract the property metadata and store locally in an indexed
    * array for fast access. The indexed array contain both the {@code Property} annotation and the
    * {@code PropertyMeta} data paired so they can be accessed using {@code getProperty(int)} and
    * {@code getMeta(int)} methods respectively.
    *
    * @param   implClass
    *          The implementation class to be processed for extraction of property metadata.
    *
    * @throws  IllegalAccessException
    *          when the implClass cannot be accessed via reflection.
    * @throws  NoSuchFieldException
    *          when the implClass cannot be accessed via reflection.
    */
   void setImplClass(Class<? extends Record> implClass)
   throws IllegalAccessException, NoSuchFieldException
   {
      this.implClass = implClass;
      Field metaField = implClass.getDeclaredField("_metadata");
      metaField.setAccessible(true);
      recordMeta = (RecordMeta) metaField.get(null);
      PropertyMeta[] propertyMetas = recordMeta.getPropertyMeta(false);
      
      int len = propertyMetas.length;
      properties = new Pair[len];
      for (int i = 0; i < len; i++)
      {
         PropertyMeta meta = propertyMetas[i];
         properties[i] = Pair.of(propsByName.get(meta.getName()), meta);
      }
   }
   
   /**
    * Add an object which describes a foreign relation from this table to another.
    *
    * @param   foreign
    *          DMO interface for the foreign table.
    * @param   info
    *          Relation descriptor.
    */
   void putRelation(Class<? extends DataModelObject> foreign, RelationInfo info)
   {
      relations.put(foreign, info);
   }
   
   /**
    * Get an iterator on all foreign relation descriptors for the current
    * DMO, which describe relations <em>from</em> this table to another.
    *
    * @return  Iterator on {@link RelationInfo} objects.
    */
   public Iterator<RelationInfo> relations()
   {
      return relations.values().iterator();
   }
   
   /**
    * Get the relation descriptor for the relation to this object's DMO class from {@code local}.
    *
    * @param   local
    *          DMO interface for the local table.
    *
    * @return  Relation descriptor or {@code null} if this DMO has no inverse foreign key relation
    *          with {@code local}.
    */
   RelationInfo getInverseRelation(Class<?> local)
   {
      return inverseRelations.get(local);
   }
   
   /**
    * Add an object which describes a foreign relation to this table from another.
    *
    * @param   local
    *          DMO interface for the local table.
    * @param   info
    *          Relation descriptor.
    */
   void putInverseRelation(Class<?> local, RelationInfo info)
   {
      inverseRelations.put(local, info);
   }
   
   /**
    * Get an iterator on all foreign relation descriptors for the current DMO, which describe
    * relations <em>to</em> this table from another.
    *
    * @return  Iterator on {@link RelationInfo} objects.
    */
   public Iterator<RelationInfo> inverseRelations()
   {
      return inverseRelations.values().iterator();
   }
   
   /**
    * Returns (after eventually has constructed) the map of getters to {@code PropertyMeta} map, used to 
    * fast-access record copy (bypassing the reflection to retrieve a field's value).
    * 
    * @return  the map of getters to {@code PropertyMeta} map.
    */
   public Map<Method, PropertyMeta> getPropsByGetterMap()
   {
      if (propsByGetter == null)
      {
         Map<Method, PropertyMeta> ret = new HashMap<>();
         boolean nonBeforeTable = afterTable.isEmpty();
         for (Pair<Property, PropertyMeta> property : properties)
         {
            PropertyMeta prop = property.getRight();
            
            // the hidden before-table specific fields are accessible only for ... before-tables
            if (nonBeforeTable && prop.getterName.startsWith("get_"))
            {
               // skip these properties (their name start with underscore) for the rest of the tables
               continue;
            }
            
            Class<?>[] sig = (prop.getExtent() == 0) ? new Class[] { } : new Class[] { int.class };
            try
            {
               ret.put(iface.getMethod(prop.getterName, sig), prop);
            }
            catch (NoSuchMethodException e)
            {
               if (log.isLoggable(Level.INFO))
               {
                  log.log(Level.INFO, 
                          "Failed to access getter for property '" + prop.getName() + "'.", e);
               }
            }
            
            Property dmoProp = property.getLeft();
            String oName = dmoProp.original;
            if (!oName.isEmpty() && dmoProp.index == 1)
            {
               // denormalized extent fields, handle only once for each original property (for 1st element)
               
               sig = new Class[] { int.class };
               try
               {
                  String getterName = (prop.getType() == logical.class ? "is" : "get") + 
                                      Character.toUpperCase(oName.charAt(0)) + oName.substring(1);
                  ret.put(iface.getMethod(getterName, sig), prop);
               }
               catch (NoSuchMethodException e)
               {
                  if (log.isLoggable(Level.INFO))
                  {
                     log.log(Level.INFO,
                        "Failed to access original indexed getter for property '" + prop.getName() + "'.", e);
                  }
               }
            }
         }
         
         propsByGetter = ImmutableMap.copyOf(ret);
      }
      
      return propsByGetter;
   }
   
   /**
    * Returns (after eventually has constructed) the map of setters to {@code PropertyMeta} map, used by
    * {@code RecordBuffer$Handler}.
    * 
    * @return  the map of setters to {@code PropertyMeta} map.
    */
   public Map<Method, PropertyMeta> getPropsBySetterMap()
   {
      if (propsBySetter == null)
      {
         Map<Method, PropertyMeta> ret = new HashMap<>();
         boolean nonBeforeTable = afterTable.isEmpty();
         for (Pair<Property, PropertyMeta> property : properties)
         {
            PropertyMeta prop = property.getRight();
            
            // the hidden before-table specific fields are accessible only for ... before-tables
            if (nonBeforeTable && prop.setterName.startsWith("set_"))
            {
               // skip these properties (their name start with underscore) for the rest of the tables
               continue;
            }
            
            Class<? extends BaseDataType> wider = toWider.getOrDefault(prop.getType(), prop.getType());
            Class<?>[] sig = (prop.getExtent() == 0) ? new Class[] { wider }
                                                     : new Class[] { int.class, wider };
            try
            {
               ret.put(iface.getMethod(prop.setterName, sig), prop);
            }
            catch (NoSuchMethodException e)
            {
               if (log.isLoggable(Level.INFO))
               {
                  log.log(Level.INFO, 
                          "Failed to access setter for property '" + prop.getName() + "'.", e);
               }
            }
            
            Property ann = property.getLeft();
            String oName = ann.original;
            if (!oName.isEmpty() && ann.index == 1)
            {
               // denormalized extent fields, handle only once for each original property (for 1st element)
               
               sig = new Class[] { int.class, wider };
               try
               {
                  String setterName = "set" + Character.toUpperCase(oName.charAt(0)) + oName.substring(1);
                  ret.put(iface.getMethod(setterName, sig), prop);
               }
               catch (NoSuchMethodException e)
               {
                  if (log.isLoggable(Level.INFO))
                  {
                     log.log(Level.INFO,
                        "Failed to access original indexed setter for property '" + prop.getName() + "'.", e);
                  }
               }
            }
         }
         
         propsBySetter = ImmutableMap.copyOf(ret);
      }
      
      return propsBySetter;
   }
   
   /**
    * Get the structure-based signature for this DMO.
    * 
    * @return  The signature of this DMO.
    */
   public DmoSignature getSignature()
   {
      return signature;
   }
   
   /**
    * Obtain a short description of this object, usually for debugging.
    * 
    * @return  a short description of this object.
    */
   @Override
   public String toString()
   {
      return "DmoMeta{" + iface.getName() + " SQL:" + sqlTable + " 4GL:" + legacyTable + '}';
   }
   
   /**
    * Constructs and returns a string containing the P4GL schema definition of the table. The result is not
    * cached so this method is to be used only for debug.
    * 
    * @return  a string containing the P4GL legacy schema definition of the table.
    */
   public String getTableDefinition()
   {
      StringBuilder sb = new StringBuilder("DEFINE ");
      sb.append(tempTable ? "TEMP-TABLE " : "TABLE ").append(legacyTable);
      if (noUndo)
      {
         sb.append(" NO-UNDO");
      }
      if (!namespaceUri.isEmpty())
      {
         sb.append("\n      NAMESPACE-URI ").append(namespaceUri);
      }
      if (!namespacePrefix.isEmpty())
      {
         sb.append("\n      NAMESPACE-PREFIX ").append(namespacePrefix);
      }
      if (!xmlNodeName.isEmpty())
      {
         sb.append("\n      XML-NODE-NAME ").append(xmlNodeName);
      }
      if (!serializeName.isEmpty())
      {
         sb.append("\n      SERIALIZE-NAME ").append(serializeName);
      }
      if (!beforeTable.isEmpty())
      {
         sb.append("\n      BEFORE-TABLE ").append(beforeTable);
      }
      for (Map.Entry<String, Property> entry : propsByName.entrySet())
      {
         Property p = entry.getValue();
         if (p.propId < 0)
         {
            continue; // this is a hidden/reserved property, skip it
         }
         sb.append("\n   FIELD ").append(p.legacy).append(" AS ").append(p._ablType.toUpperCase());
         if (p.caseSensitive)
         {
            sb.append(" CASE-SENSITIVE");
         }
         if (p.extent > 0)
         {
            sb.append(" EXTENT ").append(p.extent);
         }
         if (p.scale != 0)
         {
            sb.append(" DECIMALS ").append(p.scale);
         }
         if (p.width != 0)
         {
            sb.append(" MAX-WIDTH ").append(p.width);
         }
         if (p.initialNull)
         {
            sb.append(" INITIAL ?");
         }
         else if (!p.initial.isEmpty())
         {
            sb.append(" INITIAL ").append(p.initial);
         }
         if (p.label != null && !p.label.isEmpty())
         {
            sb.append(" LABEL ").append(p.label);
         }
         if (p.columnLabel != null && !p.columnLabel.isEmpty())
         {
            sb.append(" COLUMN-LABEL ").append(p.columnLabel);
         }
      }
      
      for (Index index : indexes)
      {
         sb.append("\n   INDEX ").append(index.legacy()).append(" AS ");
         if (index.primary())
         {
            sb.append("PRIMARY ");
         }
         if (index.unique())
         {
            sb.append("UNIQUE ");
         }
         if (index.word())
         {
            sb.append("WORD-INDEX ");
         }
         for (IndexComponent comp : index.components())
         {
            sb.append(comp.legacy()).append(comp.descending() ? " DESC " : " ");
         }
      }
      return sb.toString();
   }
   
   /**
    * Check if DMO corresponds to a meta table. The project must have been converted with support for the
    * specific meta table.
    *
    * @return {@code true} only if DMO corresponds to a system/meta table.
    */
   public boolean isMeta()
   {
      return metaTable != null;
   }
   
   /**
    * Obtains the {@link MetadataManager.SystemTable} if this corresponds to a meta table. The project must
    * have been converted with support for the specific meta table.
    * 
    * @return  The {@code SystemTable} if DMO corresponds to a system/meta table, and {@code null} otherwise.
    */
   public MetadataManager.SystemTable getMetaTable()
   {
      return metaTable;
   }
   
   /**
    * Retrieve the number of fields which are stored physically in the database table. This may not equal
    * the number of properties, as some data types require two SQL fields (i.e. datetime-tz).
    * 
    * @return   The number of fields which are generated in the database table for this model.
    */
   public int getSqlFieldCount()
   {
      return sqlFieldCount;
   }
   
   /**
    * Check if this DMO has computed columns in the underlying database representation. This is 
    * used mostly for temporary tables and may not be completely compatible with persistent tables.
    * 
    * @param    includeInvisible 
    *           {@code true} to include the columns which are defined as invisible in the underlying 
    *           database. Invisible columns are the ones which are not implicitly retrieved using 
    *           SELECT * or direct access.
    * 
    * @return   {@code true} if we also have other columns that the legacy ones in the database.
    */
   public boolean hasComputedColumns(boolean includeInvisible)
   {
      int position = includeInvisible ? 0 : 1;
      if (hasComputedColumns[position] == null)
      {
         hasComputedColumns[position] = false;
         
         // temp-tables have invisible columns; it will always match the FWD internal representation
         boolean relaxed = tempTable && !includeInvisible;
         
         if (!relaxed)
         {
            Set<Map.Entry<String, Property>> fieldEntries = this.propsByName.entrySet();
            
            Set<Property> indexedProperties = new HashSet<>();
            Iterator<P2JIndex> databaseIndexes = this.getDatabaseIndexes();
            
            while (databaseIndexes.hasNext())
            {
               P2JIndex index = databaseIndexes.next();
               ArrayList<P2JIndexComponent> components = index.components();
               for (int i = 0; i < components.size(); i++)
               {
                  P2JIndexComponent comp = components.get(i);
                  Property property = this.getFieldInfo(comp.getPropertyName());
                  indexedProperties.add(property);
               }
            }
            
            for (Map.Entry<String, Property> field : fieldEntries)
            {
               Property property = field.getValue();
               
               if (indexedProperties.contains(property) && property._isCharacter)
               {
                  hasComputedColumns[position] = true;
                  break;
               }
            }
         }
      }
      return hasComputedColumns[position];
   }
   
   /**
    * Creates, if needed, the existingFieldsMap, map of DMO property names to {@code P2JField} 
    * objects describing the corresponding fields and array of {@code Property}, 
    * existingFieldsPropertyList, holding properties for each DMO field, where the index in
    * array is the id of the property minus 1.
    */
   private synchronized void computeExistingFields()
   {
      if (existingFieldsMap == null)
      {
         Map<String, P2JField> map = new LinkedHashMap<>();
         ArrayList<Property> list;
         
         Iterator<Property> it = getFields(false);
         list = new ArrayList<>(originalProperties.size());
         while (it.hasNext())
         {
            Property p = it.next();
            if (p.propId < 0)
            {
               // skip ReservedProperties
               continue;
            }
            
            int extent = p.extent;
            if (extent != 0 && p.index > 1)
            {
               // skip denormalized fields other than the first from series
               continue;
            }
            
            Class<?> bdtType = p._fwdType;
            Object init = null;
            if (BaseDataType.class.isAssignableFrom(bdtType))
            {
               try
               {
                  if (p.initialNull)
                  {
                     init = bdtType.getConstructor().newInstance();
                  }
                  else
                  {
                     if (bdtType == date.class)
                     {
                        init = "today".equalsIgnoreCase(p.initial)
                              ? new character(p.initial)
                              : date.parseInitial(p.initial);
                     }
                     else if (bdtType == datetime.class)
                     {
                        init = "now".equalsIgnoreCase(p.initial)
                              ? new character(p.initial)
                              : datetime.parseInitial(p.initial);
                     }
                     else if (bdtType == datetimetz.class)
                     {
                        init = "now".equalsIgnoreCase(p.initial)
                              ? new character(p.initial)
                              : datetimetz.parseInitial(p.initial);
                     }
                     else if (bdtType == logical.class)
                     {
                        String fmt = p.format.isEmpty() ? logical.DEFAULT_FORMAT : p.format;
                        
                        init = new logical(p.initial, fmt);
                     }
                     else // other data types which does not have particular initialization cases
                     {
                        init = bdtType.getConstructor(String.class).newInstance(p.initial);
                     }
                  }
               }
               catch (ReflectiveOperationException e)
               {
                  if (log.isLoggable(Level.SEVERE))
                  {
                     log.log(Level.SEVERE,
                             "Failed to instantiate default value for " + p.name + " of " + iface);
                  }
               }
            }
            
            P2JField existingLegacyField = new P2JField(p.legacy,
                     ParmType.fromClass(bdtType),
                     extent,
                     p.format,
                     (BaseDataType) init,
                     p.label,
                     p.columnLabel,
                     p.caseSensitive,
                     p.codePage,
                     p.help,
                     p.serializeHidden,
                     p.serializeName,
                     p.xmlDataType,
                     p.xmlNodeName,
                     p.xmlNodeType,
                     p.mandatory,
                     p.scale,
                     p.width); 
            
            map.put(p.name, existingLegacyField);
            while (list.size() < p.id)
            {
               list.add(null);
            }
            list.set(p.id - 1, p);
         }
         
         list.trimToSize();
         existingFieldsMap = Collections.unmodifiableMap(map);
         existingFieldPropertiesList = Collections.unmodifiableList(list);
      }
   }
   
   /**
    * Check if this DMO has normalized extents. These should be loaded separately.
    * 
    * @return {@code true} if the DMO has non-expanded extents (kept in a separate SQL table).
    */
   public boolean hasNotExpandedExtents()
   {
      return nonExpandedExtents;
   }
   
   /**
    * Get the map of DMO property names to {@code P2JField} objects describing the corresponding fields,
    * creating it first, if necessary.
    * 
    * @return  Map of property names to field descriptors.
    */
   public Map<String, P2JField> getExistingFieldsMap()
   {
      if (existingFieldsMap == null)
      {
         computeExistingFields();      
      }
      
      return Collections.unmodifiableMap(existingFieldsMap);
   }
   
   /**
    * Get the list of DMO {@code Property} objects describing the corresponding fields,
    * creating it first, if necessary.
    * 
    * @return  List of field descriptors, ordered by their id and each index corresponds
    *          the objects id.
    */
   private List<Property> getExistingFieldsList()
   {
      if (existingFieldPropertiesList == null)
      {
         computeExistingFields();      
      }
      
      return existingFieldPropertiesList;
   }
   
   /**
    * Word index data. 
    */
   public static class WordIndexData
   {
      /** word table name */
      public final String wordTableName;
      
      /** extent table name */
      public final String extentTableName;
      
      /** Flag indicating that field is case sensitive */
      public final boolean caseSensitive;
      
      /**
       * Constructor
       *
       * @param   wordTableName
       *          Word table name
       * @param   extentTableName
       *          Extent table name
       * @param   caseSensitive
       *          Flag indicating that field is case sensitive 
       */
      public WordIndexData(String wordTableName, String extentTableName, boolean caseSensitive)
      {
         this.wordTableName = wordTableName;
         this.extentTableName = extentTableName;
         this.caseSensitive = caseSensitive;
      }
      
   }
}