# DynamicTablesHelper.java

    /*
    ** Module   : DynamicTablesHelper.java
    ** Abstract : Helper class for dynamic temporary tables.
    **
    ** Copyright (c) 2013-2025, Golden Code Development Corporation.
    **
    ** -#- -I- --Date-- ---------------------------------------Description---------------------------------------
    ** 001 SVL 20130624 Created initial version.
    ** 002 SVL 20130822 Continued implementation.
    ** 003 SVL 20131115 Implemented tableExists, some refactoring for dynamic buffers registration
    **                  and clean up.
    ** 004 CA  20131013 Added no-op deleted() method, required by the changes in Finalizable interface.
    ** 005 OM  20140117 Added support for runtime indexed access to fields from buffer.
    ** 006 SVL 20140123 Implemented get4GLTableName.
    ** 007 SVL 20140124 Refactored DMO class cleanup.
    ** 008 OM  20140123 Added legacy table name in createDynamicDMO. Also the fields ar normalized.
    ** 009 OM  20140128 Fixed historical annotation so the table/fields can be find by P2OLookup.
    **                  Removed ConcurrentModificationException in cleanup.
    ** 010 ECF 20140226 Renamed generateDMOAst to generateSchemaAst. Added index ASTs to the schema
    **                  AST produced by this method. Replaced Apache commons logging with J2SE
    **                  logging.
    ** 011 ECF 20140404 Reimplemented dynamic conversion to use ConversionPool.
    ** 012 OM  20140414 Added case-sensitive field support for indexed character columns.
    ** 013 ECF 20140611 Fixed regression caused by 012; avoids remapping all character computed
    **                  column information for every dynamic temp-table. DatabaseManager now handles
    **                  this mapping within its registerDynamicTable method.
    ** 014 ECF 20140922 Reduced memory leak and increased performance by caching dynamic temp-table
    **                  resources. This is an interim update, but the cleanup of expired cache
    **                  resources currently is broken, in that DMO resources can be released under
    **                  some circumstances while a Hibernate session is still using them. The
    **                  temporary workaround is to define a high cache limit, so that the cache does
    **                  not expire elements.
    ** 015 ECF 20150109 Preserve original case for legacy field names.
    ** 016 VMN 20150122 Fixes to use non-normalized legacy field and table names in annotations.
    ** 017 ECF 20150327 Added p2o_post processing step.
    ** 018 ECF 20150414 Replaced in-memory compiling of dynamically generated DMO source code with
    **                  direct-to-bytecode assembly from Java ASTs.
    ** 019 ECF 20150430 Fixed memory leak of active temp-table builders in DMOResourceManager inner
    **                  class. A proper clean-up mechanism for dynamic temp-tables which are no longer
    **                  in use and have expired from the cache is still needed.
    ** 020 ECF 20150521 Fixed memory leak of temporary buffers.
    ** 021 ECF 20150819 Fixed NPE caused by race condition in createDynamicDMO. Increased cache size.
    ** 022 OM  20151028 Avoided NPE in context cleanup().
    ** 023 ECF 20160506 Slight optimization of get4GLTableName.
    ** 024 GES 20160504 Modified initial value processing for character field types, to make it safe
    **                  for the full range of characters that might be present (including embedded
    **                  double quotes).
    ** 025 ECF 20160521 Replaced ExpiryCache with simple HashMap (TODO: switch back when cleanup code
    **                  works).
    ** 026 OM  20160905 Small optimizations.
    ** 027 SVL 20190614 Support for CLOB and BLOB fields.
    ** 028 CA  20190728 Added object type.
    **     ECF 20180802 Fixed findExistingTable to work with metadata tables.
    **     CA  20190826 Some fixes in generateSchemaAst related to initial field value.
    **     CA  20190902 Fixed problems when connecting to a FWD server running a database schema which
    **                  not actively managed by the requester, too.
    ** 029 SVL 20190801 tableExists looks in shared tables too.
    ** 030 CA  20200622 Small performance improvement - use IdentityHashMap if the key is java.lang.Class.
    ** 031 CA  20200922 For temp-tables, the reserved properties need to be emitted, too.  Fixed generateSchemaAst
    **                  which had code as if it was generating a .p2o and not a .schema AST.
    **     OM  20200924 P2JIndexComponent carries multiple information to avoid map lookups for them.
    **     OM  20201001 Improved DMO manipulation performance by caching slow Property annotation access.
    **     OM  20201002 Use DmoMeta cached information instead of map lookups.
    **     CA  20201003 Use an identity HashSet where possible.
    **     OM  20201120 Added DECIMALS attribute support.
    **     SVL 20210330 Fixed handling of default values for dynamic LOGICAL fields.
    **     SVL 20210401 "?" text is treated as an unknown default value for any data type.
    **     ECF 20210511 Refactored string key normalization work.
    **     CA  20210823 Fixed extent field check - extent 1 is allowed.
    **     HC  20211201 Fixed runtime conversion of comhandle field for dynamic temp tables.
    **     ECF 20211202 Fixed initial value bug. Some naming cleanup.
    **     OM  20211213 Added missing COLUMN-LABEL attribute to fields.
    **     CA  20211222 Fixed memory leaks: remove the dynamic temp-table from the DMO users, when the dynamic
    **                  temp-table gets deleted.
    **     OM  20220212 Use ReservedProperty static method for testing reserved properties names.
    **     OM  20220727 FieldId and PropertyId are different for denormalized extent fields.
    **     OM  20221003 When generateSchemaAst() set annotations only for present field attributes.
    **     BS  20221004 Changes for SAVE CACHE support.
    **     OM  20220706 New attributes supported in generateSchemaAst().
    **     OM  20220914 Use MAX-WIDTH to generate custom sized varchar columns.
    **     BS  20221004 Changes for SAVE CACHE support.
    **     IAS 20221006 Added RAW type to the switch in the 'generateSchemaAst'.
    **     IAS 20221013 Fixed setting of "serialize-name" and "serialize-hidden annotations".
    **     IAS 20221014 Fixed 'column-codepage' support.
    **     SVL 20230110 P2JIndex.components() provides direct access to the components array in order to improve
    **                  performance.
    **     CA  20230116 Avoid using handle.unwrap, handle.getReference or other BDT usage from within FWD runtime.
    **     OM  20230214 Added NO-UNDO property for dynamically built temp-table.
    ** 032 CA  20230215 Improved other cases to reduce BDT usage from within FWD runtime.
    ** 033 IAS 20230427 Added 'legacyName' to the 'indexAst'.
    ** 034 GBB 20230512 Logging methods replaced by CentralLogger/ConversionStatus.
    ** 035 DDF 20230608 Made the size of the DMO resources cache configurable.
    **     DDF 20230613 Removed the logic that reads the configuration size of the cache from the static block.
    **                  CacheManager will handle the finding of the configuration size and the cache creation.
    **     DDF 20230627 Made the cache final.
    ** 036 OM  20240402 Removed registration of dynamic tables with DatabaseManager.
    ** 037 CA  20240502 When a BEFORE-TABLE is being built too, for a dynamic temp-table, the associated DMOs for
    **                  both the before and after table must have the after/beforeTable annotations set.
    ** 038 AD  20240403 Added support for caching to disk dynamically generated table DMOs.
    **     AD  20240430 Unified the 2 configuration options created for reading/writing to persistent cache.
    ** 039 ICP 20240703 Added null check in initializePersistentCacheData.
    **     ICP 20240711 Ensured that newly created temporary directories are cleaned up.
    **     ICP 20240805 When the jar is created, write the jar index as well.
    **                  Ensured no duplicates in the jar.
    ** 040 OM  20240901 Improved Database API.
    ** 041 OM  20241128 Multi-tenant runtime support: selected the proper persistence context, eventually
    **                  based on [sharedDb] parameter.
    ** 042 ICP 20250210 Added directory.xml and bootstrap option to disable the update of the persistent cache.
    **                  When the update is disabled, the cache file can be now loaded from the classpath.
    **                  Fixed edge case when DMO is found in persistentCacheMap, but classloading fails.
    ** 043 ICP 20250129 Used logical constants to leverage cached instances.
    */

    /*
    ** This program is free software: you can redistribute it and/or modify
    ** it under the terms of the GNU Affero General Public License as
    ** published by the Free Software Foundation, either version 3 of the
    ** License, or (at your option) any later version.
    **
    ** This program is distributed in the hope that it will be useful,
    ** but WITHOUT ANY WARRANTY; without even the implied warranty of
    ** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    ** GNU Affero General Public License for more details.
    **
    ** You may find a copy of the GNU Affero GPL version 3 at the following
    ** location: https://www.gnu.org/licenses/agpl-3.0.en.html
    **
    ** Additional terms under GNU Affero GPL version 3 section 7:
    **
    **   Under Section 7 of the GNU Affero GPL version 3, the following additional
    **   terms apply to the works covered under the License.  These additional terms
    **   are non-permissive additional terms allowed under Section 7 of the GNU
    **   Affero GPL version 3 and may not be removed by you.
    **
    **   0. Attribution Requirement.
    **
    **     You must preserve all legal notices or author attributions in the covered
    **     work or Appropriate Legal Notices displayed by works containing the covered
    **     work.  You may not remove from the covered work any author or developer
    **     credit already included within the covered work.
    **
    **   1. No License To Use Trademarks.
    **
    **     This license does not grant any license or rights to use the trademarks
    **     Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
    **     of Golden Code Development Corporation. You are not authorized to use the
    **     name Golden Code, FWD, or the names of any author or contributor, for
    **     publicity purposes without written authorization.
    **
    **   2. No Misrepresentation of Affiliation.
    **
    **     You may not represent yourself as Golden Code Development Corporation or FWD.
    **
    **     You may not represent yourself for publicity purposes as associated with
    **     Golden Code Development Corporation, FWD, or any author or contributor to
    **     the covered work, without written authorization.
    **
    **   3. No Misrepresentation of Source or Origin.
    **
    **     You may not represent the covered work as solely your work.  All modified
    **     versions of the covered work must be marked in a reasonable way to make it
    **     clear that the modified work is not originating from Golden Code Development
    **     Corporation or FWD.  All modified versions must contain the notices of
    **     attribution required in this license.
    */

    package com.goldencode.p2j.persist;

    import antlr.*;
    import com.goldencode.asm.*;
    import com.goldencode.ast.*;
    import com.goldencode.cache.*;
    import com.goldencode.p2j.cfg.*;
    import com.goldencode.p2j.convert.*;
    import com.goldencode.p2j.directory.DirectoryService;
    import com.goldencode.p2j.pattern.*;
    import com.goldencode.p2j.persist.orm.*;
    import com.goldencode.p2j.schema.*;
    import com.goldencode.p2j.security.*;
    import com.goldencode.p2j.uast.*;
    import com.goldencode.p2j.util.*;
    import com.goldencode.p2j.util.ErrorManager;
    import com.goldencode.p2j.util.Utils;
    import com.goldencode.p2j.util.logging.*;

    import java.io.*;
    import java.util.*;
    import java.util.logging.*;

    import org.apache.commons.io.FileUtils;

    import java.util.concurrent.atomic.*;
    import java.util.function.Consumer;
    import java.util.jar.*;

    /**
     * Helper class for dynamic temporary tables.
     */
    public class DynamicTablesHelper
    {
       /** Base package for generated DMOs. */
       static final String DMO_BASE_PACKAGE = "com.goldencode.p2j.persist.dynamic";

       /** Category name for DMO interfaces, used in conversion rules. */
       static final String DMO_IFACE_CATEGORY = "iface";

       /** Param type to Progress type. */
       static final Map<ParmType, Integer> propTypeToProgress = new HashMap<>();

       /** Logger. */
       private static final CentralLogger LOG = CentralLogger.get(DynamicTablesHelper.class.getName());

       /** Singleton instance. */
       private static volatile DynamicTablesHelper instance;

       /** In-memory class loader */
       private static AsmClassLoader systemInMemoryClassLoader;

       /** Bootstrap configuration for {@link #updatePersistentCache} */
       private static Boolean bootstrapUpdateDMOCache = null;

       /** Cache for dynamically converted and compiled DMO resources */
       private final Map<CacheKey, CacheData> cache;

       /** Lock object on which to synchronize cache operations */
       private final Object cacheLock = new Object();

       /**
        * Cache used to find out if a dynamic table DMO needs to be assembled or exists saved on disk.
        * The key is a string value of a CacheKey object. The value is a list, starting with an id,
        * and followed by the names of all the interface names used to generate the DMO class.
        */
       private Map<String, List<String>> persistentCacheMap = new HashMap<>();

       /** Marks if dynamic table DMO classes generated will be saved to/read from a persistent cache. */
       private boolean usePersistentCache = false;

       /** Marks if dynamic table DMO persistent cache will be updated during this server run. */
       private boolean updatePersistentCache = false;

       /** Name of a temporary directory where .class files of dynamic table DMO classes are saved. */
       private String tempLibFilePath;

       /** Path to the jar file where dynamic table DMO classes will be placed at server shutdown. */
       private String persistentCacheJarPath;

       /** Path to the file containing data used to identify cached dynamic table DMO classes. */
       private String persistentCacheMapPath;

       /** Path to the jar file that will temprarily exist at server shutdown. */
       private String temporaryJarName;

       /** File object corresponding to the jar file with dynamic table DMO classes. */
       private File persistentCacheJarFile;

       /** List of directories that are newly created. */
       private List<File> createdDirectories = new ArrayList<>();

       /** List of entries present in the jar. */
       private static Set<String> existingEntries = new HashSet<>();

       /** File object corresponding to the file with data used to identify cached dynamic table DMO classes. */
       private File persistentCacheMapFile;

       /** Containers for context-specific data. */
       private final ContextLocal<Context> context = new ContextLocal<Context>()
       {
          public WeightFactor getWeight() { return WeightFactor.LEVEL_8; }
          public Context initialValue() { return new Context(); }
          protected void cleanup(Context ctx) { ctx.cleanup(); }
       };

       /** Counter used for generation of unique names for dynamic tables. */
       private final AtomicLong tableCounter;

       /** Cross-context manager which cleans up shared DMO resources once they go out of scope */
       private final DMOResourceManager dmoResourceManager = new DMOResourceManager();

       static
       {
          propTypeToProgress.put(ParmType.CHAR, ProgressParserTokenTypes.FIELD_CHAR);
          propTypeToProgress.put(ParmType.INT, ProgressParserTokenTypes.FIELD_INT);
          propTypeToProgress.put(ParmType.INT64, ProgressParserTokenTypes.FIELD_INT64);
          propTypeToProgress.put(ParmType.DEC, ProgressParserTokenTypes.FIELD_DEC);
          propTypeToProgress.put(ParmType.LOG, ProgressParserTokenTypes.FIELD_LOGICAL);
          propTypeToProgress.put(ParmType.DATE, ProgressParserTokenTypes.FIELD_DATE);
          propTypeToProgress.put(ParmType.DT, ProgressParserTokenTypes.FIELD_DATETIME);
          propTypeToProgress.put(ParmType.DTTZ, ProgressParserTokenTypes.FIELD_DATETIME_TZ);
          propTypeToProgress.put(ParmType.HANDLE, ProgressParserTokenTypes.FIELD_HANDLE);
          propTypeToProgress.put(ParmType.OBJECT, ProgressParserTokenTypes.FIELD_CLASS);
          propTypeToProgress.put(ParmType.RECID, ProgressParserTokenTypes.FIELD_RECID);
          propTypeToProgress.put(ParmType.ROWID, ProgressParserTokenTypes.FIELD_ROWID);
          propTypeToProgress.put(ParmType.RAW, ProgressParserTokenTypes.FIELD_RAW);
          propTypeToProgress.put(ParmType.CLOB, ProgressParserTokenTypes.FIELD_CLOB);
          propTypeToProgress.put(ParmType.BLOB, ProgressParserTokenTypes.FIELD_BLOB);
          propTypeToProgress.put(ParmType.COMHANDLE, ProgressParserTokenTypes.FIELD_COM_HANDLE);

          systemInMemoryClassLoader = AsmClassLoader.getInstance();
          if (systemInMemoryClassLoader == null)
          {
             throw new IllegalStateException("It is expected to have InMemoryClassLoader " +
                                             "in the chain of system class loaders.");
          }
       }

       /**
        * Private constructor to enforce singleton.
        */
       private DynamicTablesHelper()
       {
          cache = CacheManager.createMapCache(DynamicTablesHelper.class, null, 8192);

          initializePersistentCacheData();
          if (usePersistentCache)
          {
             tableCounter = new AtomicLong(persistentCacheMap.size() + 1);
          }
          else
          {
             tableCounter = new AtomicLong(1);
          }
       }

       /**
        * Get singleton instance of the helper.
        *
        * @return  singleton instance of the helper.
        */
       public static DynamicTablesHelper getInstance()
       {
          if (instance == null)
          {
             synchronized (DynamicTablesHelper.class)
             {
                if (instance == null)
                {
                   instance = new DynamicTablesHelper();
                   PatternEngine.setDebugLevel(ConversionDriver.MSG_NONE);
                }
             }
          }

          return instance;
       }

       /**
        * Initializes the bootstrap configuration for dynamic DMO cache updates.
        *
        * @param   config
        *          The {@link BootstrapConfig} instance used to retrieve the configuration value.
        */
       public static void bootstrap(BootstrapConfig config)
       {
          bootstrapUpdateDMOCache = config.getBoolean("persistence", "dmo", "updateDynamicDMOCache", null);
       }

       /**
        * Get DMO field name by 4GL field name for a dynamic DMO.
        *
        * @param   dmoIface
        *          DMO interface that corresponds the target dynamic DMO class.
        * @param   name4gl
        *          4GL field name.
        *
        * @return  DMO field name for the given DMO class and 4GL field name or {@code null} if no
        *          corresponding name was found.
        */
       static String getHibernateFieldName(Class<? extends DataModelObject> dmoIface, String name4gl)
       {
          DMOInfo dmoInfo = getInstance().context.get().getDMOInfo(dmoIface);
          if (dmoInfo == null)
          {
             return null;
          }

          return dmoInfo.legacyToDmoNames.get(TextOps.rightTrimLower(name4gl));
       }

       /**
        * Get DMO field name by field creation id for a dynamic DMO.
        *
        * @param   dmoIface
        *          DMO interface that corresponds the target dynamic DMO class.
        * @param   fieldId
        *          4GL field id.
        *
        * @return  DMO field name for the given DMO class and 4GL field name or {@code null} if no
        *          corresponding name was found.
        */
       static String getHibernateFieldName(Class<? extends DataModelObject> dmoIface, int fieldId)
       {
          // the algorithm here is based on the way the Hibernate field names are generated at
          // "conversion" time. See (construction of clsFieldsXXX mappings in createDynamicDMO())
          return "field" + fieldId;
       }

       /**
        * Get 4GL field name by DMO field name for a dynamic DMO.
        *
        * @param   dmoIface
        *          DMO interface that corresponds the target dynamic DMO class.
        * @param   dmoFieldName
        *          DMO field name.
        *
        * @return  4GL field name for the given DMO class and DMO field name or {@code null} if no
        *          corresponding name was found.
        */
       static String get4GLFieldName(Class<? extends DataModelObject> dmoIface, String dmoFieldName)
       {
          DMOInfo dmoInfo = getInstance().context.get().getDMOInfo(dmoIface);
          if (dmoInfo == null)
          {
             return null;
          }

          return dmoInfo.dmoToLegacyNames.get(dmoFieldName);
       }

       /**
        * Get table name and database name from the string which represents full table name
        *
        * @param fullTableName
        *        The string which represents full table name. Can have two formats:
        *        "databaseName.tableName" or "tableName".
        *
        * @return String array with two elements. The first element is table name. The second element
        *         is database name or <code>null</code> if the database name wasn't specified in the
        *         full table name.
        */
       static String[] parseTableName(String fullTableName)
       {
          String tableName;
          String databaseName = null;

          int pos = rightTrim(fullTableName).indexOf('.');
          if (pos < 0)
          {
             tableName = rightTrim(fullTableName);
          }
          else
          {
             // TODO database name should be case-insensitive, but for now it is considered to be
             // case-sensitive
             databaseName = rightTrim(fullTableName.substring(0, pos));
             tableName = rightTrim(fullTableName.substring(pos + 1));
          }

          return new String[] { tableName, databaseName };
       }

       /**
        * Allows to determine if the buffer is for a dynamic table.
        *
        * @param  buffer
        *         Buffer to check.
        *
        * @return <code>true</code> if buffer is for a dynamic table.
        */
       static boolean isDynamicTableBuffer(RecordBuffer buffer)
       {
          TempTable tempTable = buffer.getParentTable();

          return (tempTable != null && tempTable._dynamic());
       }

       /**
        * Returns the {@link TempTableBuilder} object that contains information about the 4GL
        * representation of the table or <code>null</code> if there is no corresponding dynamic table
        * for the specified record buffer.
        *
        * @param  buffer
        *         Record buffer that corresponds to the target dynamic table.
        *
        * @return See above.
        */
       static TempTableBuilder getTableInfo(RecordBuffer buffer)
       {
          return (TempTableBuilder) buffer.getParentTable();
       }

       /**
        * Determines if the specified table exists.
        *
        * @param  persistence
        *         Persistence which corresponds the target database.
        * @param  tableName4GL
        *         4GL table name.
        *
        * @return <code>true</code> if the specified table exists.
        */
       static boolean tableExists(Persistence persistence, String tableName4GL)
       {
          String normalizedTableName = TextOps.rightTrimLower(tableName4GL);

          if (persistence.isTemporary())
          {
              boolean res = BufferManager.get().getStaticTempTableDepth(normalizedTableName) != -1;
              if (!res)
              {
                 res = SharedVariableManager.lookupTempTable(normalizedTableName) != null;
              }
              return res;
          }
          else
          {
             String schema = DatabaseManager.getSchema(persistence.getDatabase(Persistence.PRIVATE_CTX));
             return TableMapper.getDMOClass(schema, normalizedTableName) != null;
          }
       }

       /**
        * Search for an existing table with the specified name.
        *
        * @param  fullTableName
        *         Table name in format "databaseName.tableName" or "tableName". In the latter case
        *         table search is performed in all permanent databases and then the temporary
        *         database. Dynamic tables are ignored.
        *
        * @return persistence which corresponds the target database or <code>null</code> if there is
        *         no such table.
        */
       static Persistence findExistingTable(String fullTableName)
       {
          String[] arr = parseTableName(fullTableName);
          String srcTableName = arr[0];
          String srcDatabaseName = arr[1];

          if (srcDatabaseName != null)
          {
             // database is explicitly specified
             if (!ConnectionManager.connected(srcDatabaseName).booleanValue())
             {
                return null;
             }

             String physicalName = ConnectionManager.pdbName(srcDatabaseName).getValue();
             Persistence persistence = PersistenceFactory.getInstance(physicalName);

             if (tableExists(persistence, srcTableName))
             {
                return persistence;
             }
             else
             {
                // special handling for metadata table
                if (srcTableName.startsWith("_"))
                {
                   return findMetaPersistence(physicalName, srcTableName);
                }

                return null;
             }
          }
          else
          {
             // TODO: OM: it looks to me like the _temp database is checked first!

             // search through all databases, check permanent databases first
             long numDbs = ConnectionManager.getConnectedDbCount();
             for (int i = 1; i <= numDbs; i++)
             {
                String databaseName = ConnectionManager.ldbNameNative(i);
                if (!ConnectionManager.connected(databaseName).booleanValue())
                {
                   continue;
                }

                Database db = ConnectionManager.get().getDatabase(databaseName);
                Persistence persistence = PersistenceFactory.getInstance(db);

                if (!tableExists(persistence, srcTableName))
                {
                   // TODO: if a buffer is dynamically created using CREATE BUFFER bh FOR TABLE "name", the
                   //       table's DMO might have not been processed yet, leading to a negative result here.
                   //       Try adding the table now

                   // special handling for metadata table
                   if (srcTableName.startsWith("_"))
                   {
                      persistence = findMetaPersistence(db.getName(), srcTableName);

                      if (persistence != null)
                      {
                         return persistence;
                      }
                   }
                   continue;
                }

                return persistence;
             }

             // check temporary database
             Persistence persistence = PersistenceFactory.getInstance(DatabaseManager.TEMP_TABLE_DB);

             if (tableExists(persistence, srcTableName))
             {
                return persistence;
             }
             else
             {
                return null;
             }
          }
       }

       /**
        * Get original 4GL table name. Despite the fact that we specify 4GL table name as an input
        * parameter, this parameter can have different letter case or may have trailing spaces (this
        * form is allowed when specifying table name in 4GL code).
        *
        * @param  persistence
        *         Persistence which corresponds the target database.
        * @param  tableName4GL
        *         4GL table name (case-insensitive, may have trailing spaces).
        *
        * @return original 4GL table name.
        */
       static String get4GLTableName(Persistence persistence, String tableName4GL)
       {
          // TODO implement when table names in the original letter case will be preserved
          return TextOps.rightTrimLower(tableName4GL);
       }

       /**
        * Traverse an AST and assign unique IDs to all of its nodes.
        *
        * @param  root
        *         The root node of the AST.
        * @param  startId
        *         First available id for assignment.
        *
        * @return new first available id for assignment.
        */
       static long assignIds(Aast root, long startId)
       {
          root.setId(startId++);

          int childCount = root.getNumImmediateChildren();
          if (childCount > 0)
          {
             for (int i = 0; i < childCount; i++)
             {
                Aast child = root.getChildAt(i);
                startId = assignIds(child, startId);
             }
          }
          return startId;
       }

       /**
        * Find the {@code Persistence} object that is associated with the given database and table.
        *
        * @param   pdb
        *          Physical database name of primary database.
        * @param   tableName
        *          Unqualified table name.
        *
        * @return  {@code Persistence} object if table is found
        */
       private static Persistence findMetaPersistence(String pdb, String tableName)
       {
          Database primaryDb = ConnectionManager.get().getDatabase(pdb);
          Database metaDb = primaryDb.toType(Database.Type.META);
          Persistence persistence = PersistenceFactory.getInstance(metaDb);

          if (tableExists(persistence, tableName))
          {
             return persistence;
          }

          return null;
       }

       /**
        * Convenience method to remove trailing whitespace (space, tab, line feed, carriage return) from the
        * string.
        *
        * @param  str
        *         Target string.
        *
        * @return the target string without trailing spaces.
        */
       private static String rightTrim(String str)
       {
          return TextOps.trim(str, null, false, false);
       }

       /**
        * Method used to save to disk a dynamic table class generated at runtime as a .class file.
        *
        * @param   className
        *          The name of the newly generated class that will be saved to disk.
        * @param   code
        *          The contents of the class file as a byte array.
        */
       private void saveDynamicDmoToDisk(String className, byte[] code)
       {
          String fileName = tempLibFilePath +
                            File.separator +
                            className.replace(".", File.separator) +
                            ".class";
          File file = new File(fileName);
          file.getParentFile().mkdirs();
          try (FileOutputStream fos = new FileOutputStream(file))
          {
             fos.write(code);
          }
          catch (IOException e)
          {
             LOG.severe("Failed to write dynamic DMO to jar file for [" + className + "]", e);
          }
       }

       /**
        * Method called at class initialization, meant to read from disk the data regarding
        * the dynamic table classes cache, saved in the form of a HashMap object.
        */
       private void initializePersistentCacheData()
       {
          DirectoryService ds = DirectoryService.getInstance();
          try
          {
             if (ds != null)
             {
                if (!ds.bind())
                {
                   throw new RuntimeException("Directory bind failed");
                }

                String directoryNodeParent = "dynamic-dmos/";
                String customUsePersistentCache = directoryNodeParent + "use-persistent-cache";
                String customUpdatePersistentCache = directoryNodeParent + "update-persistent-cache";
                String customPersistentCacheMapPath = directoryNodeParent + "persistent-cache-map-path";
                String customPersistentCacheJarPath = directoryNodeParent + "persistent-cache-jar-path";
                String customTempJarFolderPath = directoryNodeParent + "temporary-cache-folder-path";
                String customTempJarFilePath = directoryNodeParent + "temporary-cache-jar-path";

                usePersistentCache = Utils.getDirectoryNodeBoolean(ds,
                                                                   customUsePersistentCache,
                                                                   usePersistentCache,
                                                                   false);
                if (usePersistentCache)
                {
                   LOG.info("Dynamic table dmos are configured to be read from jar.");

                   persistentCacheJarPath = null;
                   temporaryJarName = "lib_cache" + File.separator + "temp_dynamic_dmos.jar";

                   persistentCacheMapPath = "dynamic_dmo_cache_data.txt";

                   tempLibFilePath = "temp_lib_cache";
                   updatePersistentCache = Utils.getDirectoryNodeBoolean(ds,
                                                                         customUpdatePersistentCache,
                                                                         updatePersistentCache,
                                                                         false);

                   persistentCacheMapPath = Utils.getDirectoryNodeString(ds,
                                                                         customPersistentCacheMapPath,
                                                                         persistentCacheMapPath,
                                                                         false);
                   persistentCacheJarPath = Utils.getDirectoryNodeString(ds,
                                                                         customPersistentCacheJarPath,
                                                                         persistentCacheJarPath,
                                                                         false);
                   tempLibFilePath = Utils.getDirectoryNodeString(ds,
                                                                  customTempJarFolderPath,
                                                                  tempLibFilePath,
                                                                  false);
                   temporaryJarName = Utils.getDirectoryNodeString(ds,
                                                                   customTempJarFilePath,
                                                                   temporaryJarName,
                                                                   false);

                   // if a bootstrap configuration is passed, the value read from directory.xml is overridden
                   if (bootstrapUpdateDMOCache != null)
                   {
                      updatePersistentCache = bootstrapUpdateDMOCache;
                   }

                   if (persistentCacheJarPath == null)
                   {
                      usePersistentCache = false;
                      LOG.severe("Did not provide path for dynamic table dmos jar.");
                   }
                }
             }
          }
          catch (Exception e)
          {
             LOG.severe("Error retrieving custom data for DynamicTableHelper.", e);
          }
          finally
          {
             if (ds != null && ds.isBound())
             {
                ds.unbind();
             }
          }

          if (usePersistentCache && updatePersistentCache)
          {
             createDirectoryHierarchy(new File(tempLibFilePath));

             persistentCacheJarFile = new File(persistentCacheJarPath);
             File parent = persistentCacheJarFile.getParentFile();
             if (parent != null && !parent.exists())
             {
                parent.mkdirs();
             }

             persistentCacheMapFile = new File(persistentCacheMapPath);
             if (!persistentCacheMapFile.exists())
             {
                cleanupDirectories();
                return;
             }

             try (FileInputStream fis = new FileInputStream(persistentCacheMapFile);
                  ObjectInputStream ois = new ObjectInputStream(fis))
             {
                persistentCacheMap = (HashMap<String, List<String>>) ois.readObject();
             }
             catch (Exception e)
             {
                LOG.severe("Failed to read dynamic table cache data map from disk.", e);
             }
          }

          if (usePersistentCache && !updatePersistentCache)
          {
             persistentCacheMapFile = new File(persistentCacheMapPath);
             if (persistentCacheMapFile.exists())
             {
                try (FileInputStream fis = new FileInputStream(persistentCacheMapFile);
                     ObjectInputStream ois = new ObjectInputStream(fis))
                {
                   persistentCacheMap = (HashMap<String, List<String>>) ois.readObject();
                }
                catch (Exception e)
                {
                   LOG.severe("Failed to read dynamic table cache data map from disk.", e);
                }
                return;
             }
             cleanupDirectories();

             // Try loading from classpath
             try (InputStream is = getClass().getClassLoader().getResourceAsStream(persistentCacheMapPath))
             {
                if (is != null)
                {
                   try (ObjectInputStream ois = new ObjectInputStream(is))
                   {
                      persistentCacheMap = (HashMap<String, List<String>>) ois.readObject();
                      LOG.info("Loaded dynamic DMO cache from classpath: " + persistentCacheMapPath);
                      return;
                   }
                }
                else
                {
                   LOG.warning("Dynamic DMO cache file not found in classpath: " + persistentCacheMapPath);
                }
             }
             catch (IOException | ClassNotFoundException e)
             {
                LOG.warning("Failed to load dynamic DMO cache from classpath: " + persistentCacheMapPath, e);
             }

             LOG.severe("Failed to load persistent cache. Dynamic DMOs will be generated at runtime.");
             usePersistentCache = false;
          }
       }

       /**
        * Method used to update/create the file on disk that retains data about the
        * dynamic table classes cache, in the form of a HashMap object.
        */
       private void updatePersistentCacheMap()
       {
          try (FileOutputStream fos = new FileOutputStream(persistentCacheMapFile);
               ObjectOutputStream oos = new ObjectOutputStream(fos))
          {
             oos.writeObject(persistentCacheMap);
          }
          catch (Exception e)
          {
             LOG.severe("Failed to save to file persistent cache map data.", e);
          }
       }

       /**
        * Method meant to create a Jar file containing dynamic table classes that were
        * cached to disk, so that they are not generated every time the server is started.
        * It creates a temporary jar file, copies all the classes from the old jar file,
        * if it exists, then takes all the new classes generated that are saved in a
        * folder that is then deleted. The old jar file is deleted, and the temporary
        * jar file is renamed to match the old one.
        */
       public void createCache()
       {
          if (!usePersistentCache)
          {
             cleanupDirectories();
             return;
          }

          if (updatePersistentCache)
          {
             File file = new File(tempLibFilePath);
             if (!file.exists())
             {
                LOG.severe("Trying to create dynamic dmos cache jar, but directory with classes is missing.");
                cleanupDirectories();
                return;
             }
             if (file.list() != null && file.list().length == 0)
             {
                LOG.info("Trying to create dynamic dmos cache jar, but directory with classes is empty.");
                cleanupDirectories();
                return;
             }

             File tempJar = new File(temporaryJarName);
             createDirectoryHierarchy(tempJar.getParentFile());
             if (tempJar.exists())
             {
                tempJar.delete();
             }
             try
             {
                tempJar.createNewFile();
             }
             catch (IOException e)
             {
                LOG.severe("Failed to create temporary JAR file for saving dynamic DMO classes jar to disk.", e);
                cleanupDirectories();
                return;
             }
             try (FileOutputStream fos = new FileOutputStream(tempJar);
                  JarOutputStream jos = new JarOutputStream(fos))
             {
                Set<String> packages = new HashSet<>();

                File[] files = file.listFiles();
                for (int i = 0; i < files.length; i++)
                {
                   createCacheHelper(files[i], jos, packages);
                }
                writeJarIndex(jos, packages);

                if (persistentCacheJarFile.exists())
                {
                   try (FileInputStream fis = new FileInputStream(persistentCacheJarFile);
                        JarInputStream jis = new JarInputStream(fis))
                   {
                      JarEntry je;
                      while ((je = jis.getNextJarEntry()) != null)
                      {
                         if (!existingEntries.contains(je.getName()))
                         {
                            jos.putNextEntry(je);
                            byte[] buffer = new byte[4096];
                            int bytesRead;
                            while ((bytesRead = jis.read(buffer)) != -1)
                            {
                               jos.write(buffer, 0, bytesRead);
                            }
                            jos.closeEntry();
                            existingEntries.add(je.getName());
                         }
                      }
                   }
                   persistentCacheJarFile.delete();
                }

                tempJar.renameTo(persistentCacheJarFile);
                updatePersistentCacheMap();
             }
             catch (Exception e)
             {
                LOG.severe("Failed to create JAR with dynamic DMO classes.", e);
             }
             finally
             {
                try
                {
                   FileUtils.cleanDirectory(file);
                   if (tempJar.exists())
                   {
                      if (!tempJar.delete())
                      {
                         LOG.warning("Failed to delete temporary JAR file: " + tempJar.getAbsolutePath());
                      }
                   }
                   cleanupDirectories();
                }
                catch (IOException e)
                {
                   LOG.severe("Failed to delete temporary dynamic DMO classes folder.", e);
                }
             }
          }
       }

        /**
        * Ensures the directory hierarchy exists by creating any missing directories.
        * Newly created directories are tracked for cleanup later.
        *
        * @param   dir
        *          The directory to create.
        */
       private void createDirectoryHierarchy(File dir)
       {
          File parent = dir;
          Stack<File> directoriesToCreate = new Stack<>();
          while (parent != null && !parent.exists())
          {
             directoriesToCreate.push(parent);
             parent = parent.getParentFile();
          }

          while (!directoriesToCreate.isEmpty())
          {
             File toCreate = directoriesToCreate.pop();
             try
             {
                if (toCreate.mkdir())
                {
                   createdDirectories.add(toCreate);
                }
                else if (!toCreate.exists())
                {
                   LOG.warning("Failed to create directory: " + toCreate.getAbsolutePath());
                }
             }
             catch (SecurityException e)
             {
                LOG.severe("Security exception while creating directory: " +
                           toCreate.getAbsolutePath()                      +
                           " - "                                           +
                           e.getMessage());
             }
          }
       }

       /**
        * Cleans up newly created directories.
        */
       private void cleanupDirectories()
       {
          for (File dir : createdDirectories)
          {
             try
             {
                if (dir.exists() && dir.list().length == 0)
                {
                   dir.delete();
                }
             }
             catch (Exception e)
             {
                LOG.warning("Failed to delete directory: " + dir.getAbsolutePath());
             }
          }
       }

       /**
        * Helper method for creating the dynamic table classes jar/cache. Receives the
        * file/directory to be added to the jar, and the stream used to write to the jar.
        * Calls itself recursively for every directory in the source File, if that is
        * a directory. Otherwise, it adds the source File to the jar.
        *
        * @param   source
        *          The class file or directory containing class files that will be added to the jar.
        * @param   target
        *          The stream used to add files to the jar.
        * @param   packages
        *          The set to collect package names.
        *
        * @throws  IOException
        *          Exception that can be thrown by the JarOutputStream object, not handled here.
        */
       private void createCacheHelper(File source, JarOutputStream target, Set<String> packages)
       throws IOException
       {
          // +1, because of the following file separator
          String name = source.getPath().replace("\\", File.separator).substring(tempLibFilePath.length() + 1);
          if (source.isDirectory())
          {
              if (!name.endsWith(File.separator))
              {
                  name += File.separator;
              }
             if (existingEntries.add(name))
             {
                JarEntry entry = new JarEntry(name);
                entry.setTime(source.lastModified());
                target.putNextEntry(entry);
                target.closeEntry();
             }
              File[] files = source.listFiles();
              for (int i = 0; i < files.length; i++)
              {
                 createCacheHelper(files[i], target, packages);
              }
          }
          else
          {
             if (existingEntries.add(name))
             {
                JarEntry entry = new JarEntry(name);
                entry.setTime(source.lastModified());
                target.putNextEntry(entry);
                try (FileInputStream fis = new FileInputStream(source);
                     BufferedInputStream bis = new BufferedInputStream(fis))
                {
                   byte[] buffer = new byte[1024];
                   while (true)
                   {
                      int count = bis.read(buffer);
                      if (count == -1)
                      {
                         break;
                      }
                      target.write(buffer, 0, count);
                   }
                   target.closeEntry();
                }
             }
             int lastSlashIndex = name.lastIndexOf('/');
             if (lastSlashIndex != -1)
             {
                String packageName = name.substring(0, lastSlashIndex).replace('/', '.');
                packages.add(packageName);
             }
          }
      }

       /**
        * Writes the jar index to the jar file's META-INF directory.
        *
        * @param   jos
        *          The JarOutputStream to write the index to.
        * @param   packages
        *          The set of package names to include in the index.
        *
        * @throws  IOException
        *          Exception that can be thrown by the JarOutputStream object, not handled here.
        */
       private static void writeJarIndex(JarOutputStream jos, Set<String> packages)
       throws IOException
       {
          String indexListPath = "META-INF/INDEX.LIST";
          if (!existingEntries.contains(indexListPath))
          {
             JarEntry indexEntry = new JarEntry(indexListPath);
             jos.putNextEntry(indexEntry);
             jos.write("JarIndex-Version: 1.0\n\n".getBytes());
             jos.write("./\n".getBytes());

             for (String pkg : packages)
             {
                jos.write((pkg + "\n").getBytes());
             }
             jos.closeEntry();
             existingEntries.add(indexListPath);
          }
       }

       /**
        * Generate DMO interface, implementation class and mapping document and register the new DMO type.
        * If requested, also caches the new classes generated to disk.
        *
        * @param   builder
        *          Temporary table builder containing definitions of the target table.
        * @param   legacyTableName
        *          The legacy name of the table.
        * @param   legacyPeerName
        *          The legacy name of the table's peer.  May be {@code null}.
        * @param   isBeforeTable
        *          Flag indicating the DMO is being built for a before table.
        *
        * @return  <code>true</code> on success.
        */
       boolean createDynamicDMO(TempTableBuilder builder,
                                String           legacyTableName,
                                String           legacyPeerName,
                                boolean          isBeforeTable)
       {
          Context local = context.get();

          Collection<P2JField> allFields = builder.getFields();
          Collection<P2JIndex> allIndexes = builder.getIndexes();

          // check the cache first
          CacheKey cacheKey = new CacheKey(allFields, allIndexes);
          CacheData cacheData;
          boolean classLoadingFailed = false;

          synchronized (cacheLock)
          {
             cacheData = cache.get(cacheKey);
          }

          DmoMeta dmoMeta = null;
          Class<? extends DataModelObject> dmoIface = null;
          Class<? extends TempTableBuffer> dmoBufIface = null;
          Class<? extends Record> dmoClass = null;
          DMOInfo dmoInfo = null;
          Map<String, Class<?>> classes;
          Map<String, String> clsFields4GLtoORM;
          Map<String, String> clsFieldsORMTo4GL;
          ArrayList<P2JIndex> fwdIndexes = null;

          try
          {
             if (cacheData == null)
             {
                fwdIndexes = new ArrayList<>();
                Long tableId = null;
                List<String> persistentData = null;
                clsFields4GLtoORM = new HashMap<>();
                clsFieldsORMTo4GL = new HashMap<>();

                long fieldId = 1;
                for (P2JField field : allFields)
                {
                   String fieldName4GL = field.getName();
                   String fieldName4GLNorm = TextOps.rightTrimLower(fieldName4GL);
                   String fieldNameORM = ReservedProperty.isReservedProperty(fieldName4GL) ? fieldName4GL
                                                                                           : "field" + (fieldId++);
                   clsFields4GLtoORM.put(fieldName4GLNorm, fieldNameORM);
                   clsFieldsORMTo4GL.put(fieldNameORM, fieldName4GL);
                }

                if (usePersistentCache)
                {
                   persistentData = persistentCacheMap.get(cacheKey.toString());

                   if (persistentData != null && persistentData.size() >= 1)
                   {
                      try
                      {
                         tableId = Long.decode(persistentData.get(0));
                      }
                      catch (Exception e)
                      {
                         tableId = null;
                      }
                   }

                   if (tableId != null)
                   {
                      String ifaceName = "DynamicRecord" + tableId;
                      String sqlTableName = "dtt" + tableId;
                      ClassLoader loader = ClassLoader.getSystemClassLoader();

                      String fullIfaceName = DMO_BASE_PACKAGE +
                                             "." +
                                             DatabaseManager.TEMP_TABLE_SCHEMA +
                                             "." +
                                             ifaceName;
                      String fullIfaceBufName = fullIfaceName + "$Buf";

                      Class<?> clazz;
                      classes = new HashMap<>();
                      try
                      {
                         for (int i = 1; i < persistentData.size(); i++)
                         {
                            clazz = loader.loadClass(persistentData.get(i));
                            classes.put(persistentData.get(i), clazz);
                         }

                         dmoIface = (Class<? extends DataModelObject>) classes.get(fullIfaceName);
                         dmoBufIface = (Class<? extends TempTableBuffer>) classes.get(fullIfaceBufName);
                         dmoMeta = DmoMetadataManager.registerDmo(dmoIface);
                         extractIndexes(dmoMeta, fwdIndexes);
                         dmoClass = dmoMeta.getImplementationClass();
                         String fullImplClassName = dmoClass.getName();
                         classes.put(fullImplClassName, dmoClass);

                         dmoInfo = new DMOInfo(classes.values(), clsFields4GLtoORM, clsFieldsORMTo4GL);

                         CacheData cd = new CacheData(dmoInfo,
                                                      fullIfaceName,
                                                      fullImplClassName,
                                                      sqlTableName,
                                                      fwdIndexes,
                                                      classes);
                         cache.put(cacheKey, cd);
                      }
                      catch (Exception e)
                      {
                         classLoadingFailed = true;
                      }
                   }
                }

                if (tableId == null || classLoadingFailed)
                {
                   if (tableId == null)
                   {
                      tableId = tableCounter.getAndIncrement();
                   }
                   String sqlTableName = "dtt" + tableId;

                   String ifaceName = "DynamicRecord" + tableId;

                   Aast root = instance.generateSchemaAst(builder,
                                                          ifaceName,
                                                          sqlTableName,
                                                          legacyTableName,
                                                          clsFields4GLtoORM,
                                                          legacyPeerName,
                                                          isBeforeTable);

                   try
                   {
                      ConversionPool.Results results;

                      results = ConversionPool.runTask(ConversionProfile.P2O, root);
                      root = (Aast) results.getStoredObject("p2o.prog");
                      ConversionPool.runTask(ConversionProfile.P2O_POST, results, root);

                      root = (Aast) results.getStoredObject("p2o.peer");
                      root.putAnnotation("permanent", false);
                      root.putAnnotation("package", DMO_BASE_PACKAGE);

                      // generate interface Java ASTs
                      results = ConversionPool.runTask(ConversionProfile.JAVA_DMO, root);
                      Aast iface = (Aast) results.getStoredObject(DMO_IFACE_CATEGORY, ifaceName);
                      String fullIfaceName = DMO_BASE_PACKAGE +
                                             "." +
                                             DatabaseManager.TEMP_TABLE_SCHEMA +
                                             "." +
                                             ifaceName;
                      String fullIfaceBufName = fullIfaceName + "$Buf";

                      // generate bytecode from Java ASTs
                      results = ConversionPool.runTask(ConversionProfile.BREW_DMO_ASM, iface);

                      // even though we've already taken the hit of converting, give the cache one last
                      // try, to minimize the chance of duplicate classes
                      synchronized (cacheLock)
                      {
                         cacheData = cache.get(cacheKey);

                         if (cacheData == null)
                         {
                            // TODO: use a separate class loader for each table, so the classes can be
                            //       garbage collected when no longer in use
                            AsmClassLoader loader = systemInMemoryClassLoader;
                            List<String> keys;
                            List<String> persistentCacheEntryValue = null;
                            if (usePersistentCache && updatePersistentCache && !classLoadingFailed)
                            {
                               persistentCacheEntryValue = new ArrayList<>();
                               persistentCacheEntryValue.add(tableId.toString());
                            }
                            classes = new HashMap<>();
                            byte[] code;

                            // load interfaces first
                            keys = (List<String>) results.getStoredObject("iface.keys");
                            String name;
                            for (int i = 0; i < keys.size(); i++)
                            {
                               name = keys.get(i);
                               code = (byte[]) results.getStoredObject("ifaces", name);
                               // enable the following line for debugging
                               // new FileOutputStream(name + ".class").write(code);
                               Class<?> clazz = loader.loadClass(name, code);
                               classes.put(name, clazz);
                               if (usePersistentCache && updatePersistentCache && !classLoadingFailed)
                               {
                                  saveDynamicDmoToDisk(name, code);
                                  persistentCacheEntryValue.add(name);
                               }
                            }

                            dmoIface = (Class<? extends DataModelObject>) classes.get(fullIfaceName);
                            dmoBufIface = (Class<? extends TempTableBuffer>) classes.get(fullIfaceBufName);
                            Consumer<DmoClass> callback = null;
                            if (usePersistentCache && updatePersistentCache && !classLoadingFailed)
                            {
                               callback = (x) ->
                               {
                                  saveDynamicDmoToDisk(x.getInternalName(), x.getBytecode());
                               };
                               persistentCacheMap.put(cacheKey.toString(), persistentCacheEntryValue);
                            }
                            dmoMeta = DmoMetadataManager.registerDmo(dmoIface, callback);
                            extractIndexes(dmoMeta, fwdIndexes);
                            dmoClass = dmoMeta.getImplementationClass();
                            String fullImplClassName = dmoClass.getName();
                            classes.put(fullImplClassName, dmoClass);

                            dmoInfo = new DMOInfo(classes.values(), clsFields4GLtoORM, clsFieldsORMTo4GL);

                            CacheData cd = new CacheData(dmoInfo,
                                                         fullIfaceName,
                                                         fullImplClassName,
                                                         sqlTableName,
                                                         fwdIndexes,
                                                         classes);
                            cache.put(cacheKey, cd);
                         }
                      }
                   }
                   finally
                   {
                      // clean up after TRPL rules in the AST registry
                      String ifacePath = DMO_BASE_PACKAGE.replace('.', File.separatorChar) + File.separator +
                                         DatabaseManager.TEMP_TABLE_SCHEMA + File.separator +
                                         ifaceName + ".java";
                      AstManager.get().removeTree(ifacePath);
                   }
                }
             }

             if (cacheData != null)
             {
                dmoInfo = cacheData.getDMOInfo();
                String fullIfaceName = cacheData.getDMOIfaceName();
                String fullIfaceBufName = fullIfaceName + "$Buf";
                classes = cacheData.getClasses();
                dmoIface = (Class<? extends DataModelObject>) classes.get(fullIfaceName);
                dmoBufIface = (Class<? extends TempTableBuffer>) classes.get(fullIfaceBufName);
                dmoMeta = DmoMetadataManager.getDmoInfo(dmoIface);
             }

             local.registerDMOInfo(dmoIface, dmoInfo);
             builder.setDmoIface(dmoMeta, dmoIface, dmoBufIface);
             local.registerBuilder(builder);
          }
          catch (Exception e)
          {
             LOG.log(Level.SEVERE, e.getMessage(), e);
             ErrorManager.displayError("Error while generating dynamic table: " + e.getMessage());

             return false;
          }

          return true;
       }

       /**
        * Deregister the given temp-table builder from the DMO resource manager.
        *
        * @param   builder
        *          Builder to be deregistered.
        */
       void deregisterBuilder(TempTableBuilder builder)
       {
          dmoResourceManager.builderInactive(builder);
       }

       /**
        * Remove this dynamic temp-table from the DMO users.
        *
        * @param    builder
        *           Builder to be deregistered.
        */
       void deregisterDmoUser(TempTableBuilder builder)
       {
          Class<? extends DataModelObject> dmoIface = builder.getDMOInterface();
          context.get().dmoUsers.getOrDefault(dmoIface, Collections.emptySet()).remove(builder);
       }

       /**
        * Extracts the list of indexes from an interface into an array.
        *
        * @param   dmoMeta
        *          the metadata collection of the DMO interface.
        * @param   fwdIndexes
        *          The destination array where the index data will be stored.
        */
       private void extractIndexes(DmoMeta dmoMeta, ArrayList<P2JIndex> fwdIndexes)
       {
          fwdIndexes.clear();
          Iterator<P2JIndex> idxIter = dmoMeta.getDatabaseIndexes();
          idxIter.forEachRemaining(fwdIndexes::add);
       }

       /**
        * Generate AST that represents the target table and has the same structure as .p2o files from
        * the conversion stage.
        *
        * @param  builder
        *         Temporary table builder containing definitions of the target table.
        * @param  ifaceName
        *         The name of the target DMO interface.
        * @param  tableName
        *         The name of the target SQL table.
        * @param  legacyTableName
        *         The legacy (Progress) name of the table.
        * @param  fields4GLtoORM
        *         Map 4GL field names to ORM names.
        * @param  legacyPeerName
        *         The legacy name of the table's peer.  May be {@code null}.
        * @param  isBeforeTable
        *         Flag indicating the DMO is being built for a before table.
        *
        * @return the root node of the generated AST.
        */
       Aast generateSchemaAst(TempTableBuilder builder,
                              String ifaceName,
                              String tableName,
                              String legacyTableName,
                              Map<String, String> fields4GLtoORM,
                              String legacyPeerName,
                              boolean isBeforeTable)
       {
          Aast root = new ProgressAst(new CommonToken(ProgressParserTokenTypes.DATABASE,
                                                      DatabaseManager.TEMP_TABLE_SCHEMA));

          ProgressAst tableSchema = new ProgressAst(new CommonToken(ProgressParserTokenTypes.TEMP_TABLE,
                                                                    tableName));
          tableSchema.putAnnotation("class-name", ifaceName);
          tableSchema.putAnnotation("table-name", tableName);
          if (legacyPeerName != null)
          {
             tableSchema.putAnnotation((isBeforeTable ? "after-table" : "before-table"), legacyPeerName);
          }
          tableSchema.putAnnotation(P2OLookup.HISTORICAL, legacyTableName);
          root.addChild(tableSchema);

          if (!builder.isUndoable())
          {
             // TODO: if other properties must be added, extract this node outside if
             ProgressAst propsAst = new ProgressAst(new CommonToken(ProgressParserTokenTypes.PROPERTIES,
                                                                   "properties"));
             tableSchema.addChild(propsAst);

             propsAst.addChild(new ProgressAst(new CommonToken(ProgressParserTokenTypes.KW_NO_UNDO, "no-undo")));
          }

          // add field nodes to table
          Map<String, Aast> fieldAsts = new HashMap<>();
          for (P2JField field : builder.getFields())
          {
             String fieldName4GL = TextOps.rightTrimLower(field.getName());
             String fieldNameORM = fields4GLtoORM.get(fieldName4GL);
             ParmType type = field.getType();
             int progressType = propTypeToProgress.get(type);
             ProgressAst tableField = new ProgressAst(new CommonToken(progressType, fieldNameORM));
             fieldAsts.put(fieldName4GL, tableField);
             ReservedProperty resPropName = ReservedProperty.getByName(fieldName4GL);
             if (resPropName != null)
             {
                tableField.putAnnotation("__fieldid", (long) resPropName.propId);
             }
             if (field.getFormat() != null)
             {
                tableField.putAnnotation(P2OLookup.FORMAT, field.getFormat());
             }
             if (field.getLabel() != null)
             {
                tableField.putAnnotation(P2OLookup.LABEL, field.getLabel());
             }
             if (field.getColumnLabel() != null)
             {
                tableField.putAnnotation(P2OLookup.COLUMN_LABEL, field.getColumnLabel());
             }
             String cp = field.getCodePage();
             if (cp != null && !cp.trim().equals(""))
             {
                tableField.putAnnotation("column-codepage", cp);
             }

             if (field.isSerializeHidden())
             {
                tableField.putAnnotation("serialize-hidden", field.isSerializeHidden());
             }
             if (field.getSerializeName() != null)
             {
                tableField.putAnnotation("serialize-name", field.getSerializeName());
             }
             tableField.putAnnotation("xml-data-type", field.getXmlDataType());
             tableField.putAnnotation("xml-node-type", field.getXmlNodeType());
             tableField.putAnnotation("xml-node-name", field.getXmlNodeName());

             tableField.putAnnotation("valexp", field.getValidateExpression());
             tableField.putAnnotation("valmsg", field.getValidateMessage());
             tableField.putAnnotation("help", field.getHelp());

             tableField.putAnnotation(P2OLookup.HISTORICAL, field.getName());
             if (field.isCaseSensitive())
             {
                tableField.putAnnotation(P2OLookup.CASE_SENSITIVE, field.isCaseSensitive());
             }
             if (field.getScale() > 0)
             {
                tableField.putAnnotation(P2OLookup.DECIMALS, (long) field.getScale());
             }
             if (field.getMaxWidth() > 0)
             {
                tableField.putAnnotation("max_wid", (long) field.getMaxWidth());
             }

             long extent = field.getExtent();
             if (extent > 0)
             {
                tableField.putAnnotation(P2OLookup.EXTENT, extent);
             }

             tableSchema.addChild(tableField);

             // add initial value node to field
             BaseDataType initial = field.getInitial();
             if (initial != null && !((initial instanceof Text) && TextOps.lengthOf((Text) initial) == 0))
             {
                Integer valType;
                if (initial.isUnknown() || (initial instanceof Text &&
                                            ResourceIdHelper.UNKNOWN_STRING.equals(initial.toStringMessage())))
                {
                   valType = ProgressParserTokenTypes.UNKNOWN_VAL;
                }
                else
                {
                   switch (type)
                   {
                      case CHAR:
                         valType = ProgressParserTokenTypes.STRING;
                         break;
                      case INT64:
                      case INT:
                      case RECID:
                         valType = ProgressParserTokenTypes.NUM_LITERAL;
                         break;
                      case DEC:
                         valType = ProgressParserTokenTypes.DEC_LITERAL;
                         break;
                      case LOG:
                         valType = new logical(initial).equals(logical.TRUE)
                                   ? ProgressParserTokenTypes.BOOL_TRUE
                                   : ProgressParserTokenTypes.BOOL_FALSE;
                         break;
                      case DATE:
                         valType = ProgressParserTokenTypes.DATE_LITERAL;
                         // TODO: what about builtins (today and now)? The above will not handle this correctly --
                         //       we already have a date object at this point and the builtin info is lost
                         break;
                      case DT:
                         valType = ProgressParserTokenTypes.DATETIME_LITERAL;
                         break;
                      case DTTZ:
                         valType = ProgressParserTokenTypes.DATETIME_TZ_LITERAL;
                         break;
                      case RAW:
                         valType = null;
                         break;
                      // TODO: other data types?
                      default:
                         valType = null;
                         if (LOG.isLoggable(Level.WARNING))
                         {
                            LOG.warning("Unsupported initial value literal type in dynamic temp table field: " +
                                        type + "; (" + legacyTableName + "." + field.getName() + ")");
                         }
                   }
                }

                String valStr;
                switch (type)
                {
                   case CHAR:
                      valStr = initial.toStringMessage();
                      valStr = character.javaRuntimeToProgressSourceString(valStr);
                      break;
                   default:
                      valStr = initial.toStringMessage();
                }

                if (valType != null)
                {
                   ProgressAst initNode = new ProgressAst(new CommonToken(ProgressParserTokenTypes.KW_INIT, ""));
                   tableField.addChild(initNode);
                   ProgressAst initVal = new ProgressAst(new CommonToken(valType, valStr));
                   initNode.addChild(initVal);
                }
             }
          }

          // add index nodes to table
          for (P2JIndex index : builder.getIndexes())
          {
             String indexName = TextOps.rightTrimLower(index.getName());
             ProgressAst indexAst = new ProgressAst(new CommonToken(ProgressParserTokenTypes.INDEX, indexName));
             tableSchema.addChild(indexAst);

             String legacyName = TextOps.rightTrimNative(index.getLegacyName());
             if (legacyName != null)
             {
                indexAst.putAnnotation(P2OLookup.HISTORICAL, legacyName);
             }
             if (index.isPrimary())
             {
                indexAst.putAnnotation(P2OLookup.PRIMARY, true);
             }

             if (index.isUnique())
             {
                indexAst.putAnnotation(P2OLookup.UNIQUE, true);
             }

             if (index.isWord())
             {
                indexAst.putAnnotation(P2OLookup.WORD, true);
             }

             // add index fields to index
             ArrayList<P2JIndexComponent> comps = index.components();
             for (int i = 0; i < comps.size(); i++)
             {
                P2JIndexComponent comp = comps.get(i);
                ProgressAst compAst = new ProgressAst(
                   new CommonToken(ProgressParserTokenTypes.INDEX_FIELD, comp.getNormalizedName()));
                indexAst.addChild(compAst);

                if (comp.isDescending())
                {
                   compAst.putAnnotation(P2OLookup.DESCEND, true);
                }

                // TODO: what about ABBREVIATED and CASE-SENSITIVE options (not handled as part of
                //       TRPL schema fixups today)?
             }
          }

          assignIds(root, 1);

          // now assign each index field's refid to the corresponding field's ID
          Aast indexAst = tableSchema.getImmediateChild(ProgressParserTokenTypes.INDEX, null);
          while (indexAst != null)
          {
             Aast compAst = indexAst.getImmediateChild(ProgressParserTokenTypes.INDEX_FIELD, null);
             while (compAst != null)
             {
                String fieldName4GL = compAst.getText();
                Aast fieldAst = fieldAsts.get(fieldName4GL);
                compAst.putAnnotation("refid", fieldAst.getId());
                compAst = indexAst.getImmediateChild(ProgressParserTokenTypes.INDEX_FIELD, compAst);
             }

             indexAst = tableSchema.getImmediateChild(ProgressParserTokenTypes.INDEX, indexAst);
          }

          return root;
       }

       /**
        * Class which contains information about a dynamically define DMO, including the compiled
        * interface and implementation classes, and mappings between the legacy and converted fields.
        */
       private static class DMOInfo
       {
          /** All classes loaded to support this DMO */
          private final Collection<Class<?>> loadedClasses;

          /** Mapping of legacy, 4GL field names to converted, property names */
          private final Map<String, String> legacyToDmoNames;

          /** Mapping of converted, property names to legacy, 4GL field names */
          private final Map<String, String> dmoToLegacyNames;

          /**
           * Constructor.
           *
           * @param   loadedClasses
           *          All classes loaded to support this DMO.
           * @param   legacyToDmoNames
           *          Mapping of legacy, 4GL field names to converted, property names.
           * @param   dmoToLegacyNames
           *          Mapping of converted, property names to legacy, 4GL field names.
           */
          DMOInfo(Collection<Class<?>> loadedClasses,
                  Map<String, String> legacyToDmoNames,
                  Map<String, String> dmoToLegacyNames)
          {
             this.loadedClasses = loadedClasses;
             this.legacyToDmoNames = legacyToDmoNames;
             this.dmoToLegacyNames = dmoToLegacyNames;
          }

          /**
           * Get all classes loaded to support this DMO.
           *
           * @return  See above.
           */
          Collection<Class<?>> getLoadedClasses()
          {
             return loadedClasses;
          }

          /**
           * Get the mapping of legacy, 4GL field names to converted, property names.
           *
           * @return  See above.
           */
          Map<String, String> getLegacyToDmoNames()
          {
             return legacyToDmoNames;
          }

          /**
           * Get the mapping of converted, property names to legacy, 4GL field names.
           *
           * @return  See above.
           */
          Map<String, String> getDmoToLegacyNames()
          {
             return dmoToLegacyNames;
          }
       }

       /**
        * A hashable key that encapsulates the characteristics which uniquely define the structure of
        * a dynamic temp-table; namely, its field and index definitions.
        */
       private static class CacheKey
       {
          /** List of field definitions for a dynamic temp-table */
          private final List<P2JField> fields;

          /** List of index definition wrappers for a dynamic temp-table */
          private final List<IndexKey> indexes;

          /**
           * Constructor.
           *
           * @param   fields
           *          Collection of field definitions for a dynamic temp-table.
           * @param   indexes
           *          Collection of index definitions for a dynamic temp-table.
           */
          CacheKey(Collection<P2JField> fields, Collection<P2JIndex> indexes)
          {
             this.fields = new ArrayList<>(fields);
             this.indexes = new ArrayList<>(indexes.size());
             for (P2JIndex next : indexes)
             {
                this.indexes.add(new IndexKey(next));
             }
          }

          /**
           * Use the field and index definitions in this key to calculate a hash code. This
           * implementation is consistent with this class' implementation of {@link #equals(Object)}.
           *
           * @return  Hash code.
           */
          @Override
          public int hashCode()
          {
             int result = 17;
             result = 37 * result + fields.hashCode();
             result = 37 * result + indexes.hashCode();

             return result;
          }

          /**
           * Use the field and index definitions in this key to determine equivalence with the given
           * object. This implementation is consistent with this class' implementation of {@link
           * #hashCode()}.
           *
           * @return  {@code true} if this object is equivalent with {@code o}, else {@code false}.
           */
          @Override
          public boolean equals(Object o)
          {
             if (this == o)
             {
                return true;
             }

             if (o instanceof CacheKey)
             {
                CacheKey that = (CacheKey) o;

                return (this.fields.equals(that.fields) && this.indexes.equals(that.indexes));
             }

             return false;
          }

          /**
           * To string method concatenating the string values of all fields and indexes.
           *
           * @return  string value of this CacheKey object.
           */
          @Override
          public String toString()
          {
             String res = "{ fields : [";
             for (int i = 0; i < fields.size(); i++)
             {
                res += fields.get(i).toStringComplete() + ",";
             }
             res += "], indexes : [";
             for (int i = 0; i < indexes.size(); i++)
             {
                res += indexes.get(i).toString() + ",";
             }
             res += "]}";
             return res;
          }

          /**
           * Instances of this class are used to wrap instances of {@link P2JIndex}, for purposes of
           * producing hash codes suitable for using instances of the enclosing class as hash map
           * keys. This class provides stricter implementations of <code>hashCode</code> and
           * <code>equals</code> than those provided by <code>P2JIndex</code> itself.
           */
          private static class IndexKey
          {
             /** The wrapped instance of an index definition */
             private final P2JIndex index;

             /**
              * Constructor.
              *
              * @param   index
              *          Index definition to be wrapped.
              */
             IndexKey(P2JIndex index)
             {
                this.index = index;
             }

             /**
              * Use the content of the wrapped index definition to produce a hash code. This is used
              * instead of {@link P2JIndex#hashCode()}, because the latter implementation of this
              * method is special-purpose, taking only the names of index components into account and
              * ignores other index metadata.
              *
              * @return  Hash code.
              */
             @Override
             public int hashCode()
             {
                int result = 17;

                result = 37 * result + index.getName().hashCode();
                result = 37 * result + (index.isPrimary() ? 1 : 0);
                result = 37 * result + (index.isUnique() ? 1 : 0);
                result = 37 * result + (index.isWord() ? 1 : 0);

                ArrayList<P2JIndexComponent> comps = index.components();
                for (int i = 0; i < comps.size(); i++)
                {
                   P2JIndexComponent comp = comps.get(i);
                   result = 37 * result + comp.hashCode();
                }

                return result;
             }

             /**
              * Overrides the default implementation to provide an implementation consistent with
              * {@link #hashCode()}.
              *
              * @return  {@code true} if this object is equivalent with {@code o}, else {@code false}.
              */
             @Override
             public boolean equals(Object o)
             {
                if (this == o)
                {
                   return true;
                }

                if (o instanceof IndexKey)
                {
                   IndexKey that = (IndexKey) o;

                   if (!this.index.getName().equals(that.index.getName()) ||
                       this.index.isPrimary() != that.index.isPrimary()   ||
                       this.index.isUnique() != that.index.isUnique()     ||
                       this.index.isWord() != that.index.isWord()         ||
                       this.index.size() != that.index.size())
                   {
                      return false;
                   }

                   ArrayList<P2JIndexComponent> comps = index.components();
                   for (int i = 0; i < comps.size(); i++)
                   {
                      P2JIndexComponent comp = comps.get(i);
                      if (!comp.equals(that.index.getComponent(i)))
                      {
                         return false;
                      }
                   }

                   return true;
                }

                return false;
             }

             /**
              * To string method returning the string value of this objects P2JIndex field.
              *
              * @return  string value of this IndexKey object.
              */
             @Override
             public String toString()
             {
                return index.toString();
             }
          }
       }

       /**
        * DMO data to be cached for a dynamically defined temp-table.
        */
       private static class CacheData
       {
          /** DMO info object */
          private final DMOInfo dmoInfo;

          /** Fully qualified DMO interface name */
          private final String dmoIfaceName;

          /** Fully qualified DMO implementation class name */
          private final String dmoClassName;

          /** Database table name */
          private final String sqlTableName;

          /** List of index definitions using converted names */
          private final ArrayList<P2JIndex> fwdIndices;

          /** Map of fully qualified class names to classes */
          private final Map<String, Class<?>> classes;

          /**
           * Constructor.
           *
           * @param   dmoInfo
           *          DMO info object.
           * @param   dmoIfaceName
           *          Fully qualified DMO interface name.
           * @param   dmoClassName
           *          Fully qualified DMO implementation class name.
           * @param   sqlTableName
           *          Database table name.
           * @param   fwdIndices
           *          List of index definitions using converted names.
           * @param   classes
           *          Map of fully qualified class names to classes.
           */
          CacheData(DMOInfo dmoInfo,
                    String dmoIfaceName,
                    String dmoClassName,
                    String sqlTableName,
                    ArrayList<P2JIndex> fwdIndices,
                    Map<String, Class<?>> classes)
          {
             this.dmoInfo = dmoInfo;
             this.dmoIfaceName = dmoIfaceName;
             this.dmoClassName = dmoClassName;
             this.sqlTableName = sqlTableName;
             this.fwdIndices = fwdIndices;
             this.classes = classes;
          }

          /**
           * Get DMO information.
           *
           * @return  See above.
           */
          DMOInfo getDMOInfo()
          {
             return dmoInfo;
          }

          /**
           * Get fully qualified DMO interface name.
           *
           * @return  See above.
           */
          String getDMOIfaceName()
          {
             return dmoIfaceName;
          }

          /**
           * Get fully qualified DMO implementation class name.
           *
           * @return  See above.
           */
          String getDMOClassName()
          {
             return dmoClassName;
          }

          /**
           * Get database table name.
           *
           * @return  See above.
           */
          String getSqlTableName()
          {
             return sqlTableName;
          }

          /**
           * Get list of index definitions using converted names.
           *
           * @return  See above.
           */
          ArrayList<P2JIndex> getFwdIndices()
          {
             return fwdIndices;
          }

          /**
           * Get map of fully qualified class names to classes.
           *
           * @return  See above.
           */
          Map<String, Class<?>> getClasses()
          {
             return Collections.unmodifiableMap(classes);
          }
       }

       /**
        * Context local information, used to clean up resources when the context ends and to respond
        * to queries about temp-table resources in use.
        * <p>
        * TODO: currently, the cleanup code is based on an incorrect assumption that the context's
        * state is reset more frequently than it actually is, AND, in some cases, we are cleaning up
        * resources prematurely, causing persistence errors when detached objects in an old Hibernate
        * session are reattached to a new one. The temporary workaround for this is a high cache
        * limit, to prevent the cache from expiring elements.
        */
       private class Context
       implements Finalizable
       {
          /** Map of DMO interfaces to sets of users of these interfaces */
          private final Map<Class<?>, Set<TempTableBuilder>> dmoUsers = new IdentityHashMap<>();

          /** Map of DMO interfaces to DMO information objects */
          private final Map<Class<? extends DataModelObject>, DMOInfo> dmoMap = new IdentityHashMap<>();

          /** Temp-table builders which are no longer in use in this context */
          private List<TempTableBuilder> finishedBuilders = null;

          /**
           * Constructor. Registers this object as a global finalizable.
           */
          private Context()
          {
             TransactionManager.registerFinalizable(this, true);
          }

          /**
           * Clean up local context - drop tables and DMO definitions.
           */
          public void finished()
          {
             if (dmoUsers.isEmpty())
             {
                return;
             }

             if (finishedBuilders == null)
             {
                finishedBuilders = new ArrayList<>();
             }

             for (Set<TempTableBuilder> set : dmoUsers.values())
             {
                finishedBuilders.addAll(set);
             }

             for (TempTableBuilder builder : finishedBuilders)
             {
                Class<? extends DataModelObject> dmoIface = builder.getDMOInterface();
                Set<TempTableBuilder> users = dmoUsers.get(dmoIface);
                if (users == null)
                {
                   continue;
                }

                users.remove(builder);
             }
          }

          /**
           * No-op.
           */
          public void deleted()
          {
          }

          /**
           * No-op.
           */
          public void iterate()
          {
          }

          /**
           * No-op.
           */
          public void retry()
          {
          }

          /**
           * Clean up DMO resources no longer needed by temp-table builders in this context.
           */
          private void cleanup()
          {
             if (finishedBuilders == null)
             {
                return;
             }

             // DMO interfaces which already have been cleaned up
             Set<Class<?>> processedDMOs = Collections.newSetFromMap(new IdentityHashMap<>());

             for (TempTableBuilder builder : finishedBuilders)
             {
                try
                {
                   Class<? extends DataModelObject> dmoIface = builder.getDMOInterface();
                   if (dmoIface == null || !processedDMOs.add(dmoIface))
                   {
                      continue;
                   }

                   dmoUsers.remove(dmoIface);
                   dmoMap.remove(dmoIface);
                }
                finally
                {
                   dmoResourceManager.builderInactive(builder);
                }
             }

             finishedBuilders = null;
          }

          /**
           * Register a dynamic temp-table builder in this context.
           *
           * @param   builder
           *          Temp-table builder to be registered.
           */
          private void registerBuilder(TempTableBuilder builder)
          {
             dmoResourceManager.builderActive(builder);

             Class<? extends DataModelObject> dmoIface = builder.getDMOInterface();
             Set<TempTableBuilder> users = dmoUsers.computeIfAbsent(dmoIface, k -> new HashSet<>());

             users.add(builder);
          }

          /**
           * Get the DMO information object associated with the given DMO interface.
           *
           * @param   dmoIface
           *          DMO interface.
           *
           * @return  See above.
           */
          private DMOInfo getDMOInfo(Class<? extends DataModelObject> dmoIface)
          {
             return dmoMap.get(dmoIface);
          }

          /**
           * Associate a DMO information object with the given DMO interface.
           *
           * @param   dmoIface
           *          DMO interface.
           * @param   dmoInfo
           *          DMO information object.
           *
           * @return  {@code true} if {@code dmoIface} was not already associated with a
           *          different object, else {@code false}.
           */
          private boolean registerDMOInfo(Class<? extends DataModelObject> dmoIface, DMOInfo dmoInfo)
          {
             return dmoMap.put(dmoIface, dmoInfo) == null;
          }
       }

       /**
        * An instance of this class manages resources requiring cleanup, across all contexts. This
        * object is a registered cache listener, receiving notifications when elements are expired
        * from the cache. These are tracked until the are no longer actively in use by any context,
        * then their associated resources are cleaned/reclaimed.
        * <p>
        * TODO: this cleanup currently is broken, as it can release resources that are still being
        * used by a Hibernate session. The temporary workaround is to define a high cache limit,
        * but this needs to be fixed.
        */
       private class DMOResourceManager
       implements CacheExpiryListener<CacheKey, CacheData>
       {
          /** Temp-table builders which are currently active across all contexts */
          private final Set<TempTableBuilder> activeBuilders = new HashSet<>();

          /** Data which has expired from the cache, but has not yet been released */
          private final Set<CacheData> expiredCacheData = new HashSet<>();

          /**
           * Constructor which registers with the dynamic DMO cache.
           */
          DMOResourceManager()
          {
    //         cache.addCacheExpiryListener(this);
          }

          /**
           * This method is invoked when the DMO cache has expired one or more elements. We use it to
           * know when a DMO's mapping can be deregistered with the Hibernate session factory and the
           * associated DMO interface and implementation class(es) can be released.
           *
           * @param   event
           *          An object containing information about the event.
           */
          public void cacheExpired(CacheExpiryEvent<CacheKey, CacheData> event)
          {
             Map<CacheKey, CacheData> expired = event.getExpiredEntries();
             for (CacheData data : expired.values())
             {
                expiredCacheData.add(data);
             }
          }

          /**
           * Track the given temp-table builder as being in use.
           *
           * @param   builder
           *          Temp-table builder to track.
           */
          void builderActive(TempTableBuilder builder)
          {
             synchronized (cacheLock)
             {
                activeBuilders.add(builder);
             }
          }

          /**
           * Stop tracking the given temp-table builder as being in use.
           *
           * @param   builder
           *          Temp-table builder to stop tracking.
           */
          void builderInactive(TempTableBuilder builder)
          {
             synchronized (cacheLock)
             {
                activeBuilders.remove(builder);
             }
          }

          /**
           * Release resources which have expired from the dynamic DMO cache, which are no longer in
           * active use. This involves deregistering DMO types with Hibernate and releasing the DMO
           * interface and class(es) that have been compiled for a dynamic temp-table.
           * <p>
           * TODO: this cleanup currently is broken, as it can release resources that are still being
           * used by a Hibernate session. Also, if there are expired cache data, but they are still
           * pinned by active builders, this method could be expensive to execute every time a new
           * table is defined. Better to run it periodically in a dedicated thread. The temporary
           * workaround is to define a high cache limit, but this needs to be fixed.
           */
          void releasePendingResources()
          {
             // TODO: probably no longer needed, or if needed, needs re-implementation
             /*
             synchronized (cacheLock)
             {
                if (expiredCacheData.isEmpty())
                {
                   return;
                }

                // collect all expired data resources
                Map<Class<?>, CacheData> deadCacheData = new HashMap<>();
                for (CacheData data : expiredCacheData)
                {
                   Map<String, Class<?>> classMap = data.getClasses();
                   String dmoIfaceName = data.getDMOIfaceName();
                   Class<?> dmoIface = classMap.get(dmoIfaceName);
                   deadCacheData.put(dmoIface, data);
                }

                // collect all currently active DMO interfaces and PersistenceClass instances
                Map<Class<?>, PersistentClass> pinnedResources = new HashMap<>();
                for (TempTableBuilder ttb : activeBuilders)
                {
                   pinnedResources.put(ttb.getDMOInterface(), ttb.getPersistentClass());
                }
                Set<Class<?>> pinnedDMOs = pinnedResources.keySet();

                // remove all the expired entries which still are pinned by active builders
                deadCacheData.keySet().removeAll(pinnedDMOs);

                MutableSessionFactoryImpl sessionFactory = (MutableSessionFactoryImpl)
                   DatabaseManager.getSessionFactory(DatabaseManager.TEMP_TABLE_DB);
                for (CacheData data : deadCacheData.values())
                {
                   // release Hibernate resources
                   String dmoImplName = data.getDMOClassName();
                   PersistentClass pc = cfg.getClassMapping(dmoImplName); // caller has cfg lock
                   cfg.remove(pc);
                   sessionFactory.remove(pc);

                   // release class resources
                   DMOInfo info = data.getDMOInfo();
                   Collection<Class<?>> loadedClasses = info.getLoadedClasses();
                   for (Class<?> clz : loadedClasses)
                   {
    //                  systemInMemoryClassLoader.deregister(clz.getName());
                   }

                   // remove this item from the set of expired entries requiring cleanup
                   expiredCacheData.remove(data);
                }
             }
             */
          }
       }
    }