TempTableBuilder.java

/*
** Module   : TempTableBuilder.java
** Abstract : Implementation of the dynamic features for constructing temp-tables
**            (attributes, methods and constructors).
**
** Copyright (c) 2013-2025, Golden Code Development Corporation.
**
** -#- -I- --Date-- ---------------------------------------Description---------------------------------------
** 001 OM  20130129 Created initial version.
** 002 VMN 20130217 Added support for handle-based attributes:
**                  ERROR (KW_ERROR)
** 003 CA  20130221 Added ADM-DATA, UNIQUE-ID, HandleChain and Deletable support.
** 004 CA  20130311 Added missing ADD-NEW-FIELD and ADD-NEW-INDEX API versions, using wrapper 
**                  types. Note that the conversion will use only the APIs with the wrapper type
**                  parameters; the other APIs (using non-wrapper parameters) are removed.
** 005 CS  20130322 Made the class non-abstract and added missing method stubs. 
** 006 CA  20130603 resourceDelete must return a boolean value.
** 006 SVL 20130624 Some methods were implemented.
** 007 CA  20130918 Added widget-pool support. The pool name must be validated before anything
**                  else, and a null name means the default pool is used.
** 008 SVL 20130822 Some methods and attributes were implemented.
** 009 CA  20131021 Adds DYNAMIC attribute support. Adds "asHandle" API to build a TempTable from
**                  a buffer associated with a temp-table.
** 010 SVL 20131115 TempTableBuilder now extends AbstractTempTable, implemented
**                  getDMOClass/Interface.
** 011 CA  20131013 Added implicit resource deletion for static temp-table cases (the static
**                  resource gets deleted ONLY when its INSTANTIATING-PROCEDURE gets deleted).
** 012 HC  20131215 Implemented ADM-DATA and UNIQUE-ID attribute.
** 013 SVL 20140106 Partially implemented copyTempTable, some error handling refactoring,
**                  implemented getExistingFields/Indexes, ERROR-STRING attribute moved to the
**                  base class.
** 014 SVL 20140123 Implemented getExistingFields/Indexes, some fixes and refactoring.
** 015 SVL 20140124 Minor fixes and refactoring.
** 016 OM  20140123 Passed the legacy name of the table to createDynamicDMO when preparing 
**                  temp-tables.
** 017 SVL 20140205 Moved some functionality to the parent class, including COPY-TEMP-TABLE.
** 018 SVL 20140221 Changes related to table deletion.
** 019 SVL 20140303 Added asTempTable.
** 020 OM  20140411 Added case-sensitivity support for character fields.
** 021 VMN 20140427 Added enhance schema name conversion support for hint "escape" attribute.
** 022 CA  20140526 handle unwrap API were switched to instance method calls.
** 023 VMN 20140715 Fixed format validation for date type.
** 024 SVL 20140709 Added createFromTableWrapper. Fixed abend from H020.
** 025 VMN 20140910 Added processing default formats for ADD-NEW-FIELD.
** 026 ECF 20140918 Changed to accommodate caching and memory optimization in DynamicTablesHelper.
**                  Requires further work to ensure proper resource cleanup (i.e., in forceClear).
** 027 VMN 20141013 Fixes for date format string validation on adding new field.
** 028 OM  20150303 Delete and removed default buffer from HandleChain when tables is cleared.
** 029 ECF 20150430 Deregister this builder with DynamicTablesHelper once its resources have been
**                  deleted.
** 030 OM  20151026 Moved setCanUndo() method and undoable field to super class.
** 031 CA  20151230 When a temp-table gets deleted, all its associated buffers must be deleted, 
**                  too.
** 032 EVL 20160223 Javadoc fixes to make compatible with Oracle Java 8 for Solaris 10.
** 033 ECF 20160225 Changes required by new PropertyHelper implementation.
** 034 IAS 20160511 Create temporary table LIKE a permanent denormalized one
** 035 SVL 20170809 Refactoring due to some functions are now used by ADD-CALC-COLUMN.
** 036 OM  20190405 Renamed createTableLike() to createLike().
** 037 OM  20190510 Clarified generation of methods and attributes for variables with BUFFER /
**                  TEMP-TABLE ABL namespaces specified.
** 038 SVL 20190614 Support for CLOB code pages.
** 039 OM  20190514 Added P11.7 syntax support for TEMP-TABLE-PREPARE.
** 040 HC  20190719 Fixed an abend during temp-table resource cleanup.
**     OM  20190721 Added validation of NAME attribute.
**     CA  20190722 Added object type.
**                  A table is 'postponedDelete' if is attempted to be deleted while referenced by
**                  a BY-REFERENCE handle.
**     CA  20190731 Added createRemoteTable.
**     EVL 20190809 Added NPE protection to createFromTableWrapper().
**     CA  20190812 Ensure the PropertyHelper caches are loaded, even if the table has never been
**                  used via a DMO in FWD.
** 041 SVL 20190801 Added getExistingTempTable. CODEPAGE-related refactoring.
** 042 CA  20191203 The field's help and serialize options must be used to uniquely identify it.
** 043 CA  20200427 Fixed an issue with an empty string definition, when creating a remote table.
** 044 ECF 20200906 New ORM implementation.
** 045 AIL 20200828 Ensure the table metadata is loaded when getting existing field.
** 046 AIL 20200909 Added back table metadata load related changes as they were dropped after rebase.
** 047 AIL 20200915 Moved prepared method to superclass.
**     OM  20200919 Improved access to dmo metadata.
**     CA  20200922 Fixed tempTablePrepareImpl - in case of a before-table, the hidden fields and the rowState
**                  index must always be added.
**     OM  20200924 Objects of this class carry multiple information to avoid map lookups for them.
**     OM  20201001 Improved DMO manipulation performance by caching slow Property annotation access.
**     OM  20201002 Use DmoMeta cached information instead of map lookups.
**     SVL 20201030 Reflect extension of PropertyDefinition.
**     SVL 20201110 Manage BufferField handles in addFieldLike.
**     OM  20201120 Renamed NAMESPACE-URI and NAMESPACE-PREFIX setters to fit buffer attributes naming
**                  convention. Allowed Object fields to be added to TEMP-TABLEs. Added CLEAR state
**                  awareness. Exposed addField() to XML deserializer.
**     OM  20201202 Avoided conditions to be generated while encountered exceptions in debug mode.
**         20210106 Dropped 2nd parameter of TEMP-TABLE-PREPARE.
**     OM  20210115 The before-table is automatically created when needed.
**     VVT 20210317 validateFormat() updated for #4880: No DisplayFormatParsingException exist in the system
**                  anymore.
**     OM  20210115 Large object fields cannot be defined in UNDOable Temp-Tables.
**     OM  20210404 Small javadoc update.
**     EVL 20210408 Fix for 4GL field name case sensitivity issue in getCodePage() method.  The name should
**                  be normalized to avoid data misses.
**     ECF 20210506 Use RecordBuffer.getDmoInfo() getter instead of direct field access to prevent
**                  NPE in proxy case.
**     ECF 20210511 Replaced DynamicTablesHelper.normalizeName with TextOps.rightTrimLower.
**     CA  20210629 Fixed word indexes when importing a dataset serialized with its schema.
**     CA  20210823 Fixed extent field check - extent 1 is allowed.
**                  Fixed case of where extent was forced to 1 instead of 0, for scalar fields, in buildField
**                  and getExistingField.
**     OM  20210930 The name of dynamic temp-tables can start with underscore.
**     ECF 20211202 Use shared collection of P2JField objects from DmoMeta, rather than creating new copies
**                  on the fly. Fixed initial value handling for temp-tables.
**     OM  20211020 Added _DATASOURCE_ROWID property.
**     CA  20211222 Fixed memory leaks: the before-table for a dynamic temp-table must get deleted when its
**                  after table gets deleted; remove the dynamic temp-table from the DMO users, when the
**                  dynamic temp-table gets deleted.
**     OM  20220120 Allowed labelStr and columnLabelStr to remain null if they were not specified.
**     CA  20221006 Added JMX instrumentation for dynamic temp-table prepare. Refs #6814
**                  Do not access tableHandle() in the FWD runtime, get the TempTable instance directly.
**                  Refs #6826
**     OM  20211110 getExistingFields() returns the original fields, not denormalized.
**     BS  20221129 direct calls to ErrorManager.displayError replaced with ErrorManager.recordOrShowError.
**     OM  20221220 Avoided using time-consuming APIs from TableMapper.
**     CA  20220531 Ensure the CREATE-LIKE source before-table is created, too, on TEMP-TABLE-PREPARE.
**     CA  20220607 ADD-NEW-FIELD ignores unknown, 0 or 1 extent value.
**     CA  20220630 Fixed cleanup/delete of a TEMP-TABLE with postponed delete (must go through the normal
**                  resource delete, and not bypass it via 'forceDelete').
**     OM  20220706 Added runtime implementation of PRIMARY attribute and fixed CREATE-LIKE method.
**     OM  20220914 Added MAX-WIDTH support for CHARACTER fields.
**     IAS 20221003 Fixed default value of the PRIMARY attribute
**     IAS 20221014 Added COMHANDLE to the 'datatypesMatchLen' map
**     IAS 20221029 Added support for missed properties.
**     IAS 20221104 Added indexes parsing for input remote table.
**     IAS 20221108 Replaced static 'createRemoteTable' with a regular one.
**     CA  20221108 BEFORE-TABLE's name is always trimmed to max 32 chars length.
**     IAS 20221111 Fixed 'schemaMarshalLevel' support.
**     CA  20221121 For a dynamic TEMP-TABLE, if the NAME attribute is changed, then the default-buffer's NAME
**                  must be changed, too.
**     SVL 20230110 P2JIndex.components() provides direct access to the components array in order to improve
**                  performance.
**     HC  20230116 Replaced some handle usages with the actual wrapped resources for
**                  performance.
**     CA  20230116 Avoid using handle.unwrap, handle.getReference or other BDT usage from within FWD runtime.
**     HC  20230119 Eliminated some of the uses of String.toUpperCase and/or String.toLowerCase for
**                  performance.
**     CA  20230206 When creating a dynamic 'like' temp-table, where the source is a temp-table handle, do not
**                  check the stack to retrieve the indexes, but instead use the actual temp-table reference.
**     OM  20230214 Added clarification comment in createTableLikeImpl().
** 048 CA  20230303 When deleting a temp-table, the delete can be vetoed if any buffer for the temp-table is
**                  associated with an OUTPUT dataset.  Refactored to move the resource-specific code in a 
**                  'allowDelete' method.
** 049 CA  20230215 Improved other cases to reduce BDT usage from within FWD runtime.
**     CA  20230221 Javadoc fixes.
** 050 IAS 20240426 Fixed the indexes order.
** 051 IAS 20230524 Added 'Override' annotaton to 'getFields()' and 'readTable' methods.
** 052 CA  20230724 Further reduce context-local usage.
** 053 IT  20230809 temp-table member of a dataset cannot be delete  without deleting the dataset first.
** 054 CA  20230917 Backed out H053, the solution is to use 'allowDelete' called for an explit DELETE OBJECT
**                  statement.
** 055 AI  20231122 Used dmoInfo.getFieldInfo instead of TableMapper.getPropertyName for getExistingField().
**         20231127 Refactored getExistingField().
** 056 AI  20231218 Removed the use of getFieldInfo and replaced with dmoMeta.byLegacyName().
** 057 HC  20240222 Enabled JMX on FWD Client.
** 058 AI  20240314 Overloaded addFieldLike with String, character.
** 059 PMP 20240321 Changed labelStr and columnLabelStr from addNewField.
** 060 CA  20240404 Static temp-table resolution by name must be done using the currently executing type and
**                  the static temp-table's defining type.
** 061 ES  20240430 Dynamic temporary buffer based on a StaticTempTable should be treated as a static buffer.
** 062 CA  20240502 When a BEFORE-TABLE is being built too, for a dynamic temp-table, the associated DMOs for
**                  both the before and after table must have the after/beforeTable annotations set.
** 063 OM  20240510 During processing of the dynamic temp-table, the date, datetime and datetime-tz literals
**                  use the session date-format instead of standard MDY/YMD.
** 064 EAB 20240423 Rewrote toString method in order to imitate the result returned by writeJson().
** 065 CA  20240523 Fixed DATASET:WRITE-/READ-XML when before tables or (in)active relations are involved.  
** 066 AI  20240531 Overloaded addFieldLike with character, String.
** 067 OM  20240909 Improved Database API.
** 068 OM  20241128 Multi-tenant runtime support: selected the proper persistence context, eventually
**                  based on [sharedDb] parameter.
** 069 AP  20250129 Refactored toString method to use the RecordBufferSerializer and do the actual JSON
**                  serialization using jackson library.
** 070 ICP 20250129 Used logical constants to leverage cached instances.
** 071 AI  20250220 Added check for JMX_DEBUG to minimize use of lambda.
** 072 ES  20250328 Set NAMESPACE-URI, NAMESPACE-PREFIX, XML-NODE-NAME and SERIALIZE-NAME as a local copy, when
**                  the defaultBuffer is not initialized. When defaultBuffer is do created, set the attributes
**                  on default buffer. 
** 072 AS  20250529 Improved createTableLikeImpl(), use existingFieldsMap() 
**                  from the dmo info for acquiring source buffer fields.
*/

/*
** This program is free software: you can redistribute it and/or modify
** it under the terms of the GNU Affero General Public License as
** published by the Free Software Foundation, either version 3 of the
** License, or (at your option) any later version.
**
** This program is distributed in the hope that it will be useful,
** but WITHOUT ANY WARRANTY; without even the implied warranty of
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
** GNU Affero General Public License for more details.
**
** You may find a copy of the GNU Affero GPL version 3 at the following
** location: https://www.gnu.org/licenses/agpl-3.0.en.html
** 
** Additional terms under GNU Affero GPL version 3 section 7:
** 
**   Under Section 7 of the GNU Affero GPL version 3, the following additional
**   terms apply to the works covered under the License.  These additional terms
**   are non-permissive additional terms allowed under Section 7 of the GNU
**   Affero GPL version 3 and may not be removed by you.
** 
**   0. Attribution Requirement.
** 
**     You must preserve all legal notices or author attributions in the covered
**     work or Appropriate Legal Notices displayed by works containing the covered
**     work.  You may not remove from the covered work any author or developer
**     credit already included within the covered work.
** 
**   1. No License To Use Trademarks.
** 
**     This license does not grant any license or rights to use the trademarks
**     Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
**     of Golden Code Development Corporation. You are not authorized to use the
**     name Golden Code, FWD, or the names of any author or contributor, for
**     publicity purposes without written authorization.
** 
**   2. No Misrepresentation of Affiliation.
** 
**     You may not represent yourself as Golden Code Development Corporation or FWD.
** 
**     You may not represent yourself for publicity purposes as associated with
**     Golden Code Development Corporation, FWD, or any author or contributor to
**     the covered work, without written authorization.
** 
**   3. No Misrepresentation of Source or Origin.
** 
**     You may not represent the covered work as solely your work.  All modified
**     versions of the covered work must be marked in a reasonable way to make it
**     clear that the modified work is not originating from Golden Code Development
**     Corporation or FWD.  All modified versions must contain the notices of
**     attribution required in this license.
*/

package com.goldencode.p2j.persist;

import com.fasterxml.jackson.core.*;
import com.fasterxml.jackson.databind.*;
import com.goldencode.p2j.convert.*;
import com.goldencode.p2j.jmx.*;
import com.goldencode.p2j.persist.orm.*;
import com.goldencode.p2j.persist.serial.*;
import com.goldencode.p2j.ui.client.format.*;
import com.goldencode.p2j.util.*;
import com.goldencode.p2j.util.logging.*;
import com.goldencode.util.*;
import java.sql.SQLException;
import java.util.*;
import java.util.function.*;

/**
 * Implementation of buffer of a dynamic TEMP-TABLE and the associated
 * attributes and methods.
 * This {@link TempTable} interface is needed by {@link CommonHandle}
 * in order to store the handle to the table.
 *
 * @author om
 */
public class TempTableBuilder
extends AbstractTempTable
{
   /** Logger */
   private static final CentralLogger LOG = CentralLogger.get(TempTableBuilder.class);
   
   /** datetime-tz 4GL type name. */
   private final static String DATETIMETZ_TYPE_NAME = "datetime-tz";
   
   /** Instrumentation for {@link #tempTablePrepareImpl(String, String, boolean)}. */
   private static final NanoTimer CREATE_DYN_TT = NanoTimer.getInstance(FwdServerJMX.TimeStat.OrmCreateDynamicTable);
   
   /** Special characters (not letters or digits) allowed in dynamic table names. */
   private final static Set<Character> allowedSpecialChars = new HashSet<>();
   
   /**
    * Used during data type name validation. Map that stores the minimal possible length of a data
    * type name abbreviation. E.g. instead "integer" you can specify "integ", "int" or "i" because
    * the minimal length for this type is 1.
    */
   private final static Map<String, Integer> datatypesMatchLen = new LinkedHashMap<>();
   
   /** Data type name to default format string. */
   private final static Map<String, String> defaultFormatStrings = new HashMap<>();
   
   /** Table fields keyed by their legacy <i>normalized</i> names. */
   private final Map<String, P2JField> fields = new CaseInsensitiveLinkedHashMap<>();
   
   /** Table indexes keyed by their <i>normalized</i> names. */
   private final Map<String, P2JIndex> indexes = new CaseInsensitiveLinkedHashMap<>();
   
    /** Object which contains metadata about the DMO. */
   private DmoMeta dmoInfo = null;
   
   /** Generated DMO buffer class. */
   private Class<? extends DataModelObject> dmoIface = null;
   
   /** Generated DMO buffer class. */
   private Class<? extends TempTableBuffer> dmoBufIface = null;
   
   /** Determines if the table object was deleted. */
   private boolean deleted = false;
   
   /**
    * Flag that indicates that an ADD-* function was called for the table (CLEAR function resets
    * this flag). The value of this flag is used in CREATE-LIKE.
    */
   private boolean addFunctionCalled = false;
   
   /** Flag indicating the delete was explicitly called for a BY-REFERENCE handle. */
   private boolean postponedDelete = false;
   
   /** The NAMESPACE-URI of the XML node for serialized data of this {@code TEMP-TABLE}. */
   private String namespaceURI = null;
   
   /** The NAMESPACE-PREFIX of the XML node for serialized data of this {@code TEMP-TABLE}. */
   private String namespacePrefix = null;
   
   /** The current value of the {@code SERIALIZE-NAME} attribute. */
   private String serializeName = null;
   
   /** The name of the XML node for serialized data of this {@code TEMP-TABLE}. */
   private String xmlNodeName = null;

   /** 
    * Flag indicating if the {@link #tempTablePrepare} should implicitly create the before-table - used when
    * the {@link #createLike} source table has a before-table.
    */
   private boolean implicitBeforeTable;
   
   static
   {
      // TODO: replace numeric values in right side with constants
      // should go first, see validateDatatype
      datatypesMatchLen.put(DATETIMETZ_TYPE_NAME, 11);
      datatypesMatchLen.put(ParmType.DT.toString(), 8);
      datatypesMatchLen.put(ParmType.INT64.toString(), 5);
      datatypesMatchLen.put(ParmType.INT.toString(), 1);
      datatypesMatchLen.put(ParmType.CHAR.toString(), 1);
      datatypesMatchLen.put(ParmType.DATE.toString(), 2);
      datatypesMatchLen.put(ParmType.DEC.toString(), 2);
      datatypesMatchLen.put(ParmType.HANDLE.toString(), 6);
      datatypesMatchLen.put(ParmType.COMHANDLE.toString(), 9);
      datatypesMatchLen.put(ParmType.OBJECT.toString(), 6);
      datatypesMatchLen.put(ParmType.LOG.toString(), 1);
      datatypesMatchLen.put(ParmType.RAW.toString(), 2);
      datatypesMatchLen.put(ParmType.RECID.toString(), 2);
      datatypesMatchLen.put(ParmType.ROWID.toString(), 3);
      datatypesMatchLen.put(ParmType.CLOB.toString(), 2);
      datatypesMatchLen.put(ParmType.BLOB.toString(), 2);
      
      for (ParmType type : new ParmType[]{ParmType.INT,
                                          ParmType.INT64,
                                          ParmType.RECID,
                                          ParmType.DEC,
                                          ParmType.LOG,
                                          ParmType.DATE,
                                          ParmType.DT,
                                          ParmType.CHAR})
      {
         String typeName = type.toString();
         defaultFormatStrings.put(typeName, DisplayFormat.instanceOfType(typeName).defaultFormatString());
      }
      // separately because used "datetimetz" and "datetime-tz"
      defaultFormatStrings.put(ParmType.DTTZ.toString(),
         DisplayFormat.instanceOfType(DATETIMETZ_TYPE_NAME).defaultFormatString());
      
      allowedSpecialChars.addAll(Arrays.asList('-', '#', '$', '%', '&', '_'));
   }
   
   /**
    * Create a new TEMP-TABLE resource.
    */
   public TempTableBuilder()
   {
      super(true);
   }

   /**
    * Create a temp-table with the specified definition, as received from a remote side.
    * 
    * @param    tableName
    *           The table name.
    * @param    props
    *           The property definitions.
    * @param    tableIndexes
    *           The index definitions.
    * @param    xmlns
    *           The XML namespace.
    * @param    xmlPrefix
    *           The XML prefix.
    * @param    schemaMarshalLevel
    *           The SCHEMA-MARSHAL level for the serialized temp-table. 
    * @return   default buffer for the table
    */
   public BufferImpl createRemoteTable(String tableName,
                                              Iterator<PropertyDefinition> props,
                                              String tableIndexes, 
                                              String xmlns,
                                              String xmlPrefix,
                                              int schemaMarshalLevel)
   {
      return createTable(tableName, props, 
               () -> parseIndexString(tableIndexes), xmlns, xmlPrefix, schemaMarshalLevel);
   }
   
   /**
    * Create a temp-table with the specified definition, as received from a remote side.
    * 
    * @param    tableName
    *           The table name.
    * @param    props
    *           The property definitions.
    * @param    indexProvider
    *           indexes' provider.
    * @param    xmlns
    *           The XML namespace.
    * @param    xmlPrefix
    *           The XML prefix.
    * @param    schemaMarshalLevel
    *           The SCHEMA-MARSHAL level for the serialized temp-table.      
    * @return   default buffer for the table
    */
   public BufferImpl createRemoteTable(String tableName,
                                              Iterator<PropertyDefinition> props,
                                              Runnable indexProvider, 
                                              String xmlns,
                                              String xmlPrefix,
                                              int schemaMarshalLevel)
   {
      return createTable(tableName, props, indexProvider, xmlns, xmlPrefix, schemaMarshalLevel);
   }
   
   /**
    * Dynamically creates a temporary table at runtime and assign it to a
    * handle. The table is empty and will be populated with fields, indexes
    * and relations subsequent statements.
    * This method is the P2J equivalent of <code>CREATE TEMP-TABLE</code>
    * statement of Progress 4GL.
    *
    * @param   tth
    *          The handle to assign the newly created temporary table to.
    * @param   widgetPool
    *          The name of the widget pool that contains the temp-table. <code>null</code> if the
    *          unnamed pool must be used.
    */
   public static void create(handle tth, String widgetPool)
   {
      create(tth, (widgetPool == null ? null : new character(widgetPool)));
   }
   
   /**
    * Dynamically creates a temporary table at runtime and assign it to a
    * handle. The table is empty and will be populated with fields, indexes
    * and relations subsequent statements.
    * This method is the P2J equivalent of <code>CREATE TEMP-TABLE</code>
    * statement of Progress 4GL.
    *
    * @param   tth
    *          The handle to assign the newly created temporary table to.
    * @param   widgetPool
    *          The name of the widget pool that contains the temp-table. <code>null</code> if the
    *          unnamed pool must be used.
    */
   public static void create(handle tth, character widgetPool)
   {
      // validate widget pool before anything else!!!
      if (!WidgetPool.validWidgetPool(widgetPool))
      {
         return;
      }
      
      TempTableBuilder builder = new TempTableBuilder();
      tth.assign(builder);
      builder.addToPool(widgetPool);
   }
   
   /**
    * Dynamically creates a temporary table at runtime and assign it to a
    * handle. The table is empty and will be populated with fields, indexes
    * and relations subsequent statements.
    * This method is the P2J equivalent of <code>CREATE TEMP-TABLE</code>
    * statement of Progress 4GL.
    *
    * @param   tth
    *          The handle to assign the newly created temporary table to.
    */
   public static void create(handle tth)
   {
      create(tth, (character) null);
   }
   
   /**
    * Get the BUFFER handle associated with this temporary buffer. This allows chaining of
    * methods/attributes on a buffer using its BUFFER interface.
    *
    * @param    buf
    *           A temporary buffer for which the associated BUFFER resource is needed.
    *
    * @return   See above.
    */
   public static handle buffer(Buffer buf)
   {
      return new handle(buf);
   }
   
   /**
    * Get the TEMP-TABLE handle associated with this temporary buffer.This allows chaining of
    * methods/attributes on a buffer using its TEMP-TABLE interface. Some attributes/methods
    * are accessible only via the proper interface. Otherwise, 4052 error will be generated at
    * runtime.
    *
    * @param    buf
    *           A temporary buffer for which the associated TEMP-TABLE resource is needed.
    *
    * @return   See above.
    */
   public static handle tempTable(Temporary buf)
   {
      return TemporaryBuffer.get(buf).tableHandle();
   }
   
   /**
    * Get the TEMP-TABLE associated with this temporary buffer. This method allows simplifying the
    * generated code when a TEMP-TABLE method/attribute of a buffer that has TEMP-TABLE namespace
    * specified. If the TEMP-TABLE is specified but the method/attribute is not from
    * {@code TempTable} the {@link #tempTable} should be used as it allows to correctly generate
    * the 4052 error message (~ is not a queryable attribute for ~).
    *
    * @param    buf
    *           A temporary buffer for which the associated TEMP-TABLE resource is needed.
    *
    * @return   See above.
    */
   public static TempTable asTempTable(Temporary buf)
   {
      return TemporaryBuffer.get(buf).getParentTable();
   }

   /**
    * Get the active static temp table with the given name.
    *
    * @param  normalizedTableName4GL
    *         Normalized legacy name of the temp table.
    *
    * @return active static temp table with the given name or <code>null</code> if a table wasn't
    *         found by the given name.
    */
   public static TempTable getExistingTempTable(String normalizedTableName4GL)
   {
      TempTable tt = BufferManager.get().getStaticTempTable(null, normalizedTableName4GL);
      if (tt == null)
      {
         Temporary buf = SharedVariableManager.lookupTempTable(normalizedTableName4GL);
         tt = ((BufferImpl) buf).buffer().getParentTable();
      }

      return tt;
   }
   
   /**
    * Get the field (legacy name) of an existing table.
    *
    * @param  persistence
    *         Persistence which corresponds the target database.
    * @param  tableName4GL
    *         4GL table name.
    * @param  fieldName4GL
    *         4GL field name.
    *
    * @return field with the specified name or {@code null} if there is no such field.
    */
   public static P2JField getExistingField(Persistence persistence, String tableName4GL, String fieldName4GL)
   {
      String normalizedTableName4GL = TextOps.rightTrimLower(tableName4GL);
      String normalizedFieldName4GL = TextOps.rightTrimLower(fieldName4GL);
      DmoMeta dmoInfo;
      
      if (persistence.isTemporary())
      {
         TempTable tt = getExistingTempTable(normalizedTableName4GL);
         dmoInfo = tt.getDmoMeta();
      }
      else
      {
         String schema = DatabaseManager.getSchema(persistence.getDatabase(Persistence.SHARED_CTX));
         dmoInfo = DmoMetadataManager.getDmoInfo(TableMapper.getDMOClass(schema, normalizedTableName4GL));
      }
      
      Property property = dmoInfo.byLegacyName(normalizedFieldName4GL);
      return property == null ? null : dmoInfo.getExistingField(property.name);
   }
   
   /**
    * Get the field of an existing table.
    *
    * @param  buffer
    *         A record buffer which backs records of the target table.
    * @param  propertyName
    *         The property name.
    *
    * @return field with the specified name or {@code null} if there is no such field.
    */
   public static P2JField getExistingField(RecordBuffer buffer, String propertyName)
   {
      if (buffer.isDynamic() && buffer.tableHandleResource()._dynamic())
      {
         TempTableBuilder table = DynamicTablesHelper.getTableInfo(buffer);
         Class<? extends DataModelObject> dmoIface = buffer.getDMOInterface();
         String fieldName4GL = DynamicTablesHelper.get4GLFieldName(dmoIface, propertyName);
         return table.getField(fieldName4GL);
      }
      else
      {
         if (buffer.isTemporary())
         {
            return buffer.getParentTable()
                         .getDmoMeta()
                         .getExistingField(propertyName);
         }
         else
         {
            return null;
         }
      }
   }
   
   /**
    * Get the index of an existing table.
    *
    * @param  buffer
    *         A record buffer which backs records of the target table.
    * @param  indexName4GL
    *         4GL index name.
    *
    * @return index with the specified name or <code>null</code> if there is no such index.
    */
   public static P2JIndex getExistingIndex(RecordBuffer buffer, String indexName4GL)
   {
      if (buffer.isDynamic() && buffer.tableHandleResource()._dynamic())
      {
         TempTableBuilder table = DynamicTablesHelper.getTableInfo(buffer);
         String normalizedIndexName = TextOps.rightTrimLower(indexName4GL);
         return table.getIndex(normalizedIndexName);
      }
      else
      {
         Collection<P2JIndex> indexes = getExistingIndexes(buffer);
         return findIndexByName(indexes, indexName4GL);
      }
   }
   
   /**
    * Get the index of an existing table.
    *
    * @param  persistence
    *         Persistence which corresponds the target database.
    * @param  tableName4GL
    *         4GL table name.
    * @param  indexName4GL
    *         4GL index name.
    *
    * @return index with the specified name or <code>null</code> if there is no such index.
    */
   public static P2JIndex getExistingIndex(Persistence persistence, String tableName4GL, String indexName4GL)
   {
      Collection<P2JIndex> indexes = getExistingIndexes(persistence, tableName4GL);
      return findIndexByName(indexes, indexName4GL);
   }
   
   /**
    * Get the indexes of an existing table.
    *
    * @param  persistence
    *         Persistence which corresponds the target database.
    * @param  tableName4GL
    *         4GL table name.
    *
    * @return collection of the indexes of the target table or <code>null</code> if there is no
    *         such table.
    */
   public static Collection<P2JIndex> getExistingIndexes(Persistence persistence, String tableName4GL)
   {
      String normalizedTableName4GL = TextOps.rightTrimLower(tableName4GL);
      Database database = persistence.getDatabase(Persistence.SHARED_CTX);
      String schema = DatabaseManager.getSchema(database);
      Class<? extends Record> dmoClass;
      TempTable tempTable = null;
      if (persistence.isTemporary())
      {
         tempTable = BufferManager.get().getStaticTempTable(null, normalizedTableName4GL);
         if (tempTable == null)
         {
            return null;
         }
         dmoClass = tempTable.getDMOClass();
      }
      else
      {
         dmoClass = TableMapper.getDMOClass(schema, normalizedTableName4GL);
         if (dmoClass == null)
         {
            return null;
         }
      }
      
      return getExistingIndexes(persistence, dmoClass, tempTable);
   }
   
   
   /**
    * Get the indexes of an existing table.
    *
    * @param  persistence
    *         Persistence which corresponds the target database.
    * @param  dmoClass
    *         DMO class.
    * @param  tempTable
    *         The temp-table instance, for temporary DMOs.
    *         
    * @return collection of the indexes of the target table or <code>null</code> if there is no
    *         such table.
    */
   public static Collection<P2JIndex> getExistingIndexes(Persistence persistence, 
                                                         Class<? extends Record> dmoClass,
                                                         TempTable tempTable)
   {
      Database database = persistence.getDatabase(Persistence.SHARED_CTX);
      String schema = DatabaseManager.getSchema(database);
      boolean tempDatabase = persistence.isTemporary();

      DmoMeta dmoInfo = DmoMetadataManager.getDmoInfo(dmoClass);
      Class<? extends DataModelObject> dmoIface = dmoInfo.getAnnotatedInterface();
      Database helperDb = tempDatabase ? database : database.toType(Database.Type.DIRTY);
      TempTableHelper helper = TempTableHelper.get(helperDb, dmoIface);
      Iterator<P2JIndex> indexes = helper.getIndexes(schema, dmoIface);
      
      List<P2JIndex> res = new ArrayList<>();
      while (indexes.hasNext())
      {
         P2JIndex index = indexes.next();
         String indexName = DBUtils.extractIndexName(index.getName());
         
         // TODO: take legacy name from P2JIndex
         String originalIndexName = tempDatabase ?
               TableMapper.getLegacyIndexName(tempTable, indexName) :
               TableMapper.getLegacyIndexName(dmoIface, indexName);
         
         P2JIndex indexCopy = new P2JIndex(index.getTable(),
                                           originalIndexName,
                                           index.isUnique(),
                                           index.isPrimary(),
                                           index.isWord());

         ArrayList<P2JIndexComponent> comps = index.components();
         for (int i = 0; i < comps.size(); i++)
         {
            P2JIndexComponent component = comps.get(i);
            String columnName = component.getPropertyName();
            if (columnName.equals(DatabaseManager.PRIMARY_KEY))
            {
               continue;
            }
            
            indexCopy.addComponent(component.getLegacyName(),
                                   component.getPropertyName(),
                                   component.getColumnName(),
                                   component.isDescending(),
                                   component.isCharType(),
                                   component.isIgnoreCase(),
                                   component.isAbbreviated());
         }
         res.add(indexCopy);
      }
      
      return res;
   }
   
   /**
    * Get the indexes of an existing table.
    *
    * @param  buffer
    *         A record buffer which backs records of the target table.
    *
    * @return collection of the indexes of the target table or <code>null</code> if there is no
    *         such table.
    */
   public static Collection<P2JIndex> getExistingIndexes(RecordBuffer buffer)
   {
      if (buffer.isTemporary())
      {
         if (buffer.isDynamic() && buffer.tableHandleResource()._dynamic())
         {
            TempTableBuilder table = DynamicTablesHelper.getTableInfo(buffer);
            return table.getIndexes();
         }
         else
         {
            TempTable tt = ((TemporaryBuffer) buffer).tableHandleResource();
            Class<? extends Record> dmoClass = tt.getDMOClass();
            return getExistingIndexes(PersistenceFactory.getInstance(buffer.getDatabase()), dmoClass, tt);
         }
      }
      else
      {
         return getExistingIndexes(PersistenceFactory.getInstance(buffer.getDatabase()),
                                   getLegacyTableName(buffer));
      }
   }
   
   /**
    * Get the fields of an existing table. The {@code P2JField.name} are legacy field names, not properties.
    * The order in which they are returned is the native order in which they were defined/added.
    *
    * @param   buffer
    *          A record buffer which backs records of the target table.
    *
    * @return  collection of the fields of the target table or {@code null} if there is no such table.
    */
   public static Collection<P2JField> getExistingFields(RecordBuffer buffer)
   {
      return buffer.getDmoInfo().getExistingFields();
   }
   
   /**
    * Build a {@link P2JField} instance with the specified details.
    * 
    * @param    prop
    *           The field's annotation.
    * @param    parmType
    *           The field's type.
    * @param    initialValue
    *           The field's initial value.
    *           
    * @return   See above.
    */
   public static P2JField buildField(Property prop, ParmType parmType, BaseDataType initialValue)
   {
      int extent = prop.extent;
      
      return new P2JField(prop.legacy,
                          parmType,
                          extent,
                          prop.format,
                          initialValue,
                          prop.label,
                          prop.columnLabel,
                          prop.caseSensitive,
                          prop.codePage,
                          prop.help,
                          prop.serializeHidden,
                          prop.serializeName,
                          prop.xmlDataType,
                          prop.xmlNodeName,
                          prop.xmlNodeType,
                          prop.mandatory,
                          prop.scale,
                          prop.width);
   }
   
   /**
    * Get the fields of an existing table. The list contains the original fields (not-denormalized).
    *
    * @param   persistence
    *          Persistence which corresponds the target database.
    * @param   tableName4GL
    *          4GL table name.
    *
    * @return collection of the fields of the target table or {@code null} if there is no such table.
    */
   public static Collection<P2JField> getExistingFields(Persistence persistence, String tableName4GL)
   {
      String normalizedTableName4GL = TextOps.rightTrimLower(tableName4GL);
      DmoMeta dmoInfo;
      
      if (persistence.isTemporary())
      {
         StaticTempTable staticTempTable = (StaticTempTable) getExistingTempTable(normalizedTableName4GL);
         dmoInfo = staticTempTable.getDmoMeta();
      }
      else
      {
         String schema = DatabaseManager.getSchema(persistence.getDatabase(Persistence.SHARED_CTX));
         dmoInfo = DmoMetadataManager.getDmoInfo(TableMapper.getDMOClass(schema, normalizedTableName4GL));
      }
      
      return dmoInfo.getExistingFields();
   }
   
   /**
    * Get the array of ORM property names ordered in the same way as they were specified in 4GL for the given
    * temp-table.
    * NB: should not be used for the denormalized tables!!!
    *
    * @param   tempTable
    *          Temp-table object which backs the target table.
    *
    * @return  array of ORM property names ordered in the same way as they were specified in 4GL for the given
    *          temp-table.
    */
   public static String[] getOrderedPropertyNames(TempTable tempTable)
   {
      DmoMeta dmoInfo = tempTable.getDmoMeta();
      int propCount = dmoInfo.getFieldCount(false);
      String[] res = new String[propCount];
      Iterator<Property> it = dmoInfo.getFields(false);
      for (int i = 0; i < propCount; i++)
      {
         res[i] = it.next().name;
      }
      
      return res;
   }
   
   /**
    * Get the array of DMO property names ordered in the same way as they were specified in
    * 4GL for the given table (for the possible denormalized tables).
    * 
    * @param  dmoIface
    *         DMO interface associated with the target table.
    * @param  tempTable
    *         Temp-table object which backs the target table or {@code null} if the table is
    *         a permanent table.
    *
    * @return array of DMO property names 
    */
   public static String[] getPropertyNamesArray(Class<? extends DataModelObject> dmoIface, TempTable tempTable)
   {
      if (tempTable == null) 
      {
         List<String> lp = TableMapper.getLegacyProperties(dmoIface);
         return lp.toArray(new String[0]);
      }

      int propCount = TableMapper.getNumFields(tempTable);
      String[] res = new String[propCount];
      for (int i = 0; i < propCount; i++)
      {
         res[i] = TableMapper.getPropertyName(tempTable, i + 1);
      }
      return res;
   }
   
   /**
    * Get legacy table name.
    *
    * @param  buffer
    *         A buffer of the target table.
    *
    * @return legacy table name.
    */
   public static String getLegacyTableName(RecordBuffer buffer)
   {
      return buffer.buffer().getDmoInfo().legacyTable;
   }
   
   /**
    * Find index in the collection of indexes by its name.
    *
    * @param  indexes
    *         Collection of indexes to be searched.
    * @param  indexName
    *         Name of the target index.
    *
    * @return index with the specified name or <code>null</code> if there is no such index.
    */
   private static P2JIndex findIndexByName(Collection<P2JIndex> indexes, String indexName)
   {
      if (indexes == null)
         return null;
      
      String normalizedIndexName = TextOps.rightTrimLower(indexName);
      
      for (P2JIndex index : indexes)
      {
         if (index.getName().toLowerCase().equals(normalizedIndexName))
            return index;
      }
      
      return null;
   }
   
   /**
    * Validates date format on adding new field.
    *
    * @param   fmt
    *          Date format.
    *
    * @throws  ErrorConditionException
    *          Date format is wrong.
    */
   public static void validateDateFormatOnAddNewField(String fmt)
   throws ErrorConditionException
   {
      int delimiterCounter = 0;
      boolean wasDelimiter = false;
      for (int i = 0; i < fmt.length(); i++)
      {
         char c = fmt.charAt(i);
         boolean delimiter = c == '.' || c == '/' || c == '-';
         if (delimiter)
         {
            delimiterCounter++;
         }
         if (delimiterCounter > 2 || (delimiter && wasDelimiter) || (!delimiter && c != '9'))
         {
            String spec = "Character number %d of format %s is invalid";
            ErrorManager.recordOrThrowError(22, String.format(spec, i + 1, fmt));
            return;
         }
         wasDelimiter = delimiter;
      }
      if (wasDelimiter || delimiterCounter < 2)
      {
         String spec = "Date format %s is incomplete";
         ErrorManager.recordOrThrowError(154, String.format(spec, fmt));
      }
   }

   /**
    * Get default format for the given data type.
    *
    * @param  datatype
    *         Data type (string representation as in {@link ParmType#toString()}.
    *
    * @return default format for the given data type.
    */
   public static String getDefaultDataTypeFormat(String datatype)
   {
      return defaultFormatStrings.get(datatype);
   }

   /**
    * Validate format for the given data type.
    *
    * @param  parmType
    *         Data type.
    * @param  format
    *         Format string to check.
    *
    * @return <code>true</code> if format is valid.
    */
   public static boolean validateFormat(ParmType parmType, String format)
   {
      try
      {
         // validate format
         switch (parmType)
         {
            case INT:
            case DEC:
            case INT64:
            case HANDLE:
            case RECID:
               new NumberFormat(format);
               break;

            case DATE:
               validateDateFormatOnAddNewField(format);
               break;

            // TODO handle datetime[-tz] format validation

            //TODO handle char format validation (e.g. "x(10c)").
         }
      }
      catch (ErrorConditionException e)
      {
         return false;
      }

      return true;
   }

   /**
    * Validates data type name which was passed to ADD-NEW-FIELD function.
    *
    * @param  datatype
    *         Data type name.
    *
    * @return corresponding {@link ParmType} or <code>null</code> if validation has failed.
    */
   public static ParmType validateDatatype(String datatype)
   {
      // get first word
      int spaceIdx = datatype.indexOf(' ');
      String compareStr = spaceIdx >= 0 ? datatype.substring(0, spaceIdx) : datatype;
      
      if (compareStr.length() > 0)
      {
         compareStr = compareStr.toLowerCase();
         if ("progress.lang.object".equals(compareStr))
         {
            compareStr = "object";
         }
         
         for (String type : datatypesMatchLen.keySet())
         {
            if (type.startsWith(compareStr))
            {
               if (compareStr.length() >= datatypesMatchLen.get(type))
               {
                  ParmType res = ParmType.fromString(type);
                  if (res == null && DATETIMETZ_TYPE_NAME.equals(type))
                  {
                     res = ParmType.DTTZ;
                  }
                  return res;
               }
            }
         }
      }

      return null;
   }
   
   /**
    * Check if this temp-table was attempted to be deleted, but was postponed because of the handle
    * being a TABLE-HANDLE BY-REFERENCE.
    * 
    * @return   The {@link #postponedDelete}.
    */
   boolean isPostponedDelete()
   {
      return postponedDelete;
   }

   /**
    * Set the {@link #postponedDelete} flag.
    * 
    * @param    postponed
    *           Flag indicating that delete is postponed.
    */
   public void setPostponedDelete(boolean postponed)
   {
      this.postponedDelete = postponed;
   }
   
   /**
    * Returns the value of the NAMESPACE-URI attribute.
    *
    * @return  the value of the NAMESPACE-URI attribute.
    */
   @Override
   public character namespaceURI()
   {
      return _prepared() ? defaultBuffer.namespaceURI() : character.of(namespaceURI);
   }
   
   /**
    * Setter for NAMESPACE-URI attribute.
    *
    * @param   uri
    *          The new value of the NAMESPACE-URI attribute.
    */
   @Override
   public void namespaceURI(String uri)
   {
      if (uri == null)
      {
         ErrorManager.recordOrShowError(4083, " NAMESPACE-URI", LegacyResource.TEMP_TABLE + " widget");
         return;
      }
      
      if (!_prepared())
      {
         namespaceURI = uri;
         return;
      }
      
      defaultBuffer.namespaceURI(uri);
   }
   
   /**
    * Returns the value of the NAMESPACE-PREFIX attribute. Only for ELEMENT or ATTRIBUTE, for other
    * attributes return unknown value.
    *
    * @return  the current value of the the NAMESPACE-PREFIX attribute.
    */
   public character namespacePrefix()
   {
      return _prepared() ? defaultBuffer.namespacePrefix() : character.of(namespacePrefix);
   }
   
   /**
    * Setter for NAMESPACE-PREFIX attribute.
    *
    * @param   prefix
    *          The new value of the NAMESPACE-PREFIX attribute.
    */
   @Override
   public void namespacePrefix(String prefix)
   {
      if (prefix == null)
      {
         ErrorManager.recordOrShowError(4083, "NAMESPACE-PREFIX", LegacyResource.TEMP_TABLE + " widget");
         return;
      }
      
      if (!_prepared())
      {
         namespacePrefix = prefix;
         return;
      }
      
      defaultBuffer.namespacePrefix(prefix);
   }
   
   /**
    * Implementation of the read access of the {@code SERIALIZE-NAME} attribute.
    *
    * @return  the current value of the {@code SERIALIZE-NAME} attribute.
    */
   @Override
   public character getSerializeName()
   {
      if (_prepared())
      {
         return defaultBuffer.getSerializeName();
      }
      
      return serializeName == null || serializeName.trim().isEmpty() ? new character(name) : 
                                                                       new character(serializeName);
   }
   
   /**
    * Implementation of the write access of the {@code SERIALIZE-NAME} attribute.
    *
    * @param   sName
    *          the new value for the {@code SERIALIZE-NAME} attribute. 
    */
   @Override
   public void setSerializeName(String sName)
   {
      if (!_prepared())
      {
         serializeName = sName;
         return;
      }
      
      defaultBuffer.setSerializeName(sName);
   }
   
   /**
    * Obtain the name of the XML element or attribute representing the target object (a {@code TEMP-TABLE})
    * name in an XML document. If not set the {@code name} is returned.
    *
    * @return  the name of the XML element for this object.
    */
   @Override
   public character getXmlNodeName()
   {
      if (_prepared())
      {
         return defaultBuffer.getXmlNodeName();
      }
      
      return xmlNodeName == null || xmlNodeName.trim().isEmpty() ? getSerializeName() : 
                                                                   new character(xmlNodeName);
   }
   
   /**
    * Sets the name of the XML element or attribute representing the target object (either a {@code DataSet},
    * a {@code TempTable}, a {@code TemporaryBuffer}, or a temp-table {@code BufferField}) name in an XML
    * document.
    *
    * @param   name
    *          The new name of the XML element for this object.
    */
   @Override
   public void setXmlNodeName(String name)
   {
      if (!_prepared())
      {
         xmlNodeName = name;
         return;
      }
      
      defaultBuffer.setXmlNodeName(name);
   }

   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   hbuf
    *          A handle to a buffer or table from which to copy the definitions.
    * @param   indexName
    *          The name of a single index which is copied from the source table.
    *
    * @return  <code>true</code> on success.
    */
   @Override
   public logical createLike(handle hbuf, String indexName)
   {
      return createTableLikeImpl(hbuf, new character(indexName), false);
   }
   
   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   hbuf
    *          A handle to a buffer or table from which to copy the definitions.
    * @param   indexName
    *          The name of a single index which is copied from the source table.
    *
    * @return  <code>true</code> on success.
    */
   @Override
   public logical createLike(handle hbuf, character indexName)
   {
      return createTableLikeImpl(hbuf, indexName, true);
   }
   
   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   tableName
    *          The name of a table to create the copy of.
    * @param   indexName
    *          The name of a single index which is copied from the source table.
    *
    * @return  <code>true</code> on success.
    */
   @Override
   public logical createLike(String tableName, String indexName)
   {
      return createTableLikeImpl(new character(tableName), new character(indexName), false);
   }
   
   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   tableName
    *          The name of a table to create the copy of.
    * @param   indexName
    *          The name of a single index which is copied from the source table.
    *
    * @return  <code>true</code> on success.
    */
   @Override
   public logical createLike(character tableName, character indexName)
   {
      return createTableLikeImpl(tableName, indexName, true);
   }
   
   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   hbuf
    *          A handle to a buffer or table from which to copy the definitions.
    *
    * @return  <code>true</code> on success.
    */
   @Override
   public logical createLike(handle hbuf)
   {
      return createTableLikeImpl(hbuf, new character(), false);
   }
   
   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   tableName
    *          The name of a table to create the copy of.
    *
    * @return  <code>true</code> on success.
    */
   @Override
   public logical createLike(String tableName)
   {
      return createTableLikeImpl(new character(tableName), new character(), false);
   }
   
   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   tableName
    *          The name of a table to create the copy of.
    *
    * @return  <code>true</code> on success.
    */
   @Override
   public logical createLike(character tableName)
   {
      return createTableLikeImpl(tableName, new character(), false);
   }
   
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    *
    * @return  <code>true</code> on success.
    */
   public logical addNewField(character name, character type)
   {
      return addNewField(name, type, null, null, null, null, null);
   }
   
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    * @param   extent
    *          An integer expression specifying the extent of an array.
    *
    * @return  <code>true</code> on success.
    */
   public logical addNewField(character name, character type, integer extent)
   {
      return addNewField(name, type, extent, null, null, null, null);
   }
   
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    * @param   extent
    *          An integer expression specifying the extent of an array.
    * @param   format
    *          The data format for the defined data type. If empty or unknown, format is
    *          replaced with default format. Null value means that format is not used in
    *          ADD-NEW-FIELD.
    *
    * @return  <code>true</code> on success.
    */
   public logical addNewField(character name, character type, integer extent, character format)
   {
      return addNewField(name, type, extent, format, null, null, null);
   }
   
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    * @param   extent
    *          An integer expression specifying the extent of an array.
    * @param   format
    *          The data format for the defined data type. If empty or unknown, format is
    *          replaced with default format. Null value means that format is not used in
    *          ADD-NEW-FIELD.
    * @param   initial
    *          An expression that evaluates to the initial value of the
    *          defined field.
    *          TODO: this method will probably be overloaded because of this.
    *
    * @return  <code>true</code> on success.
    */
   public logical addNewField(character    name,
                              character    type,
                              integer      extent,
                              character    format,
                              BaseDataType initial)
   {
      return addNewField(name, type, extent, format, initial, null, null);
   }
   
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    * @param   extent
    *          An integer expression specifying the extent of an array.
    * @param   format
    *          The data format for the defined data type. If empty or unknown, format is
    *          replaced with default format. Null value means that format is not used in
    *          ADD-NEW-FIELD.
    * @param   initial
    *          An expression that evaluates to the initial value of the
    *          defined field.
    *          TODO: this method will probably be overloaded because of this.
    * @param   label
    *          The label of the defined field. If null or unknown the name
    *          parameter will be used.
    *
    * @return  <code>true</code> on success.
    */
   public logical addNewField(character    name,
                              character    type,
                              integer      extent,
                              character    format,
                              BaseDataType initial,
                              character    label)
   {
      return addNewField(name, type, extent, format, initial, label, null);
   }
   
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    * <p>
    * When a parameter is set to null, it means it was not specified by the business logic.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    * @param   extent
    *          An integer expression specifying the extent of an array.
    * @param   format
    *          The data format for the defined data type. If empty or unknown, format is
    *          replaced with default format. Null value means that format is not used in
    *          ADD-NEW-FIELD.
    * @param   initial
    *          An expression that evaluates to the initial value of the defined field.
    *          TODO: this method will probably be overloaded because of this.
    * @param   label
    *          The label of the defined field. If <code>null</code> or unknown the name
    *          parameter will be used.
    * @param   columnLabel
    *          The label of the column associated with the defined field
    *
    * @return  <code>true</code> on success.
    */
   public logical addNewField(character    name,
                              character    type,
                              integer      extent,
                              character    format,
                              BaseDataType initial,
                              character    label,
                              character    columnLabel)
   {
      return addNewField(name, type, extent, format, initial, label, columnLabel,
               null, null, null, false, false);
   }
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    * <p>
    * When a parameter is set to null, it means it was not specified by the business logic.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    * @param   extent
    *          An integer expression specifying the extent of an array.
    * @param   format
    *          The data format for the defined data type. If empty or unknown, format is
    *          replaced with default format. Null value means that format is not used in
    *          ADD-NEW-FIELD.
    * @param   initial
    *          An expression that evaluates to the initial value of the defined field.
    *          TODO: this method will probably be overloaded because of this.
    * @param   label
    *          The label of the defined field. If <code>null</code> or unknown the name
    *          parameter will be used.
    * @param   columnLabel
    *          The label of the column associated with the defined field. If <code>null</code> 
    *          or unknown the label will be used.
    * @param   help 
    *          Field HELP attribute value.
    * @param   xmlNodeType
    *          Field XML node type. 
    * @param   codePage 
    *          Field code page. 
    * @param   serializeHidden
    *          Field SERIALIZE-HIDDEN attribute value.
    * @param   caseSensitive 
    *          Field CASE-SENSITIVE attribute value.
    *
    * @return  <code>true</code> on success.
    */
   public logical addNewField(character    name,
                              character    type,
                              integer      extent,
                              character    format,
                              BaseDataType initial,
                              character    label,
                              character    columnLabel,
                              character    help,
                              character    xmlNodeType,
                              character    codePage,
                              boolean      serializeHidden,
                              boolean      caseSensitive)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return logical.FALSE;
      }
      
      String fieldName = name.getValue();
      String typeName = type.getValue();
      
      if (fieldName == null || typeName == null)
      {
         displayUnableToEvaluate();
         return logical.FALSE;
      }
      
      if (hasField(fieldName))
      {
         // we have the field with the same name
         return logical.FALSE;
      }
      
      // validate data type
      ParmType parmType = validateDatatype(typeName);
      if (parmType == null)
      {
         ErrorManager.recordOrShowError(9051,
                                        String.format("Illegal datatype given in ADD-NEW-FIELD %s", typeName),
                                        true,
                                        false);

         return logical.FALSE;
      }
      
      String formatStr;
      
      if (format == null )
      {
         formatStr = null;
      }
      else if (format.isUnknown() || format.getValue().length() == 0)
      {
         formatStr = getDefaultDataTypeFormat(typeName);
      }
      else
      {
         formatStr = format.getValue();
         if (!validateFormat(parmType, formatStr))
         {
            ErrorManager.recordOrShowError(9052,
                                           String.format("Illegal format given in ADD-NEW-FIELD %s", formatStr),
                                           true,
                                           false);
            return logical.FALSE;
         }
      }
      
      String labelStr = label != null ? label.getValue() : fieldName;
      String columnLabelStr = columnLabel != null ? columnLabel.getValue() : labelStr;
      String helpStr = help != null ? help.getValue() : null;
      String codePageStr = codePage != null ? codePage.getValue() : null;
      String xmlNodeTypeStr = xmlNodeType != null ? xmlNodeType.getValue() : null;
      
      // Note: case-sensitivity and CLOB code page cannot be specified using P4GL ADD-NEW-FIELD method
      long ext = extent == null || extent.getValue() == 1 ? 0 : extent.getValue();
      P2JField field = new P2JField(fieldName, parmType, ext, formatStr,
                                    initial, labelStr, columnLabelStr, caseSensitive, 
                                    codePageStr, helpStr, serializeHidden, null, null,
                                    null, xmlNodeTypeStr, false, 0, 0);
      addField(field);
      return logical.TRUE;
   }
   
   /**
    * Adds a new empty index with the specified name to the temp-table. Index 
    * components must be added with the <code>ADD-INDEX-FIELD()</code> method.
    * This method is the P2J equivalent of <code>ADD-NEW-INDEX</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the index to be created.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addNewIndex(character name)
   {
      return addNewIndex(name, null, null, null);
   }
   
   /**
    * Adds a new empty index with the specified name to the temp-table. Index 
    * components must be added with the <code>ADD-INDEX-FIELD()</code> method.
    * This method is the P2J equivalent of <code>ADD-NEW-INDEX</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the index to be created.
    * @param   unique
    *          TRUE if this index is unique.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addNewIndex(character name, logical unique)
   {
      return addNewIndex(name, unique, null, null);
   }
   
   /**
    * Adds a new empty index with the specified name to the temp-table. Index 
    * components must be added with the <code>ADD-INDEX-FIELD()</code> method.
    * This method is the P2J equivalent of <code>ADD-NEW-INDEX</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the index to be created.
    * @param   unique
    *          TRUE if this index is unique.
    * @param   primary
    *          TRUE if this is the primary index.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addNewIndex(character name, logical unique, logical primary)
   {
      return addNewIndex(name, unique, primary, null);
   }
   
   /**
    * Adds a new empty index with the specified name to the temp-table. Index 
    * components must be added with the <code>ADD-INDEX-FIELD()</code> method.
    * This method is the P2J equivalent of <code>ADD-NEW-INDEX</code> method
    * of Progress 4GL.
    * <p>
    * If a parameter is null, it means it was not set.
    * 
    * @param   name
    *          The name of the index to be created.
    * @param   unique
    *          TRUE if this index is unique.
    * @param   primary
    *          TRUE if this is the primary index.
    * @param   word
    *          TRUE if this is a word index.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addNewIndex(character name, logical unique, logical primary, logical word)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return logical.FALSE;
      }
      
      String indexName = name.getValue();
      if(indexName == null)
      {
         displayUnableToEvaluate();
         return logical.FALSE;
      }
      
      if (hasIndex(indexName))
      {
         displayDuplicateIndex(indexName);
         return logical.FALSE;
      }
      
      P2JIndex index = new P2JIndex(null,  // table name is set later, after TEMP-TABLE-PREPARE
                                    indexName,
                                    unique != null && unique.getValue(),
                                    primary != null && primary.getValue(),
                                    word != null && word.getValue());
      addIndex(index);
      return logical.TRUE;
   }
   
   /**
    * Adds to the temp-table a field like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    * 
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   source
    *          Database field name from which to copy the field. The table 
    *          name must be qualified with the database name.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addFieldLike(String name, String source)
   {
      return addFieldLike(new character(name), new character(source));
   }
   
   /**
    * Adds to the temp-table a field like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    * 
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   source
    *          Database field name from which to copy the field. The table 
    *          name must be qualified with the database name.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addFieldLike(String name, character source)
   {
      return addFieldLike(new character(name), source);
   }
   
   /**
    * Adds to the temp-table a field like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    * 
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   source
    *          Database field name from which to copy the field. The table 
    *          name must be qualified with the database name.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addFieldLike(character name, String source)
   {
      return addFieldLike(name, new character(source));
   }
   
   /**
    * Adds to the temp-table a field like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    * 
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   source
    *          Database field name from which to copy the field. The table 
    *          name must be qualified with the database name.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addFieldLike(character name, character source)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return new logical(false);
      }
      
      String targetName = name.getValue();
      String fullSource = source.getValue();
      if (targetName == null || fullSource == null)
      {
         displayUnableToEvaluate();
         return new logical(false);
      }
      
      if (hasField(targetName))
      {
         // we have the field with the same name
         return new logical(false);
      }
      
      int pos = fullSource.lastIndexOf('.');
      if (pos < 0)
      {
         ErrorManager.recordOrShowError(9053,
                                        "Field-name in ADD-LIKE-FIELD must have a table qualifier",
                                        true,
                                        false);
         return new logical(false);
      }
      
      String srcFieldName = fullSource.substring(pos + 1);
      String fullTableName = fullSource.substring(0, pos);
      
      Persistence persistence = findExistingTable(fullTableName, false);
      if (persistence == null)
      {
         return new logical(false);
      }
      
      String srcTableName = DynamicTablesHelper.parseTableName(fullTableName)[0];
      P2JField srcField = getExistingField(persistence, srcTableName, srcFieldName);
      if (srcField == null)
      {
         displayCouldNotFindLikeField(srcFieldName);
         return new logical(false);
      }
      
      P2JField targetField = copyField(targetName, srcField);
      
      addField(targetField);
      return new logical(true);
   }
   
   /**
    * Adds to the temp-table a field like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method of Progress 4GL.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   source
    *          Handle to field name from which to copy the field.
    *
    * @return  <code>true</code> on success.
    */
   public logical addFieldLike(String name, handle source)
   {
      return addFieldLike(new character(name), source);
   }
   
   /**
    * Adds to the temp-table a field like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    * 
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   source
    *          Handle to field name from which to copy the field.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addFieldLike(character name, handle source)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return new logical(false);
      }
      
      String targetName = name.getValue();
      if (targetName == null || !source._isValid())
      {
         displayUnableToEvaluate();
         return new logical(false);
      }
      
      if (hasField(targetName))
      {
         // we have the field with the same name
         return new logical(false);
      }

      RecordBuffer buffer;
      String fieldNameHibernate;
      Object handleValue = source.get();
      Class<?> cls = handleValue.getClass();
      if (FieldReference.class.isAssignableFrom(cls))
      {
         FieldReference ref = (FieldReference) handleValue;
         buffer = ref.getParentBuffer();
         fieldNameHibernate = ref.getProperty();
      }
      else if (BufferField.class.isAssignableFrom(cls))
      {
         BufferFieldImpl field = (BufferFieldImpl) handleValue;
         Buffer buf = field.getBuffer();
         buffer = RecordBuffer.get((DataModelObject) buf);
         fieldNameHibernate = field.getProperty();
      }
      else
      {
         ErrorManager.recordOrShowError(9056,
                                        "ADD-LIKE-FIELD requires valid BUFFER-FIELD object",
                                        true,
                                        false);
         return new logical(false);
      }
      
      P2JField srcField = getExistingField(buffer, fieldNameHibernate);
      P2JField targetField = copyField(targetName, srcField);
      
      addField(targetField);
      
      return new logical(true);
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(String indexName, String field, String ascending)
   {
      return addFieldToIndex(new character(indexName),
                             new character(field),
                             new character(ascending));
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(String indexName, String field, character ascending)
   {
      return addFieldToIndex(new character(indexName), new character(field), ascending);
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(character indexName, String field, String ascending)
   {
      return addFieldToIndex(indexName, new character(field), new character(ascending));
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(character indexName, String field, character ascending)
   {
      return addFieldToIndex(indexName, new character(field), ascending);
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(String indexName, character field, String ascending)
   {
      return addFieldToIndex(new character(indexName), field, ascending);
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(String indexName, character field, character ascending)
   {
      return addFieldToIndex(new character(indexName), field, ascending);
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(character indexName, character field, String ascending)
   {
      return addFieldToIndex(indexName, field, new character(ascending));
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. This method requires
    * the index to be first declared with {@link #addNewIndex} and cannot be called after the 
    * {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    * @param   ascending
    *          "asc" if the ascending ordering, "desc" for descending other -&gt; error.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(character indexName, character field, character ascending)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return logical.FALSE;
      }
      
      String idxName = indexName.getValue();
      String fieldName = field.getValue();
      if (idxName == null || fieldName == null)
      {
         displayUnableToEvaluate();
         return logical.FALSE;
      }
      
      String ascVal = ascending.getValue();
      boolean asc = (ascVal == null);
      
      if (!asc)
      {
         ascVal = ascVal.toLowerCase();
         asc = ascVal.startsWith("asc");
         
         if (!asc)
         {
            if (!ascVal.startsWith("desc"))
            {
               ErrorManager.recordOrShowError(9062,
                                              "Third argument to ADD-INDEX-FIELD must be DESC or ASC, in quotes",
                                              true,
                                              false);
               return logical.FALSE;
            }
         }
      }
      
      P2JField p2jField = getField(fieldName);
      
      // index with this name exists
      P2JIndex index = getIndex(idxName);
      if (index == null)
      {
         displayCouldNotFindAddIndex(idxName);
         return logical.FALSE;
      }
      
      // field with this name exists...
      boolean invalidField = (p2jField == null);
      
      if (!invalidField)
      {
         // ...and wasn't already added to the index
         ArrayList<P2JIndexComponent> comps = index.components();
         for (int i = 0; i < comps.size(); i++)
         {
            P2JIndexComponent component = comps.get(i);
            if (hasSameNormalizedName(component.getLegacyName(), fieldName))
            // TODO: replace with: component.getNormalizedName().equals(p2jField.getNormalizedName()) 
            {
               invalidField = true;
               break;
            }
         }
      }
      
      if (invalidField)
      {
         displayDuplicateOrInvalidIndexField(idxName, fieldName);
         return logical.FALSE;
      }
      
      if (!index.isWord() && p2jField.getExtent() > 0)
      {
         ErrorManager.recordOrShowError(370, "Array fields cannot be Index-Fields", true, true);
         return logical.FALSE;
      }
      
      index.addComponent(fieldName, null, null, !asc, false, false, false);
      return logical.TRUE;
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. The new added filed will
    * sort ascending the table. This method requires the index to be first declared with 
    * {@link #addNewIndex} and cannot be called after the {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldToIndex(String indexName, String field)
   {
      return addFieldToIndex(new character(indexName),
                             new character(field),
                             new character("asc"));
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. The new added filed will
    * sort ascending the table. This method requires the index to be first declared with 
    * {@link #addNewIndex} and cannot be called after the {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    *
    * @return  <code>true</code> for success.
    */
   
   public logical addFieldToIndex(character indexName, String field)
   {
      return addFieldToIndex(indexName, new character(field), new character("asc"));
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. The new added filed will
    * sort ascending the table. This method requires the index to be first declared with 
    * {@link #addNewIndex} and cannot be called after the {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    *
    * @return  <code>true</code> for success.
    */
   
   public logical addFieldToIndex(String indexName, character field)
   {
      return addFieldToIndex(new character(indexName), field, new character("asc"));
   }
   
   /**
    * Construct an index of this temp-table by adding a new field to it. The new added filed will
    * sort ascending the table. This method requires the index to be first declared with 
    * {@link #addNewIndex} and cannot be called after the {@link #prepared()}.
    * 
    * @param   indexName
    *          The name of the index being built.
    * @param   field
    *          The name of the temp-table to be added to the index.
    *
    * @return  <code>true</code> for success.
    */
   
   public logical addFieldToIndex(character indexName, character field)
   {
      return addFieldToIndex(indexName, field, new character("asc"));
   }
   
   /**
    * Adds to the temp-table an index like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    * 
    * @param   name
    *          The name of the index to which the source index is being copied. Progress ignores
    *          this option and uses the name of the source index instead.
    * @param   sourceName
    *          The name of the index in the source table that is being copied 
    *          to the temp-table.
    * @param   dbTable
    *          The database table name from which to copy the index.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addIndexLike(String name, String sourceName, String dbTable)
   {
      return addIndexLike(new character(name), new character(sourceName), new character(dbTable));
   }
   
   /**
    * Adds to the temp-table an index like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    * 
    * @param   name
    *          The name of the index to which the source index is being copied. Progress ignores
    *          this option and uses the name of the source index instead.
    * @param   sourceName
    *          The name of the index in the source table that is being copied 
    *          to the temp-table.
    * @param   dbTable
    *          The database table name from which to copy the index.
    * 
    * @return  <code>true</code> on success.
    */
   public logical addIndexLike(character name, character sourceName, character dbTable)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return new logical(false);
      }
      
      String srcIndexName = sourceName.getValue();
      String fullTableName = dbTable.getValue();
      if (name.isUnknown() || srcIndexName == null || fullTableName == null)
      {
         displayUnableToEvaluate();
         return new logical(false);
      }
      
      Persistence persistence = findExistingTable(fullTableName, true);
      if (persistence == null)
         return new logical(false);
      
      String srcTableName = DynamicTablesHelper.parseTableName(fullTableName)[0];
      P2JIndex srcIndex = getExistingIndex(persistence, srcTableName, srcIndexName);
      if (srcIndex == null)
      {
         displayCouldNotFindLikeIndex(
               DynamicTablesHelper.get4GLTableName(persistence, srcTableName), srcIndexName);
         return new logical(false);
      }
      
      return addIndexLike(srcIndex);
   }
   
   /**
    * Adds to the temp-table an index like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the index to which the source index is being copied. Progress ignores
    *          this option and uses the name of the source index instead.
    * @param   sourceName
    *          The name of the index in the source table that is being copied
    *          to the temp-table.
    * @param   sourceBuffer
    *          Buffer handle from which to copy the index.
    *
    * @return  <code>true</code> on success.
    */
   public logical addIndexLike(String name, String sourceName, handle sourceBuffer)
   {
      return addIndexLike(new character(name), new character(sourceName), sourceBuffer);
   }
   
   /**
    * Adds to the temp-table an index like the one specified.
    * This method is the P2J equivalent of <code>ADD-LIKE-FIELD</code> method
    * of Progress 4GL.
    *
    * @param   name
    *          The name of the index to which the source index is being copied. Progress ignores
    *          this option and uses the name of the source index instead.
    * @param   sourceName
    *          The name of the index in the source table that is being copied
    *          to the temp-table.
    * @param   sourceBuffer
    *          Buffer handle from which to copy the index.
    *
    * @return  <code>true</code> on success.
    */
   public logical addIndexLike(character name, character sourceName, handle sourceBuffer)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return new logical(false);
      }
      
      String srcIndexName = sourceName.getValue();
      if (name.isUnknown() || srcIndexName == null || !sourceBuffer._isValid())
      {
         displayUnableToEvaluate();
         return new logical(false);
      }
      
      Object handleValue = sourceBuffer.get();
      if (!Buffer.class.isAssignableFrom(handleValue.getClass()))
      {
         displayCouldNotFindLikeTableFromHandle();
         return new logical(false);
      }
      
      Buffer buffer = (Buffer) handleValue;
      RecordBuffer recordBuffer = RecordBuffer.get((DataModelObject) buffer);
      
      P2JIndex srcIndex = getExistingIndex(recordBuffer, srcIndexName);
      if (srcIndex == null)
      {
         displayCouldNotFindLikeIndex(recordBuffer.getDmoInfo().legacyTable, srcIndexName);
         return new logical(false);
      }
      
      return addIndexLike(srcIndex);
   }
   
   /**
    * Returns <code>true</code> if this temp-table is in prepared state (after a successful call
    * to {@link #tempTablePrepare}).
    * This method implements the read-only PREPARED attribute from 4GL.
    *
    * @return  <code>true</code> if this table is prepared.
    */
   public boolean _prepared()
   {
      return super._name() != null;
   }
   
   /**
    * Check whether this TEMP-TABLE is in CLEAR state: I.e. the temp-table is first created or immediately
    * after the CLEAR() method is applied. The other two state of a temp-table are: UNPREPARED and PREPARED.
    * The UNPREPARED state between the first definitional method has been applied and before the
    * TEMP-TABLE-PREPARE() method is applied.
    * <p>
    * This method is FWD-internal. The programmer can check whether the temp-table is in an UNPREPARED or
    * PREPARED state by checking the PREPARED attribute, but the CLEAR state is transparent.
    *
    * @return  {@code true} if this table is in CLEAR state.
    */
   public boolean _clear()
   {
      return fields.size() == 0;
   }
   
   /**
    * Checks  whether the resource is dynamic or static.
    * 
    * @return   <code>true</code> if the resource is dynamic, <code>false</code> otherwise. 
    */
   public logical dynamic()
   {
      return new logical(_dynamic());
   }
   
   /**
    * Checks  whether the resource is dynamic or static.
    * 
    * @return   <code>true</code> if the resource is dynamic, <code>false</code> otherwise. 
    */
   public boolean _dynamic()
   {
      return true;
   }
   
   /**
    * Get the PRIMARY attribute.
    *
    * @return   The primary index.
    */
   @Override
   public character primaryIndex()
   {
      if (_prepared())
      {
         return super.primaryIndex();
      }
      
      if (indexes.isEmpty())
      {
         return new character();
      
      }
      
      String ret = null;
      String first = null;
      for (Map.Entry<String, P2JIndex> next : indexes.entrySet())
      {
         if (first == null)
         {
            first = next.getValue().getLegacyName();
         }
         if (next.getValue().isPrimary())
         {
            return new character(next.getValue().getLegacyName());
         }
      }
      
      return new character(first);
   }
   
   /**
    * Set the PRIMARY attribute.
    *
    * @param    primary
    *           The primary index.
    */
   @Override
   public void primaryIndex(String primary)
   {
      if (primary == null || _prepared())
      {
         super.primaryIndex(primary);
         return;
      }
      
      // at this point the table was NOT prepared yet. Iterate all known indexes and update the attribute
      for (Map.Entry<String, P2JIndex> next : indexes.entrySet())
      {
         next.getValue().setPrimary(next.getKey().equalsIgnoreCase(primary));
      }
   }
   
   /**
    * Returns the temp-table to it's clear state, removing all defined fields, indexes, all
    * records in the table and all corresponding buffers.
    * 
    * @return  true if operation is successful
    */
   public logical clear()
   {
      if (isUndoable() && _prepared() && hasRecords().getValue())
      {
         displayCannotClear();
         return new logical(false);
      }
      
      return new logical(forceClear());
   }
   
   /**
    * The implementation of {@code TEMP-TABLE-PREPARE} method. This method is invoked from both
    * user API and also from {@code DataSet} when a DATASET is duplicated. 
    * 
    * @param   tableName
    *          Temp-table name to be used in subsequent query statements which refer to this
    *          temp-table.
    * @param   beforeName
    *          The name of the before table name. If {@code null} the BEFORE_TABLE won't be
    *          created.  In case of creating a before-table, this is the after-table name.
    * @param   isBeforeTable
    *          Flag indicating the prepare is for a before-table.
    *          
    * @return  {@code true} on success.
    */
   logical tempTablePrepareImpl(String tableName, String beforeName, boolean isBeforeTable)
   {
      if (_prepared())
      {
         displayPreparedIgnoring();
         return logical.FALSE;
      }
      
      if (tableName == null)
      {
         ErrorManager.recordOrShowError(9046, "Unable to PREPARE TEMP-TABLE buffer object", true, false);
         forceClear();
         return logical.FALSE;
      }
      
      if (fields.size() == 0)
      {
         ErrorManager.recordOrShowError(9033,
                                        "TEMP-TABLE object must have at least one field before TEMP-TABLE-PREPARE can be called",
                                        true,
                                        false);
         displayUnableToPrepareTableFields();
         forceClear();
         return logical.FALSE;
      }
      
      // validate table name
      int nameLength = tableName.length();
      if (nameLength == 0 || nameLength > 32)
      {
         ErrorManager.recordOrShowError(1700,
                                        "Identifier was left blank or is more than 32 characters",
                                        true,
                                        false);
         displayUnableToPrepareTableFields();
         forceClear();
         return logical.FALSE;
      }
      
      char fistChar = tableName.charAt(0);
      if (fistChar != '_' && !Character.isLetter(fistChar))
      {
         ErrorManager.recordOrShowError(257,
                                        "The first character of " + tableName + " must be alphabetic",
                                        true,
                                        true);
         displayUnableToPrepareTableFields();
         forceClear();
         return logical.FALSE;
      }
      
      for (int i = 1; i < nameLength; i++)
      {
         char c = tableName.charAt(i);
         if (!Character.isLetter(c) && !Character.isDigit(c) && !allowedSpecialChars.contains(c))
         {
            ErrorManager.recordOrShowError(274,
                                           String.format("The character %s is not permitted in name %s", c, tableName),
                                           true,
                                           true);
            displayUnableToPrepareTableFields();
            forceClear();
            return logical.FALSE;
         }
      }
      
      // test whether LOBs are defined in undoable table:
      if (undoable)
      {
         Set<Map.Entry<String, P2JField>> entries = fields.entrySet();
         for (Map.Entry<String, P2JField> entry : entries)
         {
            ParmType type = entry.getValue().getType();
            if (type == ParmType.CLOB || type == ParmType.BLOB)
            {
               ErrorManager.recordOrShowError(14435);
               // Large object fields can only be defined in NO-UNDO Temp-Tables
               
               displayUnableToPrepareTableFields();
               forceClear();
               return logical.FALSE;
            }
         }
      }
      
      // validate indexes
      boolean foundPrimary = false;
      for (P2JIndex index : indexes.values())
      {
         if (index.isPrimary())
         {
            if (foundPrimary)
            {
               // there are multiple primary indexes
               ErrorManager.recordOrShowError(3389,
                                              "TEMP-TABLE  must have only one primary key",
                                              true,
                                              false);
               displayUnableToPrepareTableIndices();
               forceClear();
               return logical.FALSE;
            }
            else
            {
               foundPrimary = true;
            }
         }
         
         if (index.size() == 0)
         {
            ErrorManager.recordOrShowError(9029,
                                           String.format("TEMP-TABLE index %s needs at least 1 component", index.getName()),
                                           true,
                                           false);
            displayUnableToPrepareTableIndices();
            forceClear();
            return logical.FALSE;
         }
      }
      
      return date.withDynamicFormat(() -> {
         DynamicTablesHelper dth = DynamicTablesHelper.getInstance();
         boolean result;
         if (FwdServerJMX.JMX_DEBUG)
         {
            result = CREATE_DYN_TT.timerWithReturn(() -> dth.createDynamicDMO(this, tableName, beforeName, isBeforeTable));
         }
         else
         {
            result = dth.createDynamicDMO(this, tableName, beforeName, isBeforeTable);
         }

         if (result)
         {
            super.name(new character(tableName)); // also makes table "prepared"
            createDefaultBuffer();

            
            if (xmlNodeName != null)
            {
               defaultBuffer.setXmlNodeName(xmlNodeName);
               xmlNodeName = null;
            }
            
            if (serializeName != null)
            {
               defaultBuffer.setSerializeName(serializeName);
               serializeName = null;
            }
            
            if (namespaceURI != null)
            {
               defaultBuffer.namespaceURI(namespaceURI);
               namespaceURI = null;
            }
            
            if (namespacePrefix != null)
            {
               defaultBuffer.namespacePrefix(namespacePrefix);
               namespacePrefix = null;
            }
            
            if (beforeName != null && !isBeforeTable)
            {
               result = createBeforeTable(beforeName, tableName);
            }
            
            if (beforeTableType == BeforeType.UNKNOWN)
            {
               beforeTableType = BeforeType.SIMPLE;
            }
         }
         else
         {
            forceClear();
         }

         return logical.of(result);
      });
   }
   
   /**
    * Creates the before table and link it to current table.
    * 
    * @param   beforeName
    *          The name of the before table to be build.
    * @param   afterName
    *          The name of the after table associated with the before table.
    *
    * @return  {@code true} if operation is successful and {@code false} otherwise.
    */
   boolean createBeforeTable(String beforeName, String afterName)
   {
      boolean res;
      // create the BEFORE-TABLE
      handle btth = TypeFactory.handle();
      TempTableBuilder.create(btth);
      BufferImpl srcB4Buff = (BufferImpl) defaultBufferHandleNative();
      
      TempTableBuilder newBtt = (TempTableBuilder) btth.getResource();
      newBtt.addField(buildField(ReservedProperty._ERRORFLAG, ParmType.INT, new integer()));
      newBtt.addField(buildField(ReservedProperty._ORIGINROWID, ParmType.ROWID, new rowid()));
      newBtt.addField(buildField(ReservedProperty._DATASOURCEROWID, ParmType.ROWID, new rowid()));
      newBtt.addField(buildField(ReservedProperty._ERRORSTRING, ParmType.CHAR, new character()));
      newBtt.addField(buildField(ReservedProperty._PEERROWID, ParmType.ROWID, new rowid()));
      newBtt.addField(buildField(ReservedProperty._ROWSTATE, ParmType.INT, new integer()));
      
      newBtt.createLike(new handle(srcB4Buff));
      
      // leave only the rowState index
      newBtt.indexes.clear();
      P2JIndex rowStateIndex = new P2JIndex(beforeName, "rowState", false);
      rowStateIndex.addComponent(ReservedProperty._ROWSTATE.legacy);
      newBtt.addIndex(rowStateIndex);
      
      if (beforeName.length() > 32)
      {
         // max length is 32
         beforeName = beforeName.substring(0, 32);
      }
      res = newBtt.tempTablePrepareImpl(beforeName, afterName, true).booleanValue();
      
      if (res)
      {
         // dynamically link the new before TempTable to this after TempTable
         newBtt.beforeTableType = BeforeType.BEFORE;
         newBtt.peerTable = this;
         this.beforeTableType = BeforeType.AFTER;
         this.peerTable = newBtt;
         // then update the buffer types and link them using peerBuffer 
         srcB4Buff.computeBufferType();
         srcB4Buff.peerBuffer.computeBufferType();
      }
      return res;
   }
   
   /**
    * Signals that all the field and index definitions for a temp-table have been provided. After the call to
    * this method no fields and indexes can be added to this temporary table.<br>
    * This method is the P2J equivalent of {@code TEMP-TABLE-PREPARE} method of Progress 4GL.
    * 
    * @param   name
    *          Temp-table name to be used in subsequent query statements that refer to this temp-table.
    *
    * @return  {@code true} on success.
    */
   public logical tempTablePrepare(String name)
   {
      return implicitBeforeTable ? tempTablePrepare(name, true) : tempTablePrepareImpl(name, null, false);
   }
   
   /**
    * The implementation of {@code TEMP-TABLE-PREPARE} method. This method is invoked from internal APIs like 
    * dataset XML import. 
    * 
    * @param   tableName
    *          Temp-table name to be used in subsequent query statements which refer to this
    *          temp-table.
    * @param   beforeName
    *          The name of the before table name. If {@code null} the BEFORE_TABLE won't be
    *          created.
    *          
    * @return  {@code true} on success.
    */
   public boolean tempTablePrepare(String tableName, String beforeName)
   {
      return tempTablePrepareImpl(tableName, beforeName, false).booleanValue();
   }
   
   /**
    * Signals that all the field and index definitions for a temp-table have been provided. After the call to
    * this method no fields and indexes can be added to this temporary table.<br>
    * This method is the P2J equivalent of {@code TEMP-TABLE-PREPARE} method of Progress 4GL.
    *
    * @param   name
    *          Temp-table name to be used in subsequent query statements that refer to this temp-table.
    *
    * @return  {@code true} on success.
    */
   public logical tempTablePrepare(character name)
   {
      String tableName = name.toJavaType();
      return implicitBeforeTable ? tempTablePrepare(name, true) : tempTablePrepareImpl(tableName, null, false);
   }
   
   /**
    * Signals that all the field and index definitions for a temp-table have been provided. After
    * the call to this method no fields and indexes can be added to this temporary table.
    * This method is the P2J equivalent of {@code TEMP-TABLE-PREPARE} method of Progress 4GL.
    *
    * @param   name
    *          Temp-table name to be used in subsequent query statements which refer to this temp-table.
    * @param   before
    *          Create the {@code BEFORE-TABLE} also if {@code true}.
    *
    * @return  {@code true} on success.
    */
   @Override
   public logical tempTablePrepare(String name, boolean before)
   {
      return tempTablePrepareImpl(name, (before && name != null) ? "BI" + name : null, false);
   }
   
   /**
    * Signals that all the field and index definitions for a temp-table have been provided. After
    * the call to this method no fields and indexes can be added to this temporary table.
    * This method is the P2J equivalent of {@code TEMP-TABLE-PREPARE} method of Progress 4GL.
    *
    * @param   name
    *          Temp-table name to be used in subsequent query statements which refer to this temp-table.
    * @param   before
    *          Create the {@code BEFORE-TABLE} also if {@code true}.
    *
    * @return  {@code true} on success.
    */
   @Override
   public logical tempTablePrepare(String name, logical before)
   {
      boolean boolBef = !before.isUnknown() && before.booleanValue();
      return tempTablePrepareImpl(name, (boolBef && name != null) ? "BI" + name : null, false);
   }
   
   /**
    * Signals that all the field and index definitions for a temp-table have been provided. After
    * the call to this method no fields and indexes can be added to this temporary table.
    * This method is the P2J equivalent of {@code TEMP-TABLE-PREPARE} method of Progress 4GL.
    *
    * @param   name
    *          Temp-table name to be used in subsequent query statements which refer to this temp-table.
    * @param   before
    *          Create the {@code BEFORE-TABLE} also if {@code true}.
    *
    * @return  {@code true} on success.
    */
   @Override
   public logical tempTablePrepare(character name, boolean before)
   {
      String tableName = name.toJavaType();
      return tempTablePrepareImpl(tableName, (before && tableName != null) ? "BI" + tableName : null, false);
   }
   
   /**
    * Signals that all the field and index definitions for a temp-table have been provided. After
    * the call to this method no fields and indexes can be added to this temporary table.
    * This method is the P2J equivalent of {@code TEMP-TABLE-PREPARE} method of Progress 4GL.
    *
    * @param   name
    *          Temp-table name to be used in subsequent query statements which refer to this temp-table.
    * @param   before
    *          Create the {@code BEFORE-TABLE} also if {@code true}.
    *
    * @return  {@code true} on success.
    */
   @Override
   public logical tempTablePrepare(character name, logical before)
   {
      boolean boolBef = !before.isUnknown() && before.booleanValue();
      String tableName = name.toJavaType();
      return tempTablePrepareImpl(tableName, (boolBef && tableName != null) ? "BI" + tableName : null, false);
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   tableName
    *          The name of the table from which to copy the field definitions.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(String tableName)
   {
      return addFieldsFrom(new character(tableName));
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   hbuf
    *          A handle to a buffer or table from which to copy the field definitions.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(handle hbuf)
   {
      return addFieldsFrom(hbuf, (String) null);
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   tableName
    *          The name of the table from which to copy the field definitions.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(character tableName)
   {
      return addFieldsFrom(tableName, new character());
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   hbuf
    *          A handle to a buffer or table from which to copy the field definitions.
    * @param   except
    *          A comma-separated list of field names to be excluded from the copy process.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(handle hbuf, String except)
   {
      return addFieldsFrom(hbuf, new character(except));
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   tableName
    *          The name of the table from which to copy the field definitions.
    * @param   except
    *          A comma-separated list of field names to be excluded from the copy process.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(String tableName, String except)
   {
      return addFieldsFrom(new character(tableName), new character(except));
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   tableName
    *          The name of the table from which to copy the field definitions.
    * @param   except
    *          A comma-separated list of field names to be excluded from the copy process.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(String tableName, character except)
   {
      return addFieldsFrom(new character(tableName), except);
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   hbuf
    *          A handle to a buffer or table from which to copy the field definitions.
    * @param   except
    *          A comma-separated list of field names to be excluded from the copy process.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(handle hbuf, character except)
   {
      if (_prepared())
      {
         displayPrepared();
         return new logical(); // unknown value is returned
      }
      
      RecordBuffer recordBuffer = getRecordBufferByHandle(hbuf, CallingFunction.ADD_FIELDS_FROM);
      if (recordBuffer == null)
         return new logical(false);
      
      doAddFieldsFrom(getExistingFields(recordBuffer), except);
      return new logical(true);
   }
   
   /**
    * Adds fields from another table into this temp-table. The fields that are already in the
    * temp-table are ignored.
    * This is the P2J implementation of the ADD-FIELDS-FROM of 4GL.
    *
    * @param   tableName
    *          The name of the table from which to copy the field definitions.
    * @param   except
    *          A comma-separated list of field names to be excluded from the copy process.
    *
    * @return  <code>true</code> for success.
    */
   public logical addFieldsFrom(character tableName, character except)
   {
      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return new logical(); // unknown value is returned
      }
      
      String fullTableName = tableName.getValue();
      if (fullTableName == null)
      {
         displayUnableAddFieldsFrom();
         return new logical(false);
      }
      
      Persistence persistence = findExistingTable(fullTableName, true);
      if (persistence == null)
      {
         displayUnableAddFieldsFrom();
         return new logical(false);
      }
      
      String tblName = DynamicTablesHelper.parseTableName(fullTableName)[0];
      doAddFieldsFrom(getExistingFields(persistence, tblName), except);
      return new logical(true);
   }
   
   /**
    * Create a temp-table with the specified definition.
    * 
    * @param    tableName
    *           The table name.
    * @param    props
    *           The property definitions.
    * @param    tableIndexes
    *           The index definitions.
    * @param    xmlns
    *           The XML namespace.
    * @param    xmlPrefix
    *           The XML prefix.
    * @param    schemaMarshalLevel
    *           The SCHEMA-MARSHAL level for the serialized temp-table. 
    * @return   default buffer for the table
    */
   public BufferImpl createTable(String tableName,
                                 Iterator<PropertyDefinition> props,
                                 String tableIndexes, 
                                 String xmlns,
                                 String xmlPrefix,
                                 int schemaMarshalLevel)
   {
      return  createTable(tableName, props, 
                          () -> parseIndexString(tableIndexes), 
                          xmlns, xmlPrefix, schemaMarshalLevel);
   }

   /**
    * Create a temp-table with the specified definition.
    * 
    * @param    tableName
    *           The table name.
    * @param    props
    *           The property definitions.
    * @param    indexProvider
    *           indexes' provider.
    * @param    xmlns
    *           The XML namespace.
    * @param    xmlPrefix
    *           The XML prefix.
    * @param    schemaMarshalLevel
    *           The SCHEMA-MARSHAL level for the serialized temp-table. 
    * @return   default buffer for the table
    */
   public BufferImpl createTable(String tableName,
                                 Iterator<PropertyDefinition> props,
                                 Runnable indexProvider, 
                                 String xmlns,
                                 String xmlPrefix,
                                 int schemaMarshalLevel)
   {
      setSchemaMarshalLevel(schemaMarshalLevel);
      props.forEachRemaining(this::addNewField);
      
      indexProvider.run();
      
      tempTablePrepare(tableName);
      if (xmlns != null)
      {
         defaultBuffer.namespaceURI(xmlns);
      }
      if (xmlPrefix != null)
      {
         defaultBuffer.namespacePrefix(xmlPrefix);
      }
      
      return defaultBuffer;
   }

   /**
    * Build temporary table from a table wrapper object.
    *
    * @param tableWrapper
    *        Table wrapper containing table definitions.
    * @param tableParameterIndex
    *        1-based index of corresponding TABLE[-HANDLE] parameter (in function declaration).
    *        Used for error handling.
    */
   public void createFromTableWrapper(TableWrapper tableWrapper, int tableParameterIndex)
   {
      // simple NPE protection
      if (tableWrapper == null)
      {
         return;
      }
      List<IndexDefinition> indexDefs = tableWrapper.getIndexDefs();
      if (indexDefs != null)
      {
         createTable(tableWrapper.getTableName(), 
                  tableWrapper.propertyIterator(), 
                  () -> indexDefs.stream().forEach(idx -> idx.addTo(this)), 
                  tableWrapper.getXmlns(), 
                  tableWrapper.getXmlPrefix(),
                  tableWrapper.getSchemaMarshalLevel());
      }
      else
      {
         createTable(tableWrapper.getTableName(), 
                  tableWrapper.propertyIterator(), 
                  tableWrapper.getIndexes(), 
                  tableWrapper.getXmlns(), 
                  tableWrapper.getXmlPrefix(),
                  tableWrapper.getSchemaMarshalLevel());
      }
      TemporaryBuffer.insertAllRows(tableWrapper,
                                    (Temporary) defaultBufferHandleNative(),
                                    true,
                                    TemporaryBuffer.CopyTableMode.OUTPUT_TABLE_HANDLE_PARAM_MODE,
                                    tableParameterIndex);
   }
   
   /**
    * Obtain the handle to the buffer that was created by default for this 
    * temporary table. Every dynamic temp-table is created with at 
    * least one buffer. This buffer's object handle is returned by this
    * method. It cannot be called until the table has been prepared using 
    * <code>tempTablePrepare()</code> method.
    * This method is the P2J equivalent of <code>DEFAULT-BUFFER-HANDLE</code>
    * attribute.
    * 
    * @return  The default buffer handle for the temporary table.
    */
   public handle defaultBufferHandle()
   {
      return new handle(defaultBufferHandleNative());
   }

   /**
    * Obtain the buffer that was created by default for this temporary table. Every
    * dynamic temp-table is created with at least one buffer. This buffer object is
    * returned by this method.
    *
    * Note that it cannot be called until the table has been prepared using
    * <code>tempTablePrepare()</code> method.
    *
    * This method is the P2J equivalent of <code>DEFAULT-BUFFER-HANDLE</code>
    * attribute.
    *
    * @return  The default buffer for the temporary table.
    */
   @Override
   public Buffer defaultBufferHandleNative()
   {
      if (!_prepared())
      {
         displayNotPrepared();
         return null;
      }

      return defaultBuffer;
   }
   
   /**
    * Set table NAME attribute. Note that it can be changed only after the table has been prepared.
    *
    * @param   name
    *          New table name.
    */
   public void name(character name)
   {
      if (name.isUnknown())
      {
         ErrorManager.recordOrShowError(4083,
                                        "Unable to assign UNKNOWN value to attribute NAME on TEMP-TABLE widget",
                                        true,
                                        true);
         return;
      }
      
      if (!_prepared())
      {
         displayNotPrepared();
         ErrorManager.recordOrThrowError(3131,
               "Unable to set attribute NAME in widget of type TEMP-TABLE",
               false);
         return;
      }
      
      super.name(name);
      
      defaultBuffer.nameInternal(name);
   }
   
   /**
    * Get table NAME attribute.
    *
    * @return  name
    *          Table NAME attribute value. If the table hasn't been prepared, unknown value is
    *          returned.
    */
   public character name()
   {
      character name = super.name();
      if (name.isUnknown())
      {
         displayNotPrepared();
      }
      
      return name;
   }
   
   /**
    * Get the <code>name</code> attribute of handle as a string.
    *
    * @return See above.
    */
   @Override
   public String _name()
   {
      String name = super._name();
      
      if (name == null)
      {
         displayNotPrepared();
      }
      
      return name;
   }
   
   /**
    * Check if this resource supports the PREV-SIBLING attribute.
    * 
    * @return   Always <code>false</code>.
    */
   @Override
   protected boolean hasPrevSibling()
   {
      return false;
   }
   
   /**
    * Check if this resource supports the NEXT-SIBLING attribute.
    * 
    * @return   Always <code>false</code>.
    */
   @Override
   protected boolean hasNextSibling()
   {
      return false;
   }
   
   /**
    * Check if this resource supports the NAME attribute only in read-only mode.
    * 
    * @return   Always <code>false</code>, as NAME attribute is writable.
    */
   @Override
   protected boolean hasNameReadOnly()
   {
      return false;
   }
   
   /**
    * Allows each resource to declare whether it has some naming constraints.
    *
    * @return  This method always returns {@code false}, requiring that its name should be
    *          validated before assignment.
    *
    * @see  #validateName
    */
   @Override
   protected boolean hasNameConstraints()
   {
      return true;
   }
   
   /**
    * Reports if this object is valid for use.  
    *
    * @return   <code>true</code> if we are valid (can be used).
    */
   public boolean valid()
   {
      return !deleted;
   }
   
   /**
    * Worker to be implemented by each resource.  Called by {@link #delete()}.
    * 
    * @return   <code>true</code> if the resource was deleted.
    */
   protected boolean resourceDelete()
   {
      if (BufferManager.get().postponeTableHandleDelete(this))
      {
         return false;
      }
      
      forceDelete();
      return true;
   }
   
   /**
    * Unconditionally delete all buffers for this table, drop DMO definitions and temporary table,
    * clear TEMP-TABLE data structures and mark the table as deleted (i.e. subsequent
    * {@link #valid()} calls return <code>false</code>).
    */
   public void forceDelete()
   {
      forceClear();
      
      TempTable before = getBeforeTableNative();
      if (before != null && before.valid())
      {
         before.delete();
      }
      
      deleted = true;
   }
   
   /**
    * Check if the resource can veto the delete or not, before attempting the {@link #delete()} call.
    * 
    * @param    ref
    *           The exact handle holding the reference, on which the delete is attempted.
    * @param    explicit
    *           Flag indicating this is called via <code>DELETE OBJECT</code> from the application.
    * 
    * @return   <code>false</code> if the handle reference is registered as an OUTPUT TABLE-HANDLE parameter,
    *           or the dataset for any temp-table's buffers is registered as an OUTPUT TABLE-HANDLE parameter.
    */
   @Override
   public boolean allowDelete(handle ref, boolean explicit)
   {
      if (BufferManager.get().isOutputTableHandle(ref))
      {
         setPostponedDelete(true);
         return false;
      }
      
      DataSetManager dmgr = DataSetManager.instance();

      // check the buffers
      Iterator<Buffer> iter = allBuffers.iterator();
      while (iter.hasNext())
      {
         Buffer buffer = iter.next();
         if (buffer.valid())
         {
            DataSet bufds = ((BufferImpl) buffer)._dataSet();
            if (bufds != null && bufds.valid())
            {
               if (dmgr.isOutputDataSetHandle(bufds))
               {
                  setPostponedDelete(true);

                  return false;
               }
               else if (explicit)
               {
                  ErrorManager.recordOrThrowError(12326,
                           "Cannot delete temp-table member " + _name() +
                           " of dataset  without deleting the dataset first. (12326)",
                           false,
                           true);

                  return false;
               }
            }
         }
      }
      
      return true;
   }
   
   /**
    * Get the collection of table fields.
    *
    * @return  the collection of table fields.
    */
   @Override
   public Collection<P2JField> getFields()
   {
      return fields.values();
   }
   
   /**
    * Get field by its name.
    *
    * @param  fieldName
    *         Field name.
    *
    * @return field with the specified name (normalized field names are compared).
    */
   public P2JField getField(String fieldName)
   {
      return fields.get(TextOps.rightTrimNative(fieldName));
   }
   
   /**
    * Get index by its name.
    *
    * @param  indexName
    *         Index name.
    *
    * @return index with the specified name (normalized index names are compared).
    */
   public P2JIndex getIndex(String indexName)
   {
      return indexes.get(TextOps.rightTrimNative(indexName));
   }
   
   /**
    * Get table indexes.
    *
    * @return see above.
    */
   public Collection<P2JIndex> getIndexes()
   {
      return indexes.values();
   }
   
   /**
    * Obtain the {@code DmoMeta} structure which contains the legacy meta information.
    *
    * @return  {@code DmoMeta} structure which contains the legacy meta information
    */
   @Override
   public DmoMeta getDmoMeta()
   {
      return dmoInfo;
   }
   
   /**
    * Get DMO interface which backs this table.
    *
    * @return DMO interface which backs this table or {@code null} if the table is not
    *         initialized yet.
    */
   @Override
   public Class<? extends DataModelObject> getDMOInterface()
   {
      return dmoIface;
   }
   
   /**
    * Get buffer interface which backs this table.
    *
    * @return DMO buffer interface which backs this table or {@code null} if the table is not
    *         initialized yet.
    */
   @Override
   public Class<? extends TempTableBuffer> getDMOBufInterface()
   {
      return dmoBufIface;
   }
   
   /**
    * Get DMO implementation class associated with this table.
    *
    * @return DMO implementation class associated with this table or <code>null</code> if the
    *         table is not initialized yet.
    */
   @Override
   public Class<? extends Record> getDMOClass()
   {
      return dmoInfo.getImplementationClass();
   }
   
   /**
    * Set code page supplier for the specified CLOB field.
    *
    * @param   property
    *          The ORM property name of the CLOB field.
    * @param   supplier
    *          The code page supplier of the specified CLOB field.
    */
   @Override
   public void setCodePageSupplier(String property, Supplier supplier)
   {
      throw new IllegalStateException("Code page cannot be set for a dynamic table");
   }
   
   /**
    * Set code page for the specified CLOB field.
    *
    * @param   property
    *          The ORM property name of the CLOB field.
    * @param   codePage
    *          The code page of the specified CLOB field.
    */
   @Override
   public void setCodePage(String property, String codePage)
   {
      throw new IllegalStateException("Code page cannot be set for a dynamic table");
   }
   
   /**
    * Get code page of the specified CLOB field.
    *
    * @param   property
    *          The ORM property name of the CLOB field.
    *
    * @return  The code page of the specified CLOB field or {@code null} if not configured so the default must
    *          be used.
    */
   @Override
   public String getCodePage(String property)
   {
      String fieldName4GL = DynamicTablesHelper.get4GLFieldName(dmoIface, property);
      return fields.get(TextOps.rightTrimNative(fieldName4GL)).getCodePage();
   }
   
   /**
    * Get code page supplier of the specified CLOB field.
    *
    * @param   property
    *          The ORM property name of the CLOB field.
    *
    * @return <code>null</code> in this implementation.
    */
   public Supplier getCodePageSupplier(String property)
   {
      return null;
   }
   
   /**
    * Set DMO interface class which corresponds the table.
    *
    * @param   dmoInfo
    *          Object which contains metadata about the DMO.
    * @param   dmoIface
    *          DMO interface class which corresponds the table.
    * @param   dmoBufIface
    *          DMO buffer interface class which corresponds the table.
    */
   void setDmoIface(DmoMeta dmoInfo, 
                    Class<? extends DataModelObject> dmoIface, 
                    Class<? extends TempTableBuffer> dmoBufIface)
   {
      this.dmoInfo = dmoInfo;
      this.dmoIface = dmoIface;
      this.dmoBufIface = dmoBufIface;
   }
   
   /**
    * Determine if an ADD-* function was called for the table (CLEAR function resets the state).
    *
    * @return  {@code true} if an ADD-* function was called for the table.
    */
   protected boolean addFunctionWasCalled()
   {
      return addFunctionCalled;
   }
   
   /**
    * Add new field from property definition data.
    *
    * @param   propertyDefinition
    *          Definition of the field to add.
    */
   protected void addNewField(PropertyDefinition propertyDefinition)
   {
      ParmType parmType = ParmType.fromClass(propertyDefinition.getType());
      
      Long extent = propertyDefinition.isExtent() ? (long) propertyDefinition.getExtent() : null;

      addNewField(propertyDefinition.getLegacyName(),
                  parmType,
                  extent,
                  propertyDefinition.getFormat(),
                  propertyDefinition.getInitial(),
                  propertyDefinition.getLabel(),
                  propertyDefinition.getColumnLabel(),
                  propertyDefinition.getHelp(),
                  propertyDefinition.getXmlNodeType(),
                  propertyDefinition.getCodePage(),
                  propertyDefinition.isSerializeHidden(),
                  propertyDefinition.isCaseSensitive());
   }
   
   /**
    * Adds a field with the specified properties to the temp-table.
    * This method is the P2J equivalent of <code>ADD-NEW-FIELD</code> method
    * of Progress 4GL.
    * <p>
    * When a parameter is set to null, it means it was not specified by the business logic.
    *
    * @param   name
    *          The name of the field to be created in the temp-table.
    * @param   type
    *          The data type of the specified field.
    * @param   extent
    *          An integer expression specifying the extent of an array.
    * @param   format
    *          The data format for the defined data type. If empty or unknown, format is
    *          replaced with default format. Null value means that format is not used in
    *          ADD-NEW-FIELD.
    * @param   initial
    *          An expression that evaluates to the initial value of the defined field.
    *          TODO: this method will probably be overloaded because of this.
    * @param   label
    *          The label of the defined field. If <code>null</code> or unknown the name
    *          parameter will be used.
    * @param   columnLabel
    *          The label of the column associated with the defined field
    * @param   help 
    *          Field HELP attribute value.
    * @param   xmlNodeType
    *          Field XML node type. 
    * @param   codePage 
    *          Field code page. 
    * @param   serializeHidden
    *          Field SERIALIZE-HIDDEN attribute value.
    * @param   caseSensitive 
    *          Field CASE-SENSITIVE attribute value.
    *
    * @return  <code>true</code> on success.
    */
   private boolean addNewField(String   name,
                               ParmType type,
                               Long     extent,
                               String   format,
                               String   initial,
                               String   label,
                               String   columnLabel,
                               String   help,
                               String   xmlNodeType,
                               String   codePage,
                               boolean  serializeHidden,
                               boolean  caseSensitive)
   {

      addFunctionCalled = true;
      
      if (_prepared())
      {
         displayPrepared();
         return false;
      }
      
      if (format == null || format.length() == 0)
      {
         format = getDefaultDataTypeFormat(type.name());
      }
      else
      {
         if (!validateFormat(type, format))
         {
            ErrorManager.recordOrShowError(9052,
                                           String.format("Illegal format given in ADD-NEW-FIELD %s", format),
                                           true,
                                           false);
            return false;
         }
      }
      
      long ext = extent == null || extent == 1 ? 0 : extent;
      P2JField field = new P2JField(name, type, ext, format,
                                    new character(initial), label, columnLabel, caseSensitive, 
                                    codePage, help, serializeHidden, null, null,
                                    null, xmlNodeType, false, 0, 0);
      
      addField(field);
      return true;
   }

   /**
    * Duplicates the structure of an existing table, copying the field definitions, and
    * optionally, the indexes.
    * This method is the P2J equivalent of {@code CREATE-LIKE} method of Progress 4GL.
    *
    * @param   tableName
    *          The name of a table to create the copy of.
    * @param   indexName
    *          The name of a single index which is copied from the source table.
    * @param   handleIndex
    *          <code>true</code> if <code>indexName</code> parameter should be taken into
    *          consideration. This parameter is required because <code>unknown</code> index name
    *          represents an index with empty name rather than "index not specified" case.
    *
    * @return  <code>true</code> on success.
    */
   private logical createTableLikeImpl(character tableName,
                                       character indexName,
                                       boolean handleIndex)
   {
      if (addFunctionCalled)
      {
         displayCreateLikeNotFirst();
         return new logical();  // unknown is returned
      }
      
      addFunctionCalled = true;
      
      String fullTableName = tableName.getValue();
      if (fullTableName == null)
      {
         displayUnableCreateLike();
         return new logical(false);
      }
      
      Persistence persistence = findExistingTable(fullTableName, true);
      if (persistence == null)
      {
         displayUnableCreateLike();
         return new logical(false);
      }
      
      String tblName = DynamicTablesHelper.parseTableName(fullTableName)[0];
      addAllFields(getExistingFields(persistence, tblName));
      
      if (handleIndex)
      {
         String idxName = indexName.isUnknown() ? "" : indexName.getValue(); // correct
         P2JIndex srcIndex = getExistingIndex(persistence, tblName, idxName);
         if (srcIndex == null)
         {
            displayUnableToFindIndex(DynamicTablesHelper.get4GLTableName(persistence, tblName), idxName);
            displayUnableCreateLike();
            return new logical(false);
         }
         else
         {
            addIndex(srcIndex);
            return new logical(true);
         }
      }
      else
      {
         addAllIndexes(getExistingIndexes(persistence, tblName));
         return new logical(true);
      }
   }
   
   /**
    * Duplicates the structure of an existing table, copying the field
    * definitions, and optionally, the indexes.
    * This method is the P2J equivalent of <code>CREATE-LIKE</code> method
    * of Progress 4GL.
    *
    * @param   hbuf
    *          A handle to a buffer or table from which to copy the definitions.
    * @param   indexName
    *          The name of a single index which is copied from the source table.
    * @param   handleIndex
    *          <code>true</code> if <code>indexName</code> parameter should be taken into
    *          consideration. This parameter is required because <code>unknown</code> index name
    *          represents an index with empty name rather than "index not specified" case.
    *
    * @return  <code>true</code> on success.
    */
   private logical createTableLikeImpl(handle hbuf, character indexName, boolean handleIndex)
   {
      if (addFunctionCalled)
      {
         displayCreateLikeNotFirst();
         return logical.UNKNOWN;  // unknown is returned
      }
      
      addFunctionCalled = true;
      
      RecordBuffer recordBuffer = getRecordBufferByHandle(hbuf, CallingFunction.CREATE_LIKE);
      if (recordBuffer == null)
      {
         return logical.FALSE;
      }
      
      // check before-table
      BufferImpl buf = (BufferImpl) recordBuffer.getDMOProxy();
      implicitBeforeTable = (buf.isAfterBuffer() && buf.peerBuffer != null);

      if (fields != null)
      {
         Map<String, P2JField> existingFields = recordBuffer.getDmoInfo().getExistingFieldsMap();
         for (P2JField field : existingFields.values())
         {
            fields.put(field.getName(), field);
         }
      }

      if (handleIndex)
      {
         String idxName = indexName.isUnknown() ? "" : indexName.getValue(); // correct
         P2JIndex srcIndex = getExistingIndex(recordBuffer, idxName);
         if (srcIndex == null)
         {
            displayUnableToFindIndex(recordBuffer.getDmoInfo().legacyTable, idxName);
            displayUnableCreateLike();
            return logical.FALSE;
         }
         else
         {
            addIndex(srcIndex);
            return logical.FALSE;
         }
      }
      else
      {
         addAllIndexes(getExistingIndexes(recordBuffer));
         return logical.TRUE;
      }
      
      // NOTE: the table attributes (like NO-UNDO, NAMESPACE-URI, etc) are not copied from source table.
      //       For NO-UNDO, the default value is FALSE. If needed, it should be set manually, using
      //                            <tt-handle>:UNDO = <not no-undo value>.
   }
   
   /**
    * Adds to the temp-table an index like the one specified. This method is the P2J equivalent
    * of <code>ADD-LIKE-FIELD</code> method of Progress 4GL.
    *
    * @param   srcIndex
    *          The index in the source table that is being copied to the temp-table.
    *
    * @return  <code>true</code> on success.
    */
   private logical addIndexLike(P2JIndex srcIndex)
   {
      String srcIndexName = srcIndex.getName();
      
      if (hasIndex(srcIndexName))
      {
         displayDuplicateIndex(srcIndexName);
         displayUnableToCreateLikeIndex();
         displayUnableToCreateLikeIndex(); // this message is displayed twice
         return logical.FALSE;
      }
      
      P2JIndex targetIndex = copyIndex(srcIndex);
      boolean res = true;

      ArrayList<P2JIndexComponent> comps = srcIndex.components();
      for (int i = 0; i < comps.size(); i++)
      {
         P2JIndexComponent srcComponent = comps.get(i);
         String componentLegacyName = srcComponent.getLegacyName();
         if (!hasField(componentLegacyName))
         {
            displayDuplicateOrInvalidIndexField(srcIndexName, componentLegacyName);
            displayUnableToCreateLikeIndex();
            displayUnableToCreateLikeIndex(); // this message is displayed twice
            res = false;
            break;
         }
         else
         {
            targetIndex.addComponent(componentLegacyName, null, null,
                                     srcComponent.isDescending(),
                                     srcComponent.isCharType(),
                                     srcComponent.isIgnoreCase(),
                                     srcComponent.isAbbreviated());
         }
      }
      
      addIndex(targetIndex);
      return logical.of(res);
   }
   
   /**
    * Add specified set of fields to the table. If a field is in the except-list or was already
    * added to the table, it is ignored.
    * 
    * @param srcFields
    *        Set of fields to be added.
    * @param except
    *        Comma-separated list of field names that should be excluded from the copy process.
    */
   private void doAddFieldsFrom(Collection<P2JField> srcFields, character except)
   {
      if (srcFields != null)
      {
         Set<String> exceptList = parseFieldList(except);
         
         for (P2JField srcField : srcFields)
         {
            // names in except list should match original field names, case sensitivity isn't
            // taken into consideration, but spaces are taken into consideration
            String fieldName = srcField.getName().toLowerCase();
            if (!hasField(fieldName) && (exceptList == null || !exceptList.contains(fieldName)))
               addField(srcField);
         }
      }
   }
   
   /**
    * Create default buffer for the table.
    */
   private void createDefaultBuffer()
   {
      if (defaultBuffer == null)
      {
         Temporary buffer = TemporaryBuffer.createDefaultDynamicBufferForTempTable(this);
         defaultBuffer = (BufferImpl) buffer;
      }
   }
   
   /**
    * Display "Unable to prepare TEMP-TABLE fields" error message.
    */
   private void displayUnableToPrepareTableFields()
   {
      ErrorManager.recordOrShowError(9044, "Unable to prepare TEMP-TABLE fields", true, false);
   }
   
   /**
    * Display "Unable to prepare TEMP-TABLE indices" error message.
    */
   private void displayUnableToPrepareTableIndices()
   {
      ErrorManager.recordOrShowError(9045, "Unable to prepare TEMP-TABLE indices", true, false);
   }
   
   /**
    * Display "Unable to evaluate arguments for TEMP-TABLE method" error message.
    */
   private void displayUnableToEvaluate()
   {
      ErrorManager.recordOrShowError(9050,
                                     "Unable to evaluate arguments for TEMP-TABLE method",
                                     true,
                                     false);
   }
   
   /**
    * Display "Could not find table {tableName} during ADD-LIKE-FIELD. Table names must not be
    * abbreviated." error message.
    *
    * @param tableName
    *        Table name.
    */
   private void displayCouldNotFindLikeTable(String tableName)
   {
      ErrorManager.recordOrShowError(9054,
                                     String.format("Could not find table %s during ADD-LIKE-FIELD." +
                                                   "  Table names must not be abbreviated", tableName),
                                     true,
                                     false);
   }
   
   /**
    * Display "TEMP-TABLE method could not find LIKE table name {tableName}}" error message.
    *
    * @param tableName
    *        Table name.
    */
   private void displayCouldNotFindLikeTableName(String tableName)
   {
      ErrorManager.recordOrShowError(9057,
                                     String.format("TEMP-TABLE method could not find LIKE table name %s", tableName),
                                     true,
                                     false);
   }
   
   /**
    * Display "Duplicate index name {indexName} for temp-table" error message.
    *
    * @param indexName
    *        Index name.
    */
   private void displayDuplicateIndex(String indexName)
   {
      ErrorManager.recordOrShowError(3390,
                                     String.format("Duplicate index name %s for temp-table ", indexName),
                                     true,
                                     false);
   }
   
   /**
    * Display "Could not find source field {fieldName} during ADD-LIKE-FIELD." error message.
    *
    * @param fieldName
    *        Field name.
    */
   private void displayCouldNotFindLikeField(String fieldName)
   {
      ErrorManager.recordOrShowError(9055,
                                     String.format("Could not find source field %s during ADD-LIKE-FIELD", fieldName),
                                     true,
                                     false);
   }
   
   /**
    * Display "ADD-LIKE-INDEX could not find index {indexName} in table {tableName}" error
    * message.
    *
    * @param tableName
    *        Table name.
    * @param indexName
    *        Index name.
    */
   private void displayCouldNotFindLikeIndex(String tableName, String indexName)
   {
      String msg = String.format("ADD-LIKE-INDEX could not find index %s in table %s", indexName, tableName);
      ErrorManager.recordOrShowError(9059, msg, true, false);
   }
   
   /**
    * Display "Could not find index {indexName} during ADD-INDEX-FIELD" error message.
    *
    * @param indexName
    *        Index name.
    */
   private void displayCouldNotFindAddIndex(String indexName)
   {
      ErrorManager.recordOrShowError(9061,
                                     String.format("Could not find index %s during ADD-INDEX-FIELD", indexName),
                                     true,
                                     false);
   }
   
   /**
    * Display "Unable to create index for TEMP-TABLE in ADD-LIKE-INDEX" error message.
    */
   private void displayUnableToCreateLikeIndex()
   {
      ErrorManager.recordOrShowError(9060,
                                     "Unable to create index for TEMP-TABLE in ADD-LIKE-INDEX",
                                     true,
                                     false);
   }
   
   /**
    * Display "Attempt to add duplicate or inappropriate word index field {fieldName} to index
    * {indexName}" error message.
    *
    * @param indexName
    *        Index name.
    * @param fieldName
    *        Field name.
    */
   private void displayDuplicateOrInvalidIndexField(String indexName, String fieldName)
   {
      String msg = String.format("Attempt to add duplicate or inappropriate word index field %s to index %s",
                                 fieldName,
                                 indexName);
      ErrorManager.recordOrShowError(9063, msg, true, false);
   }
   
   /**
    * Display "Unable to find index {indexName} in table {tableName}" error message.
    *
    * @param tableName
    *        Table name.
    * @param indexName
    *        Index name.
    */
   private void displayUnableToFindIndex(String tableName, String indexName)
   {
      ErrorManager.recordOrShowError(9048,
                                     String.format("Unable to find index %s in table %s", indexName, tableName),
                                     true,
                                     false);
   }
   
   /**
    * Unconditionally delete all buffers for this table, drop DMO definitions and temporary table,
    * clear TEMP-TABLE data structures.
    *
    * @return <code>true</code> if operation was successful.
    */
   private boolean forceClear()
   {
      if (_prepared())
      {
         DynamicTablesHelper helper = DynamicTablesHelper.getInstance();
         helper.deregisterBuilder(this);
         helper.deregisterDmoUser(this);
         
         deleteAllBuffers();
         defaultBuffer = null;
      }
      
      // keep UNIQUE-ID attribute
      addFunctionCalled = false;
      undoable = true; // however default value is "false"
      fields.clear();
      indexes.clear();
      errorString = null;
      error = false;
      admData = null;
      dmoIface = null;
      name = null;
      
      return true;
   }
   
   /**
    * Split comma-separated list of field names into the set of separate field names.
    *
    * @param  list
    *         Comma-separated list of fields.
    *
    * @return The set of separate field names or <code>null</code> if there are no fields in the
    *         input list. Names are returned in lower case, spaces are kept.
    */
   private Set<String> parseFieldList(character list)
   {
      if (list.isUnknown())
      {
         return null;
      }
      
      String val = list.getValue();
      if (val.length() == 0)
      {
         return null;
      }
      
      String[] splited = val.toLowerCase().split(",");
      return new HashSet<>(Arrays.asList(splited));
   }
   
   /**
    * Search for an existing table with the specified name. Displays error message in the case of
    * error.
    *
    * @param  fullTableName
    *         Table name in format "databaseName.tableName" or "tableName". In the latter case
    *         table search is performed in all permanent databases and then the temporary
    *         database. Dynamic tables are ignored.
    * @param  tableNameSearch
    *         <code>true</code> if it is called from ADD-LIKE-INDEX, ADD-FIELDS-FROM or
    *         CREATE-LIKE, <code>false</code> if it is called from ADD-FIELD-LIKE. Affects
    *         error messaging.
    *
    * @return persistence which corresponds the target database or <code>null</code> if there is
    *         no such table.
    */
   private Persistence findExistingTable(String fullTableName, boolean tableNameSearch)
   {
      Persistence persistence = DynamicTablesHelper.findExistingTable(fullTableName);
      if (persistence == null)
      {
         if (tableNameSearch)
            displayCouldNotFindLikeTableName(fullTableName);
         else
            displayCouldNotFindLikeTable(fullTableName);
      }
      return persistence;
   }
   
   /**
    * Determines if two names has the same normalized form. See
    * {@link TextOps#rightTrimLower(String)} for more information.
    *
    * @param  name1
    *         First name to compare.
    * @param  name2
    *         Second name to compare.
    *
    * @return <code>true</code> if two names has the same normalized form.
    */
   private boolean hasSameNormalizedName(String name1, String name2)
   {
      return TextOps.rightTrimLower(name1).equals(TextOps.rightTrimLower(name2));
   }
   
   /**
    * Returns a new field with the new specified name and all other parameters copied from the
    * source field.
    *
    * @param  targetName
    *         The name of the new field.
    * @param  srcField
    *         Source field to copy.
    *
    * @return see above.
    */
   private P2JField copyField(String targetName, P2JField srcField)
   {
      return new P2JField(targetName,
                          srcField.getType(),
                          srcField.getExtent(),
                          srcField.getFormat(),
                          srcField.getInitial(),
                          srcField.getLabel(),
                          srcField.getColumnLabel(),
                          srcField.isCaseSensitive(),
                          srcField.getCodePage(),
                          srcField.getHelp(),
                          srcField.isSerializeHidden(),
                          srcField.getSerializeName(),
                          srcField.getXmlDataType(),
                          srcField.getXmlNodeName(),
                          srcField.getXmlNodeType(),
                          srcField.isMandatory(),
                          srcField.getScale(),
                          srcField.getMaxWidth());
   }
   
   /**
    * Returns a new index with all parameters copied from the source index. Index components are
    * NOT copied.
    *
    * @param  srcIndex
    *         Source index to copy.
    *
    * @return see above.
    */
   private P2JIndex copyIndex(P2JIndex srcIndex)
   {
      return new P2JIndex(srcIndex.getTable(),
                          srcIndex.getName(),
                          srcIndex.isUnique(),
                          srcIndex.isPrimary(),
                          srcIndex.isWord());
   }
   
   /**
    * Add new field to the table. Doesn't check whether the field with the same name already exists.
    * <p>
    * Note: 
    * This method was made public in order to be directly accessed from
    * {@link com.goldencode.p2j.persist.serial.XmlImport}.
    *
    * @param field
    *        Field to add.
    */
   public void addField(P2JField field)
   {
      fields.put(TextOps.rightTrimNative(field.getName()), field);
   }
   
   /**
    * Read XML/JSON data from the input source and store it in the just created empty TEMP-TABLE.
    *  
    * @param   tableName 
    *          table name
    * @param   reader 
    *          XML/JSON source reader and table creator.
    *          
    * @return  {@code true} on success.
    * 
    * @throws  PersistenceException
    *          if there is any error reading or storing XML/JSON data.
    */
   @Override
   public boolean readTable(String tableName, TableReader reader) 
   throws PersistenceException
   {
      return reader.readTable(tableName, this, RecordBuffer::loadRecord);
   }

   /**
    * Add new index to the table. Doesn't check whether the index with the same name already
    * exists.
    *
    * @param index
    *        Index to add.
    */
   private void addIndex(P2JIndex index)
   {
      indexes.put(TextOps.rightTrimNative(index.getName()), index);
   }
   
   /**
    * Checks whether the field with specified name already exist.
    *
    * @param  fieldName
    *         Field name to check.
    *
    * @return <code>true</code> if there is a field which has the same name (in normalized form,
    *         see {@link TextOps#rightTrimLower(String)}).
    */
   private boolean hasField(String fieldName)
   {
      return fields.containsKey(TextOps.rightTrimNative(fieldName));
   }
   
   /**
    * Checks whether the index with specified name already exist.
    *
    * @param  indexName
    *         Index name to check.
    *
    * @return <code>true</code> if there is an index which has the same name (in normalized form,
    *         see {@link TextOps#rightTrimLower(String)}).
    */
   private boolean hasIndex(String indexName)
   {
      return indexes.containsKey(TextOps.rightTrimNative(indexName));
   }
   
   /**
    * Add all indexes from the given collection to the table.
    *
    * @param indexes
    *        Indexes to add.
    */
   private void addAllIndexes(Collection<P2JIndex> indexes)
   {
      if (indexes != null)
      {
         for (P2JIndex index : indexes)
         {
            addIndex(index);
         }
      }
   }
   
   /**
    * Add all fields from the given collection to the table.
    *
    * @param fields
    *        Fields to add.
    */
   private void addAllFields(Collection<P2JField> fields)
   {
      if (fields != null)
      {
         for (P2JField field : fields)
         {
            addField(field);
         }
      }
   }
   
   /**
    * Add all fields from the source Buffer to this table.
    *
    * @param   source
    *          Fields to add.
    */
   private void addAllFields(BufferImpl source)
   {
      if (fields == null)
      {
         return;
      }
      
      int n = source._numFields();
      for (int k = 1; k <= n; k++)
      {
         BufferFieldImpl field = (BufferFieldImpl) source.bufferFieldNative(k);
         fields.put(field._name(), field.asP2JField());
      }
   }
   
   /**
    * Add indexes based on multiIxCols string.
    * 
    * @param tableIndexes
    *        multiIxCols string to be parsed.
    */
   private void parseIndexString(String tableIndexes)
   {
      if (tableIndexes != null)
      {
         String[] indexes = tableIndexes.split("\\.");
         for (int i = 0; i < indexes.length; i++)
         {
            String indexDef = indexes[i];
            if (indexDef.isEmpty())
            {
               continue;
            }
            String[] compAndName = indexDef.split(":");
            String typeAndComps = compAndName[0];
            String idxName = compAndName[1];
            String[] comps = typeAndComps.split(",");
            boolean unique = (i != 0) || "1".equals(comps[0]);
            boolean primary = (i == 0) && ("1".equals(comps[0]) || "0".equals(comps[0]));
            
            addNewIndex(new character(idxName),
                        logical.of(unique),
                        logical.of(primary));
            for (int j = (primary ? 1 : 0); j < comps.length; j++)
            {
               addFieldToIndex(idxName, comps[j]);
            }
         }
      }
   }

   /**
    * Returns a JSON representation of the object.
    *
    * @return JSON representation of the object, or any error that occurred.
    */
   public String toString()
   {
      if (!_prepared())
      {
         // calling writeJson()n at this time will lead to NPE
         return "{\"error\": \"dynamic temp-table not yet prepared.\"}";
      }
      if (defaultBuffer == null)
      {
         return "{\"error\": \"Default buffer is null\"}";
      }
      
      String myName = (name == null) ? "unnamed" : name().getValue();
      String staticName = "DYNAMIC " + myName + ", ID " + getUniqueID().getValue();
      
      String resultString;
      ObjectMapper objectMapper = new ObjectMapper();
      Map<String, Object> jsonObject = new HashMap<>();
      Map<String, Object> tableData = new HashMap<>();
      RecordBufferSerializer recordBufferSerializer = new RecordBufferSerializer(defaultBuffer.buffer());
      try
      {
         tableData.put(myName, recordBufferSerializer.sqlTableContent());
         jsonObject.put(staticName, tableData);
         resultString = objectMapper.writeValueAsString(jsonObject);
      }
      catch (PersistenceException | SQLException e)
      {
         resultString = "{\"error\": \"Error getting data: " + e.getMessage() + "\"}";
      }
      catch (JsonProcessingException e)
      {
         LOG.severe("JSON processing error: " + e.getMessage());
         resultString = "{\"error\": \"Error generating JSON\"}";
      }
      return resultString;
   }
}