NameConverter.java

/*
** Module   : NameConverter.java
** Abstract : Converts Progress symbols to Java/SQL-compatible symbols.
**
** Copyright (c) 2005-2025, Golden Code Development Corporation.
**
** -#- -I- --Date-- --JPRM-- -----------------------------------Description-----------------------------------
** 001 ECF 20050418   @20753 Created initial version. Converts Progress
**                           symbol names to the equivalent Java variable,
**                           method, constant, or class name, or to the
**                           equivalent SQL database, table, column, or
**                           index name. Defines default substring
**                           replacements and naming conventions which can
**                           be overridden.
** 002 ECF 20050427   @20900 Better handling for multi-word replacements.
**                           Each word of the replacement is stored in a
**                           stack of pending replacements. The nextToken
**                           method returns one word at a time as a
**                           syllable. This ensures that the specialized
**                           syllable processing methods behave
**                           consistently for multi-word replacement text.
** 003 ECF 20050427   @20931 Enhancements. We now statically load a
**                           dictionary of default match phrases and their
**                           replacements. At construction, a user-defined
**                           subset of these entries are copied to the new
**                           instance. Non-standard replacements can
**                           optionally be disabled.
** 004 ECF 20050429   @20969 Fix for no-replace mode. When tokenizing but
**                           not replacing tokens, it is necessary to do a
**                           minimal sweep for illegal characters. The set
**                           of standard replacements is used for this
**                           purpose.
** 005 ECF 20050608   @21448 Changed NameConverter(String name, int type)
**                           constructor default behavior. Now replacement
**                           is enabled by default.
** 006 ECF 20050629   @21595 Made a NameConverter instance reusable. This
**                           required moving the name and type arguments
**                           from the constructors to the convert method,
**                           and making the convert method synchronized.
**                           This removes a huge performance penalty
**                           overhead in cases where the same match phrase
**                           entries are used across many conversions,
**                           since these entries can be loaded once rather
**                           than for every conversion.
** 007 ECF 20050719   @21772 Fix to nextToken method. Algorithm attempted
**                           to match candidate strings which were too
**                           long. Now candidates which are longer than
**                           the remaining symbol text are eliminated as
**                           match candidates. This avoids partial matches
**                           from appearing in the output.
** 008 GES 20051028   @23141 Added generateBeanName().
** 009 ECF 20060114   @23915 Moved generateBeanName() to StringHelper.
**                           Method is needed at runtime; this class is
**                           intended for conversion use only.
** 010 GES 20060201   @24175 Detect and resolve conflicts of Java names
**                           with reserved Java keywords.
** 011 GES 20060310   @24972 Support a forced full text replacement mode
**                           which provides a way to limit the impact of
**                           a common match string if the replacement
**                           can be forced only in the case that there is
**                           a match with the entire name.
** 012 GES 20060311   @24975 Added load/reload from a named match list.
** 013 ECF 20060331   @25331 Fix to #012 (@24975). Recursive design of
**                           convert method was not truly recursive, as
**                           each invocation was made on a new instance of
**                           the class.
** 014 ECF 20061020   @30589 Fixed the convert() method. The syllable
**                           counter must not advance for an empty
**                           syllable.
** 015 ECF 20070304   @32264 Respect camelCase as a syllable delimiter
**                           when converting a name. This takes precedence
**                           over other syllable tokenizing, since it
**                           represents an explicit intent by the name's
**                           original author.
** 016 ECF 20070305   @32267 Fix to camelCase preprocessing. Was not
**                           handling blocks of capital letters properly.
** 017 ECF 20071219   @36501 Fixed private convert() implementation. When
**                           called recursively, this method must not
**                           attempt to resolve possible Java keyword
**                           conflicts. This resolution is only done for
**                           the fully converted text, not per syllable.
** 018 GES 20090424   @41930 Import change.
** 019 SVL 20090709   @43129 Added keyword "count". Name conflicts are
**                           resolved by postpending an underscore (instead of
**                           prepending it).
** 020 GES 20091117   @44419 Moved the standard symbol replacements into this
**                           file and modified the loading mechanism to allow
**                           database-specific defaults to be loaded
**                           automatically.
** 021 CA  20130308          Convert dots in names to the "dot" string.
** 022 ECF 20131010          Separated Java and SQL reserved words, added some SQL keywords.
** 023 GES 20140118          Added convertPackage() static method.
** 024 ECF 20140923          Added SQL reserved keyword.
** 025 ECF 20141103          Added all H2 reserved keywords.
** 026 GES 20141126          Convert hyphens to underscores in pkg names.
** 027 OM  20150522          Added resetDialects() static API that allows each schema to have each
**                           keyword exclusion list. Generified collections.
** 028 EVL 20160223          Javadoc fixes to make compatible with Oracle Java 8 for Solaris 10.
** 029 OM  20160905          Small optimizations.
** 030 OM  20170207          Added TYPE_PROPERTY for special case of DMO property names.
** 031 ECF 20170816          Added 'assert' as a reserved java name.
** 032 OM  20170823          Updated reserved Java keyword list.
** 033 GES 20180113          In (all?) converted names, the first syllable cannot start with a
**                           numeric digit.
**     ECF 20180114          Fixed typo.
**     GES 20180131          Exclude internal calls from leading digit processing.
** 034 GES 20180905          Added replacements for other characters that can appear in malformed
**                           symbols.  Added cleanJavaIdentifier() to handle illegal characters
**                           in Java identfiers.
** 035 CA  20190508          Fixed convertPackage for builtin classes.
** 036 AIL 20191015          Added object as reserved keyword, as it caused HibernateException.
** 037 CA  20200324          Converted Java method names must not collide with FWD static imported
**                           methods (BlockManager is included for now).
** 038 CA  20200427          A 4GL package starting with a digit will be prefixed with an 
**                           underscore in FWD.
**     CA  20200503          Added more FWD classes for static imports, to avoid collisions with
**                           converted method names.
**     CA  20200514          Avoid collisions with java.lang.Object inherited methods.
** 039 ECF 20200906          New ORM implementation.
** 040 CA  20201015          Replaced java.util.Stack with a non-synchronized custom implementation.
**     CA  20210113          Added method to check if a method's Java name is used by conversion rules to emit
**                           it as an unreferenced static method call.  This is required to sanitize the 
**                           implemented skeleton's method names.
**     OM  20201231          Improved list of reserved SQL keywords.
**     OM  20210121          Tests against reservedSQL in lowercase because SQL is case insensitive.
**     OM  20210203          Avoid collision of buffer name and class with SQL keywords and java.lang classes.
**     CA  20220210          Added reservedFWD, which contains the BaseDataType types.
**     CA  20220621          Added p2j.ui.Element class name as reserved.
**     CA  20220426          Added the FWD DMO annotations as reserved class names.
**     CA  20220513          TransactionManager, SessionUtils FWD type names must be reserved.
**                           All public and protected method names from java.lang.Object must be reserved.
**     ECF 20220514          Allow SQL name conversion in two new modes: with minimal change (resolving
**                           illegal characters and keyword conflicts only); and verbatim (no change at all,
**                           with the name stored in double quotes). TODO: verbatim mode probably will cause
**                           problems with DMO property annotations.
**     CA  20220712          'TransactionType' and 'ResourceType' are reserved FWD class names.
**     OM  20230125          Avoid collisions of index names with SQL reserved keywords.
** 041 GBB 20230512          Logging methods replaced by CentralLogger/ConversionStatus.
** 042 CA  20241119          A DMO alias must not emit as a FQL reserved keyword.
** 043 DDF 20250317          Made 'Agent' a reserved FWD keyword.
** 044 CA  20250319          Added 'StringHelper' and 'ConnectionManager' as reserved names.
** 045 CA  20250401          Any possible Java import from core FWD packages can result in a 'schemaconflict',
**                           so the DMO name will be qualified if they match a simple core class name.
**                           'Agent' keyword is no longer reserved.
** 046 CA  20250422          Fixed H045 - for some reason, if the same package exists in multiple jars, if a
**                           lambda is used for the scanner, Reflections API will block.  An inner class fixes
**                           this.
**     CA  20250423          'Test' and 'Fixture' are reserved names in FWD.
*/

/*
** This program is free software: you can redistribute it and/or modify
** it under the terms of the GNU Affero General Public License as
** published by the Free Software Foundation, either version 3 of the
** License, or (at your option) any later version.
**
** This program is distributed in the hope that it will be useful,
** but WITHOUT ANY WARRANTY; without even the implied warranty of
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
** GNU Affero General Public License for more details.
**
** You may find a copy of the GNU Affero GPL version 3 at the following
** location: https://www.gnu.org/licenses/agpl-3.0.en.html
** 
** Additional terms under GNU Affero GPL version 3 section 7:
** 
**   Under Section 7 of the GNU Affero GPL version 3, the following additional
**   terms apply to the works covered under the License.  These additional terms
**   are non-permissive additional terms allowed under Section 7 of the GNU
**   Affero GPL version 3 and may not be removed by you.
** 
**   0. Attribution Requirement.
** 
**     You must preserve all legal notices or author attributions in the covered
**     work or Appropriate Legal Notices displayed by works containing the covered
**     work.  You may not remove from the covered work any author or developer
**     credit already included within the covered work.
** 
**   1. No License To Use Trademarks.
** 
**     This license does not grant any license or rights to use the trademarks
**     Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
**     of Golden Code Development Corporation. You are not authorized to use the
**     name Golden Code, FWD, or the names of any author or contributor, for
**     publicity purposes without written authorization.
** 
**   2. No Misrepresentation of Affiliation.
** 
**     You may not represent yourself as Golden Code Development Corporation or FWD.
** 
**     You may not represent yourself for publicity purposes as associated with
**     Golden Code Development Corporation, FWD, or any author or contributor to
**     the covered work, without written authorization.
** 
**   3. No Misrepresentation of Source or Origin.
** 
**     You may not represent the covered work as solely your work.  All modified
**     versions of the covered work must be marked in a reasonable way to make it
**     clear that the modified work is not originating from Golden Code Development
**     Corporation or FWD.  All modified versions must contain the notices of
**     attribution required in this license.
*/

package com.goldencode.p2j.convert;

import java.beans.*;
import java.lang.reflect.*;
import java.util.*;
import java.util.function.Predicate;
import java.util.logging.*;

import org.reflections.*;
import org.reflections.scanners.*;

import com.goldencode.util.Stack;
import com.goldencode.p2j.cfg.*;
import com.goldencode.p2j.cfg.Configuration;
import com.goldencode.p2j.persist.*;
import com.goldencode.p2j.persist.annotation.*;
import com.goldencode.p2j.persist.annotation.Trigger;
import com.goldencode.p2j.persist.dialect.*;
import com.goldencode.p2j.persist.meta.*;
import com.goldencode.p2j.persist.orm.FQLLexer;
import com.goldencode.p2j.schema.*;
import com.goldencode.p2j.testengine.api.Fixture;
import com.goldencode.p2j.testengine.api.Test;
import com.goldencode.p2j.ui.Element;
import com.goldencode.p2j.ui.LogicalTerminal;
import com.goldencode.p2j.util.*;
import com.goldencode.util.*;

/**
 * Converter to generate a Java or SQL symbol name from a Progress symbol
 * (variable, function, procedure, filename, or schema construct). A
 * <code>NameConverter</code> instance is unaware of the type of symbol
 * being converted, but it is aware of the type of symbol which must result
 * from the conversion. The latter is specified during construction using
 * one of this class' type constants.
 * <p>
 * Sun-like naming conventions are used to generate Java variable, method,
 * "constant", and class names:
 * <ul>
 * <li>variable and method names are "camel"-cased, with the first letter
 *     always lower case (override {@link #processVariableSyllable} and
 *     {@link #processMethodSyllable}, respectively, to modify);
 * <li>"constant" names are all upper case, with syllables separated by an
 *     an underscore (override {@link #processConstantSyllable} to modify);
 * <li>class names are "camel"-cased, with the first letter always upper
 *     case (override {@link #processClassSyllable} to modify).
 * <li>conflicts with reserved Java keywords are resolved 
 * </ul>
 * The following naming conventions are used for SQL constructs:
 * <ul>
 * <li>database names are all lower case, with syllables separated by an
 *     underscore (override {@link #processDatabaseSyllable} to modify);
 * <li>table names are all lower case, with syllables separated by an
 *     underscore (override {@link #processTableSyllable} to modify);
 * <li>column names are all lower case, with syllables separated by an
 *     underscore (override {@link #processColumnSyllable} to modify);
 * <li>index names are all lower case, with syllables separated by an
 *     underscore (override {@link #processIndexSyllable} to modify).
 * </ul>
 *
 * The converter implementation includes a generic substring replacement
 * facility to permit custom naming conventions and to handle the fact that
 * Java and SQL symbols supported a narrower character set within their
 * respective symbol names. All replacement definitions are loaded from a
 * match phrase dictionary, which is project-specific.
 * <p>
 * Replacement entries can be overridden as a whole via the {@link
 * #setReplacements} method. Individual replacements can be overridden, and
 * arbitrary replacements added, via the {@link #putReplacement} method.
 * If more than one substring key matches within a symbol name, the following
 * precedence rules apply:
 * <ul>
 * <li>a match beginning earlier in the symbol string (left to right) is
 *     always preferred over an overlapping match beginning later in the
 *     symbol string;
 * <li>given two overlapping matches beginning at the same location in the
 *     symbol string, the more specific (i.e., longer) match string is always
 *     preferred over the less specific (i.e., shorter) match string.
 * </ul>
 *
 * Multi-word replacement strings are handled such that each word is treated
 * as a separate syllable in the output string.
 * <p>
 * Except for the type-specific rules described above, case is generally
 * preserved in the converted symbol name. However, substring matches for
 * the purpose of performing replacements are case-insensitive.
 * <p>
 * TODO:  conversion to Java variable and method names should match the
 * behavior of java.bean.Introspector.decapitalize().  Currently, it does so
 * in 99.9% of cases, but it seems that certain character combinations are
 * handled differently, such as when underscore follows a first, capital
 * letter.
 * <p>
 * TODO:  if the generic replacement functionality in this class is useful
 * in other contexts, this functionality should be abstracted into a parent
 * class (e.g., StringConverter). The key methods which would be moved into
 * this parent would be:
 * <ul>
 * <li>{@link #setReplacements} - no changes
 * <li>{@link #putReplacement} - no changes
 * <li>{@link #convert} - no changes
 * <li>{@link #nextToken} - no changes
 * <li>{@link #processSyllable} - would return syllable text unmodified by
 *     default; subclasses would override.
 * <li>{@link #prepareCandidates} - no changes
 * </ul>
 * All of the instance and class variables related to conversion state would
 * also move there. In addition to a no-arg default constructor, a convenience
 * constructor would require a map of replacements as its argument.
 */
public class NameConverter
implements MatchPhraseConstants
{
   /** Used when matching substrings within symbol; indicates no match */
   private static final int NO_MATCH = 0;
   
   /** Used when matching substrings within symbol; indicates partial match */
   private static final int PARTIAL_MATCH = 1;
   
   /** Used when matching substrings within symbol; indicates full match */
   private static final int FULL_MATCH = 2;
   
   /** The map of replacements used for minimal SQL name conversion */
   private static final Map<String, String> sqlMinimalReplacements;
   
   /** Maps digit characters to safe replacement digit names. */
   private static final Map<Character, String> digits = new HashMap<>();
   
   /** A set of Java methods which can be imported static. */
   private static final Set<String> staticImportedMethods = new HashSet<>();
   
   /** A set of Java classes which are imported by default (from package {@code java.lang}). */
   private static final Set<String> defaultImportedClasses = new HashSet<>();
   
   /** The set of methods inherited from Java's {@link java.lang.Object} root class. */
   private static final Set<String> objectMethods = new HashSet<>();

   /** Default replacement mappings. */
   private static Map<String, String> mappings = null;
   
   /** Default substring replacements, indexed by substring to be replaced */
   private static MatchPhraseDictionary defaults = null;
   
   /** Logger */
   private static final ConversionStatus LOG = ConversionStatus.get(NameConverter.class);

   /** The list of imports emitted during conversion, via {@link ConverterHelper#createImport}. */
   private static final String[] FWD_CORE_IMPORTS = new String[]
   {
      "com.goldencode.p2j.comauto.*",
      "com.goldencode.p2j.email.*",
      "com.goldencode.p2j.extension.*",
      "com.goldencode.p2j.oo.lang.*",
      "com.goldencode.p2j.persist.*",
      "com.goldencode.p2j.persist.lock.*",
      "com.goldencode.p2j.persist.trigger.*",
      "com.goldencode.p2j.reporting.*",
      "com.goldencode.p2j.security.*",
      "com.goldencode.p2j.testengine.api.*",
      "com.goldencode.p2j.uast.ProgressAst",
      "com.goldencode.p2j.ui.*",
      "com.goldencode.p2j.util.*",
      "com.goldencode.p2j.xml.*",
      "com.goldencode.util.*"
   };
   
   /** The list of all simple Java class names from the core imported packages, during conversion. */
   private static final Set<String> FWD_CORE_SIMPLE_NAMES = new HashSet<>();
   
   // get all (direct) classes in common packages:
   static
   {
      for (String s : FWD_CORE_IMPORTS)
      {
         if (s.endsWith(".*"))
         {
            s = s.substring(0, s.length() - ".*".length());
            Reflections r = new Reflections(s, Scanners.SubTypes.filterResultsBy(new Predicate<String>()
            {
               @Override
               public boolean test(String t)
               {
                  return true;
               }
            }));
            Set<Class<?>> types = r.getSubTypesOf(Object.class);
            for (Class<?> c : types)
            {
               if (c.getEnclosingClass() != null)
               {
                  // skip inner classes
                  continue;
               }

               String name = c.getName();
               if (c.getPackage().getName().equals(s))
               {
                  FWD_CORE_SIMPLE_NAMES.add(c.getSimpleName());
               }
            }
         }
         else
         {
            FWD_CORE_SIMPLE_NAMES.add(s.substring(s.lastIndexOf('.') + 1));
         }
      }
   }
   
   // Set up default substring replacements
   static
   {
      // TODO: provide a way to honor I18N here
      digits.put('0', "zero");
      digits.put('1', "one");
      digits.put('2', "two");
      digits.put('3', "three");
      digits.put('4', "four");
      digits.put('5', "five");
      digits.put('6', "six");
      digits.put('7', "seven");
      digits.put('8', "eight");
      digits.put('9', "nine");
      
      try
      {
         mappings = new HashMap<>();
         
         // Implement the default standard mappings (this may get overridden
         // by subsequent user-defined configuration files).
         mappings.put("!", "exclaim");
         mappings.put("@", "at");
         mappings.put("#", "number");
         mappings.put("$", "dollar");
         mappings.put("%", "percent");
         mappings.put("^", "caret");
         mappings.put("&", "and");
         mappings.put("*", "asterisk");
         mappings.put("+", "plus");
         mappings.put("-", "");
         mappings.put("_", "");
         mappings.put("/", "");
         mappings.put(".", "dot");
         
         defaults = MatchPhraseDictionary.load(false, mappings);
         
         sqlMinimalReplacements = new HashMap<>(mappings);
         sqlMinimalReplacements.put("-", "_");
         sqlMinimalReplacements.put("/", "_");
         sqlMinimalReplacements.remove("_");
      }
      catch (Exception exc)
      {
         throw new RuntimeException("Error", exc);
      }
   }
   
   /** All Java language reserved keywords or named literals */
   private static final Set<String> reservedJava = new HashSet<>();
   
   /** Keywords which are reserved in FWD - this includes unqualified FWD types emitted in converted code. */
   private static final Set<String> reservedFWD = new HashSet<>();
   
   /** Problematic SQL keywords */
   private static final Set<String> reservedSQL = new HashSet<>();
   
   /** Currently processed database name */
   private static String currentSchemaName = null; 
   
   static
   {
      String[] reservedNames = new String[]
      {
         "abstract",
         "assert",      // added in J2SE 1.4
         "boolean",
         "break",
         "byte",
         "case",
         "catch",
         "char",
         "class",
         "const",       // reserved but not used
         "continue",
         "default",
         "do",
         "double",
         "else",
         "enum",        // added in J2SE 5.0
         "extends",
         "final",
         "finally",
         "float",
         "for",
         "goto",        // reserved but not used
         "if",
         "implements",
         "import",
         "instanceof",
         "int",
         "interface",
         "long",
         "native",
         "new",
         "object",
         "package",
         "private",
         "protected",
         "public",
         "return",
         "short",
         "static",
         "strictfp",    // added in J2SE 1.2
         "super",
         "switch",
         "synchronized",
         "this",
         "throw",
         "throws",
         "transient",
         "try",
         "void",
         "volatile",
         "while",
         "true",        // boolean literal value
         "false",       // boolean literal value
         "null",        // null reference literal value.
      };
      reservedJava.addAll(Arrays.asList(reservedNames));
      
      // add reserved FWD types.  currently this includes BaseDataType sub-types
      for (BaseDataType.Type type : BaseDataType.Type.values())
      {
         reservedFWD.add(type.getCls().getSimpleName());
      }
      // add other types
      reservedFWD.add(Element.class.getSimpleName());
      reservedFWD.add(BlockManager.TransactionType.class.getSimpleName());
      reservedFWD.add(ResourceType.class.getSimpleName());
      
      // add DMO annotations
      reservedFWD.add(Index.class.getSimpleName());
      reservedFWD.add(IndexComponent.class.getSimpleName());
      reservedFWD.add(Indices.class.getSimpleName());
      reservedFWD.add(Property.class.getSimpleName());
      reservedFWD.add(Relation.class.getSimpleName());
      reservedFWD.add(RelationComponent.class.getSimpleName());
      reservedFWD.add(Relations.class.getSimpleName());
      reservedFWD.add(Sequence.class.getSimpleName());
      reservedFWD.add(Table.class.getSimpleName());
      reservedFWD.add(Trigger.class.getSimpleName());
      reservedFWD.add(Triggers.class.getSimpleName());
          
      // add other FWD classes
      reservedFWD.add(TransactionManager.class.getSimpleName());
      reservedFWD.add(SessionUtils.class.getSimpleName());
      reservedFWD.add(ConnectionManager.class.getSimpleName());
      reservedFWD.add(StringHelper.class.getSimpleName());
      reservedFWD.add(Test.class.getSimpleName());
      reservedFWD.add(Fixture.class.getSimpleName());

      // WARNING: Adding other classes/interfaces to this list may invalidated the already implemented
      // skeleton classes, as these must not collide with statically emitted APIs during conversion.
      Class<?>[] staticImportClasses =
      {
         BlockManager.class,
         Enum.class,
         LogicalTerminal.class,
         TextOps.class,
         ParameterOption.class
      };
      for (Class<?> cls : staticImportClasses)
      {
         Method[] mthds = cls.getDeclaredMethods();
         for (Method mthd : mthds)
         {
            int mods = mthd.getModifiers();
            if (Modifier.isPublic(mods) && Modifier.isStatic(mods))
            {
               staticImportedMethods.add(mthd.getName());
            }
         }
      }
      
      Method[] mthds = Object.class.getDeclaredMethods();
      for (Method mthd : mthds)
      {
         int mods = mthd.getModifiers();
         if (Modifier.isPublic(mods) || Modifier.isProtected(mods) )
         {
            objectMethods.add(mthd.getName());
         }
      }
   }
   
   /** A stack of pending replacement syllables detected via lookahead */
   private final Stack<String> pendingStack = new Stack<>();
   
   /** Map of substring replacements, indexed by substring to be replaced */
   private Map<String, String> replacements = null;
   
   /** Map of standard replacements, applied in case of no-replace mode */
   private Map<String, String> standards = null;
   
   /** Map of "full text only" replacements. */
   private Map<String, String> fulltxt = null;
   
   /** Array of candidates when matching substrings for replacement */
   private String[] candidates = null;
   
   /** Array of match state for each candidate during a match operation */
   private int[] matches = null;
   
   /** Original name to be converted */
   private String name = null;
   
   /** SQL name conversion override mode, if any */
   private int sqlOverride = SQL_MODE_DEFAULT;
   
   /** Type of conversion being performed (using this class' constants) */
   private int type = 0;
   
   /** Current position in original name symbol */
   private int index = 0;
   
   /** A substring match detected via lookahead while processing next token */
   private String pendingMatch = null;
   
   /**
    * Default constructor; <code>ALL</code> default match phrases from the
    * project-specific match phrase dictionary are loaded, and replacement
    * is enabled.
    */
   public NameConverter()
   {
      this(ALL, true);
   }
   
   /**
    * Constructor which accepts a code indicating which default matches to
    * pick up from the project's match phrase dictionary. The caller also
    * chooses whether matched tokens are replaced or left in place.
    *
    * @param   phrases
    *          A bitwise-OR'd combination of {@link MatchPhraseConstants}
    *          flags indicating which set of default match phrase entries
    *          to load from the project-specific match phrase dictionary.
    * @param   replace
    *          If <code>true</code>, any token from the match phrase entries
    *          defined by <code>phrases</code> which is found within
    *          <code>name</code> is replaced with its designated replacement
    *          string. If <code>false</code>, only tokens from the standard
    *          entries (if <code>phrases</code> includes the constant
    *          <code>STANDARD</code>) are replaced. All other tokens are
    *          recognized as separate syllables, but are not replaced.
    */
   public NameConverter(int phrases, boolean replace)
   {
      if (replace)
      {
         // get the requested replacement entries from the default match phrase dictionary.
         replacements = defaults.getEntries(phrases);
      }
      else
      {
         // We will need these to sweep tokens for minimal, standard
         // replacements. This ensures we don't end up with illegal
         // characters in the output string.
         standards = defaults.getEntries(STANDARD);
         
         // Copy only standard replacement entries (if included in phrases
         // parameter) into our local replacement map.
         if ((phrases & STANDARD) != 0)
         {
            replacements = defaults.getEntries(STANDARD);
         }
         else
         {
            replacements = new HashMap<>();
         }
         
         // For all non-standard entries required by the phrases parameter,
         // copy the match substrings as both the replacement key and value.
         // Any token detected will be treated as a separate syllable and
         // will be replaced with itself.
         int nonStandard = (phrases & ~STANDARD);
         Map<String, String> map = defaults.getEntries(nonStandard);
         for (String key : map.keySet())
         {
            replacements.put(key, key);
         }
      }
      
      // if space is not explicitly handled already, add a replacement for it.
      if (!replacements.containsKey(" "))
      {
         replacements.put(" ", "");
      }
      
      // Load the full text list.
      fulltxt = defaults.getFullText();
   }
   
   /**
    * Check if the given Java method name is used as a static method call emitted by the conversion rules.
    * 
    * @param    javaname
    *           The Java method name.
    *           
    * @return   <code>true</code> if this is part of {@link #staticImportedMethods} or {@link #objectMethods}.
    *           Note that some method names (like "invoke") are not emitted statically by the conversion rules,
    *           and are explicitly handled.
    */
   public static boolean isInternalMethodName(String javaname)
   {
      // this is a list of method names which are not emitted during conversion, as unreferenced static method
      // calls.
      String[] notStaticFWDApis = 
      {
         "invoke",
         "debug",
         "getClient",
         "getUserName",
         "initialize",
         "isEmpty",
         "validate"
      };
      Set<String> s = new HashSet<>(Arrays.asList(notStaticFWDApis));
      if (s.contains(javaname))
      {
         return false;
      }

      return staticImportedMethods.contains(javaname) || objectMethods.contains(javaname);
   }
   
   /**
    * Check if this simple Java name can collide with one of the {@link #FWD_CORE_SIMPLE_NAMES} resolved from
    * {@link #FWD_CORE_IMPORTS}.
    * 
    * @param    name
    *           The simple java name.
    *           
    * @return   See above.
    */
   public static boolean coreSimpleNameConflict(String name)
   {
      return name != null && !name.isEmpty() && FWD_CORE_SIMPLE_NAMES.contains(name);
   }
   
   /**
    * Reset the internal keyword exclusion list for the next schema to be processed.
    *
    * The method looks for the list of dialect used for generation of the schema and computes
    * the "least common denominator". This list is obtained by merging the ANSI SQL standard
    * keywords list with the list of each declared dialect.
    * 
    * @param   schema
    *          The schema to prepare for.
    */
   public static void resetDialects(String schema)
   {
      if (schema.equals(currentSchemaName))
      {
         // already prepared for this database (this will happen only for _temp database, this
         // method will be called for each .p.schema file that contain temp-tables but we don't
         // need to reset the exclusion list because we are handling the same database)
         return;
      }
      currentSchemaName = schema;
      
      // reset the old content
      reservedSQL.clear();
      
      // ANSI SQL reserved keywords are always on the exclusion list
      String[] ansiKeywords = {
         "by",
         "count",
         "cross",
         "current_date",
         "current_time",
         "current_timestamp",
         "distinct",
         "except",
         "exists",
         "false",
         "fetch",
         "for",
         "from",
         "full",
         "group",
         "having",
         "inner",
         "intersect",
         "is",
         "join",
         "like",
         "limit",
         "minus",
         "natural",
         "not",
         "null",
         "offset",
         "on",
         "order",
         "primary",
         "rownum",
         "select",
         "sysdate",
         "systime",
         "systimestamp",
         "today",
         "true",
         "union",
         "unique",
         "where",
      };
      reservedSQL.addAll(Arrays.asList(ansiKeywords));
      
      String[] dialects = null;
      String dialectList = null;
      if (MetadataManager.META_SCHEMA.equals(schema))
      {
         SchemaConfig.Metadata md = Configuration.getSchemaConfig().getMetadata();
         if (md != null)
         {
            schema = md.getName();
         }
      }
      if (!DatabaseManager.TEMP_TABLE_SCHEMA.equals(schema))
      {
         // the getNamespaceParameter() can potentially throw NPE if the schema is not
         // defined in as a namespace in the config file
         // TODO: leave this NPE propagate or should getNamespaceParameter() return null instead? 
         dialectList = Configuration.getSchemaConfig().getNamespaceParameter(schema, "ddl-dialects");
      }
      
      if (dialectList == null)
      {
         // no dialects declared for this database/schema (_meta & _temp) databases?
         // add automatically h2 dialect
         dialects = new String[] { "h2" };
      }
      else
      {
         dialects = dialectList.split(",");
      }
      
      for (String dialectId : dialects)
      {
         Class<? extends Dialect> dialect = DialectHelper.getDialectClass(dialectId);
         if (dialect != null)
         {
            try
            {
               Method m =  dialect.getDeclaredMethod("getReservedKeywords");
               Collection<String> keyList = (Collection<String>) m.invoke(null);
               
               reservedSQL.addAll(keyList);
            }
            catch (NoSuchMethodException | InvocationTargetException | IllegalAccessException e)
            {
               LOG.log(Level.WARNING, "'getReservedKeywords()' method is NOT defined in " + dialect);
            }
         }
         else
         {
            LOG.log(Level.WARNING, "Unknown dialect '" + dialectId + "' for '" + schema + "' schema.");
         }
      }
   }
   
   /**
    * Load or reload the match phrase dictionary definitions from the given
    * persisted match list or the default defined for the current project. 
    * Any cached version of the dictionary is replaced if it has been read
    * previously.
    *
    * @param   list
    *          The match list to use or <code>null</code> to use the default
    *          match list.
    * @param   db
    *          <code>true</code> to add the database specific default mappings,
    *          otherwise only add the code mappings.
    *
    * @throws  ConfigurationException
    *          if there is any error with the global configuration or an
    *          error loading the dictionary from persistence. A missing
    *          match list file is not fatal and <em>does not</em> trigger
    *          this exception, but does elicit a warning message to
    *          <code>stderr</code>.
    */
   public static void load(String list, boolean db)
   throws ConfigurationException
   {
      // implement the default standard mappings (this may get overridden
      // by subsequent user-defined configuration files).
      HashMap<String, String> std = new HashMap<>(mappings);
      
      defaults = MatchPhraseDictionary.load(list, std);
      
      // add database-specific mappings if requested AND only if they don't already exist.
      if (db)
      {
         Map<String, String> map = defaults.getFullText();
         
         // if project uses old database primary key name ("id"), make sure we convert any
         // occurrences to a safe default; newer default ("recid") cannot conflict, as it is a
         // reserved word in 4GL
         String pkName = Configuration.getSchemaConfig().getPrimaryKeyName();
         if ("id".equals(pkName) && map.get(pkName) == null)
         {
            defaults.putFullText("id", "identifier");
         }
         
         if (map.get("date") == null)
         {
            defaults.putFullText("date", "dateValue");
         }
      }
   }
   
   /**
    * Create a (mostly) Java-bean compatible getter or setter name for a
    * given field.  If the field is a <code>boolean</code> and a getter is
    * requested, then the prefix will be "is", otherwise getters are prefixed 
    * with "get".  All setters are prefixed with "set".  The name of the
    * field always has its first character uppercased which is slightly
    * different than how Sun defines the standard.
    *
    * @param    field
    *           The name to use as the basis, must be non-null and non-empty.
    * @param    getter
    *           If <code>true</code> the resulting bean name will be a
    *           getter.
    * @param    bool
    *           If <code>getter</code> is <code>true</code> and this is
    *           <code>true</code> then the resulting bean name will prefixed
    *           with "is" instead of "get".  Ignored for setters.
    *
    * @return   The resulting getter or setter name.
    *
    * @throws   IllegalArgumentException
    *           If the field name is <code>null</code> or is the empty string.
    */
   public static String generateBeanName(String  field, boolean getter, boolean bool)
   throws IllegalArgumentException
   {
      return StringHelper.generateBeanName(field, getter, bool);
   }
   
   /**
    * Get the name being converted.
    *
    * @return  Original name.
    */
   public String getName()
   {
      return name;
   }
   
   /**
    * Completely override the default substring replacement definitions with
    * the map specified. For each entry within the map, the key should be
    * the substring to be replaced, and the value should be the replacement
    * substring. Keys are case-insensitive; however, case is generally
    * preserved in the replacement value, though it may be overridden
    * depending upon the target symbol type.
    *
    * @param   replacements
    *          Map of substrings to match and the text which should replace
    *          them in the converted symbol name, as described above.
    */
   public void setReplacements(Map<String, String> replacements)
   {
      this.replacements = replacements;
      candidates = null;
   }
   
   /**
    * Override an individual, default substring replacement definition, or
    * add an arbitrary substring replacement definition to the default
    * definitions.
    *
    * @param   key
    *          The substring to be replaced in the original symbol name.
    *          The match against this key is case-insensitive.
    * @param   value
    *          The text to be substituted in the target symbol name, wherever
    *          <code>key</code> is found in the original. Case is generally
    *          preserved in the replacement value, though it may be
    *          overridden depending upon the target symbol type.
    */
   public void putReplacement(String key, String value)
   {
      key = key.toLowerCase();
      replacements.put(key, value);
      candidates = null;
      
      // Update standards map if necessary.
      if (standards != null && standards.containsKey(key))
      {
         standards.put(key, value);
      }
   }
   
   /**
    * Perform the symbol name conversion and return the converted name.
    * This process breaks the original symbol into "syllables", though
    * these are not traditional, English-language syllables. A syllable
    * in this context is any substring within the original symbol which
    * precedes or succeeds a substring replacement, or which is itself
    * the replacement text resulting from such an operation.
    * <p>
    * For example, the symbol <code>test-name</code> is processed as three
    * syllables: <code>test</code> (precedes the replacement of
    * "<code>-</code>") and <code>name</code> (succeeds the replacement of
    * "<code>-</code>"). In addition, the empty string which replaces the
    * hyphen character between these two syllables is itself considered as
    * the second syllable for processing purposes, even though it does not
    * appear in the converted result:
    * <ul>
    * <li>for type TYPE_VARIABLE: <code>testName</code>
    * <li>for type TYPE_METHOD: <code>testName</code>
    * <li>for type TYPE_CONSTANT: <code>TEST_NAME</code>
    * <li>for type TYPE_CLASS: <code>TestName</code>
    * <li>for type TYPE_DATABASE: <code>test_name</code>
    * <li>for type TYPE_TABLE: <code>test_name</code>
    * <li>for type TYPE_COLUMN: <code>test_name</code>
    * <li>for type TYPE_INDEX: <code>test_name</code>
    * </ul>
    *
    * If the starting symbol were instead <code>test%name</code>, the second
    * syllable would instead be <code>Pct</code> (using the default set of
    * replacements).
    * <p>
    * The {@link #processSyllable} method is called for each syllable
    * encountered. The string returned from that method is appended to the
    * target symbol. This occurs until the entire, original symbol name has
    * been processed.
    *
    * @param   name
    *          Name to be converted.
    * @param   type
    *          One of the type constants defined in this class. Indicates
    *          what type of Java or SQL symbol should result from the
    *          conversion.
    *
    * @return  The fully converted symbol name.
    */
   public synchronized String convert(String name, int type)
   {
      return convert(name, type, false);
   }
   
   /**
    * Lowercase and split the input into individual package names, then ensure that each
    * one is not a conflict with a Java reserved word. This will append an underscore to any
    * conflicting package name.
    *
    * @param    pkg
    *           A candidate Java package name, possibly including multiple period-separated
    *           directories.
    *
    * @return   The reassembled and non-conflicting result.
    */
   public static String convertPackage(String pkg)
   {
      String p = pkg.toLowerCase();
      boolean builtin = false;
      if (p.startsWith("progress.") || p.startsWith("openedge."))
      {
         builtin = true;
         pkg = pkg.substring(9);
      }
      else if (p.startsWith("ccs."))
      {
         builtin = true;
         pkg = pkg.substring(4);
      }
      // this can be static because it doesn't access any instance data nor need any of the
      // normal syllable processing
      
      // TODO: this really should be put through the standard processing because there is no
      //       guarantee that the inputs are valid Java package names; we should add TYPE_PKG...
      
      String[] pieces = pkg.toLowerCase().split("\\.");
      
      StringBuilder sb = new StringBuilder();
      
      for (int i = 0; i < pieces.length; i++)
      {
         if (i > 0)
         {
            sb.append(".");
         }
         
         String tok = resolvePossibleKeywordConflict(pieces[i], TYPE_CLASS);
         if (!tok.isEmpty() && Character.isDigit(tok.charAt(0)))
         {
            tok = "_" + tok;
         }
         sb.append(tok.replaceAll("-", "_"));
      }
      
      return (builtin ? "com.goldencode.p2j.oo." : "") + sb.toString();
   }
   
   /**
    * Retrieve a string describing the conversion type performed by this
    * converter. This is useful for reporting and debugging.
    *
    * @return  Description of conversion type.
    */
   public String describeConversion()
   {
      String desc = null;
      
      switch (type)
      {
         case TYPE_SIMPLE:
            desc = "Simple";
            break;
         case TYPE_VARIABLE:
            desc = "Java variable";
            break;
         case TYPE_METHOD:
            desc = "Java method";
            break;
         case TYPE_CONSTANT:
            desc = "Java \"constant\"";
            break;
         case TYPE_CLASS:
            desc = "Java class";
            break;
         case TYPE_DATABASE:
            desc = "SQL database";
            break;
         case TYPE_TABLE:
            desc = "SQL table";
            break;
         case TYPE_COLUMN:
            desc = "SQL column";
            break;
         case TYPE_INDEX:
            desc = "SQL index";
            break;
         case TYPE_PROPERTY:
            desc = "DMO property";
            break;
         case TYPE_DMO_ALIAS:
            desc = "DMO alias";
            break;
      }
      
      return desc;
   }
   
   /**
    * Get a string representation of this object. Reports the text being
    * converted and the type of conversion performed.
    *
    * @return  String describing this object.
    */
   public String toString()
   {
      return name + " (" + describeConversion() + ")";
   }
   
   /**
    * Process each syllable which will become a component of the converted
    * symbol name. This method is called <em>after</em> all substring
    * replacements have already been made. Depending upon the type of the
    * target symbol, this method performs different conversions on the
    * syllable text, as discussed in this class' {@link NameConverter
    * description}.
    *
    * @param   text
    *          Text which comprises the current syllable being processed.
    * @param   syllable
    *          Zero-based index of this syllable in the converted symbol
    *          name.
    *
    * @return  The current symbol after any type-specific transformation
    *          has been applied.
    */
   protected String processSyllable(String text, int syllable)
   {
      switch (type)
      {
         case TYPE_SIMPLE:
            return text;
         case TYPE_VARIABLE:
         case TYPE_DMO_ALIAS:
            return processVariableSyllable(text, syllable);
         case TYPE_METHOD:
         case TYPE_PROPERTY:
            return processMethodSyllable(text, syllable);
         case TYPE_CONSTANT:
            return processConstantSyllable(text, syllable);
         case TYPE_CLASS:
            return processClassSyllable(text, syllable);
         case TYPE_DATABASE:
            return processDatabaseSyllable(text, syllable);
         case TYPE_TABLE:
            return processTableSyllable(text, syllable);
         case TYPE_COLUMN:
            return processColumnSyllable(text, syllable);
         case TYPE_INDEX:
            return processIndexSyllable(text, syllable);
         default:
            throw new IllegalArgumentException("Illegal type: " + type);
      }
   }
   
   /**
    * Process a syllable for a Java variable symbol name.
    * <p>
    * Default processing is to lowercase the first letter of the first
    * symbol, and uppercase the first letter of all subsequent syllables.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processVariableSyllable(String text, int syllable)
   {
      return processVarMeth(text, syllable);
   }
   
   /**
    * Process a syllable for a Java method symbol name.
    * <p>
    * Default processing is to lowercase the first letter of the first
    * symbol, and uppercase the first letter of all subsequent syllables.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processMethodSyllable(String text, int syllable)
   {
      return processVarMeth(text, syllable);
   }
   
   /**
    * Process a syllable for a Java "constant" symbol name.
    * <p>
    * Default processing is to uppercase all letters and prepend an
    * underscore character to each syllable after the first.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processConstantSyllable(String text, int syllable)
   {
      if (text.length() == 0)
      {
         return text;
      }
      
      StringBuilder buf = new StringBuilder();
      
      // Set all characters to upper case;  separate syllables with an
      // underscore.
      if (syllable > 0)
      {
         buf.append('_');
      }
      
      buf.append(text.toUpperCase());
      
      return buf.toString();
   }
   
   /**
    * Process a syllable for a Java class or interface symbol name.
    * <p>
    * Default processing is to uppercase the first letter of every syllable.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processClassSyllable(String text, int syllable)
   {
      if (text.length() <= 1)
      {
         return text.toUpperCase();
      }
      
      // Set first character of every syllable to upper case.
      return Character.toUpperCase(text.charAt(0)) + text.substring(1);
   }
   
   /**
    * Process a syllable for an SQL database symbol name.
    * <p>
    * Default processing is to lowercase all letters and prepend an
    * underscore character to each syllable after the first.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processDatabaseSyllable(String text, int syllable)
   {
      return processSql(text, syllable);
   }
   
   /**
    * Process a syllable for an SQL table symbol name.
    * <p>
    * Default processing is to lowercase all letters and prepend an
    * underscore character to each syllable after the first.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processTableSyllable(String text, int syllable)
   {
      return processSql(text, syllable);
   }
   
   /**
    * Process a syllable for an SQL column symbol name.
    * <p>
    * Default processing is to lowercase all letters and prepend an
    * underscore character to each syllable after the first.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processColumnSyllable(String text, int syllable)
   {
      return processSql(text, syllable);
   }
   
   /**
    * Process a syllable for an SQL index symbol name.
    * <p>
    * Default processing is to lowercase all letters and prepend an
    * underscore character to each syllable after the first.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   protected String processIndexSyllable(String text, int syllable)
   {
      return processSql(text, syllable);
   }
   
   /**
    * Get the active SQL name conversion override mode, if any.
    * 
    * @return  Constant for the active SQL name conversion override mode.
    * 
    * @see     MatchPhraseConstants
    */
   int getSqlOverride()
   {
      return sqlOverride;
   }
   
   /**
    * Set the active SQL name conversion override mode.
    * 
    * @param   sqlOverride
    *          Constant for the active SQL name conversion override mode.
    * 
    * @see     MatchPhraseConstants
    */
   void setSqlOverride(int sqlOverride)
   {
      this.sqlOverride = sqlOverride;
   }
   
   /**
    * Perform the symbol name conversion and return the converted name.
    * This process breaks the original symbol into "syllables", though
    * these are not traditional, English-language syllables. A syllable
    * in this context is any substring within the original symbol which
    * precedes or succeeds a substring replacement, or which is itself
    * the replacement text resulting from such an operation.
    * <p>
    * For example, the symbol <code>test-name</code> is processed as three
    * syllables: <code>test</code> (precedes the replacement of
    * "<code>-</code>") and <code>name</code> (succeeds the replacement of
    * "<code>-</code>"). In addition, the empty string which replaces the
    * hyphen character between these two syllables is itself considered as
    * the second syllable for processing purposes, even though it does not
    * appear in the converted result:
    * <ul>
    * <li>for type TYPE_VARIABLE: <code>testName</code>
    * <li>for type TYPE_METHOD: <code>testName</code>
    * <li>for type TYPE_CONSTANT: <code>TEST_NAME</code>
    * <li>for type TYPE_CLASS: <code>TestName</code>
    * <li>for type TYPE_DATABASE: <code>test_name</code>
    * <li>for type TYPE_TABLE: <code>test_name</code>
    * <li>for type TYPE_COLUMN: <code>test_name</code>
    * <li>for type TYPE_INDEX: <code>test_name</code>
    * <li>for type TYPE_PROPERTY: <code>testName</code> or <code>TName</code> in certain cases,
    *       where after a lowercase char there is a set of uppercases chars. This is done in
    *       order to match the way Hibernate expectes these property names.
    * </ul>
    *
    * If the starting symbol were instead <code>test%name</code>, the second
    * syllable would instead be <code>Pct</code> (using the default set of
    * replacements).
    * <p>
    * The {@link #processSyllable} method is called for each syllable
    * encountered. The string returned from that method is appended to the
    * target symbol. This occurs until the entire, original symbol name has
    * been processed.
    * <p>
    * This method may be invoked as an internal attempt to remove any illegal
    * characters from a converted syllable.  In this case, full text only
    * mode is ignored.
    *
    * @param   name
    *          Name to be converted.
    * @param   type
    *          One of the type constants defined in this class. Indicates
    *          what type of Java or SQL symbol should result from the
    *          conversion.
    * @param   internal
    *          <code>true</code> if this method is invoked due to internal
    *          housekeeping;  <code>false</code> if invoked as the result of
    *          an external call.
    *
    * @return  The fully converted symbol name.
    */
   private String convert(String name, int type, boolean internal)
   {
      boolean preprocess = true;
      this.type = type;
      this.name = name;
      
      switch (type)
      {
         // if a SQL name conversion override mode is active, bypass the normal name conversion logic for
         // table and column names (and simple names, which will be the type when calling this method in
         // a second pass for the minimal change mode)
         case TYPE_SIMPLE:
            if (!internal)
            {
               break;
            }
            // intentional fall through if internal
         case TYPE_TABLE:
         case TYPE_COLUMN:
         case TYPE_INDEX:
            switch (sqlOverride)
            {
               case SQL_MODE_MINIMAL:
                  if (!internal)
                  {
                     // return the name as is, modifying it only to resolve a keyword conflict or to replace
                     // illegal characters
                     return processSqlMinimal(name);
                  }
                  preprocess = false;
                  break;
               case SQL_MODE_VERBATIM:
                  if (!internal)
                  {
                     // return the name as is, in double quotes
                     return "\"" + name + "\"";
                  }
                  break;
               default:
                  break;
            }
            break;
         default:
            break;
      }
      
      if (preprocess)
      {
         this.name = preprocess(name);
      }
      index = 0;
      
      // honor full text only mode, if specified for the given name AND not an internal invocation
      if (!internal)
      {
         String result = fullTextReplacement(this.name);
         if (result != null)
         {
            return processSyllable(result, 0);
         }
      }
      
      prepareCandidates();
      
      StringBuilder buf = new StringBuilder();
      
      String tok = null;
      for (int i = 0; (tok = nextToken()) != null; i++)
      {
         if (standards != null)
         {
            // Sweep tok to at least apply standard replacements, in order
            // to prevent illegal characters in the output.
            NameConverter nc = new NameConverter(0, true);
            nc.setReplacements(standards);
            tok = nc.convert(tok, type, true);
         }
         
         // the next two cases must be protected with the !internal check because in internal
         // mode i == 0 when processing subsequent syllables; this really needs to be cleaned
         // up because it breaks the contract and makes it harder to implement proper logic
         
         // for all symbol types, the first character cannot start with a digit
         if (!internal && i == 0 && tok != null && tok.length() > 0)
         {
            tok = processLeadingDigit(tok);
         }
         
         // 
         if (!internal  &&
             (type == TYPE_VARIABLE || type == TYPE_METHOD || type == TYPE_PROPERTY ||
              type == TYPE_CONSTANT || type == TYPE_CLASS  || type == TYPE_DMO_ALIAS))
         {
            tok = cleanJavaIdentifier(tok, i);
         }
         
         buf.append(processSyllable(tok, i));
         
         // Adjust the syllable count downward to skip this syllable if,
         // after conversion, it's empty.
         if (tok == null || tok.length() == 0)
         {
            i--;
         }
      }
      
      String result = buf.toString();
      
      // special handling for DMO properties: in some cases, the property name does not have the
      // first letter in lowercase. This quirk is necessary for compatibility with Hibernate's
      // internal property name conversions. 
      if (type == TYPE_PROPERTY)
      {
         if (!result.isEmpty())
         {
            char[] chars = result.toCharArray();
            chars[0] = Character.toUpperCase(chars[0]);
            result = new String(chars);
         }
         result = Introspector.decapitalize(result);
      }
      
      // resolve possible keyword conflicts on the converted output;  we don't do this when called
      // recursively, because we don't want to process individual syllables, just the final result
      if (!internal)
      {
         result = resolvePossibleKeywordConflict(result, type);
      }
      
      return result;
   }
   
   /**
    * Preprocess the given name.  This involves splitting the name into
    * syllables which are already well established by camel casing, then
    * lowercasing the entire name.  Syllables are delimited by spaces in the
    * preprocessed string.
    *
    * @param   name
    *          Name to be converted.
    *
    * @return  Preprocessed name, lowercased and split into syllables divined
    *          from camel case in the original name.
    */
   private String preprocess(String name)
   {
      StringBuilder buf = new StringBuilder();
      int len = name.length();
      char last = 0;
      for (int i = 0; i < len; i++)
      {
         char c = name.charAt(i);
         if (Character.isUpperCase(c) && i > 0 && !Character.isUpperCase(last))
         {
            buf.append(' ');
         }
         buf.append(Character.toLowerCase(c));
         last = c;
      }
      
      return buf.toString().trim();
   }
   
   /**
    * Process a SQL name with minimal change, replacing only illegal characters and resolving keyword
    * conflicts. The original case of the characters in the name is preserved.
    * 
    * @param   name
    *          Table or field name to be converted.
    * 
    * @return  The given name, converted with minimal change.
    */
   private String processSqlMinimal(String name)
   {
      String result = name;
      
      // sweep name to apply standard replacements, in order to prevent illegal characters in the output
      NameConverter nc = new NameConverter(0, true);
      nc.setSqlOverride(sqlOverride);
      nc.setReplacements(sqlMinimalReplacements);
      result = nc.convert(result, TYPE_SIMPLE, true);
      
      result = resolvePossibleKeywordConflict(result, type);
      
      return result;
   }
   
   /**
    * If the given candidate name is marked as a forced full text match (a 
    * match that is only allowed with the full text of the name rather than
    * as a partial basis), return that replacement.
    *
    * @param   candidate
    *          The string that is to be tested.
    *
    * @return  The full text replacement or <code>null</code> if this 
    *          candidate is not forced into full text mode.
    */
   private String fullTextReplacement(String candidate)
   {
      return fulltxt.get(candidate);
   }
   
   /**
    * Process Java identifiers and filter out any characters that are not allowed. Illegal
    * characters will be converted to a 'uXXXX' format where the 16-bit hexidecimal char
    * value will be rendered as text.
    */
   private String cleanJavaIdentifier(String text, int syllable)
   {
      StringBuilder sb = new StringBuilder();
      
      for (int i = 0; i < text.length(); i++)
      {
         char ch = text.charAt(i);
         
         boolean bad = (syllable == 0 && i == 0) ? !Character.isJavaIdentifierStart(ch)
                                                 : !Character.isJavaIdentifierPart(ch);
         
         if (bad)
         {
            sb.append(String.format("u%04X", (int) ch));
         }
         else
         {
            sb.append(ch);
         }
      }
      
      return sb.toString();
   }
   
   /**
    * This is the backing method behind the {@link #processVariableSyllable}
    * and {@link #processMethodSyllable} methods, which both have the same
    * default behavior.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   private String processVarMeth(String text, int syllable)
   {
      int len = text.length();
      if (len < 1)
      {
         return text;
      }
      
      StringBuilder buf = new StringBuilder();
      
      // Set first character of first syllable to lower case;  first
      // character of all other syllables to upper case.
      if (syllable == 0)
      {
         buf.append(Character.toLowerCase(text.charAt(0)));
      }
      else
      {
         buf.append(Character.toUpperCase(text.charAt(0)));
      }
      
      if (len > 1)
      {
         buf.append(text.substring(1));
      }
         
      return buf.toString();
   }
   
   /**
    * This is the backing method behind the {@link #processDatabaseSyllable},
    * {@link #processTableSyllable}, {@link #processColumnSyllable}, and
    * {@link #processIndexSyllable} methods, which all have the same default
    * behavior.
    *
    * @param   text
    *          Text of current syllable.
    * @param   syllable
    *          Zero-based index of current syllable.
    */
   private String processSql(String text, int syllable)
   {
      if (text.length() == 0)
      {
         return text;
      }
      
      StringBuilder buf = new StringBuilder();
      
      // Set all characters to lower case;  separate syllables with an
      // underscore.
      if (syllable > 0)
      {
         buf.append('_');
      }
      
      buf.append(text.toLowerCase());
      
      return buf.toString();
   }
   
   /**
    * Generate the next token (aka syllable) to be appended to the target
    * symbol name. A token generated by this method will either be:
    * <ul>
    * <li>a run of characters for which no substring replacement could be
    *     matched; or
    * <li>the replacement text for a run of characters which was matched.
    * </ul>
    * Each token consists only of those characters up to the next run as
    * described above, or to the end of the original symbol string for the
    * last such run.
    * <p>
    * For example, the symbol <code>this&amp;that</code> would generate three
    * tokens (i.e., syllables), using the default replacement map:
    * <ol>
    * <li><code>this</code> - not matched for any substring replacement; its
    *     end is determined by the fact that the next character begins a
    *     a run of characters (in this case, one character, actually) which
    *     is matched to replacement key <code>&amp;</code>;
    * <li><code>and</code> - replacement text for substring
    *     <code>&amp;</code>;
    * <li><code>that</code> - not matched for any substring replacement; its
    *     end is determined by the end of the original token string.
    * </ol>
    *
    * @return  The next token, as described above, or <code>null</code> if
    *          we have reached the end of the original symbol string and
    *          we have no new characters to return.
    */
   private String nextToken()
   {
      // If we found a full match after finding one or more non-matching
      // characters in a previous pass through this method, we will have
      // a pending match. Return the next syllable of its replacement now
      // as the next token, advancing the index to just beyond the matched
      // text.
      String replacement = getPendingReplacement();
      if (replacement != null)
      {
         // Consume the pending match.
         if (pendingMatch != null)
         {
            index += pendingMatch.length();
            pendingMatch = null;
         }
         
         return replacement;
      }
      
      resetMatchStatus();
      
      // Create buffer for next token.
      StringBuilder buf = new StringBuilder();
      
      // Offset from beginning of match keys.
      int off = 0;
      
      // Number of full key matches at the current character from the
      // original symbol name.
      int full = 0;
      
      // Number of partial key matches at the current character from the
      // original symbol name.
      int partial = 0;
      
      // Index of the character on which we last started a match, in case
      // the match turns out to be a bust and we have to push back lookahead
      // characters.
      int lastMatchStart = -1;
      
      // Scan a character at a time of the original symbol name, starting
      // at the current index, as advanced by previous calls to this method.
      int len = name.length();
      int i;
      for (i = 0; index + i < len; i++)
      {
         // next character from original symbol; lowercase unless conversion type is simple.
         char ch = name.charAt(index + i);
         if (type != TYPE_SIMPLE)
         {
            ch = Character.toLowerCase(ch);
         }
         
         // Number of remaining characters in original symbol yet to be
         // tested.
         int remaining = len - index - i;
         
         // Walk the list of match statuses for each key and update, based
         // on whether the current character matches or not.
         for (int j = 0; j < matches.length; j++)
         {
            String key = candidates[j];
            if (matches[j] != FULL_MATCH)
            {
               int keyLen = key.length();
               if (keyLen - off <= remaining &&
                   off < keyLen &&
                   Character.toLowerCase(key.charAt(off)) == ch)
               {
                  if (off + 1 == keyLen)
                  {
                     // We have successfully matched the final character of
                     // the key. Update the key's status to a full match.
                     boolean fromPartial = (matches[j] == PARTIAL_MATCH);
                     if (fromPartial)
                     {
                        partial--;
                     }
                     
                     // Only consider this a full match if we were previously
                     // a partial match or if the key is only one character
                     // long.
                     if (fromPartial || keyLen == 1)
                     {
                        matches[j] = FULL_MATCH;
                        full++;
                     }
                  }
                  else if (matches[j] == NO_MATCH && off == 0)
                  {
                     // We have successfully matched the first character
                     // Update the key's status to a partial match.
                     partial++;
                     matches[j] = PARTIAL_MATCH;
                  }
               }
               else if (matches[j] == PARTIAL_MATCH)
               {
                  // We had a partial match, but the current character did
                  // not match. Update the key's status to no match.
                  matches[j] = NO_MATCH;
                  partial--;
               }
            }
         }
         
         /*
         // DEBUG CODE
         System.out.println("ch = " + ch);
         System.out.println("candidates = " + Arrays.asList(candidates));
         System.out.print("matches = [");
         for (int x = 0; x < matches.length; x++)
         {
            if (x > 0)
               System.out.print(", ");
            System.out.print(String.valueOf(matches[x]));
         }
         System.out.println("]");
         System.out.println("i = " + i);
         System.out.println("index = " + index);
         System.out.println("len = " + len);
         System.out.println("partial = " + partial);
         System.out.println("full = " + full);
         System.out.println();
         */
         
         if (partial > 0 || full > 0)
         {
            // Increment the key offset if any key had any match.
            off++;
            
            // Remember where this match began, in case it fails and we
            // need to push back the lookahead.
            if (lastMatchStart < 0)
            {
               lastMatchStart = i;
            }
         }
         else if (partial == 0 && full == 0)
         {
            // Reset the key offset if no keys had a match.
            off = 0;
            
            // Conditionally push back.
            if (lastMatchStart >= 0)
            {
               resetMatchStatus();
               
               // Push back all characters to the position just after where
               // we last matched.
               i = lastMatchStart;
               lastMatchStart = -1;
               
               // Truncate the token buffer accordingly.
               buf.setLength(i + 1);
               
               // Skip the remainder of the outer loop, because we don't
               // want to add the current character to the buffer after
               // we've truncated it.
               continue;
            }
         }
         
         if (full > 0 && (partial == 0 || (index + i) == (len - 1)))
         {
            // At least one full match has been found, AND all partial
            // matches have resolved to either a full match or no match
            // at all OR we are at the end of the input string.
            
            // Always prefer the most specific (i.e., longest), matching
            // key.
            String key = null;
            for (int k = 0; k < matches.length; k++)
            {
               if (matches[k] == FULL_MATCH)
               {
                  String next = candidates[k];
                  if (key == null || next.length() > key.length())
                  {
                     key = next;
                  }
               }
            }
            
            // Set the pending replacement word(s) into the pending stack.
            // These will be popped as replacements are returned from this
            // method.
            setupPendingStack(key);
            
            if (lastMatchStart > 0)
            {
               // If we already found one or more non-matching characters
               // before we detected this full match, we have to return the
               // non-matching character(s) as the next token. However, we
               // set the full match into our pendingMatch instance variable,
               // so that we use it immediately on our next call into this
               // method.
               pendingMatch = key;
               index += lastMatchStart;
               buf.setLength(lastMatchStart);
               
               return buf.toString();
            }
            else
            {
               // Our matching key was not preceded by any non-matching
               // characters, so we can return the replacement text
               // immediately.
               index += key.length();
               
               return getPendingReplacement();
            }
         }
         
         // Add the character to the current token buffer.
         buf.append(ch);
      }
      
      // Update the index to reflect all "lookahead" characters processed
      // with no match.
      index += i;
      
      // Get the string of unmatched characters.
      String token = buf.toString();
      
      // Return the buffer or null if no new characters are available.
      return (token.length() > 0 ? token : null);
   }
   
   /**
    * Reset the list of potential matches so that each candidate has an
    * equal starting state of NO_MATCH. As soon any character in a
    * candidate is found to match, its corresponding entry is set to
    * PARTIAL_MATCH. If all characters match, it is set to FULL_MATCH.
    */
   private void resetMatchStatus()
   {
      Arrays.fill(matches, NO_MATCH);
   }
   
   /**
    * Set up the stack of pending replacement words. Each word will be
    * returned as a separate token (i.e., syllable) from {@link #nextToken}
    * until the stack is empty.
    *
    * @param   key
    *          A substring which was matched in the original symbol; the
    *          key to a replacement string in the map of replacements.
    */
   private void setupPendingStack(String key)
   {
      String replacement = replacements.get(key);
      String[] words = replacement.split(" ");
      for (int i = words.length - 1; i >= 0; i--)
      {
         pendingStack.push(words[i]);
      }
   }
   
   /**
    * Get the next pending replacement string, if any, based on a previous
    * substring match.
    *
    * @return  Next pending replacement syllable, or <code>null</code> if
    *          the stack of pending replacements is empty.
    */
   private String getPendingReplacement()
   {
      String replacement = null;
      if (!pendingStack.isEmpty())
      {
         try
         {
            replacement = pendingStack.pop();
         }
         catch (EmptyStackException exc)
         {
            LOG.log(Level.WARNING,"", exc);
         }
      }
      
      return replacement;
   }
   
   /**
    * Perform preparation of the data structures used to match candidate
    * substrings in the {@link #nextToken} method. This includes setting
    * each candidate key string into an array, and creating a new array
    * of integers to track the match status of each candidate key as the
    * <code>nextToken</code> method scans characters from the original
    * symbol.
    */
   private void prepareCandidates()
   {
      if (candidates != null)
      {
         return;
      }
      
      Set<String> keys = replacements.keySet();
      int size = keys.size();
      candidates = new String[size];
      matches = new int[size];
      
      Iterator<String> iter = keys.iterator();
      for (int i = 0; iter.hasNext(); i++)
      {
         String next = iter.next();
         candidates[i] = next;
      }
   }
   
   /**
    * Test the given name against the list of Java keywords and resolve any
    * conflict by adding text to the name to make it different (and thus
    * javac will compile it).  An underscore character will be prepended
    * to the name to resolve any conflicts.
    *
    * @param    possible
    *           The name to check for conflicts.
    * @param    type
    *           The type of name.  This is used to ensure that the name is
    *           a Java name (and thus requires conflict resolution).
    *
    * @return   The original name if no conflicts were detected or a
    *           disambiguated (safe) name if a conflict existed.
    */
   public static String resolvePossibleKeywordConflict(String possible, int type)
   {
      String lcPossible = possible.toLowerCase();
      boolean hit = false;
      
      switch (type)
      {
         case TYPE_VARIABLE:
         case TYPE_CONSTANT:
         case TYPE_PROPERTY:
         {
            // only emitted class fields are checked with reserved FWD names.  the fall-through to TYPE_METHOD
            // is required to check the reserved Java names, too.
            hit = reservedFWD.contains(possible);
         }
         case TYPE_METHOD:
         {
            hit = hit || reservedJava.contains(possible);
            break;
         }
         
         case TYPE_CLASS:
            hit = reservedJava.contains(lcPossible) ||
                  reservedFWD.contains(possible)    ||
                  reservedSQL.contains(lcPossible);
            if (!hit)
            {
               // exquisite search: avoid collisions with java.lang. Unfortunately,
               //          Reflections("java.lang").getSubTypesOf(Object.class);
               // cannot be used because of security reasons (?), so we built it incrementally
               hit = defaultImportedClasses.contains(possible);
               if (!hit)
               {
                  try
                  {
                     Class.forName("java.lang." + possible, false, String.class.getClassLoader());
                     hit = true;
                     defaultImportedClasses.add(possible);
                  }
                  catch (ClassNotFoundException e)
                  {
                     // this is expected: no collisions
                  }
               }
            }
            break;
         
         case TYPE_DMO_ALIAS:
            hit = FQLLexer.getReserved().contains(lcPossible);
         case TYPE_TABLE:
         {
            hit = hit                             ||
                  reservedJava.contains(possible) || 
                  reservedFWD.contains(possible)  || 
                  reservedSQL.contains(lcPossible);
            break;
         }
         
         case TYPE_DATABASE:
         case TYPE_COLUMN:
         case TYPE_INDEX:
         {
            // testing in lowercase because SQL is case-insensitive 
            hit = reservedSQL.contains(lcPossible);
            break;
         }
         
         default:
            break;
      }
      
      // methods used in static imports can't be used in definitions...
      if (hit || (type == TYPE_METHOD && (staticImportedMethods.contains(possible) || 
                                          objectMethods.contains(possible))))
      {
         possible += "_";
      }
      
      return possible;
   }
   
   /**
    * Replace the leading character with safe text if it is a digit.
    *
    * @param    text
    *           The syllable to process.
    *
    * @return   The modified text if the first character was a numeric digit.
    */
   private String processLeadingDigit(String text)
   {
      char c = text.charAt(0);
      
      return Character.isDigit(c) ? digits.get(c) + text.substring(1) : text;
   }
   
   /**
    * Entry point and test harness for this class. Expects the following
    * arguments:
    * <ol>
    * <li><code>symbol</code> (required) - Progress symbol name to be
    *     converted;
    * <li><code>type</code> (required) - type of conversion to apply. Use
    *     the integer equivalents of the <code>TYPE_*</code> constants
    *     defined by this class;
    * <li><code>replace</code> (required) - true/false. True to replace
    *     substrings as the name is tokenized; false to tokenize syllables
    *     without replacement;
    * <li><code>key</code> (optional) - substring in <code>symbol</code>
    *     to be replaced by the following parameter;
    * <li><code>replacement</code> (required) - replacement text for the
    *     immediately previous parameter.
    * </ol>
    * Any number of key/replacement pairs may be provided. If a key is
    * provided with no following replacement, that key is ignored.
    *
    * @param   args
    *          Command line parameters as described above.
    */
   public static void main(String[] args)
   {
      try
      {
         if (args.length < 3)
         {
            System.out.println("Usage: NameConverter \"symbol\" type replace phrases " +
                               "[\"key\", \"replacement\"]...");
            System.exit(-1);
         }
         
         String name = args[0];
         int type = Integer.parseInt(args[1]);
         boolean replace = Boolean.parseBoolean(args[2]);
         int phrases = ALL;
         
         if (args.length > 3)
         {
            phrases = Integer.parseInt(args[3]);
         }
         NameConverter conv = new NameConverter(phrases, replace);
         conv.setSqlOverride(SQL_MODE_MINIMAL);
         
         int i;
         for (i = 4; i + 1 < args.length; i += 2)
         {
            String key = args[i];
            String rpl = args[i + 1];
            conv.putReplacement(key, rpl);
         }
         
         if (i + 1 == args.length)
         {
            System.out.println("Extra argument '" + args[i] + "' ignored");
         }
         
         System.out.println(name + " ---> " + conv.convert(name, type) +
                            " (" + conv.describeConversion() + ")");
         /*
         long time = System.currentTimeMillis();
         for (i = 0; i < 10000; i++)
         {
            conv.convert(name, type);
         }
         time = System.currentTimeMillis() - time;
         System.out.println("Elapsed:  " + time);
         */
      }
      catch (Exception exc)
      {
         LOG.log(Level.WARNING,"", exc);
      }
   }
}