# PreselectQuery.java

    /*
    ** Module   : PreselectQuery.java
    ** Abstract : A query which enables stateful, cursored navigation of a server side result set
    **
    ** Copyright (c) 2004-2025, Golden Code Development Corporation.
    **
    ** -#- -I- --Date-- --JPRM-- -----------------------------------Description-----------------------------------
    ** 001 ECF 20051115   @23396 Created initial version, based on prototype
    **                           CursoredQuery. Backing support for queries
    **                           which must support the Progress, row-level
    **                           access semantic against a preselected result
    **                           set.
    ** 002 ECF 20051129   @23575 Added lenient-off-end mode. When in this
    **                           mode, QueryOffEndExceptions thrown by the
    **                           underlying buffer during next and previous
    **                           operations are suppressed.
    ** 003 ECF 20051216   @23750 Added scrolling support. Modified retrieval
    **                           logic to allow for repositioning. Added many
    **                           new methods.
    ** 004 ECF 20060105   @23831 Inserted abstraction layer between query and
    **                           its scrollable results. Added the Results
    **                           interface and its default implementation,
    **                           SimpleResults. This allows subclasses to
    **                           provide custom implementations by overriding
    **                           the createResults method.
    ** 005 ECF 20060202   @24358 Added temp table support. Use buffer's
    **                           multiplex ID as a query substitution
    **                           parameter when executing HQL. Parameters are
    **                           now managed separately, by component, to
    **                           enable this processing.
    ** 006 ECF 20060208   @24721 Added support for client-side where clause
    **                           processing. Results and SimpleResults are now
    **                           top level classes. Modified all single-table
    **                           constructor variants to accept an instance of
    **                           WhereExpression. Modified Component to handle
    **                           argument resolution more correctly. Results
    **                           can now be provided by an external entity
    **                           (CompoundQuery typically).
    ** 007 ECF 20060302   @25019 Add support for foreign key joins. Added new
    **                           constructor and addComponent method variants
    **                           which mirror existing versions, but accept an
    **                           additional, inverse DMO parameter. Updated
    **                           Component class, HQL generation, and query
    **                           execution to integrate natural joins.
    ** 008 ECF 20060422   @25657 Added setIterationType method. Sets retrieval
    **                           mode (first, last, next, etc.) for the most
    **                           recently added query component.
    ** 009 ECF 20060423   @25661 Allow multiple client-side where clause
    **                           expressions to be registered. This is needed
    **                           because each table in a multi-table query can
    **                           contain its own client-side where component.
    ** 010 ECF 20060424   @25670 Fix to several constructor variants. Swapped
    **                           positions of sort and inverse parameters to
    **                           match the natural order in which these
    **                           parameters are emitted in converted code.
    ** 011 ECF 20060424   @25695 Added new constructor variants. Each existing
    **                           variant now has a counterpart which does not
    **                           expect a WhereExpression parameter.
    ** 012 ECF 20060424   @25699 Roll back previous change. Fixed ordering of
    **                           some parameters in internal constructor
    **                           calls.
    ** 013 ECF 20060425   @25718 Added setLenientOffEnd method. Required for
    **                           conversion of certain OPEN QUERY constructs.
    ** 014 ECF 20060501   @25915 Added isPreselect method. Indicates query's
    **                           preselection behavior.
    ** 015 ECF 20060531   @26850 Replaced EndConditionException with
    **                           QueryOffEndException. The latter is a more
    **                           specific version of the former, used to
    **                           indicate a query has run off the end of its
    **                           result set.
    ** 016 ECF 20060602   @26944 Optimization. HQL queries now will always
    **                           attempt to load the full record, rather than
    **                           just the primary key ID. This avoids the
    **                           overhead of a separate load step.
    ** 017 ECF 20060616   @27412 Fix to regression caused by #016 (@26944).
    **                           FilteredResults needs to store DMO instance
    **                           within an array.
    ** 018 ECF 20060705   @27828 Fixed ClassCastException. Made fetch method
    **                           tolerant of a primary key ID result (instead
    **                           of DMO) in a result set.
    ** 019 ECF 20060713   @28051 Integrated HQLPreprocessor. An instance of
    **                           this class is created for each component
    **                           added to the query. This affects how the
    **                           overall HQL query is composed.
    ** 020 ECF 20060719   @28116 Rolled back #016 (@26944). With the addition
    **                           of the change broker architecture, it is
    **                           actually possible to avoid loading the record
    **                           at all in limited cases. In these cases, only
    **                           a Hibernate proxy is created, which is far
    **                           more efficient than retrieving the entire
    **                           record up front. Additionally, #016 was
    **                           problematic for locking, because the record
    **                           could be changed by another user after it was
    **                           fetched, but before the lock was acquired.
    ** 021 ECF 20060722   @28135 Changes to make inheritance of this class
    **                           more flexible. In particular, this set of
    **                           changes accommodates the AdaptiveQuery
    **                           subclass.
    ** 022 ECF 20060725   @28188 Fixed regressions and implemented
    **                           Finalizable. The latter is necessary to
    **                           support retry, and to prevent a session from
    **                           closing prematurely and discarding the
    **                           query's results.
    ** 023 ECF 20060730   @28270 Client-side where clause processing changes.
    **                           Moved FilteredResults into a top-level class.
    ** 024 ECF 20060801   @28352 Refactored execute() method and order by
    **                           clause preparation code. Query substitution
    **                           parameter preparation was moved into its own
    **                           polymorphic method. Integrated SortCriterion
    **                           class to replace code that was duplicative
    **                           with that class.
    ** 025 ECF 20060803   @28396 Further refactoring. Added getRecordBuffers()
    **                           method. Made results private again.
    ** 026 ECF 20060807   @28462 Fixed assembleOrderByClauseNoPreamble(). DMO
    **                           alias needed to be trimmed of leading space.
    ** 027 ECF 20060829   @28954 Fixed join defect in constructor. We were
    **                           creating a join and passing it to the wrong
    **                           version of addComponent().
    ** 028 ECF 20060830   @28991 Additional join fix. The fix has been moved
    **                           from a constructor to the most specific
    **                           variant of addComponent().
    ** 029 ECF 20060831   @29087 Added support for sorting by an indexed
    **                           property. Specifically, support sorting by
    **                           a property of a DMO's associated list of
    **                           composite properties.
    ** 030 ECF 20060901   @29139 Fixed NPE. Methods which accept a list of
    **                           sort criteria must handle null list case.
    ** 031 ECF 20060911   @29461 Added debug code. Report DMOs found by the
    **                           query if trace logging is enabled.
    ** 032 ECF 20060925   @29963 Check for a pending error before retrieving a
    **                           record. Retrieval is not attempted if an
    **                           error is pending in the ErrorManager.
    ** 033 ECF 20061003   @30154 Backed out #032 (@29963). This logic is only
    **                           necessary for FindQuery and AdaptiveFind.
    ** 034 ECF 20061010   @30310 Minor change to initialization logic. Joins
    **                           across different Persistence objects, rather
    **                           than across databases, are now disallowed.
    ** 035 ECF 20061019   @30528 Fixed reposition support to better handle
    **                           varying subclass behavior. This class now
    **                           implements the SessionListener interface.
    **                           Upon receiving a sessionClosing notification,
    **                           a PreselectQuery will cache all of its
    **                           results in a SimpleResults object and replace
    **                           its existing results with that cache.
    ** 036 ECF 20061023   @30607 Moved errorIfNull logic to AbstractQuery base
    **                           class. Method setErrorIfNull() is now
    **                           required by the P2JQuery interface and is
    **                           implemented in AbstractQuery instead of here.
    ** 037 ECF 20061027   @30752 Replaced Object with DataModelObject for DMO
    **                           arguments to methods/constructors. Required
    **                           for compile-time type safety.
    ** 038 ECF 20061102   @30897 Fixed issue querying row count. The cached
    **                           row count must be reset to its default value
    **                           when a new result set is retrieved. This
    **                           forces the size() method to recalculate the
    **                           proper row count.
    ** 039 ECF 20061102   @30959 Added support for reposition notification and
    **                           auto-fetch features. Cleaned up reposition
    **                           logic so that fetches after a reposition are
    **                           (hopefully) correct now.
    ** 040 ECF 20061115   @31209 Changed call to set current buffer record.
    **                           RecordBuffer API has changed to permit proper
    **                           management of placeholder snapshot record.
    **                           Added isResetSnapshot() method to allow
    **                           subclasses to override this behavior.
    ** 041 ECF 20061120   @31326 Modified Component inner class. Expose
    **                           restriction properties determined during HQL
    **                           where clause preprocessing.
    ** 042 ECF 20061204   @31576 Integrated RepositionCache. This replaced a
    **                           simple hash map approach which failed in
    **                           cases where a lookup used an ID array which
    **                           had fewer IDs than joined tables in a query.
    **                           Also emulate Progress reposition quirk which
    **                           repositions to last record in result set when
    **                           no match is found for the target recid.
    ** 043 ECF 20061206   @31587 Integrated changes to AbstractQuery's
    **                           afterReposition() signature. Failed attempts
    **                           to reposition by ID cause afterReposition()
    **                           to be invoked in notify-only mode.
    ** 044 ECF 20070225   @32227 Plug results resource leak. Clean up results
    **                           resources when enclosing block ends. This was
    **                           causing both prepared statements and result
    **                           sets to remain unclosed in the backing JDBC
    **                           connection.
    ** 045 ECF 20070225   @32232 Fixed regression caused by #044 (@32227).
    **                           Finalizable which cleans up results resources
    **                           must not be hard-coded to nearest enclosing
    **                           scope where results are created. Subclasses
    **                           may need to register this object to the
    **                           global scope instead.
    ** 046 ECF 20070228   @32247 Optimized no-lock and temp table fetches. In
    **                           these cases, it is not necessary to fetch the
    **                           record in two steps. The no-lock case makes
    **                           no guarantee that the record in memory has
    **                           not been updated by other sessions. The temp
    **                           table case guarantees that no other session
    **                           can make an update to the record in memory,
    **                           because a temp table record is inherently
    **                           scoped to a session.
    ** 047 ECF 20070308   @32334 Fixed behavior for first/last. These should
    **                           never raise an error condition, only a query
    **                           off end condition.
    ** 048 ECF 20070328   @32648 Changed to support HQLPreprocessor caching.
    **                           Use factory method to retrieve an instance of
    **                           the preprocessor rather than invoking the
    **                           constructor directly.
    ** 049 ECF 20070412   @32984 Accommodate method name change in
    **                           RecordBuffer. setCurrentRecord --> setRecord.
    ** 050 ECF 20070415   @33002 Fixed locking problem. DMOs fetched "whole"
    **                           with lock type NONE must be re-loaded with a
    **                           lock if called for by the query component or
    **                           lock type override in the retrieval method.
    **                           In certain circumstances, these were being
    **                           left without a lock.
    ** 051 ECF 20070416   @33026 Integrated user interrupt handling.
    ** 052 ECF 20070502   @33371 Changed to support HQLPreprocessor changes.
    **                           The signature of that class' primary factory
    **                           has changed.
    ** 053 ECF 20070504   @33409 Fixed load() implementation. We now retry
    **                           reposition if it fails, assuming current
    **                           result set did not contain target records.
    ** 054 ECF 20070508   @33459 Fixed sessionClosing() and session listener
    **                           registration. Was not properly scoping
    **                           registration for scrolling queries.
    ** 055 ECF 20070511   @33470 Fixed resetResults(). Additional state needed
    **                           to be reset to default values.
    ** 056 ECF 20070517   @33670 Fixed abend in sessionClosing(). Check that
    **                           all necessary buffers are still in scope
    **                           before attempting to cache preselected
    **                           results in a SimpleResults object. This
    **                           prevents a fatal error when trying to
    **                           retrieve results for a temp table which has
    **                           been dropped by the time we get the session
    **                           closing notification.
    ** 057 ECF 20070518   @33683 Changed access for getRecordBuffers(). Made
    **                           public (was protected), since this is now a
    **                           method of the Joinable interface.
    ** 058 ECF 20070706   @34402 Fixed selection of scope for SessionListener
    **                           registration. Implemented new SessionListener
    **                           method deregisteredSessionListener(). Also
    **                           replaced StringBuffer with StringBuilder.
    ** 059 ECF 20070709   @34422 Fixed sessionClosing(). Was unsafe for
    **                           certain non-scrolling results which do not
    **                           support reset.
    ** 060 ECF 20070711   @34448 Extracted ScrollingResults inner class into a
    **                           separate, top-level class.
    ** 061 ECF 20070716   @34510 Implement new cleanup() method as required by
    **                           P2JQuery interface. Check whether query is
    **                           standalone before registering a cleaner with
    **                           the TM. This prevents us from cleaning up too
    **                           aggressively in case we are wrapped by a
    **                           QueryWrapper or contained in a CompoundQuery
    **                           that is so wrapped.
    ** 062 ECF 20070720   @34611 Fixed fetch(). Always reload a DMO, even if
    **                           it is in a temp table or has no lock. This is
    **                           necessary to handle the case of fetching a
    **                           cached DMO which may have been detached from
    **                           its session previously. For a persistent DMO,
    **                           the additional overhead should be small,
    **                           since Hibernate only needs to go to the first
    **                           level cache to find the object.
    ** 063 ECF 20070720   @34621 Implemented a better fix for fetch() than
    **                           #062 (@34611). If DMO is a temp table, simply
    **                           try to reassociate it with the Hibernate
    **                           session, rather than going all the way back
    **                           to the database (if DMO was detached). Can't
    **                           do this for permanent tables, though.
    ** 064 ECF 20070821   @34907 Added support for temporary silent error
    **                           mode. This allows query substitution parms to
    **                           be processed safely during initialization of
    **                           a query.
    ** 065 ECF 20070827   @34996 Optimized certain temp table queries. Only
    **                           augment where clause with a temp buffer's
    **                           multiplex ID when necessary. Use multiplex ID
    **                           as a constant in where clause rather than as
    **                           a substitution parameter.
    ** 066 ECF 20070927   @35276 Fixed reposition logic. Any reposition must
    **                           first release all buffers. Changes to methods
    **                           to get current row and report whether query
    **                           is off-end.
    ** 067 ECF 20071030   @35634 Fixed fetch() to prevent loading stale temp
    **                           table records. It is possible for a record to
    **                           be evicted from the session cache while still
    **                           held in a result set, such that the result
    **                           set will return a stale record that is not a
    **                           detached instance, even if the session is
    **                           flushed before the fetch. We now force a
    **                           reload of fetched temp table records to avoid
    **                           this problem.
    ** 068 ECF 20071130   @36122 Removed isRefreshSnapshot() method. The
    **                           RecordBuffer placeholder snapshot is no
    **                           longer cleared by any query type when the
    **                           query does not find a record to place in the
    **                           buffer. Integrated generics.
    ** 069 ECF 20071206   @36267 Fixed off-end tracking. Replaced boolean flag
    **                           with OffEnd enum to discern between off-end
    **                           directions (i.e., off back vs off front of
    **                           result set). Current off-end status and
    **                           active sort index are reported to the record
    **                           buffer when fetching a record.
    ** 070 ECF 20071218   @36463 Fixed order by clause generation. Was failing
    **                           for sort criteria which included references
    **                           to specific extents of indexed properties.
    ** 071 ECF 20080120   @37009 Removed restriction of dealing only with IDs.
    **                           This query can now handle both Persistables
    **                           and primary key IDs when retrieving data.
    **                           This is necessary when the full record is
    **                           needed immediately, not just an ID, such as
    **                           in presort cases. Also added open() method to
    **                           set proper semantics for a converted OPEN
    **                           QUERY statement, and optionally to reposition
    **                           to the first result.
    ** 072 ECF 20080222   @37144 Fixed NPE in sessionClosing().
    ** 073 ECF 20080310   @37481 Made changes necessary to support embedded,
    **                           H2 database.
    ** 074 SVL 20080418   @38036 Added support for the inverseSorting flag.
    ** 075 SVL 20080421   @38080 Added support for legacy key join helpers.
    ** 076 SVL 20080425   @38119 lastRepo assignment has been fixed into
    **                           reposition(int).
    ** 077 ECF 20080502   @23396 Backed out #076 (@38119). The reposition(int)
    **                           implementation was generally correct and this
    **                           change caused regressions. Fixed open() to
    **                           execute the query immediately instead of
    **                           doing this lazily. We now reposition on open
    **                           only if the query is being browsed. Removed
    **                           obsolete method isRepositionOnOpen().
    ** 078 SVL 20080508   @38238 Fixed incorrect record values after
    **                           first-next, forward/backward(0) and
    **                           last-prev execution sequences.
    ** 079 ECF 20080508   @38243 Fixed regression in reposition(int) which did
    **                           not handle all non-positive rows correctly.
    **                           Refactored open() method to use parent class.
    **                           Override base lock type for temporary tables.
    **                           This allows some downstream optimizations.
    ** 080 ECF 20080509   @38252 Fixed regression in open(). Removed check for
    **                           standalone flag.
    ** 081 SVL 20080526   @38311 Preselect locking implemented.
    ** 082 SVL 20080604   @38534 Modified according to the new P2JQuery
    **                           interface. previous() fixed.
    ** 083 ECF 20080604   @38608 Added placeholder for dirty database check.
    ** 084 SVL 20080611   @38685 Behavior of previous() fixed for the case
    **                           when the off-end is FRONT.
    ** 085 CA  20080618   @38869 When the JOINed foreign buffer does not refer
    **                           any record, the query must return no results
    **                           as it can not be executed.
    ** 086 CA  20080619   @38871 If executed within an OPEN QUERY statement,
    **                           the query must be removed from the session's
    **                           listeners only when the enclosed external
    **                           scope is exit. This fixes the case when the
    **                           OPEN QUERY is executed inside a procedure.
    ** 087 ECF 20080709   @39086 Created cacheResults() method. Abstracted
    **                           from sessionClosing() and made protected so
    **                           subclasses could access this functionality.
    ** 088 ECF 20080710   @39090 Modified sessionClosing() and cacheResults().
    **                           The latter no longer throws
    **                           IllegalStateException.
    ** 089 ECF 20080714   @39127 Fixed NPE in debug logging.
    ** 090 ECF 20080723   @39173 Removed retry handling. This is now the
    **                           responsibility of subclasses.
    ** 091 CA  20080729   @39218 Added back the retry handling. Now the retry
    **                           will rely on the reposition mechanism to
    **                           retrieve the same record, but only when the
    **                           query is associated with a FOR block.
    ** 092 SVL 20080730   @39237 Component.multiplexID field was removed.
    **                           RecordBuffer's multiplex ID is used instead.
    ** 093 ECF 20080730   @39325 Added rowid support. Refactored reposition*()
    **                           methods.
    ** 094 SVL 20080808   @39348 Fixed cacheResults(). Did not handle case
    **                           when query has just executed but has not yet
    **                           retrieved any results.
    ** 095 ECF 20080822   @39556 Fix for join validation. Abstracted check
    **                           into separate method, verifyJoins(). Disable
    **                           join buffer check for server-based joins.
    **                           Other changes in support of global DMO change
    **                           events.
    ** 096 CA  20080819   @39463 Support API change in ScrollingResults.
    ** 097 CA  20080828   @39610 Fix NPE caused by repositionByID(rowid, ...).
    ** 098 ECF 20081009   @40081 Added support for inlining query substitution
    **                           parameters. Integrated HQLPreprocessor API
    **                           changes and the ParameterIndices class.
    ** 099 ECF 20081030   @40308 Fixed regression introduced by #098 (@40081).
    **                           Refactored Component inner class and code in
    **                           top-level class which uses it.
    ** 100 ECF 20081105   @40311 Implemented registerRecordChangeListeners().
    **                           Required by the Joinable interface.
    ** 101 ECF 20081106   @40324 Added _isOffEnd(). Required by P2JQuery
    **                           interface change.
    ** 102 ECF 20081107   @40405 Moved multiplex ID position in order by
    **                           clause. It used to be the first element, but
    **                           that sometimes resulted in a less efficient
    **                           index being selected for a query plan. Now it
    **                           is either the last element, or second to last
    **                           (if the primary key is part of the clause).
    ** 103 CA  20081210   @40839 Do not cleanup if the query was explicitly
    **                           closed.
    ** 104 CA  20081211   @40859 Fixed regression introduced by H103: on
    **                           sessionClosing event, if the query is closed,
    **                           return true.
    ** 105 ECF 20081215   @40914 Removed flushBuffers() method. This work is
    **                           now done when a component is added to the
    **                           query. The previous invocation, during record
    **                           retrieval, was too late. A validation error
    **                           ignores silent error mode, since the flushing
    **                           of the previous record in the buffer is not
    **                           masked by silent error mode.
    ** 106 ECF 20090202   @41248 Rolled back #081 (@38311). Prelock mechanism
    **                           has been disabled.
    ** 107 ECF 20090313   @41600 Changed createResults(). Added parameter to
    **                           FilteredResults constructor.
    ** 108 SVL 20090316   @41610 assembleOrderByClauseNoPreamble() refactored,
    **                           sort variable is protected instead of private
    **                           now.
    ** 109 GES 20090424   @41941 Import change.
    ** 110 ECF 20090512   @42153 Modified addComponent(). Now invokes flushAll()
    **                           instead of flush() on backing buffer. This
    **                           ensures all like buffers are allowed to flush
    **                           their transient records properly.
    ** 111 ECF 20090603   @42599 Fixed fetch(). A dirty record in the backing
    **                           buffer must not be replaced by the primary
    **                           database's version of that record.
    ** 112 ECF 20090604   @42606 Added support for Progress isolation leak quirk.
    **                           Modified execute() method to publish uncommitted
    **                           changes with dirty share manager if necessary.
    ** 113 ECF 20090609   @42631 Fixed execute(). We now evict, if possible, those
    **                           DMOs read into the Hibernate Session, whose
    **                           uncommitted changes are leaked to other sessions
    **                           early.
    ** 114 ECF 20090619   @42772 Added executeScroll(). Executes default variant
    **                           of Persistence.scroll(String, Object[], Type[]).
    **                           Children classes which need to invoke different
    **                           variants must override this method.
    ** 115 ECF 20090630   @42985 Rewrote reposition logic. Moved from integer to
    **                           recid type in reposition methods.
    ** 116 ECF 20090702   @43035 Fixed sessionClosing() and regressions in query
    **                           (in the Progress OPEN QUERY sense) behavior.
    **                           Added isNativeScrolling() to indicate that a base
    **                           PreselectQuery instance used for a converted 4GL
    **                           query behaves as if that query were defined as
    **                           scrolling. Added error handling in caching
    **                           results during a sessionClosing event.
    ** 117 ECF 20090703   @43051 Reimplemented load() method. This method now sets
    **                           the backing buffer to unknown mode and throws
    **                           MissingRecordException if a target record cannot
    **                           be loaded. Also fixed cleanup(); the query is now
    **                           flagged as closed once it runs.
    ** 118 ECF 20090707   @43101 Fixed defect related to processing reposition
    **                           notifications. Make sure we don't release the
    **                           buffers' current records and change other state
    **                           when processing nested reposition calls during
    **                           reposition notifications.
    ** 119 ECF 20090708   @43107 Partial rollback of #118 (@43101). Some state
    **                           must be changed when processing nested reposition
    **                           calls during reposition notifications.
    ** 120 ECF 20090711   @43192 Use new ScrollingResults c'tor.
    ** 121 ECF 20090716   @43220 Refactored for lazy record buffer initialization.
    **                           Initialize backing buffers when their scopes have
    **                           been opened. Otherwise allow initialization to be
    **                           deferred.
    ** 122 SVL 20090723   @43344 Added setNonScrolling() function.
    ** 123 ECF 20090724   @43419 Made getPersistence() protected.
    ** 124 ECF 20090729   @43448 Fixed executeQuery(). Surrounded creation of
    **                           ScrollingResults with push/pop calls for implicit
    **                           transaction.
    ** 125 CA  20090731   @43468 Register the query for off-end listener
    **                           registration after block initialization (done
    **                           only for block types FOR and FOR EACH). Added
    **                           getOffEndListeners, which returns a list of all
    **                           off-end listeners for all used buffers.
    ** 126 ECF 20090810   @43588 Changed to accommodate SessionListener API
    **                           change. sessionClosing() method was replaced with
    **                           sessionEvent().
    ** 127 ECF 20090811   @43600 Fixed resource cleanup. Refactored registration
    **                           of a cleaner object with the TM, so that
    **                           subclasses could use it. Ensured cleaner is only
    **                           registered once.
    ** 128 SVL 20091030   @44296 Fixed forward(), backward(), currentRow() and next() functions.
    ** 129 OM  20130829          Added change detection to buffer on current() methods.
    **                           Updated calls to match the new Persistence.load() signatures.
    ** 130 ECF 20131028          Import change.
    ** 131 CA  20131013          Added no-op deleted() method, required by the changes in Finalizable
    **                           interface.
    ** 132 SVL 20140210          Use HQLExpression instead of HQL string.
    ** 133 SVL 20140604          Added support for INDEX-INFORMATION.
    ** 134 ECF 20140814          Replaced RecordBuffer.flushAll with flush in addComponent method.
    ** 135 SVL 20140909          Fixed null case for peekNext/peekPrevious. Changed some access
    **                           modifiers. Added deleteResultListEntry.
    ** 136 SVL 20141209          Update row count after deleteResultListEntry.
    ** 137 SVL 20150212          Fixed last().
    ** 138 SVL 20150331          Added deleteResultListEntry(boolean).
    ** 139 OM  20150421          Changed the hqlPreprocessor of each components to expires on when
    **                           resetArgs() is called based on nullArguments fingerprint.
    **                           Replaced Apache logging with J2SE logging.
    ** 140 ECF 20150801          Implemented new Joinable methods necessary for compound query
    **                           optimization. Refactored to remove inner class Component, which is
    **                           now the top level class QueryComponent.
    ** 141 SVL 20150821          isScrolling was made public.
    ** 142 ECF 20150906          Reimplemented silent error mode processing of query substitution
    **                           parameters. Fixed NPE related to outer joins.
    ** 143 ECF 20151214          Prevent releasing the current buffer record(s) under certain
    **                           circumstances, when no record is found by the query.
    ** 144 EVL 20160217          Clean up comments from symbols invalid for Solaris to compile.
    ** 145 SVL 20160314          Added support for parameterFilter.
    ** 146 ECF 20160320          Force full records to be queried on the first pass.
    ** 147 ECF 20160512          Deregister as SessionListener during cleanup instead of letting TM
    **                           call cleaner. Fixed coreFetch issue with null record.
    ** 148 ECF 20160610          Reverted #146.
    ** 149 SVL 20160608          The query can handle null IDs (for scrolling compound queries
    **                           with OUTER join). Reposition validation moved away.
    ** 150 ECF 20160614          Added getEntities to collecte entity names associated with this
    **                           query; modified executeScroll to pass these to Persistence.scroll.
    ** 151 SVL 20160721          Proper reset on query reopen.
    ** 152 OM  20160720          Added support for loading template records.
    ** 153 ECF 20160725          Fixed buffer lookup during HQL preprocessing.
    ** 154 ECF 20160823          Improved coreFetch to better detect missing records and avoid
    **                           deadlock.
    ** 155 ECF 20160901          DirtyShareContext.isDirtyDelete signature change.
    ** 156 GES 20160804          Switched from WhereExpression to lambda usage.
    **         20160919          All initialization is now done with a default constructor followed
    **                           by a call to initialize(). This enables query instance members in
    **                           converted code to be local variables that are effectively final.
    **                           This is needed to support lambdas without moving queries to be
    **                           instance members. Queries as instance members breaks recursion
    **                           use cases.
    ** 157 SVL 20161007          Fix for the case when REPOSITION-TO-ROWID takes more ids than the
    **                           actual query table count.
    ** 158 ECF 20161014          Fixed implementation of current() method in the event there is no
    **                           current record in the buffer to reload, and this is not considered
    **                           an error in the context of the query (e.g., query represents an
    **                           outer join and record can be null).
    ** 159 OM  20170118          Fixed javadoc (Solaris compatibility).
    **     OM  20170209          Added return value for repositioning methods.
    ** 160 GES 20171220          Changes to match new silent mode implementation.
    ** 161 OM  20171129          releaseBuffers() is implemented from P2JQuery.
    **     ECF 20171202          Allow HQL to omit DMO alias qualifiers.
    ** 162 HC  20180130          Fixed an NPE condition in an outer join query where one of the query
    **                           components yields no records.
    ** 163 OM  20180515          Added dynamic filters and dynamic sort. Fixed extra joins generation
    **                           in assembleFromClause.
    ** 164 ECF 20180818          Temp-table fetch optimization.
    **     OM  20180901          Improved dynamic filtering.
    **     OM  20180924          Filtering on decimal fields uses STRING(f, format) function.
    ** 165 OM  20181023          Added external buffers when making the adaptive server join
    **                           components. Avoided NPE in repositionByID().
    ** 166 ECF 20190106          Fixed size method to not report a size of 1 for an empty result set.
    ** 167 ECF 20190318          Refetch cached full records in coreFetch, most of which are likely
    **                           detached from the Hibernate session.
    ** 168 SBI 20190401          Changed coreFetch in order to process the end of query for all query
    **                           components.
    **     ECF 20190415          Fixed quick-out test condition in cacheResults.
    **     ECF 20190427          Fixed results caching regression.
    ** 169 ECF 20190813          Change to RecordBuffer.release and RecordBuffer.setRecord APIs.
    ** 170 OM  20190720          Added repositionByID() for internal usage.
    **     CA  20190811          A dynamic query can be closed and then re-opened.
    **     HC  20190823          Fixed an NPE condition.
    ** 171 SVL 20191002          Implemented createResultListEntry.
    ** 172 CA  20191009          Added where and sort clause translation in case of bound buffers.
    ** 173 SVL 20191218          Fixed caching logic driven by REPOSITION TO ROWID.
    ** 174 CA  20200110          Fixed finalizable leak when then query is used in a QueryWrapper.
    ** 175 ECF 20200419          Removed Hibernate dependencies.
    ** 176 AIL 20200908          Reset offSet flag when adding new row through createResultListEntry.
    **     AIL 20200910          Restrict resetting offSet flag only for browsed queries.
    **     ECF 20200919          Persistence.load API signature change.
    **     OM  20201001          Dropped redundant ORDER BY elements in multi-table queries.
    **     OM  20201007          Optimized SortCriterion by using DmoMeta data instead of map lookups.
    **                           _multiplex sort component matches the index direction.
    **     OM  20201120          Adjusted error messages.
    **     CA  20210310          A dynamic predicate must set the default lock to NONE, instead of SHARE.
    **     IAS 29210401          Update _UserTableStat "read" counter
    **     IAS 20210419          UserTableStat counters' update
    **     ECF 20210502          Adjusted initial components array list size.
    **     CA  20210527          Removed isOffEnd implementation, as the AbstractQuery.isOffEnd is the correct
    **                           one.
    **     ECF 20210723          Minor cleanup.
    **     OM  20220112          Added implementation of SKIP-DELETED-RECORD attribute.
    **     CA  20220117          Fixed close for dynamic PreselectQuery's, the close(boolean) needs to be
    **                           implemented.
    **     OM  20220118          The assembleOrderByClause() should not return null value.
    **     SVL 20220127          Fixed the case when deleted records were fetched if the cursor contained full
    **                           records.
    **     CA  20220203          Fixed 'getRow' for SCROLLING mode, in this case check the cursor's row number.
    **     OM  20220225          coreFetch() returns true if at least a record of the row was fetched.
    **     OM  20220228          Reset and specially mark OUTER-JOIN buffers with no records.
    **     SVL 20220412          Fixed a problem repositioning to the first result with next(LockType).
    **     IAS 20220513          Fixed repositionByID logic for UNKNOWN value of the first argument.
    **     OM  20220530          Fixed flush of nursery (only the affected index).
    **     OM  20220701          Flatten calls to initialize().
    **     CA  20221006          Added JMX instrumentation for query assemble. Refs #6814
    **     CA  20221013          In 'repositionByID', if the row number is after the last row in the ResultSet,
    **                           assume an error (query off-end) and do not try to fetch the record.
    **     CA  20221109          Implemented runtime for EXCEPT/FIELDS options - only NO-LOCK records are loaded
    **                           as 'partial/incomplete'.   At this time, extent fields are always loaded.
    **     CA  20220531          Ensure the sortTranslated flag is set when the sort is translated with the bound
    **                           buffers.
    **     CA  20220919          Clean the current results before setting the new results.
    **     CA  20221005          Fixed NPE in assembleFromClause.
    **     OM  20221103          New class names for FQLPreprocessor, FQLExpression, FQLBundle, and FQLCache.
    **     CA  20221130          Refactored the WHERE clause translation (when the bound and definition buffers
    **                           are not the same), to be aware of the external buffers (i.e. added for CAN-FIND
    **                           sub-select clauses).  The translate will be performed before the query is being
    **                           executed.
    **     SVL 20230108          Improved performance by replacing some "for-each" loops with indexed "for" loops.
    **     SVL 20230109          Reflected PreselectQuery.components() now providing direct access to the
    **                           components array in order to improve performance.
    **     CA  20230110          Do not create 'whereExprList' until something is added to it.
    **     SVL 20230113          Improved performance by replacing some "for-each" loops with indexed "for" loops.
    **     CA  20221109          Implemented runtime for EXCEPT/FIELDS options - only NO-LOCK records are loaded
    **                           as 'partial/incomplete'.   At this time, extent fields are always loaded.
    ** 177 CA  20230227          Fixed where clause translation when the external buffer's binding and definition
    **                           does not match.
    ** 178 CA  20230223          Fixed a bug for EXCEPT field processing in 'assembleSelectClause' - the prop.id
    **                           was being used instead of prop.propId, which is not the same, in case of
    **                           denormalized extent.
    ** 178 AL2 20230315          Avoid redundant fetch.
    **         20230329          Avoid redundant fetch only when locking shouldn't be updated.
    ** 179 IAS 20230405          Fixed FILL support for recursive DATA-RELATION
    **                           Added DATE-RELATION to the 'scroll' parameters
    ** 180 OM  20230404          The join navigation key may contain null/unknown values.
    ** 180 RAA 20230406          Added isOrderByRequired() which checks if the ORDER BY clause is really
    **                           necessary (will the query retrieve a non-unique record?).
    ** 181 GBB 20230512          Logging methods replaced by CentralLogger/ConversionStatus.
    ** 182 SVL 20230516          Support for more granular configuration of dirty sharing functionality.
    ** 183 DDF 20230627          Replaced static variables of DirtyShareSupport with static method calls.
    ** 184 AB  20230803          Modified assembleWhereClause(),assembleFromClause() and prepareParameters() to
    **                           support OUTER-JOIN.
    **     AB  20230803          Modified assembleWhereClause() and prepareParameters(), renamed assembleFromClause()
    **                           to assembleFromAndWhereClaused, in order to support OUTER-JOIN.
    **     AB  20230825          Modified coreFetch() to support OUTER-JOIN. Moved the responsibility of
    **                           throwing QueryOffEndException outside for loop.
    **     AL2 20230825          Avoid calling errorNotOnFile if the fetched component was outer.
    **     AB  20230830          Modified assembleWhereClause() to assemble "on" or "where" clause, depending
    **                           on the parameter "isOn".
    ** 185 AL2 20231027          Changes signature of preserveBuffersOnEmptyResults to expose the results.
    ** 186 OM  20231115          Fixed generation of nested multiplexed sub-queries.
    ** 187 RAA 20231006          Added P2JQueryExecutor layer to the query execution.
    **     RAA 20231010          Removed DatabaseStatistics usage.
    ** 188 AD  20231108          PreselectQuery now listens for deleted records, being sensitive
    **                           to their changes.
    **     AD  20231110          Added parameter forceOnlyId for ignoring already existing results,
    **                           but use already existing primary keys to get accurate results from the database.
    **     AD  20231113          Deleted shouldRegisterRecordChangeListeners() calls in initialize methods and
    **                           refactored stateChanged() method.
    **     AD  20231124          Reset forceOnlyId in cleanup and fix forceOnlyId assignment in stateChanged().
    **     AL2 20231127          Rule out registering listeners according to shouldRegisterRecordChangeListeners.
    ** 189 AL2 20231219          Fixed NPE in PreselectQuery.close(). Dynamic query may not have yet any component.
    ** 190 ES  20231203          Changed first, last, next, previous, current, unique methods to return boolean,
    **                           instead of void. These methods now returns true if operation was succesful, and false
    **                           not and lenientOffEnd is set on true, otherwise it throwd an QQE exception.
    ** 191 RAA 20231220          Small performance improvements.
    ** 192 AB  20231002          Modified load() to handle correctly the OUTER-JOIN with preselect.
    ** 193 DDF 20240207          When caching results, the Result set of a non scrolling query may have not been
    **                           accessed yet.
    **     DDF 20240209          Do not set the results when the old and cached results are the same.
    ** 194 HC  20240222          Enabled JMX on FWD Client.
    ** 195 CA  20240331          Use ArrayList and Java for loops in query assembler.
    ** 196 ES  20240404          Persist the record in buffer in case of iteration in FOR EACH with break by statement.
    ** 197 CA  20240422          Allow the query to be reset to its initialization state, so the same instance can
    **                           be used to execute child buffer FILL operations.
    ** 198 OM  20240430          Extents are handled like scalars. The FQL converter will filter them out if
    **                           necessary, and SQLQuery will fetch them from secondary tables at hydration time.
    ** 199 CA  20240512          'prepareParameters' must resolve 'P2JQuery.ParamResolver' as these are deferred
    **                           on each iteration of the outer component.
    ** 200 SB  20240620          Use the getter value of the DMO attribute to check for null argument when
    **                           composing WHERE clause for joins. Refs #8886.
    ** 201 SB  20240709          Fix a regression where composing WHERE clause for join with IS NULL would not
    **                           remove an unknown/null parameter from the parameter list (see #8938). Refactor
    **                           the WHERE clause assembly for joins to use getParameters.
    ** 202 CA  20240809          Avoid iterator usage, for performance improvement.
    ** 203 CA  20240809          Skip using JMX timers when JMX_DEBUG flag is not set.
    ** 204 CA  20230924          Further reduce context-local lookup.
    ** 205 TJD 20230516          Bringing back support for dirty records leaks to other sessions
    **     TJD 20230724          Migrate earlyPublishEntities from List to Set to improve performance
    **     TJD 20240912          DirtyShareSupport flags logic dependency update
    ** 206 ES  20240919          Create Results based on iterating flag in executeQuery.
    ** 207 AL2 20241002          Force the caching of results only by PK if there are too many to cache.
    ** 208 AL2 20241017          Use the provided results implementation instead of the using the one in the
    **                           query. This way, AdaptiveQuery.adapter will cache the low-level results instead
    **                           of the high-level results.
    ** 209 DDF 20230712          Added support for the the FORWARD-ONLY attribute.
    **     DDF 20230713          Moved utility methods for the FORWARD-ONLY attribute to AbstractQuery.
    **     DDF 20230717          Only allow scrolling when forwardOnly is false.
    ** 210 CA  20241030          Small improvement for 'getOffEndListeners' to avoid array resize.
    ** 211 SB  20241105          Release the last bound buffer of a query if REPOSITION-BY-ID has an unknown
    **                           argument. Refs #9199.
    ** 212 LS  20241108          Created assembleComponent() that is used for every component when
    **                           assembling the whole select clause.
    ** 213 OM  20241128          Multi-tenant runtime support: selected the proper persistence context, eventually
    **                           based on [sharedDb] parameter.
    **                           Implementation of [setMultiTenantAlternative] method.
    ** 214 AL2 20241209          For assembleSelectClause() do not force onlyPk, retrieve full records when possible.
    ** 215 ES  20250129          Avoid releasing the buffers in FIRST/LAST component when reaching the QueryOffEnd.
    ** 216 ICP 20250114          Adjusted assembleWhereClause() to detect if a server side join has a subselect.
    ** 217 ES  20250218          Make sure to release FIRST/LAST component in case is null and result set of a
    **                           outer-join query is available.
    ** 218 AL2 20250226          Added shouldExecuteForward to encapsulate the decision on using ForwardResults.
    ** 219 AB2 20250419          Changed cachedResults to retrieve the array of primary keys of the object if
    **                           forceOnlyId is true. See #9722.
    ** 220 RNC 20250428          Always register the dynamic query as a global scope listener.
    ** 221 LS  20250416          Small refactoring in assembleComponent() to reuse the size of the array.
    ** 222 ES  20250508          Synchronize the query buffer if it is AUTO-SYNCHRONIZE attribute is set on true.
    */

    /*
    ** This program is free software: you can redistribute it and/or modify
    ** it under the terms of the GNU Affero General Public License as
    ** published by the Free Software Foundation, either version 3 of the
    ** License, or (at your option) any later version.
    **
    ** This program is distributed in the hope that it will be useful,
    ** but WITHOUT ANY WARRANTY; without even the implied warranty of
    ** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    ** GNU Affero General Public License for more details.
    **
    ** You may find a copy of the GNU Affero GPL version 3 at the following
    ** location: https://www.gnu.org/licenses/agpl-3.0.en.html
    **
    ** Additional terms under GNU Affero GPL version 3 section 7:
    **
    **   Under Section 7 of the GNU Affero GPL version 3, the following additional
    **   terms apply to the works covered under the License.  These additional terms
    **   are non-permissive additional terms allowed under Section 7 of the GNU
    **   Affero GPL version 3 and may not be removed by you.
    **
    **   0. Attribution Requirement.
    **
    **     You must preserve all legal notices or author attributions in the covered
    **     work or Appropriate Legal Notices displayed by works containing the covered
    **     work.  You may not remove from the covered work any author or developer
    **     credit already included within the covered work.
    **
    **   1. No License To Use Trademarks.
    **
    **     This license does not grant any license or rights to use the trademarks
    **     Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
    **     of Golden Code Development Corporation. You are not authorized to use the
    **     name Golden Code, FWD, or the names of any author or contributor, for
    **     publicity purposes without written authorization.
    **
    **   2. No Misrepresentation of Affiliation.
    **
    **     You may not represent yourself as Golden Code Development Corporation or FWD.
    **
    **     You may not represent yourself for publicity purposes as associated with
    **     Golden Code Development Corporation, FWD, or any author or contributor to
    **     the covered work, without written authorization.
    **
    **   3. No Misrepresentation of Source or Origin.
    **
    **     You may not represent the covered work as solely your work.  All modified
    **     versions of the covered work must be marked in a reasonable way to make it
    **     clear that the modified work is not originating from Golden Code Development
    **     Corporation or FWD.  All modified versions must contain the notices of
    **     attribution required in this license.
    */

    package com.goldencode.p2j.persist;

    import java.io.*;
    import java.sql.*;
    import java.util.*;
    import java.util.function.*;
    import java.util.logging.*;
    import com.goldencode.p2j.jmx.*;
    import com.goldencode.p2j.persist.dirty.*;
    import com.goldencode.p2j.persist.event.*;
    import com.goldencode.p2j.persist.lock.*;
    import com.goldencode.p2j.persist.orm.*;
    import com.goldencode.p2j.util.*;
    import com.goldencode.p2j.util.logging.CentralLogger;
    import com.goldencode.p2j.util.ErrorManager;
    import com.goldencode.util.*;

    /**
     * Query which issues an FQL query to the database server and uses a
     * scrollable database cursor to navigate over the set of results.  This
     * query is used to back converted Progress queries which either explicitly
     * or implicitly cause a result list to be preselected.  Both single- and
     * multi-table queries are supported.  Because this class implements the
     * <code>Joinable</code> interface, instances may be added as components of
     * a {@link CompoundQuery}.
     * <p>
     * The query is lazily executed the first time a record retrieval/navigation
     * method is invoked.  Only the database identifiers (i.e., primary keys) of
     * the records are retrieved by the initial query.  On each subsequent fetch,
     * a row in this result list is retrieved, where each column contains the
     * primary key for the record associated with the corresponding query
     * component.  The record for each primary key is retrieved and stored in its
     * associated buffer.
     * <p>
     * At this time, outer joins are not supported.  Furthermore, components may
     * only specify NEXT as their iteration type at this time.  FIRST and LAST
     * currently are not supported.
     * <p>
     * When used to represent a Progress query (created with DEFINE or OPEN
     * QUERY), lenient-off-end must be enabled at construction.  This mode
     * suppresses end condition exceptions for <code>next</code> and
     * <code>previous</code> commands which run off the end of the query's
     * results.
     * <p>
     * This query may be integrated with a client-side where clause expression,
     * by passing a non-<code>null</code> <code>Supplier&lt;logical&gt;</code> argument
     * to one of the single-table constructor variants.  Note that additional
     * components may not be added to an instance so created, since this type of
     * query must act against no more than one record buffer.  If a multi-table
     * preselect query is called for, <code>CompoundQuery</code> must be used,
     * though its components may be instances of this class.
     */
    public class PreselectQuery
    extends AbstractQuery
    implements Joinable,
               RecordChangeListener,
               QueryConstants,
               Finalizable,
               SessionListener
    {
       /**
        * A threshold for {@link #cacheResults(Results, boolean)} to decide when we should drop caching full DMO.
        * If this more records are cached, we will cache only the PK for the sake of the session cache performance.
        */
       protected static final int LENIENT_CACHING_THRESHOLD = 64;

       /** Empty array instance. */
       private static final ArrayList<SortCriterion> EMPTY_ARRAY = new ArrayList<>();

       /** Logger */
       private static final CentralLogger LOG = CentralLogger.get(PreselectQuery.class.getName());

       /** Instrumentation for {@link #assembleFQL()}. */
       private static final NanoTimer QUERY_ASSEMBLE = NanoTimer.getInstance(FwdServerJMX.TimeStat.OrmPreselectQueryAssemble);

       /** Has query been repositioned between result rows? */
       protected boolean betweenRows = true;

       /** Has the query run off either end of its result set? */
       protected OffEnd offEnd = OffEnd.NONE;

       /** FQL order by clause used across all query components */
       protected String sort;

       /** Components of the query;  at least 1 is required */
       protected ArrayList<QueryComponent> components;

       /**
        * Determines if the current row was deleted by DELETE-RESULT-LIST-ENTRY. Note that if the
        * query is off-end, it also can be "deleted".
        */
       protected boolean currentRowDeleted = false;

       /** Flag indicating if the {@link #sort} has been translated. */
       protected boolean sortTranslated = false;

       /** List of client-side where clause expressions */
       private List<Supplier<logical>> whereExprList = null;

       /** Persistence object which this query's components use */
       private Persistence persistence = null;

       /** Flag indicating results must be scrollable */
       private boolean scrolling = false;

       /** Cache of primary key lists to result set row numbers */
       private final RepositionCache repoCache = new RepositionCache();

       /** FQL query string to retrieve primary key ID arrays */
       private String fql = null;

       /** Result set for this query */
       private Results results = null;

       /** Does this query have any arguments which are runtime resolvable? */
       private boolean resolvableArgs = false;

       /** Number of total results returned */
       private int rowCount = -1;

       /** Is query registered for session event notification? */
       private boolean regWithSession = false;

       /** Has query been registered for resource cleanup? */
       private boolean regCleaner = false;

       /** Determines if this will be unregistered from ChangeBroker on cleanup. */
       protected boolean unregisterOnCleanup = false;

       /** Cleaner executed when this query's block is exit. */
       private final Finalizable cleaner = new Finalizable()
       {
          public void finished() { cleanup(); }
          public void iterate() {}
          public void retry() {}
          public void deleted() {}
       };

       /** Flag indicating that this instance was registered as a finalizable. */
       private boolean thisFinalizable = false;

       /** Force query to retrieve full records, rather than primary keys only */
       private boolean fullRecords = false;

       /** Force the query to retrieve only the ids */
       private boolean forceOnlyId = false;

       /** The list of sorting criteria added dynamically, at runtime. */
       private List<String> dynamicSorts = null;

       /** Flag indicated if the WHERE clause has been translated. */
       private boolean whereTranslated = false;

       /** Test whether the log messages with FINE priority are allowed. */
       private static final boolean LOG_FINE = LOG.isLoggable(Level.FINE);

       /**
        * Default constructor. Initialization is deferred until <code>initialize()</code> is
        * called.
        */
       public PreselectQuery()
       {
       }

       /**
        * Initializer logic which is typically used for a multi-table query. Details
        * specific to each table component of the query are added via one of the
        * <code>addComponent</code> method variants.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    sort
        *           FQL order by clause to be applied across all components of the query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(String sort)
       {
          // WARNING: any usage from FWD runtime of this API must give you a already translated sort
          // clause (in case of bound buffers).

          // when this is called from converted code, then the sort clause is not translated, and will
          // be translated on query execute!
          this.sort = sort;
          components = new ArrayList<>(3);

          TransactionManager.registerOffEndQuery(this);

          registerRecordChangeListeners(null);
          return this;
       }

       /**
        * Initializer logic which is typically used for a multi-table query. Details
        * specific to each table component of the query are added via one of the
        * <code>addComponent</code> method variants.
        * <p>
        * This variant is used by code which is converted from an OPEN QUERY
        * statement, which permits advancing off the end of a result set without
        * raising a <code>QueryOffEndException</code>.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    lenientOffEnd
        *           <code>true</code> if advancing past either end of the result
        *           set should NOT raise a <code>QueryOffEndException</code>;
        *           <code>false</code> if doing so should raise the exception.
        * @param    sort
        *           FQL order by clause to be applied across all components of the query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(boolean lenientOffEnd, String sort)
       {
          // WARNING: any usage from FWD runtime of this API must give you a already translated sort
          // clause (in case of bound buffers).

          // when this is called from converted code, then the sort clause is not translated, and will
          // be translated on query open!
          initialize(sort);
          this.lenientOffEnd = lenientOffEnd;

          return this;
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * The where clause must expect no substitution parameters.  A share lock
        * will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort)
       {
          return initialize(dmo, where, whereExpr, sort, null, null, null, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * The where clause must expect no substitution parameters.  A share lock
        * will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo)
       {
          return initialize(dmo, where, whereExpr, sort, indexInfo, null, null, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * The where clause must expect no substitution parameters.  A share lock
        * will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        DataModelObject   inverse)
       {
          return initialize(dmo, where, whereExpr, sort, null, inverse, null, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * The where clause must expect no substitution parameters.  A share lock
        * will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo,
                                        DataModelObject   inverse)
       {
          return initialize(dmo, where, whereExpr, sort, indexInfo, inverse, null, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * A share lock will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        Object[]          args)
       {
          return initialize(dmo, where, whereExpr, sort, null, null, args, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * A share lock will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo,
                                        Object[]          args)
       {
          return initialize(dmo, where, whereExpr, sort, indexInfo, null, args, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * A share lock will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        DataModelObject   inverse,
                                        Object[]          args)
       {
          return initialize(dmo, where, whereExpr, sort, null, inverse, args, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * A share lock will be applied to retrieved records.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo,
                                        DataModelObject   inverse,
                                        Object[]          args)
       {
          return initialize(dmo, where, whereExpr, sort, indexInfo, inverse, args, null);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * The where clause must expect no substitution parameters.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        LockType          lockType)
       {
          return initialize(dmo, where, whereExpr, sort, null, null, null, lockType);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * The where clause must expect no substitution parameters.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo,
                                        LockType          lockType)
       {
          return initialize(dmo, where, whereExpr, sort, indexInfo, null, null, lockType);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        DataModelObject   inverse,
                                        LockType          lockType)
       {
          return initialize(dmo, where, whereExpr, sort, null, inverse, null, lockType);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * The where clause must expect no substitution parameters.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo,
                                        DataModelObject   inverse,
                                        LockType          lockType)
       {
          return initialize(dmo, where, whereExpr, sort, indexInfo, inverse, null, lockType);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        Object[]          args,
                                        LockType          lockType)
       {
          return initialize(dmo, where, whereExpr, sort, null, null, args, lockType);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo,
                                        Object[]          args,
                                        LockType          lockType)
       {
          return initialize(dmo, where, whereExpr, sort, indexInfo, null, args, lockType);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        DataModelObject   inverse,
                                        Object[]          args,
                                        LockType          lockType)
       {
          return initialize(dmo, where, whereExpr, sort, null, inverse, args, lockType);
       }

       /**
        * Initializer logic designed for a single-table preselect query.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    dmo
        *           DMO proxy which defines buffer and record type.
        * @param    where
        *           FQL where clause.  May be <code>null</code>.
        * @param    whereExpr
        *           Client-side where clause expression.  May be <code>null</code>.
        * @param    sort
        *           FQL order by clause.
        * @param    indexInfo
        *           Index information string as it is returned by INDEX-INFORMATION.
        *           May be <code>null</code> if it is not an OPEN QUERY case or if
        *           you don't need debug information about selected indexes.
        * @param    inverse
        *           DMO to which this query should join via a foreign relation.
        *           May be <code>null</code>.
        * @param    args
        *           Substitution parameters to be inserted into placeholders within
        *           the where clause.
        * @param    lockType
        *           Lock type to apply to records retrieved by this query. If {@code null}, the default lock level
        *           is assumed.
        *
        * @return   This query instance.
        */
       public PreselectQuery initialize(DataModelObject   dmo,
                                        String            where,
                                        Supplier<logical> whereExpr,
                                        String            sort,
                                        String            indexInfo,
                                        DataModelObject   inverse,
                                        Object[]          args,
                                        LockType          lockType)
       {
          if (lockType == null)
          {
             lockType = getDefaultLock();
          }

          components = new ArrayList<>(1);

          setErrorIfNull(false);
          addComponent(dmo, where, indexInfo, inverse, args, lockType, NEXT, false);

          BufferReference proxy = (BufferReference) dmo;
          RecordBuffer buffer = proxy.buffer();
          RecordBuffer defBuffer = proxy.definition();

          if (!buffersMatch(buffer, defBuffer))
          {
             sort = translateSort(buffer, defBuffer, sort);
             sortTranslated = true;
          }

          this.sort = sort;
          if (whereExpr != null)
          {
             addWhereExpression(whereExpr);
          }

          buffer.getTxHelper().registerOffEndQuery(this);
          registerRecordChangeListeners(null);
          return this;
       }

       /**
        * Translate the WHERE clause for all {@link #components}.
        */
       @Override
       public void translateWhere()
       {
          if (whereTranslated)
          {
             return;
          }

          whereTranslated = true;

          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent qc = components.get(i);
             qc.translateWhere();
          }
       }

       /**
        * Set the behavior of this query as a record retrieval request moves off
        * the end of its results.  If set to lenient mode, invocations of
        * <code>next</code> and <code>previous</code> which do not find a record
        * do not raise a <code>QueryOffEndException</code>.  By default, this
        * would normally raise the exception.
        *
        * @param   lenientOffEnd
        *          <code>true</code> to suppress <code>QueryOffEndException</code>
        *          for <code>next</code> and <code>previous</code> requests;
        *          <code>false</code> to permit these exceptions.
        */
       public void setLenientOffEnd(boolean lenientOffEnd)
       {
          this.lenientOffEnd = lenientOffEnd;
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        */
       public void addComponent(DataModelObject dmo, String where)
       {
          addComponent(dmo, where, (String) null);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        */
       public void addComponent(DataModelObject dmo, String where, String indexInfo)
       {
          addComponent(dmo, where, indexInfo, null, null, getDefaultLock(), NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        */
       public void addComponent(DataModelObject dmo, String where, DataModelObject inverse)
       {
          addComponent(dmo, where, null, inverse);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                DataModelObject inverse)
       {
          addComponent(dmo, where, indexInfo, inverse, null, getDefaultLock(), NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        */
       public void addComponent(DataModelObject dmo, String where, Object[] args)
       {
          addComponent(dmo, where, (String) null, args);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        */
       public void addComponent(DataModelObject dmo, String where, String indexInfo, Object[] args)
       {
          addComponent(dmo, where, indexInfo, null, args, getDefaultLock(), NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                DataModelObject inverse,
                                Object[] args)
       {
          addComponent(dmo, where, null, inverse, args);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>share lock
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                DataModelObject inverse,
                                Object[] args)
       {
          addComponent(dmo, where, indexInfo, inverse, args, getDefaultLock(), NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo, String where, LockType lockType)
       {
          addComponent(dmo, where, (String) null, lockType);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                LockType lockType)
       {
          addComponent(dmo, where, indexInfo, null, null, lockType, NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                DataModelObject inverse,
                                LockType lockType)
       {
          addComponent(dmo, where, null, inverse, lockType);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no substitution parameters
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                DataModelObject inverse,
                                LockType lockType)
       {
          addComponent(dmo, where, indexInfo, inverse, null, lockType, NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                Object[] args,
                                LockType lockType)
       {
          addComponent(dmo, where, (String) null, args, lockType);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                Object[] args,
                                LockType lockType)
       {
          addComponent(dmo, where, indexInfo, null, args, lockType, NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                DataModelObject inverse,
                                Object[] args,
                                LockType lockType)
       {
          addComponent(dmo, where, null, inverse, args, lockType);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>inner join
        *   <li><code>NEXT</code> iteration type
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                DataModelObject inverse,
                                Object[] args,
                                LockType lockType)
       {
          addComponent(dmo, where, indexInfo, inverse, args, lockType, NEXT, false);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        * @param   iteration
        *          Iteration/retrieval mode for this component of the query.  At
        *          this time, only <code>NEXT</code> is supported.  Future
        *          support for <code>FIRST</code>, <code>LAST</code>, and
        *          <code>UNIQUE</code> is expected.
        * @param   outer
        *          <code>true</code> if an outer join is required.  At this time,
        *          only inner joins are supported.  Future support for outer
        *          joins is expected.
        *
        * @throws  UnsupportedOperationException
        *          if <code>outer</code> is <code>true</code> (temporary);
        *          if <code>iteration</code> is not <code>NEXT</code> (temporary);
        *          if <code>dmo</code>'s associated database is different than
        *          that of a previously added component's database (cross-database
        *          preselect joins are not possible).
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                Object[] args,
                                LockType lockType,
                                int iteration,
                                boolean outer)
       {
          addComponent(dmo, where, (String) null, args, lockType, iteration, outer);
       }

       /**
        * Add a component to the query.  The following defaults are used:
        * <ul>
        *   <li>no inverse join DMO
        * </ul>
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        * @param   iteration
        *          Iteration/retrieval mode for this component of the query.  At
        *          this time, only <code>NEXT</code> is supported.  Future
        *          support for <code>FIRST</code>, <code>LAST</code>, and
        *          <code>UNIQUE</code> is expected.
        * @param   outer
        *          <code>true</code> if an outer join is required.  At this time,
        *          only inner joins are supported.  Future support for outer
        *          joins is expected.
        *
        * @throws  UnsupportedOperationException
        *          if <code>outer</code> is <code>true</code> (temporary);
        *          if <code>iteration</code> is not <code>NEXT</code> (temporary);
        *          if <code>dmo</code>'s associated database is different than
        *          that of a previously added component's database (cross-database
        *          preselect joins are not possible).
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                Object[] args,
                                LockType lockType,
                                int iteration,
                                boolean outer)
       {
          BufferReference proxy = (BufferReference) dmo;
          RecordBuffer buffer = proxy.buffer();
          RecordBuffer defBuffer = proxy.definition();

          Supplier<String> translateWhere = null;
          if (where != null && (!buffersMatch(buffer, defBuffer) || anyBoundBufferInComponent()))
          {
             translateWhere = () ->
             {
                List<RecordBuffer> binding = new ArrayList<>();
                List<RecordBuffer> definition = new ArrayList<>();
                binding.add(buffer);
                definition.add(defBuffer);
                for (int i = 0; i < components.size(); i++)
                {
                   QueryComponent qc = components.get(i);
                   if (qc.getBuffer() != qc.getDefinitionBuffer())
                   {
                      binding.add(qc.getBuffer());
                      definition.add(qc.getDefinitionBuffer());
                   }
                }

                return translateWhere(binding, definition, where);
             };
          }

          QueryComponent comp = new QueryComponent(this,
                                                   (BufferReference) dmo,
                                                   null,
                                                   where,
                                                   translateWhere,
                                                   indexInfo,
                                                   lockType,
                                                   args,
                                                   iteration,
                                                   outer);
          addComponent(comp);
       }

       /**
        * Add a component to the query.
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        * @param   iteration
        *          Iteration/retrieval mode for this component of the query.  At
        *          this time, only <code>NEXT</code> is supported.  Future
        *          support for <code>FIRST</code>, <code>LAST</code>, and
        *          <code>UNIQUE</code> is expected.
        * @param   outer
        *          <code>true</code> if an outer join is required.  At this time,
        *          only inner joins are supported.  Future support for outer
        *          joins is expected.
        *
        * @throws  UnsupportedOperationException
        *          if <code>outer</code> is <code>true</code> (temporary);
        *          if <code>iteration</code> is not <code>NEXT</code> (temporary);
        *          if <code>dmo</code>'s associated database is different than
        *          that of a previously added component's database (cross-database
        *          preselect joins are not possible).
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                DataModelObject inverse,
                                Object[] args,
                                LockType lockType,
                                int iteration,
                                boolean outer)
       {
          addComponent(dmo, where, null, inverse, args, lockType, iteration, outer);
       }

       /**
        * Add a component to the query.
        *
        * @param   dmo
        *          DMO proxy which defines buffer and record type.
        * @param   where
        *          FQL where clause.  May be <code>null</code>.
        * @param   indexInfo
        *          Index information string as it is returned by INDEX-INFORMATION.
        *          May be <code>null</code> if it is not an OPEN QUERY case or if
        *          you don't need debug information about selected indexes.
        * @param   inverse
        *          DMO to which this query should join via a foreign relation.
        *          May be <code>null</code>.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          the where clause.
        * @param   lockType
        *          Lock type to apply to records retrieved by this component of
        *          the query.
        * @param   iteration
        *          Iteration/retrieval mode for this component of the query.  At
        *          this time, only <code>NEXT</code> is supported.  Future
        *          support for <code>FIRST</code>, <code>LAST</code>, and
        *          <code>UNIQUE</code> is expected.
        * @param   outer
        *          <code>true</code> if an outer join is required.  At this time,
        *          only inner joins are supported.  Future support for outer
        *          joins is expected.
        *
        * @throws  UnsupportedOperationException
        *          if <code>outer</code> is <code>true</code> (temporary);
        *          if <code>iteration</code> is not <code>NEXT</code> (temporary);
        *          if <code>dmo</code>'s associated database is different than
        *          that of a previously added component's database (cross-database
        *          preselect joins are not possible).
        */
       public void addComponent(DataModelObject dmo,
                                String where,
                                String indexInfo,
                                DataModelObject inverse,
                                Object[] args,
                                LockType lockType,
                                int iteration,
                                boolean outer)
       {
          BufferReference proxy = (BufferReference) dmo;
          RecordBuffer buffer = proxy.buffer();
          RecordBuffer defBuffer = proxy.definition();

          Supplier<String> translateWhere = null;
          if (where != null && (!buffersMatch(buffer, defBuffer) || anyBoundBufferInComponent()))
          {
             translateWhere = () ->
             {
                List<RecordBuffer> binding = new ArrayList<>();
                List<RecordBuffer> definition = new ArrayList<>();
                binding.add(buffer);
                definition.add(defBuffer);
                for (int i = 0; i < components.size(); i++)
                {
                   QueryComponent qc = components.get(i);
                   if (qc.getBuffer() != qc.getDefinitionBuffer())
                   {
                      binding.add(qc.getBuffer());
                      definition.add(qc.getDefinitionBuffer());
                   }
                }
                return translateWhere(binding, definition, where);
             };
          }

          AbstractJoin join = processJoin(proxy, (BufferReference) inverse);

          QueryComponent comp = new QueryComponent(this,
                                                   proxy,
                                                   join,
                                                   where,
                                                   translateWhere,
                                                   indexInfo,
                                                   lockType,
                                                   args,
                                                   iteration,
                                                   outer);
          addComponent(comp);
       }

       /**
        * Get the external buffers, if any, associated with this query.
        *
        * @return  Array of external buffers or {@code null}.
        */
       @Override
       public RecordBuffer[] getExternalBuffers()
       {
          return super.getExternalBuffers();
       }

       /**
        * Check if this type of query should register record change listeners. Use this
        * to avoid computing any scope for registration and end up with a no registration
        * behavior.
        *
        * @return   {@code true} if we should invest time into computing the scope in order
        *           to call {@link #registerRecordChangeListeners} or {@code false} if we
        *           can skip doing any work to honor {@link #registerRecordChangeListeners}.
        */
       @Override
       public boolean shouldRegisterRecordChangeListeners()
       {
          return true;
       }

       /**
        * Register all <code>RecordChangeListener</code>s associated with this
        * query (if any), with the context-local <code>ChangeBroker</code> at the
        * indicated scope.
        * <p>
        * This implementation does nothing.
        *
        * @param   scope
        *          Scope at which listeners should be registered with the change
        *          broker.
        */
       @Override
       public void registerRecordChangeListeners(Integer scope)
       {
          if (!shouldRegisterRecordChangeListeners())
          {
             return;
          }

          if (scope == null)
          {
             ChangeBroker.get().addListener(this);
             return;
          }

          ChangeBroker.get().addListener(this, scope);

          // if these were added as global listeners in ChangeBroker remember to remove them
          // when the query is destroyed. Otherwise, the ChangeBroker will drop them automatically
          // when the respective scope is finished.
          if (scope == 0)
          {
             unregisterOnCleanup = true;
          }
       }

       /**
        * Report the record buffers for whose changes this object is interested
        * in listening.  This listener will be registered with the {@link
        * ChangeBroker} to receive notifications of any change to DMOs whose
        * interface types match those returned by the {@link
        * RecordBuffer#getDMOInterface} methods of the returned buffers.
        *
        * @return  An iterator on the record buffers stored in each of this
        *          query's components.
        */
       @Override
       public Iterator<RecordBuffer> recordBuffers()
       {
          return Arrays.asList(getRecordBuffers()).iterator();
       }

       /**
        * Respond to a {@code RecordChangeEvent}.
        *
        * @param   event
        *          Event which describes the DMO state change.
        */
       @Override
       public void stateChanged(RecordChangeEvent event)
       throws PersistenceException
       {
          if (event.isDelete() && (event.getDMO() != null))
          {
             forceOnlyId = true;
          }
       }

       /**
        * Respond to a clean (i.e., transaction commit) session event by caching
        * all preselected results and using that cache to back the query's results
        * for all future fetches.  If all results already are cached, do nothing.
        * <p>
        * Respond to a session closing event by clearing all results, unless they
        * were just cached as described above.
        *
        * @param   event
        *          Session event type.
        *
        * @return  <code>true</code>, indicating this query should be
        *          deregistered as a session listener after this method returns;
        *          else <code>false</code>.
        *
        * @throws  PersistenceException
        *          if there is an error caching results.
        */
       public boolean sessionEvent(SessionListener.Event event)
       throws PersistenceException
       {
          if (closed)
          {
             return true;
          }

          switch (event)
          {
             case TRANSACTION_COMMITTING:
                if (results != null && !results.isResultSetCached())
                {
                   try
                   {
                      Results cached = cacheResults(results, true);
                      if (results != cached)
                      {
                         results.sessionClosing();
                         setResults(cached);
                      }
                   }
                   catch (Exception exc)
                   {
                      resetResults();

                      throw new PersistenceException("Error caching results", exc);
                   }
                }
                break;

             case SESSION_CLOSING_NORMALLY:
                if (results != null && !(results instanceof SimpleResults))
                {
                   resetResults();
                }
                break;

             case SESSION_CLOSING_WITH_ERROR:
                resetResults();
                break;

             default:
                return false;
          }

          return true;
       }

       /**
        * Invoked when this session listener is removed from the list of
        * registered session listeners.
        */
       public void deregisteredSessionListener()
       {
          regWithSession = false;
       }

       /**
        * Provides a notification that the scope in which this query was created
        * is ending.  This is a no-op implementation required by the interface.
        */
       public void finished()
       {
       }

       /**
        * Provides a notification that the external program scope in which the object is registered is
        * is being deleted and the object's reference will be lost after this method is called.
        * <p>
        * This is a no-op implementation required by the interface.
        */
       public void deleted()
       {
       }

       /**
        * Provides a notification that the block whose scope in which the object
        * is registered is about to iterate and attempt another pass.  This is a
        * no-op implementation required by the interface.
        */
       public void iterate()
       {
       }

       /**
        * Provides a notification that the block whose scope in which the object
        * is registered is about to retry the same iteration (all loop control
        * data is unchanged).  This implementation sets the state of the query as
        * if a reposition to the current row has just occurred.  This will keep
        * the subsequent call to {@link #next()} at the top of the block body from
        * advancing to the next row.
        * <p>
        * This implementation retries the current record only if the query is
        * associated with a FOR block (see {@link BlockType}).
        */
       public void retry()
       {
          BlockType type = TransactionManager.getBlockType();

          // only queries associated with a FOR block allows the retry of the
          // current record
          switch (type)
          {
             case FOR_LOOP:
             case FOR_LOOP_WHILE:
             case FOR_LOOP_TO:
             case FOR_LOOP_TO_WHILE:
             case FOR_BLOCK:
             case FOR_BLOCK_WHILE:
             case FOR_BLOCK_TO:
             case FOR_BLOCK_TO_WHILE:
                betweenRows = true;
                break;
          }

          if (LOG_FINE)
          {
             LOG.log(Level.FINE, "Preparing query for retry [" + this + "]");
          }
       }

       /**
        * Create a string representation of this object's state, primarily for debug purposes.
        *
        * @return  String representation of this object.
        */
       public String toString()
       {
          return "[" + getTableCount() + " table(s)] FQL: " + fql;
       }

       /**
        * Add a client-side where expression to this query.  Each initial query
        * result found by the server must be accepted by this expression before
        * becoming a part of the query's final result set.
        *
        * @param   whereExpr
        *          Client-side where clause expression.  May not be
        *          <code>null</code>.
        *
        * @throws  NullPointerException
        *          if <code>whereExpr</code> is <code>null</code>.
        */
       public void addWhereExpression(Supplier<logical> whereExpr)
       {
          if (whereExpr == null)
          {
             throw new NullPointerException("Client-side where expression may not be null");
          }

          if (whereExprList == null)
          {
             whereExprList = new ArrayList<>(1);
          }

          whereExprList.add(whereExpr);
       }

       /**
        * Set the iteration type for the most recently added query component.
        *
        * @param   iteration
        *          Retrieval mode from {@link QueryConstants}. Must not be FIRST or LAST (use
        *          {@link #setIterationType(int, String)} instead for these types.
        *
        * @throws  IllegalStateException
        *          If no component has yet been added.
        * @throws  IllegalArgumentException
        *          If <code>iteration</code> type is <code>FIRST</code> or <code>LAST</code>.
        */
       public void setIterationType(int iteration)
       {
          setIterationType(iteration, null);
       }

       /**
        * Set the iteration type for the most recently added query component.
        *
        * @param   iteration
        *          Retrieval mode from {@link QueryConstants}.
        * @param   sort
        *          Order by clause which should be associated with the most recently added query
        *          component. Must be non-<code>null</code> for iteration types <code>FIRST</code>
        *          and <code>LAST</code>.
        *
        * @throws  IllegalStateException
        *          If no component has yet been added.
        * @throws  IllegalArgumentException
        *          If <code>iteration</code> type is <code>FIRST</code> or <code>LAST</code>, but
        *          <code>sort</code> is <code>null</code>, or if <code>sort</code> represents an
        *          invalid FQL order by expression.
        */
       public void setIterationType(int iteration, String sort)
       {
          if (components.isEmpty())
          {
             throw new IllegalStateException(
                "Must add a query component before setting its iteration type");
          }

          QueryComponent comp = components.get(components.size() - 1);

          try
          {
             comp.setIteration(iteration, sort);
          }
          catch (PersistenceException exc)
          {
             // should only happen if we have a runtime programming error or conversion error
             // (or with bad, hand-written business logic)
             throw new IllegalArgumentException(exc);
          }
       }

       /**
        * Retrieve the first composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrieve the first record sucefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean first()
       {
          return first(null);
       }

       /**
        * Retrieve the first composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @param   lockType
        *          Lock type to apply to records retrieved (overrides default
        *          lock type set for each query component).
        *
        * @return  <code>true</code> if query could retrive the first record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean first(LockType lockType)
       {
          execute();

          boolean missing = false;
          boolean avail = false;
          do
          {
             try
             {
                avail = (missing ? results.next() : results.first());
                missing = false;

                offEnd = (avail ? OffEnd.NONE : OffEnd.BACK);
                betweenRows = false;
                fetch(avail, lockType, isErrorIfNull());
             }
             catch (MissingRecordException exc)
             {
                missing = true;
             }
          } while (missing);

          return avail;
       }

       /**
        * Retrieve the last composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        *
        * @return  <code>true</code> if query could retrive the last record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean last()
       {
          return last(null);
       }

       /**
        * Retrieve the last composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @param   lockType
        *          Lock type to apply to records retrieved (overrides default
        *          lock type set for each query component).
        *
        * @return  <code>true</code> if query could retrive the last record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean last(LockType lockType)
       {
          execute();

          boolean missing = false;
          boolean avail = false;
          do
          {
             try
             {
                avail = (missing ? results.previous() : results.last());
                missing = false;

                offEnd = (avail ? OffEnd.NONE : OffEnd.BACK);
                betweenRows = false;
                fetch(avail, lockType, isErrorIfNull());
             }
             catch (MissingRecordException exc)
             {
                missing = true;
             }
          } while (missing);

          return avail;
       }

       /**
        * Retrieve the next composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrive the next record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean next()
       {
          return next(null);
       }

       /**
        * Retrieve the next composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @param   lockType
        *          Lock type to apply to records retrieved (overrides default
        *          lock type set for each query component).
        *
        * @return  <code>true</code> if query could retrive the next record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean next(LockType lockType)
       {
          execute();

          boolean missing;
          boolean avail = false;
          do
          {
             missing = false;
             try
             {
                avail = false;
                if (offEnd == OffEnd.FRONT)
                {
                   avail = results.first();
                }
                else
                {
                   avail = (betweenRows && results.getRowNumber() != -1) || results.next();
                }

                offEnd = (avail ? OffEnd.NONE : OffEnd.BACK);
                fetch(avail, lockType, false);
             }
             catch (MissingRecordException exc)
             {
                missing = true;
             }
             finally
             {
                betweenRows = false;
             }
          } while (missing);

          return avail;
       }

       /**
        * Retrieve the previous composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrive the previous record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean previous()
       {
          return previous(null);
       }

       /**
        * Retrieve the previous composite row of results for the query.  The
        * underlying buffer for each query component is updated with the
        * appropriate record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @param   lockType
        *          Lock type to apply to records retrieved (overrides default
        *          lock type set for each query component).
        *
        * @return  <code>true</code> if query could retrive the previous record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query or the fetch of any record fails, or if no records
        *          are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail
        *          in this case.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean previous(LockType lockType)
       {
          boolean setToLast = treatPreviousAsLast();
          execute();

          boolean missing;
          boolean avail = false;
          do
          {
             missing = false;
             try
             {
                avail = (setToLast ? results.last() : results.previous());
                if (avail)
                {
                   offEnd = OffEnd.NONE;
                }
                else
                {
                   offEnd = (setToLast ? OffEnd.BACK : OffEnd.FRONT);
                }
                fetch(avail, lockType, false);
             }
             catch (MissingRecordException exc)
             {
                missing = true;
             }
             finally
             {
                betweenRows = false;
             }
          } while (missing);

          return avail;
       }

       /**
        * Retrieve a unique record (only valid for a single-table query).
        *
        * @return  <code>true</code> if query could retrive the unique record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query fails or if more than one result is found which
        *          matches the query criteria.
        * @throws  UnsupportedOperationException
        *          if this method is invoked on a multi-table query.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean unique()
       {
          return unique(null);
       }

       /**
        * Retrieve a unique record (only valid for a single-table query),
        * overriding the default lock type of the underlying query component.
        *
        * @param   lockType
        *          Lock type which should override that of the query component.
        *
        * @return  <code>true</code> if query could retrive the unique record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the query fails or if more than one result is found which
        *          matches the query criteria.
        * @throws  UnsupportedOperationException
        *          if this method is invoked on a multi-table query.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean unique(LockType lockType)
       {
          if (components.size() > 1)
          {
             throw new UnsupportedOperationException(
                "Unique retrieval only valid for a single-table preselect query");
          }

          boolean setToFirst = (results == null);
          execute();
          boolean avail = true;
          if (setToFirst)
          {
             avail = results.first();
          }

          if (!results.isFirst() || !results.isLast())
          {
             try
             {
                QueryComponent comp = components.get(0);
                RecordBuffer buffer = comp.getBuffer();
                getPersistence().uniqueResultViolation(buffer, null);
             }
             catch (PersistenceException exc)
             {
                ErrorManager.recordOrThrowError(exc);

                return false;
             }
          }

          offEnd = (avail ? OffEnd.NONE : OffEnd.BACK);

          try
          {
             fetch(avail, lockType, false);
          }
          catch (MissingRecordException exc)
          {
             ErrorManager.recordOrThrowError(exc);
          }

          return avail;
       }

       /**
        * Reload the current composite row of results for the query.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrive the current record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the reload triggers an error.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean current()
       {
          return current(null);
       }

       /**
        * Reload the current composite row of results for the query, overriding
        * the lock type to apply to each record.
        * <p>
        * If the query involves outer joins, some of the underlying buffers may
        * be empty when this method returns.
        *
        * @param   lockType
        *          Lock type which should override that of any query component.
        *
        * @return  <code>true</code> if query could retrive the current record succefully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if the reload triggers an error.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       public boolean current(LockType lockType)
       {
          if (_isOffEnd() || currentRowDeleted)
          {
             ErrorManager.displayError(4114, "No query record is available", false);
             return false;
          }

          execute();

          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             LockType lt = (lockType == null ? comp.getLockType() : lockType);

             final RecordBuffer buffer = comp.getBuffer();
             buffer.armCurrentChanged();
             buffer.reload(lt, isErrorIfNull());
             buffer.updateCurrentChanged();
          }

          return true;
       }

       /**
        * Load one or more records into their associated buffers, given an array of primary key ID values, or the
        * actual DMOs themselves.  The IDs/DMOs are arranged to coincide with the tables underlying the query,
        * from left-most to right-most (in the sense of how they are joined by the query).
        * <p>
        * In the event an expected value cannot be loaded (e.g., the record has been deleted or is otherwise no
        * longer available), this method throws {@link MissingRecordException}, after setting the associated
        * buffer(s) into {@link RecordBuffer#setUnknownMode() unknown mode}. Note that a record being locked by
        * another session does not constitute a "missing" record. Note also that this exception is not thrown
        * until all the elements of {@code data} have been processed to the extent possible.
        * <p>
        * If the QueryComponent is not the first component in {@link #components()}
        * and the query attached to it is an OUTER-JOIN, we set the buffer into
        * {@link RecordBuffer#setUnknownMode() unknown mode} otherwise the buffer is released and the method
        * returns immediately.
        *
        * @param   data
        *          Array of primary key IDs or DMOs, one per table involved in the query.
        * @param   lockType
        *          Lock type which should by used.  Set to {@code null} to allow query to use its current lock
        *          type.
        * @param   silentIfNullId
        *          Do not raise {@link MissingRecordException} if some of the provided IDs are {@code null}.
        *          {@code null} IDs are valid for queries with OUTER join.
        *
        * @throws  MissingRecordException
        *          if any record cannot be loaded, because it is no longer available (e.g., deleted, etc.).
        * @throws  PersistenceException
        *          if there is an error loading data.
        */
       public void load(Object[] data, LockType lockType, boolean silentIfNullId)
       throws PersistenceException
       {
          int len = data.length;
          if (outer && len == 1 && data[0] == null)
          {
             // quick out: this is an OUTER-JOIN component positioned on a null record
             RecordBuffer buffer = components.get(0).getBuffer();
             buffer.release(true);
             // flag the buffer as 'OUTER' result of the query
             buffer.setUnknownMode();
             return;
          }

          boolean retry = (results != null);
          Long[] ids = new Long[len];
          for (int i = 0; i < len; i++)
          {
             Object datum = data[i];
             ids[i] = datum instanceof Record ? ((Record) datum).primaryKey() : (Long) datum;
          }

          try
          {
             boolean isNullId = false;
             for (int i = 0; i < len; i++)
             {
                if (ids[i] == null)
                {
                   QueryComponent comp = components.get(i);

                   if (comp.isOuter() && i > 0)
                   {
                      RecordBuffer buffer = comp.getBuffer();
                      buffer.setUnknownMode();
                   }
                   else
                   {
                      isNullId = true;
                      break;
                   }
                }
             }

             if (isNullId)
             {
                if (silentIfNullId)
                {
                   setEmptyBuffersUnknown();
                   return;
                }
                else
                {
                   throw new MissingRecordException();
                }
             }

             if (!repositionByID(ids, true))
             {
                if (retry)
                {
                   // Query did not have the target record(s) cached in its current
                   // result set, so reset the results and try again.
                   resetResults();
                   if (!repositionByID(ids, true))
                   {
                      throw new MissingRecordException();
                   }
                }
                else
                {
                   throw new MissingRecordException();
                }
             }

             fetch(offEnd == OffEnd.NONE, lockType, false);
          }
          catch (MissingRecordException exc)
          {
             setEmptyBuffersUnknown();

             throw exc;
          }
          finally
          {
             betweenRows = false;
          }
       }

       /**
        * Get all the off-end listeners associated with this query.
        *
        * @return  the off-end listeners associated with this query.
        */
       public ArrayList<QueryOffEndListener> getOffEndListeners()
       {
          ArrayList<QueryOffEndListener> listeners = new ArrayList<>(components.size());

          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent component = components.get(i);
             listeners.add(component.getBuffer().getQueryOffEndListener());
          }

          return listeners;
       }

       /**
        * Assemble an array of primary key IDs for the current record(s) in the
        * buffers underlying this query, from left-most to right-most (in the
        * sense of how the query joins the associated tables).  This method
        * essentially takes a lightweight snapshot of the currently loaded
        * record(s), so that this may be restored later.
        *
        * @return  Array of primary key IDs, one per table involved in the query.
        *
        * @see     #load(Object[], LockType, boolean)
        */
       public Object[] getRow()
       {
          return getRow(false);
       }

       /**
        * Assemble an array of primary key IDs for the current record(s) in the
        * buffers underlying this query, from left-most to right-most (in the
        * sense of how the query joins the associated tables).  This method
        * essentially takes a lightweight snapshot of the currently loaded
        * record(s), so that this may be restored later.
        *
        * @param   forceIdOnly
        *          {@code true} only if the expected data are the PKs.
        *
        * @return  Array of primary key IDs, one per table involved in the query.
        *
        * @see     #load(Object[], LockType, boolean)
        */
       public Object[] getRow(boolean forceIdOnly)
       {
          if (results == null)
          {
             return null;
          }

          // test for empty result (may happen in case of outer joins)
          if (results.getRowNumber() == -1)
          {
             return null;
          }

          Object[] srcData = results.get(forceIdOnly);
          if (srcData == null)
          {
             return null;
          }

          int size = components.size();
          Object[] rowData = new Object[size];
          System.arraycopy(srcData, 0, rowData, 0, size);

          return rowData;
       }

       /**
        * Reposition the cursor such that a request to retrieve the next result
        * will retrieve the result which matches the specified primary key ID.  A
        * request to retrieve the previous result will retrieve the result
        * immediately previous to this, if any, in the results list.
        *
        * @param   id
        *          A primary key ID, representing the target record (in the case
        *          of a join, this ID is associated with the left-most record in
        *          the join).
        *
        * @throws  ErrorConditionException
        *          if the specified ID represents the unknown value.
        */
       public void repositionByID(recid id)
       {
          if (!validateRepositionByID(id))
          {
             return;
          }

          Long[] serIDs = new Long[] { id.longValue() };

          if (!repositionByID(serIDs, false))
          {
             ErrorManager.recordOrThrowError(7331, "", "");
             // "Cannot reposition query <name> to recid/rowid(s) given".
          }
       }

       /**
        * Reposition the cursor such that a request to retrieve the next result
        * will retrieve the result which matches the specified array of primary
        * key IDs.  A request to retrieve the previous result will retrieve the
        * result immediately previous to this, if any, in the results list.
        *
        * @param   id1
        *          A primary key ID, representing the target record (in the case
        *          of a join, this ID is associated with the left-most record in
        *          the join).
        * @param   joinIDs
        *          All remaining primary key IDs, if any, arranged from left to
        *          right to coincide with the records being joined by the
        *          underlying query.
        *
        * @throws  ErrorConditionException
        *          if any of the specified IDs represents the unknown value.
        */
       public void repositionByID(rowid id1, rowid...joinIDs)
       {
          if (id1 == null || id1.isUnknown())
          {
             QueryComponent qc = components.get(components.size() - 1);
             qc.getBuffer().release(true);
             ErrorManager.recordOrThrowError(7316, "Invalid rowid argument(s) given in query object method", false, false);
             return;
          }

          int len = joinIDs.length;
          Long[] serIDs = new Long[len + 1];
          serIDs[0] = id1.getValue();

          for (int i = 0; i < len; i++)
          {
             rowid next = joinIDs[i];

             serIDs[i + 1] = next.getValue();
          }

          if (!repositionByID(serIDs, false))
          {
             ErrorManager.recordOrThrowError(7331, "", "");
             // "Cannot reposition query <name> to recid/rowid(s) given".
          }
       }

       /**
        * Reposition the cursor such that a request to retrieve the next result will retrieve the
        * result which matches the specified array of primary key IDs. A request to retrieve the
        * previous result will retrieve the result immediately previous to this, if any, in the
        * results list.
        * <strong>Note:</strong> This is not public API.
        *
        * @param   joinIDs
        *          A set of primary key IDs, arranged from left to right to coincide with the records
        *          being joined by the underlying query.
        *
        * @throws  ErrorConditionException
        *          if any of the specified IDs represents the unknown value.
        *
        * @return  {@code true} on success.
        */
       @Override
       public boolean repositionByID(Long... joinIDs)
       {
          boolean ret = repositionByID(joinIDs, false);

          if (!ret)
          {
             ErrorManager.recordOrThrowError(7331, "", "");
             // Cannot reposition query <name> to recid/rowid(s) given.
          }

          return ret;
       }

       /**
        * Reposition the cursor to the specified row in the result set.  The row
        * indicates the position between two results, such that a call to
        * retrieve the next record will get the result at the specified row,
        * and a call to retrieve the previous result will get the result before
        * the new position.
        *
        * @param   row
        *          1-based index of the target position.
        *
        * @throws  ErrorConditionException
        *          if <code>row</code> represents unknown value.
        */
       public void reposition(NumberType row)
       {
          if (validateReposition(row))
          {
             reposition(row.intValue());
          }
       }

       /**
        * Reposition the query to the specified row in the result set.  The row
        * indicates the position between two results, such that a call to
        * retrieve the next record will get the record after the new position,
        * and a call to retrieve the previous record will get the record before
        * the new position.
        *
        * @param   row
        *          1-based index of the target position.
        *
        * @throws  ErrorConditionException
        *          if there is an error repositioning the cursor.
        */
       public void reposition(int row)
       {
          prepareReposition();

          if (results == null)
          {
             execute();
          }

          row--;   // adjust for 1-based row
          int currentRow = results.getRowNumber();
          if ((row >= 0 && currentRow == row) || results.setRowNumber(row))
          {
             offEnd = OffEnd.NONE;
             betweenRows = true;
          }
          else
          {
             offEnd = (row < 0 ? OffEnd.FRONT : OffEnd.BACK);
             betweenRows = false;
          }

          afterReposition(true);
       }

       /**
        * Advance the current cursor position forward by the specified number of rows.  This will
        * always leave the cursor between two results.  For example, given a request to move 1 row
        * forward:
        * <ul>
        *   <li>if the cursor currently is positioned <i>directly on a result</i>, this would move it
        *       ahead <i>one and a half positions</i>, to just beyond the following result;
        *   <li>if the cursor currently is positioned <i>between two results</i>, this would move it
        *       ahead <i>one position</i>, to just beyond the following result.
        * </ul>
        *
        * @param   rows
        *          Number of rows to scroll the cursor forward.
        *
        * @return  {@code true} if operation was successful and the cursor was moved the exact amount
        *          of rows as specified by the parameter. If {@code false} is returned then it is
        *          possible that cursor's position was moved but not with {@code rows} rows.
        *
        * @throws  ErrorConditionException
        *          if {@code rows} represents the unknown value; if there is an error repositioning
        *          the cursor.
        */
       public boolean forward(NumberType rows)
       {
          if (validateReposition(rows))
          {
             return forward(rows.intValue());
          }
          return false;
       }

       /**
        * Advance the current cursor position forward by the specified number of rows.  This will
        * always leave the cursor between two results.  For example, given a request to move 1 row
        * forward:
        * <ul>
        *   <li>if the cursor currently is positioned <i>directly on a result</i>, this would move it
        *       ahead <i>one and a half positions</i>, to just beyond the following result;
        *   <li>if the cursor currently is positioned <i>between two results</i>, this would move it
        *       ahead <i>one position</i>, to just beyond the following result.
        * </ul>
        *
        * @param   rows
        *          Number of rows to scroll the cursor forward.
        *
        * @return  {@code true} if operation was successful and the cursor was moved the exact amount
        *          of rows as specified by the parameter. If {@code false} is returned then it is
        *          possible that cursor's position was moved but not with {@code rows} rows.
        *
        * @throws  ErrorConditionException
        *          if there is an error repositioning the cursor.
        */
       public boolean forward(int rows)
       {
          if (rows == 0)
          {
             return true;
          }

          if (rows < 0)
          {
             return backward(-rows);
          }

          prepareReposition();

          if (results == null)
          {
             execute();
          }

          // Increment the requested number of rows by one if we are:
          // 1. on a record (i.e. not between rows) OR
          // 2. into front offend,
          // since the resulting position must be before the record *following*
          // the targeted jump.
          if (!betweenRows || (results.getRowNumber() == -1 && offEnd != OffEnd.BACK))
          {
             rows++;
          }

          boolean avail = results.scroll(rows);
          if (avail)
          {
             offEnd = OffEnd.NONE;
             betweenRows = true;
          }
          else
          {
             offEnd = OffEnd.BACK;
             betweenRows = false;
          }

          afterReposition(true);
          return avail;
       }

       /**
        * Move the current cursor position backward by the specified number of rows.  This will always
        * leave the cursor between two results.  For example, given a request to move 1 row backward:
        * <ul>
        *   <li>if the cursor currently is positioned <i>directly on a result</i>, this would move it
        *       back <i>one half position</i>, to just before the current result;
        *   <li>if the cursor currently is positioned <i>between two results</i>, this would move it
        *       back <i>one position</i>, to just before the current result.
        * </ul>
        *
        * @param   rows
        *          Number of rows to scroll the cursor backward.
        *
        * @return  {@code true} if operation was successful and the cursor was moved the exact amount
        *          of rows as specified by the parameter. If {@code false} is returned then it is
        *          possible that cursor's position was moved but not with {@code rows} rows.
        *
        * @throws  ErrorConditionException
        *          if {@code rows} represents the unknown value; if there is an error repositioning
        *          the cursor.
        */
       public boolean backward(NumberType rows)
       {
          if (validateReposition(rows))
          {
             return backward(rows.intValue());
          }
          return false;
       }

       /**
        * Move the current cursor position backward by the specified number of rows.  This will always
        * leave the cursor between two results.  For example, given a request to move 1 row backward:
        * <ul>
        *   <li>if the cursor currently is positioned <i>directly on a result</i>, this would move it
        *       back <i>one half position</i>, to just before the current result;
        *   <li>if the cursor currently is positioned <i>between two results</i>, this would move it
        *       back <i>one position</i>, to just before the current result.
        * </ul>
        *
        * @param   rows
        *          Number of rows to scroll the cursor backward.
        *
        * @return  {@code true} if operation was successful and the cursor was moved the exact amount
        *          of rows as specified by the parameter. If {@code false} is returned then it is
        *          possible that cursor's position was moved but not with {@code rows} rows.
        *
        * @throws  ErrorConditionException
        *          if there is an error repositioning the cursor.
        */
       public boolean backward(int rows)
       {
          if (rows == 0)
          {
             return true;
          }

          if (rows < 0)
          {
             return forward(-rows);
          }

          prepareReposition();

          if (results == null)
          {
             execute();
          }

          // Decrement the requested number of rows by one if we are:
          // 1. on a record (i.e. not between rows) AND
          // 2. NOT into back offend,
          // since the resulting position must be before the record *following*
          // the targeted jump.
          if (!betweenRows && offEnd != OffEnd.BACK)
          {
             rows--;
          }

          boolean avail = results.scroll(-rows);
          if (avail)
          {
             offEnd = OffEnd.NONE;
             betweenRows = true;
          }
          else
          {
             offEnd = OffEnd.FRONT;
             betweenRows = false;
          }

          afterReposition(true);
          return avail;
       }

       /**
        * Return the number of results found for this query.
        * <p>
        * Note that the first time this method is invoked, the database server may have to scroll to
        * the end of the result set, which may be a time-consuming process for a large result set.
        *
        * @return  Size of result set or value if the query is not open.
        */
       public integer size()
       {
          if (results == null)
          {
             return new integer();
          }

          if (rowCount < 0)
          {
             int row = results.getRowNumber();

             // if results.last() returns false, it means there are no results; otherwise add 1 to
             // the 0-based row number of the last result
             rowCount = results.last() ? results.getRowNumber() + 1 : 0;

             if (row == -1)
             {
                if (offEnd == OffEnd.BACK)
                {
                   results.next();
                }
                else
                {
                   results.reset();
                }
             }
             else
             {
                results.setRowNumber(row);
             }
          }

          return (new integer(rowCount));
       }

       /**
        * Progress-compatible function that returns the 1-based index of the
        * current row in the result set.
        *
        * @return  Unknown value if the query is not open or result set is empty.
        *          1 if the query is positioned before the first record.
        *          A value 1 greater than the number of rows in the query result
        *          set if the query is positioned beyond the last record.
        *          Otherwise returns the number of rows in the query result set.
        *
        */
       public integer currentRow()
       {
          if (results == null ||
              (offEnd == OffEnd.NONE && results.getRowNumber() == -1 && !isScrolling()))
          {
             return (new integer());
          }
          else
          {
             int size = size().intValue();
             int row = results.getRowNumber();

             if (size == 0)
             {
                return (new integer());
             }
             else if (offEnd == OffEnd.BACK)
             {
                return (new integer(size + 1));
             }
             else if (row == -1)
             {
                // We are at the front offend. Return unknown value if the query
                // is an adaptive one and we haven't retrieved any results using
                // it.
                return !isNativelyPreselect() && (offEnd != OffEnd.FRONT)
                       ? new integer()
                       : new integer(1);
             }
             else
             {
                return (new integer(row + 1));
             }
          }
       }

       /**
        * Get the 1-based index of the current row in the result set.
        *
        * @return  Current row or the unknown value if the query has not yet
        *          executed.
        */
       public integer currentRowImpl()
       {
          if (results == null || offEnd != OffEnd.NONE)
          {
             return (new integer());
          }

          return (new integer(results.getRowNumber() + 1));
       }

       /**
        * Indicate whether the query has run off either end of its result set,
        * due to scrolling beyond the last record or before the first record.
        *
        * @return  <code>true</code> if no record can be retrieved at the
        *          cursor's current position;  otherwise <code>false</code>.
        */
       public boolean _isOffEnd()
       {
          boolean isOffEnd = (offEnd != OffEnd.NONE);
          if (!isOffEnd && results == null)
          {
             return false;
          }

          return isOffEnd;
       }

       /**
        * Indicate whether the query will retrieve full records, rather than primary keys only.
        *
        * @return  <code>true</code> to return full records; <code>false</code> to return primary
        *          keys only.
        */
       public boolean isFullRecords()
       {
          return fullRecords;
       }

       /**
        * Force the query to retrieve full records, rather than primary keys only.
        */
       public void setFullRecords()
       {
          this.fullRecords = true;
       }

       /**
        * Reset the query by clearing its state to a known default.  This is
        * used to ensure that subsequent requests for FIRST and LAST record
        * retrievals operate on a known query/buffer state.
        * <p>
        * This implementation nulls out the current result set, so that the next
        * record retrieval request triggers a new server-side query.  Optionally,
        * it also resets resolvable substitution arguments and triggers where
        * expression arguments to be resolved, if a client-side where cause
        * expression is defined.
        *
        * @param   resolveArgs
        *          <code>true</code> if query substitution arguments should be
        *          resolved/reset as part of the reset;  <code>false</code> if
        *          existing argument values should be used for the next query
        *          execution.
        * @param   forceOrigArgs
        *          Flag indicating that the original arguments used at the construction of the query must be
        *          re-calculated.
        */
       @Override
       public void reset(boolean resolveArgs, boolean forceOrigArgs)
       {
          resetResults();

          if (resolveArgs && (resolvableArgs || forceOrigArgs))
          {
             for (int i = 0; i < components.size(); i++)
             {
                QueryComponent comp = components.get(i);
                comp.resetArgs(forceOrigArgs);
             }
          }
       }

       /**
        * Set each buffer backing a query component to unknown mode.
        *
        * @see     RecordBuffer#setUnknownMode
        */
       public void setUnknownRecord()
       {
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             comp.getBuffer().setUnknownMode();
          }
       }

       /**
        * Get the number of tables joined by this query.
        *
        * @return  Number of joined tables.
        */
       public int getTableCount()
       {
          return components == null ? 0 : components.size();
       }

       /**
        * Fetch the array of primary key IDs associated with the first result in
        * this query's result list.
        *
        * @return  Array of record IDs reflecting the first query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekFirst()
       {
          execute();
          results.first();

          return getRow();
       }

       /**
        * Fetch the array of primary key IDs associated with the last result in
        * this query's result list.
        *
        * @return  Array of record IDs reflecting the last query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekLast()
       {
          execute();
          results.last();

          return getRow();
       }

       /**
        * Fetch the array of primary key IDs associated with the next result in
        * this query's result list.
        *
        * @return  Array of record IDs reflecting the next query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekNext()
       {
          execute();
          if (results.next())
          {
             return getRow();
          }
          else
          {
             return null;
          }
       }

       /**
        * Fetch the array of primary key IDs associated with the previous result
        * in this query's result list.
        *
        * @return  Array of record IDs reflecting the previous query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekPrevious()
       {
          execute();
          if (results.previous())
          {
             return getRow();
          }
          else
          {
             return null;
          }
       }

       /**
        * Indicate whether this query preselects all of its results.
        *
        * @return  <code>true</code>.
        */
       public boolean isPreselect()
       {
          return true;
       }

       /**
        * Set the given results object into this query.
        *
        * @param   results
        *          New result set for this query.
        */
       public void setResults(Results results)
       {
          if (this.results != null)
          {
             // ensure the previous results are always cleaned up.
             this.results.cleanup();
          }
          this.results = results;
       }

       /**
        * Access the record buffers manipulated by this query, in order from
        * left-most (outermost) table of the query's join (if any) to right-most
        * (innermost) table.
        *
        * @return  Array of one or more record buffers.
        */
       @Override
       public RecordBuffer[] getRecordBuffers()
       {
          RecordBuffer[] buffers = new RecordBuffer[getTableCount()];
          ArrayList<QueryComponent> components = components();
          if (components == null)
          {
             return buffers;
          }
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             buffers[i] = comp.getBuffer();
          }

          return buffers;
       }

       /**
        * Get the query substitution arguments associated with this query, if any.
        *
        * @return  Substitution arguments, in order, or <code>null</code> if there are none.
        */
       @Override
       public Object[] getArgs()
       {
          return components.get(0).getRawArgs();
       }

       /**
        * Get the original, FQL where clause associated with this query.
        *
        * @return  Original where clause, or <code>null</code> if there is none.
        */
       @Override
       public String getOriginalWhere()
       {
          return components.get(0).getOriginalWhere();
       }

       /**
        * Get the legacy natural join, if any, associated with this query.
        *
        * @return  Legacy join object, or <code>null</code> if none.
        */
       @Override
       public AbstractJoin getJoin()
       {
          return components.get(0).getJoin();
       }

       /**
        * Force this query to operate in dynamic retrieval mode, if it supports this mode.
        *
        * @throws  UnsupportedOperationException
        *          always; this query type does not support dynamic retrieval.
        */
       @Override
       public void forceDynamicOperation()
       {
          throw new UnsupportedOperationException("PreselectQuery has no dynamic operation mode!");
       }

       /**
        * Get the sort phrase associated with this query.
        *
        * @return  Sort phrase.
        */
       public String getSortPhrase()
       {
          return sort;
       }

       /**
        * Create an adaptive query component based on the information in this query (which is
        * presumed to be a single-table query), and given a list of adaptive query components which
        * represent outer, nested query loops, which will perform a server-side join to the
        * innermost, nested query component.
        *
        * @param   joinList
        *          List of adaptive components representing nested query loops which will contain
        *          the adaptive query component returned by this method. The last component in the
        *          list represents the nested query loop immediately containing the returned query
        *          component, with which the join will be made.
        * @param   query
        *          Adaptive query which will manage the server-side join of <code>joinList</code> and
        *          the returned query component.
        * @param   iteration
        *          Iteration type: FIRST, LAST, or NEXT.
        * @param   outer
        *          <code>true</code> if the join should be a left outer join. This type of join is
        *          not supported at the time of this writing, so this parameter should always be
        *          <code>false</code>. It is here to support a planned, future enhancement.
        * @param   fallback
        *          A compound query component to use in the event the multi-table adaptive query
        *          switches from preselect to dynamic retrieval mode.
        *
        * @return  An adaptive query component suitable for a server-side join, based on the
        *          information in this query.
        */
       @Override
       public AdaptiveComponent makeAdaptiveServerJoinComponent(List<AdaptiveComponent> joinList,
                                                                AdaptiveQuery query,
                                                                int iteration,
                                                                boolean outer,
                                                                CompoundComponent fallback)
       {
          QueryComponent original = components.get(0);
          original.processDynamicFilters();
          RecordBuffer buffer = original.getBuffer();
          AbstractJoin join = original.getJoin();
          Object[] args = original.getRawArgs();
          String where = original.getOriginalWhere();
          String indexInfo = original.getIndexInfo();
          LockType lockType = original.getLockType();

          QueryComponent.ServerJoinData sjd =
             QueryComponent.prepareServerJoinData(joinList, buffer, join, args);
          BufferReference inverse = (join != null ? join.getInverse() : null);
          AdaptiveComponent comp = new AdaptiveComponent(query,
                                                         buffer,
                                                         sjd.join,
                                                         where,
                                                         null,
                                                         sort,
                                                         indexInfo,
                                                         lockType,
                                                         sjd.args,
                                                         iteration,
                                                         outer,
                                                         inverse);
          comp.setReferenceSubs(sjd.referenceSubs);

          // gather all related buffers, including [relatedBuffers] from [sjd] and eventual external
          // buffers from original query component
          Set<RecordBuffer> relatedBuffers = new HashSet<>();
          if (sjd.relatedBuffers != null)
          {
             relatedBuffers.addAll(sjd.relatedBuffers);
          }
          RecordBuffer[] externalBuffers = fallback.getQuery().getExternalBuffers();
          if (externalBuffers != null && externalBuffers.length != 0)
          {
             relatedBuffers.addAll(Arrays.asList(externalBuffers));
          }
          if (!relatedBuffers.isEmpty())
          {
             comp.setRelatedBuffers(relatedBuffers);
          }

          comp.setFallback(fallback);

          if (iteration == FIRST || iteration == LAST)
          {
             try
             {
                comp.setIteration(iteration, sort);
             }
             catch (PersistenceException exc)
             {
                // should only happen if we have a runtime programming error or conversion error
                // (or with bad, hand-written business logic)
                throw new IllegalArgumentException(exc);
             }
          }

          return comp;
       }

       /**
        * Create a preselect query component based on the information in this query (which is
        * presumed to be a single-table query), and given a list of preselect query components which
        * represent outer, nested query loops, which will perform a server-side join to the
        * innermost, nested query component.
        *
        * @param   joinList
        *          List of preselect components representing nested query loops which will contain
        *          the preselect query component returned by this method. The last component in the
        *          list represents the nested query loop immediately containing the returned query
        *          component, with which the join will be made.
        * @param   iteration
        *          Iteration type: FIRST, LAST, or NEXT.
        * @param   outer
        *          <code>true</code> if the join should be a left outer join. This type of join is
        *          not supported at the time of this writing, so this parameter should always be
        *          <code>false</code>. It is here to support a planned, future enhancement.
        *
        * @return  A preselect query component suitable for a server-side join, based on the
        *          information in this query.
        */
       @Override
       public QueryComponent makePreselectServerJoinComponent(List<QueryComponent> joinList,
                                                              int iteration,
                                                              boolean outer)
       {
          QueryComponent original = components.get(0);
          original.processDynamicFilters();
          RecordBuffer buffer = original.getBuffer();
          AbstractJoin join = original.getJoin();
          Object[] args = original.getRawArgs();
          String where = original.getOriginalWhere();
          String indexInfo = original.getIndexInfo();
          LockType lockType = original.getLockType();

          QueryComponent.ServerJoinData sjd =
             QueryComponent.prepareServerJoinData(joinList, buffer, join, args);
          QueryComponent comp = new QueryComponent(this,
                                                   buffer,
                                                   sjd.join,
                                                   where,
                                                   null,
                                                   indexInfo,
                                                   lockType,
                                                   sjd.args,
                                                   iteration,
                                                   outer);
          comp.setReferenceSubs(sjd.referenceSubs);
          comp.setRelatedBuffers(sjd.relatedBuffers);

          if (iteration == FIRST || iteration == LAST)
          {
             try
             {
                comp.setIteration(iteration, sort);
             }
             catch (PersistenceException exc)
             {
                // should only happen if we have a runtime programming error or conversion error
                // (or with bad, hand-written business logic)
                throw new IllegalArgumentException(exc);
             }
          }

          return comp;
       }

       /**
        * Indicate whether this query has a client-side where clause expression associated with it.
        *
        * @return  <code>true</code> if there is a where expression; else <code>false</code>.
        */
       @Override
       public boolean hasWhereExpression()
       {
          return whereExprList != null && !whereExprList.isEmpty();
       }

       /**
        * Clean up query resources when this query's scope ends. This implementation cleans up any
        * open results and deregisters this session listener.
        *
        * @see  P2JQuery#cleanup()
        */
       public void cleanup()
       {
          if (closed)
          {
             return;
          }

          forceOnlyId = false;
          closed = true;
          if (results != null)
          {
             results.cleanup();
             results = null;
          }

          if (regWithSession)
          {
             regWithSession = false;
             persistence.deregisterSessionListener(this);
          }

          if (unregisterOnCleanup)
          {
             ChangeBroker broker = ChangeBroker.get();
             broker.removeFromGlobal(this);

             unregisterOnCleanup = false;
          }
       }

       /**
        * Open the prepared query and reposition it to the first result if the query is being browsed.
        */
       public void open()
       {
          if (!closed)
          {
             close();
          }

          resetResults();

          setErrorIfNull(false);
          setLenientOffEnd(true);
          execute();

          if (shouldRegisterRecordChangeListeners())
          {
             int scope;
             if (isDynamicPredicate())
             {
                scope = 0;
             }
             else
             {
                scope = BufferManager.get().findNearestExternal();
                RecordBuffer[] buffers = getRecordBuffers();

                for (int i = 0; i < buffers.length; i++)
                {
                   RecordBuffer buffer = buffers[i];
                   scope = ((scope == -1)
                      ? buffer.getScopeOpenDepth()
                      : Math.min(scope, buffer.getScopeOpenDepth()));
                }
             }

             if (scope != -1)
             {
                registerRecordChangeListeners(Integer.valueOf(scope));
             }
          }

          super.open();
       }

       /**
        * Explicitly close the query.
        * <p>
        * This will remove any registered finalizable.
        */
       @Override
       public void close()
       {
          close(false);
       }

       /**
        * Explicitly close the prepared query.
        *
        * @param    inResource
        *           Flag indicating this query is used in a QUERY legacy resource.
        */
       @Override
       public void close(boolean inResource)
       {
          super.close();

          if (thisFinalizable)
          {
             TransactionManager.removeFinalizable(this);
             thisFinalizable = false;
          }

          if (regCleaner)
          {
             TransactionManager.removeFinalizable(cleaner);
             regCleaner = false;
          }

          if (inResource && !unregisterOnCleanup)
          {
             // remove it explicitly
             ChangeBroker broker = ChangeBroker.get();
             broker.removeListener(this);
          }
       }

       /**
        * Override parent's no-op implementation to track whether client code has
        * set this as a scrollable query.  While this setting does not explicitly
        * affect this query's ability to respond to reposition requests (this
        * implementation is inherently scrollable), it does provide a hint to the
        * implementation regarding the use and scope of this query.  This method
        * must be invoked before any results are retrieved.
        */
       public void setScrolling()
       {
          scrolling = true;
       }

       /**
        * Disable results list scrolling/repositioning (if it was enabled by
        * default). This method must be invoked before any results are retrieved.
        */
       public void setNonScrolling()
       {
          scrolling = false;
       }

       /**
        * Get the off-end status of this query, which indicates whether the query
        * has run off either end of its results.
        *
        * @return  One of the {@link OffEnd} constants <code>NONE</code>,
        *          <code>FRONT</code>, or <code>BACK</code>.
        */
       public OffEnd getOffEnd()
       {
          return offEnd;
       }

       /**
        * Release all buffers. The position of the query is not affected.
        */
       public void releaseBuffers()
       {
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent c = components.get(i);
             c.getBuffer().release(false);
          }
       }

       /**
        * Adds a new constraint term to the where predicate of this query, effectively filtering out
        * any rows whose referenced field does not match the requested value. Semantically, the query
        * will select rows that match
        * <p>
        * {@code new-where := (fr = val) AND (old-query)}.
        * <p>
        * Note: The constraints are exclusive meaning that if constraints will be added for same
        * field of same buffer, the query will return an empty result because the field cannot be at
        * the same time equals to two different values.
        * <p>
        * Use {@link #clearDynamicFilters} to clean all dynamically added criteria and restore the
        * sorting order of the query to its original form.
        *
        * @param   fr
        *          A reference to field used as filtering criterion. Only the rows that matches the
        *          specified value for this field will be selected. Must not be {@code null}.
        * @param   val
        *          The filtering value for this constraint. Can be {@code null} or {@code unknown}, in
        *          which case the SQL {@code null} value will be assumed. In this case the predicate
        *          will look like:
        *           <p>
        *          {@code new-where := (fr IS  NULL) AND (old-query)}
        * @param   format
        *          The format to be used when comparing values. The reason for this parameter is that
        *          for some datatype (ex: {@code decimal}) the on-screen value can be different from
        *          the database (because of rounding).
        *
        * @return  {@code true} if the filtering constraint was successfully added. If the field
        *          reference is not related to the buffer(s) of this query, {@code false} is returned.
        */
       @Override
       public boolean addDynamicFilter(FieldReference fr, BaseDataType val, String format)
       {
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             if (fr.getParentBuffer() == comp.getBuffer())
             {
                comp.addDynamicFilter(fr, val, format);
                return true;
             }
          }

          return false;
       }

       /**
        * Clears any dynamically filters added at runtime. The query predicate will be restored to its
        * form from generated at conversion time.
        */
       @Override
       public void clearDynamicFilters()
       {
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             comp.clearDynamicFilters();
          }
       }

       /**
        * Adds a new sorting criterion as the strongest criterion for sorting the result of this
        * query. The other criteria remain queued, but with lower priority.
        * <p>
        * Use {@link #clearDynamicSortCriteria} to clean all dynamically added criteria and restore the
        * sorting order of the query to its original form.
        *
        * @param   fr
        *          A reference to field used as new strongest criterion. The current criteria will
        *          be kept, but with lower priority.
        * @param   asc
        *          The direction of sorting for this criterion. Use {@code true} for ascending sort
        *          and {@code false} for descending sorting.
        */
       void addDynamicSortCriterion(FieldReference fr, boolean asc)
       {
          if (fr == null)
          {
             return; // no-op
          }
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             if (fr.getParentBuffer() == comp.getBuffer())
             {
                if (dynamicSorts == null)
                {
                   dynamicSorts = new ArrayList<>();
                }
                dynamicSorts.add(0, fr + (asc ? " asc" : " desc"));
             }
          }
       }

       /**
        * Clears any dynamically sort criteria added at runtime. The query sort order will be restored
        * to its form from generated at conversion time.
        */
       void clearDynamicSortCriteria()
       {
          dynamicSorts = null;
       }

       /**
        * Conversion of INDEX-INFORMATION attribute (KW_IDX_INFO).
        * <p>
        * Getter of INDEX-INFORMATION attribute
        *
        * @param    n
        *           An integer expression that evaluates to the level of join for which you want index
        *           information.
        *
        * @return   A character string consisting of a comma-separated list of the index or indexes
        *           the query uses at the level of join specified.
        */
       @LegacyAttribute(name = "INDEX-INFORMATION")
       public character indexInformation(NumberType n)
       {
          return new character(components.get(n.intValue() - 1).getIndexInfo());
       }

       /**
        * Indicate whether this query is a preselect query by its nature.
        *
        * @return  <code>true</code>.
        */
       public boolean isNativelyPreselect()
       {
          return true;
       }

       /**
        * Creates an entry in the result list for the current row. Implements the 4GL
        * CREATE-RESULT-LIST-ENTRY() method. Created entry contains id(s) of the record(s) currently
        * located in the backing buffer(s).
        *
        * @return <code>true</code> on success.
        */
       public logical createResultListEntry()
       {
          Results cached = cacheResults(results, false);
          if (results != cached)
          {
             setResults(cached);
          }

          Object[] row = new Object[getTableCount()];
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             RecordBuffer buffer = comp.getBuffer();
             Record dmo = buffer.getCurrentRecord();
             row[i] = dmo != null
                   ? (isFullRecords() ? dmo : dmo.primaryKey())
                   : null;
          }

          results.addRow(row, betweenRows);
          offEnd = isBrowsed() ? OffEnd.NONE : offEnd;
          betweenRows = false;
          rowCount++;
          return new logical(true);
       }

       /**
        * Deletes the current row of a query's result list. Implements the 4GL
        * DELETE-RESULT-LIST-ENTRY() method.
        *
        * @param  allowBetweenRows
        *         If <code>true</code> then the cursor can be positioned between rows (the next
        *         row will be deleted). <code>false</code> for conventional DELETE-RESULT-LIST-ENTRY()
        *         mode where the cursor should be positioned on a row (otherwise <code>false</code> is
        *         returned).
        *
        * @return <code>true</code> on success.
        */
       @Override
       public logical deleteResultListEntry(boolean allowBetweenRows)
       {
          // In allowBetweenRows mode we may want to delete multiple records without fetching them,
          // so currentRowDeleted state is ignored.
          if (currentRowDeleted && !allowBetweenRows)
          {
             return new logical(false);
          }

          Results cached = cacheResults(results, false);

          if (results != cached)
          {
             setResults(cached);
          }

          if (!_isOffEnd() &&
              cached.getNumberOfLoadedRows() > 0 &&
              (allowBetweenRows || !betweenRows))
          {
             int row = cached.getRowNumber();
             if (row >= 0)
             {
                int width = getTableCount();
                Long[] rsIDs = new Long[width];
                for (int i = 0; i < width; i++)
                {
                   rsIDs[i] = cached.getID(i);
                }

                repoCache.remove(rsIDs);
             }
             currentRowDeleted = true;
             cached.deleteRow();
             betweenRows = true;
             rowCount--;
             return new logical(true);
          }

          return new logical(false);
       }

       /**
        * Indicate whether this query's <code>scrolling</code> flag has been set.
        *
        * @return  <code>true</code> if this is a scrolling query;  else
        *          <code>false</code>.
        */
       public boolean isScrolling()
       {
          return (scrolling && !isForwardOnly()) || isNativelyScrolling();
       }

       /**
        * Indicate whether this query type is scrolling by its nature.
        * <p>
        * Subclasses which are not natively scrolling should override this method.
        *
        * @return  <code>true</code>.
        */
       protected boolean isNativelyScrolling()
       {
          return true;
       }

       /**
        * Add one or more buffers to the list of those which this query references externally. These typically
        * will be buffers referenced by converted CAN-FIND expressions embedded in the query's FQL expression,
        * which are not the primary buffers associated with the query.
        * <p>
        * If the bound buffer differs from the definition buffer (see {@link #buffersMatch}) for any of the
        * external buffers, then all {@link #components()} will have the {@link #translateWhere()} enabled
        * (if not already enabled).
        *
        * @param   dmos
        *          One or more external buffers.
        *
        * @return  This query instance.
        */
       @Override
       public P2JQuery addExternalBuffers(DataModelObject... dmos)
       {
          super.addExternalBuffers(dmos);

          RecordBuffer[] buffers = getExternalBuffers();
          if (buffers == null)
          {
             return this;
          }

          RecordBuffer[] defs = getExternalBuffersDefs();
          boolean translate = false;
          for (int i = 0; i < buffers.length && !translate; i++)
          {
             translate = !buffersMatch(buffers[i], defs[i]);
          }

          if (translate)
          {
             Function<String, String> translateWhere = (where) ->
             {
                // just create two empty lists where the binding from the external buffers will be collected.
                List<RecordBuffer> binding = new ArrayList<>();
                List<RecordBuffer> definition = new ArrayList<>();
                return translateWhere(binding, definition, where);
             };

             ArrayList<QueryComponent> components = components();
             for (int i = 0; i < components.size(); i++)
             {
                QueryComponent qc = components.get(i);
                qc.enableTranslateWhere(translateWhere);
             }
          }

          return this;
       }

       /**
        * Adds an alternative execution query, for the multi-tenant case in which the CAN-FIND nested buffer and
        * the main buffer of query component do not belong to same physical database.
        * <p>
        * This method will be generated only in case of multi-tenant projects, in the condition above-mentioned.
        *
        * @param   where
        *          FQL where clause.  May be {@code null}.
        * @param   whereExpr
        *          Client-side where clause expression. May be {@code null} but this is probably caused by an
        *          incorrect conversion.
        * @param   sort
        *          The sort order.
        *
        * @return  Self object so that the call to this method can be optionally chained with the constructor.
        */
       @Override
       public PreselectQuery setMultiTenantAlternative(String where, Supplier<logical> whereExpr, String sort)
       {
          return setMultiTenantAlternative(where, whereExpr, sort, null);
       }

       /**
        * Adds an alternative execution query, for the multi-tenant case in which the CAN-FIND nested buffer and
        * the main buffer of query component do not belong to same physical database.
        * <p>
        * This method will be generated only in case of multi-tenant projects, in the condition above-mentioned.
        *
        * @param   where
        *          FQL where clause.  May be {@code null}.
        * @param   whereExpr
        *          Client-side where clause expression. May be {@code null} but this is probably caused by an
        *          incorrect conversion.
        * @param   sort
        *          The sort order.
        * @param   args
        *          Substitution parameters for FQL queries. These will be used if not overridden by a record
        *          retrieval method.
        *
        * @return  Self object so that the call to this method can be optionally chained with the constructor.
        */
       @Override
       public PreselectQuery setMultiTenantAlternative(String where,
                                                       Supplier<logical> whereExpr,
                                                       String sort,
                                                       Object[] args)
       {
          RecordBuffer[] buffers = getExternalBuffers();
          if (buffers == null || buffers.length == 0)
          {
             return this; // no-op. This should not happen
          }

          RecordBuffer buffer = components.get(0).getBuffer();
          Persistence.Context privateContext = buffer.getPersistence().getContext(Persistence.PRIVATE_CTX);
          if (privateContext.getTenantId() == TenantManager.DEFAULT_TENANT_ID)
          {
             // running as default tenant, the query is intra-database (using the default database)
             return this;
          }

          boolean isMt = buffer.getDmoInfo().multiTenant;
          for (RecordBuffer eb : buffers)
          {
             if (eb.getDmoInfo().multiTenant != isMt)
             {
                // if the query is cross-physical database, re-initialize with the alternative execution path
                return initialize(buffer, where, whereExpr, sort, null, null, args, null);
             }
          }

          // if we reach this point, all table are either public or all private, so the optimised query can be
          // safely executed
          return  this;
       }

       /**
        * Discard the current result set, clear the reposition cache, and reset
        * result-related state to default values.
        */
       protected void resetResults()
       {
          if (results != null)
          {
             results.cleanup();
          }
          setResults(null);
          offEnd = OffEnd.NONE;
          betweenRows = true;
          rowCount = -1;
          clearRepoCache();
          currentRowDeleted = false;
       }

       /**
        * Get the query components.
        *
        * @return  Query components.
        */
       protected ArrayList<QueryComponent> components()
       {
          return components;
       }

       /**
        * Indicate whether this query will throw a
        * <code>QueryOffEndException</code> when it runs off the end of its
        * results.
        *
        * @return  <code>true</code> if an exception will be thrown, else
        *          <code>false</code>.
        */
       protected boolean isLenientOffEnd()
       {
          return lenientOffEnd;
       }

       /**
        * Indicate whether a call to move to the previous record in the result
        * list should be treated as a call to move to the last record in the list.
        * This will be the case if either there are no results available, or if
        * the query is not off the front end of its results, but the row number
        * is -1 (indicating no results have yet been retrieved).
        *
        * @return  <code>true</code> to treat a previous request as a request to
        *          move to the last record in the list;  else <code>false</code>.
        */
       protected boolean treatPreviousAsLast()
       {
          return (results == null || (offEnd != OffEnd.FRONT && results.getRowNumber() == -1));
       }

       /**
        * Walk through each component and set its backing buffer to unknown mode,
        * such that the invocation of any of its DMO-backed methods returns
        * unknown value.
        */
       protected void setEmptyBuffersUnknown()
       {
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             RecordBuffer buffer = comp.getBuffer();
             if (buffer.getCurrentRecord() == null)
             {
                buffer.setUnknownMode();
             }
          }
       }

       /**
        * Given a primary DMO buffer and an inverse DMO buffer which together
        * define a natural join, create the <code>AbstractJoin</code> helper
        * object which is used to implement the join.
        * <p>
        * It is intended that this method be called when adding a query component.
        * If no previous components have been added, a <code>DynamicJoin</code>
        * is created, and it is assumed that the inverse DMO is provided as a
        * query substitution parameter.  If a component previously has been added,
        * a <code>ServerJoin</code> is created, and it is assumed that the inverse
        * DMO will be resolved by a server-side join, rather than being provided
        * as a query substitution parameter.
        * <p>
        * If no inverse buffer is provided, no join is created.
        *
        * @param   primary
        *          Buffer reference to the primary DMO.
        * @param   inverse
        *          Buffer reference to the joined DMO, or <code>null</code> if no
        *          join is required.
        *
        * @return  A join helper object, or <code>null</code> if no natural join
        *          is required.
        */
       protected AbstractJoin processJoin(BufferReference primary, BufferReference inverse)
       {
          if (inverse == null)
          {
             return null;
          }

          AbstractJoin join = null;

          if (getTableCount() == 0)
          {
             // If we have an inverse proxy and this is the first component being
             // added, the inverse is being provided as a query substitution
             // parameter.
             join = Persistence.isForeignKeysEnabled()
                    ? new DynamicJoin(primary.buffer(), inverse.buffer())
                    : new DynamicLegacyKeyJoin(primary.buffer(), inverse.buffer());
          }
          else
          {
             // If we have an inverse proxy and this is not the first component
             // being added, the inverse is provided as a related DMO from a
             // related table.  In this case, the inverse was not provided as a
             // query substitution parameter.
             join = Persistence.isForeignKeysEnabled()
                    ? new ServerJoin(primary.buffer(), inverse.buffer())
                    : new ServerLegacyKeyJoin(primary.buffer(), inverse.buffer());
          }

          return join;
       }

       /**
        * Assemble the select clause for an FQL projection query, where the only
        * field retrieved for each buffer is the primary key. This uses the
        * Hibernate reserved keyword "id" to represent the primary key. Form of
        * the clause is:
        * <pre>
        *    select {buffer0.id} [, {buffer1.id} [...]]
        * </pre>
        *
        * @param   buf
        *          String buffer into which clause is assembled.
        */
       protected void assembleSelectClause(StringBuilder buf)
       {
          boolean needSelect = !fullRecords;

          ArrayList<QueryComponent> components = components();
          for (int i = 0; !needSelect && i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             if (!isClientWhere() && !fullRecords && comp.isIdOnly())
             {
                needSelect = true;
             }
          }

          if (!needSelect)
          {
             return;
          }

          buf.append("select ");

          components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             if (i > 0)
             {
                buf.append(", ");
             }
             assembleComponent(buf, comp, false);
          }

          buf.append(" ");
       }

       /**
        * Assemble a component that retrieves the full record or only primary key.
        * @param   buf
        *          String buffer into which clause is assembled.
        * @param   component
        *          The query component that needs to be assembled.
        * @param   forceOnlyPk
        *          Flag indicating if only the primary key should be fetched.
        */
       protected void assembleComponent(StringBuilder buf, QueryComponent component, boolean forceOnlyPk)
       {
          boolean qualified = !isOmitAlias();

          RecordBuffer buffer = component.getBuffer();

          if (qualified)
          {
             buf.append(buffer.getDMOAlias());
          }

          if (!isClientWhere() && (component.isIdOnly() && !fullRecords || forceOnlyPk))
          {
             if (qualified)
             {
                buf.append('.');
             }

             buf.append(DatabaseManager.PRIMARY_KEY);
          }
          else
          {
             Property[] iprops = included == null ? null : included.get(buffer.getDMOProxy());
             Property[] xprops = excluded == null ? null : excluded.get(buffer.getDMOProxy());
             if (iprops != null || xprops != null)
             {
                DmoMeta meta = buffer.getDmoInfo();
                int size = meta.getFieldSize();
                Property[] props = new Property[size];

                // only permanent buffers can use include/exclude
                if (iprops == null)
                {
                   // include all fields
                   Iterator<Property> piter = meta.getFields(false);
                   while (piter.hasNext())
                   {
                      Property p = piter.next();

                      if (p.propId > 0)
                      {
                         props[p.propId - 1] = p;
                      }
                   }
                }
                else
                {
                   System.arraycopy(iprops, 0, props, 0, size);
                }

                if (xprops != null)
                {
                   // exclude properties
                   for (int j = 0; j < size; j++)
                   {
                      if (xprops[j] != null) props[j] = null;
                   }
                }

                String alias = buffer.getDMOAlias();
                buf.append('.').append(DatabaseManager.PRIMARY_KEY);
                for (int j = 0; j < size; j++)
                {
                   Property p = props[j];
                   // extents are handled like scalars. The FQL converter will filter them out if necessary,
                   // and the SQLQuery will fetch them from secondary tables at hydration time
                   if (p == null)
                   {
                      continue;
                   }

                   buf.append(", ").append(alias).append(".").append(p.name);
                }
             }
          }
       }

       /**
        * Assemble the from and the where clause for an FQL query. Now this method is capable of creating
        * the OUTER-JOIN clause as well. The class name of each DMO is decapitalized to produce
        * an alias for that object. Form of the clause is:
        * <pre>
        *    from {Buffer0} as {buffer0}
        *    [[left outer] join {Buffer1} as {buffer1} on {JoinCondition1} [and {JoinCondition1_1}] [...]
        *    [, {Buffer2} as buffer2 [...]]
        *    [[left outer] join {Buffer3} as {buffer3} on {JoinCondition2} [and {JoinCondition2_1}] [...]]]
        *    [, {Buffer4} as buffer4 [...]]
        *    [where {WhereCondition1} [and {WhereCondition2}] [...]]]
        * </pre>
        * The FQL is creating by adding first the "from" KEYWORD, followed by:
        * 1) possible one or more different types of joins, each with one or more join conditions.
        * 2) not joined buffers.
        * At the end, the complete where clause is appended at the end of the FQL.
        * <p>
        *
        * @param   sortCriteria
        *          All known sort criteria. If a JOIN term is detected in the preprocessed query this list is
        *          simplified to allow a cleaner ORDER BY clause and let the query planner chose a better plan.
        *
        * @return  FqlExpression
        *          The fql expression that contains both the from and where clause.
        *
        * @throws  PersistenceException
        *          if there is an error creating the {@code FQLPreprocessor} from which the composite join data
        *          are gathered.
        */
       protected FQLExpression assembleFromAndWhereClauses(ArrayList<SortCriterion> sortCriteria)
       throws PersistenceException
       {
          FQLExpression buf = new FQLExpression();
          buf.append("from ");
          FQLExpression whereClause = new FQLExpression();
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {

             QueryComponent comp = components.get(i);
             RecordBuffer buffer = comp.getBuffer();
             if (i > 0 && comp.isOuter())
             {
                buf.append(" left outer join ");
             }
             else if (i > 0)
             {
                buf.append(", ");
             }

             buf.append(buffer.getDMOImplementationName());

             if (!isOmitAlias())
             {
                buf.append(" as ");
                buf.append(buffer.getDMOAlias());
             }
             assembleWhereClause(sortCriteria, i, whereClause, buf, comp.isOuter() && i != 0);

          }
          if (!whereClause.isEmpty())
          {
             buf.append(whereClause);
          }

          buf.append(" ");

          // this set is used to avoid duplicating joins for composite tables if they occur as
          // normal joins required by the FQL preprocessor of each component and then needed by
          // joining the composite for soring purposes
          Set<String> joins = new HashSet<>(3);

          // add any ANSI joins required by the FQL preprocessor of each component
          components = components();
          for (int idx = 0; idx < components.size(); idx++)
          {
             QueryComponent comp = components.get(idx);
             Iterator<String> joinsIter = comp.compositeJoins();
             while (joinsIter.hasNext())
             {
                String aJoin = joinsIter.next();
                buf.append(aJoin);
                buf.append(" ");
                joins.add(aJoin);
             }

             // now simplify the [sortCriteria] is a server-side JOIN was detected while preprocessing the fql
             List<FQLPreprocessor.PropertyPair> joinPairs = comp.getFQLPreprocessor().getSubstPairs();
             if (joinPairs != null)
             {
                int delCount = 0;
                SortCriterion[] criteria = new SortCriterion[sortCriteria.size()];
                sortCriteria.toArray(criteria);

                for (int i = 0; i < criteria.length; i++)
                {
                   if (criteria[i] == null)
                   {
                      continue; // already dropped
                   }

                   for (FQLPreprocessor.PropertyPair pair : joinPairs)
                   {
                      if (!pair.getOperator().equals("="))
                      {
                         continue; // ignore properties not joined with = operator
                      }

                      if (criteria[i].getOriginalName().equals(pair.getLeft()))
                      {
                         for (int j = i + 1; j < criteria.length; j++)
                         {
                            if (criteria[j] != null && criteria[j].getOriginalName().equals(pair.getRight()))
                            {
                               criteria[j] = null;
                               ++ delCount;
                               break; //j
                            }
                         }
                      }
                      else if (criteria[i].getOriginalName().equals(pair.getRight()))
                      {
                         for (int j = i + 1; j < criteria.length; j++)
                         {
                            if (criteria[j] != null && criteria[j].getOriginalName().equals(pair.getLeft()))
                            {
                               if (LOG_FINE)
                               {
                                  LOG.log(Level.FINE,
                                          "Dropped redundant sorting component " + criteria[j] +
                                          ". The duplicate is " + criteria[i]);
                               }

                               criteria[j] = null;
                               ++ delCount;
                               break; //j
                            }
                         }
                      }
                   }
                }

                // check the remaining criteria
                if (delCount != 0)
                {
                   for (int i = sortCriteria.size() - 1; i >= 0; i--)
                   {
                      if (criteria[i] == null)
                      {
                         sortCriteria.remove(i);
                      }
                   }
                }
             }
          }

          return buf;
       }

       /**
        * Assemble the restriction (i.e., where) clause for an FQL query. If more
        * than one component contains a where clause, they are combined with a
        * logical "and". Form of the clause is:
        * <pre>
        *    where ({component0_clause})
        *     [and ({component1_clause})
        *     [...]]
        * </pre>
        * <p>
        *
        * @param   sortCriteria
        *          List of <code>SortCriterion</code> objects which describe the
        *          sort behavior for this query.
        * @param   indices
        *          int which represents the position of the component we generate
        *          the where clause for.
        * @param   whereClause
        *          FQLExpression which represents the variable where the "where" clause is
        *          going to be saved.
        * @param   onClause
        *          FQLExpression which represents the variable where the "on" clause is
        *          going to be saved.
        * @param   isOn
        *          boolean that will help to distinguish between "where" and "and" and "on".
        *
        * @throws  PersistenceException
        *          if there is an error creating the <code>FQLPreprocessor</code>
        *          from which the preprocessed where clause is retrieved for each
        *          query component.
        */
       protected void assembleWhereClause(ArrayList<SortCriterion> sortCriteria,
                                          int indices,
                                          FQLExpression whereClause,
                                          FQLExpression onClause,
                                          boolean isOn)
       throws PersistenceException
       {
          FQLExpression targetClause = isOn ? onClause : whereClause;
          ArrayList<QueryComponent> components = components();
          QueryComponent comp = components.get(indices);
          RecordBuffer rb = comp.getBuffer();
          boolean multiplex = rb.isMultiplexed();
          AbstractJoin join = comp.getJoin();
          FQLExpression subClause = comp.getWhere();
          if (subClause != null)
          {
             subClause.trim();
          }
          boolean hasJoin = (join != null);
          boolean hasWhere = subClause != null && !subClause.isEmpty();

          // if the component with a server join will specify the join in a subselect,
          // we don't want the join FQL to appear in the main where clause
          if (hasJoin && join.isServerJoin())
          {
             hasJoin = !comp.isJoinWithSubselect();
          }

          boolean hasContent = (hasJoin || multiplex || hasWhere);

          if (hasContent)
          {
             if (targetClause.isEmpty() && !isOn)
             {
                targetClause.append(" where (");
             }
             else if (!targetClause.isEmpty() && isOn)
             {
                targetClause.append(" on (");
             }
             else
             {
                targetClause.append(" and (");
             }
          }

          // Determine whether where clause should be augmented with an equality test for a
          // multiplex ID.  We need to do this when:
          //    a) we are querying a multiplexed temp table;  OR
          //    b) we are querying any temp table and have a non-empty where clause
          // but we skip it if that was already added in the subclause.
          if (multiplex && (subClause == null || !subClause.wasMultiplexed()))
          {
             if (hasJoin || hasWhere)
             {
                targetClause.append("(");
             }

             targetClause.append(rb.getDMOAlias());
             targetClause.append(".");
             targetClause.append(TemporaryBuffer.MULTIPLEX_FIELD_NAME);
             targetClause.append(" = ", true);

             if (hasJoin || hasWhere)
             {
                targetClause.append(") and ");
             }
          }

          // Add subclause for a join.
          if (hasJoin)
          {
             if (multiplex || hasWhere)
             {
                targetClause.append("(");
             }

             FQLExpression[][] fqls = join.getFQL();
             FQLExpression joinWhere = new FQLExpression();
             List<Serializable> params = join.getParameters();

             for (int j = 0; j < fqls.length; j++)
             {
                boolean nullArg = false;
                if (params != null)
                {
                   Serializable val = params.get(j);
                   if (val instanceof BaseDataType)
                   {
                      BaseDataType param = (BaseDataType) val;
                      if (param.isUnknown())
                      {
                         nullArg = true;
                      }
                   }
                   else if (val == null)
                   {
                      nullArg = true;
                   }
                }

                joinWhere.append(
                         fqls[j][nullArg ? AbstractJoin.NULL_FIELD : AbstractJoin.NOT_NULL_FIELD]);

             }
             targetClause.append(joinWhere);

             if (multiplex || hasWhere)
             {
                targetClause.append(")");
             }

             if (hasWhere)
             {
                targetClause.append(" and ");
             }
          }

          if (hasWhere)
          {
             if (hasJoin || multiplex)
             {
                targetClause.append("(");
             }

             targetClause.append(subClause);

             if (hasJoin || multiplex)
             {
                targetClause.append(")");
             }
          }

          if (hasContent)
          {
             targetClause.append(") ");
          }
       }

       /**
        * Assemble the {@code order by} clause for an FQL query.
        * <p>
        * Any text-based sort criteria are augmented with the necessary upper() and rtrim() functions.
        *
        * @return  List of {@code SortCriterion} objects which describe the sort behavior for this query.
        *
        * @throws  PersistenceException
        *          if the raw sort clause provided at construction contains any unqualified property names;
        *          if the raw sort clause contains a reference to an unrecognized record buffer.
        */
       protected ArrayList<SortCriterion> assembleOrderByClause()
       throws PersistenceException
       {
          if (sort == null && dynamicSorts == null)
          {
             return EMPTY_ARRAY;
          }

          ArrayList<SortCriterion> ret = new ArrayList<>();

          if (dynamicSorts != null)
          {
             StringBuilder dynSort = new StringBuilder();
             for (Iterator<String> it = dynamicSorts.iterator(); it.hasNext(); )
             {
                dynSort.append(it.next());
                if (it.hasNext())
                {
                   dynSort.append(", ");
                }
             }
             ret.addAll(assembleOrderByClause(dynSort.toString()));

             if (sort == null)
             {
                return ret;
             }
          }

          ret.addAll(assembleOrderByClause(sort));
          return ret;
       }

       /**
        * Generates the {@code order by} clause starting from the list of criteria, including the "order by"
        * keyword. Uses the form:
        * <pre>
        *   order by {{dmo.property} {asc/desc}} [, ...]
        * </pre>
        * <p>
        *
        * @param   sortCriteria
        *          The list of criteria to be added.
        * @param   buf
        *          The string builder to add the result.
        */
       private void generateOrderBy(ArrayList<SortCriterion> sortCriteria, StringBuilder buf)
       {
          if (sortCriteria == null)
          {
             return; // quick out
          }

          boolean orderBy = true;
          for (int i = 0; i < sortCriteria.size(); i++)
          {
             SortCriterion criterion = sortCriteria.get(i);
             if (orderBy)
             {
                buf.append(" order by ");
                orderBy = false;
             }
             else
             {
                buf.append(", ");
             }
             buf.append(criterion.toString());
          }
       }

       /**
        * Assemble the {@code order by} clause for an FQL query.
        * <p>
        * Any text-based sort criteria are augmented with the necessary upper() and rtrim() functions.
        *
        * @param   sort
        *          Raw sort clause to parse and augment if necessary.
        *
        * @return  List of {@code SortCriterion} objects which describe the sort behavior for this query.
        *
        * @throws  PersistenceException
        *          if {@code sort} contains any unqualified property names;
        *          if {@code sort} clause contains a reference to an unrecognized record buffer.
        */
       protected List<SortCriterion> assembleOrderByClause(String sort)
       throws PersistenceException
       {
          List<SortCriterion> sortCriteria = buildSortCriteria(sort);

          // assemble the order by clause, given the list of SortCriterion objects.
          return assembleOrderByClause(sortCriteria);
       }

       /**
        * Returns a list <code>SortCriterion</code> objects build from the given sort clause.
        *
        * @param   sort
        *          Raw sort clause to parse and augment if necessary.
        *
        * @return  List of <code>SortCriterion</code> objects which describe the
        *          sort behavior for this query.
        *
        * @throws  PersistenceException
        *          if <code>sort</code> contains any unqualified property names;
        *          if <code>sort</code> clause contains a reference to an
        *          unrecognized record buffer.
        */
       protected ArrayList<SortCriterion> buildSortCriteria(String sort)
       throws PersistenceException
       {
          // map record buffers used in the sort clause to their identifying aliases
          Map<String, RecordBuffer> aliases = new HashMap<>();
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             RecordBuffer buffer = comp.getBuffer();
             aliases.put(buffer.getDMOAlias(), buffer);
          }

          // Parse the sort clause, extracting each qualified DMO property
          // reference, matching it to its buffer (by alias), and creating a
          // SortCriterion object for it.  Also group lists of SortCriterion
          // objects by buffer, so that each query component can store its portion
          // of the sort clause.
          Map<RecordBuffer, List<SortCriterion>> subSortMap = new HashMap<>();
          ArrayList<SortCriterion> sortCriteria = new ArrayList<>();
          String[] parts = sort.split(",");
          for (int i = 0; i < parts.length; i++)
          {
             String part1 = parts[i];
             String part = part1.trim();
             int pos = part.indexOf(".");
             if (pos < 0)
             {
                throw new PersistenceException(
                   "Order by clause contains unqualified property name:  " + part);
             }

             // Sanity.
             String alias = part.substring(0, pos);
             RecordBuffer buffer = aliases.get(alias);
             if (buffer == null)
             {
                throw new PersistenceException(
                   "Order by clause references an unrecognized buffer:  " + alias + " (" + part + ")");
             }

             // Find or create the sub-list in which to store SortCriterion
             // objects for the current buffer.
             List<SortCriterion> subSortCrit = subSortMap.get(buffer);
             if (subSortCrit == null)
             {
                subSortCrit = new ArrayList<>();
                subSortMap.put(buffer, subSortCrit);
             }

             // create the SortCriterion instance and add it to the master list and to the appropriate sub-list
             SortCriterion crit = new SortCriterion(buffer, part);

             sortCriteria.add(crit);
             subSortCrit.add(crit);
          }

          // Match the sort criteria sub-list for each buffer, generate a normalized sort clause for
          // that sub-list, and store it in the appropriate query component.
          for (Map.Entry<RecordBuffer, List<SortCriterion>> next : subSortMap.entrySet())
          {
             RecordBuffer buffer = next.getKey();
             QueryComponent comp = findComponent(buffer);
             assert (comp != null);
             List<SortCriterion> subSortCrit = next.getValue();
             String subSort = SortCriterion.toRawSortPhrase(subSortCrit);
             SortIndex sortIndex = SortIndex.get(subSort, buffer);
             comp.setSortIndex(sortIndex);
          }

          return sortCriteria;
       }

       /**
        * Assemble the {@code order by} clause for an FQL query.
        * <p>
        * Any text-based sort criteria are augmented with the necessary upper() and rtrim() functions.
        *
        * @param   sortCriteria
        *          A list of {@code SortCriterion} objects which describe the sorting to be applied.
        *
        * @return  List of {@code SortCriterion} objects which describe the sort behavior for this query.
        */
       protected ArrayList<SortCriterion> assembleOrderByClause(List<SortCriterion> sortCriteria)
       {
          RecordBuffer buffer = getRecordBuffers()[0];
          boolean multiplex = (buffer.getMultiplexID() != null);
          ArrayList<SortCriterion> workList;

          if (sortCriteria == null)
          {
             if (multiplex)
             {
                workList = new ArrayList<>(1);
             }
             else
             {
                return null;
             }
          }
          else
          {
             workList = new ArrayList<>(sortCriteria.size() + 1);
          }

          if (multiplex)
          {
             // the _multiplex is injected so that the query to benefit from the matching index

             Boolean ascIndex = IndexHelper.get(buffer.getDatabase()).getIndexDirection(buffer, sort);
             StringBuilder sb = new StringBuilder();
             sb.append(buffer.getDMOAlias()).append(".").append(TemporaryBuffer.MULTIPLEX_FIELD_NAME);
             sb.append(ascIndex == null || ascIndex ? " asc" : " desc");

             try
             {
                workList.add(new SortCriterion(buffer, sb.toString()));
             }
             catch (PersistenceException exc)
             {
                LOG.log(Level.SEVERE, "Error adding multiplex ID to order by clause", exc);
             }
          }

          if (sortCriteria != null)
          {
             workList.addAll(sortCriteria);
          }

          return workList;
       }

       /**
        * Get this query's client-side where expressions.
        *
        * @return  Unmodifiable where expression list.
        */
       protected List<Supplier<logical>> getWhereExpressions()
       {
          return whereExprList == null ? Collections.emptyList() : Collections.unmodifiableList(whereExprList);
       }

       /**
        * Get an iterator on this query's client-side where expressions.
        *
        * @return  Where expression iterator.
        */
       protected Iterator<Supplier<logical>> whereExpressions()
       {
          return getWhereExpressions().iterator();
       }

       /**
        * Compose an FQL query statement to represent the combination of all query
        * components, execute it, and store the scrollable results.  If query has
        * been executed previously and we already have a result set, simply
        * return.
        *
        * @return  <code>true</code> if query was executed for the first time;
        *          <code>false</code> if query was executed previously and we
        *          already have a result set.
        *
        * @throws  ErrorConditionException
        *          if an error occurs executing the query.
        * @throws  IllegalStateException
        *          if no query component has been added to this query.
        */
       protected boolean execute()
       {
          // translate the sort criterion
          if (sort != null && !sortTranslated)
          {
             sortTranslated = true;

             ArrayList<QueryComponent> boundComponents = null;
             for (int i = 0; i < components.size(); i++)
             {
                QueryComponent component = components.get(i);
                RecordBuffer buffer = component.getBuffer();
                RecordBuffer definition = component.getDefinitionBuffer();
                if (!buffersMatch(buffer, definition))
                {
                   if (boundComponents == null)
                   {
                      boundComponents = new ArrayList<>();
                   }

                   boundComponents.add(component);
                }
             }

             if (boundComponents != null)
             {
                // temporary switch to the definition buffer, get the sort criteria, then switch back
                int boundComponentsSize = boundComponents.size();
                RecordBuffer[] bound = new RecordBuffer[boundComponentsSize];
                RecordBuffer[] definition = new RecordBuffer[boundComponentsSize];
                int idx = 0;
                for (int i = 0; i < boundComponentsSize; i++)
                {
                   QueryComponent component = boundComponents.get(i);
                   RecordBuffer defBuffer = component.getDefinitionBuffer();
                   bound[idx] = component.getBuffer();
                   definition[idx] = defBuffer;
                   idx = idx + 1;

                   component.setBuffer(defBuffer);
                }

                List<SortCriterion> sorts;
                try
                {
                   sorts = buildSortCriteria(sort);
                }
                catch (PersistenceException e)
                {
                   throw new RuntimeException(e);
                }

                sort = translateSort(bound, definition, sorts);

                idx = 0;
                for (int i = 0; i < boundComponentsSize; i++)
                {
                   boundComponents.get(i).setBuffer(bound[idx++]);
                }
             }
          }

          if (results != null)
          {
             return false;
          }

          if (components.isEmpty())
          {
             throw new IllegalStateException(
                "Preselect query must contain at least one query component");
          }

          if (!verifyJoins())
          {
             return true;
          }

          Persistence persistence = getPersistence();

          if (!regWithSession)
          {
             // Register to receive session closing notification.
             persistence.registerSessionListener(this, SessionListener.Scope.NONE, isSharedContext());
             regWithSession = true;
          }

          // if all buffers in the query are against temp-tables, fetch full records
          boolean allTemp = true;
          ArrayList<QueryComponent> components = components();
          for (int i = 0; allTemp && i < components.size(); i++)
          {
             allTemp = components.get(i).getBuffer().isTemporary();
          }

          if (allTemp)
          {
             setFullRecords();
          }

          Results r = null;
          try
          {
             // assemble FQL statement
             fql = assembleFQL();

             // Prepare substitution parameters.
             List<Object> argList = new ArrayList<>();
             prepareParameters(argList);
             Object[] args = argList.toArray();

             if (LOG_FINE)
             {
                LOG.log(Level.FINE, "FQL:  " + fql);
                StringBuilder buf = new StringBuilder("PARMS:  {");
                for (int i = 0; i < args.length; i++)
                {
                   if (i > 0)
                   {
                      buf.append(", ");
                   }
                   Object next = args[i];
                   if (next instanceof BaseDataType)
                   {
                      buf.append(((BaseDataType) next).toStringMessage());
                   }
                   else
                   {
                      buf.append(next);
                   }
                }
                buf.append("}");
                LOG.log(Level.FINE, buf.toString());
             }

             // possibly leak uncommitted updates to other sessions
             Iterator<QueryComponent> iter = components().iterator();
             while (iter.hasNext())
             {
                QueryComponent next = iter.next();
                FQLPreprocessor fqlPreproc = next.getFQLPreprocessor();
                Set<String> earlyPublish = fqlPreproc.getEarlyPublishEntities();
                if (!earlyPublish.isEmpty())
                {
                   RecordBuffer buffer = next.getBuffer();
                   DirtyShareContext dirtyContext = buffer.getDirtyContext();
                   if (dirtyContext != null)
                   {
                      List<Record> dmos = dirtyContext.updateSnapshots(earlyPublish, persistence);

                      // Make sure DMOs do not bloat the ORM session.
                      for (Record dmo : dmos)
                      {
                         buffer.evictDMOIfUnused(dmo);
                      }
                   }
                }
             }

             Long templateRowid = getTemplateQueryRowid(args);
             if (templateRowid != null)
             {
                r = new TemplateResults(templateRowid);
                setResults(r);
             }
             else
             {
                // execute the primary query
                r = P2JQueryExecutor.getInstance().execute(getRecordBuffers()[0].getDatabase(),
                                                           PreselectQuery::executeQuery,
                                                           this,
                                                           persistence,
                                                           fql,
                                                           args);
                r = createResults(r);
                setResults(r);

                registerCleaner();
             }
          }
          catch (PersistenceException exc)
          {
             ErrorManager.recordOrThrowError(exc);
          }
          finally
          {
             rowCount = -1;
          }

          return true;
       }

       /**
        * Detect whether the components of the query are shared or private from multi-tenant point of view.
        * In case of conflict, an error is logged.
        *
        * @return  {@code true} if the buffers of teh query are (all) shared.
        */
       protected boolean isSharedContext()
       {
          int contextType = -1;
          for (QueryComponent comp : components())
          {
             boolean mt = comp.getBuffer().getDmoInfo().multiTenant;
             if (mt)
             {
                if (contextType == Persistence.SHARED_IDX)
                {
                   contextType = -2;
                   break;
                }
                contextType = Persistence.PRIVATE_IDX;
             }
             else
             {
                if (contextType == Persistence.PRIVATE_IDX)
                {
                   contextType = -2;
                   break;
                }
                contextType = Persistence.SHARED_IDX;
             }
          }

          if (contextType < 0)
          {
             LOG.log(Level.WARNING, "MTCtx: Failed TENANT context detection inPreselectQuery!");
          }

          return contextType == Persistence.SHARED_IDX;
       }

       /**
        * Indicate whether results resources should be cleaned up upon exiting
        * the global scope or upon exiting the nearest enclosing scope in which
        * the results were created.
        * <p>
        * This default implementation returns <code>false</code>, indicating
        * results should be cleaned up at the nearest enclosing scope.
        * Subclasses which require different behavior must override this method.
        *
        * @return  <code>false</code>.
        */
       protected boolean isTopLevelCleanup()
       {
          return false;
       }

       /**
        * Invoked just before query execution, this method checks whether all
        * components with a non-null JOIN have a valid inverse on which to perform
        * the join.  If not, set the query's results to an empty result set and
        * indicate failure.
        *
        * @return  <code>true</code> if verification succeeded and query execution
        *          should continue;  else <code>false</code>
        */
       protected boolean verifyJoins()
       {
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             AbstractJoin join = comp.getJoin();

             if (join != null && !join.isServerJoin())
             {
                BufferReference ref = join.getInverse();
                Record record = ref.buffer().getCurrentRecord();

                if (record == null)
                {
                   handleMissingJoinRecord();

                   return false;
                }
             }
          }

          return true;
       }

       /**
        * Respond to the condition that a component with a non-null JOIN has a
        * missing inverse on which to perform the join.
        * <p>
        * This implementation creates an empty result set.
        */
       protected void handleMissingJoinRecord()
       {
          setResults(new SimpleResults(null));
       }

       /**
        * Attempt to cache all results from the specified target result set into a {@link Results}
        * instance. If this is a scrolling query, this will cache all results in the target result
        * set, including those already visited. Otherwise, only the current result and all remaining,
        * unvisited results are cached.
        *
        * @param   target
        *          Result set which is to be cached, presumably because it will be invalid
        *          momentarily. If results previously were cached, {@code target} is simply returned.
        * @param   forceIdOnly
        *          {@code true} if the records should be cached only by their PK and not force full
        *          results iteration to hydrate the DMOs.
        *
        * @return  Results from {@code target}, if any, cached in a {@code Results} object. If there
        *          were no results available to cache, an empty result set is returned.  If any of
        *          the record buffers backing this query have been closed, {@code null} is returned.
        */
       protected Results cacheResults(Results target, boolean forceIdOnly)
       {
          if (isResultSetCached())
          {
             // results are already cached
             return target;
          }

          // If any buffer associated with this query is closed, do not
          // attempt to cache results.  Instead, clean up resources;  this
          // query is done.
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             RecordBuffer buffer = comp.getBuffer();
             if (buffer.getOpenScopeCount() < 1)
             {
                resetResults();

                return null;
             }
          }

          // Remember current row.
          int currentRow = (isScrolling() ? target.getRowNumber() : 0);

          // Cache all preselected results in a SimpleResults object and set
          // that as the query's new backing results.
          ArrayList<Object[]> rows = new ArrayList<>(64);

          int numberOfRowsCached = 0;
          if (isScrolling())
          {
             target.reset();
             while (target.next())
             {
                boolean forceIdLenientCache = ++numberOfRowsCached > LENIENT_CACHING_THRESHOLD ? forceIdOnly : false;
                rows.add(target.get(forceIdLenientCache || this.forceOnlyId));
             }
          }
          else if (target.getRowNumber() == -1)
          {
             currentRow = -1;
             while (target.next())
             {
                rows.add(target.get(++numberOfRowsCached > LENIENT_CACHING_THRESHOLD ? forceIdOnly : false));
             }
          }
          else if (target.get() != null)
          {
             do
             {
                rows.add(target.get(++numberOfRowsCached > LENIENT_CACHING_THRESHOLD ? forceIdOnly : false));
             } while (target.next());
          }

          rows.trimToSize();

          // Override SimpleResults.cleanup() to do nothing, so we don't
          // immediately discard results when the current scope ends and our
          // Finalizable cleaner (installed in execute()) does its work.
          SimpleResults cached = new SimpleResults(rows)
          {
             public void cleanup() {}
          };

          // Reset to current row.
          cached.setRowNumber(currentRow);

          return cached;
       }

       /**
        * Prepare query substitution parameters for this query.  One argument must be added to
        * {@code args} list for each {@code ?} placeholder in the FQL query string for this query.
        *
        * @param   args
        *          Argument list to be populated.
        * @throws  PersistenceException
        *          if there is an error creating an {@code FQLPreprocessor} during the gathering of
        *          query substitution parameters.
        */
       protected void prepareParameters(List<Object> args)
       throws PersistenceException
       {
          List<Object> notOuterArgs = new ArrayList<>();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent next = components.get(i);
             RecordBuffer rb = next.getBuffer();
             List<Object> targetArgs = next.isOuter() && i > 0 ? args : notOuterArgs;
             if (rb.isMultiplexed())
             {
                targetArgs.add(rb.getMultiplexID());
             }

             AbstractJoin join = next.getJoin();
             if (join != null)
             {
                List<Serializable> parameters = join.getParameters();
                if (parameters != null)
                {
                   for (int j = 0; j < parameters.size(); j++)
                   {
                      Serializable val = parameters.get(j);
                      if (val instanceof BaseDataType)
                      {
                         BaseDataType param = (BaseDataType) val;
                         if (!param.isUnknown())
                         {
                            targetArgs.add(param);
                         }
                      }
                      else if (val != null)
                      {
                         targetArgs.add(val);
                      }
                   }
                }
             }

             Object[] cArgs = next.getArgs();
             if (cArgs != null)
             {
                if (parameterFilter != null)
                {
                   cArgs = parameterFilter.filterParameters(cArgs);
                }

                for (int j = 0; j < cArgs.length; j++)
                {
                   if (cArgs[j] instanceof Resolvable)
                   {
                      targetArgs.add(((Resolvable) cArgs[j]).resolve());
                   }
                   else if (cArgs[j] instanceof P2JQuery.ParamResolver)
                   {
                      targetArgs.add(((P2JQuery.ParamResolver) cArgs[j]).resolve());
                   }
                   else
                   {
                      targetArgs.add(cArgs[j]);
                   }
                }
             }
          }
          args.addAll(notOuterArgs);
       }

       /**
        * Collect the names of the entities (DMO types) associated with this query.
        *
        * @return  Array of entity names.
        */
       protected Class<? extends Record>[] getEntities()
       {
          int len = components.size();
          Class<? extends Record>[] entities = new Class[len];
          for (int i = 0; i < len; i++)
          {
             entities[i] = components.get(i).getBuffer().getDMOImplementationClass();
          }

          return entities;
       }

       /**
        * Checks if this is a template query. A template query is a rowid query that looks for a
        * special template record that is not found in database. The template record is a record
        * having all fields set to default values. The id of such record is in P2J, by convention
        * negative.
        * In case this is a simple template query, this method returns the rowid of the template for
        * current buffer from the single.
        * <p>
        * <b>Note:</b><br>
        * The current implementation identifies only simple queries for template records:
        * <ol>
        *    <li>this {@code PreselectQuery} has a single component;</li>
        *    <li>the where predicate directly test whether the rowid/recid is equal to a value
        *          (this is done by the {@link FQLPreprocessor} of the component);</li>
        *    <li>the value (constant or passed in as query SUBST parameter) is negative (as noted
        *          above).</li>
        * </ol>
        * Although this kind of queries are sufficient for standard template record load according to
        * P4GL idiom, more complex queries can be issued, with predicates that basically match the
        * pattern but add supplementary AND/OR clauses. This kind of queries are not always of common
        * sense and the result in 4GL is not always as expected.<br>
        * For example:
        *       <p>{@code FOR EACH book WHERE RECID(book) = templateId OR book.isbn NE ?}</p>
        * will iterate only books that have the isbn not null, but will not contain the template
        * record in the result returned.
        * <p>
        * On the other hand, it does not make much sense to:
        * <ul>
        *    <li>iterate over the the template records of a table since there will always be at most
        *       one such record;</li>
        *    <li>add additional OR/AND clauses in WHERE predicate since, if the template record
        *       exists, it is well defined by it rowid, configured in database schema and
        *       immutable.</li>
        * </ul>
        *
        * @param   args
        *          The actual parameters of the query.
        *
        * @return  if this query looks for the template record of the component the rowid is returned
        *          and {@code null} otherwise.
        *
        */
       protected Long getTemplateQueryRowid(Object[] args)
       {
          if (components == null || components.size() != 1)
          {
             return null;
          }
          // TODO: need some attention:
          // - this query type is not only used for FOR EACH constructs. It is the base class for
          //       other query types as well. Need to check if template record changes applied to all
          //       these query types;

          FQLPreprocessor prep = null;
          try
          {
             prep = components.get(0).getFQLPreprocessor();
          }
          catch (PersistenceException e)
          {
             return null;
          }

          if (!prep.isFindByRowid())
          {
             return null;
          }

          long tId = prep.getFindByRowid(args);

          return tId < 0 ? tId : null;
       }

       /**
        * Execute the query to create a first level result set.
        * <p>
        * This default implementation returns an object which is a thin wrapper around
        * {@code ScrollableResults}.  Subclasses may override this method if they need a
        * {@code Results} implementation other than {@code ScrollingResults}.
        * <p>
        * In case when the {@code fql} represents a valid template record query (a test on recid /
        * rowid of the record against the unique {@code _file._template} for current record of the
        * single {@code QueryComponent}), a special {@link Results} object is returned with a single
        * element that is the template record for the buffer of the component.
        *
        * @param   persistence
        *          Object which is used to execute the query.
        * @param   fql
        *          FQL query string.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within the FQL query string.
        *
        * @return  An instance of {@code ScrollingResults} which wrappers the
        *          {@code ScrollableResults} object returned by the execution of the query.
        *
        * @throws  PersistenceException
        *          if there is an error executing the query.
        */
       protected Results executeQuery(Persistence persistence, String fql, Object[] args)
       throws PersistenceException
       {
          Long templateRowid = getTemplateQueryRowid(args);
          if (templateRowid != null)
          {
             return new TemplateResults(templateRowid);
          }

          return shouldExecuteForward() ?
                 new ForwardResults(persistence, executeScroll(persistence, fql, args))  :
                 new ScrollingResults(persistence, executeScroll(persistence, fql, args));
       }

       /**
        * Execute the query to create a {@code ScrollableResults} result set.
        *
        * @param   persistence
        *          Object which is used to execute the query.
        * @param   fql
        *          FQL query string.
        * @param   args
        *          Substitution parameters to be inserted into placeholders within
        *          within the FQL query string.
        *
        * @return  The {@code ScrollableResults} object returned by the execution of the query.
        *
        * @throws  PersistenceException
        *          if there is an error executing the query.
        */
       protected <T> ScrollableResults<T> executeScroll(Persistence persistence, String fql, Object[] args)
       throws PersistenceException
       {
          int scrollMode = isScrolling()
                           ? ResultSet.TYPE_SCROLL_INSENSITIVE
                           : ResultSet.TYPE_FORWARD_ONLY;

          return persistence.scroll(getEntities(), fql, args, 0, 0, getDataRelation(), scrollMode);
       }

       /**
        * Optionally wrapper the result set created for the query with a filtering
        * <code>Results</code> implementation (for client-side where processing).
        * It is this object which actually will be used to retrieve the primary
        * key values of records to be loaded from the database.
        * <p>
        * If no filtering is necessary, simply return the results object which
        * was passed in.
        * <p>
        * Subclasses may override this method to insert their own layers into
        * result set processing.
        *
        * @param   results
        *          An object which manipulates the database result set returned
        *          from execution of the query.
        *
        * @return  <code>Results</code> object.
        *
        * @throws  PersistenceException
        *          if there is any error creating results.
        */
       protected Results createResults(Results results)
       throws PersistenceException
       {
          if (isClientWhere())
          {
             return (new FilteredResults(results,
                                         Arrays.asList(getRecordBuffers()),
                                         getWhereExpressions(),
                                         !fullRecords));
          }

          return results;
       }

       /**
        * Fetch all records for a given result row into their backing buffers.  Each result row
        * contains one column per query component.  Each column contains the unique ID of the record
        * to be fetched or the record itself. If the latter, the record is refetched in case it has
        * changed since the record was first retrieved.  If the scrollable cursor backing this query
        * currently is at a position where no retrieval is possible, the backing buffers will have
        * their current records released.
        * <p>
        * As particular case when the query looks for the template record of the buffer of the single
        * {@link QueryComponent}, the buffer is not populated with a record fetched from backing
        * database, instead the template record for the buffer is automatically created in-memory.
        * This {@code results} object has only one element to iterate on and attempting to iterate on
        * this object will throw {@link QueryOffEndException} and release the buffer.
        *
        * @param   available
        *          {@code true} if the most recent scrollable cursor move left the cursor at an
        *          available result row;  {@code false} if the cursor cannot read a result at its
        *          current position.
        * @param   lockType
        *          Overriding lock type to apply to fetched records.  If {@code null}, default lock
        *          types are applied.
        * @param   errorIfNull
        *          {@code true} if failed retrievals should raise error.
        *
        * @throws  MissingRecordException
        *          if the record was expected to be available, but wasn't.  This typically indicates
        *          the record was deleted between the time it initially was found by the query and the
        *          time it was to be fetched to store in the record buffer.
        * @throws  ErrorConditionException
        *          if the fetch of any record fails or if no records are available and the query is
        *          set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail in this case.
        */
       protected void fetch(boolean available, LockType lockType, boolean errorIfNull)
       throws MissingRecordException
       {
          currentRowDeleted = false;

          try
          {
             // under certain circumstances, we must bypass releasing the current record(s) (if any)
             // in the buffer(s), so we throw the off-end exception directly instead of setting the
             // current buffer record to null
             if (!available   &&
                 rowCount < 0 &&
                 !errorIfNull &&
                 _isOffEnd()  &&
                 ((isPresort() && isIterating()) ||
                  preserveBuffersOnEmptyResults(results)))
             {
                if (!lenientOffEnd)
                {
                   getRecordBuffers()[0].throwOffEnd();
                }

                return;
             }

             // check if a template search was issued. In this case we have only a single component.
             // We will load the template record in the respective buffer.
             if (results instanceof TemplateResults)
             {
                QueryComponent comp = components.get(0);
                RecordBuffer buffer = comp.getBuffer();
                if (available)
                {
                   buffer.loadTemplateRecord(results.getID(0));
                }
                else
                {
                   // will release the buffer and throw QueryOffEndException
                   buffer.setRecord(null,
                                    lockType,
                                    false,
                                    lenientOffEnd,
                                    comp.getSortIndex(),
                                    offEnd,
                                    inverseSorting,
                                    false,
                                    false);
                }
             }
             else
             {
                // invalidated queries always fetch with LockType.NONE
                // SHARE or EXCLUSIVE locks needs updateLock in coreFetch
                boolean alreadyFetched = available &&
                                         results.isAutoFetch() &&
                                         !shouldUpdateAnyLock(lockType);
                if (!alreadyFetched)
                {
                   Object[] data = available ? results.get(forceOnlyId) : null;
                   coreFetch(data, lockType, available, errorIfNull, false);
                }

                if (available)
                {
                   accumulate();
                }
             }
          }
          catch (QueryOffEndException exc)
          {
             if (!lenientOffEnd)
             {
                throw exc;
             }
          }
          catch (MissingRecordException exc)
          {
             throw exc;
          }
          catch (PersistenceException exc)
          {
             ErrorManager.recordOrThrowError(exc);
          }
       }

       /**
        * Fetch all records for a given result row into their backing buffers, given an array of data which
        * contains one primary key or DMO per query component.
        *
        * @param   data
        *          An array of primary key IDs or DMOs representing records to be fetched for the composite row.
        * @param   lockType
        *          Overriding lock type to apply to fetched records.  If {@code null}, default lock types are
        *          applied.
        * @param   errorIfNull
        *          {@code true} if failed retrievals should raise error.
        * @param   silentIfNullId
        *          If {@code true}, do not raise {@link QueryOffEndException} if some of the provided IDs are
        *          {@code null}.{@code null} IDs are valid for queries with OUTER join.
        *
        * @return  {@code true} if data was fetched and is available and {@code false} is the fetch failed and no
        *          error was raised (when {@code errorIfNull} is false).
        *
        * @throws  MissingRecordException
        *          if the record was expected to be available, but wasn't.  This typically indicates the record
        *          was deleted between the time it initially was found by the query and the time it was to be
        *          fetched to store in the record buffer.
        * @throws  PersistenceException
        *          if the fetch of any record fails.
        * @throws  ErrorConditionException
        *          if no records are available and the query is set to fail in this case.
        * @throws  QueryOffEndException
        *          if no records are available and the query is set not to fail in this case.
        */
       protected final boolean coreFetch(Object[] data,
                                         LockType lockType,
                                         boolean available,
                                         boolean errorIfNull,
                                         boolean silentIfNullId)
       throws PersistenceException
       {
          int recFetched = 0;
          QueryOffEndException throwOffEndException = null;
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             RecordBuffer buffer = comp.getBuffer();
             boolean dirtyCopy = false;
             Record dmo = null;
             LockType lt = lockType;
             if (data != null && data[i] != null && !currentRowDeleted)
             {
                LockType compLock = comp.getLockType();

                // Use default lock type if not overridden.
                if (lt == null)
                {
                   lt = compLock;
                }

                // Individual result is either a DMO (i.e., Record) or a primary key ID.
                Object o = data[i];
                Long id = null;
                boolean updateLock;
                boolean fullRec = o instanceof Record;

                if (fullRec)
                {
                   dmo = (Record) o;
                   id = dmo.primaryKey();
                   updateLock = (lt != LockType.NONE);
                }
                else
                {
                   id = (Long) o;
                   updateLock = true;
                }

                // if the DMO currently in the buffer has the same ID as the ID
                // we wish to load, but also represents a dirty copy (a copy of
                // uncommitted changes in another session), leave it in the buffer
                if (buffer.isDirtyCopy())
                {
                   Record dirtyDMO = buffer.getCurrentRecord();
                   if (dirtyDMO != null && id.equals(dirtyDMO.primaryKey()))
                   {
                      dirtyCopy = true;
                      dmo = dirtyDMO;
                   }
                }

                DirtyShareContext dirtyContext = DirtyShareSupport.isEnabledCrossSession() || DirtyShareSupport.isEnabledIntraSession() ?
                                                    buffer.getDirtyContext() : null;
                String entity = buffer.getEntityName();

                // if we have an ID only, a cached DMO, or we are working with a temp-table, fetch
                // the current version of the record (dirty DMO copy excepted)
                boolean doFetch = (!fullRec || isResultSetCached() || !buffer.isTemporary()) && !dirtyCopy;

                if (doFetch)
                {
                   // a record may have been retrieved from a cache of detached objects, OR its
                   // state may have been changed (updated or deleted) since we first retrieved it,
                   // so reload it by its ID
                   Persistence p = getPersistence();
                   if (updateLock)
                   {
                      FibonacciCounter fib = new FibonacciCounter();
                      long timeout = fib.next();

                      do
                      {
                         try
                         {
                            // force the load of the entire record
                            dmo = p.load(buffer.getDMOImplementationClass(), id, lt, timeout, updateLock, null);

                            break;
                         }
                         catch (LockTimeoutException exc)
                         {
                            if (LOG_FINE)
                            {
                               String table = buffer.getTable();
                               RecordIdentifier<String> ident = new RecordIdentifier<>(table, id);
                               LOG.fine("[" +
                                        Utils.describeContext() +
                                        "] coreFetch lock timeout: " +
                                        timeout +
                                        " ms: " +
                                        ident);
                            }

                            // check if record was deleted in uncommitted transaction in another session
                            if (dirtyContext != null && dirtyContext.isDirtyDelete(entity, id, true))
                            {
                               String table = buffer.getTable();
                               RecordIdentifier<String> ident = new RecordIdentifier<>(table, id);

                               throw new MissingRecordException(ident);
                            }

                            // steadily increase the timeout value
                            timeout = fib.next();

                            // log deadlock warning if timeout is getting too long and wait
                            // indefinitely
                            if (timeout > 3600000L)
                            {
                               timeout = 0L;
                               if (LOG.isLoggable(Level.WARNING))
                               {
                                  String table = buffer.getTable();
                                  RecordIdentifier<String> ident = new RecordIdentifier<>(table, id);
                                  LOG.log(Level.WARNING,
                                          "[" +
                                          Utils.describeContext() +
                                          "] Possible deadlocked record: " + ident,
                                          new Throwable());
                               }
                            }
                         }
                      } while (true);
                   }
                   else
                   {
                      dmo = p.load(buffer.getDMOImplementationClass(),
                                   id,
                                   lt,
                                   0L,
                                   updateLock,
                                   // the entire record is loaded if the lock is not NONE
                                   updateLock || dmo == null ? null : dmo._getReadFields());
                   }
                }

                if (LOG_FINE)
                {
                   StringBuilder buf = new StringBuilder("DMO:  ");
                   if (id != null)
                   {
                      buf.append(id);
                   }
                   else
                   {
                      buf.append("<not found>");
                   }

                   LOG.log(Level.FINE, buf.toString());

                   if (LOG.isLoggable(Level.FINEST))
                   {
                      buf.setLength(0);
                      buf.append("DMO Detail:  ");
                      buf.append(dmo);

                      LOG.log(Level.FINEST, buf.toString());
                   }
                }

                // If we expected this record to be available, but it wasn't, it must have been deleted.
                // Likewise, if we know it was deleted in another context, this is an exceptional condition.
                // Throw an exception so the caller can figure out the appropriate recovery.
                if (dmo == null                                                          ||
                    dirtyContext != null && dirtyContext.isDirtyDelete(entity, id, true) ||
                    fullRec && dmo.checkState(DmoState.DELETED))
                {
                   if (_isSkipDeletedRecord() || isFillingBrowseRows())
                   {
                      String table = buffer.getTable();
                      RecordIdentifier<String> ident = new RecordIdentifier<>(table, id);

                      throw new MissingRecordException(ident);
                   }
                   else
                   {
                      // implementation of SKIP-DELETED-RECORD for -dynamic- queries. If positioned on a record
                      // which was deleted after being preselected, do not skip it, instead empty the buffer
                      // if the query attribute was set to false.
                      // NOTE: although the documentation expressly mentions that this attribute affects only
                      //       the dynamic queries, from my testcases, they seems to behave the same.
                      buffer.release(false);
                      continue; // with next buffer / component (main for)
                   }
                }
             }

             // TODO: confirm or fix behavior:  first null record will raise error or end condition;
             //       remaining buffers will not have their current records released.
             SortIndex sortIndex = comp.getSortIndex();
             boolean outerJoin = comp.isOuter() && dmo == null  && i > 0;
             try
             {
                if (dmo != null ||
                    (outerJoin && available) ||
                    (comp.getIteration() != QueryConstants.LAST &&
                     comp.getIteration() != QueryConstants.FIRST))
                {
                   buffer.setRecord(dmo,
                            lt,
                            errorIfNull || silentIfNullId,
                            lenientOffEnd,
                            sortIndex,
                            offEnd,
                            inverseSorting,
                            dirtyCopy,     // TODO:  need better dirty check
                            false);
                }

                if (outerJoin)
                {
                   buffer.setUnknownMode();
                }

                if (dmo != null)
                {
                   ++recFetched;
                }

                Buffer bufferImpl = (Buffer) buffer.getDMOProxy();
                if (bufferImpl.autoSynchronize().booleanValue())
                {
                   bufferImpl.querySynchronize();
                }
             }
             catch (QueryOffEndException ex)
             {
                if (!this.lenientOffEnd && i == components.size() - 1)
                {
                   throw ex;
                }
             }

             if (dmo == null && errorIfNull)
             {
                buffer.errorNotOnFile();
             }
          }

          return recFetched != 0;
       }

       /**
        * Indicate whether this query requires initial results to be filtered
        * through a client-side where clause expression.
        *
        * @return  <code>true</code> if client-side filter is required, else
        *          <code>false</code>.
        */
       protected final boolean isClientWhere()
       {
          return (!getWhereExpressions().isEmpty());
       }

       /**
        * Access the current <code>Results</code> object for this query.
        *
        * @return  <code>Results</code> object.
        */
       protected Results getResults()
       {
          execute();

          return results;
       }

       /**
        * Add a component to the query.
        *
        * @param   comp
        *          Query component to be added.
        *
        * @throws  UnsupportedOperationException
        *          if attempting an outer join (temporary);
        *          if iteration type is not <code>NEXT</code> (temporary);
        *          if the component's DMO's associated database is different than
        *          that of a previously added component's database (cross-database
        *          preselect joins are not possible).
        */
       protected void addComponent(QueryComponent comp)
       {
          if (isClientWhere())
          {
             throw new IllegalStateException(
                "Cannot have a multi-table query with a client-side where clause");
          }

          switch (comp.getIteration())
          {
             case NEXT:
             case FIRST:
             case LAST:
                break;
             default:
                if (!supportsArbitraryIteration())
                {
                   throw new UnsupportedOperationException(
                      "Preselect query supports only NEXT/FIRST/LAST iteration");
                }
                break;
          }

          boolean isFirst = components.isEmpty();
          RecordBuffer buffer = comp.getBuffer();

          if (buffer.getOpenScopeCount() > 0)
          {
             buffer.initialize();

             // if there is a record waiting on the queried index in the record nursery, make it visible;
             // flush the record buffer, in case a transient, newly created record currently is in the buffer
             try
             {
                // TODO: find a more direct route to the numeric index ID that does not involve a lookup by name
                String idxName = buffer.getIndexHelper().getIndexForSort(buffer, sort);
                int idxId = buffer.getDmoInfo().getRecordMeta().getIndexIdByName(idxName);
                RecordNursery nursery = buffer.getNursery();
                nursery.makeVisible(buffer.getDmoInfo().getId(), buffer.getMultiplexID(), idxId);
                buffer.flush();
             }
             catch (ValidationException | PersistenceException exc)
             {
                // a validation error can only result from flushing an unrelated record currently in
                // the buffer in preparation for a query result; since this will not be within the
                // context of any silent error mode under which this query is running, we instruct
                // the error manager to not suppress the message
                ErrorManager.throwError(exc, false);
             }

             if (isFirst)
             {
                thisFinalizable = true;
                // Register for retry support.
                TransactionManager.registerFinalizable(this, false);
             }
             // Verify that no cross-connection join is being attempted.
             else if (!getPersistence().equals(buffer.getPersistence()))
             {
                throw new UnsupportedOperationException(
                   "Preselect query cannot join across database connections");
             }
          }

          if (!isFirst)
          {
             comp.setNotTop();
          }

          // Add the component.
          components.add(comp);

          // Update cumulative substitution argument count.
          Object[] args = comp.getRawArgs();
          if (args != null)
          {
             int len = args.length;
             for (int i = 0; !resolvableArgs && i < len; i++)
             {
                if (args[i] instanceof Resolvable)
                {
                   resolvableArgs = true;
                }
             }
          }
       }

       /**
        * Does this query support non-NEXT iteration components?
        *
        * @return  <code>false</code>
        */
       protected boolean supportsArbitraryIteration()
       {
          return false;
       }

       /**
        * Indicate whether the record buffer(s) current record(s) should be preserved, in the event
        * the query does not find any results.
        * <p>
        * This implementation always returns false. Subclasses should override this behavior if
        * needed.
        *
        * @param   results
        *          The results instance to decide if they are truly empty. The results are not visible
        *          in some sub-classes, so let them be passed by parameter.
        *
        * @return  <code>false</code>.
        */
       protected boolean preserveBuffersOnEmptyResults(Results results)
       {
          return false;
       }

       /**
        * Clear the reposition cache of visited, virtual composite rows.
        */
       protected void clearRepoCache()
       {
          repoCache.clear();
       }

       /**
        * Get the FQL query which was executed to produce this query's results.
        *
        * @return  FQL query or <code>null</code> if the query has not yet been executed.
        */
       protected String getHQL()
       {
          return fql;
       }

       /**
        * Get the persistence service object used by this query.
        *
        * @return  Persistence object.
        */
       protected Persistence getPersistence()
       {
          if (persistence == null)
          {
             // Since no cross-connection joins are permitted, it is safe to set
             // the query's Persistence instance using the first component only.
             RecordBuffer buffer = components.get(0).getBuffer();
             persistence = buffer.getPersistence();
          }

          return persistence;
       }

       /**
        * Register a <code>Finalizable</code> with the transaction manager for
        * resource cleanup when this query's scope ends.
        */
       protected void registerCleaner()
       {
          // If we are not a standalone query, we assume our container will
          // register for cleanup.
          if (regCleaner || !isStandalone())
          {
             return;
          }

          if (isTopLevelCleanup())
          {
             TransactionManager.registerTopLevelFinalizable(cleaner, true);
          }
          else
          {
             TransactionManager.registerFinalizable(cleaner, false);
          }

          regCleaner = true;
       }

       /**
        * Reposition the cursor such that a request to retrieve the next result
        * will retrieve the result which matches the specified array of primary
        * key IDs.  A request to retrieve the previous result will retrieve the
        * result immediately previous to this, if any, in the results list.
        *
        * @param   ids
        *          Array of primary key IDs, arranged from left to right to
        *          coincide with the records being joined by the underlying
        *          query.
        * @param   forceFetch
        *          <code>true</code> to override the current fetch-on-reposition
        *          setting to <code>true</code>, in case the underlying
        *          <code>Results</code> implementation attempts to fetch the
        *          record during repositioning;  <code>false</code> to honor the
        *          current setting.
        *
        * @return  <code>true</code> if reposition was successful;
        *          <code>false</code> if unsuccessful.
        *
        * @throws  ErrorConditionException
        *          if there is an error repositioning the cursor.
        */
       protected boolean repositionByID(Long[] ids, boolean forceFetch)
       {
          integer oldCurrentRow = currentRow();

          prepareReposition();

          if (results == null)
          {
             execute();
          }

          ids = DBUtils.trimToTableCount(ids, this);
          int newRow = repoCache.getRow(ids);
          if (newRow >= 0)
          {
             int oldRow = results.getRowNumber();
             boolean avail;
             if (forceFetch)
             {
                boolean oldFetch = isFetchOnReposition();
                try
                {
                   setFetchOnReposition(true);
                   avail = results.setRowNumber(newRow);
                }
                finally
                {
                   setFetchOnReposition(oldFetch);
                }
             }
             else
             {
                avail = results.setRowNumber(newRow);
             }
             if (avail)
             {
                offEnd = OffEnd.NONE;
                betweenRows = true;
             }
             else
             {
                int diff = oldRow - newRow;
                offEnd = (diff < 0 ? OffEnd.FRONT : OffEnd.BACK);
                betweenRows = false;
             }

             afterReposition(true);

             return true;
          }

          // Scan all results for a match.  Start from where cache last left off.
          boolean error = false;
          int size = repoCache.size();
          results.setRowNumber(size);
          error = results.getRowNumber() == -1;
          if (!error && results.get(0) == null)
          {
             // result set is fully cached (without a match) or it is empty
             error = size > 0 || !results.first();
          }

          int width = getTableCount();
          if (!error)
          {
             // For a partial match (number of provided IDs is less than the number of query tables)
             // 4GL *in most cases* returns the *last* row in the subset. E.g. "1 2 4" is returned for
             // the request "1" here:
             // 1 1 2
             // 1 2 3
             // 1 2 4
             // That may not be true for large data sets. 4GL documentation states that the order is
             // unpredictable.
             //
             // FWD caches in repoCache only a part of the full result set, up to the max row found
             // by IDs search because of performance considerations. In order to implement finding
             // of the last row, the following mechanics are implemented:
             // 1. FWD fetches at least one extra row to know where the target subset ends.
             // 2. FWD fetches more rows to get the full subset for the outer component into the cache
             // in order to to avoid putting the part of the following subsets into the cache. This is
             // necessary to avoid quick-out check of repoCache where non-last component is mistakenly
             // returned for the subset. E.g. in the following situation "2 1 2" is returned for the
             // request "2" while it is correct to return "2 1 3":
             // 1 1 2 <- in cache
             // 1 2 3 <- in cache
             // 1 2 4 <- in cache
             // 2 1 2 <- in cache
             // 2 1 3

             Integer partialMatchRow = null;
             Integer targetRow = null;
             Serializable outerId = null;
             do
             {
                boolean match = true;
                Long[] rsIDs = new Long[width];
                for (int i = 0; i < width; i++)
                {
                   Long nextID = results.getID(i);
                   rsIDs[i] = nextID;
                   if (ids.length > i && ids[i] != null && !ids[i].equals(nextID))
                   {
                      match = false;
                   }
                }

                if (targetRow != null)
                {
                   // we already found the target row, scroll to the end of outermost subset
                   if (outerId.equals(rsIDs[0]))
                   {
                      repoCache.add(rsIDs, results.getRowNumber());
                      continue;
                   }
                   else
                   {
                      break;
                   }
                }

                if (match)
                {
                   repoCache.add(rsIDs, results.getRowNumber());
                   outerId = rsIDs[0];
                   if (rsIDs.length > ids.length)
                   {
                      //partial match
                      partialMatchRow = results.getRowNumber();
                   }
                   else
                   {
                      // complete match
                      targetRow = results.getRowNumber();
                   }
                }
                else if (partialMatchRow != null)
                {
                   // previous row was the last partial match in the subset, so it is our target
                   if (outerId.equals(rsIDs[0]))
                   {
                      repoCache.add(rsIDs, results.getRowNumber());
                   }
                   targetRow = partialMatchRow;
                }
                else
                {
                   repoCache.add(rsIDs, results.getRowNumber());
                }
             }
             while (results.next());

             if (targetRow == null && partialMatchRow != null)
             {
                targetRow = partialMatchRow;
             }

             if (targetRow != null)
             {
                results.setRowNumber(targetRow);
                offEnd = OffEnd.NONE;
                betweenRows = true;
                afterReposition(true);
                return true;
             }
          }

          offEnd = OffEnd.BACK;
          betweenRows = false;
          afterReposition(false, oldCurrentRow);

          return false;
       }

       /**
        * Prepare to process a reposition request by releasing the current
        * records, if any, in each buffer managed by the query.  This occurs in
        * Progress unconditionally, even if the reposition does not actually move
        * the cursor from its current position.
        */
       protected void prepareReposition()
       {
          // Don't release buffer if we're processing a recursive reposition that
          // was triggered by a reposition notification.
          if (isRepositionNotificationActive())
          {
             return;
          }

          if (components != null)
          {
             for (int i = 0; i < components.size(); i++)
             {
                QueryComponent next = components.get(i);
                next.getBuffer().release(false);
             }
          }
       }

       /**
        * Check if any query component (added until the current call) has bound buffers.
        *
        * @return   See above.
        */
       protected boolean anyBoundBufferInComponent()
       {
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent qc = components.get(i);
             RecordBuffer buf = qc.getBuffer();
             RecordBuffer defBuf = qc.getDefinitionBuffer();
             if (!buffersMatch(buf, defBuf))
             {
                return true;
             }
          }

          return false;
       }

       /**
        * Check if the lock should be updated for any component when fetching. This is used
        * to detect if we can bypass fetching to satisfy locks. If the provided lock type is
        * {@link LockType#NONE}, no lock update should happen. If the component specific
        * lock mode is used, ensure that there is no particular (SHARE or EXCLUSIVE) lock
        * request for any component, which may trigger fetching.
        *
        * @param   lockType
        *          The lock mode for the records after query execution.
        *
        * @return  {@code true} if one of the components has a record which should be fetched
        *          with another lock type.
        */
       protected boolean shouldUpdateAnyLock(LockType lockType)
       {
          if (lockType == LockType.NONE)
          {
             return false;
          }

          if (lockType != null)
          {
             return true;
          }

          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             LockType compLock = comp.getLockType();

             if (compLock != LockType.NONE)
             {
                return true;
             }
          }

          return false;
       }

       /**
        * Decide whether this query should have a {@link ForwardResults} back-end results provider
        * or it should eventually toggle a {@link ScrollingResults}
        *
        *
        * @return   {@code true} if this will generate {@link ForwardResults}
        */
       protected boolean shouldExecuteForward()
       {
          return !isScrolling() && isIterating();
       }

       /**
        * Find the query component associated with the given record buffer.
        *
        * @param   buffer
        *          Record buffer.
        *
        * @return  Associated query component, or <code>null</code> if not found.
        */
       private QueryComponent findComponent(RecordBuffer buffer)
       {
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             QueryComponent comp = components.get(i);
             if (comp.getBuffer() == buffer)
             {
                return comp;
             }
          }

          return null;
       }

       /**
        * Assemble the FQL which is submitted to Hibernate in order to execute
        * the database query underlying this object. The FQL generated uses the
        * theta join style and includes a select clause, a from clause, an
        * optional where clause, and an order by clause (mandatory in all cases
        * except a unique find).
        *
        * @return  String
        *          A formatted FQL statement.
        *
        * @throws  PersistenceException
        *          if the sort criteria reference unqualified or unrecognized
        *          buffer names.
        */
       private String assembleFQL()
       throws PersistenceException
       {
          StringBuilder buf = new StringBuilder();

          translateWhere();

          if (FwdServerJMX.JMX_DEBUG)
          {
             QUERY_ASSEMBLE.timer(() -> assembleFQLImpl(buf));
          }
          else
          {
             assembleFQLImpl(buf);
          }

          return buf.toString().trim();
       }

       /**
        * Assemble the FQL which is submitted to Hibernate in order to execute
        * the database query underlying this object. The FQL generated uses the
        * theta join style and includes a select clause, a from clause, an
        * optional where clause, and an order by clause (mandatory in all cases
        * except a unique find).
        *
        * @param   buf
        *          The buffer where to assemble the FQL.
        *
        * @throws  PersistenceException
        *          if the sort criteria reference unqualified or unrecognized
        *          buffer names.
        */
       private void assembleFQLImpl(StringBuilder buf)
       throws PersistenceException
       {
          // need to put together order by clause first, as the sort criteria determined in this analysis may be
          // needed when composing the from-clause and where clause
          ArrayList<SortCriterion> sortCriteria = assembleOrderByClause();

          assembleSelectClause(buf);
          buf.append(assembleFromAndWhereClauses(sortCriteria).toFinalExpression());

          if (isOrderByRequired())
          {
             generateOrderBy(sortCriteria, buf);
          }
       }

       /**
        * Function for deciding whether the ORDER BY clause is required in the FQL.
        * For the moment, the ORDER BY clause can be skipped if this query will retrieve zero or one records.
        * In other words, if this query will retrieve a unique finding, no ORDER BY is necessary.
        *
        * @return  {@code true} if the ORDER BY clause is necessary for the query,
        *          {@code false} otherwise.
        */
       private boolean isOrderByRequired()
       throws PersistenceException
       {
          ArrayList<QueryComponent> components = components();
          for (int i = 0; i < components.size(); i++)
          {
             if (!components.get(i).getFQLPreprocessor().isUniqueFind(true))
             {
                return true;
             }
          }

          return false;
       }

       /**
        * Validate a row identifier provided to a reposition request to ensure
        * that it does not represent the unknown value.
        *
        * @param   value
        *          ID to be validated.
        *
        * @return  <code>true</code> if the number is valid;  <code>false</code>
        *          if it is invalid, but we are in silent error mode.
        *
        * @throws  ErrorConditionException
        *          if the number is invalid and we are not in silent error mode.
        */
       private boolean validateRepositionByID(rowid value)
       {
          String msg = "Could not evaluate rowid during REPOSITION";

          return validateRepositionImpl(value, 7330, msg);
       }

       /**
        * Validate a record identifier provided to a reposition request to ensure
        * that it does not represent the unknown value.
        *
        * @param   value
        *          ID to be validated.
        *
        * @return  <code>true</code> if the number is valid;  <code>false</code>
        *          if it is invalid, but we are in silent error mode.
        *
        * @throws  ErrorConditionException
        *          if the number is invalid and we are not in silent error mode.
        */
       private boolean validateRepositionByID(recid value)
       {
          String msg = "Could not evaluate rowid during REPOSITION";

          return validateRepositionImpl(value, 7330, msg);
       }

       /**
        * Validate a numeric value provided to a reposition request to ensure
        * that it does not represent the unknown value.
        *
        * @param   value
        *          ID or row number/offset to be validated.
        *
        * @return  <code>true</code> if the number is valid;  <code>false</code>
        *          if it is invalid, but we are in silent error mode.
        *
        * @throws  ErrorConditionException
        *          if the number is invalid and we are not in silent error mode.
        */
       private boolean validateReposition(NumberType value)
       {
          String msg = "Could not evaluate reposition amount for query";

          return validateRepositionImpl(value, 3165, msg);
       }

       /**
        * Validate a numeric value provided to a reposition request to ensure
        * that it does not represent the unknown value.
        *
        * @param   value
        *          ID or row number/offset to be validated.
        * @param   errorNum
        *          Error number if validation fails.
        * @param   errorMsg
        *          Error message if validation fails.
        *
        * @return  <code>true</code> if the number is valid;  <code>false</code>
        *          if it is invalid.
        *
        * @throws  ErrorConditionException
        *          if the number is invalid and we are not in silent error mode.
        */
       private boolean validateRepositionImpl(BaseDataType value, int errorNum, String errorMsg)
       {
          if (value.isUnknown())
          {
             ErrorManager.displayError(errorNum, errorMsg, false);

             if (LOG.isLoggable(Level.SEVERE))
             {
                LOG.log(Level.SEVERE, errorMsg);
             }

             return false;
          }

          return true;
       }

       /**
        * Indicate whether the full result set for this query has been cached.
        *
        * @return  {@code true} if there is a result set and it has been cached.
        */
       private boolean isResultSetCached()
       {
          return results != null && results.isResultSetCached();
       }

       /**
        * Special {@link Results} implementation for template records. Since a query on rowid/recid
        * will return only a single row, this class is built around its {@code Long} rowid
        * and provide fast access for a {@link PreselectQuery} aware of this query type to iterate
        * on this single-row result.
        */
       private static class TemplateResults
       implements Results
       {
          /** The id of the template record to be loaded. */
          private final Long templateRowid;

          /**
           * The position of the cursor. This set of results always contain a single record, at
           * position 0. The natural starting position is -1, before the first row. The value 1 is
           * just after the last row. The class will maintain one of these three states.
           */
          private int cursor = -1;

          /**
           * The constructor. Builds a {@link Results} specific to a template rowid.
           *
           * @param   templateRowid
           *          The rowid of the template record. Immutable.
           */
          public TemplateResults(Long templateRowid)
          {
             this.templateRowid = templateRowid;
          }

          /**
           * Move cursor to the first results row.
           *
           * @return  {@code true} if there are any results.
           */
          @Override
          public boolean first()
          {
             cursor = 0;
             return true;
          }

          /**
           * Move cursor to the last results row.
           *
           * @return  {@code true} if there are any results.
           */
          @Override
          public boolean last()
          {
             cursor = 0;
             return true;
          }

          /**
           * Move cursor to the next results row.
           *
           * @return  {@code true} if there is a result under the cursor under the move.
           */
          @Override
          public boolean next()
          {
             ++ cursor;
             normalize();
             return isFirst();
          }

          /**
           * Move cursor to the previous results row.
           *
           * @return  {@code true} if there is a result under the cursor under the move.
           */
          @Override
          public boolean previous()
          {
             -- cursor;
             normalize();
             return isFirst();
          }

          /**
           * Is the cursor on the first row in the results set?
           *
           * @return  {@code true} if the cursor is on the first row.
           */
          @Override
          public boolean isFirst()
          {
             return cursor == 0;
          }

          /**
           * Is the cursor on the last row in the results set?
           *
           * @return  {@code true} if the cursor is on the first row.
           */
          @Override
          public boolean isLast()
          {
             return cursor == 0;
          }

          /**
           * Returns the row at the current position. It can be either only an array of PK or
           * an array of DMOs.
           *
           * @param   forceOnlyId
           *          Indicates whether it's only the primary keys that must be returned,
           *          or original data found is returned (either PK or DMO).
           *
           * @return  Object array or {@code null}.
           */
          @Override
          public Object[] get(boolean forceOnlyId)
          {
             return (cursor == 0) ? new Object[] { templateRowid } : null;
          }

          /**
           * Get the object at the current result row, at the specified column.
           *
           * @param   column
           *          Zero-based index column of the desired object.
           *
           * @return  Object at {@code column} or {@code null}.
           */
          @Override
          public Object get(int column)
          {
             return (cursor == 0 && column == 0) ? templateRowid : null;
          }

          /**
           * Get the primary key ID at the current result row, at the specified
           * column.
           *
           * @param   column
           *          Zero-based index column of the desired ID.
           *
           * @return  ID of the record or {@code null}.
           */
          @Override
          public Long getID(int column)
          {
             return (cursor == 0 && column == 0) ? templateRowid : null;
          }

          /**
           * Get the row number currently under the cursor.
           *
           * @return  Zero-based index of the current row, or {@code -1} if the cursor is not
           *          currently on a result.
           */
          @Override
          public int getRowNumber() { return cursor == 0 ? 0 : -1; }

          /**
           * Set the row number currently under the cursor.
           *
           * @param   row
           *          Zero-based index of the row to be set as the current row.
           *
           * @return  {@code true} if there is a row at the specified row number; else {@code false}.
           */
          @Override
          public boolean setRowNumber(int row)
          {
             cursor = row;
             normalize();

             return isFirst();
          }

          /**
           * Scroll the cursor ahead by the specified number of rows.  If the number is negative, the
           * cursor is moved backward.  This may put the cursor off the end (either end) of the
           * query's results.
           *
           * @param   rows
           *          Number of rows to jump ahead or back.
           *
           * @return  {@code true} if there is a row at the new location; else {@code false}.
           */
          @Override
          public boolean scroll(int rows)
          {
             cursor += rows;
             normalize();

             return isFirst();
          }

          /**
           * Reset the cursor to its natural starting position, before the first result row.
           */
          @Override
          public void reset()
          {
             cursor = -1;
          }

          /**
           * Invoked when the current Hibernate session is about to close.  In this case this is a
           * no-op since the template record is not linked to Hibernate.
           */
          @Override
          public void sessionClosing()
          {
          }

          /**
           * Clean up and release any resources which this object is holding, such as open result
           * sets. In this case it does nothing as no resources are held.
           */
          @Override
          public void cleanup()
          {
          }

          /**
           * Internal worker that keeps the correctness of cursor position: always in one of the
           * positions: -1 = before the first record, 0 = the only record is selected, +1, after the
           * last record.
           */
          private void normalize()
          {
             if (cursor > 1)
             {
                cursor = 1;
             }
             else if (cursor < -1)
             {
                cursor = -1;
             }
             // else cursor is -1 (before), 0 (on single record), 1 (after)
          }
       }
    }