# CompoundQuery.java

    /*
    ** Module   : CompoundQuery.java
    ** Abstract : Implementation of an iterative, multi-table query
    **
    ** Copyright (c) 2004-2025, Golden Code Development Corporation.
    **
    ** -#- -I- --Date-- --JPRM-- -----------------------------------Description-----------------------------------
    ** 001 ECF 20051112   @23394 Created initial version. Implementation of an
    **                           iterative, multi-table query. Component
    **                           queries are added and the join of the results
    **                           is performed at the client (i.e., P2J server,
    **                           but database client).
    ** 002 ECF 20051216   @23746 Added scrolling support. Now extends the new
    **                           DynamicQuery base class, which provides
    **                           cursoring support services. Modified
    **                           retrieval logic to allow for repositioning.
    ** 003 ECF 20060104   @23826 Changes to support new hierarchy. Added
    **                           unsupported variants of unique() method
    **                           required by P2JQuery interface. Both throw
    **                           UnsupportedOperationException.
    ** 004 ECF 20060215   @24715 Added support for client-side where clause
    **                           processing. Also refitted to implement more
    **                           correct handling of substitution parameter
    **                           resolution for iterative looping queries.
    ** 005 ECF 20060531   @26849 Replaced EndConditionException with
    **                           QueryOffEndException. The latter is a more
    **                           specific version of the former, used to
    **                           indicate a query has run off the end of its
    **                           result set.
    ** 006 ECF 20060705   @27881 Fixed preselect mode of retrieve method.
    **                           Invocations of this method after the first
    **                           would always take the dynamic route rather
    **                           than loading records by IDs.
    ** 007 ECF 20060801   @28349 Added getBreakValue() method to meet
    **                           superclass requirement. Currently throws
    **                           UnsupportedOperationException.
    ** 008 ECF 20060925   @29962 Check for a pending error before retrieving a
    **                           record. Retrieval is not attempted if an
    **                           error is pending in the ErrorManager.
    ** 009 ECF 20061003   @30152 Backed out #009 (@29962). This logic is only
    **                           necessary for FindQuery and AdaptiveFind.
    ** 010 ECF 20061006   @30196 Implemented scrolling mode. When set to
    **                           scrolling mode, non-linear scrolling is
    **                           possible in preselect mode. Previously, only
    **                           linear scrolling from first to last was
    **                           possible. This is necessitated by the fact
    **                           that CompoundQuery may back a browse widget.
    **                           Scrolling support is actually implemented in
    **                           component queries. At this level, a flag is
    **                           simply set to enable the deferred setting of
    **                           scrolling mode on component queries.
    ** 011 ECF 20061019   @30525 Fixes to better support scrolling mode.
    **                           Modified the behavior of some state variables
    **                           to fix the use of AdaptiveQuery components in
    **                           scrolling mode.
    ** 012 ECF 20061204   @31574 Implemented is/setBrowsed() methods. Browse
    **                           association status is propagated to component
    **                           queries.
    ** 013 ECF 20070323   @32548 Fixed ClassCastException in setScrolling().
    ** 014 ECF 20070504   @33407 Fixed loadByIDs(). If a component query can
    **                           not load the given ID set, return false.
    ** 015 ECF 20070509   @33453 Fixed problem of resetting component queries
    **                           too early. Added notifyRepositionListeners()
    **                           implementation to avoid sending notifications
    **                           before query is fully initialized.
    ** 016 ECF 20070511   @33471 Fixed loadByIDs(). Reset component queries
    **                           before loading records.
    ** 017 ECF 20070518   @33678 Implement recordBuffers(). Required by the
    **                           RecordChangeListener interface, which is now
    **                           implemented by the superclass.
    ** 018 ECF 20070716   @34507 Add a cleanup() method implementation. All
    **                           component queries are marked as not being
    **                           standalone, so that they do not clean up
    **                           their resources too aggressively, but rather
    **                           defer to this object's cleanup mechanism.
    ** 019 ECF 20070726   @34720 Fixed problem of outer buffers being cleared
    **                           prematurely on loop iteration. When we begin
    **                           a new iteration series, we temporarily
    **                           disable RecordBuffer's default release on
    **                           iterate behavior, but only for the current
    **                           block scope.
    ** 020 ECF 20070821   @34906 Added support for temporary silent error
    **                           mode. This allows query substitution parms to
    **                           be processed safely during initialization of
    **                           a query.
    ** 021 ECF 20071011   @35443 Optimization to loadByIDs(). The outer
    **                           table's query does not need to be reset when
    **                           loading buffers by ID.
    ** 022 ECF 20071203   @36119 Integrated generics for RecordChangeListener
    **                           implementation.
    ** 023 ECF 20080112   @36997 Removed restriction of dealing only with IDs.
    **                           This query can now handle both Persistables
    **                           and primary key IDs when retrieving data.
    **                           This is necessary when the full record is
    **                           needed immediately, not just an ID, such as
    **                           in presort cases. Removed final modifier from
    **                           class.
    ** 024 ECF 20080509   @38241 Added open() method. Necessary for converted
    **                           OPEN QUERY statements.
    ** 025 ECF 20080509   @38251 Rolled back #024 (@38241). Caused regression.
    ** 026 ECF 20080510   @38479 Changed setScrolling(). Do not process method
    **                           body more than once.
    ** 027 SIY 20080625   @38922 Removed duplicate browsed flag which under
    **                           some circumstances may be set in opposite
    **                           state to identical flag in AbstractQuery
    **                           and disable correct browse notifications.
    ** 028 ECF 20081031   @40306 Minor memory improvement.
    ** 029 ECF 20081105   @40216 Overrode parent's open() method. Registers
    **                           all query components with the ChangeBroker at
    **                           the appropriate scope.
    ** 030 ECF 20081106   @40320 Fixed open(). In case of a scrolling query,
    **                           the CompoundQuery itself must be registered
    **                           with the ChangeBroker at the proper scope.
    ** 031 CA  20081210   @40837 Implemented CLOSE QUERY statement.
    ** 032 ECF 20090223   @41351 Fixed retrieveImpl(). Reset activeIndex on
    **                           FIRST/LAST navigation types to ensure we are
    **                           fetching all components' results.
    ** 033 ECF 20090312   @42980 Improved reposition support. Track number of
    **                           result rows visited, and which direction we go
    **                           off end. Implement loadRowAtCursor() for
    **                           scrolling queries.
    ** 034 ECF 20090703   @43056 Fixed loadByValues(). If an expected record can
    **                           not be loaded, MissingRecordException is thrown
    **                           by Joinable.load() and the backing buffer is
    **                           assumed by contract to be in unknown (safe) mode,
    **                           such that a request for the buffer record's
    **                           property values will return unknown value.
    ** 035 ECF 20090716   @43215 Ensure backing record buffers are initialized
    **                           before use.
    ** 036 CA  20090727   @43393 When the query goes off-end, all buffers for this
    **                           query are registered for off-end notifications,
    **                           in case of rollback (when current block is FOR
    **                           or FOR loop).
    ** 037 CA  20090731   @43465 Register the query for off-end listener
    **                           registration after block initialization (done
    **                           only for block types FOR and FOR EACH). Added
    **                           getOffEndListeners, which returns a list of all
    **                           off-end listeners for all used buffers.
    ** 038 SVL 20091030   @44288 getOffEnd() function was made public in order to
    **                           support API change in P2JQuery.
    ** 039 SIY 20100303   @44702 Allow enabling/disabling of accumulator processing.
    ** 040 ECF 20131028          Import change.
    ** 041 SVL 20140604          Implemented indexInformation.
    ** 042 SVL 20140909          Fixed some aspects of query invalidation. Implemented
    **                           peekNext/peekPrevious and deleteResultListEntry.
    ** 043 SVL 20150331          Added deleteResultListEntry(boolean).
    ** 044 ECF 20150520          Replaced Apache commons logging with J2SE logging; cleaned up format
    **                           issues.
    ** 045 OM  20150421          resolveOnce parameter ignored for OPEN QUERY FOR constructs.
    ** 046 SVL 20150722          Updated notifyRepositionListeners signature.
    ** 047 ECF 20150806          Implemented server-join optimization algorithm. Determines whether
    **                           it is possible (and advisable) to join groups of compound query
    **                           components at the database server and does so.
    ** 048 ECF 20150906          Reimplemented silent error mode processing of query substitution
    **                           parameters. Fixed NPE related to outer joins.
    ** 049 ECF 20160105          Fixed out-of-bounds error in indexInformation method.
    ** 050 EVL 20160217          Clean up comments from symbols invalid for Solaris to compile.
    ** 051 ECF 20160206          Minor logging change.
    ** 052 SVL 20160314          Support for reference row.
    ** 053 ECF 20160320          Always return null from getBreakValue to disallow shift out of
    **                           dynamic mode.
    ** 054 SVL 20160526          Updated due to changes in PresortCompoundQuery. Fixed the case when
    **                           a record from a cached row in a scrolling query was deleted.
    ** 055 SVL 20160608          The query can handle null IDs (for scrolling compound queries
    **                           with OUTER join).
    ** 056 OM  20160629          HQLPreprocessor needs the list of fields of the current index.
    ** 057 SVL 20160721          Proper reset on query reopen.
    ** 058 ECF 20160720          Fixed buffer lookup during HQL preprocessing. Added unsupported
    **                           operation implementations of several Joinable methods.
    ** 059 SVL 20160923          Proper setScrolling() for components.
    ** 060 CA  20160928          H059 changed addComponent() so that an outer-join component is marked
    **                           as inner only if it uses one or more buffers as the current component.
    **                           H060 changes retrieveImpl to fix OUTER-JOIN cases exposed by H059.
    ** 061 GES 20160919          All initialization is now done with a default constructor followed
    **                           by a call to initialize(). This enables query instance members in
    **                           converted code to be local variables that are effectively final.
    **                           This is needed to support lambdas without moving queries to be
    **                           instance members. Queries as instance members breaks recursion
    **                           use cases.
    ** 062 CA  20161006          Update class and addComponent javadoc for H060 changes.
    ** 063 CA  20161012          Fixed scope processing when the query is from a persistent
    **                           procedure, and is part of a QUERY resource.
    ** 064 OM  20171129          releaseBuffers() is implemented from P2JQuery.
    **     OM  20171212          HQLPreprocessor.get() signature change.
    ** 065 ECF 20180201          Defer optimization until after first result is retrieved to prevent
    **                           overly aggressive buffer release in the event the query runs off end
    **                           immediately. Cleaned up inheritance hierarchy.
    **     ECF 20180220          The current deferred optimization implementation causes regressions.
    **                           It is being left intact, but disabled for the time being.
    ** 066 ECF 20180418          Optimizer fix.
    ** 067 ECF 20180618          Removed releaseBuffers from loadByValues when data is null.
    ** 068 OM  20180515          Added dynamic filters and dynamic sort. [preselected] is reset when
    **                           query is open fresh.
    ** 069 ECF 20180727          Verify an outer-joined query buffer has a record available before
    **                           invoking current() on it.
    ** 070 ECF 20180820          Added logging to Optimizer.
    **     OM  20180901          Improved dynamic filtering.
    **     OM  20180924          Filtering on decimal fields uses STRING(f, format) function.
    ** 071 OM  20181025          When peeking on a presorted set, do not add result to cursor because
    **                           it already has the full set of results.
    ** 072 ECF 20190302          Renamed setExternalBuffers to addExternalBuffers.
    ** 073 OM  20190330          Ignore close request when query not fully initialized.
    ** 074 ECF 20190414          Ensure presorted query's components fetch full records.
    ** 075 ECF 20190813          Change to RecordBuffer.release API.
    ** 076 ECF 20190802          Fixed cleanup processing to prevent a listener leak in ChangeBroker.
    **     CA  20190812          Cleanup components on close.
    **     EVL 20190815          Fix for NPE on query cleanup when components have been optimized.
    ** 077 SVL 20191002          Implemented createResultListEntry.
    ** 078 CA  20191005          A query must know if it is dynamic, when closing.
    ** 079 AIL 20200610          Added support for afterReposition: validate buffers and release last buffer.
    **     SVL 20200619          Fixed issues caused by the previous entry.
    ** 080 ECF 20200906          New ORM implementation.
    ** 081 OM  20201012          Use locally cached meta information instead of map lookup.
    **     ECF 20210502          Fixed tableCount when closing query.
    **     ECF 20210506          Use RecordBuffer.getDmoInfo() getter instead of direct field access to prevent
    **                           NPE in proxy case.
    **     ECF 20210712          When optimizing, use the iteration type of the original, outermost query in the
    **                           optimized join.
    **     ECF 20210723          Changed signature of query close worker method; do not removed query components
    **                           when closing the query, remove them in initialize method instead.
    **     OM  20220112          Added implementation of SKIP-DELETED-RECORD attribute. Fixed management of
    **                           'browsed' property.
    **     CA  20220214          Register a finalizable cleaner for the compound query (required when is used in
    **                           its own block and not via a QUERY resource).
    **     IAS 20220510          Resetting activeIndex before iterating over the components for current(LockType)
    **     IAS 20220707          Resetting nullOuter on new iteration step.
    **     CA  20220901          Refactored scope notification support: ScopeableFactory was removed, and the
    **                           registration is now specific to each type of scopeable.  For each case, the block
    **                           will be registered for scope support (for that particular scopeable) only when
    **                           the scopeable is 'active' (i.e. unnamed streams or accumulators are used).  This
    **                           allows a lazy registration of scopeables, to avoid the unnecessary overhead of
    **                           processing all the scopeables for each and every block.
    **     OM  20221103          New class names for FQLPreprocessor, FQLExpression, FQLBundle, and FQLCache.
    **     CA  20221130          Refactored the WHERE clause translation (when the bound and definition buffers
    **                           are not the same), to be aware of the external buffers (i.e. added for CAN-FIND
    **                           sub-select clauses).  The translate will be performed before the query is being
    **                           executed.
    **     SVL 20230108          Improved performance by replacing some "for-each" loops with indexed "for" loops.
    **     SVL 20230113          More performance improvements by replacing some "for-each" loops with indexed
    **                           "for" loops.
    ** 082 CA  20230221          Javadoc fixes.
    ** 083 IAS 20230405          Fixed FILL support for recursive DATA-RELATION
    ** 084 OM  20230404          The join navigation key may contain null/unknown values.
    ** 085 GBB 20230512          Logging methods replaced by CentralLogger/ConversionStatus.
    ** 086 RAA 20230522          Added instance of P2JQueryStatistics for storing some statistics.
    **     RAA 20230526          Function processComponent now goes through P2JQueryLogger
    **                           in case logging is intended.
    ** 087 OM  20230410          Removed outer-join to inner-join optimisation in addComponent().
    ** 088 CA  20230414          In a compound query, the component groups must always have the EACH components at
    **                           the beginning, in a contiguous prefix.  They can not be preceded by FIRST or
    **                           other single-record components.
    ** 089 RAA 20230607          P2J Queries are now executed in P2JQueryExecutor instead of P2JQueryLogger.
    ** 090 AB  20230803          Deleted part of isServerJoinPossible() that sets "outer" components automatically
    **                           as not join-able.
    **     AB  20230828          Modified isServerJoinPossible() to not allow combining components when the total
    **                           number of columns is bigger than the limit of the dialect.
    **     AB  20230828          Modified isServerJoinPossible() to log the case when we have to many columns
    **                           in select.
    **     AL2 20230828          Made optimized compound components outer if their first component is outer.
    ** 091 IAS 20230921          Added initializa2f() method.
    ** 092 AB  20230925          Modified isServerJoinPossible() to not allow optimizing when the keyword
    **                           "contains" is present in the FQL.
    **     AB  20230926          Added checks related to "contains" with non-constant expression
    **                           in isServerJoinPossible().
    **     AB  20230929          Changed iterating method from for-each to for in isServerJoinPossible(), fixed
    **                           spacing in javadoc.
    ** 093 RAA 20231006          Added getCombinedFqls and getCombinedExecutionTime.
    **     RAA 20231011          Removed databaseName from getCombinedExecutionTime.
    **     RAA 20231122          Removed database caching from retrieveImpl. Refactored getCombinedExecutionTime
    **                           to use indexed looping.
    ** 093 ES  20231206          Changed first, last, next, previous, current, unique methods to return boolean,
    **                           instead of void. These methods now returns true if operation was successful, and false
    **                           not and lenientOffEnd is set on true, otherwise it thrown an QQE exception.
    ** 094 ASL 20240105          Fixed switch statement CURRENT case mistake to call
    **                           current() instead of unique().
    ** 095 ASL 20240111          Added server join constraint for single component groups containing outer joins.
    ** 096 OM  20240106          Do not optimize queries with "contains()" function in their predicate.
    ** 097 CA  20240314          Mark the query to retrieve the entire result set (useful for FILL operations).
    ** 098 CA  20240320          Avoid BDTs within internal FWD runtime.  Replaced iterators with Java 'for',
    **                           where it applies.
    ** 099 ES  20240327          Added cases for invalidation of of join in PreselectQuery.
    ** 100 ES  20240404          Persist the record in buffer in case of iteration in FOR EACH with break by statement.
    **                           Setting iteration and presort flag for query components.
    ** 101 CA  20240422          Allow the query to be reset to its initialization state, so the same instance can
    **                           be used to execute child buffer FILL operations.
    ** 102 CA  20240425          Do not join components which use the same SQL table.
    ** 103 AL2 20240424          Detect if the optimized query is going to be invalidated early.
    **                           If so, deny the optimization.
    ** 104 CA  20240809          Avoid iterator usage, for performance improvement.
    ** 105 DDF 20230712          Added setForwardOnly() method.
    **     DDF 20230713          Complete setForwardOnly() method since utility methods for the FORWARD-ONLY
    **                           attribute were moved to AbstractQuery.
    **     DDF 20230717          Added missing @Override annotation.
    **     DDF 20240130          Set the forwardOnly flag when adding a new component.
    ** 106 SB  20241105          Release the last bound buffer of a query if REPOSITION-BY-ID has an unknown
    **                           argument. Refs #9199.
    ** 107 OM  20241128          Multi-tenant runtime support: selected the proper persistence context, eventually
    **                           based on [sharedDb] parameter.
    **                           Implementation of [setMultiTenantAlternative] method.
    ** 108 AOG 20250127          Cache the entire result set when last(boolean, LockType) is called and
    **                           the query is attached to a browse.
    ** 109 AL2 20250416          Enforce database join if force-database-join attribute is set.
    ** 110 RNC 20250428          Always register the dynamic query as a global scope listener.
    ** 111 SP  20250509          Reload component query buffers up to activeIndex for scrolling queries.
    ** 112 AL2 20250520          Better support for CURRENT to honor record set by previous loadByValues.
    ** 113 AL2 20250521          Loading a prefix of buffers should honor the provided lock type.
    */

    /*
    ** This program is free software: you can redistribute it and/or modify
    ** it under the terms of the GNU Affero General Public License as
    ** published by the Free Software Foundation, either version 3 of the
    ** License, or (at your option) any later version.
    **
    ** This program is distributed in the hope that it will be useful,
    ** but WITHOUT ANY WARRANTY; without even the implied warranty of
    ** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    ** GNU Affero General Public License for more details.
    **
    ** You may find a copy of the GNU Affero GPL version 3 at the following
    ** location: https://www.gnu.org/licenses/agpl-3.0.en.html
    **
    ** Additional terms under GNU Affero GPL version 3 section 7:
    **
    **   Under Section 7 of the GNU Affero GPL version 3, the following additional
    **   terms apply to the works covered under the License.  These additional terms
    **   are non-permissive additional terms allowed under Section 7 of the GNU
    **   Affero GPL version 3 and may not be removed by you.
    **
    **   0. Attribution Requirement.
    **
    **     You must preserve all legal notices or author attributions in the covered
    **     work or Appropriate Legal Notices displayed by works containing the covered
    **     work.  You may not remove from the covered work any author or developer
    **     credit already included within the covered work.
    **
    **   1. No License To Use Trademarks.
    **
    **     This license does not grant any license or rights to use the trademarks
    **     Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
    **     of Golden Code Development Corporation. You are not authorized to use the
    **     name Golden Code, FWD, or the names of any author or contributor, for
    **     publicity purposes without written authorization.
    **
    **   2. No Misrepresentation of Affiliation.
    **
    **     You may not represent yourself as Golden Code Development Corporation or FWD.
    **
    **     You may not represent yourself for publicity purposes as associated with
    **     Golden Code Development Corporation, FWD, or any author or contributor to
    **     the covered work, without written authorization.
    **
    **   3. No Misrepresentation of Source or Origin.
    **
    **     You may not represent the covered work as solely your work.  All modified
    **     versions of the covered work must be marked in a reasonable way to make it
    **     clear that the modified work is not originating from Golden Code Development
    **     Corporation or FWD.  All modified versions must contain the notices of
    **     attribution required in this license.
    */

    package com.goldencode.p2j.persist;

    import java.util.*;
    import java.util.function.*;
    import java.util.logging.*;
    import com.goldencode.p2j.persist.event.*;
    import com.goldencode.p2j.persist.hql.*;
    import com.goldencode.p2j.persist.lock.*;
    import com.goldencode.p2j.persist.orm.P2JQueryExecutor;
    import com.goldencode.p2j.util.*;
    import com.goldencode.p2j.util.ErrorManager;
    import com.goldencode.p2j.util.logging.*;

    /**
     * A client-side (relative to the database server) implementation of a
     * multi-table, join query.  Queries of individual tables are added via one
     * of the <code>addComponent</code> method variants.  This query type can
     * combine both {@link PreselectQuery}s and {@link RandomAccessQuery}s as its
     * components.  However, preselected tables must always be to the left of
     * (i.e., outward from) non-preselected tables in the query.  That is, a
     * <code>PreselectQuery</code> object can never be added, once a
     * <code>RandomAccessQuery</code> object has been added to a compound query.
     * <p>
     * Inner and left outer joins are supported.  However, inner joins must
     * always occur to the left (i.e., outward from) outer joins.  If, at the
     * time a component query specified as an inner join is added, there are
     * previously added outer-join components which in their <code>where</code>
     * clause use one or more buffers as the inner-join component's
     * <code>where</code> clause, these outer-join components will automatically
     * be converted to an inner join, to satisfy the condition that all inner
     * joins are on the left of the outer-join components.
     * <p>
     * This query retrieves result records in two modes:  iterating and random
     * access.  Iterating mode is active when using the {@link #iterate} method
     * to advance the query to the next, composite row of records in the result
     * set.  Random access mode is active when using any of the other retrieval
     * methods (<code>first</code>, <code>last</code>, <code>next</code>,
     * <code>previous</code>, <code>current</code>).  The difference between the
     * modes is that iterating mode always terminates with a
     * <code>QueryOffEndException</code> when no further record retrieval is
     * possible;  random access mode does not.  Random access mode thus allows
     * client code to advance off the end of the result set and continue
     * processing.
     * <p>
     * <code>CompoundQuery</code> abstracts the use of multiple query
     * implementations to achieve a dynamic, multi-table query.  It delegates the
     * record retrieval work to the underlying queries, but manages the overall
     * navigation of the results.  Delegate queries must implement the {@link
     * Joinable} interface.  Each "row" in the "result set" is a logical
     * construct, the virtual composite of multiple record buffers.  Advancing to
     * a new row (be it forwards, backwards, first, last) means advancing the
     * most specific query possible to the next matching record in its
     * predetermined progression.  Once the most specific query can no longer
     * advance, the next most specific query is advanced, the most specific query
     * is reset to its starting state, and we try to find a joining match for the
     * most specific query again, given that its outer query has a new value with
     * which to join.  This continues until the most specific possible query has
     * advanced, or we have moved to our least specific query component, and no
     * further match is possible.  At this point, the query is "off-end".  For
     * iterating mode, this is the end of query execution and an end condition is
     * raised.  In random access mode, further navigation is possible.
     * <p>
     * <b>Substitution Parameters</b><br>
     * Query substitution parameters require different modes of resolution,
     * depending upon the type of parameter and the nature of the query.  A query
     * may be set at construction to resolve all of its parameters once, up front
     * or iteratively, each time an iterative loop construct is entered.  The
     * former corresponds with Progress' OPEN QUERY style queries;  the latter
     * corresponds with all other iterative record retrieval constructs in
     * Progress (e.g., FOR EACH, DO PRESELECT, REPEAT PRESELECT, REPEAT FOR).
     * <p>
     * The number of times a parameter is resolved and the timing of such
     * resolution becomes important if an iterative record retrieval construct
     * modifies the values of such parameters during record retrieval.  This may
     * occur, for example, if the body of an outer FOR loop modifies the value of
     * a variable used in the where clause of an inner FOR loop.  While less of
     * an issue for preselect type queries, these can still be affected if the
     * where clause of a nested query component invokes a client-side function,
     * which may in turn modify state on which the query or related logic depends.
     * <p>
     * The following conversion rules apply:
     * <pre>
     * ------------------------------------------------------------------------
     * Progress Construct                             CompoundQuery Modes
     *                                        resolveOnce             preselect
     * ------------------------------------------------------------------------
     * OPEN QUERY FOR                         true (forced to false)  false
     * OPEN QUERY PRESELECT                   true (ignored?)         true
     * FOR {EACH | FIRST | LAST}              false                   false
     * DO PRESELECT                           false                   true
     * REPEAT PRESELECT                       false                   true
     * ------------------------------------------------------------------------
     * </pre>
     */
    public class CompoundQuery
    extends DynamicQuery
    implements QueryConstants,
               RecordChangeListener
    {
       /** Logger */
       private static final CentralLogger log = CentralLogger.get(CompoundQuery.class.getName());

       /** Defer optimization until after first composite row has been retrieved (experimental) */
       // TODO: current implementation of this feature introduces regressions; needs to be fixed
       private static final boolean deferOptimization = false;

       /**
        * Determines if the current row was deleted by DELETE-RESULT-LIST-ENTRY. Note that if the
        * query is off-end, it also can be "deleted".
        */
       protected boolean currentRowDeleted = false;

       /** Determines if we have fetched a result row. Only for non-scrolling queries. */
       protected boolean foundFirst = false;

       /** Determines if this will be unregistered from ChangeBroker on cleanup. */
       protected boolean unregisterOnCleanup = false;

       /** Force preselection of all results? */
       private boolean preselect;

       /** Query components originally added to the compound query */
       private final ArrayList<CompoundComponent> originalComponents = new ArrayList<>(2);

       /** Resolve arguments once, up front, or on every loop cycle? */
       private boolean resolveOnce;

       /** Query components currently in use by the compound query */
       private ArrayList<CompoundComponent> components = null;

       /** Have results been preselected? */
       private boolean preselected = false;

       /** Current index position in each component query loop */
       private int[] counters = null;

       /**
        * Specifies if snapshot values (values at the time when the record was loaded) should be used
        * as FQL sub-query parameters for specific component instead of the actual values. The array
        * has size <code>components.size() - 1</code> and excludes the first (outermost) component
        * because it cannot have FQL which includes fields of an outer component.
        */
       private boolean[] useSnapshot = null;

       /** Index of component query currently being iterated */
       private int activeIndex = -1;

       /** Total number of tables involved in this compound query */
       private int tableCount = 0;

       /** Postpone accumulation on retrieve. */
       private boolean postponeAccumulation = false;

       /** Map of all buffers participating in this query to their 0-based index positions */
       private Map<RecordBuffer, Integer> buffersMap = null;

       /** Has query been registered for resource cleanup? */
       private boolean regCleaner = false;

       /** Flag indicating the entire result set will be retrieved (useful for FILL operations). */
       private boolean forceFullResults = false;

       /** Cleaner executed when this query's block is exit. */
       private final Finalizable cleaner = new Finalizable()
       {
          public void finished() { cleanup(); }
          public void iterate() {}
          public void retry() {}
          public void deleted() {}
       };

       /** Flag indicated if the WHERE clause has been translated. */
       private boolean whereTranslated = false;

       /**
        * Default constructor. Initialization is deferred until <code>initialize()</code> is
        * called.
        */
       public CompoundQuery()
       {
       }

       /**
        * Clear the current results from the query, so the same query instance can be executed again, without
        * having to {@link #open() re-open} it.
        */
       @Override
       public void clearResults()
       {
          super.clearResults();

          ArrayList<CompoundComponent> components = getComponents();
          for (int i = 0; i < components.size(); i++)
          {
             CompoundComponent cc = components.get(i);
             cc.getQuery().clearResults();
          }
       }

       /**
        * Initialization logic.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @param    preselect
        *           {@code true} to force preselection of all results; {@code false} otherwise.
        *
        * @return   The query instance.
        */
       public CompoundQuery initialize(boolean resolveOnce, boolean preselect)
       {
          this.resolveOnce = false; // TODO: remove resolveOnce parameter completely
          this.preselect = preselect;

          // initialize will be invoked on an existing query after close(), when the query is re-opened in
          // dynamic mode (i.e., dynamically prepared), so reset any structural state here
          originalComponents.clear();
          tableCount = 0;

          TransactionManager.registerOffEndQuery(this);

          return this;
       }

       /**
        * Initialization logic.
        * <p>
        * This code is intentionally separated from the constructor. The purpose of this separation
        * is to allow construction to occur in the prior scope while initialization can still occur
        * within a method or lambda that is inside the next block's scope.
        *
        * @return   This query instance.
        */
       public CompoundQuery initialize2f()
       {
          return initialize(false, false);
       }

       /**
        * Create a cursor for this query and ensure that any component queries are scrollable, whether they
        * already have been added, or will be added in the future.
        */
       public void setScrolling()
       {
          if (!isScrolling())
          {
             super.setScrolling();

             if (components != null)
             {
                for (int i = 0; i < components.size(); i++)
                {
                   CompoundComponent component = components.get(i);
                   component.getQuery().setScrolling();
                }
             }

             for (int i = 0; i < originalComponents.size(); i++)
             {
                CompoundComponent originalComponent = originalComponents.get(i);
                originalComponent.getQuery().setScrolling();
             }
          }
       }

       /**
        * Set the forward-only attribute for any component queries to the given value.
        *
        * @param   value
        *          <code>true</code> to enable forward-only, <code>false</code> to disable it.
        */
       @Override
       public void setForwardOnly(boolean value)
       {
          super.setForwardOnly(value);

          if (components != null)
          {
             for (int i = 0; i < components.size(); i++)
             {
                CompoundComponent component = components.get(i);
                component.getQuery().setForwardOnly(value);
             }
          }

          for (int i = 0; i < originalComponents.size(); i++)
          {
             CompoundComponent originalComponent = originalComponents.get(i);
             originalComponent.getQuery().setForwardOnly(value);
          }
       }

       /**
        * Get the number of tables joined by this query.
        *
        * @return  Number of tables joined by this query.
        */
       public int getTableCount()
       {
          return tableCount;
       }

       /**
        * Get all the off-end listeners associated with this query.
        *
        * @return  the off-end listeners associated with this query.
        */
       public ArrayList<QueryOffEndListener> getOffEndListeners()
       {
          LinkedHashSet<QueryOffEndListener> listeners = new LinkedHashSet<>();

          List<CompoundComponent> compoundComponents = getComponents();
          for (int i = 0; i < compoundComponents.size(); i++)
          {
             CompoundComponent comp = compoundComponents.get(i);
             listeners.addAll(comp.getQuery().getOffEndListeners());
          }

          return new ArrayList<>(listeners);
       }

       /**
        * Mark the query to retrieve the entire result set (useful for FILL operations).
        */
       @Override
       public void hintFullResults()
       {
          this.forceFullResults = true;
       }

       /**
        * Add a joinable query to the compound query.  If this is not the first
        * component to be added to the query, its results will be joined to the
        * immediately previous component by an inner join.  This query will be
        * iterated using the <code>NEXT</code> iteration type.
        *
        * @param   query
        *          Query which will perform the fundamental work for this
        *          component when the compound query is iterated.
        *
        * @throws  IllegalStateException
        *          if this method is invoked during an iteration cycle.
        *
        * @see     #iterate
        */
       public void addComponent(Joinable query)
       {
          addComponent(query, NEXT, false);
       }

       /**
        * Add a joinable query to the compound query, and specify the nature of
        * the iteration which will be performed (first, last, next, etc.) on that
        * component, when the compound query is iterated.  If this is not the
        * first component to be added to the query, its results will be joined to
        * the immediately previous component by an inner join.
        *
        * @param   query
        *          Query which will perform the fundamental work for this
        *          component when the compound query is iterated.
        * @param   iteration
        *          A {@link QueryConstants constant} indicating the type of work
        *          to be performed upon an iteration of the compound query.  This
        *          parameter determines whether the underlying query will be asked
        *          to perform a first, last, next, previous, or unique record
        *          retrieval.
        *
        * @throws  IllegalStateException
        *          if this method is invoked during an iteration cycle.
        *
        * @see     #iterate
        */
       public void addComponent(Joinable query, int iteration)
       {
          addComponent(query, iteration, false);
       }

       /**
        * Add a joinable query to the compound query, and specify the nature of
        * the iteration which will be performed (first, last, next, etc.) on that
        * component, when the compound query is iterated.  If this is not the
        * first component to be added to the query, the type of join performed
        * with the immediately previous component is determined by the value of
        * <code>outer</code>.
        *
        * @param   query
        *          Query which will perform the fundamental work for this
        *          component when the compound query is iterated.
        * @param   iteration
        *          A {@link QueryConstants constant} indicating the type of work
        *          to be performed upon an iteration of the compound query.  This
        *          parameter determines whether the underlying query will be asked
        *          to perform a first, last, next, previous, or unique record
        *          retrieval.
        * @param   outer
        *          <code>true</code> if this component should be joined to the
        *          previously added component via a left outer join;
        *          <code>false</code> if an inner join is required.
        *          <p>
        *          If, at the time a component query specified as an inner join is added, there are
        *          previously added outer-join components which in their <code>where</code> clause
        *          use one or more buffers as the inner-join component's <code>where</code> clause,
        *          these outer-join components will automatically be converted to an inner join,
        *          to satisfy the condition that all inner joins are on the left of the outer-join
        *          components.
        *
        * @throws  IllegalStateException
        *          if this method is invoked during an iteration cycle.
        *
        * @see     #iterate
        */
       public void addComponent(Joinable query, int iteration, boolean outer)
       {
          if (activeIndex >= 0)
          {
             throw new IllegalStateException("Cannot add query component while iterating");
          }

          initializeComponentQuery(query);

          query.setSkipDeletedRecord(isSkipDeletedRecord());
          addCompoundComponent(new CompoundComponent(query, iteration, outer));
       }

       /**
        * Add one or more buffers to the list of those which this query references externally. These
        * typically will be buffers referenced by converted CAN-FIND expressions embedded in the
        * query's FQL expression, which are not the primary buffers associated with the query.
        * <p>
        * This method overrides the default implementation to set external buffers on the most
        * recently added component of this compound query. It therefore must be called immediately
        * after the target component is added and before the query is opened, optimized, or executed.
        *
        * @param   dmos
        *          One or more external buffers to add to the most recently added query component.
        *
        * @throws  IllegalStateException
        *          if called before a component has been added or after query has been optimized.
        */
       @Override
       public P2JQuery addExternalBuffers(DataModelObject... dmos)
       {
          if (components != null)
          {
             throw new IllegalStateException(
                "Cannot set external buffers on a query component once query has been optimized");
          }

          int size = originalComponents.size();
          if (size == 0)
          {
             throw new IllegalStateException("Add a query component before setting external buffers");
          }

          CompoundComponent comp = originalComponents.get(size - 1);
          comp.getQuery().addExternalBuffers(dmos);

          return this;
       }

       /**
        * Adds an alternative execution query, for the multi-tenant case in which the CAN-FIND nested buffer and
        * the main buffer of query component do not belong to same physical database.
        * <p>
        * This method will be generated only in case of multi-tenant projects, in the condition above-mentioned.
        *
        * @param   where
        *          FQL where clause.  May be {@code null}.
        * @param   whereExpr
        *          Client-side where clause expression. May be {@code null} but this is probably caused by an
        *          incorrect conversion.
        * @param   sort
        *          The sort order.
        *
        * @return  Self object so that the call to this method can be optionally chained with the constructor.
        */
       @Override
       public P2JQuery setMultiTenantAlternative(String where, Supplier<logical> whereExpr, String sort)
       {
          return setMultiTenantAlternative(where, whereExpr, sort, null);
       }

       /**
        * Adds an alternative execution query, for the multi-tenant case in which the CAN-FIND nested buffer and
        * the main buffer of query component do not belong to same physical database.
        * <p>
        * This method will be generated only in case of multi-tenant projects, in the condition above-mentioned.
        *
        * @param   where
        *          FQL where clause.  May be {@code null}.
        * @param   whereExpr
        *          Client-side where clause expression. May be {@code null} but this is probably caused by an
        *          incorrect conversion.
        * @param   sort
        *          The sort order.
        * @param   args
        *          Substitution parameters for FQL queries. These will be used if not overridden by a record
        *          retrieval method.
        *
        * @return  Self object so that the call to this method can be optionally chained with the constructor.
        */
       @Override
       public P2JQuery setMultiTenantAlternative(String where,
                                                 Supplier<logical> whereExpr,
                                                 String sort,
                                                 Object[] args)
       {
          if (components != null)
          {
             throw new IllegalStateException(
                   "Cannot setMultiTenantAlternative on a query component once query has been optimized");
          }

          int size = originalComponents.size();
          if (size == 0)
          {
             throw new IllegalStateException("Add a query component before setting external buffers");
          }

          CompoundComponent comp = originalComponents.get(size - 1);
          comp.getQuery().setMultiTenantAlternative(where, whereExpr, sort, args);

          return this;
       }

       /**
        * Setter for SKIP-DELETED-RECORD query attribute.
        *
        * @param   on
        *          New value for the attribute.
        */
       @Override
       public void setSkipDeletedRecord(logical on)
       {
          for (int i = 0; i < originalComponents.size(); i++)
          {
             CompoundComponent next = originalComponents.get(i);
             next.getQuery().setSkipDeletedRecord(on);
          }

          if (components != null && components != originalComponents)
          {
             for (int i = 0; i < components.size(); i++)
             {
                CompoundComponent next = components.get(i);
                next.getQuery().setSkipDeletedRecord(on);
             }
          }
       }

       /**
        * Indicate to this query that whether it is associated with a browse widget or not.
        *
        * @param   browsed
        *          {@code true} to indicate the query has an associated browse widget, else {@code false}.
        */
       @Override
       public void setBrowsed(boolean browsed)
       {
          super.setBrowsed(browsed);
          this.setComponentsBrowsed();
       }

       /**
        * Open the query once all query components have been added.
        */
       @Override
       public void open()
       {
          if (cursor != null)
          {
             cursor.reset();
          }
          currentRowDeleted = false;
          foundFirst = false;
          counters = null;
          useSnapshot = null;
          activeIndex = -1;
          preselected = false;

          boolean globalScope = registerRecordChangeListeners();

          super.open();

          if (preselect)
          {
             preselectResults();
          }

          initReferenceRowSupport(globalScope);
       }

       /**
        * Advance the compound query, triggering one or more of the underlying
        * queries to retrieve a new record.  This produces a virtual, composite
        * row of results which is the combination of the records retrieved by
        * each component query.
        *
        * @throws  QueryOffEndException
        *          if no more records can be retrieved by the underlying queries.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean iterate()
       {
          return retrieve(NEXT, null, true) != null;
       }

       /**
        * Retrieve the first virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrieve the first record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean first()
       {
          return first(false, null);
       }

       /**
        * Retrieve the first virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        *
        * @return  <code>true</code> if query could retrieve the first record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean first(boolean iterating)
       {
          return first(iterating, null);
       }

       /**
        * Retrieve the first virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the first record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean first(LockType lockType)
       {
          return first(false, lockType);
       }

       /**
        * Retrieve the first virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the first record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean first(boolean iterating, LockType lockType)
       {
          boolean done = false;

          if (isScrolling())
          {
             if (preselect)
             {
                preselectResults();
             }

             Object[] data = cursor.getFirst();
             if (data == null && cursor.isFirstRowDeleted() && cursor.isForwardScroll())
             {
                return next(iterating, lockType);
             }

             done = loadByValues(data, lockType, iterating);

             if (data != null && !done)
             {
                return next(iterating, lockType);
             }
          }

          if (!done)
          {
             Object[] ids = retrieve(FIRST, lockType, iterating);
             if (isScrolling() && !isPresorted())
             {
                cursor.addResultFirst(ids);
             }

             return ids != null;
          }

          return true;
       }

       /**
        * Retrieve the last virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrieve the last record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean last()
       {
          return last(false, null);
       }

       /**
        * Retrieve the last virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        *
        * @return  <code>true</code> if query could retrieve the last record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean last(boolean iterating)
       {
          return last(iterating, null);
       }

       /**
        * Retrieve the last virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the last record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean last(LockType lockType)
       {
          return last(false, lockType);
       }

       /**
        * Retrieve the last virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the last record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean last(boolean iterating, LockType lockType)
       {
          boolean done = false;

          if (isScrolling())
          {
             if (isBrowsed() && !cursor.isFullSet())
             {
                Object[] data = cursor.getNext();
                Object[] lastData = null;

                while (data != null)
                {
                   lastData = data;
                   data = cursor.getNext();
                }

                if (lastData != null)
                {
                   loadByValues(lastData, lockType, iterating);
                }

                Object[] ids = retrieve(NEXT, lockType, iterating);

                while (ids != null)
                {
                   cursor.addResultNext(ids);
                   ids = retrieve(NEXT, lockType, iterating);
                }
                // Mark that the set is full.
                cursor.addResultNext(null);
                // current row is null after addResultNext(null) and we need to go back to the last row
                previous(iterating, lockType);

                return true;
             }

             if (preselect)
             {
                preselectResults();
             }

             Object[] data = cursor.getLast();
             if (data == null && cursor.isFirstRowDeleted() && !cursor.isForwardScroll())
             {
                return previous(iterating, lockType);
             }

             done = loadByValues(data, lockType, iterating);

             if (data != null && !done)
             {
                return previous(iterating, lockType);
             }
          }

          if (!done)
          {
             Object[] ids = retrieve(LAST, lockType, iterating);
             if (isScrolling() && !isPresorted())
             {
                cursor.addResultLast(ids);
             }

             return ids != null;
          }

          return true;
       }

       /**
        * Retrieve the next virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrieve the next record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean next()
       {
          return next(isIterating(), null);
       }

       /**
        * Retrieve the next virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        *
        * @return  <code>true</code> if query could retrieve the next record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean next(boolean iterating)
       {
          return next(iterating, null);
       }

       /**
        * Retrieve the next virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the next record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean next(LockType lockType)
       {
          return next(false, lockType);
       }

       /**
        * Retrieve the next virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the next record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean next(boolean iterating, LockType lockType)
       {
          if (isScrolling())
          {
             if (preselect)
             {
                preselectResults();
             }

             Object[] data;

             do
             {
                data = cursor.getNext();

                if (data == null && (cursor.isFullSet() ||
                                     !cursor.isForwardScroll()) && cursor.isFoundFirst())
                {
                   // back off-end
                   releaseBuffers();

                   return false;
                }

                if (data != null)
                {
                   if (loadByValues(data, lockType, iterating))
                   {
                      return true;
                   }
                }

             } while (data != null);
          }

          Object[] ids = retrieve(NEXT, lockType, iterating);

          if (isScrolling() && !isPresorted())
          {
             cursor.addResultNext(ids);
          }

          return ids != null;
       }

       /**
        * Retrieve the previous virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrieve the previous record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean previous()
       {
          return previous(false, null);
       }

       /**
        * Retrieve the previous virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        *
        * @return  <code>true</code> if query could retrieve the previous record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean previous(boolean iterating)
       {
          return previous(iterating, null);
       }

       /**
        * Retrieve the previous virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the previous record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean previous(LockType lockType)
       {
          return previous(false, lockType);
       }

       /**
        * Retrieve the previous virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the previous record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean previous(boolean iterating, LockType lockType)
       {
          if (isScrolling())
          {
             if (preselect)
             {
                preselectResults();
             }

             Object[] data;

             do
             {
                data = cursor.getPrevious();

                if (data == null && (cursor.isFullSet() ||
                                     cursor.isForwardScroll()) && cursor.isFoundFirst())
                {
                   // front off-end
                   releaseBuffers();

                   if (iterating)
                   {
                      throw new QueryOffEndException("",
                            getOffEndListeners().toArray(new QueryOffEndListener[0]));
                   }
                   return false;
                }

                if (data != null)
                {
                   if (loadByValues(data, lockType, iterating))
                   {
                      return true;
                   }
                }

             } while (data != null);
          }

          Object[] ids = retrieve(PREVIOUS, lockType, iterating);
          if (isScrolling() && !isPresorted())
          {
             cursor.addResultPrevious(ids);
          }

          return ids != null;
       }

       /**
        * Retrieve the current virtual, composite row of results for the compound
        * query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @return  <code>true</code> if query could retrieve the current record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean current()
       {
          return current(null);
       }

       /**
        * Retrieve the current virtual, composite row of results for the compound
        * query, overriding the lock type to apply to each underlying query.
        * <p>
        * This is a combination of records retrieved by each component query.  If
        * the compound query involves outer joins, some of the underlying buffers
        * may be empty when this method returns.
        *
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *
        * @return  <code>true</code> if query could retrieve the current record successfully;
        *          <code>false</code> if unsuccessful and <code>lenientOffEnd</code> is set on true.
        *
        * @throws  ErrorConditionException
        *          if an underlying query triggers an error.
        */
       public boolean current(LockType lockType)
       {
          if (_isOffEnd())
          {
             ErrorManager.displayError(4114, "No query record is available", false);

             return false;
          }

          if (currentRowDeleted)
          {
             if (isScrolling())
             {
                ErrorManager.displayError(4114, "No query record is available", false);
             }

             return false;
          }
          activeIndex = 0;
          return retrieve(CURRENT, lockType, false) == null;
       }

       /**
        * Fetch the array of primary key IDs associated with the first result in
        * this query's result list.
        * <p>
        * This implementation actually updates the underlying record buffer(s).
        *
        * @return  Array of record IDs reflecting the first query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekFirst()
       {
          return retrieve(FIRST, LockType.NONE, false, true);
       }

       /**
        * Fetch the array of primary key IDs associated with the last result in
        * this query's result list.
        * <p>
        * This implementation actually updates the underlying record buffer(s).
        *
        * @return  Array of record IDs reflecting the last query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekLast()
       {
          return retrieve(LAST, LockType.NONE, false, true);
       }

       /**
        * Fetch the array of primary key IDs associated with the next result in this query's result
        * list.
        * <p>
        * This implementation actually updates the underlying record buffer(s). The difference between
        * this function and {@link #next()} function is that this function calls
        * {@link Scrollable#peekNext()} on sub-queries in order to get the next record from underlying
        * results, while {@link #next()} returns the next record after the current row.
        *
        * @return  Array of record IDs reflecting the next query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekNext()
       {
          Object[] res = retrieve(NEXT, LockType.NONE, false, true);
          if (isScrolling() && !isPresorted())
          {
             cursor.addResultNext(res);
          }

          return res;
       }

       /**
        * Fetch the array of primary key IDs associated with the previous result
        * in this query's result list.
        * <p>
        * This implementation actually updates the underlying record buffer(s). The difference between
        * this function and {@link #previous()} function is that this function calls
        * {@link Scrollable#peekPrevious()} on sub-queries in order to get the next record from
        * underlying results, while {@link #previous()} returns the next record after the current row.
        *
        * @return  Array of record IDs reflecting the previous query result, or
        *          <code>null</code> if there is no such result.
        */
       public Object[] peekPrevious()
       {
          Object[] res = retrieve(PREVIOUS, LockType.NONE, false, true);
          if (isScrolling() && !isPresorted())
          {
             cursor.addResultPrevious(res);
          }

          return res;
       }

       /**
        * Notify all registered listeners that query is repositioned or closed.
        * If this is invoked before any query components have been added to the
        * compound query, it is ignored.
        *
        * @param   closed
        *          A <code>true</code> value will inform all listeners that the
        *          query was explicitly closed.
        * @param   error
        *          <code>true</code> if an error happened during reposition.
        * @param   targetRepositionRow
        *          Row used as the target for reposition. If it was deleted, reposition with fetching
        *          may end up on a different row.
        */
       @Override
       public void notifyRepositionListeners(boolean closed, boolean error, int targetRepositionRow)
       {
          List<CompoundComponent> comps = getComponents();
          if (comps != null && !comps.isEmpty())
          {
             super.notifyRepositionListeners(closed, error, targetRepositionRow);
          }
       }

       /**
        * Report the record buffers for whose changes this object is interested
        * in listening.  This listener will be registered with the {@link
        * ChangeBroker} to receive notifications of any change to DMOs whose
        * interface types match those returned by the {@link
        * RecordBuffer#getDMOInterface} methods of the returned buffers.
        *
        * @return  An iterator on the record buffers stored in each of this
        *          query's components.
        */
       public Iterator<RecordBuffer> recordBuffers()
       {
          if (buffersMap != null)
          {
             return buffersMap.keySet().iterator();
          }

          final List<CompoundComponent> comps = getComponents();

          if (comps == null)
          {
             return Collections.emptyIterator();
          }

          Iterator<RecordBuffer> iter = new Iterator<RecordBuffer>()
          {
             private Iterator<CompoundComponent> compIter = comps.iterator();

             private RecordBuffer nextBuffer = null;

             private RecordBuffer[] bufs = null;

             private int index = -1;

             public boolean hasNext()
             {
                if (nextBuffer == null)
                {
                   advance();
                }

                return (nextBuffer != null);
             }

             public RecordBuffer next()
             {
                if (!hasNext())
                {
                   throw new NoSuchElementException();
                }

                try
                {
                   return nextBuffer;
                }
                finally
                {
                   nextBuffer = null;
                }
             }

             public void remove()
             {
                throw new UnsupportedOperationException();
             }

             private void advance()
             {
                if (bufs != null)
                {
                   if (++index < bufs.length)
                   {
                      nextBuffer = bufs[index];

                      return;
                   }

                   bufs = null;
                }

                // at this moment bufs is always null
                if (compIter.hasNext())
                {
                   CompoundComponent nextComp = compIter.next();
                   Joinable query = nextComp.getQuery();
                   bufs = query.getRecordBuffers();
                   index = -1;
                }
                else
                {
                   nextBuffer = null;
                   return;
                }

                // Assumes that array will not be empty or null.
                nextBuffer = bufs[++index];
             }
          };

          return iter;
       }

       /**
        * Respond to a record change event.
        *
        * @param   event
        *          Event which describes the DMO state change.
        *
        * @throws  PersistenceException
        *          if any error occurs while processing the state change event.
        */
       @Override
       public void stateChanged(RecordChangeEvent event)
       throws PersistenceException
       {
          if (useSnapshot == null)
          {
             // query opened but not iterated yet
             return;
          }

          if (cursor != null)
          {
             cursor.storeReferenceRow(event);
          }

          // update useSnapshot array
          RecordBuffer targetBuffer = event.getSource();
          Long eventRecordId = event.getDMO() != null ? event.getDMO().primaryKey() : null;

          if (eventRecordId == null)
          {
             return;
          }

          Iterator<RecordBuffer> iter = recordBuffers();
          while (iter.hasNext())
          {
             RecordBuffer buffer = iter.next();
             if (buffer.getDMOInterface().equals(targetBuffer.getDMOInterface()))
             {
                int index = buffersMap.get(buffer);
                if (index >= buffersMap.size() - 1)
                {
                   // ignore last component
                   break;
                }

                Long id;
                if (cursor != null && cursor.size().intValue() > 0)
                {
                   Object o = cursor.peekLastLoadedResult()[index];

                   id = o instanceof Record ? ((Record) o).primaryKey(): (Long) o;
                }
                else
                {
                   id = buffer._rowID();
                }

                if (id != null && eventRecordId.equals(id))
                {
                   Arrays.fill(useSnapshot, index, useSnapshot.length, true);
                }
                else
                {
                   break;
                }
             }
          }
       }

       /**
        * Change the standalone nature of this query.  A query is standalone by
        * default, but can be changed to non-standalone by the wrapper which
        * contains it.
        *
        * @param   standalone
        *          <code>true</code> to set query as standalone;
        *          <code>false</code> to set query as contained.
        */
       public void setStandalone(boolean standalone)
       {
          // TODO: do we need the following?
          /*
          if (!standalone && preselect)
          {
             resolveOnce = true;
          }
          */

          super.setStandalone(standalone);
       }

       /**
        * Explicitly close the prepared query.
        */
       @Override
       public void close()
       {
          close(false);
       }

       /**
        * Explicitly close the prepared query. This resets execution state, but it should not change the query's
        * structural state (i.e., the original components), in case the query is re-opened afterward.
        *
        * @param   inResource
        *          Flag indicating this query is used in a QUERY legacy resource.
        */
       @Override
       public void close(boolean inResource)
       {
          if (inResource && !unregisterOnCleanup)
          {
             // remove it explicitly
             ChangeBroker broker = ChangeBroker.get();
             broker.removeListener(this);
          }

          super.close();

          // reset the table count if the query was optimized
          if (components != originalComponents)
          {
             tableCount = 0;

             for (int i = 0; i < originalComponents.size(); i++)
             {
                CompoundComponent comp = originalComponents.get(i);
                tableCount += comp.getQuery().getTableCount();
             }
          }

          activeIndex = -1;
          components = null;

          if (regCleaner)
          {
             TransactionManager.removeFinalizable(cleaner);
             regCleaner = false;
          }
       }

       /**
        * Clean up resources associated with this query when its scope ends.
        */
       public void cleanup()
       {
          if (closed)
          {
             return;
          }

          // make sure we clean up every query associated with this compound query, including the
          // original component queries and any optimized component join queries
          ArrayList<CompoundComponent> comps = originalComponents;
          if (components != null && components.size() < originalComponents.size())
          {
             comps = new ArrayList<>(originalComponents);
             comps.addAll(components);
          }

          for (int i = 0; i < comps.size(); i++)
          {
             CompoundComponent next = comps.get(i);
             Joinable query = next.getQuery();
             query.cleanup();
          }

          if (unregisterOnCleanup)
          {
             ChangeBroker.get().removeFromGlobal(this);
          }
       }

       /**
        * Report the off-end status of this query, which is essentially the
        * off-end status of its outermost component query.  The query is not
        * considered off-end in its starting state;  that is, if a result has not
        * yet been retrieved.
        *
        * @return  One of the {@link OffEnd} constants <code>NONE</code>,
        *          <code>FRONT</code>, or <code>BACK</code>.
        */
       public OffEnd getOffEnd()
       {
          AbstractQuery query = (AbstractQuery) getComponents().get(0).getQuery();

          return query.getOffEnd();
       }

       /**
        * Conversion of INDEX-INFORMATION attribute (KW_IDX_INFO).
        *
        * Getter of INDEX-INFORMATION attribute
        *
        * @param    n
        *           An integer expression that evaluates to the level of join for which you want index
        *           information.
        *
        * @return   A character string consisting of a comma-separated list of the index or indexes
        *           the query uses at the level of join specified.
        */
       @LegacyAttribute(name = "INDEX-INFORMATION")
       public character indexInformation(NumberType n)
       {
          return originalComponents.get(n.intValue() - 1).getQuery().indexInformation();
       }

       /**
        * Deletes the current row of a query's result list. Implements the 4GL
        * DELETE-RESULT-LIST-ENTRY() method.
        *
        * @param  allowBetweenRows
        *         If <code>true</code> then the cursor can be positioned between rows (the next
        *         row will be deleted). <code>false</code> for conventional DELETE-RESULT-LIST-ENTRY()
        *         mode where the cursor should be positioned on a row (otherwise <code>false</code> is
        *         returned).
        *
        * @return <code>true</code> on success.
        */
       public logical deleteResultListEntry(boolean allowBetweenRows)
       {
          if (isScrolling())
          {
             boolean res = cursor.deleteResultListEntry(allowBetweenRows);
             if (res)
             {
                currentRowDeleted = true;
             }
             return new logical(res);
          }
          else if (!currentRowDeleted && foundFirst)
          {
             currentRowDeleted = true;

             return new logical(true);
          }

          return new logical(false);
       }

       /**
        * Creates an entry in the result list for the current row. Implements the 4GL
        * CREATE-RESULT-LIST-ENTRY() method. Created entry contains id(s) of the record(s) currently
        * located in the backing buffer(s).
        *
        * @return <code>true</code> on success.
        */
       @Override
       public logical createResultListEntry()
       {
          if (!deferOptimization)
          {
             // safe to call multiple times; returns immediately if called previously
             maybeOptimize();
          }

          Object[] data = new Object[tableCount];
          int index = 0;
          List<CompoundComponent> compoundComponents = getComponents();
          for (int i = 0; i < compoundComponents.size(); i++)
          {
             CompoundComponent component = compoundComponents.get(i);
             Joinable query = component.getQuery();
             RecordBuffer[] queryBuffers = query.getRecordBuffers();

             for (RecordBuffer buffer : queryBuffers)
             {
                if (!buffer.isAvailable())
                {
                   data[index++] = null;
                }
                else
                {
                   data[index++] = buffer._rowID();
                }
             }
          }

          cursor.addResultAtCurrentPosition(data);
          return new logical(true);
       }

       /**
        * Set the recursive DATA-RELATION the query is to FILL for.
        * @param relation
        *       the recursive DATA-RELATION the query is to FILL for
        */
       @Override
       public void setDataRelation(DataRelation relation)
       {
          super.setDataRelation(relation);
          if (relation == null)
          {
             List<CompoundComponent> comps = this.getComponents();
             if (!comps.isEmpty()) {
                 final CompoundComponent comp = comps.get(0);
                 comp.setDataRelation(relation);
             }
          }
       }

       /**
        * Function that combines the FQL of the components from this query.
        * The FQLs are separated by a comma.
        *
        * This function is used for logging purposes.
        *
        * @return  String
        *          The combined FQLs.
        */
       public String getCombinedFqls()
       {
          // Will return null if not all components have a valid fql
          StringBuilder fql = new StringBuilder();
          List<CompoundComponent> comps = getComponents();
          int size = comps.size();
          for (int i = 0; i < size; i++)
          {
             Joinable query = comps.get(i).getQuery();
             String result = null;
             if (query instanceof PreselectQuery)
             {
                result = ((PreselectQuery) query).getHQL();
             }
             else if (query instanceof RandomAccessQuery)
             {
                result = ((RandomAccessQuery) query).getBundleFql();
             }

             if (result == null)
             {
                return null;
             }

             fql.append(result);
             if (i < size - 1)
             {
                fql.append(", ");
             }
          }

          return fql.toString();
       }

       /**
        * Function that sums and returns the execution time of the components, denoting the total time of
        * execution of this query.
        *
        * This function is used for logging purposes.
        *
        * @return  double
        *          The total time of execution of this {@code CompoundQuery}.
        */
       public double getCombinedExecutionTime()
       {
          // It presumes that all components have been executed
          double totalTime = 0;
          List<CompoundComponent> comps = getComponents();
          int size = comps.size();
          for (int i = 0; i < size; i++)
          {
             Joinable query = comps.get(i).getQuery();
             String result = null;
             if (query instanceof PreselectQuery)
             {
                result = ((PreselectQuery) query).getHQL();
             }
             else if (query instanceof RandomAccessQuery)
             {
                result = ((RandomAccessQuery) query).getBundleFql();
             }

             // Statistics about the query executed by this component
             P2JQueryStatistics statistics = P2JQueryStatistics.getStatistics(query, result);
             totalTime += statistics.getTime();
          }

          return totalTime;
       }

       /**
        * Reposition the cursor such that a request to retrieve the next result
        * will retrieve the result which matches the specified array of primary
        * key IDs.  A request to retrieve the previous result will retrieve the
        * result immediately previous to this, if any, in the results list.
        *
        * @param   id1
        *          A primary key ID, representing the target record (in the case
        *          of a join, this ID is associated with the left-most record in
        *          the join).
        * @param   joinIDs
        *          All remaining primary key IDs, if any, arranged from left to
        *          right to coincide with the records being joined by the
        *          underlying query.
        */
       @Override
       public void repositionByID(rowid id1, rowid...joinIDs)
       {
          if (id1 == null || id1.isUnknown())
          {
             RecordBuffer[] buffers = getRecordBuffersInternal();
             RecordBuffer lastSetBuffer = buffers[buffers.length - 1];
             lastSetBuffer.release(true);
             ErrorManager.recordOrThrowError(7316, "Invalid rowid argument(s) given in query object method", false, false);
             return;
          }

          super.repositionByID(id1, joinIDs);
       }

       /**
        * Register a <code>Finalizable</code> with the transaction manager for
        * resource cleanup when this query's scope ends.
        */
       protected void registerCleaner()
       {
          // If we are not a standalone query, we assume our container will
          // register for cleanup.
          if (regCleaner || !isStandalone())
          {
             return;
          }

          TransactionManager.registerFinalizable(cleaner, false);

          regCleaner = true;
       }

       /**
        * This method is an extension to the AbstractQuery normal behavior. On a
        * reposition, the buffers should be validated and eventually fire their
        * write triggers. Furthermore, the bottom-most buffer should be released.
        * The other buffers should be indeterminably released - this feature is safe to
        * ignore.
        *
        * @param   permitFetch
        *          <code>true</code> to permit immediate record fetch;
        *          <code>false</code> to override <code>fetchOnReposition</code>
        *          flag and explicitly disallow immediate record fetch.
        * @param   error
        *          <code>true</code> if error happened during reposition.
        */
       @Override
       protected void afterReposition(boolean permitFetch, boolean error)
       {
          if (error)
          {
             super.afterReposition(permitFetch, true);
             return;
          }

          // after a reposition, the write triggers should be fired as part of a validation scenario
          RecordBuffer[] buffers = getRecordBuffersInternal();
          for (RecordBuffer buffer : buffers)
          {
             buffer.validate(false, false);
          }

          if (buffers.length > 0)
          {
             // the bottom-most buffer should be always released after repositioning
             RecordBuffer buf = buffers[buffers.length - 1];
             buf.release(true);
          }

          super.afterReposition(permitFetch, false);
       }

       /**
        * Init structures required to use reference row: create the map of all buffers, register
        * parameter filters for sub-queries and register this query as a record change listener.
        *
        * @param   globalScope
        *          Flag indicating if the block at which this query is registered as record change listener is
        *          the global scope.
        */
       protected void initReferenceRowSupport(boolean globalScope)
       {
          if (!preselect && buffersMap == null)
          {
             // cache set of all buffers and set parameter filters
             buffersMap = new LinkedHashMap<>();
             int bufferIndex = 0;
             List<CompoundComponent> compoundComponents = getComponents();
             for (int i = 0; i < compoundComponents.size(); i++)
             {
                CompoundComponent component = compoundComponents.get(i);
                Joinable query = component.getQuery();
                RecordBuffer[] queryBuffers = query.getRecordBuffers();

                for (RecordBuffer buffer : queryBuffers)
                {
                   buffersMap.put(buffer, bufferIndex);

                   if (bufferIndex > 0)
                   {
                      query.setParameterFilter(new SnapshotParameterFilter(bufferIndex));
                   }
                   bufferIndex++;
                }
             }

             ChangeBroker.get().addListener(this);
             if (globalScope)
             {
                unregisterOnCleanup = true;
             }
          }
       }

       /**
        * Return an object which represents a property value which defines the
        * end of a <i>sort band</i>.  A <i>sort band</i> is a series of one or
        * more records which share the same value for a particular property,
        * where that property is used as the coarsest sorting criterion defined
        * by this query's <code>order by</code> clause.
        * <p>
        * If the break value for this query currently is non-<code>null</code>,
        * this indicates that the most recently retrieved record has crossed a
        * <i>sort band</i> boundary, and that a new band has begun.  This value
        * will only be non-<code>null</code> at the point at which such a record
        * is found, and will be <code>null</code> for all other records.
        * <p>
        * Note:  break values are only tracked for invocations of
        * <code>next</code> and <code>previous</code>.  Other retrievals will
        * result in break value being <code>null</code>.
        *
        * @return  {@code null} always; when a compound query is the backing query for
        *          a dynamic results object, it should never return a non-null value,
        *          because it cannot shift into a preselect mode.
        */
       protected Object getBreakValue()
       {
          return null;
       }

       /**
        * Retrieve a virtual, composite row of results for the compound query.
        * This method implements a client-side join of multiple, single-row
        * record retrievals performed by the underlying, component queries.
        * It works first from outermost component inward, retrieving a record at
        * each level.  When each component's backing buffer contains a record,
        * the composite retrieval is complete and the method returns.  On
        * subsequent iterations, the innermost component is advanced first.  If
        * no more matches are possible at this level, the algorithm moves outward
        * one level and attempts to advance that level.  On success, it moves
        * inward again, on failure, it moves outward again.  This continues until
        * either a new composite row of results is assembled, or no further
        * records can be found.  This progression ensures that each inner query
        * cycles through all of its matches before the next outer query advances.
        * <p>
        * <b>Preselect Mode</b><br>
        * <code>CompoundQuery</code> generally operates as a dynamic query,
        * though it can be used to generate a fixed list of preselected results.
        * When in this mode, the first time this method is executed, a results
        * list is generated.  This is done by retrieving each virtual, composite
        * row which meets the overall query's criteria, and caching the primary
        * keys of these records in the underlying cursor.  Each underlying query
        * is assigned the correct subset of these IDs, such that subsequent
        * retrieval requests can safely use the cursor to retrieve the correct
        * records.  As a result, in preselect mode, this method is only ever
        * invoked once per query.
        *
        * @param   navigation
        *          Type of advancement applied to the compound query;  i.e., how
        *          the virtual composite result set is navigated as a whole:
        *          forward (next), backward (previous), jump to first or last, or
        *          reload current records in all buffers.
        * @param   lockType
        *          Overriding lock type to apply to each underlying buffer.
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        *
        * @return  An array of the primary key IDs for every table involved in
        *          the query, (which together identify a unique, virtual record),
        *          if a result row was successfully retrieved;
        *          <code>null</code> if no more results were available (and we
        *          were not iterating).
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an error occurred retrieving the record for an underlying
        *          query.
        */
       protected Object[] retrieve(int navigation, LockType lockType, boolean iterating)
       {
          return retrieve(navigation, lockType, iterating, false);
       }

       /**
        * Retrieve a virtual, composite row of results for the compound query.
        * This method implements a client-side join of multiple, single-row
        * record retrievals performed by the underlying, component queries.
        * It works first from outermost component inward, retrieving a record at
        * each level.  When each component's backing buffer contains a record,
        * the composite retrieval is complete and the method returns.  On
        * subsequent iterations, the innermost component is advanced first.  If
        * no more matches are possible at this level, the algorithm moves outward
        * one level and attempts to advance that level.  On success, it moves
        * inward again, on failure, it moves outward again.  This continues until
        * either a new composite row of results is assembled, or no further
        * records can be found.  This progression ensures that each inner query
        * cycles through all of its matches before the next outer query advances.
        * <p>
        * <b>Preselect Mode</b><br>
        * <code>CompoundQuery</code> generally operates as a dynamic query,
        * though it can be used to generate a fixed list of preselected results.
        * When in this mode, the first time this method is executed, a results
        * list is generated.  This is done by retrieving each virtual, composite
        * row which meets the overall query's criteria, and caching the primary
        * keys of these records in the underlying cursor.  Each underlying query
        * is assigned the correct subset of these IDs, such that subsequent
        * retrieval requests can safely use the cursor to retrieve the correct
        * records.  As a result, in preselect mode, this method is only ever
        * invoked once per query.
        *
        * @param   navigation
        *          Type of advancement applied to the compound query;  i.e., how
        *          the virtual composite result set is navigated as a whole:
        *          forward (next), backward (previous), jump to first or last, or
        *          reload current records in all buffers.
        * @param   lockType
        *          Overriding lock type to apply to each underlying buffer.
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        * @param   peek
        *          <code>true</code> if sub-queries should be interrogated using
        *          {@link Scrollable#peekNext()}/{@link Scrollable#peekPrevious()}
        *          rather than {@link P2JQuery#next()}/{@link P2JQuery#previous()}.
        *          The difference is that in the first case sub-queries return the
        *          next/previous record from underlying results, while in the second
        *          case they return the next/previous record after the current row.
        *
        * @return  An array of the primary key IDs for every table involved in
        *          the query, (which together identify a unique, virtual record),
        *          if a result row was successfully retrieved;
        *          <code>null</code> if no more results were available (and we
        *          were not iterating).
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an error occurred retrieving the record for an underlying
        *          query.
        */
       protected Object[] retrieve(int navigation, LockType lockType, boolean iterating, boolean peek)
       {
          translateWhere();

          initReferenceRowSupport(false);

          Object[] rowData;

          if (!peek)
          {
             OffEnd offEnd = getOffEnd();
             if (!(offEnd == OffEnd.FRONT && navigation == PREVIOUS) &&
                 !(offEnd == OffEnd.BACK && navigation == NEXT))
             {
                currentRowDeleted = false;
             }
          }

          registerCleaner();

          if (preselect)
          {
             preselectResults();

             // Fulfill the retrieval request using the cached results.
             switch (navigation)
             {
                case NEXT:
                   rowData = cursor.getNext();
                   break;
                case FIRST:
                   rowData = cursor.getFirst();
                   break;
                case PREVIOUS:
                   rowData = cursor.getPrevious();
                   break;
                case LAST:
                   rowData = cursor.getLast();
                   break;
                default:
                   rowData = null;
             }

             boolean loaded = loadByValues(rowData, lockType, iterating);

             if (!isScrolling() && !foundFirst && rowData != null && loaded)
             {
                foundFirst = true;
             }
          }
          else
          {
             // Non-preselect mode;  retrieve composite record dynamically.
             rowData = retrieveImpl(navigation, lockType, iterating, peek);
          }

          return rowData;
       }

       /**
        * Determines if the query is presorted, i.e. if the query is sorted up front in some special
        * way.
        *
        * @return <code>true</code> if the query is presorted.
        */
       protected boolean isPresorted()
       {
          return false;
       }

       /**
        * If the query is a presort query, then presort the results stored in the cursor.
        * No-op for this implementation.
        */
       protected void presort()
       {
       }

       /**
        * Preselect all results by creating a common cursor and {@link SimpleResults} for each
        * component.
        */
       protected void preselectResults()
       {
          if (preselected)
          {
             return;
          }

          registerCleaner();

          Object[] rowData;

          // Create a cursor.
          setScrolling();

          int size = 0;

          // Logging setup.
          boolean trace = log.isLoggable(Level.FINEST);
          StringBuilder buf = null;
          String nl = null;
          if (trace)
          {
             nl = System.getProperty("line.separator");
             buf = new StringBuilder("Preselected IDs:");
          }

          // Prepare per-table, preliminary result sets.
          Set<List<Object>>[] idSets = null;
          int[] widths = null;

          // Preselect all records which match the query's criteria.
          do
          {
             rowData = retrieveImpl(NEXT, LockType.NONE, false, false);
             cursor.addResultNext(rowData);

             // initialize idSets and widths below retrieveImpl, so we are working with the
             // optimized form of the compound query
             if (size == 0)
             {
                size = components.size();

                idSets = new Set[size];
                for (int i = 0; i < size; i++)
                {
                   idSets[i] = new LinkedHashSet<>();
                }

                widths = new int[size];
                Iterator<CompoundComponent> iter = components.iterator();
                for (int i = 0; iter.hasNext(); i++)
                {
                   widths[i] = iter.next().getQuery().getTableCount();
                }
             }

             if (rowData != null)
             {
                int index = 0;
                for (int i = 0; i < size; i++)
                {
                   // How many tables does this joinable represent?
                   int w = widths[i];

                   // Initialize the sub-array for this joinable component.
                   // We use an ArrayList instead of an array directly so we
                   // can add this to a Set.
                   List<Object> subIDs = new ArrayList<>();
                   for (int j = 0; j < w && index < tableCount; j++, index++)
                   {
                      subIDs.add(rowData[index]);
                   }
                   idSets[i].add(subIDs);
                }

                if (trace)
                {
                   buf.append(nl);
                   buf.append("  ");
                   for (int i = 0; i < size; i++)
                   {
                      buf.append(" ");
                      buf.append(rowData[i]);
                   }
                }
             }
          }
          while (rowData != null);

          if (isPresorted())
          {
             presort();
          }

          // Set up and install final result sets into underlying queries.
          // This is necessary because inner tables in a multi-table query
          // will have an incorrect result set left over from the first pass
          // scan for the last iteration of their enclosing loop.  This
          // would wreak havoc when trying to load from IDs later.
          for (int i = 0; i < size; i++)
          {
             List<Object[]> rows = new ArrayList<>();
             Iterator<List<Object>> iter2 = idSets[i].iterator();
             while (iter2.hasNext())
             {
                List<Object> subIDs = iter2.next();
                rows.add(subIDs.toArray());
             }

             SimpleResults results = new SimpleResults(rows);
             Joinable query = components.get(i).getQuery();
             query.setResults(results);
          }

          // Reset state and cursor.
          activeIndex = -1;
          preselected = true;
          cursor.addResultNext(null);
          cursor.beforeFirst();

          if (trace)
          {
             log.log(Level.FINEST, buf.toString());
          }
       }

       /**
        * Synchronize the query with the cursor by causing the query to load the
        * current row's results (if any) into its backing buffer(s).
        */
       protected void loadRowAtCursor()
       {
          Object[] row = cursor.getRow();
          if (row != null)
          {
             loadByValues(row, null, false);
          }
       }

       /**
        * Load a virtual, composite record by loading each record buffer managed
        * by this query with a DMO, given its primary key ID in the specified
        * <code>ids</code> array.  The IDs are arranged to coincide with the
        * tables underlying the query, from left-most to right-most (in the sense
        * of how they are joined by the query).
        *
        * @param   data
        *          Array of primary key IDs or DMOs, one per table involved in the
        *          query
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *          Set to <code>null</code> to allow underlying query to use its
        *          existing lock type.
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        *
        * @return  <code>true</code> if <code>ids</code> was not
        *          <code>null</code>;  else <code>false</code>.
        *
        * @see     #collectRowData
        */
       protected boolean loadByValues(Object[] data, LockType lockType, boolean iterating)
       {
          return loadByValues(data, lockType, iterating, true);
       }

       /**
        * Load a virtual, composite record by loading each record buffer managed
        * by this query with a DMO, given its primary key ID in the specified
        * <code>ids</code> array.  The IDs are arranged to coincide with the
        * tables underlying the query, from left-most to right-most (in the sense
        * of how they are joined by the query).
        *
        * @param   data
        *          Array of primary key IDs or DMOs, one per table involved in the
        *          query
        * @param   lockType
        *          Lock type which should override that of any underlying query.
        *          Set to <code>null</code> to allow underlying query to use its
        *          existing lock type.
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        * @param   loadAll
        *          <code>true</code> to load all components
        *          <code>false</code> to only load components up to the current active index;
        *
        * @return  <code>true</code> if <code>ids</code> was not
        *          <code>null</code>;  else <code>false</code>.
        *
        * @see     #collectRowData
        */
       protected boolean loadByValues(Object[] data, LockType lockType, boolean iterating, boolean loadAll)
       {
          if (data == null)
          {
             if (iterating)
             {
                throw new QueryOffEndException("",
                      getOffEndListeners().toArray(new QueryOffEndListener[0]));
             }

             return false;
          }

          int compsLoadCount = (loadAll) ? components.size() : activeIndex;

          // Reset active index and counters.
          if (loadAll)
          {
             activeIndex = 0;
             counters = new int[components.size()];
          }

          // Iterate over the query components, updating the buffers of each
          // with the appropriate record.
          int index = 0;
          for (int i = 0; i < compsLoadCount; i++)
          {
             // Get the next query component.
             CompoundComponent comp = components.get(i);

             // Get the underlying query.
             Joinable query = comp.getQuery();

             // How many tables does this joinable query represent?
             int width = query.getTableCount();

             // Initialize the sub-array for this joinable component.
             Object[] subData = new Object[width];
             for (int j = 0; j < width && index < tableCount; j++, index++)
             {
                subData[j] = data[index];
             }

             try
             {
                query.load(subData, lockType, true);
             }
             catch (MissingRecordException exc)
             {
                releaseAllBuffers();
                return false;
             }
             catch (PersistenceException exc)
             {
                if (log.isLoggable(Level.WARNING))
                {
                   log.log(Level.WARNING, "Error loading data", exc);
                }

                return false;
             }
             catch (QueryOffEndException exc)
             {
                return false;
             }
             catch (ConditionException exc)
             {
                if (loadAll)
                {
                   activeIndex = -1;
                }

                throw exc;
             }

             if (loadAll)
             {
                // Reflect the fact that this component has been updated.
                counters[i]++;
                activeIndex++;
             }
          }

          if (loadAll)
          {
             activeIndex--;
          }
          else
          {
             activeIndex = compsLoadCount;
          }

          return true;
       }

       /**
        * Indicate whether this object can be optimized.
        *
        * @return  <code>true</code> by default. Subclasses should override this behavior as needed.
        */
       protected boolean canOptimize()
       {
          return true;
       }

       /**
        * Register the query within each component as a record change listener.
        *
        * @return  <code>true</code> if the global scope is used for registration.
        */
       private boolean registerRecordChangeListeners()
       {
          // Find for each compound query component the most outer scope among
          // nearest external scope and the outermost scopes of buffers used by
          // this component and register the component into this scope.
          int externalScope = BufferManager.get().findNearestExternal();
          int lowestScope = externalScope;

          List<CompoundComponent> compoundComponents = getComponents();
          for (int i = 0; i < compoundComponents.size(); i++)
          {
             CompoundComponent comp = compoundComponents.get(i);
             Joinable query = comp.getQuery();
             if (!query.shouldRegisterRecordChangeListeners())
             {
                continue;
             }

             int scope;
             if (query.isDynamicPredicate())
             {
                scope = 0;
             }
             else
             {
                scope = externalScope;
                RecordBuffer[] buffers = query.getRecordBuffers();

                for (RecordBuffer buffer : buffers)
                {
                   scope = ((scope == -1)
                      ? buffer.getScopeOpenDepth()
                      : Math.min(scope, buffer.getScopeOpenDepth()));
                }
             }

             lowestScope = ((lowestScope == -1) ? scope : Math.min(lowestScope, scope));

             if (scope != -1)
             {
                query.registerRecordChangeListeners(Integer.valueOf(scope));
             }
          }

          return lowestScope == 0;
       }

       /**
        * Method which attempts to optimize this object's query components into one or more
        * server-side joins. Returns immediately if this is not the first attempt to do so.
        * <p>
        * This method must only be invoked after the first query result is fetched via the
        * un-optimized query, to ensure we correctly preserve the pre-query state of the involved
        * record buffers, in the event the compound query runs off end immediately.
        */
       private void maybeOptimize()
       {
          if (components != null)
          {
             // already optimized (or at least tried)
             return;
          }

          try
          {
             if (!canOptimize())
             {
                components = originalComponents;

                return;
             }

             new Optimizer();

             if (deferOptimization && components.size() < originalComponents.size())
             {
                // register the new, optimized queries as change listeners; if a query was left
                // unoptimized, the change broker is tolerant of a listener being added multiple times
                registerRecordChangeListeners();
             }
          }
          finally
          {
             if (forceFullResults)
             {
                ArrayList<CompoundComponent> comps = getComponents();
                for (int i = 0; i < comps.size(); i++)
                {
                   CompoundComponent cc = comps.get(i);
                   cc.getQuery().hintFullResults();
                }
             }

             if (isPresort() && isIterating())
             {
                ArrayList<CompoundComponent> comps = getComponents();
                for (int i = 0; i < comps.size(); i++)
                {
                   CompoundComponent cc = comps.get(i);
                   cc.getQuery().setIterating(true);
                   cc.getQuery().setPresort(true);
                }
             }
          }
       }

       /**
        * Get an array of all record buffers managed by this query.
        *
        * @return  All record buffers managed by this query.
        */
       private RecordBuffer[] getRecordBuffersInternal()
       {
          RecordBuffer[] bufs = new RecordBuffer[getTableCount()];

          int i = 0;
          ArrayList<CompoundComponent> compoundComponents = getOriginalComponents();
          for (int j = 0; j < compoundComponents.size(); j++)
          {
             CompoundComponent comp = compoundComponents.get(j);
             for (RecordBuffer buf : comp.getQuery().getRecordBuffers())
             {
                bufs[i++] = buf;
             }
          }

          return bufs;
       }

       /**
        * Iterate the component queries of this object and set their browsed flags to match that of
        * this object.
        */
       private void setComponentsBrowsed()
       {
          boolean browsed = isBrowsed();

          for (int i = 0; i < originalComponents.size(); i++)
          {
             CompoundComponent next = originalComponents.get(i);
             next.getQuery().setBrowsed(browsed);
          }

          if (components != null && components != originalComponents)
          {
             for (int i = 0; i < components.size(); i++)
             {
                CompoundComponent next = components.get(i);
                next.getQuery().setBrowsed(browsed);
             }
          }
       }

       /**
        * Assemble an array of primary key IDs or DMOs for the current records in
        * the buffers underlying this query, from left-most to right-most (in the
        * sense of how the query joins the associated tables).  This method
        * essentially takes a lightweight snapshot of the currently loaded
        * records, so that this may be restored later.
        * <p>
        * This snapshot is only generated for scrolling queries.
        *
        * @param   navigation
        *          Type of advancement applied to the compound query;  i.e., how
        *          the virtual composite result set is navigated as a whole:
        *          forward (next), backward (previous), jump to first or last, or
        *          reload current records in all buffers.
        *
        * @return  Array of primary key IDs or DMOs, one per table involved in the
        *          query.
        *
        * @see     #loadByValues
        */
       private Object[] collectRowData(int navigation)
       {
          if (isScrolling())
          {
             switch (navigation)
             {
                case NEXT:
                case LAST:
                   if (cursor.isOffEnd(OffEnd.BACK))
                   {
                      return null;
                   }
                   break;
                case PREVIOUS:
                case FIRST:
                   if (cursor.isOffEnd(OffEnd.FRONT))
                   {
                      return null;
                   }
                default:
                   break;
             }
          }
          else
          {
             if (isOffEnd().booleanValue())
             {
                return null;
             }
          }

          Object[] rowData = new Object[tableCount];
          int dstIdx = 0;
          ArrayList<CompoundComponent> comps = getComponents();
          for (int i = 0; i < comps.size(); i++)
          {
             Joinable query = comps.get(i).getQuery();
             Object[] subData = query instanceof AdaptiveQuery
                                ? ((AdaptiveQuery) query).getCachedRow()
                                : query.getRow();
             if (subData != null)
             {
                System.arraycopy(subData, 0, rowData, dstIdx, subData.length);
             }
             dstIdx += query.getTableCount();
          }

          return rowData;
       }

       /**
        * Retrieve a virtual, composite row of results for the compound query.
        * This method implements a client-side join of multiple, single-row
        * record retrievals performed by the underlying, component queries.
        * It works first from outermost component inward, retrieving a record at
        * each level.  When each component's backing buffer contains a record,
        * the composite retrieval is complete and the method returns.  On
        * subsequent iterations, the innermost component is advanced first.  If
        * no more matches are possible at this level, the algorithm moves outward
        * one level and attempts to advance that level.  On success, it moves
        * inward again, on failure, it moves outward again.  This continues until
        * either a new composite row of results is assembled, or no further
        * records can be found.  This progression ensures that each inner query
        * cycles through all of its matches before the next outer query advances.
        *
        * @param   navigation
        *          Type of advancement applied to the compound query;  i.e., how
        *          the virtual composite result set is navigated as a whole:
        *          forward (next), backward (previous), jump to first or last, or
        *          reload current records in all buffers.
        * @param   lockType
        *          Overriding lock type to apply to each underlying buffer.
        * @param   iterating
        *          <code>true</code> if this method is being invoked within the
        *          context of an iterating loop, which can only be exited by
        *          raising an end condition;  <code>false</code> if end condition
        *          should not be raised, even if no further advancement of the
        *          composite result is possible.
        * @param   peek
        *          <code>true</code> if sub-queries should be interrogated using
        *          {@link Scrollable#peekNext()}/{@link Scrollable#peekPrevious()}
        *          rather than {@link P2JQuery#next()}/{@link P2JQuery#previous()}.
        *          The difference is that in the first case sub-queries return the
        *          next/previous record from underlying results, while in the second
        *          case they return the next/previous record after the current row.
        *
        * @return  An array of the primary key IDs for every table involved in
        *          the query, (which together identify a unique, virtual record),
        *          if a result row was successfully retrieved;
        *          <code>null</code> if no more results were available (and we
        *          were not iterating).
        *
        * @throws  QueryOffEndException
        *          if no more results were available in iterating mode.
        * @throws  ErrorConditionException
        *          if an error occurred retrieving the record for an underlying
        *          query.
        */
       private Object[] retrieveImpl(int navigation, LockType lockType, boolean iterating, boolean peek)
       {
          translateWhere();

          if (!deferOptimization)
          {
             // safe to call multiple times; returns immediately if called previously
             maybeOptimize();
          }

          ArrayList<CompoundComponent> comps = getComponents();

          int size = comps.size();
          if (size > 0)
          {
             final CompoundComponent comp = comps.get(0);
             comp.setDataRelation(this.getDataRelation());
          }

          // determine which outer components returned no record for the current search
          boolean[] nullOuter = new boolean[size];

          // If we are getting the first or last compound row, reset activeIndex
          // so that we are forced to get all components.  This prevents interim
          // state from interfering with a first/last request.
          switch (navigation)
          {
             case FIRST:
             case LAST:
                activeIndex = -1;
                break;
             default:
                break;
          }

          // Is this the first iteration in a new series?
          if (activeIndex < 0)
          {
             activeIndex = 0;

             // Create array to hold current loop counts, each initialized to 0.
             counters = new int[size];
             useSnapshot = new boolean[getTableCount() - 1];

             // Disable outer component queries' buffers' release-on-iterate
             // default behavior for the enclosing scope.  This is necessary to
             // prevent the premature release of an outer buffer's current
             // record while we are manually joining it with an inner buffer's
             // records.
             int end = size - 1;
             for (int i = 0; i < end; i++)
             {
                CompoundComponent comp = comps.get(i);
                RecordBuffer[] buffers = comp.getQuery().getRecordBuffers();
                int len = buffers.length;
                for (int j = 0; j < len; j++)
                {
                   buffers[j].disableReleaseOnIterate();
                }
             }
          }

          Object[] rowData = null;

          // Array of indexes in rowData at which sub-data for each component begins
          int[] subIndexes = null;

          if (peek)
          {
             rowData = new Object[tableCount];
             subIndexes = new int[size];
             int dstIndex = 0;
             int j = 0;
             for (int i = 0; i < comps.size(); i++)
             {
                CompoundComponent component = comps.get(i);
                Joinable query = component.getQuery();
                subIndexes[j++] = dstIndex;
                dstIndex += query.getTableCount();
             }
          }

          if (activeIndex > 0 && isScrolling())
          {
             Object[] row = cursor.getRow();
             if (row != null)
             {
                loadByValues(row, lockType, false, false);
             }
          }

          boolean offEnd = false;
          boolean advancing = false;
          Database database = null;
          for (int i = activeIndex; i < size && i >= 0; activeIndex = i)
          {
             nullOuter[i] = false;
             CompoundComponent comp = comps.get(i);
             RecordBuffer[] recordBuffers = comp.getQuery().getRecordBuffers();
             comp.getQuery().setLenientOffEnd(true);

             if (recordBuffers != null && recordBuffers.length > 0)
             {
                database = recordBuffers[0].getDatabase();
             }

             try
             {
                Object[] subData = P2JQueryExecutor.getInstance().execute(database,
                                                                          (String) null,
                                                                          CompoundQuery::processComponent,
                                                                          this,
                                                                          navigation,
                                                                          comp,
                                                                          counters[i],
                                                                          lockType,
                                                                          peek);

                if (subData == null)
                {
                   if (comp.isOuter() && counters[i] == 0)
                   {
                      // if this is the first usage of this outer join component, and no record was found
                      // then either advance or go back, depending on if this component was queried
                      // before
                      boolean prevNullOuter = nullOuter[i];

                      if (!prevNullOuter && advancing)
                      {
                         nullOuter[i] = true;
                         i++;
                      }
                      else
                      {
                         advancing = false;
                         i--;
                      }

                      comp.getQuery().setUnknownRecord();

                      Arrays.fill(useSnapshot, false);
                   }
                   else
                   {
                      // this branch treats the case of an inner join or an outer join component (which
                      // did found records)

                      // Reset counter for this component.
                      counters[i] = 0;
                      if (i > 0)
                      {
                         Arrays.fill(useSnapshot, i - 1, useSnapshot.length, false);
                      }

                      // If we've propagated back up to the left-most component and
                      // it has no more results, we're done.
                      offEnd = (--i < 0);
                      advancing = false;
                      if (offEnd)
                      {
                         // If not iterating, assume client code is relying on
                         // an availability test to end loop.
                         break;
                      }
                   }

                   continue;
                }

                if (peek)
                {
                   try
                   {
                      comp.getQuery().load(subData, lockType, false);
                   }
                   catch (MissingRecordException exc)
                   {
                      continue;
                   }

                   System.arraycopy(subData, 0, rowData, subIndexes[i], subData.length);
                }

                counters[i]++;

                if (cursor != null)
                {
                   cursor.resetReferenceRow(i);
                }

                advancing = true;
                i++;
             }
             catch (ErrorConditionException exc)
             {
                activeIndex = -1;

                throw exc;
             }
             catch (PersistenceException exc)
             {
                ErrorManager.recordOrThrowError(exc);
             }
          }

          // the next record search must start with the left-most component which was
          // not an outer join component with no record found.
          while (activeIndex > 0 && nullOuter[activeIndex - 1])
          {
             activeIndex = activeIndex - 1;
          }
          activeIndex = activeIndex - 1;

          if (log.isLoggable(Level.FINEST))
          {
             log.log(Level.FINEST, "activeIndex = " + activeIndex);
             log.log(Level.FINEST, "nullOuter = " + Arrays.toString(nullOuter));
             StringBuilder buf = new StringBuilder("counters:  ");
             for (int i = 0; i < size; i++)
             {
                buf.append(counters[i]);
                buf.append(" ");
             }
             log.log(Level.FINEST, buf.toString());
          }

          if (!offEnd && !peek)
          {
             if (!postponeAccumulation)
             {
                // Notify accumulators of a successful iteration.
                accumulate();
             }

             incrementMoves();

             if (!isScrolling() && !foundFirst)
             {
                foundFirst = true;
             }
          }

          if (peek && !offEnd)
          {
             // In rowData we have only sub-data for components which have been iterated.
             // Fill data for remaining components.
             for (int i = 0; i < subIndexes.length; i++)
             {
                int index = subIndexes[i];
                if (rowData[index] == null)
                {
                   Joinable query = comps.get(i).getQuery();
                   Object[] subData = query instanceof AdaptiveQuery
                                      ? ((AdaptiveQuery) query).getCachedRow()
                                      : query.getRow();
                   if (subData != null)
                   {
                      System.arraycopy(subData, 0, rowData, index, subData.length);
                   }
                }
             }
          }

          if (!offEnd)
          {
             if (!peek)
             {
                rowData = collectRowData(navigation);
             }

             if (deferOptimization)
             {
                // An attempt to optimize the compound query only will be made once, after the first
                // result is obtained successfully. On subsequent attempts, the optimize method is
                // effectively a no-op which returns immediately. We wait to optimize until after the
                // first result is fetched, because if we optimize before this and the optimized
                // query runs off end, the individual buffers may not be in the correct state. If an
                // individual query component fails to retrieve a record, any buffers of inner
                // components must maintain their original, pre-query state. If a query is optimized
                // such that a server-side join fails to return a composite record, we don't know
                // which buffers should be cleared and which should retain their pre-query state.
                // Although it might be considered an application-level bug to rely on this behavior,
                // we must maintain compatibility with this quirk to avoid discrepancies in behavior.
                maybeOptimize();
             }
          }

          return (offEnd ? null : rowData);
       }

       /**
        * Switch on postponed accumulation on next retrieve.
        */
       protected void postponeAccumulation()
       {
          postponeAccumulation = true;
       }

       /**
        * Switch on postponed accumulation on next retrieve.
        */
       protected void restoreAccumulation()
       {
          postponeAccumulation = false;
       }

       /**
        * Release all backing buffers of for all components.
        */
       protected void releaseAllBuffers()
       {
          ArrayList<CompoundComponent> compoundComponents = getComponents();
          for (int i = 0; i < compoundComponents.size(); i++)
          {
             CompoundComponent comp = compoundComponents.get(i);
             RecordBuffer[] bufs = comp.getQuery().getRecordBuffers();
             for (RecordBuffer buf : bufs)
             {
                buf.release(false);
             }
          }
       }

       /**
        * Add a component query to this compound query and adjust the table count accordingly.
        *
        * @param   component
        *          Query component to be added.
        */
       void addCompoundComponent(CompoundComponent component)
       {
          if (isForwardOnly())
          {
             component.getQuery().setForwardOnly(true);
          }

          originalComponents.add(component);

          // Update total table count.
          tableCount += component.getQuery().getTableCount();
       }

       /**
        * Retrieve the list of original query components (unoptimized).
        *
        * @return  See above.
        */
       ArrayList<CompoundComponent> getOriginalComponents()
       {
          return originalComponents;
       }

       /**
        * Retrieve the list of query components. Depending on when this is called (before or after
        * optimization), it may return the list of original, unoptimized components, or the list of
        * optimized components.
        *
        * @return  See above.
        */
       private ArrayList<CompoundComponent> getComponents()
       {
          return components != null ? components : originalComponents;
       }

       /**
        * Initialize state of a component query.
        *
        * @param   query
        *          Component query to be initialized.
        */
       private void initializeComponentQuery(Joinable query)
       {
          // Mark component query as not standalone.
          query.setStandalone(false);

          // In a compound query, joined first and last record retrievals must
          // not throw an ErrorConditionException if no record is found.
          query.setErrorIfNull(false);

          // Does the component query need to scroll?
          if (isScrolling())
          {
             query.setScrolling();
          }

          // a query which sorts on the client (application server) needs full records (not just
          // primary keys) to do so
          if (isPresorted())
          {
             query.setFullRecords();
          }

          // If necessary, indicate to the component query that it should behave
          // as if it were associated with a browse.
          if (isBrowsed())
          {
             query.setBrowsed(isBrowsed());
          }
       }

       /**
        * Advance a single query component, according to the overall query
        * navigation direction, and that component's natural iteration type.  It
        * is the intersection of these two criteria which determines the actual
        * type of retrieval which is performed.
        *
        * @param   navigation
        *          Type of advancement applied to the compound query;  i.e., how
        *          the virtual composite result set is navigated as a whole:
        *          forward (next), backward (previous), jump to first or last, or
        *          reload current records in all buffers.
        * @param   component
        *          Query component being asked to retrieve a record.
        * @param   count
        *          Index of this query component's result within the current join.
        * @param   lockType
        *          Lock type to apply to the retrieved record for this query
        *          component
        * @param   peek
        *          <code>true</code> if sub-queries should be interrogated using
        *          {@link Scrollable#peekNext()}/{@link Scrollable#peekPrevious()}
        *          rather than {@link P2JQuery#next()}/{@link P2JQuery#previous()}.
        *          The difference is that in the first case sub-queries return the
        *          next/previous record from underlying results, while in the second
        *          case they return the next/previous record after the current row.
        *          Peeking works only for NEXT and PREV iteration types.
        *
        * @throws  QueryOffEndException
        *          if query component is being asked to retrieve a FIRST, LAST,
        *          or UNIQUE record, and it has already done so in the current
        *          iteration pass (prevents looping infinitely on these
        *          constructs).
        *
        * @return  If <code>peek</code> is <code>true</code>: an array of
        *          the primary key IDs for the query component, if a result was
        *          successfully retrieved; <code>null</code> if no more results
        *          were available.<p>
        *          If <code>peek</code> is <code>false</code>: always returns
        *          <code>null</code>.
        *
        * @see     QueryConstants#RETRIEVE_MODES
        */
       private Object[] processComponent(int navigation,
                                         CompoundComponent component,
                                         int count,
                                         LockType lockType,
                                         boolean peek)
       {
          int iteration = component.getIteration();
          int retrieveMode = RETRIEVE_MODES[navigation][iteration];

          // Trigger end condition for one-shot iteration types if iterating and
          // we've already retrieved a record for this component in this iteration
          // pass.
          switch (retrieveMode)
          {
             case FIRST:
             case LAST:
             case UNIQUE:
                if (count > 0)
                {
                   return null;
                }
                break;

             default:
                break;
          }

          Joinable query = component.getQuery();

          // Force the query to clear its state at the beginning of a loop pass.
          if (count == 0)
          {
             query.reset(!resolveOnce, false);
          }

          // Perform component query record retrieval.
          if (peek)
          {
             switch (retrieveMode)
             {
                case NEXT:
                   return query.peekNext();

                case PREVIOUS:
                   return query.peekPrevious();

                case FIRST:
                   return query.peekFirst();

                case LAST:
                   return query.peekLast();

                default:
                   throw new UnsupportedOperationException(
                      "Unsupported record retrieval mode for peeking:  " + retrieveMode);
             }
          }
          else
          {
             boolean available = false;
             switch (retrieveMode)
             {
                case FIRST:
                   available = query.first(lockType);
                   break;
                case LAST:
                   available = query.last(lockType);
                   break;
                case NEXT:
                   available = query.next(lockType);
                   break;
                case PREVIOUS:
                   available = query.previous(lockType);
                   break;
                case UNIQUE:
                   available = query.unique(lockType);
                   break;
                case CURRENT:
                   RecordBuffer[] buffers = query.getRecordBuffers();
                   int len = buffers.length;

                   if (!component.isOuter())
                   {
                      available = query.current(lockType);
                   }
                   else
                   {
                      boolean avail = true;
                      for (int i = 0; i < len && avail; i++)
                      {
                         avail = buffers[i].isAvailable();
                      }
                      if (avail)
                      {
                         available = query.current(lockType);
                      }
                   }

                   // getting the row after CURRENT is not easy because it may
                   // be loaded by a loadValues and not part of results or cursor
                   if (!available)
                   {
                      return null;
                   }

                   if (this.isScrolling())
                   {
                      Object[] row = new Object[len];
                      for (int i = 0; i < len; i++)
                      {
                         row[i] = buffers[i].getCurrentRecord();
                      }

                      return row;
                   }
                   else
                   {
                      return query.getRow();
                   }

                default:
                   throw new UnsupportedOperationException(
                      "Unknown record retrieval mode:  " + retrieveMode);
             }

             if (available)
             {
                return query.getRow();
             }
          }

          return null;
       }

       /**
        * Translate the WHERE clause for all {@link #getComponents() components}.
        * <p>
        * This is done only once.
        */
       private void translateWhere()
       {
          if (whereTranslated)
          {
             return;
          }

          whereTranslated = true;
          ArrayList<CompoundComponent> compoundComponents = getComponents();
          for (int i = 0; i < compoundComponents.size(); i++)
          {
             CompoundComponent cc = compoundComponents.get(i);
             cc.getQuery().translateWhere();
          }
       }

       /**
        * Releases buffers from all components. The row position is not affected.
        */
       @Override
       public void releaseBuffers()
       {
          for (RecordBuffer buf : getRecordBuffersInternal())
          {
             buf.release(false);
          }
       }

       /**
        * Adds a new constraint term to the where predicate of this query, effectively filtering out
        * any rows whose referenced field does not match the requested value. Semantically, the query
        * will select rows that match
        * <p>
        * {@code new-where := (fr = val) AND (old-query)}.
        * <p>
        * Note: The constraints are exclusive meaning that if constraints will be added for same
        * field of same buffer, the query will return an empty result because the field cannot be at
        * the same time equals to two different values.
        * <p>
        * Use {@link #clearDynamicFilters} to clean all dynamically added criteria and restore the
        * sorting order of the query to its original form.
        *
        * @param   fr
        *          A reference to field used as filtering criterion. Only the rows that matches the
        *          specified value for this field will be selected. Must not be {@code null}.
        * @param   val
        *          The filtering value for this constraint. Can be {@code null} or {@code unknown}, in
        *          which case the SQL {@code null} value will be assumed. In this case the predicate
        *          will look like:
        *           <p>
        *          {@code new-where := (fr IS  NULL) AND (old-query)}
        * @param   format
        *          The format to be used when comparing values. The reason for this parameter is that
        *          for some datatype (ex: {@code decimal}) the on-screen value can be different from
        *          the database (because of rounding).
        *
        * @return  {@code true} if the filtering constraint was successfully added. If the field
        *          reference is not related to the buffer(s) of this query, {@code false} is returned.
        */
       @Override
       public boolean addDynamicFilter(FieldReference fr, BaseDataType val, String format)
       {
          for (int i = 0; i < originalComponents.size(); i++)
          {
             CompoundComponent comp = originalComponents.get(i);
             Joinable query = comp.getQuery();
             if (query.addDynamicFilter(fr, val, format))
             {
                components = null; // force re-optimize server-side joined queries
                return true;
             }
          }
          return false;
       }

       /**
        * Clears any dynamically filters added at runtime. The query predicate will be restored to its
        * form from generated at conversion time.
        */
       @Override
       public void clearDynamicFilters()
       {
          for (int i = 0; i < originalComponents.size(); i++)
          {
             CompoundComponent comp = originalComponents.get(i);
             comp.getQuery().clearDynamicFilters();
          }

          components = null; // force re-optimize server-side joined queries
       }

       /**
        * Inner class which attempts to optimize the enclosing, compound query's individual query
        * components into server-side joins. This object will inspect and analyze the components to
        * determine whether they are likely to be suitable, server-side join candidates. If so, it
        * will create one or more instances of {@link AdaptiveQuery} or {@link PreselectQuery}
        * (depending on whether the original compound query is set to preselect its results) from
        * the suitable candidates it finds. Only query components from adjacent loops are joined,
        * as many as possible while still maintaining the likelihood that the join will be performant
        * (which is the whole point of the optimization).
        * <p>
        * The analysis considers factors such as the type of iteration (NEXT, FIRST, LAST) of the
        * original components, their substitution arguments, whether they utilize client-side
        * filtering, the complexity of their where clauses, the columns on which they would join and
        * the participation of those columns in database indexes, and of course whether the tables
        * being queried reside in the same physical database.
        * <p>
        * Zero or more optimized joins may result from this analysis. If the algorithm determines no
        * join is advisable, the original query components will be used unchanged. The algorithm will
        * attempt to create as few, multi-table, server-side join queries as possible from the
        * original components, ideally 1; however, the resulting, optimized queries may not represent
        * the entire original query, or even adjacent sets of nested queries. For example, a compound
        * query with 5 original components may be optimized to a single query representing all 5
        * tables; or a query for 2 of the tables and another for the other 3; 5 separate separate
        * queries (i.e., the original state); 2 and 2; 4 and 1; etc. The original, enclosing compound
        * query still will drive the record retrieval mechanics, it will just have fewer components
        * to which it delegates the actual work.
        */
       private class Optimizer
       {
          /**
           * Constructor which immediately performs the analysis and if possible, the optimizations.
           * Loops through each of the original query components and submits them to an analysis of
           * their suitability to be joined together at the database.
           */
          private Optimizer()
          {
             int max = originalComponents.size();
             components = new ArrayList<>(max);

             ArrayList<ArrayList<CompoundComponent>> grouped = new ArrayList<>(max);
             ArrayList<CompoundComponent> currentGroup = new ArrayList<>(max);
             grouped.add(currentGroup);

             // chunk original query components into sub-groups of server-joinable components; each
             // sub-group will have one component if no join is possible to that component, multiple
             // components if a server join among those components is possible, or may be empty

             boolean top = true;
             int i = 0;

             boolean finer = log.isLoggable(Level.FINER);

             Iterator<CompoundComponent> iter = originalComponents.iterator();
             while (iter.hasNext())
             {
                i++;

                CompoundComponent currentComp = iter.next();

                boolean possible = isServerJoinPossible(currentGroup, currentComp, top, i);

                if (activeIndex == i - 1)
                {
                   // set activeIndex to index the correct component in the potentially reduced list
                   // of joined components
                   activeIndex = grouped.size() - (possible ? 1 : 0);
                }

                if (possible)
                {
                   // the current component is join-able; add it to the current group
                   currentGroup.add(currentComp);

                   // nothing more to do for this component
                   continue;
                }

                if (currentGroup.isEmpty())
                {
                   // current group is empty and we determined that a downward join was not suitable
                   // for this component, which is the only type of join we could achieve at this
                   // point; all we can do is add the component to the group and create a new group
                   // for the next pass
                   currentGroup.add(currentComp);
                   currentGroup = new ArrayList<>(max);
                   grouped.add(currentGroup);
                   top = false;

                   continue;
                }

                if (finer)
                {
                   log.log(Level.FINER, "no upward server join", new Throwable());
                }

                // current group is not empty and we determined that an upward join was not suitable
                // for this component, so we have to close off the last group and create a new one
                currentGroup = new ArrayList<>(max);
                grouped.add(currentGroup);
                top = false;

                // if there are no components after this one, we are done...
                if (!iter.hasNext())
                {
                   // ...but we have to store the last failing component before we leave
                   currentGroup.add(currentComp);

                   break;
                }

                // now try again to see if the current component is suitable for a downward join as
                // the top of the new group
                possible = isServerJoinPossible(currentGroup, currentComp, top, i);

                currentGroup.add(currentComp);

                if (!possible)
                {
                   if (finer)
                   {
                      log.log(Level.FINER, "no downward server join", new Throwable());
                   }

                   // the current component is not suitable for a downward join either, so create a
                   // new group for the next pass
                   currentGroup = new ArrayList<>(max);
                   grouped.add(currentGroup);
                }
             }

             for (int j = 0; j < grouped.size(); j++)
             {
                ArrayList<CompoundComponent> subgroup = grouped.get(j);
                int subSize = subgroup.size();

                if (subSize == 0)
                {
                   continue;
                }

                CompoundComponent comp;

                if (subSize == 1)
                {
                   // nothing to join; use original component
                   comp = subgroup.get(0);
                }
                else
                {
                   Joinable optimizedQuery;
                   if (preselect)
                   {
                      optimizedQuery = serverJoinPreselect(subgroup);
                   }
                   else
                   {
                      optimizedQuery = serverJoinAdaptive(subgroup);
                   }

                   if (optimizedQuery == null)
                   {
                      components.addAll(subgroup);
                      continue;
                   }

                   initializeComponentQuery(optimizedQuery);

                   if (log.isLoggable(Level.FINE))
                   {
                      log.fine("Optimized join of " + subSize + " compound query components");
                   }

                   // assign the iteration type of the outermost query in the original subgroup to the new,
                   // joined component
                   int iteration = subgroup.get(0).getIteration();
                   comp = new CompoundComponent(optimizedQuery, iteration, subgroup.get(0).isOuter());
                }

                components.add(comp);
             }

             setComponentsBrowsed();
          }

          /**
           * Check if the two components use the same SQL table, in any referenced buffer.
           *
           * @param    c1
           *           The first component.
           * @param    c2
           *           The second component.
           *
           * @return   {@code true} if at least a buffer in one component has the same SQL name as in the other
           *           component.
           */
          private boolean hasSameSqlTable(CompoundComponent c1, CompoundComponent c2)
          {
             RecordBuffer[] buffs1 = c1.getQuery().getRecordBuffers();
             Set<String> sqlNames = new HashSet<>();
             for (int i = 0; i < buffs1.length; i++)
             {
                sqlNames.add(buffs1[i].getTable());
             }
             RecordBuffer[] buffs2 = c2.getQuery().getRecordBuffers();
             for (int i = 0; i < buffs2.length; i++)
             {
                if (sqlNames.contains(buffs2[i].getTable()))
                {
                   return true;
                }
             }

             return false;
          }

          /**
           * Determine whether a server-side join is possible (and advisable) for the given component
           * and the zero or more candidates above it in the nesting. The analysis visits the original
           * query components from outermost to innermost nesting, so the component being tested will
           * always represent the innermost query being visited.
           *
           * @param   candidates
           *          The list of candidate components, to the last of which the current candidate
           *          (<code>nextComp</code>) will potentially have an upward join. This list may
           *          be empty, indicating that <code>nextComp</code> is potentially the outermost
           *          query being considered for the current group of components being joined. Each
           *          component in the list represents a single-table component query. This argument
           *          may not be <code>null</code>.
           * @param   nextComp
           *          The single-table query currently being analyzed to join (upward) with the last
           *          of the queries in the <code>candidates</code> list (if any), and potentially
           *          downward with additional component queries (if any).
           * @param   top
           *          <code>true</code> if this component and the list of <code>candidates</code>
           *          are at the outermost nesting level.
           * @param   originalPosition
           *          The one-based position of <code>nextComp</code> in the original list of query
           *          components. Currently only used for debug logging.
           *
           * @return  <code>true</code> if the algorithm has determined that <code>nextComp</code>
           *          should be part of a server-side join.
           */
          private boolean isServerJoinPossible(ArrayList<CompoundComponent> candidates,
                                               CompoundComponent nextComp,
                                               boolean top,
                                               int originalPosition)
          {
             int iteration = nextComp.getIteration();
             Joinable query = nextComp.getQuery();
             RecordBuffer buffer = query.getRecordBuffers()[0];
             boolean fine = log.isLoggable(Level.FINE);

             // calculate the number of fields in the possible optimized component in an worst case scenario
             // if the number is bigger than the max admitted by the dialect, we should not optimize
             int totalFields = buffer.getDmoInfo().getFieldCount(false);
             for (CompoundComponent candidate : candidates)
             {
                totalFields += candidate.getQuery().getRecordBuffers()[0].getDmoInfo().getFieldCount(false);

                if (hasSameSqlTable(candidate, nextComp))
                {
                   return false;
                }
             }

             if (totalFields > buffer.getDialect().getMaxEntriesInSelect())
             {
                if (fine)
                {
                   log.fine(" --- too many columns: ");
                }
                return false;
             }

             if (candidates.size() == 1 && candidates.get(0).isOuter())
             {
                if (fine)
                {
                   log.fine("   --- can't optimize if the previous component is the first one"
                         + " and is an outer component.");
                }

                return false;
             }

             String where = query.getOriginalWhere();

             if (fine)
             {
                StringBuilder buf = new StringBuilder(top ? "*" : " ");
                buf.append(originalPosition);

                switch (iteration)
                {
                   case NEXT:
                      buf.append("N");
                      break;
                   case FIRST:
                      buf.append("F");
                      break;
                   case LAST:
                      buf.append("L");
                      break;
                   default:
                      buf.append(iteration);
                      break;
                }

                AbstractJoin legacyJoin = query.getJoin();

                buf.append(legacyJoin != null ? "J" : " ");
                buf.append(" ");
                buf.append(buffer.getTable());
                buf.append(" [");
                buf.append(where);
                buf.append("]");

                if (legacyJoin != null)
                {
                   buf.append(" / [");

                   FQLExpression[][] fqls = legacyJoin.getFQL();
                   FQLExpression joinWhere = new FQLExpression();
                   for (FQLExpression[] fql : fqls)
                   {
                      // log a generic query, where all fields are not unknown
                      joinWhere.append(fql[AbstractJoin.NOT_NULL_FIELD]);
                   }

                   buf.append(joinWhere.toFinalExpression(false));
                   buf.append("]");
                }

                log.fine(buf.toString());
             }

             boolean isFirst = candidates.isEmpty();

             switch (iteration)
             {
                case FIRST:
                case LAST:
                case NEXT:
                   break;
                default:
                   return false;
             }

             // presence of client-side where expression disallows server-side join
             if (query.hasWhereExpression())
             {
                if (fine)
                {
                   log.fine("   --- client-side where expression");
                }

                return false;
             }

             // if the query references composites, it will not join well
             // TODO: fix this; it is due to the fact that in the generated SQL, Hibernate creates
             // an inner join on the list's parent id, but the test for the composite's index is
             // relegated to the where clause; this works OK for single entity queries, but compounds
             // the query plan's complexity when joining multiple entities
             Object[] args = query.getArgs();
             FQLPreprocessor fqlp = null;
             if (where != null)
             {
                try
                {
                   fqlp = FQLPreprocessor.get(where,
                                              buffer.getDatabase(),
                                              buffer.getDialect(),
                                              query.getReferencedBuffers(),
                                              args,
                                              null,
                                              false,
                                              null,
                                              null,
                                              true,
                                              null);
                   if (fqlp.hasAnsiJoins() || fqlp.hasContains())
                   {
                      if (fine)
                      {
                         log.fine("   --- composite reference");
                      }

                      return false;
                   }
                }
                catch (PersistenceException exc)
                {
                   if (log.isLoggable(Level.WARNING))
                   {
                      log.log(Level.WARNING, "Error optimizing compound query", exc);
                   }

                   return false;
                }
             }

             // There are 2 cases when we should not allow the optimization regarding "contains" keyword:
             // 1) when a previous component has a contains keyword with non-constants expression and
             //    the current component is outer
             // 2) when a previous component is outer and
             //    the current one has a contains keyword with non-constant expression
             boolean nonConstantContains = fqlp != null && fqlp.hasNonConstantContains();
             int len = candidates.size();
             for (int i = 0; i < len; i++)
             {
                CompoundComponent candidate = candidates.get(i);
                Joinable candidateQuery = candidate.getQuery();
                RecordBuffer candidateBuffer = candidateQuery.getRecordBuffers()[0];
                Object[] candidateArgs = candidateQuery.getArgs();
                String candidateWhere  = candidateQuery.getOriginalWhere();
                FQLPreprocessor candidateFqlp = null;

                if (candidateWhere != null)
                {
                   try
                   {
                      candidateFqlp = FQLPreprocessor.get(candidateWhere,
                                                          candidateBuffer.getDatabase(),
                                                          candidateBuffer.getDialect(),
                                                          candidateQuery.getReferencedBuffers(),
                                                          candidateArgs,
                                                          null,
                                                          false,
                                                          null,
                                                          null,
                                                          true,
                                                          null);
                   }
                   catch (PersistenceException exc)
                   {
                       if (log.isLoggable(Level.WARNING))
                       {
                          log.log(Level.WARNING, "Error optimizing compound query", exc);
                       }

                      return false;
                   }

                   if (nextComp.isOuter() && candidateFqlp.hasNonConstantContains())
                   {

                      if (fine)
                      {
                         log.fine("   --- can't optimize if a previous component has a contains keyword " +
                                  "with non-constants expression and the current component is outer");
                      }

                      return false;
                   }

                   if (candidate.isOuter() && nonConstantContains)
                   {

                      if (fine)
                      {
                         log.fine("   --- can't optimize if a previous component is outer and the " +
                                  "current one has a contains keyword with non-constants expression");
                      }

                      return false;
                   }
                }
            }

             // no further checks are necessary if this is the first component in a series
             if (isFirst)
             {
                if (fine)
                {
                   log.fine("   +++ first in new series (" + (top ? "" : "not ") + "outermost)");
                }

                return true;
             }

             // server-side join only possibly if query components reference the same database
             Database database = buffer.getDatabase();
             Joinable first = candidates.get(0).getQuery();
             Database firstDatabase = first.getRecordBuffers()[0].getDatabase();
             if (!database.equals(firstDatabase))
             {
                if (fine)
                {
                   log.fine("   --- databases do not match");
                }

                return false;
             }

             int countMulti = 0;
             int countSingle = 0;
             CompoundComponent parent = null;
             int parentIteration = NONE;

             for (int i = 0; i < candidates.size(); i++)
             {
                CompoundComponent comp = candidates.get(i);
                parent = comp;
                parentIteration = parent.getIteration();
                switch (comp.getIteration())
                {
                   case NEXT:
                      countMulti++;
                      break;
                   case FIRST:
                   case LAST:
                      countSingle++;
                      break;
                   default:
                      break;
                }
             }

             boolean mixed = countSingle > 0 && countMulti > 0;

             switch (iteration)
             {
                case NEXT:
                   if (mixed && parentIteration != NEXT)
                   {
                       // mixed iteration types should be contiguous to simplify the sorting
                       if (fine)
                       {
                          log.fine("   --- non-contiguous iteration types");
                       }

                       return false;
                   }
                   else if (countSingle > 0 && countMulti == 0 && iteration == NEXT)
                   {
                      if (fine)
                      {
                         log.fine("   --- NEXT can't follow FIRST/LAST/UNIQUE");
                      }

                      return false;
                   }
                   break;
                case FIRST:
                case LAST:
                   if (mixed && parentIteration == NEXT)
                   {
                      // mixed iteration types should be contiguous to simplify the sorting
                      if (fine)
                      {
                         log.fine("   --- non-contiguous iteration types");
                      }

                      return false;
                   }
                   break;
                default:
                   break;
             }

             // if not in preselect mode, we have to be careful about enabling server-side joins,
             // since a poor join query can be much slower than the original nested loops, and we may
             // well not need the full results, as in preselect mode, so the cost won't necessarily
             // be amortized
             if (!preselect)
             {
                // need to keep the sorting as simple as possible to make sure a server join that is
                // supposed to improve performance doesn't produce a very inefficient query plan; a
                // NEXT iteration always will add one or more elements to ORDER BY; FIRST/LAST won't

                // make some best-guess decisions on an appropriate join strategy, based on the
                // current component and its enclosing components

                switch (iteration)
                {
                   case NEXT:
                      /*
                      if (countMulti > 1)
                      {
                         // limit to 2 levels of NEXT iteration components
                         return false;
                      }
                      */
                      /*
                      if (!top && parentIteration == NEXT)
                      {
                         // do not allow NEXT/NEXT joins below the outermost nesting loop; if the
                         // resulting server-side join is at all inefficient, the impact would be
                         // magnified
                         return false;
                      }
                      */
                      if (parentIteration == NEXT)
                      {
                         Joinable pQuery = parent.getQuery();
                         if (!validateJoin(pQuery, query, args, database, fqlp, true, fine))
                         {
                            return false;
                         }
                      }

                      break;

                   case FIRST:
                   case LAST:

    //                  if (parentIteration != NEXT)
    //                  {
                         // there's very little advantage (and usually a heavy penalty) to joining
                         // a FIRST/LAST to another FIRST/LAST
    //                     return false;
    //                  }
                      if (!validateJoin(parent.getQuery(), query, args, database, fqlp, false, fine))
                      {
                         return false;
                      }

                      break;

                   default:
                      break;
                }
             }

             if (fine)
             {
                log.fine("   +++ passed all validation");
             }

             return true;
          }

          /**
           * Validate whether a server-side join between <code>parentQuery</code> and
           * <code>query</code> is advisable to produce a performant result. At this point, calling
           * code has determined the join is physically possible; it is this method's job to
           * determine whether this would be a good idea. The join may be rejected at this point due
           * to perceived where clause or join parameter complexity, poor matches of join columns,
           * etc.
           *
           * @param   parentQuery
           *          The nesting query.
           * @param   query
           *          The nested query.
           * @param   args
           *          Query substitution parameters, if any, for the original query.
           * @param   database
           *          Database containing the tables referenced by <code>parentQuery</code> and
           *          <code>query</code>.
           * @param   fqlp
           *          FQL preprocessor used in the analysis of the query's where clause, if any. May
           *          by <code>null</code>.
           * @param   noOpenCrossJoin
           *          <code>true</code> if an unrestricted cross join from <code>query</code> to
           *          <code>parentQuery</code> must be rejected; <code>false</code> if it is
           *          permissible.
           * @param   fine
           *          <code>true</code> if FINE level logging is enabled, else <code>false</code>.
           *
           * @return  <code>true</code> if the join is recommended, <code>false</code> if rejected.
           */
          private boolean validateJoin(Joinable parentQuery,
                                       Joinable query,
                                       Object[] args,
                                       Database database,
                                       FQLPreprocessor fqlp,
                                       boolean noOpenCrossJoin,
                                       boolean fine)
          {
             RecordBuffer parentBuf = parentQuery.getRecordBuffers()[0];
             AbstractJoin legacyJoin = query.getJoin();

             if (noOpenCrossJoin && args == null && legacyJoin == null)
             {
                // open cross join == bad
                if (fine)
                {
                   log.fine("   --- unrestricted cross join - no where clause or legacy join");
                }

                return false;
             }

             if (legacyJoin != null && parentBuf != legacyJoin.getInverse().buffer())
             {
                // a legacy join must reference the direct parent
                if (fine)
                {
                   log.fine(
                      "   --- join inverse does not reference buffer of immediate enclosing loop");
                }
                return false;
             }

             if (forceDatabaseJoin)
             {
                return true;
             }

             ArrayList<FieldReference> refList = null;
             if (args != null || legacyJoin != null)
             {
                refList = new ArrayList<>();

                if (args != null)
                {
                   for (Object arg : args)
                   {
                      if (arg instanceof FieldReference)
                      {
                         refList.add((FieldReference) arg);
                      }
                   }
                }

                if (legacyJoin != null)
                {
                   refList.addAll(legacyJoin.getUnresolvedParameters());
                }
             }

             if (refList != null)
             {
                FieldReference joinRef = null;
                Class<? extends DataModelObject> dmoIface = parentBuf.getDMOInterface();
                for (int i = 0; i < refList.size(); i++)
                {
                   FieldReference ref = refList.get(i);
                   boolean test;
                   if (ref.getDMO() != null)
                   {
                      // more exact: we know the actual buffer instances are the same
                      test = ref.getParentBuffer() == parentBuf;
                   }
                   else
                   {
                      // less exact: we know buffer types are the same; hopefully the actual buffer
                      // instances are, too; have to settle for this because ref.getParentBuffer()
                      // will throw an NPE when the ref's DMO is null
                      test = ref.getDMOInterface().equals(dmoIface);
                   }

                   if (test)
                   {
                      if (joinRef != null)
                      {
                         // don't want to join by more than one column
                         if (fine)
                         {
                            log.fine("   --- more than one join property");
                         }

                         return false;
                      }

                      joinRef = ref;
                   }
                }

                if (noOpenCrossJoin && joinRef == null)
                {
                   // open cross join == bad
                   if (fine)
                   {
                      log.fine("   --- unrestricted cross join - no join property identified");
                   }

                   return false;
                }

                if (joinRef != null)
                {
                   try
                   {
                      // we need to know the index of the joining reference in the array of
                      // substitution parameters
                      Integer refSubstIndex = null;

                      RecordBuffer buffer = query.getRecordBuffers()[0];

                      if (fqlp == null && legacyJoin != null)
                      {
                         Object[] parms = legacyJoin.getUnresolvedParameters().toArray();
                         FQLExpression[][] fqls = legacyJoin.getFQL();

                         FQLExpression joinWhere = new FQLExpression();
                         for (int j = 0; j < fqls.length; j++)
                         {
                            boolean nullArg = (parms.length != 0) &&
                                              ((parms[j] == null) ||
                                               (parms[j] instanceof BaseDataType &&
                                                ((BaseDataType) parms[j]).isUnknown()));
                            joinWhere.append(fqls[j][nullArg ? AbstractJoin.NULL_FIELD : AbstractJoin.NOT_NULL_FIELD]);
                         }

                         fqlp = FQLPreprocessor.get(joinWhere.toFinalExpression(false),
                                                    buffer.getDatabase(),
                                                    buffer.getDialect(),
                                                    query.getReferencedBuffers(),
                                                    parms,
                                                    null,
                                                    false,
                                                    null,
                                                    null,
                                                    true,
                                                    null);
                         for (int i = 0; i < parms.length; i++)
                         {
                            if (parms[i] == joinRef)
                            {
                               refSubstIndex = i;
                            }
                         }
                      }

                      if (refSubstIndex == null && args != null)
                      {
                         for (int i = 0; i < args.length; i++)
                         {
                            if (args[i] == joinRef)
                            {
                               refSubstIndex = i;
                            }
                         }
                      }

                      if (fqlp != null && refSubstIndex != null)
                      {
                         if (!evaluatePropertyMatches(fqlp, refSubstIndex, buffer, fine))
                         {
                            return false;
                         }
                      }
                      else
                      {
                         if (fine)
                         {
                            if (fqlp == null)
                            {
                               log.fine("   --- unexpected - missing fql preprocessor");
                            }
                            else
                            {
                               log.fine("   --- unexpected - error finding join ref in args");
                            }
                         }

                         return false;
                      }
                   }
                   catch (PersistenceException exc)
                   {
                      if (log.isLoggable(Level.WARNING))
                      {
                         log.log(Level.WARNING, "Error retrieving index data", exc);
                      }

                      return false;
                   }
                }
             }

             return true;
          }

          /**
           * Analyze the property match expressions found in the current component's where clause or
           * join clause FQL, to determine suitability for a server-side join. This phase of the
           * analysis considers the type of match forming the join and whether the property used for
           * the join is the leading component of any index on the backing table.
           *
           * @param   fqlp
           *          FQL preprocessor which has parsed the where or join clause.
           * @param   joinSubstIndex
           *          Index of the substitution parameter which represents the field reference used
           *          to join the table referenced by the current query to the table in the nesting
           *          query above it.
           * @param   buffer
           *          Record buffer referenced by the current query.
           * @param   fine
           *          <code>true</code> if FINE level logging is enabled, else <code>false</code>.
           *
           * @return  <code>true</code> if an analysis of the property match expressions in the FQL
           *          informs a decision to create a server-side join, else <code>false</code>.
           *
           * @throws  PersistenceException
           *          if there is an error looking up index information.
           */
          private boolean evaluatePropertyMatches(FQLPreprocessor fqlp,
                                                  int joinSubstIndex,
                                                  RecordBuffer buffer,
                                                  boolean fine)
          throws PersistenceException
          {
             List<FQLPreprocessor.PropertyMatch> matches = fqlp.getPropertyMatches();
             if (matches == null)
             {
                if (fine)
                {
                   log.fine("   --- no match expression found in where clause");
                }

                return false;
             }

             String joinProp = null;
             int i = 0;

             for (FQLPreprocessor.PropertyMatch pm : matches)
             {
                if (pm.rvalType == HQLParserTokenTypes.SUBST && i++ == joinSubstIndex)
                {
                   if (pm.operator != HQLParserTokenTypes.EQUALS)
                   {
                      if (fine)
                      {
                         log.fine("   --- server join requires equality match");
                      }

                      return false;
                   }

                   joinProp = pm.property;
                }

                if (!pm.alias.equals(buffer.getDMOAlias()))
                {
                   if (fine)
                   {
                      log.fine("   --- DMO alias in where clause does not reference current buffer");
                   }

                   return false;
                }
             }

             if (joinProp == null)
             {
                if (fine)
                {
                   log.fine("   --- unexpected - no reference to join property in where clause [" +
                            joinProp +
                            "]");
                }

                return false;
             }

             if (!buffer.getDmoInfo().isLeadingIndexComponent(joinProp))
             {
                if (fine)
                {
                   log.fine("   --- join property not a leading index component [" + joinProp + "]");
                }

                return false;
             }

             return true;
          }

    //      /**
    //       * Determine whether the given DMO property represents the leading component of any index
    //       * on the table represented by the given buffer.
    //       *
    //       * @param   buffer
    //       *          Record buffer which represents the table associated with the property.
    //       * @param   property
    //       *          DMO property to be tested.
    //       *
    //       * @return  <code>true</code> if <code>property</code> is a leading index component; else
    //       *          <code>false</code>.
    //       *
    //       * @throws  PersistenceException
    //       *          if there is an error looking up the backing table's indexes.
    //       */
    //      private boolean isLeadingIndexComponent(RecordBuffer buffer, String property)
    //      throws PersistenceException
    //      {
    //         String schema = buffer.getSchema();
    //         Database database = buffer.getDatabase();
    //         Class<? extends DataModelObject> dmoIface = buffer.getDMOInterface();
    //
    //         Iterator<LinkedHashSet<String>> indexIter =
    //            DMOIndex.indexProperties(schema, database, dmoIface);
    //         while (indexIter.hasNext())
    //         {
    //            LinkedHashSet<String> props = indexIter.next();
    //            Iterator<String> propsIter = props.iterator();
    //            if (propsIter.hasNext())
    //            {
    //               String leadingProp = propsIter.next();
    //               if (property.equals(leadingProp))
    //               {
    //                  return true;
    //               }
    //            }
    //         }
    //
    //         return false;
    //      }

          /**
           * Create a multi-table, adaptive query from the given compound query components, which
           * will be joined at the database server.
           *
           * @param   originals
           *          Compound query components which need to be joined.
           *
           * @return  Multi-table adaptive query. If the {@link AdaptiveQuery} detects that it
           *          is not able to be properly optimized, this will return {@code null} instead.
           */
          private AdaptiveQuery serverJoinAdaptive(ArrayList<CompoundComponent> originals)
          {
             AdaptiveQuery optimizedQuery = new AdaptiveQuery().initialize();

             List<AdaptiveComponent> acList = new ArrayList<>();

             for (int i = 0; i < originals.size(); i++)
             {
                CompoundComponent next = originals.get(i);
                int iteration = next.getIteration();
                boolean outer = next.isOuter();
                Joinable original = next.getQuery();
                AdaptiveComponent serverCopy =
                   original.makeAdaptiveServerJoinComponent(acList,
                                                            optimizedQuery,
                                                            iteration,
                                                            outer,
                                                            next);
                optimizedQuery.addComponent(serverCopy);
                acList.add(serverCopy);
             }

             if (optimizedQuery.earlyInvalidate())
             {
                return null;
             }

             return optimizedQuery;
          }

          /**
           * Create a multi-table, preselect query from the given compound query components, which
           * will be joined at the database server.
           *
           * @param   originals
           *          Compound query components which need to be joined.
           *
           * @return  Multi-table preselect query.
           */
          private PreselectQuery serverJoinPreselect(ArrayList<CompoundComponent> originals)
          {
             // generate sort clause from the original queries
             StringBuilder buf = new StringBuilder();
             for (int i = 0; i < originals.size(); i++)
             {
                CompoundComponent next = originals.get(i);
                if (next.getIteration() == NEXT)
                {
                   Joinable j = next.getQuery();
                   String sort = j.getSortPhrase();
                   if (sort != null && sort.length() > 0)
                   {
                      if (buf.length() > 0)
                      {
                         buf.append(", ");
                      }
                      buf.append(sort);
                   }
                }
             }
             String sort = buf.toString();
             if (sort.trim().length() == 0)
             {
                sort = null;
             }

             PreselectQuery optimizedQuery = new PreselectQuery().initialize(sort);

             List<QueryComponent> qcList = new ArrayList<>();

             for (int i = 0; i < originals.size(); i++)
             {
                CompoundComponent next = originals.get(i);
                int iteration = next.getIteration();
                boolean outer = next.isOuter();
                Joinable original = next.getQuery();
                QueryComponent serverCopy = original.makePreselectServerJoinComponent(qcList,
                                                                                      iteration,
                                                                                      outer);
                optimizedQuery.addComponent(serverCopy);
                qcList.add(serverCopy);
             }

             return optimizedQuery;
          }
       }

       /**
        * Filter that handles parameters passed to the component's subquery which handles the target
        * buffer.
        */
       private class SnapshotParameterFilter
       implements ParameterFilter
       {
          /** Index of target buffer in the set of buffers belonging to this compound query. */
          final int bufferIndex;

          /**
           * Create filter that handles parameters passed to the component's subquery which handles
           * the target buffer.
           *
           * @param   bufferIndex
           *          Index of target buffer in the set of buffers belonging to this compound query.
           */
          public SnapshotParameterFilter(int bufferIndex)
          {
             this.bufferIndex = bufferIndex;
          }

          /**
           * Handles parameters passed to the component's subquery which handles the target buffer:
           * according to the state kept in {@link #useSnapshot} it returns original parameters or
           * may substitute some of them with snapshot values for specific outer components.
           *
           * @param   parameters
           *          Component's subquery parameters.
           *
           * @return  see above.
           */
          @Override
          public Object[] filterParameters(Object[] parameters)
          {
             if (parameters == null || parameters.length == 0 || !useSnapshot[bufferIndex - 1])
             {
                return parameters;
             }

             Object[] res = new Object[parameters.length];
             for (int i = 0; i < parameters.length; i++)
             {
                Object arg = parameters[i];
                if (arg instanceof FieldReference)
                {
                   FieldReference ref = (FieldReference) arg;
                   RecordBuffer buffer = ref.getParentBuffer();
                   if (buffersMap.containsKey(buffer))
                   {
                      Record snapshot = null;
                      if (cursor != null)
                      {
                         Record[] referenceRow = cursor.getReferenceRow();
                         if (referenceRow != null)
                         {
                            snapshot = referenceRow[buffersMap.get(buffer)];
                         }
                      }

                      if (snapshot == null)
                      {
                         snapshot = buffer.getSnapshot();
                      }

                      res[i] = ref.getTemp(snapshot);
                      continue;
                   }
                }

                res[i] = arg;
             }

             return res;
          }
       }
    }