Validation.java
/*
** Module : Validation.java
** Abstract : Database record validation and persistence operation.
**
** Copyright (c) 2004-2024, Golden Code Development Corporation.
**
** -#- -I- --Date-- ---------------------------------------Description---------------------------------------
** 001 ECF 20200207 First revision. Validates non-nullable fields and unique constraints.
** OM 20200214 Added maximum index size validation implementation.
** CA 20200604 Track if during validation a newly created record was saved.
** ECF 20200909 Add StaleRecordException to those methods which can throw it.
** 002 ECF 20200925 Create session if needed.
** 20200929 Performance: UNION ALL multiple unique index validation SQL statements together to
** reduce number of queries prepared/sent.
** OM 20201001 Improved DMO manipulation performance by caching slow Property annotation access.
** AIL 20201020 Closed unclosed prepared statement.
** ECF 20201201 Fixed FFC invalidation for flush update case when record was validated previously.
** 20210126 Ensure all unvalidated unique indices are validated at flush time.
** ECF 20210117 Do not update DMO state when explicitly validating.
** OM 20210419 After a record is successfully inserted in database table, the buffer storing it is
** removed from early inserts list of dirty context.
** ECF 20210711 Confirm a unique constraint violation detected by query, in case unflushed changes to the
** suspect record clear the violation.
** ECF 20210713 Improved efficiency of unique constraint check by query.
** OM 20210927 FFC invalidates the records whose changed fields are used in query predicates.
** ECF 20211005 Reworked FFC invalidation for individual records for performance.
** 20211008 Slightly more aggressive FFC invalidation for non-indexed property updates.
** OM 20220701 In case of flush(), save the data in a micro-transaction before setting up the savepoint
** for the new innermost block.
** OM 20221012 The TriggerTracker are bound to DMO, not to the buffer they are contained into.
** TJD 20220504 Upgrade do Java 11 minor changes
** RAA 20221221 Modified the execution of sql's. They now pass through the SQLStatementLogger
** in case logging is intended.
** OM 20221117 Some properties (datetime-tz) may take a different number of positional parameters when
** used in insert/update or where predicate.
** CA 20230110 getDirtyIndices and getUnvalidatedIndices now return null instead of an empty BitSet, when
** no indices are found.
** 003 RAA 20230518 Replaced lambdas with method referencing when using SQLStatementLogger.
** 004 RAA 20230607 SQLs are now executed through SQLExecutor instead of SQLStatementLogger.
** 005 ECF 20230608 The write trigger should have a dedicated savepoint scoped to it.
** 006 RAA 20230615 Added SQL as a parameter when using SQLExecutor.
** 007 DDF 20230705 Replaced static field initialization with a method called at server bootstrap.
** 008 DDF 20231127 Used a dialect specific method to handle unique constraint violation.
** 009 RAA 20230724 Replaced FunctionWithException with QueryStatement.
** 010 RAA 20230802 Added support for VALIDATE keyword.
** RAA 20230802 Added support for H2 unique index validation.
** RAA 20230802 Split flush function into two for more straight-forward use when dealing with VALIDATE.
** RAA 20230802 In case a flush with VALIDATE fails, but it gets cured, a second flush is now attempted.
** RAA 20231127 Field allowDBUniqueCheck is now directly assigned instead of being passed as a parameter.
** RAA 20240316 Rebase fix: Added condition to skip the server-side index validation.
** 011 CA 20240320 When flush is not active and inside a FULL tx (thus the temp-table buffers are not
** auto-commit, use 'validateUniqueByQuery' for temp-tables.
** 012 AL2 20240329 Set allowDbUniqueCheck using a bracket pattern (set and reset in finally).
** Fixed case where the validation cure was not having an opened transaction.
** Disable allowDbUniqueCheck when doing a cure.
** 013 AI 20240418 Allow invalidate of reverse lookup even if dirty prop is indexed.
** 014 CA 20240507 Unique index validation must validate only dirty indexes, even for NEW records, if we are
** not in flush mode.
** 015 BS 20230511 Shared cache instance per persistent database.
** 016 CA 20240827 Notify that a record has been flushed - for non-temp records, this will allow invalidation
** of the FastFindCache for that DMO.
** 017 OM 20240909 FFCAche does not require knowledge of current tenant.
** 018 TJD 20221125 Update DirtyShareContext on every index update
** TJD 20230210 Mark DMO validated before trying to put it into nursery
** TJD 20230428 Null check for unique and non-unique dirty indices
** TJD 20240110 Fixes for DirtyShare cross session.
** TJD 20240308 Allow RecordNursery to work over records modified in another transaction
** 019 AL2 20241022 Reduce logging level when validation is cured.
** 020 AS 20241004 Added support for validating a subset of unique indexes, used in batch assigns.
** 021 AL2 20240829 Use different endpoints into the record nursery for transient and updated records.
** AL2 20240903 Send dirty props before flush to record nursery to be aware of the actual changes.
** AL2 20241112 Adapt to new record nursery remove signature.
** 022 AL2 20241217 Don't notify shadow non-transient record index updates to the record nursery.
** 023 AS 20250220 Refactored validateMaybeFlush to not return the getActiveUpdateDiffs.
*/
/*
** This program is free software: you can redistribute it and/or modify
** it under the terms of the GNU Affero General Public License as
** published by the Free Software Foundation, either version 3 of the
** License, or (at your option) any later version.
**
** This program is distributed in the hope that it will be useful,
** but WITHOUT ANY WARRANTY; without even the implied warranty of
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
** GNU Affero General Public License for more details.
**
** You may find a copy of the GNU Affero GPL version 3 at the following
** location: https://www.gnu.org/licenses/agpl-3.0.en.html
**
** Additional terms under GNU Affero GPL version 3 section 7:
**
** Under Section 7 of the GNU Affero GPL version 3, the following additional
** terms apply to the works covered under the License. These additional terms
** are non-permissive additional terms allowed under Section 7 of the GNU
** Affero GPL version 3 and may not be removed by you.
**
** 0. Attribution Requirement.
**
** You must preserve all legal notices or author attributions in the covered
** work or Appropriate Legal Notices displayed by works containing the covered
** work. You may not remove from the covered work any author or developer
** credit already included within the covered work.
**
** 1. No License To Use Trademarks.
**
** This license does not grant any license or rights to use the trademarks
** Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
** of Golden Code Development Corporation. You are not authorized to use the
** name Golden Code, FWD, or the names of any author or contributor, for
** publicity purposes without written authorization.
**
** 2. No Misrepresentation of Affiliation.
**
** You may not represent yourself as Golden Code Development Corporation or FWD.
**
** You may not represent yourself for publicity purposes as associated with
** Golden Code Development Corporation, FWD, or any author or contributor to
** the covered work, without written authorization.
**
** 3. No Misrepresentation of Source or Origin.
**
** You may not represent the covered work as solely your work. All modified
** versions of the covered work must be marked in a reasonable way to make it
** clear that the modified work is not originating from Golden Code Development
** Corporation or FWD. All modified versions must contain the notices of
** attribution required in this license.
*/
package com.goldencode.p2j.persist.orm;
import java.math.*;
import java.sql.*;
import java.util.*;
import java.util.logging.Level;
import com.goldencode.p2j.directory.*;
import com.goldencode.p2j.persist.*;
import com.goldencode.p2j.persist.dialect.*;
import com.goldencode.p2j.persist.orm.SQLExecutor.QueryStatement;
import com.goldencode.p2j.persist.orm.types.*;
import com.goldencode.p2j.util.*;
import com.goldencode.p2j.util.logging.CentralLogger;
/**
* An object which performs validation of the state of a DMO, and optionally flushes the DMO's
* data to the database, if the validation passes successfully. Before validation is performed,
* this object determines whether validation and flushing is actually necessary, based on the
* state of the DMO. This determination can be overridden to force validation under certain
* conditions.
* <p>
* Validation consists of an optional (configurable) maximum index size check, a non-null field
* check, and a unique constraint violation check.
* <p>
* Instances of this class are meant for a single use.
*/
public class Validation
implements DmoState
{
/** Logger */
private static final CentralLogger LOG = CentralLogger.get(Validation.class);
/** Record buffer requesting validation */
private final RecordBuffer buffer;
/** Record whose data is to be validated and optionally flushed to the database */
private final BaseRecord dmo;
/** Incubator for newly created records, until they are first flushed to the database */
private final RecordNursery nursery;
/** The name of the buffer. It is used when printing the error message, if any. */
private final String bufferName;
/** Tracks unique constraints in uncommitted transactions */
private final UniqueTracker uniqueTracker;
/** Context-local API helper for unique constraint tracking */
private final UniqueTracker.Context uniqueTrackerCtx;
/** Whether to flush the record; {@code false} to validate only */
private final boolean flush;
/** Whether to update the DMO state as a side effect of validation */
private final boolean updateState;
/**
* The optional multiplex value, in case of validation of temp-table, otherwise ignored. It is only used
* when invalidating the FastFind cache for the current table.
*/
private final Integer multiplex;
/** Flag indicating the record was flushed to the database (either newly inserted or updated) */
private boolean flushed = false;
/**
* The maximum permitted size for all combined fields of an index. By default this
* is equal to Progress version 10.x limit of 1971 bytes.
* <p>
* Set it to 0 to disable P4GL index-key limit check. The dialect specific limit is still
* enforced by each dialect because otherwise irrecoverable errors may occur when persisting
* records with wider key length.
*/
private static int P4GL_MAX_INDEX_SIZE = DBUtils.DEFAULT_MAX_INDEX_SIZE;
/**
* Constructor.
*
* @param buffer
* Record buffer which requested validation be performed.
* @param dmo
* Record whose data is to be validated and optionally flushed to the database.
* @param multiplex
* The optional multiplex value, in case of validation of temp-table, otherwise ignored. It is
* used only when invalidating the FastFind cache for the current table. Must be {@code null} for
* permanent tables. Must be positive for temp-tables.
* @param bufferName
* The name of the buffer. It is used when printing the error message, if any.
* @param nursery
* Incubator for newly created records, until they are first flushed to the database.
* @param uniqueTracker
* Tracks unique constraints in uncommitted transactions.
* @param uniqueTrackerCtx
* Context-local API helper for unique constraint tracking.
* @param flush
* {@code true} to validate and persist DMO's data; {@code false} to validate only.
* @param updateState
* Update the DMO's index and validation state as a side effect of validation. This should be set
* to {@code true} for implicit/organic validation; {@code false} for explicit validation.
*/
public Validation(RecordBuffer buffer,
BaseRecord dmo,
Integer multiplex,
String bufferName,
RecordNursery nursery,
UniqueTracker uniqueTracker,
UniqueTracker.Context uniqueTrackerCtx,
boolean flush,
boolean updateState)
{
this.buffer = buffer;
this.dmo = dmo;
this.multiplex = multiplex;
this.bufferName = bufferName;
this.nursery = nursery;
this.uniqueTracker = uniqueTracker;
this.uniqueTrackerCtx = uniqueTrackerCtx;
this.flush = flush;
this.updateState = updateState;
}
/**
* Constructs and returns the message that is printed with error 132 when the attempt to save (update or
* insert) a record will lead to a collision in a unique index.
*
* @param uIndex
* The index that failed validation, as a {@code BitSet} of its components.
* @param bufferName
* The legacy buffer name that failed the validation.
* @param dmo
* The record currently stored in buffer that caused the validation failure.
*
* @return The message to be displayed when a unique index fails validation (error 132).
*/
public static String getFailUniqueIndexMessage(BitSet uIndex, String bufferName, BaseRecord dmo)
{
if (uIndex == null)
{
return "unknown";
}
PropertyMeta[] props = dmo._recordMeta().getPropertyMeta(false);
StringBuilder sb = new StringBuilder();
sb.append(bufferName).append(" already exists with");
for (int i = uIndex.nextSetBit(0); i >= 0; i = uIndex.nextSetBit(i + 1))
{
Object datum = dmo.data[i];
if (props[i].getType() == recid.class)
{
// RECID values are not displayed here
continue;
}
else
{
if (props[i].getType() == character.class)
{
datum = "\"" + datum + "\"";
}
else
{
BaseDataType asBDT = props[i].getDataHandler().convert(datum);
datum = asBDT.toStringMessage();
}
}
sb.append(' ').append(props[i].getLegacyName()).append(' ').append(datum);
}
return sb.toString();
}
/**
* This method will check whether the record is in a state which requires validation and
* flushing, and will perform both if it is. The flush is an optional operation, and is only
* performed if this object was constructed with the {@code persist} option set to {@code
* true}.
* <p>
* The successful completion of this method indicates that either validation and flushing
* were determined to not yet be appropriate, or that they were, and the operation(s)
* completed successfully. Callers should check the state of the record after this method
* returns to determine which case it was.
* <p>
* If validation is determined to be appropriate, but it fails, an exception is thrown.
*
* @throws PersistenceException
* if there was a database error during validation or data persistence. This
* generally will not be recoverable.
* @throws ValidationException
* if there was a validation error. This represents an application level data
* validation failure and is recoverable.
* @throws StaleRecordException
* if the record being validated/flushed is stale. A stale record should not get this
* far under normal circumstances.
*/
public void validateMaybeFlush()
throws PersistenceException,
ValidationException,
StaleRecordException
{
try
{
boolean isNew = dmo.checkState(NEW);
// get the offset of the property updated
BitSet offsets = dmo.getActiveOffsets();
// validate mandatory constraint, if any, on the updated field
if (multiplex == null)
{
// mandatory constraints are only enforced for persistent tables
checkNotNull(offsets);
if (isNew)
{
// if we're flushing, check all non-nullable properties; otherwise, only check dirty properties
int failingOffset = checkNotNull(!flush);
if (failingOffset >= 0)
{
failNotNull(failingOffset);
}
}
}
// TODO: this check needs to be split apart to only check those indices needing it, like we do with
// unique constraint validation
checkMaxIndexSize();
// validate fully updated unique indices, if any, or all indices if flushing or not updating index
// state
BitSet valUnique = dmo.getUnvalidatedIndices(offsets, flush || !updateState);
if (valUnique != null && !valUnique.isEmpty())
{
// validate unique indices (and flush, if flush was requested)
validateUniqueIndices(valUnique);
}
else if (flush || (buffer.isAutoCommit() && !dmo.needsValidation()))
{
// we need to flush to the database if:
// * we've been explicitly told to do so, but there were no remaining unique indices to validate;
// * we are auto-commit and there are no remaining indices to be updated
flush();
}
if (!flushed && updateState)
{
if (isNew)
{
// update the nursery if we did not already flush and:
// * the table has no remaining index needing update; or
// * any indices for a record were updated
boolean indexUpdated = false;
BitSet unique = dmo.getDirtyIndices(offsets, true, flush);
if (unique != null && !unique.isEmpty())
{
indexUpdated(unique, true);
indexUpdated = true;
}
BitSet nonunique = dmo.getDirtyIndices(offsets, false, flush);
if (nonunique != null && !nonunique.isEmpty())
{
indexUpdated(nonunique, false);
indexUpdated = true;
}
// update at least primary index
if (!indexUpdated)
{
nursery.indexUpdated(buffer, null, true, dmo);
}
}
else if (dmo.checkState(CHANGED))
{
nursery.indexUpdated(buffer, dmo);
}
dmo.markValidated(offsets);
}
}
catch (ValidationException exc)
{
nursery.remove(buffer, dmo);
throw exc;
}
}
/**
* Method called at server bootstrap that initializes values from the directory
* configuration. Until this method is called, default values are used.
*/
public static void bootstrap()
{
DirectoryService ds = DirectoryService.getInstance();
if (ds != null)
{
if (!ds.bind())
{
throw new RuntimeException("Directory bind failed");
}
try
{
P4GL_MAX_INDEX_SIZE = Utils.getDirectoryNodeInt(ds,
"persistence/max-index-size",
DBUtils.DEFAULT_MAX_INDEX_SIZE,
false);
}
finally
{
ds.unbind();
}
}
}
/**
* Validate the unique indices represented by the given index bit set. If the validation was configured
* to flush the record, it will be persisted (inserted or updated) if validation passes.
*
* @param check
* A bit set whose set bits represent the 0-based positions of the unique indices to be
* validated in the target record's unique index array, as stored in the DMO's {@link RecordMeta}
* structure.
*
* @throws PersistenceException
* if there was a database error during validation or data persistence. This
* generally will not be recoverable.
* @throws ValidationException
* if there was a validation error. This represents an application level data
* validation failure and is recoverable.
* @throws StaleRecordException
* if the record being validated/flushed is stale. A stale record should not get this
* far under normal circumstances.
*/
private void validateUniqueIndices(BitSet check)
throws PersistenceException,
ValidationException,
StaleRecordException
{
// unique constraint checking takes place in two phases:
// 1. we check whether any uncommitted transaction has made a matching update to a unique
// index;
// 2. we attempt to insert/update the DMO into the database, which will fail if there is a
// unique constraint violation (for validation-only, we roll back the change).
UniqueTracker.Token token = null;
Session session = buffer.getSession(true);
try
{
// validate against uncommitted transaction data; no-op for temp-table
token = uniqueTrackerCtx.lockAndChange(uniqueTracker, dmo, bufferName, flush);
boolean allowDBUniqueCheck;
boolean isCheckEmpty = check != null && check.isEmpty();
if (isCheckEmpty)
{
allowDBUniqueCheck = false;
}
else
{
allowDBUniqueCheck = canSkipServerSideUniqueCheck() && (flush || buffer.isAutoCommit());
}
if ((multiplex != null || !flush) && !isCheckEmpty && !allowDBUniqueCheck)
{
// validate by executing unique index queries
validateUniqueByQuery(check);
}
if (flush || buffer.isAutoCommit())
{
// validation will be done by the database as part of the flush (for persistent tables)
boolean oldValue = session.getPersister().setAllowDBUniqueCheck(allowDBUniqueCheck);
try
{
flush();
}
finally
{
// reset the persister allowDBUniqueCheck state
session.getPersister().setAllowDBUniqueCheck(oldValue);
}
}
// validation was performed and succeeded; changes may have been persisted to database;
// the new/dirty/changed state of the record will have have been updated by the save
// mark record as having passed validation (whether or not data was persisted)
dmo.updateState(session, INVALID, false);
}
catch (ValidationException exc)
{
if (token != null)
{
// token will be non-null when the UniqueTracker update was successful, but the
// database update was not; roll back tracker changes; no-op for temp-table
uniqueTrackerCtx.rollbackChange(uniqueTracker, token);
}
if (updateState)
{
// mark record as having failed validation; it will stay this way until it passes
dmo.updateState(session, INVALID, true);
}
throw exc;
}
finally
{
// unlock tracker's resources locked above, no matter how we exit; no-op for temp-tables
// and for no-unique-constraint case
uniqueTrackerCtx.unlock(uniqueTracker, token);
}
}
/**
* One or more indices were updated as the result of the record's properties being touched. Notify the
* record to update its meta state, and update the newly created record nursery.
*
* @param indices
* Bit set representing those indices that were updated.
* @param unique
* {@code true} if {@code indices} refers to unique indices; else {@code false}.
*/
private void indexUpdated(BitSet indices, boolean unique)
throws PersistenceException
{
for (int i = indices.nextSetBit(0); i >= 0; i = indices.nextSetBit(i + 1))
{
int idxUid = (i + 1) * (unique ? 1 : -1);
dmo.indexUpdated(idxUid);
}
nursery.indexUpdated(buffer, indices, unique, dmo);
}
/**
* Flush the record being validated to the database.
*
* @throws PersistenceException
* if there is any non-validation database error.
* @throws ValidationException
* if there is an error validating a unique or not-null constraint.
* @throws StaleRecordException
* if the record being flushed is stale. A stale record should not get this far under normal
* circumstances.
*/
private void flush()
throws PersistenceException,
ValidationException,
StaleRecordException
{
// notify the FastFindCache about the results which are about to be invalidated. In the event of
// an error persisting the record, we will have invalidated the cache unnecessarily, but it's better
// to lose the optimization than to get incorrect results in a parallel thread if it was accessing
// the cache between save and invalidate operations
RecordMeta recMeta = dmo._recordMeta();
FastFindCache ffc = FastFindCache.getInstance(recMeta.isTemporary(), buffer.getDatabase());
boolean wasNew = dmo.checkState(NEW);
if (wasNew)
{
// invalidate cached results for all indexes on insert
ffc.invalidate(recMeta.getDmoMeta().getId(), multiplex);
}
else
{
BitSet dirtyOffsets = dmo.getActiveOffsets();
BitSet dirtyUnique = dmo.getDirtyIndices(dirtyOffsets, true, false);
BitSet dirtyNonunique = dmo.getDirtyIndices(dirtyOffsets, false, false);
if ((dirtyUnique != null && !dirtyUnique.isEmpty()) ||
(dirtyNonunique != null && !dirtyNonunique.isEmpty()))
{
// one or more index was touched; invalidate cached results across entire indices
ffc.invalidate(recMeta.getDmoMeta().getId(), multiplex, dirtyUnique, dirtyNonunique);
}
BitSet dirtyProps = (BitSet) dmo.getDirtyProps().clone();
// if any property was touched, invalidate cached result(s) for the touched record
if (!dirtyProps.isEmpty())
{
ffc.invalidate(dmo.getClass().getName(), dmo.primaryKey(), dirtyProps);
}
}
Session session = buffer.getSession(true);
SavepointManager.PersistenceCode lastBlockSetup = null;
if (!dmo.checkState(NOUNDO))
{
// preemptively set lazy savepoints in the savepoint manager before we create a "micro" savepoint
// here, because otherwise, the save here will trigger the savepoint manager to lazily create them,
// and we do not want those savepoints nested within this one
SavepointManager savepointManager = session.getSavepointManager();
if (savepointManager != null)
{
// this is a no-op if we are not in a transaction or we are in a full transaction block
lastBlockSetup = savepointManager.lazilySetSavepoints(true);
}
}
// if not already fired, do it now. When the trigger is fired the oldDMO is reset, preventing double
// firing
buffer.maybeFireWriteTrigger();
BitSet dirtyProps = (BitSet) dmo.getDirtyProps().clone();
try
{
flushImpl(session, wasNew);
}
finally
{
nursery.remove(buffer, dmo, dirtyProps);
}
flushed = true;
if (lastBlockSetup != null)
{
lastBlockSetup.run();
}
}
/**
* The implementation of the flushing process to the database.
* In case a flush with VALIDATE fails, the intention is to see if it can be "cured". If that is the case,
* a normal flush will be attempted.
*
* @param session
* The session in use.
* @param wasNew
* Is the DMO marked with NEW?
*
* @throws PersistenceException
* if there is any non-validation database error.
* @throws ValidationException
* if there is an error validating a unique or not-null constraint.
* @throws StaleRecordException
* if the record being flushed is stale. A stale record should not get this far under normal
* circumstances.
*/
private void flushImpl(Session session,
boolean wasNew)
throws PersistenceException,
ValidationException,
StaleRecordException
{
// create a "micro" savepoint, so we can roll back the save on failure
Savepoint sp = session.setSavepoint();
boolean fullTx = false;
if (sp == null)
{
// we are not currently within a transaction, so create a short-lived transaction to persist
// the change
fullTx = session.beginTransaction(null);
}
boolean commit = true;
try
{
// this will perform the appropriate insert or update, based on the state of the DMO
session.save(dmo, true);
if (wasNew)
{
buffer.recordInserted();
}
}
catch (PersistenceException exc)
{
commit = false;
// rollback to the savepoint or rollback the micro-transaction
if (sp != null)
{
session.rollbackSavepoint(sp);
}
else if (fullTx)
{
session.rollback();
}
// go up the exception chain and look for a SQL exception:
Throwable cause = exc.getCause();
boolean isAllowDBUniqueCheck = session.getPersister().getAllowDBUniqueCheck();
while (cause != null &&
!(cause instanceof SQLException) &&
cause != cause.getCause())
{
cause = cause.getCause();
}
while (cause instanceof SQLException)
{
SQLException sqle = (SQLException) cause;
// UNIQUE VIOLATION - unique_violation
if (buffer.getDialect().isUniqueConstraintViolation(sqle))
{
// if the right index is found, this method will not return normally, instead it will
// throw a specific exception
validateUniqueByQuery(null);
if (isAllowDBUniqueCheck)
{
// We were not able to execute the query with VALIDATE, but a "cure" occurred,
// meaning that we can try executing the query without validation.
if (LOG.isLoggable(Level.FINE))
{
LOG.fine("Flush with VALIDATE failed due to " + sqle);
}
boolean oldAllowDBUniqueCheck = session.getPersister().setAllowDBUniqueCheck(false);
try
{
flushImpl(session, wasNew);
}
finally
{
session.getPersister().setAllowDBUniqueCheck(oldAllowDBUniqueCheck);
}
return;
}
break;
}
// if this exception comes from H2, then we need to move to the inner exception (a.k.a.
// JdbcSQLIntegrityConstraintViolationException), because the outer one (SQLException) does not
// have the SQLState set accordingly.
cause = sqle.getCause();
}
// if the exception could not be identified as a SQL UNIQUE VIOLATION exception or the
// generateUniqueIndexesCondition() method could not identify the right index to compose the
// P4GL error message, we let the caller to handle the exception
throw new ValidationException(exc.getMessage(), exc);
}
finally
{
if (commit)
{
buffer.recordFlushed();
// release the savepoint or commit/roll-back the micro-transaction
if (sp != null)
{
session.releaseSavepoint(sp);
}
else if (fullTx)
{
session.commit();
}
}
}
}
/**
* Get the state of the {@link #flushed} flag.
*
* @return See above.
*/
public boolean wasFlushed()
{
return flushed;
}
/**
* This method checks whether the record fits the maximum allowable index size for 4GL and the
* current configured dialect. If there is at least one index that will pass this limit when
* the change is applied a {@code ValidationException} is thrown. Otherwise the method returns
* peacefully.
*/
private void checkMaxIndexSize()
throws ValidationException
{
Dialect dialect = buffer.getDialect();
int dialectMaxIndexSize = dialect.getIndexLengthLimit();
if (P4GL_MAX_INDEX_SIZE == 0 && dialectMaxIndexSize == 0)
{
// quick-out for index-key size validation is disabled
return;
}
RecordMeta recordMeta = dmo._recordMeta();
PropertyMeta[] properties = recordMeta.getPropertyMeta(false);
BitSet[] uIndexes = recordMeta.getUniqueIndices();
int[] uiSizes = new int[uIndexes.length];
int[] duiSizes = new int[uIndexes.length];
BitSet[] nIndexes = recordMeta.getNonuniqueIndices();
int[] niSizes = new int[nIndexes.length];
int[] dniSizes = new int[nIndexes.length];
BitSet idxProps = recordMeta.getAllIndexedProperties();
// walk changed bitset and update the uiSizes and niSizes with the space used
for (int i = idxProps.nextSetBit(0); i >= 0; i = idxProps.nextSetBit(i + 1))
{
// compute the size for an index property
PropertyMeta crtProp = properties[i];
int size = crtProp.getDataHandler().getFieldSizeInIndex(dmo.data[i]);
// search the unique indexes where it is used and add the size there. If an index is
// breaking one of the constraints, stop with error and skip additional computations
for (int uComp = 0, uIndexesLength = uIndexes.length; uComp < uIndexesLength; uComp++)
{
if (uIndexes[uComp].get(i))
{
uiSizes[uComp] += size;
duiSizes[uComp] += dialect.computeIndexKeySize(crtProp.getType(), dmo.data[i]);
checkIndexSize(uiSizes[uComp], duiSizes[uComp], dialectMaxIndexSize, uComp, true);
}
}
// do the same for non-unique indexes
for (int nComp = 0, nIndexesLength = nIndexes.length; nComp < nIndexesLength; nComp++)
{
if (nIndexes[nComp].get(i))
{
niSizes[nComp] += size;
dniSizes[nComp] += dialect.computeIndexKeySize(crtProp.getType(), dmo.data[i]);
checkIndexSize(niSizes[nComp], dniSizes[nComp], dialectMaxIndexSize, nComp, false);
}
}
}
// if we reach this point then we are good to go, no index constraints were broken
}
/**
* Validates an intermediary index size against the maximum P4GL and dialect specific
* constraints.
*
* @param p4glSize
* The current calculated size of the space occupied by {@code dmo} in P4GL. May be
* incomplete.
* @param sqlSize
* The current calculated size of the space occupied by {@code dmo} in SQL. May be
* incomplete.
* @param dialectMaxSize
* The maximum supported index size by the dialect used in FWD.
* @param indexPos
* The position of the index.
* @param unique
* {@code true} only if the current index is unique.
*
* @throws ValidationException
* When the {@code p4glSize} already overflows the index size constraint.
*/
private void checkIndexSize(int p4glSize,
int sqlSize,
int dialectMaxSize,
int indexPos,
boolean unique)
throws ValidationException
{
int x = (P4GL_MAX_INDEX_SIZE != 0 && p4glSize > P4GL_MAX_INDEX_SIZE) ? 1 :
(dialectMaxSize != 0 && sqlSize > dialectMaxSize) ? 2 : 0;
if (x == 0)
{
return;
}
String legacyTableName = (dmo instanceof TempRecord ? "(TEMP-TABLE) " : "") + bufferName;
String message;
if (x == 1)
{
// Note that, when displayed in 4GL, the (129) represents the 'old' index limitation of
// OpenEdge <= v10.1B, not an error code. They increased the limitation in newer
// versions, but the error ocode remained the same.
message = "The total length of the fields in an index exceeds max key size. " +
"Index " + dmo._recordMeta().getIndexLegacyName(indexPos, unique) +
" of table " + legacyTableName + " (129)";
}
else
{
// this record is stored successfully on P4GL but the current dialect cannot handle
// it so a SQL error will be emitted if we continue with the flush to database
message = "The total length of the fields in an index exceeds " +
"maximum key size supported by this dialect. " +
"Index " + dmo._recordMeta().getIndexLegacyName(indexPos, unique) +
" of table " + legacyTableName;
}
ErrorManager.displayError(11353, message, false);
// the message is usually displayed multiple times with temp-tables
throw new StopConditionException(message, 11353);
}
/**
* Check whether properties are both mandatory and are currently {@code null}. If so,
* throw an exception.
*
* @param offsets
* Offsets of the properties to be checked in the DMO.
*
* @throws ValidationException
* if the property is mandatory and is currently {@code null}.
*/
private void checkNotNull(BitSet offsets)
throws ValidationException
{
BitSet nonNullable = dmo._recordMeta().nonNullableProps;
for (int i = offsets.nextSetBit(0); i != -1; i = offsets.nextSetBit(i + 1))
{
if (nonNullable.get(i) && dmo.data[i] == null)
{
failNotNull(i); // err: 110
}
}
}
/**
* Check whether any property is both mandatory and is currently {@code null}. If so, throw an exception.
* This method should be invoked only for transient records which have the initial/default value set to
* unknown (?). Normally the field assignment is tested for null/unknown values at the assign moment.
*
* @return Negative value if the check succeeded, else the non-negative offset value of the first field
* in the DMO's data array to fail the non-null test.
*/
private int checkNotNull(boolean onlyCheckDirty)
throws ValidationException
{
BitSet nonNullable = dmo._recordMeta().nonNullableProps;
BitSet currentlyNull = dmo.getNullProps();
BitSet check = currentlyNull;
if (onlyCheckDirty)
{
check = (BitSet) check.clone();
check.and(dmo.getDirtyProps());
}
if (nonNullable.intersects(check))
{
BitSet copy = (BitSet) nonNullable.clone();
copy.and(check);
int offset = copy.nextSetBit(0);
return offset;
}
return -1;
}
/**
* The property at the given offset is mandatory but is currently {@code null}. Throw an
* exception to signal the error.
*
* @param offset
* Offset of the property in the DMO.
*
* @throws ValidationException
* always, to indicate the mandatory property error.
*/
private void failNotNull(int offset)
throws ValidationException
{
DmoMeta dmoInfo = DmoMetadataManager.getDmoInfo(dmo.getClass());
Property fi = dmoInfo.getProperty(offset);
String msg = bufferName + "." + fi.legacy + " is mandatory, but has unknown (?) value";
if (updateState)
{
dmo.updateState(buffer.getSession(false), INVALID, true);
}
throw new ValidationException(msg, 110);
}
/**
* Validate the DMO with respect to the unique constraints, if any, of its table. This occurs
* in two steps. The first checks whether any uncommitted transactions across sessions contain
* any record with which this record would conflict, in terms of a unique constraint. This
* step is a no-op for temp-tables, which are scoped to a single user session. The second step
* attempts to insert (for a newly created record) or update (for a previously persisted
* record) the record in its database. If this operation is meant to validate only, this
* insert/update is rolled back. If either operation fails, an exception is thrown.
* <p>
* If this method returns normally, the record is considered valid from the standpoint of
* unique constraints.
*
* @return {@code true} if the record was saved (or updated) and this was not a validation-only
* operation.
*
* @throws PersistenceException
* if there was a database error during validation or data persistence. This
* generally will not be recoverable.
* @throws ValidationException
* if there was a validation error. This represents an application level data
* validation failure and is recoverable.
*/
/*
private boolean checkUniqueConstraints()
throws PersistenceException,
ValidationException
{
boolean queryCheckMethod = false;
int dirtyOffset = dmo.getActiveOffset();
BitSet checkUniqIndices = null;
RecordMeta recordMeta = dmo._recordMeta();
// if (!force)
// {
// determine whether unique constraint check and flush is warranted based on current state
if (dmo.checkState(NEW))
{
// for a transient (newly created) record, at least one index needs to be fully dirty
// (even if not changed) to trigger validation and flush OR the table must have no indices
if (dmo.isIndexed() && !dmo.isAnyIndexFullyDirty())
{
// validation was not performed; do not update any record/property state
return false;
}
// if we are in a single property validation for a transient record, we need to check
// whether ANY mandatory property is null; if so, this error may mask any unique constraint
// violation when we try to persist the DMO (depending on the database implementation);
// also use this method if 2 or fewer unique indices are fully dirty in connection with
// the most recent change
checkUniqIndices = dmo.getFullyDirtyUniqueIndices(dirtyOffset);
queryCheckMethod = singleProp &&
hasUniqueConstraint &&
rollback &&
(checkNotNull() >= 0 || checkUniqIndices.cardinality() <= 2);
}
else
{
// for an existing, persisted record, any update that touches an index component
// triggers validation and possibly flush OR the table must have no indices
if (dmo.isIndexed() && !dmo.getDirtyProps().intersects(recordMeta.allIndexedProps))
{
// validation was not performed; do not update any record/property state
return false;
}
// checkUniqIndices = dmo.getDirtyUniqueIndices(dirtyOffset);
// queryCheckMethod = singleProp &&
// hasUniqueConstraint &&
// rollback &&
// checkUniqIndices.cardinality() <= 2;
}
// }
// unique constraint checking takes place in two phases:
// 1. we check whether any uncommitted transaction has made a matching update to a unique
// index;
// 2. we attempt to insert/update the DMO into the database, which will fail if there is a
// unique constraint violation (for validation-only, we roll back the change).
UniqueTracker.Token token = null;
boolean saved = false;
try
{
if (hasUniqueConstraint)
{
// validate against uncommitted transaction data; no-op for temp-table
token = uniqueTrackerCtx.lockAndChange(uniqueTracker, dmo, bufferName);
}
if (queryCheckMethod)
{
// use the query method of checking unique constraints instead of the persist & rollback
// method; this will not get hung up on a null mandatory field in a transient DMO
validateUniqueByQuery(checkUniqIndices);
}
else if (hasUniqueConstraint || !rollback)
{
// notify the FastFindCache about the indices which are about to be invalidated. In the event of
// an error persisting the record ([saved] set to false below), we will have invalidated the cache
// unnecessarily, but it's better to lose the optimization than to get incorrect results in a
// parallel thread if it was accessing the cache between save and invalidate operations
FastFindCache ffc = FastFindCache.getInstance(recordMeta.isTemporary());
if (dmo.checkState(NEW))
{
// invalidate all indexes on insert
ffc.invalidate(recordMeta.getDmoMeta().getId(), multiplex);
}
else
{
// invalidate only affected indexes on update
BitSet dirtyUnique = dmo.getDirtyIndices(recordMeta.getUniqueIndices(), dirtyOffset);
BitSet dirtyNonunique = dmo.getDirtyIndices(recordMeta.getNonuniqueIndices(), dirtyOffset);
ffc.invalidate(recordMeta.getDmoMeta().getId(), multiplex, dirtyUnique, dirtyNonunique);
}
// validate against data already committed in the database; this has to be done for the validate
// and persist case, even if we have no unique constraint
saved = validateUniqueCommitted();
}
// validation was performed and succeeded; changes may have been persisted to database;
// the new/dirty/changed state of the record will have have been updated by the save
// mark record as having passed validation (whether or not data was persisted)
dmo.updateState(INVALID, false);
// log the validation event for possible rollback
dmo.logValidation(session);
}
catch (ValidationException exc)
{
if (token != null)
{
// token will be non-null when the UniqueTracker update was successful, but the
// database update was not; roll back tracker changes; no-op for temp-table
uniqueTrackerCtx.rollbackChange(uniqueTracker, token);
}
// mark record as having failed validation; it will stay this way until it passes
dmo.updateState(INVALID, true);
throw exc;
}
finally
{
// unlock tracker's resources locked above, no matter how we exit; no-op for temp-tables
// and for no-unique-constraint case
uniqueTrackerCtx.unlock(uniqueTracker, token);
// if full validation was done and we persisted the record, clear NEEDS_VALIDATION flag
if (!singleProp && saved)
{
dmo.updateState(NEEDS_VALIDATION, false);
}
}
return saved;
}
*/
/**
* Iterate the specified unique indices and decide whether a record already exists with the unique
* combination of index components in the DMO being validated. In this case, emit a specific validation
* error. This method makes sure the proper index is identified to match the 4GL error message. For this,
* the order in which the unique indexes are tested is essential.
*
* @param check
* A bit set whose set bits represent the positions of the unique indices in the DMO's array
* of unique indices to be checked. If {@code null}, check all the unique indices.
*
* @throws ValidationException
* when the method detects a collision of current DMO with another record stored in the database.
* @throws PersistenceException
* if an error occurred while accessing the database.
*/
private void validateUniqueByQuery(BitSet check)
throws PersistenceException,
ValidationException
{
if (check != null && check.isEmpty())
{
return;
}
Session session = buffer.getSession(true);
RecordMeta recordMeta = dmo._recordMeta();
PropertyMeta[] props = recordMeta.getPropertyMeta(false);
BitSet[] uniqueIndices = recordMeta.uniqueIndices;
String[] uniqueSQLs = recordMeta.getUniqueIndicesValidationSQLs(
Dialect.getDialect(session.getDatabase()));
int len = uniqueIndices.length;
if (check == null)
{
// check all unique indices
check = new BitSet();
check.set(0, len);
}
String query;
if (check.cardinality() > 1)
{
// check multiple indices using UNION ALL
StringBuilder buf = new StringBuilder();
for (int k = check.nextSetBit(0), count = 0; k >= 0; k = check.nextSetBit(k + 1), count++)
{
if (count > 0)
{
buf.append(" union all ");
}
buf.append(uniqueSQLs[k]);
}
query = buf.toString();
}
else
{
query = uniqueSQLs[check.nextSetBit(0)];
}
try (PreparedStatement ps = session.getConnection().prepareStatement(query))
{
int psOffset = 0;
for (int k = check.nextSetBit(0); k >= 0; k = check.nextSetBit(k + 1))
{
psOffset = setUniqueQueryParameters(ps, uniqueIndices[k], recordMeta, props, psOffset);
}
QueryStatement f = PreparedStatement::executeQuery;
ResultSet rs = SQLExecutor.getInstance().execute(session.getDatabase(), query, f, ps);
while (rs.next())
{
// violated indices are detected in P4GL legacy order; for each one, confirm there is no
// modified DMO in memory which cures the violation
int i = rs.getInt(1);
BitSet uIndex = uniqueIndices[i];
if (confirmUniqueConstraintViolation(session, i, uIndex, uniqueSQLs[i], recordMeta, props))
{
throw new ValidationException(getFailUniqueIndexMessage(uIndex, bufferName, dmo), 132);
}
// if we exit the loop normally, all violations are cured by in-memory changes (which may yet
// have to be validated and flushed themselves)
}
}
catch (SQLException sqlEx)
{
throw new PersistenceException("Failed to test uniqueness", sqlEx);
}
}
/**
* Set the substitution parameters for a prepared statement used to check uniqueness against the records
* in a database.
*
* @param ps
* Prepared statement.
* @param uIndex
* A bit set describing which data elements participate in the unique index being checked by the
* query.
* @param recordMeta
* Metadata for the DMOs associated with the table being queried.
* @param props
* Array of DMO metadata objects, each describing a property of the DMO interface associated with
* the table being queried.
* @param psOffset
* Zero-based offset of the first parameter to be substituted in the prepared statement by this
* method (this method can be called multiple times for the same prepared statement, if multiple
* individual statements are UNION'd together for a unique constraint check).
*
* @return The one-based offset of the last parameter substituted by the current invocation.
*
* @throws SQLException
* if there is a SQL error substituting parameters.
*/
private int setUniqueQueryParameters(PreparedStatement ps,
BitSet uIndex,
RecordMeta recordMeta,
PropertyMeta[] props,
int psOffset)
throws SQLException
{
TypeManager.getTypeHandler(Long.class).setParameter(ps, ++psOffset, dmo.primaryKey(), false);
if (recordMeta.isTemporary())
{
TypeManager.getTypeHandler(Integer.class)
.setParameter(ps, ++psOffset, ((TempRecord) dmo)._multiplex(), false);
}
for (int i = uIndex.nextSetBit(0); i >= 0; i = uIndex.nextSetBit(i + 1))
{
Object datum = dmo.data[i];
if (datum != null)
{
if (props[i].getType() == character.class)
{
datum = TextOps.trim((String) datum, " ", false, true);
if (!props[i].isCaseSensitive())
{
datum = ((String) datum).toUpperCase();
}
}
else if (props[i].getType() == decimal.class)
{
// must round [datum] to field's decimals
datum = ((BigDecimal) datum).setScale(props[i].getDecimals(), RoundingMode.HALF_UP);
}
}
props[i].getDataHandler().setParameter(ps, ++psOffset, datum, false);
}
return psOffset;
}
/**
* Function that checks if VALIDATE keyword can be used with this statement.
*
* @return boolean
* {@code true} if there exists support for the VALIDATE keyword.
*/
private boolean canSkipServerSideUniqueCheck()
{
return buffer != null && buffer.getPersistence().allowDBUniqueCheck();
}
/**
* Verify a unique constraint violation detected by querying the database, by confirming that the record
* found to be causing the constraint violation does not have unflushed changes associated with it in
* memory, awaiting its own validation and flush operation. If a modified, unflushed copy of the suspect
* record is found in the current session cache, its data elements which participate in the violated
* constraint are compared with those causing the violation. If any element compares differently, the
* constraint violation is rejected.
*
* @param session
* Current database session.
* @param pos
* The position of the unique index being tested in the array of the target DMO's unique indices.
* @param index
* A bit set describing which data elements participate in the unique index being tested.
* @param sql
* The original SQL statement associated with the unique constraint check for the unique index
* being tested. It is modified to return the primary key of the violating record.
* @param recordMeta
* Metadata for the DMOs associated with the table being queried.
* @param props
* Array of DMO metadata objects, each describing a property of the DMO interface associated with
* the table being queried.
*
* @return {@code true} if the unique constraint violation was confirmed; {@code false} if it was
* rejected based on unflushed, in-memory changes.
*
* @throws PersistenceException
* if there is an error getting a JDBC connection or accessing the session cache.
* @throws SQLException
* if there is an error preparing or executing the query.
*/
private boolean confirmUniqueConstraintViolation(Session session,
int pos,
BitSet index,
String sql,
RecordMeta recordMeta,
PropertyMeta[] props)
throws PersistenceException,
SQLException
{
// modify SELECT statement to select primary key instead of the position of the violated index
int len = ("select " + pos).length();
String sqlPK = "select " + Session.PK + sql.substring(len);
try (PreparedStatement ps = session.getConnection().prepareStatement(sqlPK))
{
setUniqueQueryParameters(ps, index, recordMeta, props, 0);
ResultSet rs = ps.executeQuery();
if (!rs.next())
{
// if there is no violation at this point, another session may have cleared it asynchronously
// (should not happen for a temp-table)
return false;
}
long id = rs.getLong(1);
BaseRecord cached = session.getCached(dmo.getClass(), id);
if (cached != null && cached.checkState(DmoState.CHANGED))
{
// if any data element in the changed record does not match the corresponding unique check datum,
// the violation is cleared; note that the changed record may still fail its own validation later
for (int i = index.nextSetBit(0); i >= 0; i = index.nextSetBit(i + 1))
{
Object uniqueDatum = dmo.data[i];
Object cachedDatum = cached.data[i];
if (!Objects.equals(uniqueDatum, cachedDatum))
{
return false;
}
}
}
}
// no unflushed record clears the violation, so it is confirmed
return true;
}
/**
* Use the database to validate and optionally persist all current changes in the record.
* This method sets a savepoint and attempts to insert/update the record. If it succeeds,
* the data is considered valid from a unique constraint perspective.
* <p>
* If this object was constructed to not persist the data, the current transaction is rolled
* back to this savepoint. Otherwise, the transaction is allowed to continue.
* <p>
* It is possible that this method is called when there is no active transaction within which
* to set a savepoint. This can occur when inserting or updating a temp table record outside
* an application transaction. In this case, we create a database transaction instead of a
* savepoint (unless this operation is validation only). The rollback semantics are the same
* whether using a savepoint or a transaction.
*
* @return {@code true} if the record was saved (or updated) and we are not in a validation-only
* check.
*
* @throws PersistenceException
* if there was a database error during validation or data persistence. This
* generally will not be recoverable.
* @throws ValidationException
* if there was a validation error. This represents an application level data
* validation failure and is recoverable.
*/
/*
private boolean validateUniqueCommitted()
throws PersistenceException,
ValidationException
{
// if there is at least one unique index to validate against, create a savepoint, so we can roll
// back the save on failure
Savepoint sp = session.setSavepoint();
boolean fullTx = false;
if (sp == null)
{
// we are not currently within a transaction, so create a short-lived transaction to persist
// the change
fullTx = session.beginTransaction(null);
}
try
{
// this will perform the appropriate insert or update, based on the state of the DMO
session.save(dmo, !rollback);
}
catch (PersistenceException exc)
{
// validation failed, or we had a different error; roll back to savepoint
// TODO: is this always safe (or even possible) in the event of a non-unique constraint
// related database error?
if (sp != null)
{
session.rollbackSavepoint(sp);
}
else if (fullTx)
{
session.rollback();
}
// go up the exception chain and look for a SQL exception:
Throwable cause = exc.getCause();
while (cause != null &&
!(cause instanceof SQLException) &&
cause != cause.getCause())
{
cause = cause.getCause();
}
if (cause instanceof SQLException)
{
SQLException sqle = (SQLException) cause;
if ("23505".equals(sqle.getSQLState())) // UNIQUE VIOLATION - unique_violation
{
// if the right index is found, this method will not return normally, instead it will
// throw a specific exception
validateUniqueByQuery(null);
}
}
// if the exception could not be identified as a SQL UNIQUE VIOLATION exception or the
// generateUniqueIndexesCondition() method could not identify the right index to compose the
// P4GL error message, we let the caller to handle the exception
throw new ValidationException(exc.getMessage(), exc);
}
// if only validating and not persisting, roll back; else, commit
if (rollback)
{
if (sp != null)
{
session.rollbackSavepoint(sp);
}
else if (fullTx)
{
session.rollback();
}
}
else
{
if (sp != null)
{
session.releaseSavepoint(sp);
}
else if (fullTx)
{
session.commit();
}
}
return !rollback;
}
*/
}