JsonExport.java
/*
** Module : JsonExport.java
** Abstract : Write temp-table and dataset data into a JSON stream.
**
** Copyright (c) 2019-2025, Golden Code Development Corporation.
**
** -#- -I- --Date-- ---------------------------------------Description---------------------------------------
** 001 ECF 20190107 Created initial version.
** 002 OM 20190327 Renamed DataSource to avoid conflicts with DataSet source.
** 003 OM 20190626 Added serialization support for DATASETs.
** HC 20190707 Implemented read(dataSet) and read(tempTable) methods of JsonObject legacy class.
** 004 CA 20190720 Default encoding is UTF-8, if not specified.
** CA 20190728 If the value is unknown, write a null.
** CA 20190811 Fixed problems with DATASET export.
** 005 ECF 20200906 New ORM implementation.
** 006 CA 20200914 Added blob, raw, rowid and handle support.
** 007 CA 20200927 Use IdentityHashMap instead of plain map when the key is a Class.
** 008 OM 20201120 Improved compatibility with P4GL.
** ME 20210130 Added serializeTempTable method into JsonArray.
** 20210319 Refactor to reuse code for JsonArray/JsonObject Read methods.
** ME 20210324 Add method to serialize 4GL error as Json Object.
** CA 20210510 Fixed JSON and XML serialization when XML-NODE-NAME/SERIALIZE-NAME field options are set.
** CA 20210511 Fixed a XmlTempTableSchema usage.
** OM 20210809 Completed implementation of SERIALIZE-ROW|WRITE-JSON}(JsonObject|JsonArray).
** CA 20220120 Do not use TypeFactory.object for internal usages, use ObjectVar if the reference must be
** tracked.
** OM 20220212 Use ReservedProperty static method for testing reserved properties names.
** CA 20220923 Variable definitions (including associated with parameters) must be done always outside of
** the BlockManager API
** CA 20221006 Do not access tableHandle() in the FWD runtime, get the TempTable instance directly.
** Refs #6826
** CA 20221017 Honor DATASET:SERIALIZE-HIDDEN at JsonObject serialization.
** HC 20230116 Replaced some handle usages with the actual wrapped resources for
** performance.
** CA 20230116 Avoid using handle.unwrap, handle.getReference or other BDT usage from within FWD runtime.
** 009 IAS 20230303 Fixed <code>decimal</code> serialization
** 20230315 Added support for the DATA-REALATION:FOREIGN-KEY-HIDDEN attribute.
** 010 IAS 20230415 Fixed support for PARENT-ID-RELATION.
** 011 GBB 20230512 Logging methods replaced by CentralLogger/ConversionStatus.
** 012 CA 20231026 Do not use ObjectVar instances within FWD runtime.
** 013 CA 20240227 Fixed SERIALIZE-ROW to not include a root buffer name key when exporting to JsonObject.
** 014 CA 20240320 Performance improvement for WRITE-JSON:
** Avoid BDTs within internal FWD runtime.
** Replaced iterators with Java 'for', where it applies.
** Read the temp-table fields directly from the Record.data.
** Directly set the record in the parent buffer instead of using a findUnique
** 015 TJD 20240123 Java 17 compatibility updates
** 016 SP 20240627 Error 4065 must not set ERROR-STATUS:ERROR flag.
** 017 AP 20240910 Improved support for WRITE-JSON to JsonObject by using the same serialization
** implementation for all output types.
** 018 RNC 20241101 Added stripping of BigDecimal's trailing zeros in writeDatum().
** 019 ES 20250327 Use seralizationName of the buffer, instead of temp-table. In case of TEMP-TABLE:writeJson
** use seralizationName of the defaultBuffer.
*/
/*
** This program is free software: you can redistribute it and/or modify
** it under the terms of the GNU Affero General Public License as
** published by the Free Software Foundation, either version 3 of the
** License, or (at your option) any later version.
**
** This program is distributed in the hope that it will be useful,
** but WITHOUT ANY WARRANTY; without even the implied warranty of
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
** GNU Affero General Public License for more details.
**
** You may find a copy of the GNU Affero GPL version 3 at the following
** location: https://www.gnu.org/licenses/agpl-3.0.en.html
**
** Additional terms under GNU Affero GPL version 3 section 7:
**
** Under Section 7 of the GNU Affero GPL version 3, the following additional
** terms apply to the works covered under the License. These additional terms
** are non-permissive additional terms allowed under Section 7 of the GNU
** Affero GPL version 3 and may not be removed by you.
**
** 0. Attribution Requirement.
**
** You must preserve all legal notices or author attributions in the covered
** work or Appropriate Legal Notices displayed by works containing the covered
** work. You may not remove from the covered work any author or developer
** credit already included within the covered work.
**
** 1. No License To Use Trademarks.
**
** This license does not grant any license or rights to use the trademarks
** Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
** of Golden Code Development Corporation. You are not authorized to use the
** name Golden Code, FWD, or the names of any author or contributor, for
** publicity purposes without written authorization.
**
** 2. No Misrepresentation of Affiliation.
**
** You may not represent yourself as Golden Code Development Corporation or FWD.
**
** You may not represent yourself for publicity purposes as associated with
** Golden Code Development Corporation, FWD, or any author or contributor to
** the covered work, without written authorization.
**
** 3. No Misrepresentation of Source or Origin.
**
** You may not represent the covered work as solely your work. All modified
** versions of the covered work must be marked in a reasonable way to make it
** clear that the modified work is not originating from Golden Code Development
** Corporation or FWD. All modified versions must contain the notices of
** attribution required in this license.
*/
package com.goldencode.p2j.persist.serial;
import java.io.*;
import java.lang.reflect.*;
import java.math.*;
import java.sql.Timestamp;
import java.time.OffsetDateTime;
import java.util.*;
import com.fasterxml.jackson.core.*;
import com.goldencode.p2j.oo.core.Assert;
import com.goldencode.p2j.oo.json.objectmodel.*;
import com.goldencode.p2j.oo.lang.*;
import com.goldencode.p2j.persist.*;
import com.goldencode.p2j.persist.Record;
import com.goldencode.p2j.persist.orm.*;
import com.goldencode.p2j.util.*;
import com.goldencode.p2j.util.ErrorManager;
import com.goldencode.p2j.util.logging.*;
/**
* An implementation of the 4GL WRITE-JSON method, which writes the records of a temp-table into
* a JSON stream.
* <p>
* Runtime support is partial at this time; not all modes and features are supported.
*/
public final class JsonExport
{
/** Logger */
private static final CentralLogger LOG = CentralLogger.get(JsonExport.class.getName());
/** Map of data types to functions which write JSON values */
private final Map<Class<? extends BaseDataType>, ThrowingConsumer> fnMap = new IdentityHashMap<>();
/**
* Child fields for NESTED relations with FOREIGN-KEY-HIDDEN = true per child table name.
* These fields are skipped on serialization.
*
*/
private Map<String, Set<String>> childFieldsByChild = new HashMap<>();
/** Child fields for the current NESTED relations with FOREIGN-KEY-HIDDEN = true*/
private Set<String> childFields = Collections.emptySet();
/** Cache the {@link TempTableSchema} structures. */
private Map<TemporaryBuffer, TempTableSchema> tableSchemas = new IdentityHashMap<>();
/** Encoding for JSON string */
public static JsonEncoding parseEncoding(character encoding)
{
if (encoding == null || "UTF-8".equalsIgnoreCase(encoding.toStringMessage()))
{
return JsonEncoding.UTF8;
}
JsonEncoding jenc = (encoding.isUnknown() || "".equals(encoding.toStringMessage()))
? JsonEncoding.UTF8
: null;
if (jenc == null)
{
try
{
jenc = JsonEncoding.valueOf(encoding.toStringMessage());
}
catch (IllegalArgumentException exc)
{
// TODO: proper error reporting
ErrorManager.noRecordOrThrowError(exc.getMessage());
}
}
return jenc;
}
/**
* Constructor specific to DATASET serialization. It will immediately performs the export.
*
* @param dataSet
* The target object to be serialized.
* @param target
* Data target.
* @param pretty
* {@code True} to format the JSON to make it more readable; {@code false} to write it
* as unformatted text.
* @param encoding
* The encoding to use.
* @param noInit
* {@code True} to omit data for fields whose values match their initial values.
* @param noOuter
* {@code True} to omit data for the outermost object from the output.
* @param beforeImage
* {@code True} to write before-image data and error information, else {@code false}.
*
* @return {@code true} on success and {@code false} otherwise.
*
* @throws PersistenceException
* On persistence issues.
*/
public boolean exportDataSet(DataSet dataSet,
TargetData target,
boolean pretty,
JsonEncoding encoding,
boolean noInit,
boolean noOuter,
boolean beforeImage)
throws PersistenceException
{
dataSet.getRelations().filter(r -> r.getForeignKeyHidden().booleanValue() ||
r.isParentIdRelation().booleanValue()).
forEach(r -> {
childFieldsByChild.put(r.getChildTable(),
new HashSet<>(Arrays.asList(r.getRelationFields(false))));
});
if (!target.configureSupportedTargets(TargetData.TD_WRITE_JSON,
TargetData.TD_WRITE_JSON,
LegacyResource.DATASET + " widget"))
{
return false;
}
try
{
if (target.isJson())
{
return writeDatasetToJson(dataSet, target, noInit, noOuter, beforeImage);
}
else
{
try (OutputStream out = target.getStream())
{
StreamJsonSerializer sjs = new StreamJsonSerializer(out, encoding, pretty);
exportDataSet(dataSet, noInit, noOuter, beforeImage, sjs, false);
sjs.closeStream();
}
}
return true;
}
catch (IOException exc)
{
throw new PersistenceException("Error exporting JSON data", exc);
}
catch (RuntimeException exc)
{
throw new PersistenceException(exc.getMessage(), exc);
}
}
/**
* Constructor specific to DATASET serialization. It will immediately performs the export.
*
* @param dataSet
* The target object to be serialized.
* @param noInit
* {@code True} to omit data for fields whose values match their initial values.
* @param noOuter
* {@code True} to omit data for the outermost object from the output.
* @param beforeImage
* {@code True} to write before-image data and error information, else {@code false}.
* @param json
* This is the target for the deserialized json data.
* @param skipStartAndEndObject
* Flag used to skip the creation and removing of the first JSON node as these will be performed
* somewhere else.
*
* @throws PersistenceException
* On persistence issues.
*/
public void exportDataSet(DataSet dataSet,
boolean noInit,
boolean noOuter,
boolean beforeImage,
JsonStructureCallback json,
boolean skipStartAndEndObject)
throws PersistenceException
{
try
{
if (!skipStartAndEndObject)
{
json.startObject();
}
serializeDataSet(json, dataSet, noInit, noOuter, beforeImage);
if (!skipStartAndEndObject)
{
json.endObject();
}
}
catch (IOException exc)
{
throw new PersistenceException("Error exporting JSON data", exc);
}
catch (RuntimeException exc)
{
throw new PersistenceException(exc.getMessage(), exc);
}
}
/**
* Serializes TEMP-TABLE to json.
*
* @param buffer
* Temp-table buffer in which data is stored.
* @param target
* Data target.
* @param formatted
* {@code True} to format the JSON to make it more readable; {@code false} to write it
* as unformatted text. Unknown value is treated as {@code false}.
* @param encoding
* The JSON encoding to use.
* @param omitInitialValues
* {@code True} to omit data for fields whose values match their initial values.
* @param omitOuterObject
* {@code True} to omit data for the outermost object from the output.
*
* @return {@code true} on success.
*
* @throws PersistenceException
* On persistence issues.
*/
public boolean exportTempBuffer(TemporaryBuffer buffer,
TargetData target,
logical formatted,
JsonEncoding encoding,
logical omitInitialValues,
logical omitOuterObject)
throws PersistenceException
{
if (!target.configureSupportedTargets(TargetData.TD_WRITE_JSON,
TargetData.TD_WRITE_JSON,
LegacyResource.TEMP_TABLE + " widget"))
{
return false;
}
if (target.isJson())
{
return writeTableToJson(buffer, target);
}
try (OutputStream out = target.getStream())
{
boolean pretty = formatted != null && !formatted.isUnknown() && formatted.booleanValue();
StreamJsonSerializer sjs = new StreamJsonSerializer(out, encoding, pretty);
exportTempBuffer(buffer, omitInitialValues, omitOuterObject, sjs);
sjs.closeStream();
return true;
}
catch (IOException exc)
{
throw new PersistenceException("Error exporting JSON data", exc);
}
catch (RuntimeException exc)
{
throw new PersistenceException(exc.getMessage(), exc);
}
}
/**
* Serializes TEMP-TABLE to json.
*
* @param buffer
* Temp-table buffer in which data is stored.
* @param omitInitialValues
* {@code True} to omit data for fields whose values match their initial values.
* @param omitOuterObject
* {@code True} to omit data for the outermost object from the output.
* @param json
* This is the target for the deserialized json data.
*
* @throws PersistenceException
* On persistence issues.
*/
public void exportTempBuffer(TemporaryBuffer buffer,
logical omitInitialValues,
logical omitOuterObject,
JsonStructureCallback json)
throws PersistenceException
{
boolean omitOuter = !omitOuterObject.isUnknown() && omitOuterObject.booleanValue();
boolean noInit = !omitInitialValues.isUnknown() && omitInitialValues.getValue();
boolean beforeFlags = ((BufferImpl) buffer.getDMOProxy()).isBeforeBuffer();
try
{
serializeTempTable(json, null, buffer, null, noInit, omitOuter, beforeFlags, false, false);
}
catch (IOException exc)
{
throw new PersistenceException("Error exporting JSON data", exc);
}
catch (RuntimeException exc)
{
throw new PersistenceException(exc.getMessage(), exc);
}
}
/**
* Exports a single record in JSON format.
*
* @param bufferImpl
* The buffer holding the record to be exported.
* @param targetData
* The output structure.
* @param formatted
* Use {@code true} to have a pretty output. Otherwise, the output is generated on a single line.
* @param jsonEncoding
* The encoding to be used.
* @param omitInitial
* If {@code true} the initial/default values are skipped.
* @param omitOuter
* Omit outer values.
*
* @return The buffer holding the record to be exported.
*/
public boolean exportRecord(BufferImpl bufferImpl,
TargetData targetData,
boolean formatted,
JsonEncoding jsonEncoding,
boolean omitInitial,
boolean omitOuter)
{
if (!targetData.configureSupportedTargets(TargetData.TD_SERIALIZE_ROW,
TargetData.TD_WRITE_JSON,
LegacyResource.BUFFER + " widget"))
{
return false;
}
TemporaryBuffer tempBuffer = (TemporaryBuffer) bufferImpl.buffer();
TempTableSchema schema = new TempTableSchema(tempBuffer, true);
// if (checkSupportedTypes(schema))
// {
// return false;
// }
if (targetData.isJson())
{
return writeRecordToJson(bufferImpl, targetData, schema);
}
try (OutputStream out = targetData.getStream())
{
StreamJsonSerializer json = new StreamJsonSerializer(out, jsonEncoding, formatted);
if (!omitOuter)
{
json.startObject();
json.fieldName(bufferImpl.doGetName());
}
writeRecord(json, tempBuffer, tempBuffer.getCurrentRecord(), schema, bufferImpl.doGetName(),
true, false, false, false, omitInitial);
if (!omitOuter)
{
json.endObject();
}
json.closeStream();
}
catch (IOException e)
{
return false;
}
return true;
}
/**
* Populate the JSON object target with data from the buffer.
*
* @param buffer
* The buffer containing data.
* @param target
* The object containing JSON target.
* @param schema
* The schema used for operation.
*
* @return {@code true} if the operation is successful.
*/
private boolean writeRecordToJson(BufferImpl buffer, TargetData target, TempTableSchema schema)
{
String opType = target.getType();
if (!Serializator.TYPE_JSON_OBJECT.equalsIgnoreCase(opType))
{
ErrorManager.recordOrShowError(15344, opType, "SERIALIZE-ROW");
// Invalid mode '<mode>' for <method>. (15344)
return false;
}
Object ufo = target.getDestination();
if (!(ufo instanceof ObjectVar) || !(((ObjectVar<?>) ufo).ref() instanceof JsonObject))
{
ErrorManager.recordOrShowError(19079, "JsonObject", "SERIALIZE-ROW");
// <argument> argument for <method-name> is not valid. (19079)
ErrorManager.recordOrShowError(4065, false, "SERIALIZE-ROW", "BUFFER widget");
// **The <attribute> attribute on the <widget id> has invalid arguments. (4065)
return false;
}
// good to go
object<? extends JsonObject> root = asJsonObject((TempRecord) buffer.buffer().getCurrentRecord(),
schema);
if (root == null)
{
return false;
}
JsonObject json = (JsonObject) (((ObjectVar<?>) ufo).ref());
clearObject(json);
json.copy(root.ref());
return true; // all good!
}
/**
* Resets a {@code JsonConstruct} object by dropping all its properties or elements.
* <p>
* TODO: optimize this by providing a method to clear the [properties]/[elements] member of the [json]
*
* @param json
* The object to be reset.
*/
private void clearObject(JsonConstruct json)
{
if (json instanceof JsonObject)
{
JsonObject jo = (JsonObject) json;
for (Map.Entry<String, Object> property : jo.getProperties())
{
jo.remove(new character(property.getKey()));
}
}
else if (json instanceof JsonArray)
{
((JsonArray) json).setLength(new int64(0));
}
else
{
throw new RuntimeException("Is there some other kind of JsonConstruct?");
}
}
/**
* Write the content of the whole table to the destination JSON object found in target.
*
* @param buffer
* The buffer whose table is to be saved to JSON object.
* @param target
* The target which contains the destination object.
*
* @return {@code true} on success.
*/
private boolean writeTableToJson(TemporaryBuffer buffer, TargetData target)
{
String opType = target.getType();
boolean isJOTarget = Serializator.TYPE_JSON_OBJECT.equalsIgnoreCase(opType);
boolean isJATarget = Serializator.TYPE_JSON_ARRAY.equalsIgnoreCase(opType);
if (!isJOTarget && !isJATarget)
{
ErrorManager.recordOrShowError(15344, opType, "SERIALIZE-ROW");
// Invalid mode '<mode>' for <method>. (15344)
return false;
}
Object ufo = target.getDestination();
TempTableSchema schema = new TempTableSchema(buffer, true);
if (isJOTarget)
{
if (!(ufo instanceof ObjectVar) || !(((ObjectVar<?>) ufo).ref() instanceof JsonObject))
{
ErrorManager.recordOrShowError(19079, "JsonObject", "WRITE-JSON");
// <argument> argument for <method-name> is not valid. (19079)
ErrorManager.recordOrShowError(4065, false, "WRITE-JSON", "BUFFER widget");
// **The <attribute> attribute on the <widget id> has invalid arguments. (4065)
return false;
}
if (!((BufferImpl) buffer.getDMOProxy()).bufferRelease().booleanValue())
{
return false;
}
// good to go
final object<? extends JsonArray> root = new object<>(JsonArray.class);
root.assign(ObjectOps.newInstance(JsonArray.class));
buffer.readAllRows((dmo) -> {
object<? extends JsonObject> node = asJsonObject(dmo, schema);
if (node == null)
{
return;
}
root.ref().add_2(node);
});
JsonObject json = (JsonObject) (((ObjectVar<?>) ufo).ref());
clearObject(json);
json.add(((BufferImpl) buffer.getDMOProxy()).getSerializeName(), root);
}
else //if (isJATarget)
{
if (!(ufo instanceof ObjectVar) || !(((ObjectVar<?>) ufo).ref() instanceof JsonArray))
{
ErrorManager.recordOrShowError(19079, "JsonArray", "WRITE-JSON");
// <argument> argument for <method-name> is not valid. (19079)
ErrorManager.recordOrShowError(4065, false, "WRITE-JSON", "BUFFER widget");
// **The <attribute> attribute on the <widget id> has invalid arguments. (4065)
return false;
}
if (!((BufferImpl) buffer.getDMOProxy()).bufferRelease().booleanValue())
{
return false;
}
JsonArray json = (JsonArray) (((ObjectVar<?>) ufo).ref());
clearObject(json);
// good to go
buffer.readAllRows((dmo) -> {
object<? extends JsonObject> node = asJsonObject(dmo, schema);
if (node == null)
{
return;
}
json.add_2(node);
});
}
return true;
}
/**
* Creates a new JSON Object starting from the information which can be found in the buffer sent as
* parameter. Each field is converted to a property in the JSON Object.
*
* @param dmo
* The record which contains the data to be used for creating the object to be returned.
* @param schema
* Metadata information about the structure of the record.
*
* @return The JSON object constructed as described above. Or {@code null} on error.
*/
private object<? extends JsonObject> asJsonObject(TempRecord dmo, TempTableSchema schema)
{
final object<? extends JsonObject> root = new object<>(JsonObject.class);
root.assign(ObjectOps.newInstance(JsonObject.class));
ArrayList<TempTableSchema.Column> columns = schema.columns();
for (int k = 0; k < columns.size(); k++)
{
TempTableSchema.Column column = columns.get(k);
String name = column.getSerializeName();
if (ReservedProperty.isReservedProperty(name))
{
// some fields which start with "__" are hidden fields (specific to BEFORE tables)
continue;
}
Integer extent = column.getExtent();
if (extent != null && extent > 0)
{
final object<? extends JsonArray> jsonExtent = new object<>(JsonArray.class);
root.assign(ObjectOps.newInstance(JsonArray.class));
for (int i = 0; i < extent; i++)
{
BaseDataType datum = null;
try
{
datum = (BaseDataType) column.getGetter().invoke(dmo, i);
}
catch (IllegalAccessException | InvocationTargetException e)
{
ErrorManager.recordOrThrowError(-3, "", null, null);
return null;
}
addPropertyToJsonArray(jsonExtent.ref(), column, datum);
}
root.ref().add(new character(column.getSerializeName()), jsonExtent);
}
else // plain property
{
BaseDataType datum = null;
try
{
datum = (BaseDataType) column.getGetter().invoke(dmo);
}
catch (IllegalAccessException | InvocationTargetException e)
{
ErrorManager.recordOrThrowError(-4, "", null, null);
return null;
}
addPropertyToJson(root.ref(), column, datum);
}
}
return root;
}
/**
* Adds a value as a new property to a JSON object. Only for scalar values.
*
* @param dest
* The destination JSON object.
* @param column
* The column which provides the type and name of the property.
* @param datum
* The actual data to be set as property's value.
*/
private void addPropertyToJson(JsonObject dest, TempTableSchema.Column column, BaseDataType datum)
{
switch (column.getType().getSimpleName().toLowerCase())
{
case "character": dest.add(new character(column.getSerializeName()), (character) datum); break;
case "comhandle": dest.add(new character(column.getSerializeName()), (comhandle) datum); break;
case "date": dest.add(new character(column.getSerializeName()), (date) datum); break;
case "datetime": dest.add(new character(column.getSerializeName()), (datetime) datum); break;
case "datetimetz": dest.add(new character(column.getSerializeName()), (datetimetz) datum); break;
case "decimal": dest.add(new character(column.getSerializeName()), (decimal) datum); break;
case "handle": dest.add(new character(column.getSerializeName()), (handle) datum); break;
case "int64": dest.add(new character(column.getSerializeName()), (int64) datum); break;
case "integer": dest.add(new character(column.getSerializeName()), (integer) datum); break;
case "logical": dest.add(new character(column.getSerializeName()), (logical) datum); break;
case "longchar": dest.add(new character(column.getSerializeName()), (longchar) datum); break;
case "memptr": dest.add(new character(column.getSerializeName()), (memptr) datum); break;
case "raw": dest.add(new character(column.getSerializeName()), (raw) datum); break;
case "recid": dest.add(new character(column.getSerializeName()), (recid) datum); break;
case "rowid": dest.add(new character(column.getSerializeName()), (rowid) datum); break;
case "object": dest.add_1(new character(column.getSerializeName()), (object) datum); break;
case "blob": dest.add(new character(column.getSerializeName()), (blob) datum); break;
case "clob": dest.add(new character(column.getSerializeName()), (clob) datum); break;
}
}
/**
* Adds a value as a new indexed property in JSON array object.
*
* @param dest
* The destination JSON array object.
* @param column
* The column which provides the type of the property.
* @param datum
* The actual data to be set as property's value.
*/
private void addPropertyToJsonArray(JsonArray dest, TempTableSchema.Column column, BaseDataType datum)
{
switch (column.getType().getSimpleName().toLowerCase())
{
case "character": dest.add((character) datum); break;
case "comhandle": dest.add((comhandle) datum); break;
case "date": dest.add((date) datum); break;
case "datetime": dest.add((datetime) datum); break;
case "datetimetz": dest.add((datetimetz) datum); break;
case "decimal": dest.add((decimal) datum); break;
case "handle": dest.add((handle) datum); break;
case "int64": dest.add((int64) datum); break;
case "integer": dest.add((integer) datum); break;
case "logical": dest.add((logical) datum); break;
case "longchar": dest.add((longchar) datum); break;
case "memptr": dest.add((memptr) datum); break;
case "raw": dest.add((raw) datum); break;
case "recid": dest.add((recid) datum); break;
case "rowid": dest.add((rowid) datum); break;
case "object": dest.add_2((object) datum); break;
case "blob": dest.add((blob) datum); break;
case "clob": dest.add((clob) datum); break;
}
}
/**
* Write the content of the whole dataset to the destination JSON object found in target.
*
* @param dataSet
* The buffer whose table is to be saved to JSON object.
* @param target
* The target which contains the destination object.
* @param noInit
* {@code true} to omit data for fields whose values match their initial values.
* @param noOuter
* {@code true} to omit data for the outermost object from the output.
* @param beforeImage
* {@code true} to write before-image data and error information, else {@code false}.
*
* @return {@code true} on success.
*
* @throws PersistenceException
* On persistence issues.
*/
private boolean writeDatasetToJson(DataSet dataSet,
TargetData target,
boolean noInit,
boolean noOuter,
boolean beforeImage)
throws PersistenceException
{
String opType = target.getType();
boolean isJOTarget = Serializator.TYPE_JSON_OBJECT.equalsIgnoreCase(opType);
if (!isJOTarget)
{
ErrorManager.recordOrShowError(19043, "WRITE-JSON");
// JsonArray is not a valid target-type for <method> method. (19043)
ErrorManager.recordOrShowError(4065, false, "WRITE-JSON", "DATASET widget");
// **The <attribute> attribute on the <widget id> has invalid arguments. (4065)
return false;
}
Object ufo = target.getDestination();
if (!(ufo instanceof ObjectVar) || !(((ObjectVar<?>) ufo).ref() instanceof JsonObject))
{
ErrorManager.recordOrShowError(19079, "JsonObject", "WRITE-JSON");
// <argument> argument for <method-name> is not valid. (19079)
ErrorManager.recordOrShowError(4065, false, "WRITE-JSON", "DATASET widget");
// **The <attribute> attribute on the <widget id> has invalid arguments. (4065)
return false;
}
JsonObject json = (JsonObject) (((ObjectVar<?>) ufo).ref());
clearObject(json);
JsonConstruct.ObjectBuilder jsonExportOutput =
new JsonConstruct.ObjectBuilder(json);
exportDataSet(dataSet, noInit, noOuter, beforeImage, jsonExportOutput, true);
// this call should be performed as the first Json node is added through the ObjectBuilder constructor
jsonExportOutput.endObject();
return true;
}
/**
* Actual implementation of a table serialization. This is called for both for TEMP-TABLEs
* (once) and for DATASETs (one time for each member buffer, and supplementary for any eventual
* before-buffer)
*
* @param json
* The target of the deserialized json data.
* @param relation
* The relation on which this buffer is a child of.
* @param buffer
* A buffer of the temp-table to be serialized.
* @param overrideBeforeName
* Overrides the name of the table. The BEFORE-TABLE data is serialized under the name
* of the main (AFTER) buffer.
* @param noInit
* Skip the initial values.
* @param omitOuter
* {@code True} to omit data for the outermost object from the output.
* @param hiddenFields
* {@code true} to add the internal flags specific to BEFORE-BUFFER records.
* @param dsFlags
* {@code true} to add the internal flags specific to DataSet TempTable records.
* @param dsBeforeImage
* {@code true} to write the before-image if available.
*
* @return {@code true} if everything went fine, and {@code false} if a non-fatal error was encountered
*
* @throws IOException
* If exceptions occur during the serialization.
*/
private boolean serializeTempTable(JsonStructureCallback json,
DataRelation relation,
TemporaryBuffer buffer,
String overrideBeforeName,
boolean noInit,
boolean omitOuter,
boolean hiddenFields,
boolean dsFlags,
boolean dsBeforeImage)
throws IOException
{
TempTableSchema schema = tableSchemas.computeIfAbsent(buffer, (k) -> new TempTableSchema(buffer, true));
boolean isBefore = (overrideBeforeName != null);
Buffer currentBuffer = (Buffer) buffer.getDMOProxy();
String serializeName = currentBuffer.getSerializeName().toJavaType();
String tableName = isBefore
? overrideBeforeName
: serializeName != null && !serializeName.isEmpty()
? serializeName
: schema.getTableName();
// if (checkSupportedTypes(schema))
// {
// return false;
// }
if (relation == null)
{
// reads the records in order of primary index, if available, else in ascending primary key order
if (!omitOuter)
{
json.startObject();
}
if (dsFlags || !omitOuter)
{
json.fieldName(tableName);
}
json.startArray();
buffer.readAllRows(
(dmo) -> writeRecord(json,
buffer,
dmo,
schema,
tableName,
hiddenFields,
dsFlags,
dsFlags && isBefore,
dsBeforeImage,
noInit));
json.endArray();
if (!omitOuter)
{
json.endObject();
}
return true;
}
// build a query depending on the relation, and add each record from the relation
QueryWrapper query = (QueryWrapper) relation.getQuery();
query.hintFullResults();
query.queryOpen();
query.first();
boolean hasRecords = !query._isOffEnd();
if (hasRecords)
{
json.fieldName(tableName);
json.startArray();
}
while (!query._isOffEnd())
{
writeRecord(json,
buffer,
buffer.getCurrentRecord(),
schema,
tableName,
hiddenFields,
dsFlags,
dsFlags && isBefore,
dsBeforeImage,
noInit);
query._getNext();
}
query.close();
if (hasRecords)
{
json.endArray();
}
return true;
}
/**
* Actual implementation of a table serialization into a JsonArray (only rows).
*
* @param json
* The target of the deserialized json data.
* @param buffer
* A buffer of the temp-table to be serialized.
* @param omitOuter
* {@code True} to omit data for the outermost object from the output.
* @param noInit
* Skip the initial values.
*
* @throws IOException
* If exceptions occur during the serialization.
*/
public boolean serializeTempTable(JsonStructureCallback json,
TemporaryBuffer buffer,
boolean omitOuter,
boolean noInit)
throws IOException
{
TempTableSchema schema = new TempTableSchema(buffer, true);
TempTable tempTable = buffer.getParentTable();
String serializeName = tempTable.getSerializeName().toJavaType();
String tableName = serializeName != null && !serializeName.isEmpty() ? serializeName
: schema.getTableName();
if (!omitOuter)
{
json.fieldName(tableName);
json.startArray();
}
buffer.readAllRows((dmo) -> writeRecord(json, buffer, dmo, schema, tableName, false, false,
false, false, noInit));
if (!omitOuter)
{
json.endArray();
}
return true;
}
/**
* Actual implementation of a temp-table buffer serialization into a JsonArray (only current row).
*
* @param json
* The target of the deserialized json data.
* @param buffer
* A buffer record to be serialized.
* @param noInit
* Skip the initial values.
*
* @throws IOException
* If exceptions occur during the serialization.
*/
public boolean serializeRecord(JsonStructureCallback json, TemporaryBuffer buffer, boolean noInit)
throws IOException
{
if (json != null && buffer != null)
{
try
{
writeRecord(json, buffer.getCurrentRecord(), new TempTableSchema(buffer, true), false, noInit);
return true;
}
catch (IllegalAccessException | IllegalArgumentException | InvocationTargetException exc)
{
throw new RuntimeException("Error exporting JSON data", exc);
}
}
return false;
}
/**
* Constructor specific to DATASET serialization in a JSON Object.
*
* @param json
* This is the target for the deserialized json data.
* @param dataSet
* The target object to be serialized.
* @param noInit
* {@code True} to omit data for fields whose values match their initial values.
* @param noOuter
* {@code True} to omit data for the outermost object from the output.
* @param beforeImage
* {@code True} to write before-image data and error information, else {@code false}.
*
* @throws IOException
* On persistence issues.
*/
public void serializeDataSet(JsonStructureCallback json,
DataSet dataSet,
boolean noInit,
boolean noOuter,
boolean beforeImage)
throws IOException
{
boolean writeDataset = !noOuter && !dataSet.getSerializeHidden().booleanValue();
if (writeDataset)
{
json.fieldName(dataSet.getSerializeName().toStringMessage());
json.startObject();
}
int ttCount = dataSet.numBuffers().intValue();
boolean hasChanges = false;
for (int k = 0; k < ttCount; k++)
{
BufferImpl after = dataSet.getBufferByIndex(k + 1);
if (!after.isAfterBuffer())
{
continue; // SIMPLE buffers don't have changes
}
BufferImpl before = (BufferImpl) after.beforeBufferNative();
TemporaryBuffer buffer = (TemporaryBuffer) before.buffer();
if (buffer.count() > 0)
{
hasChanges = true;
break;
}
}
if (beforeImage /*&& hasChanges */)
{
json.fieldName("prods:hasChanges");
json.booleanValue(true);
}
nextBuffer: for (int k = 0; k < ttCount; k++)
{
BufferImpl after = (BufferImpl) dataSet.getBufferByIndex(k + 1);
List<DataRelation> rels = dataSet.getRelations(after, false, true, true);
for (DataRelation rel : rels)
{
if (rel.isNested().booleanValue() || rel.isParentIdRelation().booleanValue())
{
// if [after] is a child buffer in a active nested relation, its data was (or
// will be serialized with the parent)
continue nextBuffer;
}
}
// if there are no active relation with [after] as child buffer, serialize it now
TemporaryBuffer buffer = (TemporaryBuffer) after.buffer();
if (buffer.count() > 0)
{
serializeTempTable(
json, null, buffer, null, noInit, true, false, true, beforeImage);
}
}
if (beforeImage)
{
json.fieldName("prods:before");
json.startObject();
for (int k = 0; k < ttCount; k++)
{
BufferImpl after = (BufferImpl) dataSet.getBufferByIndex(k + 1);
if (!after.isAfterBuffer())
{
continue;
}
BufferImpl before = (BufferImpl) after.beforeBufferNative();
TemporaryBuffer buffer = (TemporaryBuffer) before.buffer();
if (buffer.count() <= 0)
{
continue;
}
String nameOverride = after.name().toStringMessage();
serializeTempTable(json, null, buffer, nameOverride, noInit, true, false, true, beforeImage);
}
json.endObject();
}
if (writeDataset)
{
json.endObject();
}
}
/**
* Serialize a single record to the JSON output.
*
* @param json
* Target of the deserialized json data.
* @param buffer
* The buffer to which the DMO belongs.
* @param dmo
* DMO representing the record.
* @param schema
* The temp-table schema.
* @param hiddenFields
* Write the hidden fields ({@code __ERROR_FLAG__}, {@code __ORIGIN_ROWID__},
* {@code __ERROR_STRING__}, {@code __AFTER_ROWID__}, {@code __ROW_STATE__}).
* @param dsFlags
* {@code true} to add the internal flags specific to DataSet TempTable records.
* @param skipCreateFields
* {@code true} to skip CREATED records in before table.
* @param dsBeforeImage
* {@code true} to write the before-image if available.
* @param noInit
* Flag indicating the initial values are to be omitted.
*/
private void writeRecord(JsonStructureCallback json,
TemporaryBuffer buffer,
TempRecord dmo,
TempTableSchema schema,
String rowName,
boolean hiddenFields,
boolean dsFlags,
boolean skipCreateFields,
boolean dsBeforeImage,
boolean noInit)
{
if (dmo == null)
{
return;
}
Integer rowState = dmo._rowState();
if (rowState == null)
{
rowState = Buffer.ROW_UNMODIFIED;
}
if (skipCreateFields && rowState == Buffer.ROW_CREATED)
{
// NOTE: it seems like the CREATED records are NOT serialized at all for BEFORE TEMP-TABLEs, so these
// values should not reach past this point, so the records are cut here
return;
}
try
{
json.startObject();
if (dsFlags && dsBeforeImage && rowState != Buffer.ROW_UNMODIFIED)
{
json.fieldName("prods:id");
// this links the BEFORE and AFTER image. Because the deleted records (from
// BEFORE-TABLE) do not have a peer their rowid will be used, as an exception.
if (rowState == Buffer.ROW_DELETED || !schema.isBeforeImage())
{
// DELETED before, CREATED or MODIFIED after images
json.stringValue(rowName + dmo.primaryKey());
}
else
{
// CREATED or MODIFIED before images
json.stringValue(rowName + dmo._peerRowid());
}
json.fieldName("prods:rowState");
json.stringValue(Util.getRowStateAsString(rowState));
}
writeRecord(json, dmo, schema, hiddenFields, noInit);
if (dsFlags)
{
BufferImpl buf = (BufferImpl) buffer.getDMOProxy();
DataSet ds = buf._dataSet();
if (ds != null)
{
// position the buffer on this DMO
buf.buffer().loadRecord(dmo);
ArrayList<DataRelation> childRels = ds.getRelations(buf, true, false, true);
for (int i = 0; i < childRels.size(); i++)
{
DataRelation dr = childRels.get(i);
if (dr.isNested().booleanValue() || dr.isParentIdRelation().booleanValue())
{
BufferImpl childBuffer = (BufferImpl) dr.getChildBufferNative();
TemporaryBuffer child = (TemporaryBuffer) childBuffer.buffer();
Set<String> cFields = this.childFields;
childFields = childFieldsByChild.getOrDefault(childBuffer._name(),
Collections.emptySet());
try
{
serializeTempTable(json, dr, child, null, noInit, true, false, true, dsBeforeImage);
}
finally
{
this.childFields = cFields;
}
}
}
}
}
json.endObject();
}
catch (IllegalAccessException |
IllegalArgumentException |
InvocationTargetException |
IOException exc)
{
throw new RuntimeException("Error exporting JSON data", exc);
}
}
/**
* Serialize a single buffer record to the JSON output.
*
* @param json
* Target of the deserialized json data.
* @param dmo
* DMO representing the record.
* @param schema
* The temp-table schema.
* @param hiddenFields
* Write the hidden fields ({@code __ERROR_FLAG__}, {@code __ORIGIN_ROWID__},
* {@code __ERROR_STRING__}, {@code __AFTER_ROWID__}, {@code __ROW_STATE__}).
* @param noInit
* Flag indicating the initial values are to be omitted.
*/
private boolean writeRecord(JsonStructureCallback json,
TempRecord dmo,
TempTableSchema schema,
boolean hiddenFields,
boolean noInit)
throws IOException,
IllegalAccessException,
IllegalArgumentException,
InvocationTargetException
{
if (json != null && dmo != null)
{
ArrayList<TempTableSchema.Column> columns = schema.columns();
for (int k = 0; k < columns.size(); k++)
{
TempTableSchema.Column column = columns.get(k);
String name = column.getSerializeName();
if (!hiddenFields && ReservedProperty.isReservedProperty(name) ||
childFields.contains(column.getFieldName()) || column.isSerializeHidden())
{
// fields which start with "__" are hidden fields (specific to BEFORE tables)
continue;
}
Integer extent = column.getExtent();
if (extent != null && extent > 0)
{
if (noInit)
{
boolean untouched = true;
for (int i = 0; i < extent; i++)
{
if (column.isChanged((BaseDataType) column.getGetter().invoke(dmo, i)))
{
untouched = false;
break; // drop all other tests from this extent
}
}
if (untouched)
{
continue; // skip to next column
}
}
json.fieldName(name);
json.startArray();
for (int i = 0; i < extent; i++)
{
writeDatum(json, column, dmo, i);
}
json.endArray();
}
else
{
if (noInit)
{
if (!column.isChanged((BaseDataType) column.getGetter().invoke(dmo)))
{
continue; // skip to next column
}
}
writeDatum(json, column, dmo, null);
}
}
return true;
}
return false;
}
/**
* Checks supported data types. The evaluation is delegated to {@code Util.checkSupportedTypes()}.
* The method returns {@code true} and prints an error messages is unsupported types are detected.
*
* @return {@code true} if a type problem was discovered.
*/
private boolean checkSupportedTypes(TempTableSchema schema)
{
String unsupportedType = Util.checkSupportedTypes(schema);
if (unsupportedType != null)
{
// NOTE: this test is executed for each serialized row! Error 15391 is printed multiple
// times in message line, one for each record.
ErrorManager.recordOrShowError(15391, unsupportedType);
// Unsupported data type for JSON serialization: <datatype>.
return true;
}
return false;
}
/**
* Write a single data value to the JSON generator's output stream, according to the
* instructions stored in the temp-table schema.
*
* @param json
* Target of the deserialized json data.
* @param column
* Temp-table column schema information.
* @param dmo
* DMO containing datum associated with the specified column.
* @param index
* Zero-based index of an extent field element. Should be {@code null} for a scalar field.
*
* @return {@code false} on errors. The error was already reported through {@code ErrorManager}.
*
* @throws IOException
* if the JSON generator cannot write data.
* @throws InvocationTargetException
* if the method invoked to get the data from the DMO fails.
* @throws IllegalArgumentException
* if an argument passed to a DMO getter is invalid.
* @throws IllegalAccessException
* if there is an access/security problem.
*/
private boolean writeDatum(JsonStructureCallback json,
TempTableSchema.Column column,
Record dmo,
Integer index)
throws IOException,
IllegalAccessException,
IllegalArgumentException,
InvocationTargetException
{
boolean extent = index != null;
Object datum = OrmUtils.getField(dmo, column.offset + (index == null ? 0 : index));
if (!extent)
{
json.fieldName(column.getSerializeName());
}
if (datum == null)
{
json.nullValue();
return true;
}
else if (!Util.checkSupportedType(column))
{
ErrorManager.recordOrThrowError(17586, Util.getLegacyType(column.getType()));
// Unsupported data type for JSON serialization: <datatype>.
return false;
}
// lookup JSON writer function by data type; we cache lambdas to minimize use of
// isAssignableFrom native method call
Class<? extends BaseDataType> type = column.getType();
ThrowingConsumer fn = fnMap.get(type);
if (fn == null)
{
if (int64.class.isAssignableFrom(type))
{
fn = (d) -> json.numberValue(((Number) d).longValue());
}
else if (Text.class.isAssignableFrom(type))
{
fn = (d) -> json.stringValue((String) d);
}
else if (logical.class.isAssignableFrom(type))
{
fn = (d) -> json.booleanValue((Boolean) d);
}
else if (decimal.class.isAssignableFrom(type))
{
// strip trailing zeros, but assure at least one
fn = (d) ->
{
BigDecimal value = ((BigDecimal) d).setScale(((BigDecimal) d).scale(), RoundingMode.HALF_UP)
.stripTrailingZeros();
d = value.scale() <= 0 ? value.setScale(1) : value;
json.numberValue((BigDecimal) d);
};
}
else if (datetimetz.class.isAssignableFrom(type))
{
fn = (d) ->
{
date p2jDate = new datetimetz((OffsetDateTime) d);
if (p2jDate.isUnknown())
{
json.nullValue();
}
else
{
json.stringValue(p2jDate.getIsoDate());
}
};
}
else if (datetime.class.isAssignableFrom(type))
{
fn = (d) ->
{
date p2jDate = new datetime((Timestamp) d);
if (p2jDate.isUnknown())
{
json.nullValue();
}
else
{
json.stringValue(p2jDate.getIsoDate());
}
};
}
else if (date.class.isAssignableFrom(type))
{
fn = (d) ->
{
date p2jDate = new date((Date) d);
if (p2jDate.isUnknown())
{
json.nullValue();
}
else
{
json.stringValue(p2jDate.getIsoDate());
}
};
}
else if (rowid.class.isAssignableFrom(type))
{
fn = (d) ->
{
Long r = (Long) d;
json.stringValue(Util.encodeBase64(r));
};
}
else if (blob.class.isAssignableFrom(type) || raw.class.isAssignableFrom(type))
{
fn = (d) ->
{
byte[] b = (byte[]) d;
Base64.Encoder enc = Base64.getEncoder();
json.stringValue(enc.encodeToString(b));
};
}
else if (handle.class.isAssignableFrom(type))
{
fn = (d) ->
{
Long h = (Long) d;
json.numberValue(h);
};
}
else
{
ErrorManager.recordOrShowError(15391, Util.getLegacyType(type));
// Unsupported data type for JSON serialization: <datatype>.
return false;
}
// TODO: other data types ?
if (fn != null)
{
fnMap.put(type, fn);
}
}
if (fn == null)
{
ErrorManager.recordOrShowError(15391, Util.getLegacyType(type));
// Unsupported data type for JSON serialization: <datatype>.
return false;
}
fn.accept(datum);
return true;
}
/**
* Write a single data value of type {@code rowid} passed as a long value, along with its field
* name.
*
* @param json
* Target of the deserialized json data.
* @param name
* The name of the field.
* @param rowidAsLong
* The {@code rowid} value as {@code Long}.
*/
private void writeRowid(JsonStructureCallback json, String name, Long rowidAsLong)
throws IOException
{
json.fieldName(name);
if (rowidAsLong == null)
{
json.nullValue();
return;
}
json.stringValue(Util.encodeBase64(rowidAsLong));
}
/**
* Serialize 4GL legacy error as Json object.
*
* Used by Json entity/body writers in .net package, might be part of some new class in .json package.
* @param poError The 4GL legacy error.
*
* @return The JSON serialization of the error as a Json object.
*/
public static object<? extends JsonObject> serializeError(object<? extends LegacyError> poError)
{
final object<? extends JsonObject> oResponse = new object<>(JsonObject.class);
final object<? extends JsonObject> oError = new object<>(JsonObject.class);
final object<? extends JsonArray> oErrorList = new object<>(JsonArray.class);
integer iLoop = TypeFactory.integer();
Assert.notNull(poError, new character("Error"));
oResponse.assign(ObjectOps.newInstance(JsonObject.class));
oErrorList.assign(ObjectOps.newInstance(JsonArray.class));
if ((ObjectOps.typeOf(poError, AppError.class)).booleanValue())
{
oResponse.ref().add(new character("_retVal"), ObjectOps.cast(poError, AppError.class).ref().getReturnValue());
}
oResponse.ref().add(new character("_errors"), oErrorList);
for (int i = 0; i < poError.ref().getNumMessages().intValue(); i++)
{
iLoop.assign(i + 1);
oError.assign(ObjectOps.newInstance(JsonObject.class));
oErrorList.ref().add_2(oError);
oError.ref().add(new character("_errorMsg"), poError.ref().getMessage(iLoop));
oError.ref().add(new character("_errorNum"), poError.ref().getMessageNum(iLoop));
}
if ((SessionUtils.isDebugAlert()).booleanValue())
{
oResponse.ref().add(new character("_type"), poError.ref().getLegacyClass().ref().getTypeName());
if (!poError.ref().getCallStack().isUnknown())
{
oErrorList.assign(ObjectOps.newInstance(JsonArray.class));
oResponse.ref().add(new character("_stack"), oErrorList);
String[] callStack = TextOps.entries(poError.ref().getCallStack().getValue(), "\\n");
for (int i = 0; i < callStack.length; i++)
{
oErrorList.ref().add(new character(callStack[i]));
}
}
}
// TODO: accessing any method in LegacyClass throws NPE
// ProcedureManager.CalleeInfoImpl.push #5351
// oProp.assign(poError.ref().getLegacyClass().ref().getProperty(new character("InnerError")));
//
// if (_and(and(oProp.isValid(), () -> isEqual(oProp.ref().getDataType(), DataType.object_)), () -> LegacyClass.getLegacyClass(oProp.ref().getDataTypeName()).ref().isA(ObjectOps.getLegacyClass(LegacyError.class))))
// {
// oInner.assign(oProp.ref().get(poError));
//
// if ((oInner.isValid()).booleanValue())
// {
// this.writeError(oInner);
// }
// }
return oResponse;
};
/**
* A consumer which accepts one value and throws IOException, to avoid having to handle this
* exception in every defined lambda expression.
*/
@FunctionalInterface
private interface ThrowingConsumer
{
/**
* Accept and process a data value.
*
* @param datum
* Data value to be processed.
*
* @throws IOException
* if an I/O error occurs.
*/
public void accept(Object datum)
throws IOException;
}
}