Project

General

Profile

Bug #10309

« Previous | Next »

DATASET WRITE-JSON does not change value for a property

Added by Teodor Gorghe about 1 year ago. Updated 10 months ago.

Status:
Test
Priority:
Normal
Assignee:
Teodor Gorghe
Target version:
-
Start date:
Due date:
% Done:

100%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:
reviewer:
Alexandru Lungu, Constantin Asofiei
production:
No
env_name:
topics:

TestWriteDatasetProperty.cls (16.4 KB) Teodor Gorghe, 07/21/2025 05:20 AM

History

#1 Updated by Teodor Gorghe about 1 year ago

When the WRITE-JSON is used to store output into a property longchar, FWD fails on this because the value stored is actually a copy of it's reference:

#3 Updated by Teodor Gorghe about 1 year ago

  • Status changed from New to WIP
  • Assignee set to Teodor Gorghe

Created task branch 10309a.

#4 Updated by Teodor Gorghe about 1 year ago

  • % Done changed from 0 to 30

#5 Updated by Teodor Gorghe about 1 year ago

Committed revision 16057 on task branch 10309a:
- Added conversion and runtime support for property reference on builtin 4GL WRITE methods.

I am attaching the test class file and the tests results below:


Some of the tests fail, but the content gets written into the variable. These tests fail (not related with properties) because of:
- Strange behavior of 4GL XDocument XML implementation, as pointed on #9447.
- SAX-WRITER FWD issue: 4GL writes into longchar before SAX-WRITER:END-DOCUMENT() call. See testSaxWriterSetOutputDestinationBeforeEnd.
- WRITE-XML and WRITE-XMLSCHEMA issue: FWD XML writer uses the shorter syntax on empty XML nodes. 4GL puts the closing tags every time.

#6 Updated by Constantin Asofiei 12 months ago

In SaxWriterImpl, why call TextOps.concat(ref.get(), buf) ?

#7 Updated by Teodor Gorghe 12 months ago

Constantin Asofiei wrote:

In SaxWriterImpl, why call TextOps.concat(ref.get(), buf) ?

I have looked up on LongcharOutputStream and did it in the same way.
I was thinking about on multiple flush calls, but how the logic is right now, I don't see more flush call, but just one, on stream close. This can be used in the future, for the issue described in #10309-5.

#8 Updated by Constantin Asofiei 12 months ago

  • Status changed from Review to Internal Test

#9 Updated by Teodor Gorghe 12 months ago

Ovidiu, testing for this branch is done. The changes are OK and no issues were found on all applications.

#10 Updated by Ovidiu Maxiniuc 12 months ago

OK, ABL OO is not in my domain of expertise, but Constantin reviewed it already.
Yet, I see the %Done is still at 30. I assume was not updated, otherwise remaining work will use a new subsequent branch.

Alexandru, please queue the merge for this branch when appropriate.

#11 Updated by Teodor Gorghe 12 months ago

  • % Done changed from 30 to 100

#12 Updated by Alexandru Lungu 12 months ago

  • Status changed from Internal Test to Merge Pending

Please merge 10309a to trunk after 10308a.

#13 Updated by Teodor Gorghe 12 months ago

  • Status changed from Merge Pending to Test

Branch 10309a was merged to trunk rev 16096 and archived.

#14 Updated by Teodor Gorghe 11 months ago

  • Status changed from Test to WIP
  • % Done changed from 100 to 50

There is a regression on conversion which was not noticed during internal testing. After merging, the original test case still works.
We have this test case:

The conversion fails with the following message:

     [java] ./p-test-w-prop.p
     [java] Elapsed job time:  00:00:01.539
     [java] EXPRESSION EXECUTION ERROR:
     [java] ---------------------------
     [java] lastid = createJavaAst(java.string, #(java.lang.String) ref.getAnnotation("name"), lastid)
     [java]                                                             ^  { Cannot invoke "com.goldencode.ast.Aast.getAnnotation(String)" because the return value of "com.goldencode.expr.CompiledExpression.getVar(int)" is null }
     [java] ---------------------------
     [java] ERROR:
     [java] com.goldencode.p2j.pattern.TreeWalkException: ERROR!  Active Rule:
     [java] -----------------------
     [java]       RULE REPORT      
     [java] -----------------------
     [java] Rule Type :   WALK
     [java] Source AST:  [ : ] BLOCK/ASSIGNMENT/EXPRESSION/COLON/METH_LOGICAL/OBJECT_INVOCATION/ @12:45 {17179869286}
     [java] Copy AST  :  [ : ] BLOCK/ASSIGNMENT/EXPRESSION/COLON/METH_LOGICAL/OBJECT_INVOCATION/ @12:45 {17179869286}
     [java] Condition :  lastid = createJavaAst(java.string, #(java.lang.String) ref.getAnnotation("name"), lastid)
     [java] Loop      :  false
     [java] --- END RULE REPORT ---
     [java] 
     [java] 
     [java] 
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1102)
     [java]     at com.goldencode.p2j.convert.TransformDriver.processTrees(TransformDriver.java:591)
     [java]     at com.goldencode.p2j.convert.ConversionDriver.back(ConversionDriver.java:593)
     [java]     at com.goldencode.p2j.convert.TransformDriver.executeJob(TransformDriver.java:1010)
     [java]     at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:1314)
     [java] Caused by: com.goldencode.expr.ExpressionException: Expression execution error @1:61 [OBJECT_INVOCATION id=17179869286]
     [java]     at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:275)
     [java]     at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:210)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.apply(PatternEngine.java:1696)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1582)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.processAst(PatternEngine.java:1515)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1065)
     [java]     ... 4 more
     [java] Caused by: com.goldencode.expr.ExpressionException: Expression execution error @1:61
     [java]     at com.goldencode.expr.Expression.execute(Expression.java:495)
     [java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:500)
     [java]     at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:751)
     [java]     at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:717)
     [java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:537)
     [java]     at com.goldencode.p2j.pattern.Rule.executeActions(Rule.java:751)
     [java]     at com.goldencode.p2j.pattern.Rule.coreProcessing(Rule.java:717)
     [java]     at com.goldencode.p2j.pattern.Rule.apply(Rule.java:537)
     [java]     at com.goldencode.p2j.pattern.RuleContainer.apply(RuleContainer.java:597)
     [java]     at com.goldencode.p2j.pattern.RuleSet.apply(RuleSet.java:98)
     [java]     at com.goldencode.p2j.pattern.RuleContainer.apply(RuleContainer.java:597)
     [java]     at com.goldencode.p2j.pattern.RuleSet.apply(RuleSet.java:98)
     [java]     at com.goldencode.p2j.pattern.AstWalker.walk(AstWalker.java:262)
     [java]     ... 9 more
     [java] Caused by: java.lang.NullPointerException: Cannot invoke "com.goldencode.ast.Aast.getAnnotation(String)" because the return value of "com.goldencode.expr.CompiledExpression.getVar(int)" is null
     [java]     at com.goldencode.expr.CE13454.execute(Unknown Source)
     [java]     at com.goldencode.expr.Expression.execute(Expression.java:398)
     [java]     ... 21 more

I am investigating this right now and I will fix it ASAP.

#15 Updated by Teodor Gorghe 11 months ago

  • Status changed from WIP to Review
  • % Done changed from 50 to 100
  • reviewer Alexandru Lungu added

Committed revision 16103 on task branch 10309b:
- Fixed regression from 10309a. I have tested with the initial test case, the second test case (which was the regression) and with properties from the static class.

Alexandru, please review my changes.

#16 Updated by Alexandru Lungu 11 months ago

  • Status changed from Review to Internal Test

I don't have full expertise to review these changes, but as they are decremental in relation to 10309a, I suppose they are safe as long as it still fixes #10309-5 tests. Please also do some testing with:

  • output param is a field in a table (e.g. define temp-table tt field OutputMessage as longchar): WRITE-JSON ("LONGCHAR":U, tt.OutputMessage, FALSE, "UTF-8":U, TRUE)
  • output param is a field of an object stored in a field of a table (e.g. define temp-table tt field f1 as ObjUtil.): WRITE-JSON ("LONGCHAR":U, tt.f1.OutputMessage, FALSE, "UTF-8":U, TRUE)
  • output param to a dynamic buffer field: WRITE-JSON ("LONGCHAR":U, buffer tt:handle:BUFFER-FIELD:buffer-value(), FALSE, "UTF-8":U, TRUE)@
    • I am not entirely sure if this works, but I know that buffer tt:handle:BUFFER-FIELD(1):buffer-value() = 1 is a valid statement, so it seems like buffer-value is writable.
  • output param to a this member: WRITE-JSON ("LONGCHAR":U, this-object:OutputMessage, FALSE, "UTF-8":U, TRUE)

Integrate #10309-14 test in the unit test suite. Add tests as suggested.
Please send me an email on what application did you encounter #10309-14 and where (what file/line).

#17 Updated by Teodor Gorghe 11 months ago

Alexandru Lungu wrote:

I don't have full expertise to review these changes, but as they are decremental in relation to 10309a, I suppose they are safe as long as it still fixes #10309-5 tests. Please also do some testing with:

  • output param is a field in a table (e.g. define temp-table tt field OutputMessage as longchar): WRITE-JSON ("LONGCHAR":U, tt.OutputMessage, FALSE, "UTF-8":U, TRUE)
  • output param is a field of an object stored in a field of a table (e.g. define temp-table tt field f1 as ObjUtil.): WRITE-JSON ("LONGCHAR":U, tt.f1.OutputMessage, FALSE, "UTF-8":U, TRUE)
  • output param to a dynamic buffer field: WRITE-JSON ("LONGCHAR":U, buffer tt:handle:BUFFER-FIELD:buffer-value(), FALSE, "UTF-8":U, TRUE)@
    • I am not entirely sure if this works, but I know that buffer tt:handle:BUFFER-FIELD(1):buffer-value() = 1 is a valid statement, so it seems like buffer-value is writable.
  • output param to a this member: WRITE-JSON ("LONGCHAR":U, this-object:OutputMessage, FALSE, "UTF-8":U, TRUE)

Integrate #10309-14 test in the unit test suite. Add tests as suggested.
Please send me an email on what application did you encounter #10309-14 and where (what file/line).

Committed revision 1795 on testcases.

The changes are the following:

Test results (FWD):

      ├─ testTt3PropertyWriteJson ✔
      ├─ testTtpropertyWriteCharacterJson ✔
      ├─ testTtpropertyWriteInvalidPolyJson ✔
      ├─ testTtpropertyThisObjectJson ✔
      ├─ testTtpropertyObjectJson ✔
      ├─ testTtpropertyStaticObjectJson ✔

  • You can't define longchar fields on a temp-table, but you can define clob fields on persistent table on 4GL. You can also define clob TEMP-TABLE fields on FWD.
    This has nothing to do with properties, it gets to getBufferFieldName call before and after 10309 branches. Conversion does not fail (before and after). I think is is OK, please tell me if additional tests are required. If I find a WRITE-XML code which writes into a CLOB field, I will make a fix for this.

#18 Updated by Alexandru Lungu 11 months ago

Please merge 10309b to trunk now.

#19 Updated by Alexandru Lungu 11 months ago

  • Status changed from Internal Test to Merge Pending

#20 Updated by Alexandru Lungu 11 months ago

After merge, please also add some tests with:

  • output into memptr variable (i.e. def var m as memptr., ds:write-json('memptr', m).
  • output into a JsonObject variable (i.e. def var x as Progress.Json.ObjectModel.JsonObject., ds:write-json('JsonObject', x).

#21 Updated by Teodor Gorghe 11 months ago

  • Status changed from Merge Pending to Test

Branch 10309b was merged to trunk rev 16104 and archived.

#22 Updated by Teodor Gorghe 11 months ago

Committed revision 1796 on test cases:
- modified tests/oo/basic/property/TestWriteDatasetProperty.cls with memptr and JsonObject test case on WRITE-JSON. There are no issues on property refs.

#23 Updated by Teodor Gorghe 10 months ago

  • Status changed from Test to WIP

Created task branch 10309c.

I have missed a test case, when the temp-table/dataset handle is obtained using a getter:

CLASS tests.BaseProperty ABSTRACT:
    DEFINE PUBLIC PROPERTY baseLongchar AS LONGCHAR NO-UNDO GET. SET.
    DEFINE PUBLIC PROPERTY dsHandle AS HANDLE NO-UNDO GET. SET.
END CLASS.


BLOCK-LEVEL ON ERROR UNDO, THROW.

USING OpenEdge.Core.Assert FROM PROPATH.

CLASS tests.TestWriteDatasetPropertyInherit INHERITS tests.BaseProperty:
    DEFINE TEMP-TABLE tt1
        FIELD f1 AS INTEGER.

    DEFINE TEMP-TABLE tt2
        FIELD f2 AS INTEGER.

    DEFINE TEMP-TABLE tt3
        FIELD f1 AS Progress.Lang.Object.

    DEFINE TEMP-TABLE tt4
        FIELD OutputMessage AS CHARACTER.

    DEFINE DATASET ds1 FOR tt1, tt2.

    @Test.
    METHOD PUBLIC VOID TestBasePropertyDatasetHandlePropWriteJSON():
        DEFINE VARIABLE cExpected AS LONGCHAR NO-UNDO.
        dsHandle = DATASET ds1:HANDLE.
        cExpected = '~{"ds1":~{~}~}'.
        BaseLongchar = "".
        dsHandle:WRITE-JSON ("LONGCHAR":U, BaseLongchar, FALSE, "UTF-8":U, TRUE). // It gets converted into a getter call.
        Assert:Equals(cExpected, BaseLongchar).
    END METHOD.

    @Test.
    METHOD PUBLIC VOID TestBasePropertyDatasetHandlePropWriteXML():
        DEFINE VARIABLE cExpected AS LONGCHAR NO-UNDO.
        dsHandle = DATASET ds1:HANDLE.
        cExpected = '<?xml version="1.0"?><ds1 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"></ds1>'.
        BaseLongchar = "".
        dsHandle:WRITE-XML ("LONGCHAR":U, BaseLongchar). // It gets converted into a getter call.
        Assert:Equals(cExpected, BaseLongchar).
    END METHOD.

    @Test.
    METHOD PUBLIC VOID TestBasePropertyDatasetHandlePropWriteXMLSchema():
        DEFINE VARIABLE cExpected AS LONGCHAR NO-UNDO.
        dsHandle = DATASET ds1:HANDLE.
        cExpected = '<?xml version="1.0"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="" xmlns:prodata="urn:schemas-progress-com:xml-prodata:0001"><xsd:element name="ds1" prodata:proDataSet="true"><xsd:complexType><xsd:sequence><xsd:element name="tt1" minOccurs="0" maxOccurs="unbounded" prodata:undo="true"><xsd:complexType><xsd:sequence><xsd:element name="f1" type="xsd:int" nillable="true"/></xsd:sequence></xsd:complexType></xsd:element><xsd:element name="tt2" minOccurs="0" maxOccurs="unbounded" prodata:undo="true"><xsd:complexType><xsd:sequence><xsd:element name="f2" type="xsd:int" nillable="true"/></xsd:sequence></xsd:complexType></xsd:element></xsd:sequence></xsd:complexType></xsd:element></xsd:schema>'.
        BaseLongchar = "".
        dsHandle:WRITE-XMLSCHEMA ("LONGCHAR":U, BaseLongchar). // It gets converted into a getter call.
        Assert:Equals(cExpected, BaseLongchar).
    END METHOD.
END CLASS.

#24 Updated by Teodor Gorghe 10 months ago

  • Status changed from WIP to Review

Committed revision 16161 on task branch 10309c:
- Added support for handle method/properties on 'is_method_output_argument'.

#25 Updated by Constantin Asofiei 10 months ago

The correct function is this:

      <function name="is_method_output_argument">
         <parameter  name="target"  type="com.goldencode.ast.Aast" />
         <variable   name="ref"     type="com.goldencode.ast.Aast" />
         <variable   name="ref2"    type="com.goldencode.ast.Aast" />
         <return     name="match"   type="java.lang.Boolean" />

         <rule>match = false</rule>

         <rule>ref = target.parent</rule>
         <rule>ref.type == prog.meth_logical
            <action>ref2 = ref.parent</action>
            <rule>ref2.type == prog.colon
               <action>ref2 = ref2.firstChild</action>
               <rule>((evalLib("type_pair", ref, prog.meth_logical, prog.kw_wr_json)  or
                       evalLib("type_pair", ref, prog.meth_logical, prog.kw_wr_xmlsc) or
                       evalLib("type_pair", ref, prog.meth_logical, prog.kw_wr_xml))
                                                                     and this.indexPos == 1) or
                     (evalLib("type_pair", ref, prog.meth_logical, prog.kw_serialzr)
                                                                     and this.indexPos == 2) or
                     ((evalLib("type_pair", ref, prog.meth_logical, prog.kw_save) or
                       evalLib("type_pair", ref, prog.meth_logical, prog.kw_set_odst))
                                                                     and this.indexPos == 1)
                  <action>match = true</action>
               </rule>
            </rule>
         </rule>
      </function>

What matters is that there is a call for a certain handle method, and type_pair takes care of it; it doesn't matter who performs the call.

#26 Updated by Constantin Asofiei 10 months ago

  • Status changed from Review to Internal Test

The change is in 10309c rev 16162.

#27 Updated by Constantin Asofiei 10 months ago

  • Status changed from Internal Test to Merge Pending

Teodor, please merge now.

#28 Updated by Teodor Gorghe 10 months ago

  • Status changed from Merge Pending to Test

Branch 10309c was merged to trunk rev. 16205 and archived.

Also available in: Atom PDF