Project

General

Profile

Bug #10560

Wrong buffer picking for ReadXml when seralization name of a buffer correspond to the name of the buffer from dataset

Added by Eduard Soltan 10 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
-
Start date:
Due date:
% Done:

100%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:
production:
No
env_name:
topics:

Related issues

Related to Base Language - Bug #10352: Fix READ-JSON dataset handle method's behaviour Closed

History

#1 Updated by Eduard Soltan 10 months ago

  • Subject changed from Wrong buffer picking for writeXml when seralization name of a buffer correspond to the name of the buffer from dataset to Wrong buffer picking for ReadXml when seralization name of a buffer correspond to the name of the buffer from dataset

I have the following test case:

DEFINE TEMP-TABLE Main NO-UNDO
    FIELD ProcessField_id  AS RECID XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE First NO-UNDO NAMESPACE-URI "http://www.openapplications.org/oagis/10":U
    FIELD Main_id AS RECID XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE Second NO-UNDO NAMESPACE-URI "http://www.openapplications.org/oagis/10":U
    FIELD Main_id AS RECID XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE ID NO-UNDO
    FIELD ID_Text1   AS CHARACTER 
    FIELD First_id   AS RECID      XML-NODE-TYPE "HIDDEN":U
    FIELD Second_id  AS RECID      XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE oaID NO-UNDO NAMESPACE-URI "http://www.openapplications.org/oagis/10":U SERIALIZE-NAME "ID":U  LIKE ID.

DEFINE DATASET dsProcessField  XML-NODE-TYPE "HIDDEN":U
    FOR Main, First, Second, ID, oaID
    PARENT-ID-RELATION FOR Main, First   PARENT-ID-FIELD Main_id
    PARENT-ID-RELATION FOR Main, Second  PARENT-ID-FIELD Main_id
    PARENT-ID-RELATION FOR First, oaID   PARENT-ID-FIELD First_id
    PARENT-ID-RELATION FOR Receiver, ID  PARENT-ID-FIELD Second_id.

DEFINE VARIABLE cSourceType             AS CHARACTER NO-UNDO.
DEFINE VARIABLE cReadMode               AS CHARACTER NO-UNDO.
DEFINE VARIABLE lOverrideDefaultMapping AS LOGICAL   NO-UNDO.
DEFINE VARIABLE cFile                   AS CHARACTER NO-UNDO.
DEFINE VARIABLE cSchemaLocation         AS CHARACTER NO-UNDO.
DEFINE VARIABLE cFieldTypeMapping       AS CHARACTER NO-UNDO.
DEFINE VARIABLE cVerifySchemaMode       AS CHARACTER NO-UNDO.
DEFINE VARIABLE lRetOK                  AS LOGICAL   NO-UNDO.
DEFINE VARIABLE httCust                 AS HANDLE    NO-UNDO.

CREATE TEMP-TABLE httCust.
ASSIGN
  cSourceType             = "file" 
  cFile                   = "dset.xml" 
  cReadMode               = "empty" 
  cSchemaLocation         = "" 
  lOverrideDefaultMapping = ?
  cFieldTypeMapping       = ?
  cVerifySchemaMode       = ?.

lRetOK = dataset dsProcessField:READ-XML(cSourceType, cFile, cReadMode, cSchemaLocation, lOverrideDefaultMapping, cFieldTypeMapping, cVerifySchemaMode).

find first oaID.

message oaId.ID_Text1.

One thing to note here is that the oaID and ID tables will be serialized on the same XML tag <ID>.

This will cause a problem at readXML, because we will pick the buffer in which to put a new record using DataSet.getBufferByName method. This method only uses the actual buffer name to make the matching. Hence the wrong buffer is selected ID, instead of oaID.

I thing we should implement something different here, to look for serialization names of the buffers that are in a relationship with the current buffer.

#3 Updated by Constantin Asofiei 10 months ago

There is another case where there is a field f1 serialize-name "oa:f1", and the XML has the field like f1 - in this and only this case, it will be picked up even if the XML file doesn't have the oa namespace.

I have a fix for this and the issue reported by Eduard, but needs more testing. I'll create and commit it to 10560a.

#4 Updated by Constantin Asofiei 10 months ago

  • Related to Bug #10352: Fix READ-JSON dataset handle method's behaviour added

#5 Updated by Constantin Asofiei 10 months ago

Eduard, see 10560a rev 16164 for my current changes. Please try to break them (see the TODO in XmlImport) by creating misc relations.

#6 Updated by Eduard Soltan 10 months ago

I have created the following dataset structure, one thing to note that Second table has a relationship with with ID and oaID table (serialization-name ID) :

DEFINE TEMP-TABLE Main NO-UNDO
    FIELD first_id  AS RECID XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE Second NO-UNDO NAMESPACE-URI "http://www.openapplications.org/oagis/10":U
    FIELD Main_id AS RECID XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE Third NO-UNDO NAMESPACE-URI "http://www.openapplications.org/oagis/10":U
    FIELD Main_id AS RECID XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE ID NO-UNDO
    FIELD ID_Text1        AS CHARACTER  XML-NODE-TYPE "TEXT":U
    FIELD Second_id       AS RECID      XML-NODE-TYPE "HIDDEN":U
    FIELD Third_id        AS RECID      XML-NODE-TYPE "HIDDEN":U.

DEFINE TEMP-TABLE oaID NO-UNDO NAMESPACE-URI "http://www.openapplications.org/oagis/10":U SERIALIZE-NAME "ID":U
         FIELD ID_Text1    AS CHARACTER  XML-NODE-TYPE "TEXT":U
         FIELD Second_id   AS RECID      XML-NODE-TYPE "HIDDEN":U
         FIELD Third_id    AS RECID      XML-NODE-TYPE "HIDDEN":U.

DEFINE DATASET dsProcessField  XML-NODE-TYPE "HIDDEN":U
    FOR Main, Second, Third, ID, oaID
    PARENT-ID-RELATION FOR Main, Second PARENT-ID-FIELD Main_id
    PARENT-ID-RELATION FOR Main, Third  PARENT-ID-FIELD Main_id
    PARENT-ID-RELATION FOR Second, ID    PARENT-ID-FIELD Second_id
    PARENT-ID-RELATION FOR Second, oaID PARENT-ID-FIELD Second_id
    PARENT-ID-RELATION FOR Third, ID    PARENT-ID-FIELD Third_id.

Exported the following structure into an XML:

create Main.

create Second.
Second.Main_id = RECID(Main).
create Third.
Third.Main_id = recid(Main).

create oaID.
oaID.ID_Text1 = 'text1'.
oaID.Second_id = recid(Second).
release oaID.

create ID.
ID.ID_Text1 = 'text2'.
ID.Second_id = recid(Second).
release ID.

create ID.
ID.ID_Text1 = 'text3'.
ID.Third_id = recid(Third).
release ID.

release Second.
release Third.
release Main.

And got the following result:

<?xml version="1.0"?>
<Main xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Second xmlns="http://www.openapplications.org/oagis/10">
    <ID>text1</ID>
    <ID>text2</ID>
  </Second>
  <Third xmlns="http://www.openapplications.org/oagis/10">
    <ID>text3</ID>
  </Third>
</Main>

Later when trying to load the same XML into the dataset, both <ID> nodes load into a single table. It chooses the first relationship that matches the serialization-name of the XML tag. And the second table that has the same serialization-name, but it's relationship is just ignored.

#7 Updated by Eduard Soltan 10 months ago

I think I found a problem in rev. 16164.

In XmlImport.resolveTableByXmlName the current context should be used as opposed to the context saved on stack ImportContext ctx = peekContext();

Committed on 10560a, rev. 16165.

#8 Updated by Teodor Gorghe 10 months ago

On XmlImport.resolveTableByXmlName, I don't think is right to use the ctx.ttname, because the value is the local name for the current XML node and the relation.getParentTable() contains the legacy name.
I suggest changing it to ctx.tt._name(). This fixes some unit tests.

#9 Updated by Teodor Gorghe 10 months ago

  • Assignee set to Eduard Soltan
  • Status changed from New to WIP
  • % Done changed from 0 to 90

#10 Updated by Teodor Gorghe 10 months ago

Should I commit the change from #10560-8?

#11 Updated by Eduard Soltan 10 months ago

Teodor Gorghe wrote:

Should I commit the change from #10560-8?

Go ahead.

#12 Updated by Teodor Gorghe 10 months ago

  • Status changed from WIP to Review
  • % Done changed from 90 to 100
  • reviewer Constantin Asofiei added

Committed revision 16166 on task branch 16166:
- Resolving the parent buffer by its legacy name instead of the XML local name.

#13 Updated by Constantin Asofiei 10 months ago

Teodor Gorghe wrote:

Committed revision 16166 on task branch 16166:
- Resolving the parent buffer by its legacy name instead of the XML local name.

What task branch is this in?

#14 Updated by Teodor Gorghe 10 months ago

Oops, this commit is in this task, 10560a.

#15 Updated by Constantin Asofiei 10 months ago

I've committed my changes to 10560a rev 16167 - use a context in 'resolveTableByXmlName'.

#16 Updated by Teodor Gorghe 10 months ago

I have a test case which does not work with 10352a, trunk or this branch:

Constantin, should I create a new task or work here/on 10352a?

#17 Updated by Constantin Asofiei 10 months ago

Work it on this branch.

#18 Updated by Teodor Gorghe 10 months ago

For the test case from #10560-16, 4GL passes, but FWD fails because the name field is empty.

I am currently exploiting all hidden behavior from 4GL.

What I have found:
  1. When the parentId field has XML-NODE-TYPE to HIDDEN, 4GL acts like FWD. The name field is empty and there is no Name record.
  2. When the f1 field is a type of TEXT, the serialized XML gets something like that:
    When this is read with READ-XML, the name field from tt1 table is written twice (same as FWD).

#19 Updated by Teodor Gorghe 10 months ago

Later update:
- And when the Name temp-table has a structure, like this

Then, the temp-table gets serialized like as the #10560-16 case.
- When the Name temp-table has 2 node fields, and the parentId is hidden, the read-xml acts like #10560-16.

#20 Updated by Constantin Asofiei 10 months ago

  • Status changed from Review to WIP
  • Assignee changed from Eduard Soltan to Teodor Gorghe
  • % Done changed from 100 to 90

As I understand there is still more work on this.

#21 Updated by Constantin Asofiei 10 months ago

Teodor: please make sure to run your tests in a later version than OE 11.6 - especially when you find 'weird quirks'.

#22 Updated by Teodor Gorghe 10 months ago

I know that this behavior also happens for OE 11.7. Do we have another OE version available?

#23 Updated by Teodor Gorghe 10 months ago

I have tested with OE 12.8 and I can confirm that everything that happens on OE 11.6 is the same as OE 12.8.

#24 Updated by Teodor Gorghe 10 months ago

Committed revision 16168 on task branch 10560a:
- Added support on DATASET:READ-XML for matching XML field and temp-table with same serializable name/XML node name.

Constantin, the work is done, but I am not very confident on my changes, might be some regressions. I am starting to do some testing, but I have committed because I need a review.

#25 Updated by Teodor Gorghe 10 months ago

  • Status changed from WIP to Review

#26 Updated by Teodor Gorghe 10 months ago

Committed revision 16169 on task branch 10560a:
- Added some missing javadoc description. Added one more case which 4GL treats as a temp-table node.

#27 Updated by Constantin Asofiei 9 months ago

Review of 10560a rev 16169:
  • please make it compile in Java 8 - see XmlImport line 3949, the .toList() call;
  • update the copyright year
  • LookAheadXmlStreamReader
    • not all methods use the replay stack - see isStandalone(), isWhiteSpace() and others
    • move the defined classes to the end of the file

#28 Updated by Teodor Gorghe 9 months ago

Committed revision 16170 on task branch 10560a:
- Addressed code review #10560-27.

#29 Updated by Teodor Gorghe 9 months ago

Committed revision 16171 on task branch 10560a:
- Replaced ctx.ttName with ctx.tt._name().

#30 Updated by Constantin Asofiei 9 months ago

Teodor: please rebase 10560a.

#31 Updated by Teodor Gorghe 9 months ago

Rebased task branch 10560a to trunk revision 16250.

#32 Updated by Constantin Asofiei 6 months ago

  • Status changed from Review to Internal Test

#33 Updated by Teodor Gorghe 6 months ago

Rebased task branch 10560a to trunk revision 16358.

#34 Updated by Constantin Asofiei 6 months ago

  • Status changed from Internal Test to Merge Pending

Merge to trunk after 6457a

#35 Updated by Teodor Gorghe 6 months ago

  • % Done changed from 90 to 100
  • Status changed from Merge Pending to Test

Branch 10560a was merged to trunk rev. 16384 and archived.

#36 Updated by Constantin Asofiei 6 months ago

  • Status changed from Test to Closed

Also available in: Atom PDF