FWD » Core Development » Base Language

In 4GL, it is possible to import an XML file into a DATASET handle without a pre-existing schema. The dataset structure is expected to be created dynamically based on the XML content.

PROCEDURE test:
    DEF VAR ds1 AS HANDLE NO-UNDO.
    CREATE DATASET ds1.
    ds1:READ-XML("file", "tmp.xml", "EMPTY", ?, ?).
END.

When executing a similar test in FWD, the XML import fails with the following error, indicating a problem with temp-table preparation:

╷
└─ FWD Test ✔
   └─ XmlToJson ✔
      └─ xmltoJsontest ✘ TEMP-TABLE object must have at least one field before TEMP-TABLE-PREPARE can be called. (9033)

The possible cause may be at: XmlImport.java:568:

if (isRoot)
{
   isRoot = false;
   dsName = name;
   nsUri = reader.getNamespaceURI();
   nsPrefix = reader.getPrefix();
   dataSetNameMatch = ds._name().equals(dsName) ||
                      ds.getSerializeName().getValue().equals(dsName) ||
                      ds.getXmlNodeName().getValue().equals(dsName);
}
...

Error 9033 may happen because there isn't a temporary table created for the root node. There's no schema to back this up and also, the logic above doesn't take that into consideration.

Further debugging reveals that even if the initial error is manually bypassed (e.g., by creating a temp-table), the XML parsing is flawed.

• Any attributes present on the root XML element are ignored.
• The hierarchical structure (nesting) of the XML is not correctly interpreted, leading to an incomplete or malformed dataset.

For this test:

def var ds1 as handle.
create dataset ds1.
ds1:READ-XML("file", "tmp.xml", "EMPTY", ?, ?).
ds1:write-xml("file", "out.xml").

message ds1:name ds1:xml-node-type.

For this XML:

<?xml version="1.0" encoding="UTF-8"?>
<ViewResourceMetadata IsDefaultHybridBrowse="true">
    <Test> hello </Test>
</ViewResourceMetadata>

<?xml version="1.0"?>
<ViewResourceMetadata xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" IsDefaultHybridBrowse="true">
  <Test> hello </Test>
</ViewResourceMetadata>

For this test:

<foo>
   <ViewResourceMetadata xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" IsDefaultHybridBrowse="true">
      <Test> hello </Test>
  </ViewResourceMetadata>
</foo>

<?xml version="1.0"?>
<foo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ViewResourceMetadata IsDefaultHybridBrowse="true">
    <Test> hello </Test>
  </ViewResourceMetadata>
</foo>

Conclusion: if the root node has attributes, the the XML has a single-table dataset (as you can't have multiple root nodes).

Please check other DATASET attributes related to XML and serialization.

For the following code:

DEFINE VARIABLE ds1 AS HANDLE NO-UNDO.
CREATE DATASET ds1.
ds1:READ-XML("file", "in.xml", "EMPTY", ?, ?).
ds1:WRITE-XML ("file", "out.xml").

The following XML:

<?xml version="1.0" encoding="UTF-8"?>
<ViewResourceMetadata IsDefaultHybridBrowse="true">
   <Test> hello </Test>
</ViewResourceMetadata>

fails with the following error message:
TEMP-TABLE object must have at least one field before TEMP-TABLE-PREPARE can be called. (9033) in latest trunk revision.

But I've found that the following XML:

<?xml version="1.0" encoding="UTF-8"?>
<ViewResourceMetadata IsDefaultHybridBrowse="true">
   <Test testatr="test"> hello </Test>
</ViewResourceMetadata>

The only change is that the Test node has an attribute in the passing test, but doesn't have any in the failing test.
I need to investigate this more as I surely didn't understand the underlying issue myself.

I've made some tests and looked over the existing code.

The used read-write FWD procedure:

DEFINE VARIABLE ds1 AS HANDLE NO-UNDO.
CREATE DATASET ds1.
ds1:READ-XML("file", "in.xml", "EMPTY", ?, ?).
ds1:WRITE-XML ("file", "out.xml").

message ds1:name ds1:xml-node-type.

• Issue 1:

The following XML undergoes a read-write procedure in FWD:

<?xml version="1.0" encoding="UTF-8"?>
<RootTest attr1="test">
   <Variable>test</Variable>
</RootTest>

Gets converted to:

<?xml version='1.0'?>
<RootTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <Variable/>
</RootTest>

Here the dataset's name is RootTest and XML node-type ELEMENT!

The problem is that the XML isn't wrapped in a NewDataSet hidden new root node.
I couldn't find anything about creating a NewDataSet wrapper hidden node.

In Importer.java:706, method setupDataset(String name), the dataset is set to NewDataSet if and only if there isn't another name provided.
But in XmlImport.java:529 a name is always provided - in XmlImport.java:569 the dataset's name is set to the START_ELEMENT's name.

For the following code (I've enforced a NewDataSet root with the same structure as above):

<?xml version="1.0" encoding="UTF-8"?>
<NewDataSet>
   <RootTest attr1="test">
      <Variable>test</Variable>
   </RootTest>
</NewDataSet>

We get:

<?xml version='1.0'?>
<NewDataSet xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <RootTest attr1="test">
      <Variable>
         test
      </Variable>
   </RootTest>
</NewDataSet>

We can see that no data is lost now. So we must fix the NewDataSet node that hasn't been added in the previous examples.

• Issue 2:

We can see, even with the NewDataSet node that fixes the TEMP-TABLE object must have at least one field before TEMP-TABLE-PREPARE can be called. (9033) error, we get an incomplete dataset.

Original XMLOriginal XML

<?xml version="1.0" encoding="UTF-8"?>
<NewDataSet>
<PlanetarySystemRegistry IsHabitable="true" IsCharted="true" HasWarning="false" SystemId="urn:planet:id:corp.xyz.KEPLER_186F" SystemNameCode="KEPLER_186F" PrimaryStarId="urn:star:corp.xyz.KEPLER_186" NavDataUri="urn:nav:system:corp.xyz.KEPLER_186F">
   <OrbitalBody>
      <PrimaryMoon MoonId="urn:moon:primary:corp.xyz.KEPLER_186F_A"/>
      <AtmosphereComposition>RANDOM DATA IN A DIFFERENT PLACE</AtmosphereComposition>
      <TectonicPlates>
         <Plate>
            <RegionName>NorthernCrater.Ridge</RegionName>
            <CoreSampleId>magmaSamples[0].olivine</CoreSampleId>
         </Plate>
      </TectonicPlates>
   </OrbitalBody>
   <StarData>
      <StellarClassification>urn:class:star:KEPLER_186</StellarClassification>
      <StarMetadata/>
      <SolarFlarePatterns>
         <FlarePattern/>
      </SolarFlarePatterns>
   </StarData>
   <GeologicalSurvey SurveyId="urn:survey:program:corp.xyz.KEPLER_186F:geo"/>
   <SurfaceTemperature>ANOTHER RANDOM VALUE IN A DIFFERENT PLACE</SurfaceTemperature>
</PlanetarySystemRegistry>
</NewDataSet>

Read and written XMLRead and written XML

<?xml version='1.0'?>
<NewDataSet
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <PlanetarySystemRegistry NavDataUri="urn:nav:system:corp.xyz.KEPLER_186F" PrimaryStarId="urn:star:corp.xyz.KEPLER_186" SystemNameCode="KEPLER_186F" SystemId="urn:planet:id:corp.xyz.KEPLER_186F" HasWarning="false" IsCharted="true" IsHabitable="true">
        <OrbitalBody>
            <PrimaryMoon/>
            <AtmosphereComposition>RANDOM DATA IN A DIFFERENT PLACE</AtmosphereComposition>
            <TectonicPlates>
                <Plate>
                    <RegionName>NorthernCrater.Ridge</RegionName>
                    <CoreSampleId>magmaSamples[0].olivine</CoreSampleId>
                </Plate>
            </TectonicPlates>
        </OrbitalBody>
        <StarData>
            <StellarClassification>urn:class:star:KEPLER_186</StellarClassification>
            <StarMetadata/>
            <SolarFlarePatterns>
                <FlarePattern/>
            </SolarFlarePatterns>
        </StarData>
    </PlanetarySystemRegistry>
</NewDataSet>

<GeologicalSurvey SurveyId="urn:survey:program:corp.xyz.KEPLER_186F:geo"/>
<SurfaceTemperature>ANOTHER RANDOM VALUE IN A DIFFERENT PLACE</SurfaceTemperature>

These two nodes are lost because persisted = true at XmlImport.java:3343.

if( --nest == -1)
{
   return persisted ? true : persist(maker, readValues, parent != null);
}

I'll be investigating more in-depth these issues.

Teodor, what else is needed here?

Yes, there is still more work to do, but it is not related with an issue reported by a customer.
I have worked for a fix on this, there are lot of changes of READ-XML when inferring the dataset, but I think I shall commit the changes as it is (some of it is a workaround).

Rebased to trunk rev 16358

10203a rev 16361 fixes copyright years. Otherwise, changes are OK. This was already tested via #10615.

Merge to trunk after 11048a