FWD » Core Development » Database

Need to run customer reports to generate exact list of methods and attributes required.

Based on a review of the recent reports, the following dynamic DB method is in use but is not yet implemented:

DELETE-RESULT-LIST-ENTRY()

Based on a review of the recent reports, the following dynamic DB attributes are in use but are not yet implemented:

BUFFER-DELETE
EMPTY-TEMP-TABLE

Final conversion testing passed. This is checked into bzr as 10200.

Greg Shah wrote:

Final conversion testing passed. This is checked into bzr as 10200.

This will fail runtime regression testing. You need to update p2j/pl/p2jpl.ddr to change the signatures there, too, so that PL/Java has the right method names for these UDFs.

Passed runtime regression testing and is checked into bzr as revision 10222.

The following Buffer methods require dynamic database or metadata support to be in place, but are not directly related to dynamic temp-table generation or dynamic query generation:

• bufferField (requires metadata)
• findFirst (requires dynamic database)
• findNext (requires dynamic database)
• findLast (requires dynamic database)
• findPrevious (requires dynamic database)
• findUnique (requires dynamic database)

Did some research regarding these methods, focusing on bufferField().

My first thought were to 'capture' the missing information needed for evaluation of the functions.
For the static tables (including temp-tables), the name / index of the field can be added to dmo_index.xml.
For dynamic defined tables the name/index will be managed at runtime. A common point for both kind of fields can be some kind of map, but at this moment I am not sure where this metadata should be stored.

All metadata from the _field, _file, _index, _index_field, _user, and _lock tables are being stored in an embedded H2 database and will be available through an API to the persistence runtime in a class called MetaSchema. There will be one instance of this class for each permanent database in an application. I am working on this task, but it is not ready yet. Much of the DMOIndex functionality will be moved into this API eventually (though not initially), so I do not want to expand use of the DMOIndex class and dmo_index.xml file at this time.

We can ignore the last entry about MetaSchema. This approach has been replaced by the use of annotations, which should be enhanced/added where necessary.

For M7, we need to add support for the BUFFER-field attributes in use. At least DATA-TYPE is needed. The attributes which are linked to clauses at the field definition in the schema file or at the define temp-table statement should emit as annotations at the DMO impl class, for both permanent and temporary DMOs.

The following need implementation in BufferImpl:

// has TODO regarding generating a properly converted sort clause (currently null); otherwise
// implemented
public logical findByRowID(rowid id, LockType lockType)

// all no-whereClause find* methods are implemented; those with where clauses (i.e., requiring
// dynamic database support) need to be implemented
public logical findFirst(character whereClause)
public logical findFirst(character whereClause, LockType lockType)
public logical findFirst(character whereClause, long lock)
public logical findFirst(character whereClause, NumberType lock)
public logical findFirst(character whereClause, long lock, long nowait)
public logical findFirst(character whereClause, NumberType lock, long nowait)
public logical findFirst(character whereClause, long lock, NumberType nowait)
public logical findFirst(character whereClause, NumberType lock, NumberType nowait)
public logical findLast(character whereClause)
public logical findLast(character whereClause, LockType lockType)
public logical findLast(character whereClause, long lock)
public logical findLast(character whereClause, NumberType lock)
public logical findLast(character whereClause, long lock, long nowait)
public logical findLast(character whereClause, NumberType lock, long nowait)
public logical findLast(character whereClause, long lock, NumberType nowait)
public logical findLast(character whereClause, NumberType lock, NumberType nowait)
public logical findUnique(character whereClause)
public logical findUnique(character whereClause, LockType lockType)
public logical findUnique(character whereClause, long lock)
public logical findUnique(character whereClause, NumberType lock)
public logical findUnique(character whereClause, long lock, long nowait)
public logical findUnique(character whereClause, NumberType lock, long nowait)
public logical findUnique(character whereClause, long lock, NumberType nowait)
public logical findUnique(character whereClause, NumberType lock, NumberType nowait)

public integer numFields()
public integer currentIteration()          [LE: currentIteration() not needed for M7! Should return unknown value]
public logical isError()
public void setError(boolean value)
public void setError(logical value)
public character getDbName()
public character getTable()
public character getNamespaceURI()
public void setNamespaceURI(String uri)
public void setNamespaceURI(character uri)

// return handle to specified buffer field; HIGH priority
public handle bufferField(NumberType fieldNumber)
public handle bufferField(character fieldName)
public handle bufferField(int fieldNumber)

// should return value of buffer field with given name (converted form of 4GL :: notation)
public BaseDataType dereference(String memberName)

The following need implementation in FieldReference:

// some of these may be able to use existing data in FieldReference, others will require new
// annotations; still others could benefit from a link back to the containing buffer
public integer extent()
public character initial()
public character stringValue()
public character stringValue(int index)
public character stringValue(NumberType index)
public character columnLabel()
public void changeColumnLabel(character columnLabel)
public character bufferName()
public logical literalQuestion()
public void changeLiteralQuestion(logical literalQuestion)
public character validateMessage()
public void changeValidateMessage(character validateMessage)
public logical mandatory()
public character validateExpression()
public void changeValidateExpression(character validateExpression)
public integer position()
public character getDbName()
public character getTable()
public decimal getWidthChars()
public void setWidthChars(NumberType widthChars)
public void setWidthChars(double widthChars)
public character getColumnLabel()
public void setColumnLabel(character columnLabel)
public void setColumnLabel(String columnLabel)
public character getDataType()
public void setDataType(String dataType)      [LE: cannot set data type for a buffer field]
public void setDataType(character dataType)   [LE: cannot set data type for a buffer field]
public character getLabel()
public void setLabel(String label)
public void setLabel(character label)

// there seems to be some redundancy between the first 2 of the following methods and the
// remaining 3
public character format()
public void changeFormat(character format)
public character getFormat()
public void setFormat(String format)
public void setFormat(character format)

// casts newValue to BaseDataType; are non-BDT values possible?
public void changeValue(Object newValue)

// always returns true; in what circumstance would it return false?
public boolean valid()

// not sure why this is implemented for FieldReference; DELETE cannot be called on a
// BUFFER-FIELD handle; current implementation returns false, which is probably correct
protected boolean resourceDelete()

The following need implementation in QueryWrapper:

public character indexInformation()
public character indexInformation(int n)
public character indexInformation(NumberType n)
public logical createResultListEntry()
public logical deleteResultListEntry()

I realize the previous 3 entries are not necessarily all about dynamic attributes and methods, but I needed to summarize all the remaining, unimplemented stubs in one place, so we can attack them.

Ovidiu: I would like you to work on the BufferImpl pieces, except public handle bufferField(character fieldName).

Vadim: I would like you to work on the FieldReference pieces and public handle bufferField(character fieldName), which you will need to do the FieldReference work. Please post the BufferImpl.bufferField update here as soon as it is working, so Ovidiu can merge early.

Constantin: you left TODOs in the QueryWrapper methods listed above in note 20. Did you determine these were not necessary for M7?

Implementation note: where new Java annotations are needed on DMOs to implement some of the above attributes, please minimize their use. For instance, only add an annotation for the MANDATORY attribute if the field is indeed declared as MANDATORY in the schema. Similarly, if there is no validation expression for a field, there should be no corresponding annotation.

The lack of an annotation should determine the default condition of the attribute, thus minimizing annotation clutter in these classes.

Eric Faulhaber wrote:

Constantin: you left TODOs in the QueryWrapper methods listed above in note 20. Did you determine these were not necessary for M7?

Correct, these are not needed for M7.

The question about BufferImpl.bufferField(int fieldNumber) implementation.

I am going to reuse bufferField(String fieldName), but it is not clear how to find field name by field number.

We can add fieldNumber to @LegacyField annotation

@LegacyField(name = "field1", fieldNumber = "5")

with corresponding changes in TableMapper, but it will work only for permanent tables.

Just noticed that I needed to start from bufferField(character fieldName). It will reuse bufferField(String fieldName) of course:

   public handle bufferField(character fieldName)
   {
      if (fieldName.isUnknown())
      {
         final String errMsg1 =
            "Argument for BUFFER-FIELD method for buffer %s could not be evaluated.";
         ErrorManager.recordOrShowError(
            7349, String.format(errMsg1, name()), false, false, false);
         final String errMsg2 =
            "Lead attributes in a chained-attribute expression (a:b:c) must be type " +
            "HANDLE or a user-defined type and valid (not UNKNOWN)";
         ErrorManager.recordOrShowError(10068, errMsg2, false, false, false);
         return new handle();
      }
      else
      {
         return bufferField(fieldName.getValue());
      }
   }

Vadim: the bufferField(character) implementation looks good to me.

You don't need to implement bufferField(int) or bufferField(NumberType) right now. I just wanted you to be able to get a handle to a FieldReference from a Buffer, so you can write and convert BUFFER-FIELD test cases, in order to implement and test the methods in note 19.

Please note that all BUFFER-FIELD methods and attributes need to work for both permanent and temporary tables (unless the corresponding 4GL implementation behaves otherwise).

Once you are sufficiently comfortable that your bufferField(character) implementation works properly with your test cases, please go ahead and commit it to bzr without regression testing. Since it has no conversion impact, is not used by the regression testing environment, and does not affect existing runtime behavior, there is no point in spending the time regression testing this change.

Note 25 corrected, added one more error message after comparing with 4GL behavior.

The following also need implementation in BufferImpl:

/**
  * Obtain the handle to the buffer that was created by default for this temporary table. Every
  * dynamic temp-table is created with at least one buffer. This buffer's object handle is
  * returned by this method.
  */
public handle defaultBufferHandle()

Vadim wrote (in email):

The question: 'initial' was added in *.p2o as Ast node in contrast to other attributes added as annotations. I understood it was done for specifying its value type.

But INITIAL attribute should return character value based on formatting:

//The value of the INITIAL schema field (which is always CHARACTER), formatted with
//the buffer-field’s format. If the INITIAL schema field has the Unknown value (?), the
//value of the INITIAL attribute is the null string (“”).

Are there any method in P2J implementing such formatting?

See character.toString(String) [LE: BaseDataType.toString(String)] and its related methods. This should give you most of what you want in terms of formatting, except that these methods will return "?" for unknown value.

It seems TableMapper needs refactoring if we plan to add some new annotations to LegacyField like now I added format(). I mean creation of common methods like

public static String getLegacyFieldAnnotation(Buffer buffer, String property, String annotation)
...

inatead of

getLegacyFieldName
getLegacyFieldFormat
getLegacyFieldInitial
...

While I do see some value in not repeating the pattern

if (buf.isTemporary())
   ...
else 
   ...

Yes, agreed. I made internal methods returning LegacyFieldInfo, bit public methods are the same.

I think for INITIAL attribute we need two string annotations: initialValue and initialType, assigning them from text/type respectively:

<ast col="0" id="34359738474" line="0" text="0" type="NUM_LITERAL"/>

or

<ast col="0" id="42949672966" line="0" text="qwerty" type="STRING"/>

initial() attribute should return

baseDataType.toString(format()).

But I did not find so far methods allowing creation BaseDataType instance from a pair of strings initialValue and initialType:

BaseDataType baseDataType = BaseDataType.getValue("0", "NUM_LITERAL")

or

BaseDataType baseDataType = BaseDataType.getValue("qwerty", "STRING")

Should I use BaseDataType.generateUnknown(Class cls) or BaseDataType.generateDefault(Class cls)

for corresponding class and assign string value to resulting baseDataType instance? How to find such class?

But I did not find so far methods allowing creation BaseDataType instance from a pair of strings initialValue and initialType

AFAIK, there is no such method. Why do you need to store initialType as an annotation at all? You already know (i.e., can determine through reflection) the data type of any DMO property. I'm pretty sure every subclass of BaseDataType has a constructor which takes a String argument. You can invoke it using java.lang.reflect.Constructor, passing in the intialValue String as the argument. From there you can get the formatted string as discussed previously.

Vadim, in the version of BufferImpl you just checked in (rev 10433), you removed the original copyright year of 2013 and replaced it with 2014. We want to preserve the initial copyright year. So, when first editing a source file in a new year, please leave the original copyright year and add the new one. For example:

Copyright (c) 2013-2014, Golden Code Development Corporation.

Eric Faulhaber wrote:

Vadim, in the version of BufferImpl you just checked in (rev 10433), you removed the original copyright year of 2013 and replaced it with 2014. We want to preserve the initial copyright year. So, when first editing a source file in a new year, please leave the original copyright year and add the new one. For example:
[...]
Whoever edits BufferImpl next, please update the copyright notice accordingly.

OK. Sorry, it was automatically.

Eric Faulhaber wrote:

But I did not find so far methods allowing creation BaseDataType instance from a pair of strings initialValue and initialType

AFAIK, there is no such method. Why do you need to store initialType as an annotation at all? You already know (i.e., can determine through reflection) the data type of any DMO property. I'm pretty sure every subclass of BaseDataType has a constructor which takes a String argument. You can invoke it using java.lang.reflect.Constructor, passing in the intialValue String as the argument. From there you can get the formatted string as discussed previously.

I thought we need to store the difference between property type and initial value type. For example Progress allows both numeric and string based initialization for integer:

define temp-table t
     field f1 as integer initial 16  format "999" 
     field f2 as integer initial "16"  format "999".

But after parsing, in *.p2o I do not see any difference:

<ast col="0" id="42949672972" line="0" text="16" type="NUM_LITERAL"/>

for both fields. So, as you wrote I will annotate only with using this parsed string value and use property class for receive BaseDataType value for toString(format).

I'm pretty sure the double quotes around the numeric value represent optional syntax. It is the value itself that is important, and our parser drops the double quotes (or we remove them in later TRPL processing -- I don't remember which).

• findFirst, findLast, findUnique - implemented, testing in progress, to add error messages if the case
• numFields - implemented & tested
• currentIteration - for the moment just returns unknown value
• isError & setError s - locally implemented, investigating dependency to BEFORE-TABLE clause. At least this is the message I get when I try to alter the attribute (Cannot set ERROR if there is no BEFORE-TABLE for ...). On the other hand, we do not support at all BEFORE-TABLE clause nor the before-buffer and after-buffer attributes.
• getDbName - implemented & tested
• getTable - implemented & tested
• getNamespaceURI & setNamespaceURI s - implemented locally in BufferImpl. Looks like the we also do not support this clause at conversion time. I searched this clause/attribute in the customer's server project files and dis not found any evidence of their usage.
• dereference - implemented and tested. the conversion gives some warnings when running on some pieces of code with this and in some case, it fails or generated code is not compilable. This happens because of the POLY aspect of the feature and of the bufferFiled:value. The errors can be avoided by always casting the displayed result to char, for example, using the string function.
• findByRowID - in progress, investigating behavior in Progress
• bufferField(int/numeric) - in progress

PS: I will fix the (c) year mentioned in note 38.

LITERAL-QUESTION attribute

Needed to support -literalquestion startup parameter and special processing for quoted character values

When TRUE, the AVM treats a quoted character value as a literal character value. That is, it does not remove enclosing quotes, trailing blanks, or formatting insertion characters. 
When FALSE, the default value, the AVM treats a quoted character value as a non-literal character value. That is, it removes enclosing quotes, trailing blanks, and formatting insertion characters. For example:
The AVM treats "abc   " as abc.
The AVM treats a quoted question mark character ("?") as the Unknown value (?).

Do P2J already has such option for different quoted character values processing and runtime options support?

Do P2J already has such option for different quoted character values processing and runtime options support?

No.

But the description does not provide enough information to understand what this option really does.

1. Does it affect preprocessing?
2. Does it affect parsing?
3. Does it affect character type processing in regular 4GL code (e.g. expressions).
4. Does it affect character type processing in 4GL code that is related to database processing?
5. Does it affect character type processing in the database itself?
6. Does it affect character type comparisons or are there actual data differences?
7. What does it mean by its comment about the unknown value? As far as I know this is not how the 4GL works. For example:

def var txt as char init "?".
message "'" + txt + "'; unknown = " + string(txt eq ?).

will display '?'; unknown = no.

We need to understand much more clearly what it is that is really needed here. And we may want to limit our efforts to providing minimum support to allow the current code to work, if full support is any significant effort.

MANDATORY attribute

Mandatory fields — If checked, mandatory fields are included in the RowObjectValidate validation procedure.

Whether there is support of RowObjectValidate in P2J?

AFAIK, RowObjectValidate is just an ADM SmartObjects procedure that we convert. Does this have any bearing on the implementation of the MANDATORY attribute itself? Where did you find this statement?

Vadim, current status (implemented methods are marked in bold)¶

BufferImpl¶

public handle bufferField(character fieldName).

FieldReference¶

public integer extent()
public character initial()
public character stringValue()
public character stringValue(int index)
public character stringValue(NumberType index)
public character columnLabel()
public void changeColumnLabel(character columnLabel)
public character bufferName()
public logical literalQuestion()
public void changeLiteralQuestion(logical literalQuestion)
public character validateMessage()
public void changeValidateMessage(character validateMessage)
public logical mandatory()
public character validateExpression()
public void changeValidateExpression(character validateExpression)
public integer position()
public character getDbName()
public character getTable()
public decimal getWidthChars()
public void setWidthChars(NumberType widthChars)
public void setWidthChars(double widthChars)
public character getColumnLabel()
public void setColumnLabel(character columnLabel)
public void setColumnLabel(String columnLabel)
public character getDataType()
public void setDataType(String dataType) [LE: cannot set data type for a buffer field]
public void setDataType(character dataType) [LE: cannot set data type for a buffer field]
public character getLabel()
public void setLabel(String label)
public void setLabel(character label)

// there seems to be some redundancy between the first 2 of the following methods and the
// remaining 3
public character format() REMOVED
public void changeFormat(character format) REMOVED
public character getFormat()
public void setFormat(String format)
public void setFormat(character format)
// casts newValue to BaseDataType; are non-BDT values possible?
public void changeValue(Object newValue)

// always returns true; in what circumstance would it return false?
public boolean valid()

// not sure why this is implemented for FieldReference; DELETE cannot be called on a
// BUFFER-FIELD handle; current implementation returns false, which is probably correct
protected boolean resourceDelete()

Vadim, to me it doesn't look like format and changeFormat are actually in use. I don't see any rules to generate these methods in annotations/methods_attributes.rules, but I do see getFormat and setFormat (which are part of the CommonField interface) generated from KW_FORMAT. In fact, I searched the project for the text "changeFormat" and the only place it appears is in FieldReference.

Vadim Nebogatov wrote:

public void setDataType(String dataType) [LE: cannot set data type for a buffer field]
public void setDataType(character dataType) [LE: cannot set data type for a buffer field]

Is there any error raised/shown? What happens when you use something like bf:data-type = "bogus", bf being a buffer-field handle?

// there seems to be some redundancy between the first 2 of the following methods and the
// remaining 3
public character format()
public void changeFormat(character format)
public character getFormat()
public void setFormat(String format)
public void setFormat(character format)

As Eric noted, format and changeFormat should be removed.

// casts newValue to BaseDataType; are non-BDT values possible?
public void changeValue(Object newValue)

Yes, I think it should be possible (i.e. literals could be used). Depending on the buffer-field's type, you will have to create a new BDT instance (with the new value) and use FieldReference.set to change it.

// always returns true; in what circumstance would it return false?
public boolean valid()

I think this should call BufferImpl.valid - as they should be linked with the parent buffer.

// not sure why this is implemented for FieldReference; DELETE cannot be called on a
// BUFFER-FIELD handle; current implementation returns false, which is probably correct
protected boolean resourceDelete()

I don't understand this. If a buffer-field is saved in a handle var, calling DELETE on that var, what will happen? Will there be an error?

Constantin Asofiei wrote:

// not sure why this is implemented for FieldReference; DELETE cannot be called on a
// BUFFER-FIELD handle; current implementation returns false, which is probably correct
protected boolean resourceDelete()

I don't understand this. If a buffer-field is saved in a handle var, calling DELETE on that var, what will happen? Will there be an error?

Sorry, this was my comment and I see how it is misleading. It was not based on testing, but on the 4GL documentation for the DELETE() method, which states:

Applies to: BROWSE widget (column), COMBO-BOX widget, RADIO-SET widget, SELECTION-LIST widget

Constantin Asofiei wrote:

Vadim Nebogatov wrote:

public void setDataType(String dataType) [LE: cannot set data type for a buffer field]
public void setDataType(character dataType) [LE: cannot set data type for a buffer field]

Is there any error raised/shown? What happens when you use something like bf:data-type = "bogus", bf being a buffer-field handle?

Sorry, again this is my comment based on the 4GL documentation for the DATA-TYPE attribute:

This attribute is writeable for combo-boxes (only before realization), fill-ins, and text widgets only.

Eric Faulhaber wrote:

AFAIK, RowObjectValidate is just an ADM SmartObjects procedure that we convert. Does this have any bearing on the implementation of the MANDATORY attribute itself? Where did you find this statement?

I found this statement in dybas.pdf (Progress Dynamics Basic Development, Progress ABL v11).
No, it does not stop implementation of the MANDATORY attribute.

The same for the LITERAL-QUESTION attribute, I have implemented attribute support without proper analysys according note 44.

Eric Faulhaber wrote:

Vadim, to me it doesn't look like format and changeFormat are actually in use. I don't see any rules to generate these methods in annotations/methods_attributes.rules, but I do see getFormat and setFormat (which are part of the CommonField interface) generated from KW_FORMAT. In fact, I searched the project for the text "changeFormat" and the only place it appears is in FieldReference.

format and changeFormat removed and note 47 updated

Ovidiu working on BufferImpl.java

// has TODO regarding generating a properly converted sort clause (currently null); otherwise implemented
public logical findByRowID(rowid id, LockType lockType)
Don't think that sorting need to be enforced here (and Constantin agreed with) because the rowid s are unique so there is nothing to sort. The only logical thing would be an index on the rowid that would help quickly find the needed record.

// all no-whereClause find* methods are implemented; those with where clauses (i.e., requiring dynamic database support) need to be implemented
public logical findFirst(character whereClause)
public logical findFirst(character whereClause, LockType lockType)
public logical findFirst(character whereClause, long lock)
public logical findFirst(character whereClause, NumberType lock)
public logical findFirst(character whereClause, long lock, long nowait)
public logical findFirst(character whereClause, NumberType lock, long nowait)
public logical findFirst(character whereClause, long lock, NumberType nowait)
public logical findFirst(character whereClause, NumberType lock, NumberType nowait)
public logical findLast(character whereClause)
public logical findLast(character whereClause, LockType lockType)
public logical findLast(character whereClause, long lock)
public logical findLast(character whereClause, NumberType lock)
public logical findLast(character whereClause, long lock, long nowait)
public logical findLast(character whereClause, NumberType lock, long nowait)
public logical findLast(character whereClause, long lock, NumberType nowait)
public logical findLast(character whereClause, NumberType lock, NumberType nowait)
public logical findUnique(character whereClause)
public logical findUnique(character whereClause, LockType lockType)
public logical findUnique(character whereClause, long lock)
public logical findUnique(character whereClause, NumberType lock)
public logical findUnique(character whereClause, long lock, long nowait)
public logical findUnique(character whereClause, NumberType lock, long nowait)
public logical findUnique(character whereClause, long lock, NumberType nowait)
public logical findUnique(character whereClause, NumberType lock, NumberType nowait)
All the other overloaded forms will are using a predicate, not a where clause. The predicate is a little more complex, here is the syntax:

[ WHERE [ logical-expression ] ] [ USE-INDEX index-name ]

public integer numFields() - implemented
public integer currentIteration() - not needed for M7! For the moment return unknown value
public logical isError() - implemented
public void setError(boolean value) - implemented
public void setError(logical value) - implemented
public character getDbName() - implemented
public character getTable() - implemented
public character getNamespaceURI() - implemented
public void setNamespaceURI(String uri) - implemented
public void setNamespaceURI(character uri) - implemented

// return handle to specified buffer field; HIGH priority
public handle bufferField(NumberType fieldNumber) - implemented
public handle bufferField(character fieldName) - implemented by Vadim
public handle bufferField(int fieldNumber) - implemented

// should return value of buffer field with given name (converted form of 4GL :: notation)
public BaseDataType dereference(String memberName) - implemented

Eric Faulhaber wrote:

Sorry, this was my comment and I see how it is misleading. It was not based on testing, but on the 4GL documentation for the DELETE() method, which states:

OK, note that a DELETE() method call is different from a DELETE OBJECT statement. Also:
- the Buffer-field resource doesn't support DELETE(), so that doesn't apply to us.
- resourceDelete is executed only when DELETE OBJECT or DELETE WIDGET is called on that buffer-field; and this is what we need to test.

Some more details on implementation of bufferField(int):

I have done some researches that clarified me that the BUFFER-FIELD attribute uses the ORDER clause to index them in the sequence of the buffer. Fortunately, much of the code for sorting the fields was already in place, I only had to annotate the fields so they could be index-find at runtime. Currently, the dmo properties look something like:

   @LegacyField(name = "book-id", fieldId = 1)
   private final integer bookId;

• DataModelWorker.java - added fieldId annotation (for both simple and extent fields)
• TableMapper.java - added:
  public static String getPropertyName(TempTable table, int fieldId)
public static String getPropertyName(String schema, String table, int fieldId)
protected String getPropertyName(K legacyKey, int fieldId)
  and in private static class LegacyTableInfo, an Integer to LegacyIndexInfo map that is used by above methods.
• LegacyField interface -
  added int fieldId();
• DynamicTablesHelper.java added:
  public static String getHibernateFieldName(Class dmoIface, int fieldId)
• dmo_comp.rules, dmo_common.rules, schema/java_templates.tpl and dmo_props.rules
  to define the template for annotation and set it into generated dmo implementation classes.

Ovidiu, please check if you can extract the code which adds the new annotations into a separate update; I think is worth releasing this on its own, as other work relies on it.

Agreed. Consider whether runtime regression testing is necessary for this update. If its impact is only on new runtime features and there are no changes to existing runtime classes (e.g., RecordBuffer, etc.), or there are only additive changes to existing runtime classes that won't impact old code paths, then you can do only conversion regression testing before checking it in.

I also added some more optional annotations like

/** Data member */
   @LegacyField( name = "fc" , format = "qwertyu2" , initial = "qwertyu1" , columnLabel = "qwertyu3")
   private final character fc;

/** Data member */
   @LegacyField( name = "fi" , format = "99999" , initial = "0")
   private final integer fi;

So, we will need to merge our changes.

POSITION attribute

The position of a buffer-field within the database record.

From tests, it is just field order (fieldId) plus 1. It is interesting it always starts from 2 both for temporary and permanent tables.

Two cosmetics found in HEIGHT-CHARS implementation: HEIGHT-CHARS instead of HEIGHT and setable with one t

Now

**HEIGHT-CHARS is not a settable attribute for BUFFER-FIELD widget. (4052)

Should be

**HEIGHT is not a setable attribute for BUFFER-FIELD widget. (4052)

Fix them?

VALIDATE-EXPRESSION attribute

I missed annotation of this attribute. In *.dict it looks like

        <ast col="3" id="8589935371" line="500" text="VALEXP" type="VALEXP">
          <ast col="6" id="8589935794" line="1" text="&gt;" type="GT">
            <ast col="1" id="8589935795" line="1" text="cost" type="FIELD_DEC">
              <annotation datatype="java.lang.Long" key="oldtype" value="2348"/>
              <annotation datatype="java.lang.String" key="schemaname" value="p2j_test.book.cost"/>
              <annotation datatype="java.lang.String" key="bufname" value="p2j_test.book"/>
              <annotation datatype="java.lang.String" key="dbname" value="p2j_test"/>
              <annotation datatype="java.lang.Long" key="recordtype" value="12"/>
              <annotation datatype="java.lang.String" key="name" value="cost"/>
              <annotation datatype="java.lang.Long" key="type" value="370"/>
              <annotation datatype="java.lang.String" key="format" value="&quot;-&gt;&gt;,&gt;&gt;9.99&quot;"/>
              <annotation datatype="java.util.ArrayList" key="label">
                <listitem datatype="java.lang.String" value="&quot;Cost&quot;"/>
              </annotation>
              <annotation datatype="java.lang.String" key="columnlabel" value="&quot;Cost&quot;"/>
            </ast>
            <ast col="8" id="8589935796" line="1" text="10.0" type="DEC_LITERAL"/>
          </ast>
        </ast>

Do P2J has common (serialize) method allowing to convert such expression back to string?

Can you post a small testcase showing how this behaves? If it works how I think it works (i.e. VALIDATE-EXPRESSION can be ANY 4GL-style expression and gets accessed by i.e. UPDATE statements), than we have a problem: the VAL-EXP (for a field in a perm table) is not converted at the DMO; instead, they get converted where they are used (i.e. when a field has a val-exp, converting an UPDATE field will emit the val-exp too). My feeling is that we have yet another piece of 4GL code to dynamically convert at runtime...

No special test is required: it is from p2j_test.dict for p2j_test.book.cost field.

Ah, I get it know, you want to get the legacy 4GL expression. The best way is to save it in an annotation at the field, during the schema conversion step. And later on emit it as an annotation at the DMO property.

What I was talking about is: can the validate-expression be changed at runtime (does a h:validate-expression="something" work for a buffer field)? If yes, is the new value seen by an UPDATE statement?

Constantin Asofiei wrote:

Ah, I get it know, you want to get the legacy 4GL expression. The best way is to save it in an annotation at the field, during the schema conversion step. And later on emit it as an annotation at the DMO property.

Schema conversion step starts from *.dict file received as result of parsing. Am I correct? I need to serialize whole VALEXP ast node (shown above) back to string just to use in annotation.

What I was talking about is: can the validate-expression be changed at runtime (does a h:validate-expression="something" work for a buffer field)? If yes, is the new value seen by an UPDATE statement?

I just implemented such setter for buffer field. But it is not used so far in UPDATE statement.

Vadim Nebogatov wrote:

POSITION attribute
From tests, it is just field order (fieldId) plus 1. It is interesting it always starts from 2 both for temporary and permanent tables.

My theory is that position 1 is reserved for the implicit/internal recid field.

Vadim Nebogatov wrote:

Two cosmetics found in HEIGHT-CHARS implementation: HEIGHT-CHARS instead of HEIGHT and setable with one t
Fix them?

OK.

Vadim Nebogatov wrote:

Schema conversion step starts from *.dict file received as result of parsing. Am I correct? I need to serialize whole VALEXP ast node (shown above) back to string just to use in annotation.

Sorry, I meant schema parsing. See schema.g valexp rule - the expression is saved as a string, and later on parsed (in SchemaLoader.postProcessImport).

Eric Faulhaber wrote:

Vadim Nebogatov wrote:

Two cosmetics found in HEIGHT-CHARS implementation: HEIGHT-CHARS instead of HEIGHT and setable with one t
Fix them?

OK.

Actually, better postpone this. We've encountered same problem for width-chars and width at another task (#2016).

LE: just took a look at the implementation and the attribute name is hard-coded, not determined from annotations - so go ahead and implement them. Sorry for the confusion.

The VAL-EXP attribute actually is used in both read and write modes in the current project. In the function where it is written to, the comments seem to confirm your finding that the text is simply saved in the attribute, but changing it does not change the actual validation, which is irrevocably set at compile time.

See if the following version of the valExp rule in schema.g works:

valExp
   :
      k:KW_VALEXP^ s:STRING
      {
         String text = removeBogusQuotes(#s.getText());
         text = text.replaceAll("\"\"", "\"");
         #s.setText(text);
         #k.setType(VALEXP);
         #k.putAnnotation("original_text", text);
      }
   ;

I think this will add an annotation named "original_text" to the VALEXP node, with the pre-parsed validation expression. You may have to play with this a bit (like storing the annotation before we mess with the quotes) to get the text to match exactly what the VAL-EXP attribute reports at runtime. I'll leave this to you to test.

Eric Faulhaber wrote:

The VAL-EXP attribute actually is used in both read and write modes in the current project. In the function where it is written to, the comments seem to confirm your finding that the text is simply saved in the attribute, but changing it does not change the actual validation, which is irrevocably set at compile time.

See if the following version of the valExp rule in schema.g works:
[...]

I think this will add an annotation named "original_text" to the VALEXP node, with the pre-parsed validation expression. You may have to play with this a bit (like storing the annotation before we mess with the quotes) to get the text to match exactly what the VAL-EXP attribute reports at runtime. I'll leave this to you to test.

It works nice.

Ovidiu Maxiniuc wrote:

// has TODO regarding generating a properly converted sort clause (currently null); otherwise implemented
public logical findByRowID(rowid id, LockType lockType)
Don't think that sorting need to be enforced here (and Constantin agreed with) because the rowid s are unique so there is nothing to sort. The only logical thing would be an index on the rowid that would help quickly find the needed record.

Every table has a database-level index on the primary key id, which is what we use as recid/rowid for that table. This is a new index (i.e., not carried over from an existing 4GL index), because the id column is a surrogate primary key added by conversion.

The reason I made this comment about the sort clause is because for every RandomAccessQuery (the parent of FindQuery) instance associated with a non-temp-table, we analyze the sort clause to determine the original index which would have been used by the 4GL for that query (see line 679 in RandomAccessQuery). This is for the purpose of dirty share checking later on in RandomAccessQuery.executeImpl. Without debugging through this code, I can't remember right now if we would actually invoke the dirty share code which would use the RandomAccessQuery.index field at line 2897 in that class for this type of FIND query. You should convert and run a test case which does a static FIND by recid/rowid to find out if having this index be null will cause a problem. I know DefaultDirtyShareManager locks on that index and will trigger an NPE if invoked, but the question is: does dirty share checking come into play at all in this case, or do we bypass it for a FIND by recid/rowid query?

Ovidiu Maxiniuc wrote:

I have attached the intermediary update that contains some my changes for this issue.
I am putting this update to conversion test right now. I expect only the dmo field annotation to be changed at conversion time.
The runtime should not be affected in majic as it does not use the new features.

The conversion test finished as expected. I have checked some dmos and the the fieldId annotation looks fine (the fieldId will not occur in their natural order in src; this is fine because because the extent fields are grouped in Composite static classes).

Vadim, since my update only contains a changes for a single annotation I think, it's easier to integrate into your code. However, the update is just an intermediate version for other methods (I rolled those back to a compilable form).
I also added the NAME R/O attribute to FieldInterface/FieldReference, which I needed in my tests.
I wait for Eric to had the opportunity to review it before releasing it into bzr, but I think you can merge the changes if this is OK for you.

Eric Faulhaber wrote:

The reason I made this comment about the sort clause is because for every RandomAccessQuery (the parent of FindQuery) instance associated with a non-temp-table, we analyze the sort clause to determine the original index which would have been used by the 4GL for that query (see line 679 in RandomAccessQuery). This is for the purpose of dirty share checking later on in RandomAccessQuery.executeImpl. Without debugging through this code, I can't remember right now if we would actually invoke the dirty share code which would use the RandomAccessQuery.index field at line 2897 in that class for this type of FIND query. You should convert and run a test case which does a static FIND by recid/rowid to find out if having this index be null will cause a problem. I know DefaultDirtyShareManager locks on that index and will trigger an NPE if invoked, but the question is: does dirty share checking come into play at all in this case, or do we bypass it for a FIND by recid/rowid query?

Indeed, the RecordIdentifier c'tor requires that both parameters to be not null or NPEs are thrown. This happens when the buffer is dirty, no matter if he 'static' FIND by ROWID is used or the method of the handle. As consequence, the dirty share manager must do its job so the sorting hql has to be dynamically constructed in the BufferImpl.

Vadim, in last Thursday's email status report, you wrote:

I also noticed that FieldReference is not proper place for keep field attributes especially as for changeable (write) attributes because FieldReference is recreated constantly. So, I refactored and now all attributes are stored and updated in TableMapper, namely in LegacyFieldInfo.

Please put information like this into Redmine, since it will be seen by other developers familiar with the code, and they may have comments.

This seems OK to me. Stas, Constantin: any concerns with this approach?

Code review om_upd20140117a:

Code seems OK to me, though given your changes to FieldReference, you should coordinate with Vadim on the use of TableMapper to store legacy information instead (see note 79).

This seems OK to me. Stas, Constantin: any concerns with this approach?

No concerns. Attributes should be stored in TableMapper.

Stanislav Lomany wrote:

This seems OK to me. Stas, Constantin: any concerns with this approach?

Attributes should be stored in TableMapper.

Correct.

I think fieldId should be also added to LegacyFieldInfo along with other attributes. It will allow to find fieldId by property name in order to implement position() and possible for some other methods.

Code review 20140121a:

• In schema/fixups.xml, the rule which tests for type == prog.kw_mand is not correct. This keyword will not appear in *.dict files (it represents the MANDATORY handle attribute in 4GL business logic). The schema keyword used in *.dict files is prog.mandatory, and there already is a rule that converts this to an AST annotation.
• In email on January 10, I thought we had come to the conclusion that BufferImpl.defaultBufferHandle did not need to implemented after all, because it was implemented in TempTableBuilder and would not be reached in BufferImpl. Did you discover otherwise?
• Is the dataType annotation actually necessary? Can't we determine the appropriate, starting value for the DATA-TYPE attribute from the actual data type of the DMO property?

After these issues are addressed/answered and the remaining items are implemented, please run a conversion-only regression test.

Eric Faulhaber wrote:

• In schema/fixups.xml, the rule which tests for type == prog.kw_mand is not correct. This keyword will not appear in *.dict files (it represents the MANDATORY handle attribute in 4GL business logic). The schema keyword used in *.dict files is prog.mandatory, and there already is a rule that converts this to an AST annotation.

Removed, it was actually not used. MANDATORY support is based on existing "not-null" annotation. I noticed some later that such annotation just means mandatory and forgot to remove in fixups.xml

• In email on January 10, I thought we had come to the conclusion that BufferImpl.defaultBufferHandle did not need to implemented after all, because it was implemented in TempTableBuilder and would not be reached in BufferImpl. Did you discover otherwise?

I understood it should be implemented nevertheless... OK, I removed these changes.

• Is the dataType annotation actually necessary? Can't we determine the appropriate, starting value for the DATA-TYPE attribute from the actual data type of the DMO property?

I removed this annotation and used data type from DMO property. I added this annotation only because it is already presented in *.p2o.

Code review om_upd20140122a:

Looks good, but I also would like Constantin to review the dynamic query generation parts based on his work in DynamicQueryHelper.

• In BufferImpl.findByRowID, we have RecordBuffer buffer = buffer() at line 1551, but then we call buffer() many times more in the method. Please use the local variable instead.
• At line 1575 (same method), we manufacture a sort clause if the effort to find the primary index was unsuccessful. How would we get here? Is it possible to define a table without a primary index?
• Same method: in multiple places, we are calling printStackTrace. If the error is worth reporting, log it instead.
• Same method: not new code, but the FindQuery c'tor doesn't look right to me. We are passing an HQL where clause with no query substitution parameter (we embed the ID value directly), but we also are providing a substitution parameter in the 5th argument to the c'tor. I would expect Hibernate to error out on this. Shouldn't the 2nd argument be: buffer.getDMOAlias() + ".id = ?"? Better yet, cache the DMO alias, because we are getting it 2 or 3 times in this method.
• BufferImpl._find: we are hard-coding the query name to "query0". If I'm reading the DynamicQueryHelper code correctly, the idea is that the name is very short-lived, so there won't be any collisions, right?
• Regarding the LockType error processing, since the matrix of errors is more complicated than originally considered, perhaps it is better OO to throw an exception and allow the caller to compose the proper error message? The downside of this approach is that it would require us to refactor the places we call LockType.fromLockNoWait, so we only do the error handling in one place per query type. Not one to figure out right now, though...please open an issue for M11 for this problem.
• Vadim: please note the changes to the FieldReference.value methods.
• The PatternEngine change seems reasonable, in that we always want the resource cleanup, regardless of success. However, I'm concerned it may have some unexpected side effects, because I think many of the post-rules were written with the assumption of success. For a failed static/batch conversion, we won't be using the results anyway, so this is fine, but are there side effects we need to consider for a failed dynamic/runtime conversion? I guess we'll still be getting an exception and tossing the results. What problem specifically are you solving with this change? A resource leak?

One more thought on BufferImpl.findByRowID: we are using IndexHelper to find the primary index, but IndexHelper is not aware of dynamically defined temp-tables. This may answer my question above of how we get to the point where finding the primary index has failed. It may not matter because we don't do dirty share checking for temp-tables (well, technically we do, but it is a no-op), and that is the only thing we would need a sort clause for here. Would it make sense to always use the id column/property for the sort clause when dealing with temp-tables in this method?

Sorry, one more (minor) issue on om_upd20140122a: why did you change the initialization of the DynamicQueryHelper.wrapperToTokenType variable from

   /** Map of P2J wrappers to their associated token type. */
   private static final Map<String, Integer> wrapperToTokenType = new HashMap<String, Integer>();

   /** Map of P2J wrappers to their associated token type. */
   private static final Map<String, Integer> wrapperToTokenType = new HashMap<>();

Isn't this short-hand going to generate a compiler warning?

Eric Faulhaber wrote:

Sorry, one more (minor) issue on om_upd20140122a: why did you change the initialization of the DynamicQueryHelper.wrapperToTokenType variable from
[...]
Isn't this short-hand going to generate a compiler warning?

It should not. The diamond (new in Java 7) should force the compiler to do its job and infer the generic type that the variable was declared with. The absence of the diamond will, indeed emit an unchecked conversion warning.

Constantin Asofiei wrote:

Eric Faulhaber wrote:

Sorry, this was my comment and I see how it is misleading. It was not based on testing, but on the 4GL documentation for the DELETE() method, which states:

OK, note that a DELETE() method call is different from a DELETE OBJECT statement. Also:
- the Buffer-field resource doesn't support DELETE(), so that doesn't apply to us.
- resourceDelete is executed only when DELETE OBJECT or DELETE WIDGET is called on that buffer-field; and this is what we need to test.

Is current implementation of FieldReference.resourceDelete() correct? What should be tested in such case?

I see only one usage of resourceDelete() related with buffers – TempTableBuilder.forceClear().
resourceDelete() is last not-implemented method in note 47.

Vadim Nebogatov wrote:

Is current implementation of FieldReference.resourceDelete() correct? What should be tested in such case?

I'm not sure, though I expect it is just a matter of cleaning up resources in use for the FieldReference, if any. I suppose you should try to invoke some methods or attributes on the BUFFER-FIELD handle after DELETE OBJECT is called to be sure we report/handle errors identically.

Constantin: while we can write and convert a 4GL test case which calls DELETE OBJECT on a BUFFER-FIELD handle, what exactly are we checking for afterward, to know whether we've gotten our implementation of resourceDelete correct? Do you have related test cases for other resource types we can look at for inspiration?

Ovidiu, Vadim: it looks like you are both very close to being finished with your respective pieces. Although we originally intended to do only a conversion regression test, I think the changes have been invasive enough in some cases (especially once Ovidiu gets the 0117a FieldReference changes fixed and put back in some form), that we need to do a full regression test after all.

Constantin: please have a quick look at Ovidiu's changes related to DynamicTablesHelper in om_upd20140122a.zip.

Ovidiu: once you have gotten Constantin's feedback and addressed the remaining issues from my code review in note 89, and the H033 FieldReference issue, I would like you to take the lead with merging and regression testing your combined changes. Vadim, please post your latest update for Ovidiu to merge. My understanding is that it addresses my code review items and is complete except for resourceDelete. If/when you have changes for that method, please post that separately. Ovidiu, once you have that final piece, please regression test the combined update.

Questions/concerns, anyone?

• what I don't like is that we are duplicating code in parseQuery and parseFindQuery. You can add a DynamicQuery$QueryParser inner class which:
  - has an abstract preparePredicate(String) method, which receives as input the original predicate and returns a valid 4GL code snippet. So, for your case, a FIND FIRST ... <lock-type> will be returned. And for the dyn query case, a OPEN QUERY... will be returned.
  - has an abstract postprocessJavaAst(JavaAst) method, which receives as input the created JavaAst to post-process it, depending on the case: find query or open-query. This will just call the proper runPattern(jcode, "convert/fix_in_mem_<something> false); code.
  - has a parse method, which contains the currently DynamicQueryParser.parseQuery code, changed to properly call preparePredicate and postProcessJavaAst.
  - has two subclasses, FindQueryParser for the find-query case, and OpenQueryParser for the dyn query case. There will be only one instances of each class, in some DynamicQueryParser.findQueryParser and DynamicQueryParser.openQueryParser constants. These two classes must not save any data in instance fields; save the lock-type in a WorkArea.lockType field, and use it later in preparePredicate.
  - if you notice something else which needs to be abstracted, let me know.
  - when i.e. DynamicQueryHelper.parseFindQuery is called, this will save the lock-type in the WorkArea and delegate the call to the FindQueryParser.parse API. Similar for the DynamicQueryHelper.parse.
• also, you've duplicated compileFindQuery from compileQuery only because your getter in the dynamic java code is named getFindQuery instead of getQuery... this is not needed, let the getter be named getQuery and remove compileFindQuery.
• I see you are treating parsing/conversion errors. This is not needed. All predicates passed to DynamicQueryHelper must be valid 4GL code. If the input is not valid 4GL code, then the 4GL code must be fixed. We don't need and don't want to start hunting for all the possible 4GL compile problems. Remove the showFindConversionError and let only the showConversionError be called throughout the code.
• for the BUFFER:FIND-* cases, the dynamic query is short lived, and I expected to be used like:
  - use DynamicQueryHelper to parse the predicate and create a dynamic query
  - execute the query
  - discard it
  Currently, the dynamic query obtained in BufferImpl._find is not being discarded; this will lead to a mem leak in the class loader, as we need to remove the in-mem compiled class.

If you think you can do these changes and get them regression tested by tomorrow, go ahead and fix all issues. Else, solve only the class loader mem leak (by calling DynamicQueryHelper.unregisterQuery in a finally block in BufferImpl._find and will address the other issues after M7.

About the query name: it can be hard-coded to anything, this is used only in error messages. But instead of query0, please use something like buffer-find-<first/last/etc>, to give a clue to the user to what we are converting.

Vadim Nebogatov wrote:

Is current implementation of FieldReference.resourceDelete() correct? What should be tested in such case?

1. call DELETE OBJECT on the buffer-field
2. delete the dynamic or static buffer
3. delete the dynamic temp-table
4. after DELETE is called, check how the var is reported using VALID-HANDLE
5. you can use DELETE OBJECT ... NO-ERROR to check for any reported errors, via ERROR-STATUS:ERROR, ERROR-STATUS:NUM-MESSAGES and ERROR-STATUS:GET-MESSAGE.
6. a quick example of how the code can look:
   

   def var hf as handle. /* assume this refers to a valid buffer-field */
   def var h as handle. /* assume this refers to same buffer-field as h, a valid buffer (static or dynamic) or a valid dynamic temp-table, to which hf belongs. */
   
   /* 1. check if any error condition is thrown. */
   do on error undo, leave:
     delete object h.
     message "no error condition is thrown".
   end.
   /* check how the buffer-field is reported. */
   message valid-handle(hf).
   
   /* 2. check how error messages are recorded. */
   delete object h.
   message error-status:error error-status:num-messages error-status:get-message(1).
   
   /* check how the buffer-field is reported. */
   message valid-handle(hf).
   

Code review om_upd20140123a:

I'm good with the changes, assuming you will come back around after M7 and finish up the refactoring Constantin recommended for the dynamic find queries. I noticed that the version of TableMapper included in this update was not different from the latest bzr revision. Note that Stanislav has changes to TempTableBuilder as well, but your change is small and should merge easily. Please make one minor change for me: looks like a typo in QueryHelper; search on "processer" and replace with "processor".

• I'm OK with using the QueryProcessor interface instead of the QueryParser abstract class (actually, this is cleaner than my approach, I'm glad you got the idea and made it better).
• add javadoc for the processor parameter in parse
• remove/comment the System.err.println(pcode); at line 194
• I would like to use inner classes and not anonymous classes for the two implementations of QueryProcessor interface. This way, the openQueryProcessor and findQueryProcessor fields can be moved in the proper place, and not just be at the end of the class.

I tested the following test case based on note 98

def var hf as handle. /* assume this refers to a valid buffer-field */
def var h as handle. /* assume this refers to same buffer-field as h, a valid buffer (static or dynamic) or a valid dynamic temp-table, to which hf belongs. */
def var hd as handle.

define temp-table tt1
     field fc as character
     field fe as character extent 7.

def var hBuffer1 as handle.
hBuffer1 =  buffer tt1:handle.

create tt1.
tt1.fc = "aa".

hf = hBuffer1:buffer-field("fc").
h = hBuffer1:buffer-field("fc").

/* buffer-field */

/* 1. check if any error condition is thrown. */
do on error undo, leave:
  delete object h.
  message "no error condition is thrown".
end.
/* check how the buffer-field is reported. */
message valid-handle(hf).

/* 2. check how error messages are recorded. */
delete object h.
message error-status:error error-status:num-messages error-status:get-message(1).

/* check how the buffer-field is reported. */
message valid-handle(hf).

/* buffer */

hd = buffer tt1:handle.

/* 1. check if any error condition is thrown. */
do on error undo, leave:
  delete object hd.
  message "no error condition is thrown".
end.
/* check how the buffer-field is reported. */
message valid-handle(hf).

/* 2. check how error messages are recorded. */
delete object hd.
message error-status:error error-status:num-messages error-status:get-message(1).

/* check how the buffer-field is reported. */
message valid-handle(hf).

/* temp-table */

hd = temp-table tt1:handle.

/* 1. check if any error condition is thrown. */
do on error undo, leave:
  delete object hd.
  message "no error condition is thrown".
end.
/* check how the buffer-field is reported. */
message valid-handle(hf).

/* 2. check how error messages are recorded. */
delete object hd.
message error-status:error error-status:num-messages error-status:get-message(1).

/* check how the buffer-field is reported. */
message valid-handle(hf).

Results are the same both on P2J and 4GL:

no error condition is thrown
yes
no 0 
yes
no error condition is thrown
yes
no 0 
yes
no error condition is thrown
yes
no 0 
yes

Vadim: this seems like the BUFFER-FIELD can never be deleted explicitly. I forgot about dynamic buffers - please test this too:

def temp-table tt1 field f1 as int.
create tt1.
tt1.f1 = 10.

def var h as handle.
create buffer h for table "tt1".

def var hf as handle.
hf = h:buffer-field("f1").

delete object h.
message valid-handle(hf). /* this shows no */

Tested, results are the same:

no

FieldReference.value(int) was broken. I'll include this fix in my next update.

public FieldReference(DataModelObject dmo,
                         String property,
                         boolean uppercase,
                         NumberType index)
   {
      ...
      // do not initialize sizer if this is a reference to an element of an extent
      this.sizer = (index == null) ? getSizer(dmoIface, property) : null;

Stanislav Lomany wrote:

// do not initialize sizer if this is a reference to an element of an extent

Don't you mean:

// only initialize sizer if this is a reference to an element of an extent

Technically note is correct. I can write "initialize sizer for extent fields and do not initialize for non-extent fields or extent elements".