FWD » Core Development » Database

I just discovered that we don't cover all the conversions between datatypes. We need to implement additional constructors for classes extending BDT to allow conversion from FieldReference objects. Here are more details.
Consider the following code (based on code from a customer application):

define temp-table tt1
    field f1-dte as date
    field f2-int as integer
    field f3-dte as date
    field f4-chr as char
    field f5-dec as decimal.

define temp-table tt2
    field f1-dte as date
    field f2-int as integer
    field f3-int as integer
    field f4-chr as character.

FOR 
    EACH tt1 
       WHERE  tt1.f1-dte = 02/05/05 
         AND  tt1.f2-int = 10 
         AND  tt1.f3-dte = 04/04/05 
         AND  tt1.f4-chr = 'ABC' NO-LOCK, 
    FIRST  tt2 
       WHERE  tt2.f1-dte = 02/05/05 
         AND  tt2.f2-int = 10 
         AND  tt2.f4-chr = 'ABC' 
         AND  (tt1.f5-dec = 0 
            OR tt2.f3-int = INTEGER(tt1.f5-dec))

query0 = new CompoundQuery(false, false);
query0.addComponent(new AdaptiveQuery(tt1, "tt1.f1Dte = '2005-02-05' and tt1.f2Int = 10 and tt1.f3Dte = '2005-04-04' and upper(tt1.f4Chr) = 'ABC'", null, "tt1.id asc", LockType.NONE));
query0.addComponent(new RandomAccessQuery(tt2, "tt2.f1Dte = '2005-02-05' and tt2.f2Int = 10 and upper(tt2.f4Chr) = 'ABC'", whereExpr0, "tt2.id asc"), QueryConstants.FIRST);

    WhereExpression whereExpr0 = new WhereExpression() {
       public Object[] getSubstitutions() {
          return new Object[] {
             new IntegerExpression() {
                public integer execute() {
                    return new integer(new FieldReference(tt1, "f5Dec"));
                }
             }
          };
       }

This is not compilable because we don't have such integer constructor at this moment. The issue was not visible by the compiler because this query is generated at runtime and only the classes that handle dynamic queries will encounter it.

The implementation is as simple as adding add a new constructor with at least a single Accessor parameter, because using FieldReference would add an unwanted reference to persistence. A second optional parameter of type character can be used to specify the format to be used during the conversion.
The affected datatypes are integer, int64, decimal, date, datetime, datetimetz and logical.
The STRING function is implemented as a static method valueOf in character class and it also must be overloaded.

Ovidiu Maxiniuc wrote:

I guess we should keep the new constructor and replace the return line like:

return isEqual(tt2.getF3Int(), new integer(args[0]);

I think this proposed conversion is correct, but we don't need the new constructors. Each data type should already have a c'tor that accepts BaseDataType (please confirm this). Assuming we make the getSubstitutions conversion change above, args[0] should already be a decimal value by the time it is passed into the generated evaluate(BDT[]) method. It will have been obtained by resolving the FieldReference object returned by getSubstitutions (see WhereExpression.evaluate(), specifically line 197).

Yes, this is correct, the new constructors are not really needed.
I checked all datatypes classes and they all have a c'tor that accepts BaseDataType.

The fix for moving the field evaluation along with the function that use it as parameter is done by changing the evalLib("function_calls") or from line 196 of annotations/where_clause_post2.rules. I am working to rewrite this to something that can identify subtrees functions calls that don't contain references to fields that are not part of the current buffer. These subtrees must be kept in getSubstitutions().
If the subtree contains references to this kind of buffers, they are moved to evaluate() method and only the field references will get an entry in the getSubstitutions() returned array.

If the the tree would have already been processed some annotation could be in handy to identify these cases. But in walk-rules stage I must do an iteration of the subtree to detect non current-buffer field references.

Code review 0311a:

I am not sure I wrote the function call annotation in the right place (index_selection.rules) but it can be moved.

Right, I don't think index_selection.rules is the most appropriate place for these changes, since they are about query substitution parameters, not index selection. This processing probably best belongs in where_clause_prep.rules, which is similar in structure to index_selection.rules, and which is where we do a lot of annotation regarding query substitution parameters.

I think the rest of the update is OK, but it's hard to tell whether it will cause broader changes than what you intend. This should become clearer with conversion regression testing.

Code review 0313b:

Just thought of one edge case which might be problematic: consider a WHERE clause with a single, non-current-buffer field (would have to be a FIELD_LOGICAL) and nothing else. Wouldn't this mean the field's parent is the KW_WHERE node, which would be marked with the "where_not_subst" annotation? Perhaps a silly case which would better be refactored differently...but would it break the update?

Otherwise looks good. If it needs to be fixed to deal with the above case, please do so. Then, please conversion regression test. If the converted output is different, but looks correct, runtime regression test as well.

• in CTRL+C the clients were killed twice in ctrlc_11_session3 so there were semaphore issues in ctrlc_11_session1 and ctrlc_11_session4.
• in main tc_job_clock_002 failed beside tc_job_002. This is a known-to fail test.

I will restart the runtime test and hope the tests not executed will have a second chance now.

I guess after the second run, update passed the test.

Please commit and distribute the update.

Update committed to bzr as revno 10815 and distributed by mail.