FWD » Core Development » Database

I tried to simplify the sample. Suppose we have a simple function:

FUNCTION f1 RETURNS LOGICAL (d AS DATE).
   RETURN YES.
END FUNCTION.

A statement like:

MESSAGE CAN-FIND(FIRST Customer 
      WHERE Customer.Address <> "COM" 
        AND f1(Customer.description)
).

FindQuery(Customer.Buf, String, WhereExpression, String, logical, LockType)

Here are the converted code for two cases (which attempt to call two new constructors), with and without another joined table:

FIND FIRST Book.
MESSAGE CAN-FIND(FIRST Customer 
      WHERE Customer.Address <> Book.book-title 
        AND Customer.Balance <> 0 
        AND (f1(Customer.description) AND f2(Customer.uniq))
).

MESSAGE CAN-FIND(FIRST Customer 
      WHERE Customer.Address <> "COM" 
        AND f1(Customer.description)
).

message(new FindQuery(customer, "upper(customer.address) != ? and customer.balance != 0", whereExpr0, "customer.id asc", and(f1(customer.getDescription()), new LogicalExpression()
{
   public logical execute()
   {
      return f2(customer.isUniq());
   }
}), new Object[]
{
   toUpperCase((character) new FieldReference(book, "bookTitle", true).getValue())
}, LockType.NONE).hasAny());

message(new FindQuery(customer, "upper(customer.address) != 'COM'", whereExpr1, "customer.id asc", f1(customer.getDescription()), LockType.NONE).hasAny());

   WhereExpression whereExpr0 = new WhereExpression()
   {
      public logical evaluate(final BaseDataType[] args)
      {
         return and(f1(customer.getDescription()), new LogicalExpression()
         {
            public logical execute()
            {
               return f2(customer.isUniq());
            }
         });
      }
   };

   WhereExpression whereExpr1 = new WhereExpression()
   {
      public logical evaluate(final BaseDataType[] args)
      {
         return f1(customer.getDescription());
      }
   };

Ovidiu Maxiniuc wrote:

The parameter is in fact an echo of the content of the client-side WhereExpression: f1(customer.getDescription()) and no additional constructors should be implemented but the extra parameter should be removed.

I think your analysis is exactly correct. The user defined function has been moved correctly to the client-side WhereExpression. The parameter should not be emitted into the FindQuery constructor. It must be that we are making a duplicate of the expression which calls the function, attaching it to the body of the WhereExpression.evaluate method, but then not hiding the original expression we copied. Actually, hiding may not be enough, we may need to remove it.

I currently am working in the same area, so I am curious to know what you find.

Another can-find leak:

message CAN-FIND (FIRST book WHERE book.book-id > 0 and CAN-DO("A,H",TRIM(book.book-title))). /* this fails */
message CAN-FIND (FIRST book WHERE CAN-DO("A,H",TRIM(book.book-title))). /* this works */

I did some patch (removing the extra expression from where node after generating the anonymous inner class needed by client-side expression) which will fix the issue in some cases. Then I start testing with more complex queries. I found another strange conversion issue (may be the same, cannot tell yet).
Here is the source code:

FIND FIRST Book.
MESSAGE CAN-FIND(FIRST Customer 
      WHERE f1(Customer.description) AND f2(Book.book-title <> "")

WhereExpression whereExpr0 = new WhereExpression()
{
   public logical evaluate(final BaseDataType[] args)
   {
      return f1(customer.getDescription());
   }
};

new FindQuery(book, (String) null, null, "book.bookId asc").first();
message(new FindQuery(customer, (String) null, whereExpr0, "customer.id asc", f1(customer.getDescription()), new Object[]
{
   and(f1(customer.getDescription()), new LogicalExpression()
   {
      public logical execute()
      {
         return f2(isNotEqual((character) new FieldReference(book, "bookTitle").getValue(), ""));
      }
   })
}, LockType.NONE).hasAny());

You can see that the whereExpr0 is broken, the correct form of it can be found into the 6th parameter, Object[], which in turn should have been empty.

In my previous note there the and operator from where clause contains a reference to a buffer other than the one that the current Query applies.
In the phase that separates server-side from client-side branched, the f2(Book.book-title <> "") is not marked as client_branch or client_tint as the function that do this is only called for nodes marked as "current_buffer".

This is the change in where_clause_prep2.rules, line ~190:

...
<rule>
   evalLib("fields") and
   ancestor(prog.kw_where, -1) and
   (!getNoteBoolean("hql") or clientCF)

   <rule>
      parent.type == prog.func_poly and
      #(long) parent.getAnnotation("oldtype") == prog.kw_input
      <action on="false">execLib("mark_client_branch", copy, 1)</action>
   </rule>

   <rule>!getNoteBoolean("current_buffer")
      <!-- related buffer references that are the root of a
           sub-expression are not nested refs BUT a more deeply
           nested related buffer reference IS a nested ref -->
      <rule>
         !clientCF and
         getNoteBoolean("related_buffer") and
         (!isNote("sub_expression") or
          !getNoteBoolean("sub_expression"))
         <action>execLib("mark_client_branch", copy, 2)</action>
      </rule>
      <rule>
         ...

I'm getting closer. Here what I found:

The extra node (f1 [FUNC_LOGICAL]) should be made hidden so it will not be emitted in java source. This should happen in where_clause_post.rules: ~715:

<rule> parent.type == prog.kw_where and
       (type == prog.expression or evalLib("type_pair", parent.parent.parent, prog.func_logical, prog.kw_can_find))
   <action>copy.setHidden(true)</action>
</rule>

At this moment, I use the following simplified testcase:

MESSAGE CAN-FIND(FIRST Customer WHERE f1(Customer.description)).

MESSAGE CAN-FIND(FIRST Customer WHERE 
         Customer.Address <> "COM" AND 
         Customer.Name = "gigi").

MESSAGE CAN-FIND(FIRST Customer WHERE 
         Customer.Address <> "COM" AND 
         f1(Customer.description)).

The first 2 of them are converted successfully, but not the 3rd (some kind of mix between server and client side).

I couldn't get my mind of client_taint and I took a quick look where is used. In where_clause_post.rules, at line 413, there is this rule:

      <!-- prune server side only child node of an AND where the other child
           is a client branch;  promote client branch to replace parent AND
           node -->
      <rule>
         parent.type == prog.kw_and                      and
         parent.isAnnotation("client_taint")             and
         #(boolean) parent.getAnnotation("client_taint") and
         ancestor(prog.kw_where, -1)                     and
         evalLib("is_server_branch")

This moves a client branch node (like a function call) to be the child of the KW_WHERE node. If you disable this rule, the code converts with no compile errors, but the server-side condition is duplicated at the client-side WhereExpression.

I think this was intended to ensure that the WhereExpression doesn't contain tests which can be done at the server-side.

After some more tests, it seems like this update does not fix the issue, in fact it is too restrictive, it will cut more branches including from the evaluate() of the whereExprN anonymous declarations. Here are some examples:

public logical evaluate(final BaseDataType[] args)
{
   return and(or(f2(customer.isUniq()), (logical) args[0]));
}
...
public logical evaluate(final BaseDataType[] args)
{
   return ;
}

MESSAGE "2" + CAN-FIND(FIRST Customer WHERE
         f1(Customer.description)                                      AND 
         Customer.Address <> Book.book-title AND Customer.Balance <> 0 AND
         (f2(Customer.uniq) OR f2(Book.book-title <> "3"))).

FOR EACH Customer WHERE Customer.Address <> "0" AND f1(Customer.description):
END.

The <action>sibref.setHidden(true)</action> from where_clause_post.rules at line 438 from my previous update seems correct, however, the clause is broken somewhere in the next step in where_clause_post2.rules.
There is an <action>copy.remove()</action> at line 235 (or so) that leads somewhere.
However, there are some extra nodes that are mixed between [CLIENT_WHERE] and [CLIENT_WHERE_SUBST], I mean , args [VAR_LOGICAL] looks wrong placed in [CLIENT_WHERE].

Ovidiu, I was having many similar issues with client-side query substitution parameters and CAN-FIND while fixing #2059. Please see if my fix candidate (ecf_upd20130312a.zip, attached to #2059) improves things here. I would recommend trying it with no other changes.

Ovidiu, I would like to close out this issue. Please confirm that the test case(s) you were using work correctly with the latest revision of P2J in bzr; upload your test cases here and check them into the testcases project.