Project

General

Profile

Bug #7235

Avoid selecting all fields for an CAN-FIND sub-select

Added by Alexandru Lungu over 1 year ago. Updated 12 months ago.

Status:
Closed
Priority:
Normal
Target version:
-
Start date:
Due date:
% Done:

100%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:

7235-20230419.patch Magnifier (3.09 KB) Dănuț Filimon, 04/19/2023 05:17 AM

7235-20230425.patch Magnifier (2.14 KB) Dănuț Filimon, 04/25/2023 05:50 AM

History

#1 Updated by Alexandru Lungu over 1 year ago

4GL code:

find first tt where can-find(FIRST tt2 where tt2.f1 = tt.f1).
Java code:
new FindQuery(tt, "exists(from Tt2_1_1 as tt2 where upper(tt2.f1) = upper(tt.f1))", null, "tt.recid asc").addExternalBuffers(tt2).first();
SQL:
select [9 columns] from tt3 tt_1_1__im0_ where tt_1_1__im0_._multiplex = ? and exists(select [9 columns] from tt4 tt2_1_1__i1_ where tt2_1_1__i1_.f1 = tt_1_1__im0_.f1 or tt2_1_1__i1_.f1 is null and tt_1_1__im0_.f1 is null) order by    tt_1_1__im0_._multiplex asc, tt_1_1__im0_.recid asc limit ?

I see that we don't generate an ORDER BY or LIMIT for the sub-select, which is good. However, we still generate the field list (all fields of tt2 in this case). This is not required, as the data won't reach the FWD buffer for tt2. It can be replaced with 1 or some other constant. Maybe a smart SQL engine will optimize this out, but even so, we still lose time on SQL generation, parsing and caching.

#2 Updated by Alexandru Lungu about 1 year ago

  • Status changed from New to WIP
  • Assignee set to Dănuț Filimon

#3 Updated by Dănuț Filimon about 1 year ago

I've attached a patch for the current issue.
When using the function exists and the first child is of type SUBSELECT, 2 new nodes are created (SELECT and NUM_LITERAL), for the SUBSELECT and one for the current node.
A few changes were made to FQLPreprocessor.emit so that the subselect is emitted correctly:
  • if the immediate child is a NUM_LITERAL, it will add a space before and after the text node.
  • when using functions, it will usually add a comma but if the first child of the AST is a SELECT, it won't happen.
  • Another case where if the parent is a SUBSELECT, the NUM_LITERAL shouldn't be able to fall in the default case.
    The change to FqlToSqlConverter is to prevent generating a column name for the NUM_LITERAL.

I've initially tried to make use of FqlToSqlConverter to generate the subselect statement, but it may be too late to handle it here.
Please review the current changes.

#4 Updated by Alexandru Lungu about 1 year ago

  • % Done changed from 0 to 80

Review of #7235 patch in #7235-3.

  • When you append the NUM_LITERAL, you also append SELECT (which is OK). You use subSelectNode.graftAt(selectValueNode, 0); and next.graftAt(selectNode, 0);. Is subSelectNode.graftAt(selectNode, 0); OK, for the select token? Maybe next = subSelectNode, but I doubt this as "exists".equals(next.getText()) && next.getFirstChild().getType() == SUBSELECT. Please check that the SELECT keyword is placed correctly (as a left sibling of the num literal, as a child of the subselect).
  • Don't inject SELECT and NUM_LITERAL if SELECT already exists! You may already preprocess an FQL which is of type EXISTS (SELECT 1 FROM tt); you don't want to end up with EXISTS (SELECT 1 SELECT 1 FROM tt).
  • I think you can drop child.getType() != NUM_LITERAL in FqlToSqlConverter. It is not that bad to have SELECT 1 as col1 from tt.
  • Don't use if (ast.getFirstChild().getType() != SELECT) to rule out the commas, as they are used to separate a functions parameters. Your problem is somewhere else regarding commas. MY_FUNC((SELECT 1 FROM tt), 2) is a valid syntax, where the subselect is a function argument. Your conditional removes the comma between the first and second arguments. Maybe you can use nextChild with SUBSELECT to check which children are you iterating:
    // SUBSELECT case
    Aast child = ast.getChildAt(index);
    // this only handled num literals as selection fields; this can be extended to support booleans/string/table fields, etc.
    if (child.getType() == NUM_LITERAL)
    {
        // add comma only if (ast.getChildAt(index + 1) != FROM)
        // add space
    }
    
  • Let the NUM_LITERAL be emitted by the default case (buf.append(text))

Please address the first point. That is, make sure SELECT, NUM_LITERAL and FROM are on the same level, children of SUBSELECT.
Afterward, check that injecting two NUM_LITERAL (EXISTS(SELECT 1, 1 FROM TT)) produces the right FQL (with proper spacing and commas). This should be work.

#5 Updated by Dănuț Filimon about 1 year ago

Alexandru Lungu wrote:

... Please check that the SELECT keyword is placed correctly (as a left sibling of the num literal, as a child of the subselect).

I reorganized the nodes since they were not placed correctly.

... Your conditional removes the comma between the first and second arguments. Maybe you can use nextChild with SUBSELECT to check which children are you iterating: ...

I took the code suggestion and changed it a little bit. Instead of checking the next node and see if it's a FROM, I check the previous node for a SELECT, this way I can generate statements like select 1, 2, 3 from. At the same time, another space needs to be added when if the child is a FROM in order to avoid obtaining select 1, 2, 3from.

I attached a new patch where I made the suggested changes, please review.

#6 Updated by Alexandru Lungu about 1 year ago

  • Status changed from WIP to Review
  • % Done changed from 80 to 100

Added this patch to 7026c.
Tested this. I got a -0.24% performance improvement with a 7026c profile run.
Just rebased 7026c with trunk.

Fully reviewed tested and profiled. 7026c is ready for trunk.
I intent to bump the last FWD-H2 revision with 7026c (rev. 17).

#7 Updated by Alexandru Lungu about 1 year ago

  • Status changed from Review to Test

Merged 7026c to trunk as rev. 14553.

#8 Updated by Eric Faulhaber 12 months ago

Can this be closed?

#9 Updated by Alexandru Lungu 12 months ago

Yes.

#10 Updated by Eric Faulhaber 12 months ago

  • Status changed from Test to Closed

Also available in: Atom PDF