FWD » Core Development » Database

I profiled the INDEX-INFORMATION implementation in FWD on a POC that runs in ~10.000ms. ~10ms are spent inside INDEX-INFORMATION. Most of the uses of INDEX-INFORMATION in the customer application I tested are for a not-opened dynamic query. I find the following code problematic, inside QueryWrapper.indexInformation:

if (this.closed)
{
    getDelegate().notifyQueryOpenListeners(false);
}

query0.addQueryOpenListener((evaluate) -> 
{
   interpreter.processInstancesWith((o) -> 
   {
      if (o instanceof P2JQuery)
      {
         ((P2JQuery) o).setDynamicPredicate(query0.isDynamicPredicate());
      }
   });
   interpreter.interpret("execute");

   if (evaluate)
   {
      interpreter.evaluateDynamicCalls();
   }
});

My concern is that QueryWrapper.indexInformation is called 10~20 times (twice per query component) in my POC and each time the interpreter runs "execute" to extract the index information of the same query.

Ovidiu, OFC, we can't mark the query as opened to avoid this. Can we at least short-circuit this listener if the dynamic query was already parsed and interpreted? I have 20 compound queries with 5 components each generated for the same logical dynamic query, and this happens only to provide QueryWrapper.indexInformation. Note that the listener shouldn't evaluate dynamic calls in this case. All in all, such short-circuit may also boost some other dependents beside INDEX-INFORMATION.

This is a low priority optimization. In a large customer application POC, the time spent in INDEX-INFORMATION is only 0.1%. However, if we can do a quick fix for this, I think we can take it as it is in 7026c.

Alexandru Lungu wrote:

Ovidiu, OFC, we can't mark the query as opened to avoid this. Can we at least short-circuit this listener if the dynamic query was already parsed and interpreted? I have 20 compound queries with 5 components each generated for the same logical dynamic query, and this happens only to provide QueryWrapper.indexInformation. Note that the listener shouldn't evaluate dynamic calls in this case. All in all, such short-circuit may also boost some other dependents beside INDEX-INFORMATION.

Actually, the execute method should be executed exactly one time, when the dynamic query is open. It might be difficult to test is the method was already executed and skip subsequent calls.

Rather it seems to me that the getDelegate().notifyQueryOpenListeners(false); is not called at the right moment/condition in QueryWrapper.indexInformation(). There are two calls for notifying these listeners. The one from queryOpen() is logical, but in indexInformation() it does not make sense. In 4GL, "INDEX-INFORMATION requires query <name> to be QUERY-PREPARED or open".

I will think how can we solve this.

Ovidiu Maxiniuc wrote:

Rather it seems to me that the getDelegate().notifyQueryOpenListeners(false); is not called at the right moment/condition in QueryWrapper.indexInformation(). There are two calls for notifying these listeners. The one from queryOpen() is logical, but in indexInformation() it does not make sense. In 4GL, "INDEX-INFORMATION requires query <name> to be QUERY-PREPARED or open".

In my case, the query is prepared, but not opened. In FWD this means that the dynamic query is parsed, but not instantiated. The open listener itself is the one instantiating. To retrieve the index information you need a P2JQuery, but not an opened the query, so as a "hack" you can just notify the open listeners, without opening.

Also note that this happens only when processor.delayedExecute().

Well, the listener was added exactly for this reason, to invoke the execute() method later, asynchronously from prepare event, as you noted only for delayed executions. What execute() does is to construct the P2JQuery (by adding/configuring its components).

Invoking the execute() method multiple times will mess with P2JQuery setup. In worse case, it will be reconstructed again, from the scratch, ending up with a viable P2JQuery object, and in worst case a malformed object could be built if the components are added as duplicate or in incorrect order.

An alternative solution would be to mark the method that it was executed at first callback of open query listener and avoid the whole work on subsequent calls. Of course, the flag will have to be reset when the query is closed.

I have reviewed fix_7258.patch. I think it works exactly how was described in previous notes.

As an optimization, we may want to merge the open and close the listeners as a single class and invoke the appropriate method for each event. This may not directly improve performance, but would take less memory. This is not mandatory, but maybe we should take it into account.

Ovidiu, I am planning to merge 7258a today. Move this to Test after you check #7158-7: these are the same changes as #7258-5, but with merged open/close listeners.

Branch 7258a has been merged to trunk at revision 14549.