Bug #10362
FastFindCache.L2Key uses the buffer alias in the FQL
50%
Related issues
History
#1 Updated by Artur Școlnic 12 months ago
define temp-table tt1 field f1 as int index idx f1. create tt1. tt1.f1 = 1. release tt1. define buffer tt1buf for tt1. find tt1 where tt1.f1 = 1. find tt1buf where tt1buf.f1 = 1.
The second find will not retrieve the record from the cache because of the different fql tt1.f1 = 1 vs tt1buf.f1 = 1.
We need to drop the alias from the where clause before using it to create the cache key.
#3 Updated by Artur Școlnic 12 months ago
- Assignee set to Artur Școlnic
#4 Updated by Artur Școlnic 12 months ago
- Related to Bug #10307: FFcache keys don't match for the values inlined vs passed as args added
#5 Updated by Artur Școlnic 12 months ago
- Status changed from New to WIP
I think the buffer has enough information to replace the alias with the implicit buffer/table name.
#6 Updated by Artur Școlnic 12 months ago
- Status changed from WIP to Review
- reviewer Alexandru Lungu, Constantin Asofiei added
I committed the code to 10362a. Alexandru or Constantin, please review.
#7 Updated by Alexandru Lungu 11 months ago
- Status changed from Review to WIP
- % Done changed from 0 to 50
I think the solution is quite fragile, considering that .replace can match other tokens that are not the actual alias. For example tt1buf.f1tt1buf = 1 will be converted to tt1.f1tt1 = 1, but f1tt1 field does not exist. The same may apply to string literals, so that tt1buf.f1 = "tt1buf" will end up as tt1.f1 = "tt1", which is functionally incorrect.
This technique should be moved after actually parsing the FQL into AST (FqlPreprocessor). This way we can distinguish between ALIAS token and other tokens (fields, string literals, etc.). The "normalizedWhere" should be retrieved from something like preproc.getNormalizedWhere().
Please check FqlPreprocessor.emit that converts the processed AST into a proper FQL. buf is the actual FQLExpression that is computed by traversing the tree and appending string tokens.
Also, please refer to recommendations in #9802:
OK, then the only way we can do this is via FQLPreprocessor.
This is already used by the 'activeBundle', and we may need to refactor the FFCache usage in RAQ, so that we don't parse the WHERE string twice (once for checking FFCache, once for checking the 'activeBundle').
But FQLHelper has a cache on its own, so maybe the overhead will not be too much.
Please mind that FQLPreprocessor already has some infrastructure with dropAlias. I can't recall what was this about, maybe it is worth looking into it. Also consider the informational mode that avoids some overhead.
Now that I look into emit, there is a translate flag, so maybe you can leverage that inside getNormalizedWhere (by calling emit with translate). You shall eventually temporarily update replacementAlias and dropAlias properly to do the magic.
Also, please mind the FQLPreprocessor(List<RecordBuffer> bound, List<RecordBuffer> definition, String where, boolean singleBuffer) constructor that does this exact same thing from scratch. However, you will need to reuse the existing preprocessor to avoid double-parsing as Constantin suggested.
#8 Updated by Alexandru Lungu 11 months ago
- Related to Bug #9016: Improve performance in FQLPreprocessor by using astCache added
#9 Updated by Alexandru Lungu 11 months ago
I think #9016 can fix the problem easier. If we use the astCache for the private FQLPreprocessor(List<RecordBuffer> bound, List<RecordBuffer> definition, String where, boolean singleBuffer) c'tor, then I guess in #10362 you can simply use this ctor and translate from bound (explicit buffer) to definition (implicit buffer) easier. Of course, a more friendly signature is required so that you intake only:
Map<String, Map<String, String>> aliasToMappings = new HashMap<>();
Map<String, String> aliases = new HashMap<>();
... or other structures required. The point is that 95% of the business logic is already in that c'tor of FQLPreprocessor. You need to adapt it to your need (mapping in between DMO aliases) and implement #9016 to avoid double-parsing.
#10 Updated by Artur Școlnic 11 months ago
From FQLPreprocessor():
* @param definition
* The buffers as it was used to generate the {@code where} clause at conversion time.
If I understood correctly, the alias is replaced only if one is used in the where clause, but the query is executed using another buffer, in other words, in the example from note 1, the definition buffer is still the explicit one.
Using the FQLPreprocessor maybe would work, if there would be a way to get the default buffer from an explicit one. There is the getDefaultBuffer, but it works only for temp tables.
In fact, I checked in a customer project and
String alias = buf.getDMOAlias();
String defAlias = def.getDMOAlias();
are always the same for permanent tables, so I am not sure if these mappings actually do something.
#11 Updated by Artur Școlnic 11 months ago
Considering that the buffer is already used in the FF Key as a separate parameter, I don't see any downside to dropping the buffer prefix from the field in the where clause, we would use just f1 = 1.
#12 Updated by Artur Școlnic 11 months ago
From RAQ.initialize
this.buffer = ((BufferReference) dmo).buffer();
RecordBuffer defBuffer = ((BufferReference) dmo).definition();
if (!buffersMatch(buffer, defBuffer))
{
...
return translateWhere(binding, definition, where); // fql pre-processing
for explicitly defined buffers (like tt1buf), buffers always match, the
definition method returns the same bound buffer, so the FQLPreprocessor is never used, I am not sure if this is expected or not.The bottom line is that we should not introduce additional overhead with fql processing or String manipulation, if not already present.
#13 Updated by Artur Școlnic 11 months ago
- Status changed from WIP to Review
Considering that FQLPreprocessor is not already used in the discussed cases and the where clause is not processed in any way, there is only one place where we can still traverse it at no (much) additional cost, the hash code calculation.
Instead of using the built in hash calculation, I wrote a custom hash code method that calculates the hash code while traversing the string, but ignores the buffer name.
This solves the cases like the one in note 1.
The code is in 10362a/16086.
Constantin, please review.
#14 Updated by Constantin Asofiei 7 months ago
- Status changed from Review to WIP
I don't think the approach will work - if I'm not mistaken, a DOT can be part of literals (like digits or strings).
We need to think of something better, maybe at conversion - do not emit the alias if the FQL uses a single buffer?