Project

General

Profile

Activity

From 02/27/2021 to 03/28/2021

03/29/2021

11:54 PM Feature #5219 (New): implement native full-text search for word index and CONTAINS support
Eric Faulhaber

03/28/2021

11:33 AM Feature #1587: implement full support for word indexes
For the performance testing (@words/words-perf.p@) I use the following data:
The test table definition is:...
Igor Skornyakov
04:56 AM Feature #1587: implement full support for word indexes
Added documentation for the new @CONTAINS@ support to the 'Internals' section of Wiki. See [[p2j:Word Index and CONTA... Igor Skornyakov

03/26/2021

05:04 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> For the "how it works" documentation you just posted, you could add a page to the @Internals...
Igor Skornyakov
05:02 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> BTW: maybe it makes sense to add documentation to the Wiki? If so, then to what section?
...
Eric Faulhaber
04:36 PM Feature #1587: implement full support for word indexes
h1. Word tables and related database objects
For the table *{T}* and field *{f}* with word index on it the word ta...
Igor Skornyakov
02:13 PM Feature #1587: implement full support for word indexes
Merged wors tables tests' data to the @fwd@ test database. Performed cleanup of the tests.
Committed to @xfer.gold...
Igor Skornyakov
01:33 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> What is the problem and do you have an idea about the level of effort required to fix it?
...
Igor Skornyakov
01:22 PM Feature #1587: implement full support for word indexes
What is the problem and do you have an idea about the level of effort required to fix it? Eric Faulhaber
08:14 AM Feature #1587: implement full support for word indexes
Found a regression in the denormalized extents' support.
Working on this.
Igor Skornyakov
07:23 AM Feature #1587: implement full support for word indexes
I've just found a stupid bug in the @schema_word_tables_<db>_<dialect>.sql@ script generation. It is not a problem fo... Igor Skornyakov

03/23/2021

12:17 PM Bug #4163 (Closed): SHARED non-NEW buffer definitions must act as the equivalent of FIND or CREATE
Greg Shah

03/22/2021

10:55 AM Support #5183: database collation
I've committed a newer version of the collation data collector program to 3821c/12160.
The @locale/cp_coll/@ direc...
Eric Faulhaber
05:26 AM Feature #1587: implement full support for word indexes
Re-worked ordering looks a little slower than before. However, it is still much faster than with UDF.
| expr ...
Igor Skornyakov

03/21/2021

05:16 PM Feature #1587: implement full support for word indexes
Implicit sorting for @_temp@ and 'dirty' databases is implemented. Will be committed tomorrow after additional testin... Igor Skornyakov
09:04 AM Feature #1587: implement full support for word indexes
Re-worked implicit ordering for word tables. See #1587-462.
Committed to 3821c/12157.
Igor Skornyakov

03/19/2021

01:18 PM Feature #1587: implement full support for word indexes
I've realized that the implicit ordering described in #1587-407 is not correct and can result in the repeated records... Igor Skornyakov

03/18/2021

11:11 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Igor Skornyakov wrote:
>
> > What about ordering for @_temp@ and 'dirty'?
>
> If you c...
Igor Skornyakov
11:09 AM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> What about ordering for @_temp@ and 'dirty'?
If you can get this done today or tomorro...
Eric Faulhaber
07:54 AM Feature #1587: implement full support for word indexes
A refactored version of the word tables' support and creation of word tables for the @_temp@ database (currently disa... Igor Skornyakov
07:42 AM Feature #1587: implement full support for word indexes
The results of the performance test ("warmed" mode). Last 4 columns are times in ms:
| expr | records | 4...
Igor Skornyakov
05:33 AM Feature #1587: implement full support for word indexes
After adding "warming up" to my @words/words-perf.p@ performance test I've noticed the following. The screen remains ... Igor Skornyakov
04:40 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> In 3821c/12143, I have added the 1688 conversion-time warning and code to remove the subscri...
Igor Skornyakov
04:36 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> I'm sure we can adapt the @orm.Persister@ class to allow for some trigger-like feature which...
Igor Skornyakov
12:01 AM Feature #1587: implement full support for word indexes
In 3821c/12143, I have added the 1688 conversion-time warning and code to remove the subscript from an extent field l... Eric Faulhaber

03/17/2021

06:23 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> Without triggers, we have to add the word tables population/update on every insert/update...
Eric Faulhaber
02:04 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> The current implementation of triggers in H2 seems to be a dealbreaker, in that the trigger ...
Igor Skornyakov
01:53 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> It seems that for H2 temporary tables it is impossible to use triggers. H2 trigger is invok...
Eric Faulhaber
09:47 AM Feature #1587: implement full support for word indexes
It seems that for H2 temporary tables it is impossible to use triggers. H2 trigger is invoked with a separate new con... Igor Skornyakov
08:31 AM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> I've encountered a strange problem with trigger for @_temp@:
> [...]
>
> The @OnInsertW...
Igor Skornyakov
07:30 AM Feature #1587: implement full support for word indexes
I've encountered a strange problem with trigger for @_temp@:... Igor Skornyakov
03:52 AM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> No runtime changes are required. By "will just not work" I mean that the corresponding piec...
Igor Skornyakov
03:30 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Until now, I think yes. I think the other related tables (normalized extent field tables), i...
Igor Skornyakov
03:27 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Sorry, to clarify "will just not work", do you mean that removing the subscript at conversio...
Igor Skornyakov
04:36 PM Feature #3574 (Closed): finish implementation of temp-table XML support
Greg Shah
04:26 PM Feature #3574: finish implementation of temp-table XML support
Yes, please do. I am not aware of any remaining issue. Ovidiu Maxiniuc
02:30 PM Feature #3574: finish implementation of temp-table XML support
Can we close this? Greg Shah
02:29 PM Bug #5176 (Closed): Buffer field attribute leak
Greg Shah

03/16/2021

06:41 PM Feature #1587: implement full support for word indexes
Ovidiu Maxiniuc wrote:
> The only case that comes into my mind is that there is a single statement and the delimiter...
Eric Faulhaber
06:35 PM Feature #1587: implement full support for word indexes
I think the missing of @dialect.getDelimiter()@ is just an omission. I am sorry for that. Please add it in @TempTable... Ovidiu Maxiniuc
06:34 PM Feature #1587: implement full support for word indexes
Actually, we do append the delimiter to the end of the DROP INDEX statements, which are added to the JDBC batch first... Eric Faulhaber
06:26 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> I think it is needed. Without it, the statements are not properly terminated and cannot b...
Eric Faulhaber
06:22 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> Actually, I remove the subscript at runtime (now silently) at a very early stage of the p...
Eric Faulhaber
06:19 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> > Is there any reason why we do not append @dialect.getDelimiter()@ to the @drop table@ stat...
Igor Skornyakov
06:12 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> Is there any reason why we do not append @dialect.getDelimiter()@ to the @drop table@ sta...
Eric Faulhaber
05:40 PM Feature #1587: implement full support for word indexes
Is there any reason why we do not append @dialect.getDelimiter()@ to the @drop table@ statement in the TempTableHelpe... Igor Skornyakov
04:18 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:

> I can make the conversion changes (including adding the 1688 warning), but I wanted to un...
Igor Skornyakov
03:57 PM Feature #1587: implement full support for word indexes
> This is a little bit of a grey area, because the converted output we are talking about is a string containing a con... Greg Shah
03:52 PM Feature #1587: implement full support for word indexes
Greg Shah wrote:
> In other words, it is OK in the input 4GL code but we should only ever generate correct/clean Jav...
Eric Faulhaber
03:39 PM Feature #1587: implement full support for word indexes
> Actually, I do not think that leaving the subscript in the converted code is misleading since the definition of th... Greg Shah
03:05 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> If we fix the conversion to drop the extent field dereference, will the current runtime be a...
Igor Skornyakov
02:14 PM Feature #1587: implement full support for word indexes
If we fix the conversion to drop the extent field dereference, will the current runtime be able to handle the FQL? If... Eric Faulhaber
02:00 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Please refresh my memory...for the 1688 warning, is adding the conversion-time warning the o...
Igor Skornyakov
01:06 PM Feature #1587: implement full support for word indexes
Please refresh my memory...for the 1688 warning, is adding the conversion-time warning the only thing left to be done... Eric Faulhaber
04:01 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Igor, what is left for this task, besides adding the 1688 error to conversion?
Eric,
I u...
Igor Skornyakov
03:56 AM Feature #1587: implement full support for word indexes
Igor, what is left for this task, besides adding the 1688 error to conversion? Eric Faulhaber

03/15/2021

03:00 PM Feature #1587: implement full support for word indexes
* Re-worked implicit ordering to be compatible with one for @CONTAINS@ with UDF described in #1587-423
* Refactored ...
Igor Skornyakov
03:29 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> One question: for a given record, is there any efficiency/performance difference evaluating ...
Igor Skornyakov
01:35 PM Bug #5176: Buffer field attribute leak
Eric Faulhaber wrote:
> Code review 3821c/12108:
> Generally, the changes look good to me, though the change was so...
Ovidiu Maxiniuc

03/14/2021

05:27 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> Hopefully is not not just a 'late night idea' ))
Well, it seems like an interesting on...
Eric Faulhaber

03/12/2021

05:36 PM Feature #1587: implement full support for word indexes
BTW. I've just realized that it is possible to add a 'natural' implicit ordering to the queries using the @CONTAINS@ ... Igor Skornyakov
02:27 PM Feature #1587: implement full support for word indexes
I'm ok with using GUIDs. Eric Faulhaber
02:25 PM Feature #1587: implement full support for word indexes
Greg Shah wrote:
> Do dynamic temp-tables have names?
Yes, they are referenced in subsequent dynamic queries by n...
Eric Faulhaber
02:20 PM Feature #1587: implement full support for word indexes
Greg Shah wrote:
> Do dynamic temp-tables have names?
If they are in the H2 database they should have at least so...
Igor Skornyakov
02:16 PM Feature #1587: implement full support for word indexes
Do dynamic temp-tables have names? Greg Shah
02:10 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Igor Skornyakov wrote:
> When you say "ensure that data objects created for word tables are...
Igor Skornyakov
02:09 PM Feature #1587: implement full support for word indexes
Some google searches suggest '@@@' is a valid character in an SQL table name, but not in a 4GL table name. This will ... Eric Faulhaber
01:59 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> Another question. I need to ensure that data objects created for word tables are unique, ev...
Eric Faulhaber
01:45 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Yes, the dirty database should work properly with extent fields, if we go with the UDF appro...
Igor Skornyakov
01:42 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> So I have to add a check for @isDirty@ and do not re-write the query for the 'dirty' data...
Eric Faulhaber
01:32 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Igor Skornyakov wrote:
> > What is 'dirty' database and should I consider such databases fo...
Igor Skornyakov
01:22 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> What is 'dirty' database and should I consider such databases for the word tables support?
...
Eric Faulhaber
05:28 AM Feature #1587: implement full support for word indexes
Another question. I need to ensure that data objects created for word tables are unique, even in the scope of a sessi... Igor Skornyakov
05:19 AM Feature #1587: implement full support for word indexes
Sorry for my ignorance. What is 'dirty' database and should I consider such databases for the word tables support?
T...
Igor Skornyakov
04:17 PM Bug #5056 (Closed): Temp-tables and unique indexes
Eric Faulhaber
04:00 PM Bug #5056: Temp-tables and unique indexes
The validation works correctly for temp-tables now. It does not really make sense why OE shows error 132 multiple tim... Ovidiu Maxiniuc
11:39 AM Bug #5176: Buffer field attribute leak
Code review 3821c/12108:
Generally, the changes look good to me, though the change was so dense in @TableMapper@, ...
Eric Faulhaber

03/11/2021

02:41 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Just saw this in the H2 mailing list, FWIW. Noel is one of the main contributors to the proj...
Igor Skornyakov
02:30 PM Feature #1587: implement full support for word indexes
Just saw this in the H2 mailing list, FWIW. Noel is one of the main contributors to the project. The OP was reporting... Eric Faulhaber
07:49 AM Bug #5121: Converted temp table should be using the legacy name instead of the converted name in ...
Eric Faulhaber wrote:
> Ovidiu, can you think of an efficient way to give access to @SharedVariableManager.lookupTem...
Ovidiu Maxiniuc
09:22 PM Bug #5121: Converted temp table should be using the legacy name instead of the converted name in ...
Roger Borrello wrote:
> Greg Shah wrote:
> > I think the legacy name can be found in @expected@. It is the DMO we ...
Eric Faulhaber

03/10/2021

04:27 PM Bug #5176 (WIP): Buffer field attribute leak
I committed the fix in 3821c as revision 12108. The update is a aggregate, it contains fixes for other smaller issues... Ovidiu Maxiniuc
04:08 PM Bug #5121: Converted temp table should be using the legacy name instead of the converted name in ...
Eric? Greg Shah
03:51 PM Bug #5121: Converted temp table should be using the legacy name instead of the converted name in ...
Greg Shah wrote:
> I think the legacy name can be found in @expected@. It is the DMO we are searching for, which in...
Roger Borrello
01:17 PM Bug #5121: Converted temp table should be using the legacy name instead of the converted name in ...
I think the legacy name can be found in @expected@. It is the DMO we are searching for, which inherently maps 1-to-1... Greg Shah
12:02 PM Bug #5121: Converted temp table should be using the legacy name instead of the converted name in ...
At this point in processing I don't think we know the legacy name (since we haven't found it).
From @SharedVariabl...
Roger Borrello
01:44 PM Bug #4173 (Closed): BUFFER-COPY/BUFFER-COMPARE compares table fields by name rather than data typ...
Greg Shah
01:29 PM Bug #4173 (WIP): BUFFER-COPY/BUFFER-COMPARE compares table fields by name rather than data type a...
This can be closed; the premises in #4173 are not accurate as there is a difference in implementation between @bufer-... Adrian Lungu
11:01 AM Bug #4173: BUFFER-COPY/BUFFER-COMPARE compares table fields by name rather than data type and order
Is this still an open issue? Greg Shah
01:42 PM Feature #3958 (Closed): BUFFER-COPY optimization
Greg Shah
01:22 PM Feature #3958: BUFFER-COPY optimization
This can be closed; the BUFFER-COPY implementation reached all the optimizations proposed here. Adrian Lungu
11:00 AM Feature #3958: BUFFER-COPY optimization
What is left to do on this task? Greg Shah
01:26 PM Feature #4011: database/persistence layer performance improvements
We have the idea in #4011-774 implemented as @DmoSignature@. Adrian Lungu
11:01 AM Feature #4011: database/persistence layer performance improvements
Is #4011-774 complete? Greg Shah
01:19 PM Feature #4055: optimize temp-table output parameter copying
#4055-63 keeps track of the things left to do. The opened ones are still some possible directions, which were explore... Adrian Lungu
10:59 AM Feature #4055: optimize temp-table output parameter copying
What is left to do on this task? Greg Shah
11:21 AM Feature #1587: implement full support for word indexes
The final logic for implicit sorting appeared to be more tricky than I've described initially. In particular, a situa... Igor Skornyakov

03/09/2021

05:04 PM Feature #1587: implement full support for word indexes
Ovidiu Maxiniuc wrote:
> These two classes are really specialised:
> * the @DDLGeneratorWorker@ is used by static c...
Igor Skornyakov
05:02 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Not as such. The documentation on FWD internals is the least developed of the various books....
Igor Skornyakov
04:25 PM Feature #1587: implement full support for word indexes
These two classes are really specialised:
* the @DDLGeneratorWorker@ is used by static conversion to get DDLs for pe...
Ovidiu Maxiniuc
03:39 PM Feature #1587: implement full support for word indexes
Not as such. The documentation on FWD internals is the least developed of the various books.
The code which create...
Eric Faulhaber
03:15 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> OK, please post any questions you may have about the temp-table creation, DDL, etc. Ovidiu o...
Igor Skornyakov
03:07 PM Feature #1587: implement full support for word indexes
OK, please post any questions you may have about the temp-table creation, DDL, etc. Ovidiu or I will be able to get y... Eric Faulhaber
01:57 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Igor Skornyakov wrote:
> Given that support for H2 already is implemented, what is the incr...
Igor Skornyakov
01:02 PM Feature #1587: implement full support for word indexes
Igor Skornyakov wrote:
> According to #1587-327, you wanted also to implement word tables for @_temp@ db.
Given...
Eric Faulhaber
11:58 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Igor, other than finishing the sort implementation (and gauging its performance) and the 168...
Igor Skornyakov
11:50 AM Feature #1587: implement full support for word indexes
Igor, other than finishing the sort implementation (and gauging its performance) and the 1688 conversion warning, wha... Eric Faulhaber
02:48 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> W.r.t. the command line options, we don't want to use system properties long term. For any o...
Igor Skornyakov
02:44 PM Bug #5176: Buffer field attribute leak
I have really thought of this issue during last days.
The @DmoMeta@ carries immutable data, that is: information ...
Ovidiu Maxiniuc
11:53 AM Bug #5176: Buffer field attribute leak
Ovidiu, if every buffer has its own writable version of a certain set of attributes, does it make sense for each buff... Eric Faulhaber
11:47 AM Support #5183: database collation
Some additional research to consider:
*1.* Locales
A "locale":https://en.wikipedia.org/wiki/Locale_(computer_so...
Greg Shah

03/08/2021

05:54 PM Feature #1587: implement full support for word indexes
Code review 3821c, rev 12079, 12089:
The logic changes look good to me. I have not tested them.
W.r.t. the comm...
Eric Faulhaber
03:35 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Another question/concern: it seems like this approach requires the CTE. It follows that it w...
Igor Skornyakov
03:31 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> In the absence of a BY clause, I don't know of any guaranteed ordering of the results of a q...
Igor Skornyakov
11:03 PM Feature #1587: implement full support for word indexes
Another question/concern: it seems like this approach requires the CTE. It follows that it would disrupt your more pe... Eric Faulhaber
10:46 PM Feature #1587: implement full support for word indexes
This is a very interesting proposal, and I appreciate the creativity in coming up with it.
In the absence of a BY ...
Eric Faulhaber

03/07/2021

10:57 AM Feature #1587: implement full support for word indexes
I doubt that it is possible to implement an implicit ordering imposed by @CONTAINS@ which is 100% compatible with 4GL... Igor Skornyakov
10:25 AM Feature #1587: implement full support for word indexes
I've created the following test to analyze the implicit sorting imposed by @CONTAINS@ (see @testcases/words/words-ord... Igor Skornyakov

03/06/2021

02:41 PM Feature #1587: implement full support for word indexes
The problem with ... Igor Skornyakov
08:21 AM Feature #1587: implement full support for word indexes
Added CNF ordering (see #1576-386).
Committed to 3821c/12089
Igor Skornyakov
05:10 AM Feature #1587: implement full support for word indexes
I understand that the order of terms in the @CONTAINS@@ expression is important for implicit sorting. So I'm going to... Igor Skornyakov
11:24 AM Support #5183: database collation
In the "Internationalize ABL Application" docs there are good details about codepages and all the character configura... Greg Shah

03/05/2021

05:17 PM Support #5183: database collation
The attached program (@cp_coll.p@) was written as a starting point to review the 4GL's collation, inspired by the exi... Eric Faulhaber
05:05 PM Support #5183: database collation
A review of how FWD collates its database data has raised some questions as to whether we are doing it correctly for ... Eric Faulhaber
05:00 PM Support #5183 (WIP): database collation
Eric Faulhaber

03/04/2021

08:02 AM Bug #5176: Buffer field attribute leak
My understanding is that in the 4GL every temp-table *instance* is a completely separate table. Multiple table insta... Greg Shah
08:10 PM Bug #5176: Buffer field attribute leak
I will try to summarize an ugly issue I am investigating lately.
I was working on a issue related to #3574-11 and ...
Ovidiu Maxiniuc
07:07 PM Bug #5176 (Closed): Buffer field attribute leak
Ovidiu Maxiniuc
06:29 AM Feature #1587: implement full support for word indexes
I've noticed that queries with @CONTAINS@ run faster with CTE when the expression is 'simple' (its CNF contains only ... Igor Skornyakov

03/03/2021

03:46 PM Feature #1587: implement full support for word indexes
Added SQL re-writing using common table expressions (WITH clause).
Committed to 3821c/12077.
For PostgreSQL, it i...
Igor Skornyakov

03/02/2021

02:41 AM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> Well, it is extra work to add it in @HQLPreprocessor@ and I imagine there probably is some w...
Igor Skornyakov

03/01/2021

05:59 PM Feature #1587: implement full support for word indexes
Well, it is extra work to add it in @HQLPreprocessor@ and I imagine there probably is some work to ignore it when gen... Eric Faulhaber
05:19 PM Feature #1587: implement full support for word indexes
Eric Faulhaber wrote:
> The RTRIM is injected for text property references by the @HQLPreprocessor@. It is needed fo...
Igor Skornyakov
05:09 PM Feature #1587: implement full support for word indexes
The RTRIM is injected for text property references by the @HQLPreprocessor@. It is needed for normal text property eq... Eric Faulhaber
04:44 PM Feature #1587: implement full support for word indexes
I've noticed some strange thing.
Consider the following 4GL code:...
Igor Skornyakov
 

Also available in: Atom