FWD » Core Development » Database

02/20/2024

07:57 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

After testing POC with JMXs I found out that @(select count()) = 1@ is used often and a lot more than with @FIRST@ or... Dănuț Filimon

03:13 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

I've made a few examples to check if there are other methods used and got the following:
* @CAN-FIND(pt1)@ generated... Dănuț Filimon

02/19/2024

02:13 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Checking for @exists@ in the FQL seems like a reasonable approach. Right now, it is only used to implement server-sid... Eric Faulhaber

02/15/2024

04:23 AM Feature #6628 (WIP): port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> OK, please go ahead and implement my suggestions. That is:
>
> * augment @FQLPreprocessor... Dănuț Filimon

02/14/2024

07:30 AM Feature #6720: lazy hydration

I changed @BaseRecord.copy@, to set @hydrator@ and @liveProps@ of the new record ... Eduard Soltan

04:02 AM Feature #6720: lazy hydration

Lets go ahead with some statistics. From where is @getData@ called and how many times. Please find the back traces of... Alexandru Lungu

02:09 AM Feature #6720: lazy hydration

On delete of a temporary buffer and a deleted buffer is placed in before table.
But to make this copy a snapshot o... Eduard Soltan

02/13/2024

02:22 PM Feature #6720: lazy hydration

Short update from a live discussion with Eduard: the regression seems to be related with the @BaseRecord.getData@ cal... Alexandru Lungu

02/12/2024

02:39 AM Bug #7496: finish support for query:forward-only attribute

Rebased *7496a* with *trunk/rev.14981*. *7496a* is now at revision *14991*. Dănuț Filimon

02:29 AM Bug #7334 (Test): Reclaim used sessions to improve performance

Dănuț Filimon

02:29 AM Bug #7334 (Internal Test): Reclaim used sessions to improve performance

Alexandru Lungu wrote:
> This reached trunk a while ago. Is there anything left on this task? If not, please move it... Dănuț Filimon

02:26 AM Bug #7334: Reclaim used sessions to improve performance

This reached trunk a while ago. Is there anything left on this task? If not, please move it to test. Alexandru Lungu

02/09/2024

10:36 AM Bug #7924: add support for creating an empty database

Issue has been fixed in *7924a.rev.14794*. All field types can be augmented with @__null@ for @MariaDbDialect@ when t... Dănuț Filimon

04:57 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

Alexandru Lungu wrote:
> Eduard Soltan wrote:
> > For now in we assign to @updateLock@ the following expression: @u... Eduard Soltan

04:04 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

Eduard Soltan wrote:
> For now in we assign to @updateLock@ the following expression: @updateLock = (lt != LockType.... Alexandru Lungu

03:58 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

For now in we assign to @updateLock@ the following expression: @updateLock = (lt != LockType.NONE);@.
I think that... Eduard Soltan

03:59 AM Bug #7999: FWD does not honor FIELDS/EXCEPT at dynamic queries

Ovidiu Maxiniuc wrote:
> ** line 1368: @byLegacyName()@ requires that the argument to be normalized (lowercase). Yo... Eduard Soltan

02/08/2024

02:14 PM Bug #8249: Introduce a fast fill method to optimize Dataset.fill

I discussed with Alexandru [ad] and there is a reasonable observation that the fill queries are usual @AdaptiveQuery@... Alexandru Lungu

09:20 AM Bug #8249: Introduce a fast fill method to optimize Dataset.fill

A little note:
@DataSet.fill@ calls @BufferImpl.fill@ which calls @FillWorker.fill@ (class inside @BufferImpl@), whi... Alexandru Donica

11:00 AM Bug #7924: add support for creating an empty database

Dănuț,
@sql_only_user__null@ must be generated.
This thing seemed familiar to me. I did a bit of research and ... Ovidiu Maxiniuc

10:12 AM Bug #7999: FWD does not honor FIELDS/EXCEPT at dynamic queries

Review of 7999a/14954-14958.
The code looks promising, most of the issues below are low priority. but there are a co... Ovidiu Maxiniuc

09:08 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

Eduard, this basically means that the condition in #8259-2 is incorrect.
Please post here a cut-out to the problem... Alexandru Lungu

08:49 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

Alexandru Lungu wrote:
> I wonder if NO-LOCK to SHARE-LOCK is considered an lock update.
It is. Any change in loc... Eric Faulhaber

08:03 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

I wonder if NO-LOCK to SHARE-LOCK is considered an lock update. Alexandru Lungu

07:20 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

Alexandru Lungu wrote:
> My whole concern here is that we do an extra session cache look-up for almost any record ... Eduard Soltan

05:55 AM Bug #8259: Reduce the work done by PreselectQuery.coreFetch

Eduard detected recently that there is some work being done in @PreselectQuery.coreFetch@ regarding an extra fetch.
... Alexandru Lungu

04:29 AM Bug #8259 (New): Reduce the work done by PreselectQuery.coreFetch

Alexandru Lungu

09:01 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

In terms of JMX, results look promising. Out of a million of "Select all" type of queries, the @DirectAccess@ method ... Radu Apetrii

06:42 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

Radu, if the solution is stable. Lets get some profiling results asap.
Please go in-depth with some JMX to check the... Alexandru Lungu

06:35 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

Alexandru Lungu wrote:
> You can have compound queries. A component can be instantiated as an @AdaptiveQuery@. See @... Radu Apetrii

05:24 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

You can have compound queries. A component can be instantiated as an @AdaptiveQuery@. See @for each tt, each tt2@, as... Alexandru Lungu

05:09 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

Alex/Greg: Are there any other ways of generating a "Select all" type of *static* query that turns into an @AdaptiveQ... Radu Apetrii

07:19 PM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

I've committed to:
* *7252b rev. 14934* a change for better handling of index choosing in @AdaptiveQuery.getAllLazyD... Radu Apetrii

02/07/2024

11:53 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

Greg Shah wrote:
> It seems better to "remember" this information than to calculate it at runtime.
I'll add this id... Radu Apetrii

04:46 AM Bug #8251: Fine tune CompoundQuery optimization when contains clauses are used

This is a follow up of #8246.
Compound queries using @contains@ clause are no longer optimized:
* Check if this c... Alexandru Lungu

04:43 AM Bug #8251 (New): Fine tune CompoundQuery optimization when contains clauses are used

Alexandru Lungu

02/06/2024

10:34 AM Bug #8249: Introduce a fast fill method to optimize Dataset.fill

This task is aimed to optimize the @Dataset.fill@ implementation as it represents a bottle-neck in a large customer a... Alexandru Lungu

10:00 AM Bug #8249 (New): Introduce a fast fill method to optimize Dataset.fill

Alexandru Lungu

08:49 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

For static queries, we know the 4GL name of the selected index at conversion time. We can easily pass that to the qu... Greg Shah

07:26 AM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

For reference, if one were to assume that an index called @idx2@ would be created for table @tt@, then the following ... Radu Apetrii

11:02 PM Bug #7252: Use direct-access to support for-each queries without WHERE and BY clauses

I've finally managed to isolate that NPE issue I encountered. Long story short (I say "short" because I had a really ... Radu Apetrii

03:21 AM Bug #7924: add support for creating an empty database

Ovidiu, the problem does not start from the index, but from the table field. If you take a look at @DDLGeneratorWorke... Dănuț Filimon

01:07 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

OK, please go ahead and implement my suggestions. That is:
* augment @FQLPreprocessor@ to detect and expose whethe... Eric Faulhaber

02/05/2024

02:35 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

It sounds like these suuggested changes are simple and low risk. I don't think we have to quantify the benefit in th... Greg Shah

11:56 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

OK, thanks. The change seems to make logical sense, but it got me looking at this part of the code again, and I am a ... Eric Faulhaber

10:59 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> Dănuț Filimon wrote:
> > There's a single change in @SQLQuery.resetErrorHandler@ that shoul... Dănuț Filimon

10:53 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Dănuț Filimon wrote:
> There's a single change in @SQLQuery.resetErrorHandler@ that should be mentioned. Instead of ... Eric Faulhaber

03:18 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

There's a single change in @SQLQuery.resetErrorHandler@ that should be mentioned. Instead of calling the dialect spec... Dănuț Filimon

03:14 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

6628b is already in trunk and reached 7156b, so functionally we are OK in 7156b. 6628c can go in trunk without confli... Alexandru Lungu

03:10 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric, do you still want 6628c to be merged? Dănuț Filimon

02:10 PM Bug #7924: add support for creating an empty database

Review of 7924a/rev.14793.
The code seems fine, except two issues:
1. In case of nullable fields, why injection o... Ovidiu Maxiniuc

11:10 AM Bug #7924 (Review): add support for creating an empty database

*Committed 7924a/rev.14793*.
Based on the error from #7924-14 I noticed that the column does not exist because it ... Dănuț Filimon

03:56 AM Bug #7924: add support for creating an empty database

The changes work well, I tested conversion with hotel_gui and compared the index ddls for mariadb (before and after).... Dănuț Filimon

02/02/2024

12:04 PM Feature #7382: Check performance of delete from vs drop table in H2

Done Alexandru Lungu

12:02 PM Feature #7382: Check performance of delete from vs drop table in H2

Alexandru Lungu wrote:
> Right, it does! I uploaded @devsrv01:/tmp/fwd-h2-1.40-trunk.jar@. Please upload to public r... Constantin Asofiei

11:59 AM Feature #7382: Check performance of delete from vs drop table in H2

Right, it does! I uploaded @devsrv01:/tmp/fwd-h2-1.40-trunk.jar@. Please upload to public repository.
We can then up... Alexandru Lungu

11:55 AM Feature #7382: Check performance of delete from vs drop table in H2

Alexandru Lungu wrote:
> Branch 7382a was merged into trunk as rev 14966 and archived.
Doesn't this require update ... Constantin Asofiei

11:52 AM Feature #7382: Check performance of delete from vs drop table in H2

Branch 7382a was merged into trunk as rev 14966 and archived. Alexandru Lungu

09:29 AM Feature #7382: Check performance of delete from vs drop table in H2

Don't we need to get 7382a into trunk to take advantage of the improvements? Greg Shah

11:36 AM Feature #6496 (Test): add equivalent support for -trig command line option

Branch 6496b was merged into trunk as rev 14965 and archived. Alexandru Lungu

09:17 AM Feature #6496 (Merge Pending): add equivalent support for -trig command line option

Constantin Asofiei

02:29 AM Bug #7924: add support for creating an empty database

Ovidiu Maxiniuc wrote:
> I think you can access the @not-null@ annotation when configuring the index components.
> ... Dănuț Filimon

12:22 AM Bug #7924: add support for creating an empty database

I think you can access the @not-null@ annotation when configuring the index components.
* in index components are ... Ovidiu Maxiniuc

02:15 AM Bug #7496: finish support for query:forward-only attribute

After taking the minimal test case and looking into it, I noted the following:
* with no changes, the cursor already... Dănuț Filimon

01:02 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

The fix for #6628-214 and my mentions from #6628-215 were committed to *6628c/rev.14964*. Dănuț Filimon

02/01/2024

05:04 PM Feature #7382 (Test): Check performance of delete from vs drop table in H2

That is great. These changes are already in the H2 trunk - I will move this into Test. Alexandru Lungu

08:47 AM Feature #7382: Check performance of delete from vs drop table in H2

I ran the tests, and there were no more tests that failed compared to the baseline. Alexandru Donica

05:03 PM Feature #6496: add equivalent support for -trig command line option

AFAIK, these were tested on a large customer application regression tests and POC by actually setting the flag and de... Alexandru Lungu

09:26 AM Feature #6496 (Internal Test): add equivalent support for -trig command line option

What testing remains? Greg Shah

01:28 PM Bug #7924: add support for creating an empty database

The order in the ruleset does not matter too much (in this case), but the structure of the tree will drive the rules ... Greg Shah

05:25 AM Bug #7924: add support for creating an empty database

I've looked into @DDLGeneratorWorker@ and how we can make use of the @mandatory@ and while debugging the conversion I... Dănuț Filimon

11:47 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Dănuț Filimon wrote:
> POC regression testing is ok with those changes.
Danut, in which branch are these fixes? C... Eric Faulhaber

04:49 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

POC regression testing is ok with those changes. Dănuț Filimon

03:07 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

I've looked into #6628-214 and fixed all the problems except the renamed file. I also looked in detail at all classes... Dănuț Filimon

07:46 AM Bug #8196 (Test): Reduce number of AbstractTempTable._hasRecords calls to avoid BufferManager.act...

Branch 8196a was merged into trunk as rev.14961 and archived. Dănuț Filimon

07:28 AM Bug #8196 (Merge Pending): Reduce number of AbstractTempTable._hasRecords calls to avoid BufferMa...

8196a can merge after 8107a. Greg Shah

03:29 AM Bug #8196: Reduce number of AbstractTempTable._hasRecords calls to avoid BufferManager.activeBuffers

This is good news. The changes are quite safe from my POV.
This is ready for merge. Greg, this is a performance im... Alexandru Lungu

01/31/2024

04:09 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

There are still some things to fix in r14958:
* both @FWDPostgreSQLConnectionCustomizer@ and @FWDMariaDBConnectionCu... Ovidiu Maxiniuc

10:34 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Branch 6628b was merged to trunk as rev.14958 and archived. Dănuț Filimon

09:45 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

> Create 6628b, hotfix and merge it?
Yes, go ahead now. Greg Shah

08:48 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

I've made a mistake while rebasing 6628a and changed @int rowSize = rowStructures.size();@ to @int rowSize = rowStruc... Dănuț Filimon

06:53 AM Feature #6628 (Test): port PostgreSQL native user defined functions to MariaDB

Branch 6628a was merged to trunk as rev.14956 and archived. Dănuț Filimon

06:30 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

You can merge now. Greg Shah

02:25 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

@guarded.sql@ is the latest version obtained after executing the command mentioned in #6628-202.
Greg, let me know... Dănuț Filimon

11:20 AM Feature #6496: add equivalent support for -trig command line option

Sorry for mistaking the files.
Commited change to branch 6496b rev 14945. Alexandru Donica

09:58 AM Feature #6496: add equivalent support for -trig command line option

Code review 6496b. The @SourceNameMapper@ changes look good. Just please update the copyright year in @StartupParamet... Hynek Cihlar

06:34 AM Feature #6496: add equivalent support for -trig command line option

Ovidiu Maxiniuc wrote:
> +Review of 6496b rev 14943.+
> * @DatabaseTriggerManager.java@
> ** line 270: if the co... Alexandru Donica

06:32 AM Feature #6496: add equivalent support for -trig command line option

Hynek: Please review the @SourceNameMapper@ changes. Greg Shah

04:38 AM Feature #6496: add equivalent support for -trig command line option

Alexandru [ad], please address the review. We need to get this merged asap. Alexandru Lungu

07:07 AM Bug #8196: Reduce number of AbstractTempTable._hasRecords calls to avoid BufferManager.activeBuffers

POC regression tests passed and the results of the performance tests showed an improvement of *5.27%* for the average... Dănuț Filimon

04:37 AM Feature #7382 (Internal Test): Check performance of delete from vs drop table in H2

I am OK with the changes; merged 7382a_h2 to FWD-H2 trunk as rev. 40. Archived 7382a_h2.
Alexandru [ad], please pi... Alexandru Lungu

01/30/2024

05:59 PM Feature #6496: add equivalent support for -trig command line option

+Review of 6496b rev 14943.+
* the copyright year must be updated (and the branch rebased) in each altered file;
... Ovidiu Maxiniuc

03:50 PM Feature #6496: add equivalent support for -trig command line option

Alexandru Donica wrote:
> Commited to branch 6496b rev 14943.
Ovidiu, please review, at least the trigger impleme... Eric Faulhaber

05:39 PM Feature #6628 (Merge Pending): port PostgreSQL native user defined functions to MariaDB

If @guarded.sql@ was manually committed before, please do the same with the latest version generated from @udfs.sql@ ... Eric Faulhaber

12:50 PM Feature #6825: improve table meta runtime (TableMapper and other)

Alexandru [ad], AFAIK, you already have some changes on @TableMapper@ to improve @BufferFieldImpl.asP2JField@. This i... Alexandru Lungu

11:12 AM Bug #8214: Split Persister into SQLPersister and InMemoryPersister

I attempted to test my changes on 8214a and 8214a_h2 but got no viable improvement. Interestingly, 80% of the updates... Alexandru Lungu

10:01 AM Bug #7167 (Closed): Associating records from opened buffers to new sessions is slow

I understand the work begun here is being continued in #8196. Eric Faulhaber

08:31 AM Bug #8196: Reduce number of AbstractTempTable._hasRecords calls to avoid BufferManager.activeBuffers

Alexandru Lungu wrote:
> The changes look good to me!
> Danut, you can inline @isActiveBuffer@ by doing @return <la... Dănuț Filimon

08:19 AM Bug #8196 (Internal Test): Reduce number of AbstractTempTable._hasRecords calls to avoid BufferMa...

The changes look good to me!
Danut, you can inline @isActiveBuffer@ by doing @return <large-if-condition> && buffer.... Alexandru Lungu

07:06 AM Bug #8196: Reduce number of AbstractTempTable._hasRecords calls to avoid BufferManager.activeBuffers

*Committed 8196a/rev.14937*. I've created a separate method to check each buffer individually and it can be used dire... Dănuț Filimon

06:26 AM Bug #8196: Reduce number of AbstractTempTable._hasRecords calls to avoid BufferManager.activeBuffers

Note that @activeBuffers@ iterates @RecordBuffer@ while we want to iterate @Buffer@ instances, so those need to be ca... Dănuț Filimon

05:38 AM Bug #8196: Reduce number of AbstractTempTable._hasRecords calls to avoid BufferManager.activeBuffers

I suspect the code duplicate to be quite bad architecturally.
Lets overload @activeBuffers(Persistence, boolean)@ ... Alexandru Lungu

04:55 AM Bug #7496: finish support for query:forward-only attribute

Alexandru Lungu wrote:
> Danut, can you cut-out it even more.
>
> 1. Please check if this can be reproduced with... Dănuț Filimon

04:14 AM Bug #7496: finish support for query:forward-only attribute

Danut, can you cut-out it even more.
1. Please check if this can be reproduced with temp-tables - it is a bit eas... Alexandru Lungu

03:48 AM Bug #7496: finish support for query:forward-only attribute

*Rebased 7496a* with *trunk/rev.14950*. *7496a* is not at revision *14959*. I also committed the adjustments which fi... Dănuț Filimon

04:43 AM Bug #7999: FWD does not honor FIELDS/EXCEPT at dynamic queries

Alexandru Lungu wrote:
> I see changes in @database_access.rules@. Are these mandatory and require reconversion?
... Eduard Soltan

04:37 AM Bug #7999: FWD does not honor FIELDS/EXCEPT at dynamic queries

Eduard, I rebased 7999a to latest trunk. It is now at rev. 14958. Please do the testing of it with POC and a large cu... Alexandru Lungu

04:19 AM Bug #7999: FWD does not honor FIELDS/EXCEPT at dynamic queries

Committed on 7999a, revision 14837.
Addressed review in #7999-7. Also for second case in #7999-1, with @DEFINE QUE... Eduard Soltan

02:31 AM Bug #7924 (WIP): add support for creating an empty database

Ovidiu Maxiniuc wrote:
> Indeed, the @addr_id integer not null@ column should not create @(addr_id_null, addr_id)@ p... Dănuț Filimon

01/29/2024

05:09 PM Bug #7924: add support for creating an empty database

Dănuț Filimon wrote:
> Ovidiu Maxiniuc wrote:
> > As noted in my previous note, this is the cause of the failure: t... Ovidiu Maxiniuc

05:05 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Alexandru Lungu wrote:
> Danut, any other testing required?
There is no testing left that I can do. Currently ChUI ... Dănuț Filimon

04:58 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Danut, any other testing required?
Beside the history entry fixes, what about the @guarded.sql@ - was it generated... Alexandru Lungu

01/26/2024

10:50 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Can we merge this to trunk? Greg Shah

10:06 AM Bug #7999: FWD does not honor FIELDS/EXCEPT at dynamic queries

Eduard, please address the review to get this done. Alexandru Lungu

05:38 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

I retested the changes (the patch from #6714-14) but had to make two small adjustments:
* The usage of the @Procedur... Dănuț Filimon