FWD » Core Development » Database

09/14/2022

03:33 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> I do not understand. There is nothing particular to those functions. I did a quick test and... Igor Skornyakov

03:29 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

I do not understand. There is nothing particular to those functions. I did a quick test and the functions are correct... Ovidiu Maxiniuc

02:35 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

In the converted queries there are no 'signature suffix' for @indexOf@ and @lookup@.
Ovidiu, I undestand that the ... Igor Skornyakov

05:08 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

There is a problem with creation of @SQLWarning@ with MariaDB. Unlike PostgreSQL @RAISE NOTICE@, @SIGNAL SQLSTATE '01... Igor Skornyakov

11:29 AM Support #6713: check if it is faster to use separate temp-table instances instead of multiplex id

Please consider several "abstract" testcases (eventually with @etime@) which can show clear time differences in diffe... Alexandru Lungu

11:12 AM Feature #6582: implement multi-table AdaptiveQuery

I committed 6582a/rev. 14164 including all the latest changes done to increase stability of the multi-table @Adaptive... Alexandru Lungu

09/13/2022

07:27 PM Feature #6457: finish COPY-LOB support

> The BOM handling has a testcase already written. See @testcases/copy_lob/tests/bom.p@.
OK, the first issue I me... Stanislav Lomany

07:04 AM Feature #6457: finish COPY-LOB support

The BOM handling has a testcase already written. See @testcases/copy_lob/tests/bom.p@. Greg Shah

01:49 PM Feature #6371: implement SAVE CACHE statement

Ovidiu Maxiniuc wrote:
> Eric Faulhaber wrote:
> > What I am not sure about is from where we gather the legacy info... Eric Faulhaber

01:27 PM Feature #6371: implement SAVE CACHE statement

Eric Faulhaber wrote:
> What I am not sure about is from where we gather the legacy information (i.e., the informati... Ovidiu Maxiniuc

01:38 AM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> I updated @DatabaseManager@ to handle registration of the new interface.
I didn't really... Eric Faulhaber

01:16 PM Feature #6582: implement multi-table AdaptiveQuery

The non-scrolling query examples that were working on 3821c are now working properly on 6582a as well. Note that I mo... Alexandru Lungu

10:30 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> Any issues coming out of testing? Any headway on the error handling solution? Thanks.
Not... Igor Skornyakov

10:09 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Any issues coming out of testing? Any headway on the error handling solution? Thanks. Eric Faulhaber

10:16 AM Feature #6453 (WIP): temp-table features

Eric Faulhaber

09/12/2022

04:05 PM Feature #6371: implement SAVE CACHE statement

I updated @DatabaseManager@ to handle registration of the new interface. I am still working on removal of the last ca... Boris Schegolev

12:27 PM Support #6679: H2 general performance tuning

Is the H2 storage engine API implemented in a clean manner or is the storage engine implemented as pieces of code spr... Greg Shah

08:39 AM Support #6679: H2 general performance tuning

> I wonder if there is any case in which a physical 4GL user can have multiple transactions to one single temporary t... Greg Shah

08:35 AM Support #6679: H2 general performance tuning

Greg Shah wrote:
> Clearly, we cannot move ahead with the new version of H2. At least we cannot without something th... Alexandru Lungu

05:44 AM Support #6679: H2 general performance tuning

Clearly, we cannot move ahead with the new version of H2. At least we cannot without something that changes the perf... Greg Shah

05:24 AM Support #6679: H2 general performance tuning

Eric Faulhaber wrote:
> I don't know how the tests were performed for the bottom table, which actually is the more ... Dănuț Filimon

10:10 AM Feature #6348: implement support for MariaDB

I've found the following tool which can be useful - https://www.linuxbabe.com/ubuntu/mysql-mariadb-database-performan... Igor Skornyakov

06:11 AM Feature #6348: implement support for MariaDB

Igor Skornyakov wrote:
> It looks that we need to use @timestamp(3)@/@datetime(3)@ instead of @timestamp@/@datetime@... Ovidiu Maxiniuc

05:07 AM Feature #6453: temp-table features

6129a Greg Shah

04:53 AM Feature #6453: temp-table features

Ovidiu,
Which branch should I use for this task?
Thank you. Igor Skornyakov

09/09/2022

12:13 PM Bug #6744: rollback when there are catch/finally blocks opening their own transactions

I'm seeing an abend with a customer app where a savepoint is no longer valid (when trying to rollback-/commit-). I'v... Constantin Asofiei

12:08 PM Bug #6744 (New): rollback when there are catch/finally blocks opening their own transactions

Constantin Asofiei

12:08 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor, I think @PRIMARY@ has already been implemented in branch 6129a. I think that is your ... Igor Skornyakov

12:06 PM Feature #6453: temp-table features

Igor, I think @PRIMARY@ has already been implemented in branch 6129a. I think that is your target. Anyway that branch... Ovidiu Maxiniuc

12:30 AM Feature #6453: temp-table features

Igor, please see #6453-3 and ask Ovidiu any questions you have. Eric Faulhaber

12:05 PM Bug #6699 (Review): TemporaryBuffer$Multiplexer memory leak in global block

The fix for this is in #6641-34, 6129a rev 14388. Constantin Asofiei

10:19 AM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

Eric Faulhaber wrote:
> Constantin Asofiei wrote:
> > I think I have a better fix for this leak; in @TemporaryBuffe... Constantin Asofiei

10:19 AM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

Constantin Asofiei wrote:
> I think I have a better fix for this leak; in @TemporaryBuffer.initialize@ do this:
> [... Eric Faulhaber

05:49 AM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

I think I have a better fix for this leak; in @TemporaryBuffer.initialize@ do this:... Constantin Asofiei

10:17 AM Feature #6582: implement multi-table AdaptiveQuery

I started using the tests from @uast/adaptive_scrolling@ to test the changes for multi-table @AdaptiveQuery@. Almost ... Alexandru Lungu

06:05 AM Feature #6418: represent extent fields as arrays

Small update on this topic: I ran the tests from the previous post, but with a small change regarding the tables. Ins... Radu Apetrii

04:51 AM Feature #6348: implement support for MariaDB

It looks that we need to use @timestamp(3)@/@datetime(3)@ instead of @timestamp@/@datetime@ in the tables' DDL for Ma... Igor Skornyakov

01:43 AM Bug #6741: Word index support for MariaDB.

Implement word index support for MariaDB using word tables Igor Skornyakov

01:41 AM Bug #6741 (New): Word index support for MariaDB.

Igor Skornyakov

01:40 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> Is there any value in implementing a @CONTAINS@ UDF? As I recall, we can't fully support the... Igor Skornyakov

12:46 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Finished port of PostgreSQL UDFs (except ones related to error handling and word tables) to... Eric Faulhaber

12:23 AM Feature #6507: cross-database joins

Greg Shah wrote:
> Is there a reason to implement this for a specific database backend?
Yes. Some databases (e.g.... Eric Faulhaber

11:01 PM Support #6679: H2 general performance tuning

Dănuț Filimon wrote:
> Alexandru Lungu wrote:
> > retest the Hotel GUI examples from #6679-21. Check if we have reg... Eric Faulhaber

09/08/2022

07:24 PM Bug #6703 (WIP): loading schema table DMO interface with no defined fields

Constantin, I reviewed it and your fix in 6129a/14273 seems logical.
I also added two changes in 6129a/14387:
* @... Ovidiu Maxiniuc

09/07/2022

04:16 PM Support #6713: check if it is faster to use separate temp-table instances instead of multiplex id

Eric Faulhaber wrote:
> I thought this naming convention was just for the DMO interfaces. When did we start using th... Ovidiu Maxiniuc

03:52 PM Support #6713: check if it is faster to use separate temp-table instances instead of multiplex id

Ovidiu Maxiniuc wrote:
> We are already forced to use a naming schema for temp-tables as @<converted-original-name... Eric Faulhaber

01:52 PM Support #6713: check if it is faster to use separate temp-table instances instead of multiplex id

The investigation should take into account that, beside the expected direct SQL internal benefits, should also includ... Ovidiu Maxiniuc

02:58 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Finished port of PostgreSQL UDFs (except ones related to error handling and word tables) to MariaDB.
Committed to 61... Igor Skornyakov

09:43 AM Feature #6418: represent extent fields as arrays

I've done some more testing, this time regarding the SQLs for three different approaches of storing an extent field a... Radu Apetrii

09:25 AM Support #6679: H2 general performance tuning

Alexandru Lungu wrote:
> retest the Hotel GUI examples from #6679-21. Check if we have regressions with prepared sta... Dănuț Filimon

02:09 AM Support #6679: H2 general performance tuning

Eric Faulhaber wrote:
> Is this something which is fixed with the patched v214?
No, this is related to the FWD ... Dănuț Filimon

09/06/2022

02:34 PM Support #6679: H2 general performance tuning

I'm trying to use H2 v214 to see if it still shows a memory leak I see when running a converted application with v200... Eric Faulhaber

10:15 AM Support #6679: H2 general performance tuning

Some things to note for your testing:
* We do not use persistent tables with H2 for customer applications. All use... Eric Faulhaber

09:56 AM Support #6679: H2 general performance tuning

Alexandru Lungu wrote:
> retest 2.1.210 (and eventually 2.1.212 and 2.1.214) with one-by-one strategy. Check if we s... Dănuț Filimon

11:26 AM Feature #6582: implement multi-table AdaptiveQuery

Alexandru Lungu wrote:
in my mind, the "dynamic mode" is still initially represented by the CompundQuery b... Alexandru Lungu

11:24 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

- removed the comment -
It was meant to be posted in #6582 Alexandru Lungu

09/05/2022

04:21 PM Feature #6371: implement SAVE CACHE statement

It is not ideal to use @DynamicTablesHelper@ (in its current form) to create persistent table DMO interfaces/classes.... Eric Faulhaber

03:47 PM Support #6679: H2 general performance tuning

Alexandru Lungu wrote:
> [...]
Thank you for the explanation.
> I will update Performance_Testing_the_H2_Datab... Eric Faulhaber

12:12 PM Support #6679: H2 general performance tuning

The tests from our current Performance_Testing_the_H2_Database are decent for 1.4.200, but are no quite accurate for ... Alexandru Lungu

10:31 AM Support #6679: H2 general performance tuning

Dănuț Filimon wrote:
> Eric Faulhaber wrote:
> > Dănuț Filimon wrote:
> > > Committed fwd-h2/rev.9. I applied a fe... Eric Faulhaber

09:46 AM Support #6679: H2 general performance tuning

Eric Faulhaber wrote:
> Dănuț Filimon wrote:
> > Committed fwd-h2/rev.9. I applied a few changes from versions 2.1.... Dănuț Filimon

02:22 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDF: ... Igor Skornyakov

03:55 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

I was thinking about error handling support ('guarded' UDFs) for MariaDB.
It looks that it can be implemented using ... Igor Skornyakov

04:28 AM Feature #6418: represent extent fields as arrays

> The question I would like to ask is: Is there any reason why normalized is preferred over denormalized, since the l... Greg Shah

02:58 AM Feature #6418: represent extent fields as arrays

I have tested these two methods of representing extent fields (normalized and denormalized) over a series of tests an... Radu Apetrii

01:57 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

I was in a session the other day which was offered at the PUG Challenge 2022. This was presented by Gus, who designe... Greg Shah

09/02/2022

02:14 PM Feature #6720: lazy hydration

If we are to lazily hydrate records, any column data which was not loaded in the initial hydration pass will have to ... Eric Faulhaber

01:03 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDFs: ... Igor Skornyakov

04:06 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDFs: ... Igor Skornyakov

10:26 AM Support #6679: H2 general performance tuning

Dănuț Filimon wrote:
> Eric Faulhaber wrote:
> > Have you discovered any cases where H2 v2.1.210patched was slower ... Eric Faulhaber

09:46 AM Support #6679: H2 general performance tuning

Eric Faulhaber wrote:
> Have you discovered any cases where H2 v2.1.210patched was slower than v1.4.200patched (Page... Dănuț Filimon

09:07 AM Feature #6418 (WIP): represent extent fields as arrays

Alexandru Lungu

09:06 AM Support #6713 (WIP): check if it is faster to use separate temp-table instances instead of multip...

Alexandru Lungu

02:47 AM Feature #6729: improve database and temp-table toString to write all records without relying on w...

This is in my opinion a very important problem that needs to be fixed asap. Enabling the DataSet.fullDebugString allo... Stefan Brands

09/01/2022

03:59 PM Support #6679: H2 general performance tuning

Thank you for all the research and testing. In all the work so far...
* Have you discovered any cases where H2 v2.... Eric Faulhaber

08/31/2022

02:50 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDFs: ... Igor Skornyakov

09:33 AM Feature #6729: improve database and temp-table toString to write all records without relying on w...

We need to allow the dataset or temp-table 'toString' representation (like in a debugger) to show all records in all ... Constantin Asofiei

09:30 AM Feature #6729 (New): improve database and temp-table toString to write all records without relyin...

Constantin Asofiei

08/30/2022

03:33 PM Feature #6371: implement SAVE CACHE statement

I pushed the latest version (see revision 6371a/14100). I tried to use existing DynamicTablesHelper implementation in... Boris Schegolev

02:01 PM Feature #4722: data import should be able to run with only the converted application jar file (an...

Constantin Asofiei wrote:
> Import always requires @ant deploy.prepare@, as it needs the application's jar. Did you... Roger Borrello

01:49 PM Feature #4722: data import should be able to run with only the converted application jar file (an...

Import always requires @ant deploy.prepare@, as it needs the application's jar. Did you have this in our @deploy/lib... Constantin Asofiei

01:48 PM Feature #4722: data import should be able to run with only the converted application jar file (an...

The issue was that I had been trying to link p2j to the @convert@ deployment and *not* run @ant deploy.prepare@, whic... Roger Borrello

11:40 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
> > Igor Skornyakov wrote:
> > > This revision should be OK. Let m... Igor Skornyakov

11:19 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Igor Skornyakov wrote:
> > This revision should be OK. Let me test the imports with my ve... Eric Faulhaber

10:37 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> This revision should be OK. Let me test the imports with my version of MariaDB.
Eric. W... Igor Skornyakov

10:28 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> I was using 3821c/14206. The latest revision is 14207, but I do not see a newer version of @... Igor Skornyakov

09:39 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Eric Faulhaber wrote:
> > Igor, is MariaDB 10.8 the minimum version on which the UDFs are ... Eric Faulhaber

04:21 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> Igor, is MariaDB 10.8 the minimum version on which the UDFs are supported? I've been trying ... Igor Skornyakov

04:07 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor, is MariaDB 10.8 the minimum version on which the UDFs are supported? I've been trying to get Hotel GUI going wi... Eric Faulhaber

07:16 AM Support #6679: H2 general performance tuning

Eric Faulhaber wrote:
> The statements under test ultimately were the same for 200patched and 210patched, correct?
... Dănuț Filimon

06:12 AM Support #6679: H2 general performance tuning

Committed fwd-h2/rev.7 containing version 2.1.210 without any patches from version 1.4.200.
Committed fwd-h2/rev.8... Dănuț Filimon

08/29/2022

12:56 PM Feature #4722: data import should be able to run with only the converted application jar file (an...

Greg Shah wrote:
> > Could that be where the issue is? Should the jar include these in the cvt directory?
>
> Yes... Roger Borrello

09:48 AM Feature #4722: data import should be able to run with only the converted application jar file (an...

> Could that be where the issue is? Should the jar include these in the cvt directory?
Yes and probably yes. You ... Greg Shah

09:41 AM Feature #4722: data import should be able to run with only the converted application jar file (an...

Using a conversion with 3821c_14194, my @ant import@ for a customer application is receiving this in the import log:
... Roger Borrello

07:21 AM Feature #6720: lazy hydration

In query processing, it is my understanding that a substantial percentage of time is spent in hydration of returned r... Greg Shah

06:39 AM Feature #6720 (WIP): lazy hydration

Greg Shah

07:03 AM Feature #6721: calculate field lists at conversion time where possible

There are use cases of buffers in which it is not possible to statically calculate the exact fields used at runtime (... Greg Shah

06:54 AM Feature #6721 (New): calculate field lists at conversion time where possible

Greg Shah

06:21 AM Support #6679: H2 general performance tuning

After running some tests regarding the performances of 4GL, SQL and P2J, I have generated a table that is meant to de... Radu Apetrii

05:40 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I added 3821c/rev. 14206 including a new MBean for query profiling: @QueryProfiler@. It stores a map from an SQL prep... Alexandru Lungu

04:53 AM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

Constantin Asofiei wrote:
> Actually, the previous patch is incorrect. I think this code in @TemporaryBuffer.define... Eric Faulhaber

08/28/2022

04:30 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Greg Shah wrote:
> Actually, it looks like the documentation already exists in [[p2j:Profiling#JMX-MBean-Instrumenta... Igor Skornyakov

08/27/2022

11:57 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Added @rindexof_<suffix>@, @rtrimws_<suffix>@, and @lc_t@ UDFs corresponding to @R-INDEX@, @RIGHT-TRIM@ and @LC@ 4GL ... Igor Skornyakov

08:27 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Implemented @substitute_tf@ UDF (see #6628-102).
Committed to 3821c/14202. Igor Skornyakov

05:19 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> I think that we can implement a @substitute@ for MariaDB as a two-argument UDF with JSON ... Igor Skornyakov

05:15 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

> I think that we can implement a @substitute@ for MariaDB as a two-argument UDF with JSON array as a second argument... Igor Skornyakov

03:01 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> @substitute@ could be used as a UDF, but it is not in this initial application. Again, this ... Igor Skornyakov

09:41 PM Bug #6275 (Review): SKIP-DELETED-RECORD implementation

Stanislav Lomany

09:41 PM Bug #6275 (WIP): SKIP-DELETED-RECORD implementation

Stanislav Lomany

09:40 PM Bug #6275: SKIP-DELETED-RECORD implementation

> A single question: calls to @stopOnSkippedRecord@ occur on @first@, @next@, and @previous@ navigation. Shouldn't th... Stanislav Lomany

08/26/2022

03:06 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDFs: ... Igor Skornyakov

12:55 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> For the initial application (at least based on static code analysis), only @r-index@ and @ri... Igor Skornyakov

10:53 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

For the initial application (at least based on static code analysis), only @r-index@ and @right-trim@ are used in con... Eric Faulhaber

04:01 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> The following is a list of 4GL builtin functions used within WHERE clauses, based on their p... Igor Skornyakov

12:12 PM Feature #6371: implement SAVE CACHE statement

Yes, something like that, but please note I've already stubbed out @DatabaseManager.schemaChanged@ in 6371a/14098. Pl... Eric Faulhaber

12:02 PM Feature #6371: implement SAVE CACHE statement

So, the integration part should be something like that, right?... Boris Schegolev

03:17 AM Feature #6371: implement SAVE CACHE statement

6371a/14098 contains a first pass implementation of conversion support for SAVE CACHE. The statement converts to a st... Eric Faulhaber

10:56 AM Bug #6715: BufferManager.validate throws IllegalStateException if a validation fails via a Valida...

Converting a legacy error into an abend seems like something easily avoided. Greg Shah

03:47 AM Bug #6715: BufferManager.validate throws IllegalStateException if a validation fails via a Valida...

If validation fails in @BufferManager.validate@, this abends:... Constantin Asofiei

03:45 AM Bug #6715 (New): BufferManager.validate throws IllegalStateException if a validation fails via a ...

Constantin Asofiei

04:08 AM Bug #6703: loading schema table DMO interface with no defined fields

A fix for the DMO interface loading (for runtime and DB import) is in 6129a/14273.
This does not solve all issues,... Constantin Asofiei

08/25/2022

06:03 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> This list comes from an application which we must support fully on MariaDB in September. P... Igor Skornyakov

05:59 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Eric Faulhaber wrote:
> > So, porting these final 10 gets us to parity with the other dial... Eric Faulhaber

05:41 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> Does this mean 306 of 316 Maria UDFs are implemented now? As I understand it, the 316 total ... Igor Skornyakov

05:26 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Ported the following UDFs:
> [...]
>
> Committed to 3821c/14199.
>
> The following ... Eric Faulhaber

04:08 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDFs: ... Igor Skornyakov

01:06 PM Bug #6688 (Closed): ASSIGN trigger conversion when field has DECIMALS clause

Constantin Asofiei wrote:
> Ovidiu Maxiniuc wrote:
> > With 3821c your testcases correctly converts to
> > [...]
... Constantin Asofiei

12:30 PM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

It seems to me that when processing certain temp-table copying use cases, there may be an opportunity for optimizatio... Greg Shah

12:24 PM Support #6714 (Review): check if there is an optimization opportunity when copying temp-tables ba...

Greg Shah

12:18 PM Support #6713: check if it is faster to use separate temp-table instances instead of multiplex id

It seems like for tables with very large numbers of records might be slow when using a multiplex approach as opposed ... Greg Shah

12:17 PM Support #6713 (WIP): check if it is faster to use separate temp-table instances instead of multip...

Greg Shah

12:15 PM Support #6712: check generated dynamic temp-table caching to ensure that all cache keys work when...

Greg Shah wrote:
> Recently, Eric found a flaw in our cache key implementation which resulted in the cache entry not... Eric Faulhaber

12:12 PM Support #6712: check generated dynamic temp-table caching to ensure that all cache keys work when...

Recently, Eric found a flaw in our cache key implementation which resulted in the cache entry not being properly foun... Greg Shah

12:09 PM Support #6712 (New): check generated dynamic temp-table caching to ensure that all cache keys wor...

Greg Shah

11:58 AM Feature #6711: skip schema AST creation and go directly to .p2o in dynamic temp-table generation

With relatively minor changes to the conversion, we could skip schema AST creation and go directly to .p2o in dynamic... Greg Shah

11:57 AM Feature #6711 (New): skip schema AST creation and go directly to .p2o in dynamic temp-table gener...

Greg Shah

11:56 AM Feature #6710: persist dynamic temp-tables across server restarts

A customer (thank you Stefan!) suggested that we could cache the generated DMOs etc... for dynamic temp-tables such t... Greg Shah

11:55 AM Feature #6710 (New): persist dynamic temp-tables across server restarts

Greg Shah

11:54 AM Support #6709: track nested FIND inside a related FOR loop

We have the perception that we have a problem with nested FINDs that reference a buffer or buffers controlled by a co... Greg Shah

11:49 AM Support #6709 (Review): track nested FIND inside a related FOR loop

Greg Shah

11:47 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

It seems that we may shift into dynamic mode in @AdaptiveQuery@ more frequently than one might expect. I would have ... Greg Shah

11:42 AM Support #6708 (Review): track/report at runtime when AdaptiveQuery shifts into dynamic mode

Greg Shah

11:37 AM Support #6707: evaluate the effectiveness of the current approach to ProgressiveResults

Currently @ProgressiveResults@ always starts with a set size of 1. This was done because the following idiom was see... Greg Shah

11:30 AM Support #6707 (New): evaluate the effectiveness of the current approach to ProgressiveResults

Greg Shah

11:33 AM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

Actually, the previous patch is incorrect. I think this code in @TemporaryBuffer.define@ at line 1668 (branch 6129a)... Constantin Asofiei

10:26 AM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

The fix is this:... Constantin Asofiei

09:25 AM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

The recreate is this:... Constantin Asofiei

10:05 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I had forgotten too. It was Alexandru's note in #6695-11 that reminded me. ;) Greg Shah

09:36 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Greg Shah wrote:
> Actually, it looks like the documentation already exists in [[p2j:Profiling#JMX-MBean-Instrumen... Igor Skornyakov

09:28 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Igor Skornyakov wrote:
> Greg Shah wrote:
> > Yes, no problem.
>
> Greg,
> It looks that I need some time to re... Greg Shah

07:45 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Greg Shah wrote:
> Yes, no problem.
Greg,
It looks that I need some time to recall the details. Maybe I will pre... Igor Skornyakov

05:34 AM Bug #6703: loading schema table DMO interface with no defined fields

If the .df has a table with no fields defined, import or loading this DMO interface fails with this:... Constantin Asofiei

05:33 AM Bug #6703 (WIP): loading schema table DMO interface with no defined fields

Constantin Asofiei

03:49 AM Feature #6582: implement multi-table AdaptiveQuery

I attached here the query conversion tests I used for reference. These are procedures which have structures involving... Alexandru Lungu

08/24/2022

05:19 PM Feature #6700: refactor prodataset conversion to generate a separate/common class which can be re...

Similar to how we convert temp-tables, the idea is to refactor the code to have datasets convert as a separate class ... Greg Shah

05:11 PM Feature #6700 (New): refactor prodataset conversion to generate a separate/common class which can...

Greg Shah

03:38 PM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

Constantin Asofiei wrote:
> Eric: do you recall if the @global@ parameter at the @TemporaryBuffer.define@ is used ... Eric Faulhaber

02:14 PM Bug #6699: TemporaryBuffer$Multiplexer memory leak in global block

For a very long-running FWD context, there is an accumulation of @TemporaryBuffer$Multiplexer@ instances in the @fina... Constantin Asofiei

02:11 PM Bug #6699 (WIP): TemporaryBuffer$Multiplexer memory leak in global block

Constantin Asofiei

03:36 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDFs: ... Igor Skornyakov

03:01 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Alexandru, I had missed your description of the tracing in #6695-11 when I posted my note.
If the hit/miss counts ... Eric Faulhaber

02:50 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Alexandru Lungu wrote:
> I locally added JMX beans for cache hits and misses and I get the same results (@4.99@ on @... Eric Faulhaber

10:24 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Greg Shah wrote:
> Make sure that the 4GL code is compiled before you run it. Otherwise the compilation cost in the... Alexandru Lungu

09:52 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Greg Shah wrote:
> > I locally added JMX beans for cache hits and misses and I get the same results (4.99 on PRESELE... Alexandru Lungu

09:50 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Yes, no problem. Greg Shah

09:45 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Greg Shah wrote:
> Igor: Please add details to [[p2j:JMX Instrumentation]] to allow the rest of the team to easily f... Igor Skornyakov

09:41 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

> I locally added JMX beans for cache hits and misses and I get the same results (4.99 on PRESELECT NO-UNDO, 0.0 on @... Greg Shah

09:28 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I locally added JMX beans for cache hits and misses and I get the same results (@4.99@ on @PRESELECT NO-UNDO@, @0.0 o... Alexandru Lungu

06:47 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

> > I also instrumented FWD with the attached patch (based on 3821c/14190),
>
> Forgot to attach the patch. This m... Greg Shah

05:33 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

These are some very insightful results!
@0.499@ rate for @PRESELECT NO-UNDO@ is exactly what I expected: only the ... Alexandru Lungu

10:46 AM Support #6679: H2 general performance tuning

Dănuț Filimon wrote:
> I ran a test on the sql statements obtained from the Hotel GUI application.
>
> The prepar... Eric Faulhaber

08:32 AM Support #6679: H2 general performance tuning

I ran a test on the sql statements obtained from the Hotel GUI application.
The prepared statements obtained from ... Dănuț Filimon

10:24 AM Feature #6348: implement support for MariaDB

The Hotel GUI project is broken, when running @./prepare_hotel.sh@ on a fresh checkout (interestingly, running again ... Constantin Asofiei

08/23/2022

06:32 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Eric Faulhaber wrote:
> I also instrumented FWD with the attached patch (based on 3821c/14190),
Forgot to attac... Eric Faulhaber

06:25 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I wanted to see where the time was being spent in both the @PreselectQuery@ case and the @CompoundQuery@ case. I also... Eric Faulhaber

12:25 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

There is a case in which the multi-table @PreselectQuery@ works slower than a @CompoundQuery@:... Alexandru Lungu

12:18 PM Feature #6695 (Review): Multi-table preselect query may underperform due to repetitive fetching

Alexandru Lungu

02:30 PM Bug #6275: SKIP-DELETED-RECORD implementation

> A single question: calls to @stopOnSkippedRecord@ occur on @first@, @next@, and @previous@ navigation. Shouldn't th... Greg Shah

01:55 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported the following UDFs: ... Igor Skornyakov

12:34 PM Feature #6582: implement multi-table AdaptiveQuery

> I should note that in a real application, it would be very unusual to have a wide open cross join like this, which ... Alexandru Lungu

11:10 AM Feature #6582: implement multi-table AdaptiveQuery

Alexandru Lungu wrote:
> > I am facing a three times slower query in @AdaptiveQuery@ comparing to @CompoundQuery@ (a... Eric Faulhaber

04:45 AM Support #6679: H2 general performance tuning

Committed 3821c/rev.14187 where I replaced @.nextvalue@ with @next value for@ in order to prepare for the 2.1.210 H2 ... Dănuț Filimon

03:52 AM Support #6679: H2 general performance tuning

Eric Faulhaber wrote:
> What is currently calling .nextval?
@P2JH2Dialect.getSequencePrefetchString()@, @FastCopy... Dănuț Filimon

08/22/2022

03:42 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ported matches, matcheslist, lookup.
Committed to 3821c/14184. Igor Skornyakov

11:22 AM Support #6679: H2 general performance tuning

Dănuț Filimon wrote:
> Regarding the syntax difference about nextval in #6679-11.
>
> Since we can't use @.nextva... Eric Faulhaber

10:09 AM Support #6679: H2 general performance tuning

Regarding the syntax difference about nextval in #6679-11.
Since we can't use @.nextval@ in version 2.1.210 of H2,... Dănuț Filimon

10:38 AM Feature #6582: implement multi-table AdaptiveQuery

> I am facing a three times slower query in @AdaptiveQuery@ comparing to @CompoundQuery@ (a read-only example). This ... Alexandru Lungu

08:40 AM Feature #6348: implement support for MariaDB

> The dialect generates now string columns as varchar(100). I did not push the system intentionally, but with the scr... Greg Shah

08/20/2022

07:45 AM Feature #6582: implement multi-table AdaptiveQuery

Eric Faulhaber wrote:
> Alexandru Lungu wrote:
>> Initially, I expect to see the AdaptiveQuery working out-of-the-b... Alexandru Lungu

06:42 AM Support #6679: H2 general performance tuning

> OTOH, I am a bit concerned that the performance drops off again in versions >210. Does the H2 history log suggest t... Alexandru Lungu

09:22 PM Feature #6348: implement support for MariaDB

Committed revision 14182.
Eric Faulhaber wrote:
> Any update on the status of running Hotel GUI with MariaDB?
Th... Ovidiu Maxiniuc

08/19/2022

03:47 PM Support #6679: H2 general performance tuning

Alexandru, I'll leave it to you to set the assignee. If both gentlemen are working on this task, you can leave it bla... Eric Faulhaber

03:39 PM Support #6679 (WIP): H2 general performance tuning

The fact that there are so many transactions being executed in the tests actually is encouraging to me. Since transac... Eric Faulhaber

03:17 PM Support #6679: H2 general performance tuning

Around 20 transactions are done by one user per iteration. There are 200 users per tests (except one test with 10 use... Alexandru Lungu

10:58 AM Support #6679: H2 general performance tuning

Just curious: do the tests use a lot of transactions? This is where we have seen a serious bottleneck in the past. Eric Faulhaber

10:13 AM Support #6679: H2 general performance tuning

I tested version 210 patched with the Hotel_GUI application and there were no major problems, except a syntax differe... Dănuț Filimon

08:28 AM Support #6679: H2 general performance tuning

> Yes, the tests for 1.4.200 patched use PageStore and the later versions use MVStore.
Fantastic! I think it is f... Greg Shah

05:26 AM Support #6679: H2 general performance tuning

Greg Shah wrote:
> Are the above comparisons using 1.4.200 patched in PageStore mode versus the later versions in mv... Dănuț Filimon

03:00 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> According to https://dev.mysql.com/doc/refman/8.0/en/signal.html, using SIGNAL with a SQLSTA... Igor Skornyakov

02:54 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

According to https://dev.mysql.com/doc/refman/8.0/en/signal.html, using SIGNAL with a SQLSTATE which begins with @01@... Eric Faulhaber

01:58 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Does anybody know a replacement of PostgreSQL @RAISE NOTICE@ statement for MariaDB?
I've used it for debugging compl... Igor Skornyakov

11:40 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Update.
Here is the complete list of MariaDB UDFs to be implemented:... Igor Skornyakov

11:27 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Eric Faulhaber wrote:
> What is the revised total number of UDFs for this dialect, and how many currently have a fir... Igor Skornyakov

09:58 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

What is the revised total number of UDFs for this dialect, and how many currently have a first pass implementation? I... Eric Faulhaber

03:00 PM Feature #6348: implement support for MariaDB

Ovidiu Maxiniuc wrote:
> My attempt to add a new guest caused a @Guest Accommodation Report@ with 225 entries (eve... Eric Faulhaber

11:37 AM Feature #6348: implement support for MariaDB

Now that you are working with Hotel GUI (i.e., something that, while not a "real" application, is more than a simple ... Eric Faulhaber

09:49 AM Feature #6348: implement support for MariaDB

Ovidiu Maxiniuc wrote:
> With hotel_gui/r248 and 3821c/14168, I am able to start the client and authenticate. The @G... Eric Faulhaber

10:36 AM Feature #6582: implement multi-table AdaptiveQuery

Alexandru Lungu wrote:
> Initially, I expect to see the @AdaptiveQuery@ working out-of-the-box with the current im... Eric Faulhaber

07:55 AM Bug #6688: ASSIGN trigger conversion when field has DECIMALS clause

Ovidiu Maxiniuc wrote:
> With 3821c your testcases correctly converts to
> [...]
>
> This was fixed recently r1... Constantin Asofiei

07:54 AM Bug #6688: ASSIGN trigger conversion when field has DECIMALS clause

With 3821c your testcases correctly converts to ... Ovidiu Maxiniuc

03:33 AM Bug #6688: ASSIGN trigger conversion when field has DECIMALS clause

There is a problem when converting a runtime ASSIGN database trigger, with a field having DECIMALS and/or INITIAL cla... Constantin Asofiei

03:31 AM Bug #6688 (Closed): ASSIGN trigger conversion when field has DECIMALS clause

Constantin Asofiei

08/18/2022

05:25 PM Support #6679: H2 general performance tuning

Dănuț Filimon wrote:
> The tests have 200 threads with 5/10 loops. The full tests have 200-2000 randomly generated... Eric Faulhaber

05:18 PM Support #6679: H2 general performance tuning

Greg Shah wrote:
> This sounds quite exciting.
>
> Are the above comparisons using 1.4.200 patched in PageStore m... Eric Faulhaber

03:23 PM Support #6679: H2 general performance tuning

This sounds quite exciting.
Are the above comparisons using 1.4.200 patched in PageStore mode versus the later ver... Greg Shah

10:01 AM Support #6679: H2 general performance tuning

Using the resources available in [[p2j:Performance Testing the H2 Database]] I have done tests using Apache JMeter an... Dănuț Filimon

08/17/2022

04:06 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> Please expand to see {{collapse(the full list of expected UDF in FWD)
> [...]
> }}
>
>... Igor Skornyakov

04:04 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Please expand to see {{collapse(the full list of expected UDF in FWD)... Ovidiu Maxiniuc

01:57 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> I used @meld@ to compare two text captions:
> # as the 'file name' suggests, I executed a ... Igor Skornyakov

01:53 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> Now, as I wrote the text above, I noticed that the UDFs are defined once for *ALL* database... Igor Skornyakov

01:46 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Greg Shah wrote:
> > I think it would be better to decide on a name casing (all lowercase maybe?)
>
> +1 for all ... Igor Skornyakov

01:45 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Added placeholder for the error handling UDFs in 3821c/14169.
>
> Please note that some ... Ovidiu Maxiniuc

01:40 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> > Anyway, I think that it is a little late to change the approach.
> I am not sure what yo... Igor Skornyakov

01:26 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Well, this is a possible option. It is a natural choice since UDFs a 4GL functions' counter... Ovidiu Maxiniuc

12:11 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> I think the best would be to scan for @@HQLFunction@ annotation in the list of @public st... Igor Skornyakov

12:03 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

> I think it would be better to decide on a name casing (all lowercase maybe?)
+1 for all lowercase names. Greg Shah

11:59 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Greg Shah wrote:
> Why would this emit into the converted code? Our converted code should not be dialect-specific t... Ovidiu Maxiniuc

11:53 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> I've generated templates for MariaDB UDFs from PostgreSQL ones. This is why we have no e.g.... Ovidiu Maxiniuc

11:16 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

> Devil's advocate point: there already are significant differences in syntax and builtin function names across datab... Greg Shah

11:12 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Please note however, that adding "signature" suffix to the UDF names increases their total number by ~50%.
For examp... Igor Skornyakov

11:09 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> Please note also that I decided do not add underscore to the names of UDFs w/o argument. So... Eric Faulhaber

11:05 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Devil's advocate point: there already are significant differences in syntax and builtin function names across databas... Eric Faulhaber

09:12 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Greg Shah wrote:
> If I understand correctly, different database "dialects" (in FWD) will have different function na... Igor Skornyakov

08:19 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

If I understand correctly, different database "dialects" (in FWD) will have different function names for the same 4GL... Greg Shah

06:54 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Added additional UDFs' signatures in 3821c/14171.
@fwdsession@ UDF does not exist for PostgreSQL. Actually I do no... Igor Skornyakov

03:40 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Please note also that I decided do not add underscore to the names of UDFs w/o argument. So it should be @getfwdversi... Igor Skornyakov

02:48 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Added placeholder for the error handling UDFs in 3821c/14169.
Please note that some native UDFs (such as @totimest... Igor Skornyakov

01:30 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> Igor,
>
> I implemented the decoration of the UDFs as described in note-31. I encountere... Igor Skornyakov

10:57 AM Support #6679 (Internal Test): H2 general performance tuning

Alexandru Lungu

03:45 AM Feature #6582: implement multi-table AdaptiveQuery

Committed 6582a/rev. 14160 including some fixes for order by clause and iteration type in multi-table @AdaptiveQuery@... Alexandru Lungu

08:14 PM Feature #6348: implement support for MariaDB

With hotel_gui/r248 and 3821c/14168, I am able to start the client and authenticate. The @Guests@, @Rates@ and @Rooms... Ovidiu Maxiniuc

08/16/2022

07:39 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor,
I implemented the decoration of the UDFs as described in note-31. I encountered some differences. I know you... Ovidiu Maxiniuc

01:20 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Refactored @ScriptRunner@ moving dialect-specific logic into the corresponding subclasses. Added support for MariaDB ... Igor Skornyakov

01:10 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> No, I do not work on this. In fact I've planned to finish with porting UDFs first.
Ovi... Eric Faulhaber

12:17 PM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > OK. Switching to the @ScriptRunner@ and most commonly used UDFs... Igor Skornyakov

10:42 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Igor Skornyakov wrote:
> OK. Switching to the @ScriptRunner@ and most commonly used UDFs.
> Please note however tha... Ovidiu Maxiniuc

02:41 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

Changed @build.gradle@ to add MariaDB @udfs.sql@ to the p2j.jar in 3821c/14165. Igor Skornyakov

01:31 AM Feature #6628: port PostgreSQL native user defined functions to MariaDB

OK. Switching to the @ScriptRunner@ and most commonly used UDFs.
Please note however that we also need UDFs names re... Igor Skornyakov

10:50 AM Feature #6348: implement support for MariaDB

Ovidiu Maxiniuc wrote:
> I can confirm the char r-trimming is provided out-of-the-box and char-indexes seems to wo... Eric Faulhaber