FWD » Core Development » Database

11/12/2022

04:54 AM Feature #6453: temp-table features

In addition to the previous note. I
Why not extend @XMLStreamWriter@ so that it will care about indentation itself? ... Igor Skornyakov

03:49 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> > > * @TempTableResultSet@:387: you are creating an intermediary @mprops@ tree for collecti... Igor Skornyakov

11/11/2022

03:41 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > > * @TempTableResultSet@:387: you are creating an intermediary ... Igor Skornyakov

03:34 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> > * @TempTableResultSet@:387: you are creating an intermediary @mprops@ tree for collecting... Ovidiu Maxiniuc

01:37 PM Feature #6453: temp-table features

Fixed @WRITE-XML@ support for @DATASET@.
See the attached .diff file for 6129b/14320.
The only remaining thing is... Igor Skornyakov

01:34 PM Feature #6453: temp-table features

Fixed @WRITE-XML@ support for @DATASET@ Igor Skornyakov

07:24 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Review of @marshal4.diff@
> Overall: a big update, handling multiple issues. Nice work. He... Igor Skornyakov

02:34 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > I work with branch 6129b. Please find the .diff for rev.14320 a... Igor Skornyakov

09:20 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> I work with branch 6129b. Please find the .diff for rev.14320 attached.
Review of @marsh... Ovidiu Maxiniuc

11/10/2022

03:51 PM Feature #6453: temp-table features

Completely (as far as my tests demonstrate) fixed @WRITE-XMLSCHEMA@ for standalone tables and datasets, including ext... Igor Skornyakov

06:09 AM Bug #6923: inner transaction block reverts too much

Update: If you replace the middle transaction block with the code below, the program works fine. It reverts only the ... Radu Apetrii

05:26 AM Bug #6923: inner transaction block reverts too much

Here is the code that I've been working with, on 3821c, rev.14354:... Radu Apetrii

05:25 AM Bug #6923 (Pending): inner transaction block reverts too much

Radu Apetrii

01:42 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> * @BaseRecord@:
> ** @setDatum()@: the conditional update of @readFields@ should happen at... Constantin Asofiei

09:55 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Eric/Ovidiu, please review the attached patch.
I like the update. The boolean arrays ... Ovidiu Maxiniuc

11/09/2022

04:43 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Tijs Wickardt wrote:
> > # Give each ORM data object a ByteArray, holding the raw (unse... Tijs Wickardt

04:38 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Tijs Wickardt wrote:
> # Give each ORM data object a ByteArray, holding the raw (unserialized) data of the last fetc... Constantin Asofiei

03:43 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Eric/Ovidiu, please review the attached patch. The details are these:
> * I couldn't f... Tijs Wickardt

02:20 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Another note: the @orm.Loader.load@ will retrieve from the database the full records, but they will not be populated ... Constantin Asofiei

02:14 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Eric/Ovidiu, please review the attached patch. The details are these:
* I couldn't find a good way to add support f... Constantin Asofiei

09:35 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

There is another finding: for EXCLUSIVE-LOCK, the record is always fully-fetched. In FWD, the partial record is allo... Constantin Asofiei

01:52 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> Constantin Asofiei wrote:
> > I have a (minimal) set of changes to allow EXCEPT/FIELDS at ... Constantin Asofiei

08:18 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

> This is odd. I did not try it. The logical implementation is to fully fetch the respective at that moment. Is this ... Greg Shah

02:37 PM Feature #6453: temp-table features

It appears that we have more problems with DATASET support than I initially reported in #6453-147. The most important... Igor Skornyakov

12:49 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > Fixed DATASET (un)marshaling. See the attached .diff file.
> >... Igor Skornyakov

07:48 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Fixed DATASET (un)marshaling. See the attached .diff file.
> Please review. There are many... Ovidiu Maxiniuc

12:47 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I have taken into account what you said and I have made some changes that appear in this new patch. They were tested ... Radu Apetrii

07:36 AM Bug #5307 (WIP): AdaptiveQuery goes into dynamic mode too aggressively

I've been trying to reproduce the behavior mentioned in #5307-1 using the example provided and failed. When debugging... Dănuț Filimon

03:13 AM Support #6712: check generated dynamic temp-table caching to ensure that all cache keys work when...

IIRC, the problem was that we rely on @P2JField.computeHash@, and the @name@ field can vary, depending on the call pa... Eric Faulhaber

11/08/2022

05:23 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> I have a (minimal) set of changes to allow EXCEPT/FIELDS at a query, with these notes:
... Ovidiu Maxiniuc

11:14 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Things get tricky when you have an ignored field and you update that field - this is allowed, and OE will save the re... Constantin Asofiei

09:53 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Another issue is that the records which are partially hydrated can not be cached (@FastFindCache@ or @orm.Session.cac... Constantin Asofiei

08:38 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

I have a (minimal) set of changes to allow EXCEPT/FIELDS at a query, with these notes:
* temp-tables do not support ... Constantin Asofiei

12:43 PM Bug #6919: fix ErrorManager.displayError in TempTableBuilder

FWD uses @ErrorManager.displayError@ instead of going through @recordOrShowError@, in i.e. @TempTableBuilder.tempTabl... Constantin Asofiei

12:36 PM Bug #6919 (Review): fix ErrorManager.displayError in TempTableBuilder

Constantin Asofiei

06:28 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Constantin Asofiei wrote:
> > Igor, you need to use @ErrorManager.recordOrThrow@ most like... Igor Skornyakov

05:10 AM Feature #6453: temp-table features

Fixed DATASET (un)marshaling. See the attached .diff file.
Please review. There are many pended changes. I would pre... Igor Skornyakov

11/07/2022

12:03 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Very well. This is s strong reason to refactor the inter-process serialization (the @Extern... Igor Skornyakov

11:37 AM Feature #6453: temp-table features

> Thank you, Ovidiu. However, in my tests, there is no difference between a standalone TEMP-TABLE and one which is a ... Ovidiu Maxiniuc

11:23 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Yes, this is true. Another reason (actually more important, at least at the beginning) was ... Igor Skornyakov

11:13 AM Feature #6453: temp-table features

Yes, this is true. Another reason (actually more important, at least at the beginning) was that, in the initial simpl... Ovidiu Maxiniuc

07:10 AM Feature #6453: temp-table features

> I think it will take a day or two.
Please do resolve these issues now. If it is going to take much longer than... Greg Shah

02:47 AM Feature #6453: temp-table features

In addition to the previous note.
In fact, there are many places when we work with a standalone temp table in a diff... Igor Skornyakov

02:35 AM Feature #6453: temp-table features

For some reason FWD processes @TEMP-TABLE@ which is part of the @DATASET@ input parameter differently than when it is... Igor Skornyakov

11/06/2022

12:54 PM Feature #6453: temp-table features

Greg Shah wrote:
> Optimally, yes. But I know Eric has other tasks for you as well. How long do you estimate the f... Igor Skornyakov

11:46 AM Feature #6453: temp-table features

Optimally, yes. But I know Eric has other tasks for you as well. How long do you estimate the fixes to take? Greg Shah

05:38 AM Feature #6453: temp-table features

I see a number of incompatibilities in the @NAMESPACE-URI@ and @NAMESPACE-PREFIX@ support in FWD for @DATASET@.
# FW... Igor Skornyakov

11/04/2022

04:58 PM Feature #6371: implement SAVE CACHE statement

Serialization Exception was a secondary issue. It was actually caused by unsuccessful class lookup. This is now fixed... Boris Schegolev

12:24 PM Feature #6453: temp-table features

I've fixed the issue with the indexes (un)marshaling by adding optional structured index data to the @TableWrapper@. ... Igor Skornyakov

04:03 AM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
>
> I'm still thinkin... Igor Skornyakov

11/03/2022

07:12 PM Feature #6371: implement SAVE CACHE statement

I pushed support for class unloading. I still need to do some refactoring and fix a @java.io.NotSerializableException... Boris Schegolev

10:29 AM Feature #6453: temp-table features

Fixed support for a binded table parameter.
Please review (see the attached .diff file).
Thank you.
Working ... Igor Skornyakov

10:18 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> In 4GL when @SCHEMA-MARCHAL@ is @NONE@ and the input parameter of the called procedure is d... Igor Skornyakov

05:24 AM Feature #6453: temp-table features

In 4GL when @SCHEMA-MARCHAL@ is @NONE@ and the input parameter of the called procedure is defined as @table FOR <loca... Igor Skornyakov

08:34 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

Committed 6708a/rev.14327. Added an initial version for tracing AdaptiveQuery and important data related to it.
Al... Dănuț Filimon

11/02/2022

03:52 PM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
Yes, this helps. Than... Igor Skornyakov

12:10 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> It seems now that the full set of knowledge about the indexes is needed. So I am definite... Igor Skornyakov

11:07 AM Feature #6453: temp-table features

Eric Faulhaber wrote:
> > Please let me know what you think about my suggestion #6453-120 and related idea about the... Ovidiu Maxiniuc

10:37 AM Feature #6453: temp-table features

Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
I'm still thinking about the index issue. Constantin Asofiei

09:39 AM Feature #6453: temp-table features

I've added reporting @12323@ error reporting ('A NO-SCHEMA-MARSHAL table cannot be used as a parameter where the rece... Igor Skornyakov

01:05 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Please let me know what you think about my suggestion #6453-120 and related idea about th... Eric Faulhaber

02:23 PM Feature #4056: instrument FWD and supporting libraries to measure database performance

I've committed to 6129b/14318 the changes in DatasetWrapper (serialization ID) and SortIndex (made the class public),... Constantin Asofiei

11/01/2022

04:10 PM Feature #6453: temp-table features

I'm working now on table parameter issues described in #6453-133 and #6453-134.
After that I plan to return to work ... Igor Skornyakov

08:02 AM Feature #6453: temp-table features

One more problem with table parameter in remote call.
If the parameter is declared like this:... Igor Skornyakov

06:52 AM Feature #6453: temp-table features

Added 4GL test for the @NONE@ value of the @SCHEMA-MARSHAL@.
FWD results are different from the 4GL ones Igor Skornyakov

04:26 AM Feature #6453: temp-table features

Marian Edu wrote:
> [[https://docs.progress.com/bundle/openedge-develop-abl-applications-122/page/Using-a-temp-table... Igor Skornyakov

03:57 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Sorry for my ignorance. How the declaration of such parameter should look like?
[[https:... Marian Edu

03:47 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
>
> > Thank you Marian. But my question was about the situations when ... Igor Skornyakov

03:42 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Thank you Marian. But my question was about the situations when @NONE@ value of the @SCHE... Marian Edu

03:34 AM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor Skornyakov wrote:
> > At this moment we keep index data in the @TableParameter@ as... Igor Skornyakov

03:26 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> At this moment we keep index data in the @TableParameter@ as a string created by @TableMapp... Constantin Asofiei

11:11 AM Feature #6418: represent extent fields as arrays

Radu Apetrii wrote:
> Greg Shah wrote:
> > The above is my opinion. Eric makes the final decision here.
> Any dec... Eric Faulhaber

09:29 AM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

Please ensure that the solution is built such that all dialects (not just MariaDB) can leverage this same approach. ... Greg Shah

09:33 PM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

I committed an intermediary solution. The occurrences of properties are no longer wrapped in @upper()@ and/or @rtrim(... Ovidiu Maxiniuc

08:33 AM Feature #6898: PostgreSQL schema improvements

We have multiple requests for changes to the default approach to some PostgreSQL schema. We have a small window righ... Greg Shah

08:05 AM Feature #6898 (New): PostgreSQL schema improvements

Greg Shah

08:32 AM Support #4058: consider denormalizing tables as the default approach

We've discussed offline this many times recently. I want to record some thoughts from those discussions:
* The cu... Greg Shah

10/31/2022

06:14 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Radu, in my last post I started right into the code review comments, but I want to note that I appreciate your recogn... Eric Faulhaber

05:38 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Code review @6695.patch@:
Interesting solution. Why is it only applicable for NO-UNDO temp-tables?
The patch do... Eric Faulhaber

07:16 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I have attached a patch that includes the changes regarding preselect queries on temporary no-undo tables. They were ... Radu Apetrii

04:03 PM Feature #6453: temp-table features

Greg Shah wrote:
> > Sorry, I do not understand. I see both @commons-lang@ and @commons-lang3@.
> > Can I use them ... Igor Skornyakov

04:02 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Greg Shah wrote:
> > [[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|Li... Greg Shah

03:21 PM Feature #6453: temp-table features

Greg Shah wrote:
> [[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|List of Supported 3rd Party Li... Igor Skornyakov

02:46 PM Feature #6453: temp-table features

[[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|List of Supported 3rd Party Libraries]] Greg Shah

01:17 PM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, review for the #6453-117 patch:
> * @DynamicTablesHelper@ - please do not use @St... Igor Skornyakov

12:49 PM Feature #6453: temp-table features

Igor, review for the #6453-117 patch:
* @DynamicTablesHelper@ - please do not use @StringUtils.isEmpty@ in apache co... Constantin Asofiei

10:21 AM Feature #6453: temp-table features

At this moment we keep index data in the @TableParameter@ as a string created by @TableMapper.getLegacyIndexInfo@.
... Igor Skornyakov

02:55 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor, sorry for being late on this one... it just slipped my mind I guess. The use for that erro... Igor Skornyakov

02:49 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> The support for @SCHEMA-MARSHAL@ @NONE@ value is still missing (see #6453-116). In partic... Marian Edu

10/29/2022

12:49 PM Feature #6453: temp-table features

Fixed (un)marshalling of fields and @NAMESPACE-URI@/@NAMESPACE-PREFIX@ for table parameter.
However, indexes are sti... Igor Skornyakov

10/28/2022

06:06 PM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> I pushed support for indexes (revision 6371a/14117). There's an issue with detecting if the... Eric Faulhaber

05:15 PM Feature #6371: implement SAVE CACHE statement

I pushed support for indexes (revision 6371a/14117). There's an issue with detecting if the index is primary - this i... Boris Schegolev

03:42 PM Bug #6812: fix FastFindCache to use a shared instance per persistent database

Changed the issue description to match the point made in #6812-3 and #6812-4. Eric Faulhaber

04:36 AM Feature #6895: improve performance of BufferReference methods buffer() and definition()

@RecordBuffer.buffer()@ and @RecordBuffer.define()@ (declared by the @BufferReference@ interface, and implemented by ... Eric Faulhaber

04:35 AM Feature #6895 (New): improve performance of BufferReference methods buffer() and definition()

Eric Faulhaber

03:21 AM Feature #6453: temp-table features

Marian Edu wrote:
> I wouldn't bother too much with MIN/FULL differences... the real use case here was the 'NONE' va... Igor Skornyakov

10/27/2022

11:49 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

I've fixed the issue with the wrong times - the thread-local must be static, for it to work as 'thread-local'. See 6... Constantin Asofiei

10:26 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Extended the method tracing with a new file: @summary_trace.csv@ including
* Summary table of scenarios: root, cou... Alexandru Lungu

04:06 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

The aggregate times per bucket are not displayed yet; working on it.
I will check how the sums are computed (and th... Alexandru Lungu

02:55 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

There are also some cases where a method's aggregate and own times are reported as >100% of the bucket total:
!405... Eric Faulhaber

01:03 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Alexandru, I like the HTML UI, nice work! Is the Grand Totals by package breakdown somewhere in there?
For a large... Eric Faulhaber

08:03 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Ovidiu: Please review. Greg Shah

06:09 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Apart from the tests mentioned in #6714-4, I have also made tests in which the called external procedure passes the t... Dănuț Filimon

06:48 AM Feature #6453: temp-table features

Greg Shah wrote:
> Now is the time to get all details exactly correct. I'm glad it is less likely to be an issue, b... Igor Skornyakov

06:27 AM Feature #6453: temp-table features

Now is the time to get all details exactly correct. I'm glad it is less likely to be an issue, but please do impleme... Greg Shah

03:41 AM Feature #6453: temp-table features

Marian Edu wrote:
> I wouldn't bother too much with MIN/FULL differences... the real use case here was the 'NONE' va... Igor Skornyakov

03:39 AM Feature #6453: temp-table features

It looks that when @SCHEMA-MARSHAL@ is @FULL@, 4GL serializes all field attributes of the table parameter of the remo... Igor Skornyakov

01:59 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Ovidiu Maxiniuc wrote:
> > My surprise was to see the last htt-MIN.xsd and htt-FULL.xsd ar... Marian Edu

10/26/2022

03:22 PM Feature #6348: implement support for MariaDB

I have an update which passes my (rather simple) testcase (unlike the current implementation from 6129 branches). I r... Ovidiu Maxiniuc

06:48 AM Feature #6348: implement support for MariaDB

It seems like sequences are not implemented in MariaDB for general usage. There must be some kind of use case that a... Greg Shah

09:27 PM Feature #6348: implement support for MariaDB

Further information on sequences.
It seems that we cannot rely on @lastval()@ method at all to detect the current ... Ovidiu Maxiniuc

02:29 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> My surprise was to see the last htt-MIN.xsd and htt-FULL.xsd are pretty identical. Initiall... Igor Skornyakov

02:26 PM Feature #6453: temp-table features

When I initially added this attribute I used a possible older manuals. It seems it is unchanged, though.
I think you... Ovidiu Maxiniuc

01:39 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> A bit strange. The OE versions looks identical in both FULL and NONE case.
> https://docs.... Igor Skornyakov

01:24 PM Feature #6453: temp-table features

A bit strange. The OE versions looks identical in both FULL and NONE case.
https://docs.progress.com/ru-RU/bundle/op... Ovidiu Maxiniuc

12:02 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Constantin,
> If you have time can you please advise me how to run this test with FWD? I... Igor Skornyakov

06:08 AM Feature #6453: temp-table features

Please find the final version of my test and its results attached.
It looks that currently we serialize less fields'... Igor Skornyakov

05:59 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > When running attached test with an application server, I see the fol... Igor Skornyakov

05:41 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> When running attached test with an application server, I see the following in the server lo... Marian Edu

05:36 AM Feature #6453: temp-table features

When running attached test with an application server, I see the following in the server log:... Igor Skornyakov

02:40 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > Marian,
> > Sorry for disturbing you, but I've stuck with the probl... Igor Skornyakov

02:02 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Marian,
> Sorry for disturbing you, but I've stuck with the problem of running my test wit... Marian Edu

01:06 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> OK, I understand now. If the ASTs are found in cache, the time _"was 40-50ms for dynamic ca... Constantin Asofiei

12:59 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

OK, I understand now. If the ASTs are found in cache, the time _"was 40-50ms for dynamic case"_ which is pretty close... Ovidiu Maxiniuc

12:47 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> That is correct.
> I see that there are 2 buffers involved so there is a bit of initializa... Constantin Asofiei

12:46 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

That is correct.
I see that there are 2 buffers involved so there is a bit of initialization and maybe some optimiza... Ovidiu Maxiniuc

12:29 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> The engine for dynamic queries will replace the inlined values with parameters. So all @con... Constantin Asofiei

12:07 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> The dynamic case is 3-6 times slower than OE, and the static case 2-5 times slower. I a... Ovidiu Maxiniuc

11:56 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

> I am thinking of a specialized AST (JavaAst) with runtime annotations as members instead. No more access using Map ... Greg Shah

11:48 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu, the problem is when the query predicate for the dynamic query is really unique (think a UUID is used as an op... Constantin Asofiei

11:35 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> Ovidiu, I don't see any calls originating from @interpret@, into the application (via Co... Ovidiu Maxiniuc

11:17 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> Improvements need to be made to dynamic query interpreter, too. It takes ~280ms for 650... Ovidiu Maxiniuc

11:16 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu, I don't see any calls originating from @interpret@, into the application (via ControlFlowOps). Most of the t... Constantin Asofiei

11:04 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Do you have a statistic of where is the most time consumed?
Usually the AST tree is simple enough and there are on... Ovidiu Maxiniuc

09:56 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Improvements need to be made to dynamic query interpreter, too. It takes ~280ms for 6500 'interpret' calls. Constantin Asofiei

10:55 AM Bug #6887: improve performance of dataset/table[-handle] parameters

In a large application, there is this JMX reported times (count/millis/nanos):... Constantin Asofiei

10:48 AM Bug #6887 (Review): improve performance of dataset/table[-handle] parameters

Constantin Asofiei

08:38 AM Feature #6582: implement multi-table AdaptiveQuery

This is the latest comparison of the scenarios in the context of the new H2 feature: incremental indexed sorting (658... Alexandru Lungu

07:09 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

As I recently got time to work with our LTW engine, I suggest extending it to cover the run-time tracking / reporting... Alexandru Lungu

06:28 AM Bug #5306: AdaptiveQuery sets a found record into the buffer twice in dynamic mode

The same issue I encountered when implemented #6582. When @AdaptiveQuery@ was in dynamic mode, the record was fetched... Alexandru Lungu

05:53 AM Feature #6815: configure all cache sizes in the directory, and create documentation for them

This includes the @query_cache_size=1024@ for temp-tables in @H2Helper.setCommonInMemoryProperties@. Constantin Asofiei

05:01 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

I have applied the changes you suggested. Also, I have tested all of them against a large customer application and th... Radu Apetrii

10/25/2022

04:21 PM Feature #6371: implement SAVE CACHE statement

Pushed point 2 from list above. See revision 6371a/14116.
Point 1 is still in progress. Boris Schegolev

12:13 PM Feature #6453: temp-table features

Marian,
Sorry for disturbing you, but I've stuck with the problem of running my test with the OE App. Server.
The t... Igor Skornyakov

11:53 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Committed 6129b/rev. 14307 including the new support for the HTML presentation of the method tracing data. There are ... Alexandru Lungu

10:16 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Alexandru Lungu wrote:
> Eric Faulhaber wrote:
> > Please commit the same update to branch 6129b today. Thank you.
... Eric Faulhaber

04:03 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Eric Faulhaber wrote:
> Please commit the same update to branch 6129b today. Thank you.
Done it in 6129b/rev. 143... Alexandru Lungu

02:10 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Dănuț Filimon wrote:
> Committed 3821c/rev.14313. Added AspectJ on @com.goldencode.p2j.persist.orm@ subpackage while... Eric Faulhaber

08:41 PM Bug #6837 (WIP): prevent unnecessary upper/rtrim injection in queries if the data type doesn't re...

Eric Faulhaber wrote:
> I agree that we don't want dialect-specific metadata in DMO annotations. OTOH, I think it ma... Ovidiu Maxiniuc

10/24/2022

07:23 PM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

I agree that we don't want dialect-specific metadata in DMO annotations. OTOH, I think it makes sense to store that m... Eric Faulhaber

06:34 PM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

The dropping of the @upper()@ / @rtrim()@ functions injected during conversion must be done anyway. Initially, it wil... Ovidiu Maxiniuc

01:08 PM Bug #4071: FWD server stops allowing new web client logins after abend

I'll also try to force the race condition Greg detected using a small sleep. Tijs Wickardt

12:49 PM Bug #4071: FWD server stops allowing new web client logins after abend

Okay. I'll try to hammer the system (testcases and/or <app>) concurrently, trying to overlap client shutdowns. Tijs Wickardt

12:14 PM Bug #4071: FWD server stops allowing new web client logins after abend

Unfortunately, I never found a reliable way to recreate this condition, else it probably would be fixed already. Eric Faulhaber

11:39 AM Bug #4071: FWD server stops allowing new web client logins after abend

Eric: Do you recall the details on how this problem can be recreated? Tijs is asking for guidance in #6787-70.
If... Greg Shah

07:45 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Greg Shah wrote:
> > Greg, is this a good moment to add the patch to 3821c?
>
> How safe are the changes? Have w... Radu Apetrii

07:25 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

> Greg, is this a good moment to add the patch to 3821c?
How safe are the changes? Have we tested them in one of ... Greg Shah

06:52 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

I reviewed the @6830.patch@. Nice job!
A couple of notes related to affected code:
* in @FastCopyHelper.java@ I s... Ovidiu Maxiniuc

06:15 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Here's the patch you requested. It was built on 3821c, rev. 14307.
I also modified @ReportApi.persistSearchHistory@ ... Radu Apetrii

07:05 AM Bug #6868: MariaDB Dialect can't properly check if a sequence exists

Radu Apetrii wrote:
> As a side note, I noticed there was a slight difference in results when running the test from ... Ovidiu Maxiniuc

06:23 AM Bug #6868: MariaDB Dialect can't properly check if a sequence exists

As a side note, I noticed there was a slight difference in results when running the test from #6868-1 in both Postgre... Radu Apetrii

10/21/2022

07:10 PM Feature #6371: implement SAVE CACHE statement

I pushed an update that handles the properties of columns (types, nullable, default, etc.), provides null-safe readin... Boris Schegolev

09:39 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Committed 3821c/rev.14313. Added AspectJ on @com.goldencode.p2j.persist.orm@ subpackage while keeping @com.goldencode... Dănuț Filimon

03:43 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Radu, please post a patch here for review, and mention the branch and revision it was built on. Constantin Asofiei

10/20/2022

11:55 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Radu Apetrii wrote:
> * I could not test @MariaDbDialect.getSequenceSetValString@ because I cannot access sequences ... Ovidiu Maxiniuc

06:57 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

I have tested these prepared statements (from the last post) and I got the following results:
* Out of 16 modificati... Radu Apetrii

11:41 AM Bug #6868 (Feedback): MariaDB Dialect can't properly check if a sequence exists

I assume you use 3821c.
This is an issue which was addressed in another branch. To allow you advance until the merge... Ovidiu Maxiniuc

05:57 AM Bug #6868: MariaDB Dialect can't properly check if a sequence exists

I tried executing with MariaDB (10.3.34) database this example:... Radu Apetrii

05:50 AM Bug #6868 (Feedback): MariaDB Dialect can't properly check if a sequence exists

Radu Apetrii

09:13 AM Feature #6418: represent extent fields as arrays

Greg Shah wrote:
> The above is my opinion. Eric makes the final decision here.
Any decision yet?
Radu Apetrii

09:10 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

There is already some work for #6720 which is close to this thread.
*I am planning to move my changes from 6582a to ... Radu Apetrii

06:23 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

I got it now. It was a mistake from my part (I was not using @-a@ option on the target server to be traced, but on th... Alexandru Lungu

05:17 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

> The queries (adaptive, preselect or compound) are not considered for "debug weaving" nor for "debug not weaving" (b... Greg Shah

12:24 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Eric Faulhaber wrote:
> For reference, I am attaching another @trace.csv@ file that was generated after implementi... Eric Faulhaber

10/19/2022

06:24 PM Feature #6720: lazy hydration

Yes, that is what I calling partial record hydration. Greg Shah

03:18 PM Feature #6720: lazy hydration

Greg, I think that #5731-399 doesn't implement lazy hydration. It just extends the usage of FWD Persistence API for e... Sergey Ivanovskiy

09:42 AM Feature #6720: lazy hydration

Code posted in #5731-399 implements partial record hydration. Greg Shah

04:01 PM Feature #4056: instrument FWD and supporting libraries to measure database performance

TBH, I've never fully understood the AspectJ weaving implementation, though back when I initially implemented this, I... Eric Faulhaber

04:09 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

I could set-up this feature with 3821c/rev.14305 and Hotel GUI. I am starting a standard server and then I attach the... Alexandru Lungu

09:41 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Code posted in #5731-399 implements partial record hydration. Greg Shah

09:07 AM Feature #6453: temp-table features

Greg Shah wrote:
> We need to match the results exactly. No short cuts.
I see. So I definitely need to be able t... Igor Skornyakov

09:02 AM Feature #6453: temp-table features

We need to match the results exactly. No short cuts. Greg Shah

08:56 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> BTW: what can go wrong if we ignore @SCHEMA-MARSHAL@ and always send *all* schema attribu... Marian Edu

08:51 AM Feature #6453: temp-table features

Marian Edu wrote:
> If it's easier for you we can write the 4GL tests on our side and then you run those on FWD, e... Igor Skornyakov

08:06 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> If I will be able to run at least one test from @testcases/appsrv@ with 4GL I believe I w... Marian Edu

06:11 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > Marian,
> > I've finished (hopefully) with other substasks and star... Igor Skornyakov

05:54 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Marian,
> I've finished (hopefully) with other substasks and started working on testing @S... Marian Edu

04:08 AM Feature #6453: temp-table features

Marian Edu wrote:
> Greg Shah wrote:
> > Constantin/Marian: Can you please advise Igor on how he can test the behav... Igor Skornyakov

07:42 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

I have created tests with the mentioned specifications and all passed (same results were obtained from the executed s... Dănuț Filimon

10/18/2022

04:58 PM Feature #6371: implement SAVE CACHE statement

I have successfully loaded a table / new DMO into my server runtime. Code is in 6371a/14114 revision. Boris Schegolev

11:22 AM Bug #6852: Problem with datetime-tz literal string conversion.

The field definition... Igor Skornyakov

11:21 AM Bug #6852 (New): Problem with datetime-tz literal string conversion.

Igor Skornyakov

11:16 AM Feature #6453: temp-table features

Eric Faulhaber wrote:
> Yes, please do, and let's focus on the remaining core items of this task. What is left?
... Igor Skornyakov

11:13 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> As far as I understand @relex()@ cannot properly and reliably detect the literal type since... Eric Faulhaber

08:21 AM Feature #6453: temp-table features

As far as I understand @relex()@ cannot properly and reliably detect the literal type since it does not consider @vty... Igor Skornyakov

06:58 AM Feature #6453: temp-table features

Greg Shah wrote:
> Look at @initializer_constant@ in @progress.g@. There is a @relex()@ step that occurs when the l... Igor Skornyakov

06:57 AM Feature #6453: temp-table features

Look at @initializer_constant@ in @progress.g@. There is a @relex()@ step that occurs when the literal is quoted tex... Greg Shah

06:49 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Igor Skornyakov wrote:
> > I've noticed the following conversion problem:
> > The field d... Igor Skornyakov

06:02 AM Feature #6453: temp-table features

Fixed @COLUMN-CODEPAGE@ and @now@/@today@ @INITIAL@ attribute values for dynamic temp-tables.
Please review.
Thank ... Igor Skornyakov

05:08 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> I've noticed the following conversion problem:
> The field definition
> [...]
> is conve... Igor Skornyakov

09:16 PM Bug #6301: incorrect initialization of date-related fields with today and now literals

Eric Faulhaber wrote:
> A fix is committed to 3821c/14279. I have tested with the above test case.
>
> Ovidiu, pl... Ovidiu Maxiniuc

10/17/2022

02:17 PM Bug #6301: incorrect initialization of date-related fields with today and now literals

Fixed in 6129b/14302. Tested with this simple test case:... Eric Faulhaber

02:01 PM Bug #6301 (Test): incorrect initialization of date-related fields with today and now literals

Igor reported a regression for a table with two fields of type @datetime@ and @datetime-tz@, respectively, both with ... Eric Faulhaber

02:13 PM Feature #6453: temp-table features

I've noticed the following conversion problem:
The field definition... Igor Skornyakov

10:33 AM Feature #6457: finish COPY-LOB support

> What are the values you're trying to use for the codepages/collation tables - for cpstream and cpinternal?
I wan... Stanislav Lomany

08:42 AM Feature #6457: finish COPY-LOB support

Stanislav Lomany wrote:
> > @i18n/4gl/cp_collation.xml@ @i18n/4gl/cp_conversion.xml@ @i18n/4gl/chr_conversion.xml@ @... Marian Edu

10/16/2022

03:05 PM Feature #6457: finish COPY-LOB support

> @i18n/4gl/cp_collation.xml@ @i18n/4gl/cp_conversion.xml@ @i18n/4gl/chr_conversion.xml@ @i18n/4gl/is_conversion.xml@... Stanislav Lomany

10/15/2022

03:29 AM Feature #6348: implement support for MariaDB

There is another issue in MariaDB - sequences and tables are in the same namespace. In OE, IIRC they are not in the ... Constantin Asofiei

09:33 PM Feature #6348: implement support for MariaDB

Constantin Asofiei wrote:
> 6129a/14487 contains some fixes related to sequences in MariaDB:
> * fixes related to s... Ovidiu Maxiniuc

10/14/2022

05:04 PM Bug #6767: Possible name clashes in DMO classes

Eric: Please review. Greg Shah

03:04 PM Bug #6715: BufferManager.validate throws IllegalStateException if a validation fails via a Valida...

Tijs Wickardt wrote:
> Eric Faulhaber wrote:
> > Tijs, could you please check the files you committed related to th... Tijs Wickardt

11:19 AM Bug #6715: BufferManager.validate throws IllegalStateException if a validation fails via a Valida...

Eric Faulhaber wrote:
> Tijs, could you please check the files you committed related to the @TailMap@ implementation... Tijs Wickardt

11:14 AM Bug #6715: BufferManager.validate throws IllegalStateException if a validation fails via a Valida...

Constantin Asofiei wrote:
> In some cases, a ValidationException (which is from a bug) is wrapped in a ErrorConditio... Eric Faulhaber

11:07 AM Bug #6715: BufferManager.validate throws IllegalStateException if a validation fails via a Valida...

In some cases, a ValidationException (which is from a bug) is wrapped in a ErrorConditionException, like this:... Constantin Asofiei

11:02 AM Bug #6715: BufferManager.validate throws IllegalStateException if a validation fails via a Valida...

Yes, @ErrorConditionException@ should be allowed through; it should not be wrapped in @IllegalStateException@. I thin... Eric Faulhaber

12:51 PM Feature #6348: implement support for MariaDB

Fun fact (nothing more): you can set NTFS to case sensitive in Windows.
That will OE classes in Windows to become ca... Tijs Wickardt

12:24 PM Feature #6348: implement support for MariaDB

Constantin Asofiei wrote:
> Regarding case-sensitivity in MariaDB: this is OS-dependent, it will be case-insensitive... Ovidiu Maxiniuc

12:23 PM Feature #6348: implement support for MariaDB

Regarding case-sensitivity in MariaDB: this is OS-dependent, it will be case-insensitive on Windows. Constantin Asofiei

12:23 PM Feature #6348: implement support for MariaDB

Ovidiu Maxiniuc wrote:
> How are you able to keep then in various casing?
Set @ conversion="minimal"@ in p2j.cfg.xm... Constantin Asofiei

12:17 PM Feature #6348: implement support for MariaDB

Constantin Asofiei wrote:
> Ovidiu, the main issue is the case-sensitivity of sequence names (and tables, btw) in Ma... Ovidiu Maxiniuc

11:01 AM Feature #6348: implement support for MariaDB

Ovidiu, with an app I'm seeing lots of @MISSING: MariaDbDialect.isQueryRangeParameterInlined()@ - is this a TODO rela... Constantin Asofiei

06:22 AM Feature #6348: implement support for MariaDB

6129a/14487 contains some fixes related to sequences in MariaDB:
* fixes related to sequence management in MariaDB -... Constantin Asofiei

12:07 PM Feature #6453: temp-table features

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
>
> > Looking at 3821c I'm not sure that it contains the fix for t... Igor Skornyakov

11:27 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Looking at 3821c I'm not sure that it contains the fix for the problem.
We have some t... Eric Faulhaber

08:42 AM Feature #6453: temp-table features

Fixed a number of issues with @WRITE-XML@ and @READ-XMLSCHEMA@.
Committed to 6129a/14488.
At this moment I see ... Igor Skornyakov

08:30 AM Feature #6453: temp-table features

Greg Shah wrote:
> Are you saying that Eric's solution in 3821c revs 14279 and 14280 is not sufficient to resolve th... Igor Skornyakov

08:16 AM Feature #6453: temp-table features

Are you saying that Eric's solution in 3821c revs 14279 and 14280 is not sufficient to resolve the issue you have found? Greg Shah

08:09 AM Feature #6453: temp-table features

Greg Shah wrote:
> > In @TableMapper.loadFields@ for if @INITIAL@ is @NOW@ or @TODAY@ for date/datetime field we cal... Igor Skornyakov

08:05 AM Feature #6453: temp-table features

> In @TableMapper.loadFields@ for if @INITIAL@ is @NOW@ or @TODAY@ for date/datetime field we calculate the value of ... Greg Shah

07:59 AM Feature #6453: temp-table features

In @TableMapper.loadFields@ for if @INITIAL@ is @NOW@ or @TODAY@ for date/datetime field we calculate the value of co... Igor Skornyakov

07:10 AM Bug #6829: H2 forces re-parse of all prepared statements when metadata is changed

Constantin Asofiei wrote:
> Alex, there's something else I'd like to understand: I've increased the FWD's @psCache@ ... Alexandru Lungu