FWD » Core Development » Database

11/17/2022

06:39 PM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

FYI, I'm digging back through the history on this one. I'll pick it back up in the morning. Greg Shah

08:02 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

The patch is built on top of 6129b/14324. Constantin Asofiei

08:01 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

Greg, the problem is that for a @DEF PARAM BUFFER ... FOR some-buf@ statement, @some-buf@ gets promoted because @def_... Constantin Asofiei

02:52 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

The problem is when a buffer is defined explicitly as a parameter - FWD doesn't resolve the unqualified field properl... Constantin Asofiei

02:46 AM Bug #6940 (Test): unqualified field name resolution when there is an explicit buffer defined in a...

Constantin Asofiei

03:18 PM Feature #6453: temp-table features

Only one change before the commit, please: in @XmlImport.java@, instead of ... Ovidiu Maxiniuc

02:20 PM Feature #6453 (Review): temp-table features

Igor Skornyakov wrote:
> Ovidiu Maxiniuc wrote:
> > Review of @read-xmlschema.diff@:
> [...]
> Can I commit the c... Eric Faulhaber

05:03 AM Feature #6453: temp-table features

Tests used for this task committed to @xfer.goldencode.com/opt/testcases/temp-table-marshal@. rev. 1300.
Igor Skornyakov

01:58 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Review of @read-xmlschema.diff@:
> * @AbstractTempTable@: the new method lacks the javadoc... Igor Skornyakov

02:32 PM Feature #6371: implement SAVE CACHE statement

FYI, I am re-working the class unloading, which will not work currently in "real" use scenarios. When we create a DMO... Eric Faulhaber

02:02 PM Bug #6923: inner transaction block reverts too much

Eric Faulhaber wrote:
> ...the fact that the results are different in your first test case between temp-table and pe... Eric Faulhaber

01:52 PM Bug #6923: inner transaction block reverts too much

When you nest transactions like:... Eric Faulhaber

06:50 AM Bug #6923 (WIP): inner transaction block reverts too much

An update to this issue:
I narrowed down the code, in order to describe what I have found until this moment. This is... Radu Apetrii

12:02 PM Feature #6418: represent extent fields as arrays

Alexandru Lungu wrote:
> Eric, is there any prior partial implementation/discussion into the array fields?
There ... Eric Faulhaber

03:29 AM Feature #6418: represent extent fields as arrays

Eric, is there any prior partial implementation/discussion into the array fields? If no, this task can be the lead ef... Alexandru Lungu

07:35 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

The @2137_20221114a.patch@ patch was committed to 3821c/14369, with these notes:
* only NO-LOCK records are loaded a... Constantin Asofiei

07:34 AM Bug #6941: conversion issue when using undo in do transaction

There is a problem when converting examples that contain an @undo@ statement inside a @do transaction@.
For example,... Radu Apetrii

07:20 AM Bug #6941 (Rejected): conversion issue when using undo in do transaction

Radu Apetrii

05:40 AM Feature #6816: improve PreselectQuery.assembleHQL

For an initial check, I recommend implementing a very small suite of performance tests (~6) for @PreselectQuery@ to t... Alexandru Lungu

03:33 AM Feature #6816 (WIP): improve PreselectQuery.assembleHQL

Alexandru Lungu

11/16/2022

03:54 PM Feature #6453: temp-table features

Review of @read-xmlschema.diff@:
* @AbstractTempTable@: the new method lacks the javadoc;
* @XmlImport@:
** method... Ovidiu Maxiniuc

11:59 AM Feature #6453: temp-table features

Finished with fixing @READ-XMLSCHEMA@ support for DATASET.
Please review.
Thank you.
I understand that the tas... Igor Skornyakov

01:01 PM Feature #6459: query features

Igor, please focus only on the first bullet point (i.e., @QUERY@ handle) items. Eric Faulhaber

12:28 PM Feature #6448: make location of LOB files configurable for import

Eric Faulhaber wrote:
> Currently, the location is hard-coded to a @lobs@ sub-directory of the directory containing ... Eric Faulhaber

10:16 AM Feature #6938: XML seriliazation improvement

At this moment the @XMLExport@ class uses manual formatting of the document based on @indent()@/@leneBreak()@ calls (... Igor Skornyakov

10:08 AM Feature #6938 (New): XML seriliazation improvement

Igor Skornyakov

10:14 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

3821c is fine. Thanks. Eric Faulhaber

09:04 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

I have no objection.
Eric: Where should the changes go? I'd prefer 3821c. Greg Shah

06:27 AM Support #6708 (WIP): track/report at runtime when AdaptiveQuery shifts into dynamic mode

Greg, can we consider moving this change to 3821c or 6129b? It does not change the FWD core behavior; it only augment... Alexandru Lungu

06:34 AM Support #6709: track nested FIND inside a related FOR loop

This is a good analysis. It certainly highlights the differences in control flow and danger of refactoring such code... Greg Shah

05:57 AM Support #6709 (WIP): track nested FIND inside a related FOR loop

Dănuț Filimon

05:36 AM Support #6709: track nested FIND inside a related FOR loop

From what I understood, we want to test if either nested find or joined approach is better. After a few tests using t... Dănuț Filimon

05:56 AM Feature #5355 (WIP): _Usertablestat metadata improvements

Alexandru Lungu

05:47 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Any news about the review? Dănuț Filimon

11/15/2022

09:33 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Right. I failed to notice the @field@ is updated in inner loop. Ovidiu Maxiniuc

08:07 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> Review of @2137_20221114a.patch@:
> I have a single issue in the new patch, in @AbstractQu... Constantin Asofiei

06:45 AM Feature #6928: H2 UPDATE performance improvement

In H2, when doing an UPDATE statement,the approach is to:
* create a new row
* assign all columns not part of the U... Constantin Asofiei

03:49 AM Feature #6928 (Closed): H2 UPDATE performance improvement

Constantin Asofiei

11/14/2022

06:37 PM Feature #6371: implement SAVE CACHE statement

Pushed revision 6371a/14121. Based on testing, added more mappings for column types and moved mapping to @Dialect@ pl... Boris Schegolev

02:03 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Ovidiu, please review attached. This adds denormalized extent for FIELDS/EXCEPT. norma... Ovidiu Maxiniuc

10:31 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu, please review attached. This adds denormalized extent for FIELDS/EXCEPT. normalized extent is still not sup... Constantin Asofiei

01:24 PM Feature #6453: temp-table features

It appears that @READ-XMLSCHEMA@ for the @DATASET@ is broken even in the situation of a single namespace (w/o externa... Igor Skornyakov

11:01 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Certainly, the patch is clearly an improvement over the current revision on bzr. Please c... Igor Skornyakov

10:44 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> > What I meant was that a @ClassCastException@ could be thrown if code inside FWD will ca... Ovidiu Maxiniuc

10:42 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > Why not extend @XMLStreamWriter@ so that it will care about ind... Igor Skornyakov

10:37 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Why not extend @XMLStreamWriter@ so that it will care about indentation itself? [...] What ... Ovidiu Maxiniuc

12:28 PM Bug #6887: improve performance of dataset/table[-handle] parameters

I've edited previous note to add @delete object hds1.@ to @dstest.p@ - with this, OE is in the 200ms range, while FWD... Constantin Asofiei

12:04 PM Bug #6887: improve performance of dataset/table[-handle] parameters

A starting point for a test to check the performance is this, made of two programs:
# @dstest.p@ which defines inter... Constantin Asofiei

11:14 PM Bug #6887: improve performance of dataset/table[-handle] parameters

Constantin, is this something you think can reasonably be reproduced with some simplified test cases, which would ill... Eric Faulhaber

09:07 AM Bug #6829: H2 forces re-parse of all prepared statements when metadata is changed

Alexandru, I have in a heap dump @UnclosablePreparedStatement@ instances with ~90KB retained heap. But in at least a... Constantin Asofiei

06:23 AM Bug #6829: H2 forces re-parse of all prepared statements when metadata is changed

I've done some simple debugging in Hotel GUI with @UnclosablePreparedStatement@. The @psCache@ structure is an @Expir... Alexandru Lungu

07:46 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

Committed 6708a/rev.14328. Made improvements to AdaptiveQuery method tracing.
I made changes and now it is possibl... Dănuț Filimon

11/12/2022

04:54 AM Feature #6453: temp-table features

In addition to the previous note. I
Why not extend @XMLStreamWriter@ so that it will care about indentation itself? ... Igor Skornyakov

03:49 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> > > * @TempTableResultSet@:387: you are creating an intermediary @mprops@ tree for collecti... Igor Skornyakov

11/11/2022

03:41 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > > * @TempTableResultSet@:387: you are creating an intermediary ... Igor Skornyakov

03:34 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> > * @TempTableResultSet@:387: you are creating an intermediary @mprops@ tree for collecting... Ovidiu Maxiniuc

01:37 PM Feature #6453: temp-table features

Fixed @WRITE-XML@ support for @DATASET@.
See the attached .diff file for 6129b/14320.
The only remaining thing is... Igor Skornyakov

01:34 PM Feature #6453: temp-table features

Fixed @WRITE-XML@ support for @DATASET@ Igor Skornyakov

07:24 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Review of @marshal4.diff@
> Overall: a big update, handling multiple issues. Nice work. He... Igor Skornyakov

02:34 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > I work with branch 6129b. Please find the .diff for rev.14320 a... Igor Skornyakov

09:20 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> I work with branch 6129b. Please find the .diff for rev.14320 attached.
Review of @marsh... Ovidiu Maxiniuc

11/10/2022

03:51 PM Feature #6453: temp-table features

Completely (as far as my tests demonstrate) fixed @WRITE-XMLSCHEMA@ for standalone tables and datasets, including ext... Igor Skornyakov

06:09 AM Bug #6923: inner transaction block reverts too much

Update: If you replace the middle transaction block with the code below, the program works fine. It reverts only the ... Radu Apetrii

05:26 AM Bug #6923: inner transaction block reverts too much

Here is the code that I've been working with, on 3821c, rev.14354:... Radu Apetrii

05:25 AM Bug #6923 (Pending): inner transaction block reverts too much

Radu Apetrii

01:42 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> * @BaseRecord@:
> ** @setDatum()@: the conditional update of @readFields@ should happen at... Constantin Asofiei

09:55 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Eric/Ovidiu, please review the attached patch.
I like the update. The boolean arrays ... Ovidiu Maxiniuc

11/09/2022

04:43 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Tijs Wickardt wrote:
> > # Give each ORM data object a ByteArray, holding the raw (unse... Tijs Wickardt

04:38 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Tijs Wickardt wrote:
> # Give each ORM data object a ByteArray, holding the raw (unserialized) data of the last fetc... Constantin Asofiei

03:43 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Eric/Ovidiu, please review the attached patch. The details are these:
> * I couldn't f... Tijs Wickardt

02:20 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Another note: the @orm.Loader.load@ will retrieve from the database the full records, but they will not be populated ... Constantin Asofiei

02:14 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Eric/Ovidiu, please review the attached patch. The details are these:
* I couldn't find a good way to add support f... Constantin Asofiei

09:35 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

There is another finding: for EXCLUSIVE-LOCK, the record is always fully-fetched. In FWD, the partial record is allo... Constantin Asofiei

01:52 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> Constantin Asofiei wrote:
> > I have a (minimal) set of changes to allow EXCEPT/FIELDS at ... Constantin Asofiei

08:18 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

> This is odd. I did not try it. The logical implementation is to fully fetch the respective at that moment. Is this ... Greg Shah

02:37 PM Feature #6453: temp-table features

It appears that we have more problems with DATASET support than I initially reported in #6453-147. The most important... Igor Skornyakov

12:49 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > Fixed DATASET (un)marshaling. See the attached .diff file.
> >... Igor Skornyakov

07:48 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Fixed DATASET (un)marshaling. See the attached .diff file.
> Please review. There are many... Ovidiu Maxiniuc

12:47 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I have taken into account what you said and I have made some changes that appear in this new patch. They were tested ... Radu Apetrii

07:36 AM Bug #5307 (WIP): AdaptiveQuery goes into dynamic mode too aggressively

I've been trying to reproduce the behavior mentioned in #5307-1 using the example provided and failed. When debugging... Dănuț Filimon

03:13 AM Support #6712: check generated dynamic temp-table caching to ensure that all cache keys work when...

IIRC, the problem was that we rely on @P2JField.computeHash@, and the @name@ field can vary, depending on the call pa... Eric Faulhaber

11/08/2022

05:23 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> I have a (minimal) set of changes to allow EXCEPT/FIELDS at a query, with these notes:
... Ovidiu Maxiniuc

11:14 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Things get tricky when you have an ignored field and you update that field - this is allowed, and OE will save the re... Constantin Asofiei

09:53 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Another issue is that the records which are partially hydrated can not be cached (@FastFindCache@ or @orm.Session.cac... Constantin Asofiei

08:38 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

I have a (minimal) set of changes to allow EXCEPT/FIELDS at a query, with these notes:
* temp-tables do not support ... Constantin Asofiei

12:43 PM Bug #6919: fix ErrorManager.displayError in TempTableBuilder

FWD uses @ErrorManager.displayError@ instead of going through @recordOrShowError@, in i.e. @TempTableBuilder.tempTabl... Constantin Asofiei

12:36 PM Bug #6919 (Review): fix ErrorManager.displayError in TempTableBuilder

Constantin Asofiei

06:28 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Constantin Asofiei wrote:
> > Igor, you need to use @ErrorManager.recordOrThrow@ most like... Igor Skornyakov

05:10 AM Feature #6453: temp-table features

Fixed DATASET (un)marshaling. See the attached .diff file.
Please review. There are many pended changes. I would pre... Igor Skornyakov

11/07/2022

12:03 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Very well. This is s strong reason to refactor the inter-process serialization (the @Extern... Igor Skornyakov

11:37 AM Feature #6453: temp-table features

> Thank you, Ovidiu. However, in my tests, there is no difference between a standalone TEMP-TABLE and one which is a ... Ovidiu Maxiniuc

11:23 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Yes, this is true. Another reason (actually more important, at least at the beginning) was ... Igor Skornyakov

11:13 AM Feature #6453: temp-table features

Yes, this is true. Another reason (actually more important, at least at the beginning) was that, in the initial simpl... Ovidiu Maxiniuc

07:10 AM Feature #6453: temp-table features

> I think it will take a day or two.
Please do resolve these issues now. If it is going to take much longer than... Greg Shah

02:47 AM Feature #6453: temp-table features

In addition to the previous note.
In fact, there are many places when we work with a standalone temp table in a diff... Igor Skornyakov

02:35 AM Feature #6453: temp-table features

For some reason FWD processes @TEMP-TABLE@ which is part of the @DATASET@ input parameter differently than when it is... Igor Skornyakov

11/06/2022

12:54 PM Feature #6453: temp-table features

Greg Shah wrote:
> Optimally, yes. But I know Eric has other tasks for you as well. How long do you estimate the f... Igor Skornyakov

11:46 AM Feature #6453: temp-table features

Optimally, yes. But I know Eric has other tasks for you as well. How long do you estimate the fixes to take? Greg Shah

05:38 AM Feature #6453: temp-table features

I see a number of incompatibilities in the @NAMESPACE-URI@ and @NAMESPACE-PREFIX@ support in FWD for @DATASET@.
# FW... Igor Skornyakov

11/04/2022

04:58 PM Feature #6371: implement SAVE CACHE statement

Serialization Exception was a secondary issue. It was actually caused by unsuccessful class lookup. This is now fixed... Boris Schegolev

12:24 PM Feature #6453: temp-table features

I've fixed the issue with the indexes (un)marshaling by adding optional structured index data to the @TableWrapper@. ... Igor Skornyakov

04:03 AM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
>
> I'm still thinkin... Igor Skornyakov

11/03/2022

07:12 PM Feature #6371: implement SAVE CACHE statement

I pushed support for class unloading. I still need to do some refactoring and fix a @java.io.NotSerializableException... Boris Schegolev

10:29 AM Feature #6453: temp-table features

Fixed support for a binded table parameter.
Please review (see the attached .diff file).
Thank you.
Working ... Igor Skornyakov

10:18 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> In 4GL when @SCHEMA-MARCHAL@ is @NONE@ and the input parameter of the called procedure is d... Igor Skornyakov

05:24 AM Feature #6453: temp-table features

In 4GL when @SCHEMA-MARCHAL@ is @NONE@ and the input parameter of the called procedure is defined as @table FOR <loca... Igor Skornyakov

08:34 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

Committed 6708a/rev.14327. Added an initial version for tracing AdaptiveQuery and important data related to it.
Al... Dănuț Filimon

11/02/2022

03:52 PM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
Yes, this helps. Than... Igor Skornyakov

12:10 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> It seems now that the full set of knowledge about the indexes is needed. So I am definite... Igor Skornyakov

11:07 AM Feature #6453: temp-table features

Eric Faulhaber wrote:
> > Please let me know what you think about my suggestion #6453-120 and related idea about the... Ovidiu Maxiniuc

10:37 AM Feature #6453: temp-table features

Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
I'm still thinking about the index issue. Constantin Asofiei

09:39 AM Feature #6453: temp-table features

I've added reporting @12323@ error reporting ('A NO-SCHEMA-MARSHAL table cannot be used as a parameter where the rece... Igor Skornyakov

01:05 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Please let me know what you think about my suggestion #6453-120 and related idea about th... Eric Faulhaber

02:23 PM Feature #4056: instrument FWD and supporting libraries to measure database performance

I've committed to 6129b/14318 the changes in DatasetWrapper (serialization ID) and SortIndex (made the class public),... Constantin Asofiei

11/01/2022

04:10 PM Feature #6453: temp-table features

I'm working now on table parameter issues described in #6453-133 and #6453-134.
After that I plan to return to work ... Igor Skornyakov

08:02 AM Feature #6453: temp-table features

One more problem with table parameter in remote call.
If the parameter is declared like this:... Igor Skornyakov

06:52 AM Feature #6453: temp-table features

Added 4GL test for the @NONE@ value of the @SCHEMA-MARSHAL@.
FWD results are different from the 4GL ones Igor Skornyakov

04:26 AM Feature #6453: temp-table features

Marian Edu wrote:
> [[https://docs.progress.com/bundle/openedge-develop-abl-applications-122/page/Using-a-temp-table... Igor Skornyakov

03:57 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Sorry for my ignorance. How the declaration of such parameter should look like?
[[https:... Marian Edu

03:47 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
>
> > Thank you Marian. But my question was about the situations when ... Igor Skornyakov

03:42 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Thank you Marian. But my question was about the situations when @NONE@ value of the @SCHE... Marian Edu

03:34 AM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor Skornyakov wrote:
> > At this moment we keep index data in the @TableParameter@ as... Igor Skornyakov

03:26 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> At this moment we keep index data in the @TableParameter@ as a string created by @TableMapp... Constantin Asofiei

11:11 AM Feature #6418: represent extent fields as arrays

Radu Apetrii wrote:
> Greg Shah wrote:
> > The above is my opinion. Eric makes the final decision here.
> Any dec... Eric Faulhaber

09:29 AM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

Please ensure that the solution is built such that all dialects (not just MariaDB) can leverage this same approach. ... Greg Shah

09:33 PM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

I committed an intermediary solution. The occurrences of properties are no longer wrapped in @upper()@ and/or @rtrim(... Ovidiu Maxiniuc

08:33 AM Feature #6898: PostgreSQL schema improvements

We have multiple requests for changes to the default approach to some PostgreSQL schema. We have a small window righ... Greg Shah

08:05 AM Feature #6898 (New): PostgreSQL schema improvements

Greg Shah

08:32 AM Support #4058: consider denormalizing tables as the default approach

We've discussed offline this many times recently. I want to record some thoughts from those discussions:
* The cu... Greg Shah

10/31/2022

06:14 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Radu, in my last post I started right into the code review comments, but I want to note that I appreciate your recogn... Eric Faulhaber

05:38 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Code review @6695.patch@:
Interesting solution. Why is it only applicable for NO-UNDO temp-tables?
The patch do... Eric Faulhaber

07:16 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I have attached a patch that includes the changes regarding preselect queries on temporary no-undo tables. They were ... Radu Apetrii

04:03 PM Feature #6453: temp-table features

Greg Shah wrote:
> > Sorry, I do not understand. I see both @commons-lang@ and @commons-lang3@.
> > Can I use them ... Igor Skornyakov

04:02 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Greg Shah wrote:
> > [[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|Li... Greg Shah

03:21 PM Feature #6453: temp-table features

Greg Shah wrote:
> [[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|List of Supported 3rd Party Li... Igor Skornyakov

02:46 PM Feature #6453: temp-table features

[[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|List of Supported 3rd Party Libraries]] Greg Shah

01:17 PM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, review for the #6453-117 patch:
> * @DynamicTablesHelper@ - please do not use @St... Igor Skornyakov

12:49 PM Feature #6453: temp-table features

Igor, review for the #6453-117 patch:
* @DynamicTablesHelper@ - please do not use @StringUtils.isEmpty@ in apache co... Constantin Asofiei

10:21 AM Feature #6453: temp-table features

At this moment we keep index data in the @TableParameter@ as a string created by @TableMapper.getLegacyIndexInfo@.
... Igor Skornyakov

02:55 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor, sorry for being late on this one... it just slipped my mind I guess. The use for that erro... Igor Skornyakov

02:49 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> The support for @SCHEMA-MARSHAL@ @NONE@ value is still missing (see #6453-116). In partic... Marian Edu

10/29/2022

12:49 PM Feature #6453: temp-table features

Fixed (un)marshalling of fields and @NAMESPACE-URI@/@NAMESPACE-PREFIX@ for table parameter.
However, indexes are sti... Igor Skornyakov

10/28/2022

06:06 PM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> I pushed support for indexes (revision 6371a/14117). There's an issue with detecting if the... Eric Faulhaber

05:15 PM Feature #6371: implement SAVE CACHE statement

I pushed support for indexes (revision 6371a/14117). There's an issue with detecting if the index is primary - this i... Boris Schegolev

03:42 PM Bug #6812: fix FastFindCache to use a shared instance per persistent database

Changed the issue description to match the point made in #6812-3 and #6812-4. Eric Faulhaber

04:36 AM Feature #6895: improve performance of BufferReference methods buffer() and definition()

@RecordBuffer.buffer()@ and @RecordBuffer.define()@ (declared by the @BufferReference@ interface, and implemented by ... Eric Faulhaber

04:35 AM Feature #6895 (New): improve performance of BufferReference methods buffer() and definition()

Eric Faulhaber

03:21 AM Feature #6453: temp-table features

Marian Edu wrote:
> I wouldn't bother too much with MIN/FULL differences... the real use case here was the 'NONE' va... Igor Skornyakov

10/27/2022

11:49 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

I've fixed the issue with the wrong times - the thread-local must be static, for it to work as 'thread-local'. See 6... Constantin Asofiei

10:26 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Extended the method tracing with a new file: @summary_trace.csv@ including
* Summary table of scenarios: root, cou... Alexandru Lungu

04:06 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

The aggregate times per bucket are not displayed yet; working on it.
I will check how the sums are computed (and th... Alexandru Lungu

02:55 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

There are also some cases where a method's aggregate and own times are reported as >100% of the bucket total:
!405... Eric Faulhaber

01:03 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Alexandru, I like the HTML UI, nice work! Is the Grand Totals by package breakdown somewhere in there?
For a large... Eric Faulhaber

08:03 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Ovidiu: Please review. Greg Shah

06:09 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Apart from the tests mentioned in #6714-4, I have also made tests in which the called external procedure passes the t... Dănuț Filimon

06:48 AM Feature #6453: temp-table features

Greg Shah wrote:
> Now is the time to get all details exactly correct. I'm glad it is less likely to be an issue, b... Igor Skornyakov

06:27 AM Feature #6453: temp-table features

Now is the time to get all details exactly correct. I'm glad it is less likely to be an issue, but please do impleme... Greg Shah

03:41 AM Feature #6453: temp-table features

Marian Edu wrote:
> I wouldn't bother too much with MIN/FULL differences... the real use case here was the 'NONE' va... Igor Skornyakov

03:39 AM Feature #6453: temp-table features

It looks that when @SCHEMA-MARSHAL@ is @FULL@, 4GL serializes all field attributes of the table parameter of the remo... Igor Skornyakov

01:59 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Ovidiu Maxiniuc wrote:
> > My surprise was to see the last htt-MIN.xsd and htt-FULL.xsd ar... Marian Edu

10/26/2022

03:22 PM Feature #6348: implement support for MariaDB

I have an update which passes my (rather simple) testcase (unlike the current implementation from 6129 branches). I r... Ovidiu Maxiniuc

06:48 AM Feature #6348: implement support for MariaDB

It seems like sequences are not implemented in MariaDB for general usage. There must be some kind of use case that a... Greg Shah

09:27 PM Feature #6348: implement support for MariaDB

Further information on sequences.
It seems that we cannot rely on @lastval()@ method at all to detect the current ... Ovidiu Maxiniuc

02:29 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> My surprise was to see the last htt-MIN.xsd and htt-FULL.xsd are pretty identical. Initiall... Igor Skornyakov

02:26 PM Feature #6453: temp-table features

When I initially added this attribute I used a possible older manuals. It seems it is unchanged, though.
I think you... Ovidiu Maxiniuc

01:39 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> A bit strange. The OE versions looks identical in both FULL and NONE case.
> https://docs.... Igor Skornyakov

01:24 PM Feature #6453: temp-table features

A bit strange. The OE versions looks identical in both FULL and NONE case.
https://docs.progress.com/ru-RU/bundle/op... Ovidiu Maxiniuc

12:02 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Constantin,
> If you have time can you please advise me how to run this test with FWD? I... Igor Skornyakov

06:08 AM Feature #6453: temp-table features

Please find the final version of my test and its results attached.
It looks that currently we serialize less fields'... Igor Skornyakov

05:59 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > When running attached test with an application server, I see the fol... Igor Skornyakov

05:41 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> When running attached test with an application server, I see the following in the server lo... Marian Edu

05:36 AM Feature #6453: temp-table features

When running attached test with an application server, I see the following in the server log:... Igor Skornyakov

02:40 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > Marian,
> > Sorry for disturbing you, but I've stuck with the probl... Igor Skornyakov

02:02 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Marian,
> Sorry for disturbing you, but I've stuck with the problem of running my test wit... Marian Edu

01:06 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> OK, I understand now. If the ASTs are found in cache, the time _"was 40-50ms for dynamic ca... Constantin Asofiei

12:59 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

OK, I understand now. If the ASTs are found in cache, the time _"was 40-50ms for dynamic case"_ which is pretty close... Ovidiu Maxiniuc

12:47 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> That is correct.
> I see that there are 2 buffers involved so there is a bit of initializa... Constantin Asofiei

12:46 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

That is correct.
I see that there are 2 buffers involved so there is a bit of initialization and maybe some optimiza... Ovidiu Maxiniuc

12:29 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> The engine for dynamic queries will replace the inlined values with parameters. So all @con... Constantin Asofiei

12:07 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> The dynamic case is 3-6 times slower than OE, and the static case 2-5 times slower. I a... Ovidiu Maxiniuc

11:56 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

> I am thinking of a specialized AST (JavaAst) with runtime annotations as members instead. No more access using Map ... Greg Shah

11:48 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu, the problem is when the query predicate for the dynamic query is really unique (think a UUID is used as an op... Constantin Asofiei

11:35 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> Ovidiu, I don't see any calls originating from @interpret@, into the application (via Co... Ovidiu Maxiniuc

11:17 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> Improvements need to be made to dynamic query interpreter, too. It takes ~280ms for 650... Ovidiu Maxiniuc

11:16 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu, I don't see any calls originating from @interpret@, into the application (via ControlFlowOps). Most of the t... Constantin Asofiei

11:04 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Do you have a statistic of where is the most time consumed?
Usually the AST tree is simple enough and there are on... Ovidiu Maxiniuc

09:56 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Improvements need to be made to dynamic query interpreter, too. It takes ~280ms for 6500 'interpret' calls. Constantin Asofiei

10:55 AM Bug #6887: improve performance of dataset/table[-handle] parameters

In a large application, there is this JMX reported times (count/millis/nanos):... Constantin Asofiei

10:48 AM Bug #6887 (Review): improve performance of dataset/table[-handle] parameters

Constantin Asofiei

08:38 AM Feature #6582: implement multi-table AdaptiveQuery

This is the latest comparison of the scenarios in the context of the new H2 feature: incremental indexed sorting (658... Alexandru Lungu

07:09 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

As I recently got time to work with our LTW engine, I suggest extending it to cover the run-time tracking / reporting... Alexandru Lungu

06:28 AM Bug #5306: AdaptiveQuery sets a found record into the buffer twice in dynamic mode

The same issue I encountered when implemented #6582. When @AdaptiveQuery@ was in dynamic mode, the record was fetched... Alexandru Lungu

05:53 AM Feature #6815: configure all cache sizes in the directory, and create documentation for them

This includes the @query_cache_size=1024@ for temp-tables in @H2Helper.setCommonInMemoryProperties@. Constantin Asofiei

05:01 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

I have applied the changes you suggested. Also, I have tested all of them against a large customer application and th... Radu Apetrii

10/25/2022

04:21 PM Feature #6371: implement SAVE CACHE statement

Pushed point 2 from list above. See revision 6371a/14116.
Point 1 is still in progress. Boris Schegolev

12:13 PM Feature #6453: temp-table features

Marian,
Sorry for disturbing you, but I've stuck with the problem of running my test with the OE App. Server.
The t... Igor Skornyakov

11:53 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Committed 6129b/rev. 14307 including the new support for the HTML presentation of the method tracing data. There are ... Alexandru Lungu

10:16 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Alexandru Lungu wrote:
> Eric Faulhaber wrote:
> > Please commit the same update to branch 6129b today. Thank you.
... Eric Faulhaber

04:03 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Eric Faulhaber wrote:
> Please commit the same update to branch 6129b today. Thank you.
Done it in 6129b/rev. 143... Alexandru Lungu

02:10 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Dănuț Filimon wrote:
> Committed 3821c/rev.14313. Added AspectJ on @com.goldencode.p2j.persist.orm@ subpackage while... Eric Faulhaber

08:41 PM Bug #6837 (WIP): prevent unnecessary upper/rtrim injection in queries if the data type doesn't re...

Eric Faulhaber wrote:
> I agree that we don't want dialect-specific metadata in DMO annotations. OTOH, I think it ma... Ovidiu Maxiniuc

10/24/2022

07:23 PM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

I agree that we don't want dialect-specific metadata in DMO annotations. OTOH, I think it makes sense to store that m... Eric Faulhaber

06:34 PM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

The dropping of the @upper()@ / @rtrim()@ functions injected during conversion must be done anyway. Initially, it wil... Ovidiu Maxiniuc

01:08 PM Bug #4071: FWD server stops allowing new web client logins after abend

I'll also try to force the race condition Greg detected using a small sleep. Tijs Wickardt

12:49 PM Bug #4071: FWD server stops allowing new web client logins after abend

Okay. I'll try to hammer the system (testcases and/or <app>) concurrently, trying to overlap client shutdowns. Tijs Wickardt

12:14 PM Bug #4071: FWD server stops allowing new web client logins after abend

Unfortunately, I never found a reliable way to recreate this condition, else it probably would be fixed already. Eric Faulhaber

11:39 AM Bug #4071: FWD server stops allowing new web client logins after abend

Eric: Do you recall the details on how this problem can be recreated? Tijs is asking for guidance in #6787-70.
If... Greg Shah

07:45 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Greg Shah wrote:
> > Greg, is this a good moment to add the patch to 3821c?
>
> How safe are the changes? Have w... Radu Apetrii

07:25 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

> Greg, is this a good moment to add the patch to 3821c?
How safe are the changes? Have we tested them in one of ... Greg Shah

06:52 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

I reviewed the @6830.patch@. Nice job!
A couple of notes related to affected code:
* in @FastCopyHelper.java@ I s... Ovidiu Maxiniuc

06:15 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Here's the patch you requested. It was built on 3821c, rev. 14307.
I also modified @ReportApi.persistSearchHistory@ ... Radu Apetrii

07:05 AM Bug #6868: MariaDB Dialect can't properly check if a sequence exists

Radu Apetrii wrote:
> As a side note, I noticed there was a slight difference in results when running the test from ... Ovidiu Maxiniuc

06:23 AM Bug #6868: MariaDB Dialect can't properly check if a sequence exists

As a side note, I noticed there was a slight difference in results when running the test from #6868-1 in both Postgre... Radu Apetrii

10/21/2022

07:10 PM Feature #6371: implement SAVE CACHE statement

I pushed an update that handles the properties of columns (types, nullable, default, etc.), provides null-safe readin... Boris Schegolev

09:39 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Committed 3821c/rev.14313. Added AspectJ on @com.goldencode.p2j.persist.orm@ subpackage while keeping @com.goldencode... Dănuț Filimon

03:43 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Radu, please post a patch here for review, and mention the branch and revision it was built on. Constantin Asofiei

10/20/2022

11:55 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

Radu Apetrii wrote:
> * I could not test @MariaDbDialect.getSequenceSetValString@ because I cannot access sequences ... Ovidiu Maxiniuc

06:57 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

I have tested these prepared statements (from the last post) and I got the following results:
* Out of 16 modificati... Radu Apetrii

11:41 AM Bug #6868 (Feedback): MariaDB Dialect can't properly check if a sequence exists

I assume you use 3821c.
This is an issue which was addressed in another branch. To allow you advance until the merge... Ovidiu Maxiniuc

05:57 AM Bug #6868: MariaDB Dialect can't properly check if a sequence exists

I tried executing with MariaDB (10.3.34) database this example:... Radu Apetrii

05:50 AM Bug #6868 (Feedback): MariaDB Dialect can't properly check if a sequence exists

Radu Apetrii

09:13 AM Feature #6418: represent extent fields as arrays

Greg Shah wrote:
> The above is my opinion. Eric makes the final decision here.
Any decision yet?
Radu Apetrii

09:10 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

There is already some work for #6720 which is close to this thread.
*I am planning to move my changes from 6582a to ... Radu Apetrii

06:23 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

I got it now. It was a mistake from my part (I was not using @-a@ option on the target server to be traced, but on th... Alexandru Lungu

05:17 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

> The queries (adaptive, preselect or compound) are not considered for "debug weaving" nor for "debug not weaving" (b... Greg Shah

12:24 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Eric Faulhaber wrote:
> For reference, I am attaching another @trace.csv@ file that was generated after implementi... Eric Faulhaber

10/19/2022

06:24 PM Feature #6720: lazy hydration

Yes, that is what I calling partial record hydration. Greg Shah

03:18 PM Feature #6720: lazy hydration

Greg, I think that #5731-399 doesn't implement lazy hydration. It just extends the usage of FWD Persistence API for e... Sergey Ivanovskiy

09:42 AM Feature #6720: lazy hydration

Code posted in #5731-399 implements partial record hydration. Greg Shah

04:01 PM Feature #4056: instrument FWD and supporting libraries to measure database performance

TBH, I've never fully understood the AspectJ weaving implementation, though back when I initially implemented this, I... Eric Faulhaber

04:09 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

I could set-up this feature with 3821c/rev.14305 and Hotel GUI. I am starting a standard server and then I attach the... Alexandru Lungu

09:41 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Code posted in #5731-399 implements partial record hydration. Greg Shah

09:07 AM Feature #6453: temp-table features

Greg Shah wrote:
> We need to match the results exactly. No short cuts.
I see. So I definitely need to be able t... Igor Skornyakov

09:02 AM Feature #6453: temp-table features

We need to match the results exactly. No short cuts. Greg Shah

08:56 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> BTW: what can go wrong if we ignore @SCHEMA-MARSHAL@ and always send *all* schema attribu... Marian Edu

08:51 AM Feature #6453: temp-table features

Marian Edu wrote:
> If it's easier for you we can write the 4GL tests on our side and then you run those on FWD, e... Igor Skornyakov

08:06 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> If I will be able to run at least one test from @testcases/appsrv@ with 4GL I believe I w... Marian Edu

06:11 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > Marian,
> > I've finished (hopefully) with other substasks and star... Igor Skornyakov

05:54 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Marian,
> I've finished (hopefully) with other substasks and started working on testing @S... Marian Edu

04:08 AM Feature #6453: temp-table features

Marian Edu wrote:
> Greg Shah wrote:
> > Constantin/Marian: Can you please advise Igor on how he can test the behav... Igor Skornyakov

07:42 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

I have created tests with the mentioned specifications and all passed (same results were obtained from the executed s... Dănuț Filimon