FWD » Core Development » Database

11/24/2022

10:57 AM Feature #6582: implement multi-table AdaptiveQuery

> consider way stronger optimizations directly in the H2 engine: this involves some extended work into the H2 optimiz... Greg Shah

10:56 AM Feature #6582: implement multi-table AdaptiveQuery

*CHECKPOINT*
I am pending the work here as I can't find any more consistent optimizations to motivate that multi-t... Alexandru Lungu

10:24 AM Support #6709: track nested FIND inside a related FOR loop

> This is my intent, but I wanted to share my work until now.
Understood. No worries.
> Also, I guess you mean... Greg Shah

05:00 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> Neither annotations/annotations.xml and annotations/annotations_prep.xml can't be used for the ne... Dănuț Filimon

10:05 AM Feature #6453: temp-table features

> Eric, what should we do: remove this list or expose this as a enhanced feature with a method in SESSION handle?
... Greg Shah

01:15 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> > Actually, if we replace throwing @XmlStreamException@ with @return false@ this will resul... Igor Skornyakov

08:13 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> In fact, I do not understand why we add @TempTableBuilder@ to the @HandleChain@. I have not... Ovidiu Maxiniuc

09:43 AM Feature #6448 (WIP): make location of LOB files configurable for import

Igor Skornyakov

09:42 AM Feature #6459: query features

I do not see how we can reliably figure out how the @CACHE@ attribute value affects the performance. Such things usua... Igor Skornyakov

09:39 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

Committed 3821c/rev.14384. I applied all the suggestions mentioned by Alexandru in #6708-7 and moved all the changes ... Dănuț Filimon

09:06 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

I rebased 6708a with 3821c. Danut, you can now move the changes from 6708a to 3821c. Alexandru Lungu

11/23/2022

01:39 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> In fact, the caller of this method just returns @false@ on @XmlStreamException@. Since this... Igor Skornyakov

06:52 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> This class is unique in the fact that we have control of the short lifetime of the @TempTab... Igor Skornyakov

11:16 AM Support #6709: track nested FIND inside a related FOR loop

Code Review 6709-rpt.diff
Neither @annotations/annotations.xml@ and @annotations/annotations_prep.xml@ can't be us... Greg Shah

07:40 AM Support #6709: track nested FIND inside a related FOR loop

I took a different approach after taking a look at #3194 and modified how the report is made. I created a new rule se... Dănuț Filimon

06:20 AM Bug #6923: inner transaction block reverts too much

I attached a patch with some changes that, in my opinion, make more sense than the current implementation. Mainly, th... Radu Apetrii

11/22/2022

01:52 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Sorry, but I'm confused. We create a new @TempTableBuilder@ instance e.g. in the @readTable... Ovidiu Maxiniuc

12:41 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> The problem is that @TempTableBuilder@ -s are @HandleChain@ objects which are kept in a lin... Igor Skornyakov

08:41 AM Feature #6459: query features

According to my tests, both forms of @FIRST-OF@/@LAST-OF@ are fully supported.
I've also fixed the problem with @... Igor Skornyakov

11/21/2022

05:58 PM Bug #6919 (WIP): fix ErrorManager.displayError in TempTableBuilder

I am attaching my changes for review. This should be the complete implementation, I will do more testing tomorrow. Boris Schegolev

02:57 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Related to 6129b/14329: after @ErrorManager.recordOrShowError@ do not @throw@ a new excepti... Igor Skornyakov

02:21 PM Feature #6453: temp-table features

Igor,
Related to 6129b/14329: after @ErrorManager.recordOrShowError@ do not @throw@ a new exception. After the 4GL... Ovidiu Maxiniuc

09:52 AM Feature #6453: temp-table features

Greg Shah wrote:
> >> how safe is to abend here?
> > I think that it is safe because actually, it should never happ... Igor Skornyakov

07:30 AM Feature #6453: temp-table features

>> how safe is to abend here?
> I think that it is safe because actually, it should never happen if the schema is co... Greg Shah

07:40 AM Bug #6940 (Test): unqualified field name resolution when there is an explicit buffer defined in a...

Code Review Task Branch 6129b Revision 14327
The concept of the change is sound. Combined with the fact that it d... Greg Shah

07:38 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

In the end there was no information in email, @testcases/uast/@, Redmine or @bzr@ that could lead be back to the orig... Greg Shah

11/19/2022

08:16 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

There is no change in the application used for task #346. In the customer's app, all changes look 'expected', the pa... Constantin Asofiei

08:00 AM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Review for 6129b/14326:
> * how safe is to abend here?
> [...]
I think that it is saf... Igor Skornyakov

07:41 AM Feature #6453: temp-table features

Review for 6129b/14326:
* how safe is to abend here?... Constantin Asofiei

11/18/2022

05:51 PM Feature #6450 (Closed): buffer/buffer-field features

Eric Faulhaber

01:59 PM Feature #6450: buffer/buffer-field features

Yes, I think so.
Note that, unlike #6444 (where the changes were committed to 3821c), this task uses 6129a/b. Ovidiu Maxiniuc

10:50 AM Feature #6450: buffer/buffer-field features

Can this be closed? Eric Faulhaber

01:50 PM Feature #6444: dataset improvements

Eric Faulhaber wrote:
> Ovidiu, could you please review the list above and indicate what the actual support is curre... Ovidiu Maxiniuc

01:14 PM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

The change was introduced in trunk revision 4785:... Greg Shah

12:56 PM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

I think the intent was to promote the parameter buffer, and not the 'source' for the parameter buffer. For example, ... Constantin Asofiei

12:13 PM Feature #6459: query features

Greg Shah wrote:
> There are 2 forms of these: built-in function and handle-based method.
>
> Built-In Function f... Igor Skornyakov

11:42 AM Feature #6459: query features

There are 2 forms of these: built-in function and handle-based method.
Built-In Function form:... Greg Shah

10:21 AM Feature #6459: query features

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
>
> > In 6129b both @FIRST-OF@ and @LAST-OF@ are set to @rw.cvt_l... Igor Skornyakov

10:20 AM Feature #6459: query features

The @convert/methods_attributes.rules@ has support for both methods. Subclasses of @P2JQuery@ like @Persorter@ have ... Greg Shah

10:05 AM Feature #6459: query features

Igor Skornyakov wrote:
> In 6129b both @FIRST-OF@ and @LAST-OF@ are set to @rw.cvt_lvl_full | rw.rt_lvl_full@
... Eric Faulhaber

09:42 AM Feature #6459: query features

Greg Shah wrote:
> > As far as I can see from my test, @FIRST-OF@, @LAST-OF@ methods are fully supported.
>
> At ... Igor Skornyakov

09:17 AM Feature #6459: query features

> As far as I can see from my test, @FIRST-OF@, @LAST-OF@ methods are fully supported.
At a minimum, the gap marki... Greg Shah

08:37 AM Feature #6459: query features

As far as I can see from my test, @FIRST-OF@, @LAST-OF@ methods are fully supported.
However, I see the following is... Igor Skornyakov

09:56 AM Feature #6453 (Closed): temp-table features

Eric Faulhaber

01:36 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Only one change before the commit, please: in @XmlImport.java@, instead of [...]use[...]
>... Igor Skornyakov

08:24 AM Bug #6923: inner transaction block reverts too much

Thank you for the explanations!
I've taken them on board and I have seen that the bug is in fact about the restorati... Radu Apetrii

08:01 AM Support #6709: track nested FIND inside a related FOR loop

Nice! Please post the diff so I can review the approach. It can be tricky to do this kind of static analysis becaus... Greg Shah

05:51 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> We are trying to evaluate how often in real 4GL code does this condition occur? Getting a measure... Dănuț Filimon

11/17/2022

06:39 PM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

FYI, I'm digging back through the history on this one. I'll pick it back up in the morning. Greg Shah

08:02 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

The patch is built on top of 6129b/14324. Constantin Asofiei

08:01 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

Greg, the problem is that for a @DEF PARAM BUFFER ... FOR some-buf@ statement, @some-buf@ gets promoted because @def_... Constantin Asofiei

02:52 AM Bug #6940: unqualified field name resolution when there is an explicit buffer defined in an inter...

The problem is when a buffer is defined explicitly as a parameter - FWD doesn't resolve the unqualified field properl... Constantin Asofiei

02:46 AM Bug #6940 (Test): unqualified field name resolution when there is an explicit buffer defined in a...

Constantin Asofiei

03:18 PM Feature #6453: temp-table features

Only one change before the commit, please: in @XmlImport.java@, instead of ... Ovidiu Maxiniuc

02:20 PM Feature #6453 (Review): temp-table features

Igor Skornyakov wrote:
> Ovidiu Maxiniuc wrote:
> > Review of @read-xmlschema.diff@:
> [...]
> Can I commit the c... Eric Faulhaber

05:03 AM Feature #6453: temp-table features

Tests used for this task committed to @xfer.goldencode.com/opt/testcases/temp-table-marshal@. rev. 1300.
Igor Skornyakov

01:58 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Review of @read-xmlschema.diff@:
> * @AbstractTempTable@: the new method lacks the javadoc... Igor Skornyakov

02:32 PM Feature #6371: implement SAVE CACHE statement

FYI, I am re-working the class unloading, which will not work currently in "real" use scenarios. When we create a DMO... Eric Faulhaber

02:02 PM Bug #6923: inner transaction block reverts too much

Eric Faulhaber wrote:
> ...the fact that the results are different in your first test case between temp-table and pe... Eric Faulhaber

01:52 PM Bug #6923: inner transaction block reverts too much

When you nest transactions like:... Eric Faulhaber

06:50 AM Bug #6923 (WIP): inner transaction block reverts too much

An update to this issue:
I narrowed down the code, in order to describe what I have found until this moment. This is... Radu Apetrii

12:02 PM Feature #6418: represent extent fields as arrays

Alexandru Lungu wrote:
> Eric, is there any prior partial implementation/discussion into the array fields?
There ... Eric Faulhaber

03:29 AM Feature #6418: represent extent fields as arrays

Eric, is there any prior partial implementation/discussion into the array fields? If no, this task can be the lead ef... Alexandru Lungu

07:35 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

The @2137_20221114a.patch@ patch was committed to 3821c/14369, with these notes:
* only NO-LOCK records are loaded a... Constantin Asofiei

07:34 AM Bug #6941: conversion issue when using undo in do transaction

There is a problem when converting examples that contain an @undo@ statement inside a @do transaction@.
For example,... Radu Apetrii

07:20 AM Bug #6941 (Rejected): conversion issue when using undo in do transaction

Radu Apetrii

05:40 AM Feature #6816: improve PreselectQuery.assembleHQL

For an initial check, I recommend implementing a very small suite of performance tests (~6) for @PreselectQuery@ to t... Alexandru Lungu

03:33 AM Feature #6816 (WIP): improve PreselectQuery.assembleHQL

Alexandru Lungu

11/16/2022

03:54 PM Feature #6453: temp-table features

Review of @read-xmlschema.diff@:
* @AbstractTempTable@: the new method lacks the javadoc;
* @XmlImport@:
** method... Ovidiu Maxiniuc

11:59 AM Feature #6453: temp-table features

Finished with fixing @READ-XMLSCHEMA@ support for DATASET.
Please review.
Thank you.
I understand that the tas... Igor Skornyakov

01:01 PM Feature #6459: query features

Igor, please focus only on the first bullet point (i.e., @QUERY@ handle) items. Eric Faulhaber

12:28 PM Feature #6448: make location of LOB files configurable for import

Eric Faulhaber wrote:
> Currently, the location is hard-coded to a @lobs@ sub-directory of the directory containing ... Eric Faulhaber

10:16 AM Feature #6938: XML seriliazation improvement

At this moment the @XMLExport@ class uses manual formatting of the document based on @indent()@/@leneBreak()@ calls (... Igor Skornyakov

10:08 AM Feature #6938 (New): XML seriliazation improvement

Igor Skornyakov

10:14 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

3821c is fine. Thanks. Eric Faulhaber

09:04 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

I have no objection.
Eric: Where should the changes go? I'd prefer 3821c. Greg Shah

06:27 AM Support #6708 (WIP): track/report at runtime when AdaptiveQuery shifts into dynamic mode

Greg, can we consider moving this change to 3821c or 6129b? It does not change the FWD core behavior; it only augment... Alexandru Lungu

06:34 AM Support #6709: track nested FIND inside a related FOR loop

This is a good analysis. It certainly highlights the differences in control flow and danger of refactoring such code... Greg Shah

05:57 AM Support #6709 (WIP): track nested FIND inside a related FOR loop

Dănuț Filimon

05:36 AM Support #6709: track nested FIND inside a related FOR loop

From what I understood, we want to test if either nested find or joined approach is better. After a few tests using t... Dănuț Filimon

05:56 AM Feature #5355 (WIP): _Usertablestat metadata improvements

Alexandru Lungu

05:47 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Any news about the review? Dănuț Filimon

11/15/2022

09:33 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Right. I failed to notice the @field@ is updated in inner loop. Ovidiu Maxiniuc

08:07 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> Review of @2137_20221114a.patch@:
> I have a single issue in the new patch, in @AbstractQu... Constantin Asofiei

06:45 AM Feature #6928: H2 UPDATE performance improvement

In H2, when doing an UPDATE statement,the approach is to:
* create a new row
* assign all columns not part of the U... Constantin Asofiei

03:49 AM Feature #6928 (Closed): H2 UPDATE performance improvement

Constantin Asofiei

11/14/2022

06:37 PM Feature #6371: implement SAVE CACHE statement

Pushed revision 6371a/14121. Based on testing, added more mappings for column types and moved mapping to @Dialect@ pl... Boris Schegolev

02:03 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Ovidiu, please review attached. This adds denormalized extent for FIELDS/EXCEPT. norma... Ovidiu Maxiniuc

10:31 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu, please review attached. This adds denormalized extent for FIELDS/EXCEPT. normalized extent is still not sup... Constantin Asofiei

01:24 PM Feature #6453: temp-table features

It appears that @READ-XMLSCHEMA@ for the @DATASET@ is broken even in the situation of a single namespace (w/o externa... Igor Skornyakov

11:01 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Certainly, the patch is clearly an improvement over the current revision on bzr. Please c... Igor Skornyakov

10:44 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> > What I meant was that a @ClassCastException@ could be thrown if code inside FWD will ca... Ovidiu Maxiniuc

10:42 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > Why not extend @XMLStreamWriter@ so that it will care about ind... Igor Skornyakov

10:37 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Why not extend @XMLStreamWriter@ so that it will care about indentation itself? [...] What ... Ovidiu Maxiniuc

12:28 PM Bug #6887: improve performance of dataset/table[-handle] parameters

I've edited previous note to add @delete object hds1.@ to @dstest.p@ - with this, OE is in the 200ms range, while FWD... Constantin Asofiei

12:04 PM Bug #6887: improve performance of dataset/table[-handle] parameters

A starting point for a test to check the performance is this, made of two programs:
# @dstest.p@ which defines inter... Constantin Asofiei

11:14 PM Bug #6887: improve performance of dataset/table[-handle] parameters

Constantin, is this something you think can reasonably be reproduced with some simplified test cases, which would ill... Eric Faulhaber

09:07 AM Bug #6829: H2 forces re-parse of all prepared statements when metadata is changed

Alexandru, I have in a heap dump @UnclosablePreparedStatement@ instances with ~90KB retained heap. But in at least a... Constantin Asofiei

06:23 AM Bug #6829: H2 forces re-parse of all prepared statements when metadata is changed

I've done some simple debugging in Hotel GUI with @UnclosablePreparedStatement@. The @psCache@ structure is an @Expir... Alexandru Lungu

07:46 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

Committed 6708a/rev.14328. Made improvements to AdaptiveQuery method tracing.
I made changes and now it is possibl... Dănuț Filimon

11/12/2022

04:54 AM Feature #6453: temp-table features

In addition to the previous note. I
Why not extend @XMLStreamWriter@ so that it will care about indentation itself? ... Igor Skornyakov

03:49 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> > > * @TempTableResultSet@:387: you are creating an intermediary @mprops@ tree for collecti... Igor Skornyakov

11/11/2022

03:41 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > > * @TempTableResultSet@:387: you are creating an intermediary ... Igor Skornyakov

03:34 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> > * @TempTableResultSet@:387: you are creating an intermediary @mprops@ tree for collecting... Ovidiu Maxiniuc

01:37 PM Feature #6453: temp-table features

Fixed @WRITE-XML@ support for @DATASET@.
See the attached .diff file for 6129b/14320.
The only remaining thing is... Igor Skornyakov

01:34 PM Feature #6453: temp-table features

Fixed @WRITE-XML@ support for @DATASET@ Igor Skornyakov

07:24 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Review of @marshal4.diff@
> Overall: a big update, handling multiple issues. Nice work. He... Igor Skornyakov

02:34 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > I work with branch 6129b. Please find the .diff for rev.14320 a... Igor Skornyakov

09:20 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> I work with branch 6129b. Please find the .diff for rev.14320 attached.
Review of @marsh... Ovidiu Maxiniuc

11/10/2022

03:51 PM Feature #6453: temp-table features

Completely (as far as my tests demonstrate) fixed @WRITE-XMLSCHEMA@ for standalone tables and datasets, including ext... Igor Skornyakov

06:09 AM Bug #6923: inner transaction block reverts too much

Update: If you replace the middle transaction block with the code below, the program works fine. It reverts only the ... Radu Apetrii

05:26 AM Bug #6923: inner transaction block reverts too much

Here is the code that I've been working with, on 3821c, rev.14354:... Radu Apetrii

05:25 AM Bug #6923 (Pending): inner transaction block reverts too much

Radu Apetrii

01:42 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> * @BaseRecord@:
> ** @setDatum()@: the conditional update of @readFields@ should happen at... Constantin Asofiei

09:55 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Eric/Ovidiu, please review the attached patch.
I like the update. The boolean arrays ... Ovidiu Maxiniuc

11/09/2022

04:43 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Tijs Wickardt wrote:
> > # Give each ORM data object a ByteArray, holding the raw (unse... Tijs Wickardt

04:38 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Tijs Wickardt wrote:
> # Give each ORM data object a ByteArray, holding the raw (unserialized) data of the last fetc... Constantin Asofiei

03:43 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> Eric/Ovidiu, please review the attached patch. The details are these:
> * I couldn't f... Tijs Wickardt

02:20 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Another note: the @orm.Loader.load@ will retrieve from the database the full records, but they will not be populated ... Constantin Asofiei

02:14 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Eric/Ovidiu, please review the attached patch. The details are these:
* I couldn't find a good way to add support f... Constantin Asofiei

09:35 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

There is another finding: for EXCLUSIVE-LOCK, the record is always fully-fetched. In FWD, the partial record is allo... Constantin Asofiei

01:52 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Ovidiu Maxiniuc wrote:
> Constantin Asofiei wrote:
> > I have a (minimal) set of changes to allow EXCEPT/FIELDS at ... Constantin Asofiei

08:18 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

> This is odd. I did not try it. The logical implementation is to fully fetch the respective at that moment. Is this ... Greg Shah

02:37 PM Feature #6453: temp-table features

It appears that we have more problems with DATASET support than I initially reported in #6453-147. The most important... Igor Skornyakov

12:49 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > Fixed DATASET (un)marshaling. See the attached .diff file.
> >... Igor Skornyakov

07:48 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Fixed DATASET (un)marshaling. See the attached .diff file.
> Please review. There are many... Ovidiu Maxiniuc

12:47 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I have taken into account what you said and I have made some changes that appear in this new patch. They were tested ... Radu Apetrii

07:36 AM Bug #5307 (WIP): AdaptiveQuery goes into dynamic mode too aggressively

I've been trying to reproduce the behavior mentioned in #5307-1 using the example provided and failed. When debugging... Dănuț Filimon

03:13 AM Support #6712: check generated dynamic temp-table caching to ensure that all cache keys work when...

IIRC, the problem was that we rely on @P2JField.computeHash@, and the @name@ field can vary, depending on the call pa... Eric Faulhaber

11/08/2022

05:23 PM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Constantin Asofiei wrote:
> I have a (minimal) set of changes to allow EXCEPT/FIELDS at a query, with these notes:
... Ovidiu Maxiniuc

11:14 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Things get tricky when you have an ignored field and you update that field - this is allowed, and OE will save the re... Constantin Asofiei

09:53 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

Another issue is that the records which are partially hydrated can not be cached (@FastFindCache@ or @orm.Session.cac... Constantin Asofiei

08:38 AM Feature #2137: runtime support for FIELDS/EXCEPT record phrase options

I have a (minimal) set of changes to allow EXCEPT/FIELDS at a query, with these notes:
* temp-tables do not support ... Constantin Asofiei

12:43 PM Bug #6919: fix ErrorManager.displayError in TempTableBuilder

FWD uses @ErrorManager.displayError@ instead of going through @recordOrShowError@, in i.e. @TempTableBuilder.tempTabl... Constantin Asofiei

12:36 PM Bug #6919 (Review): fix ErrorManager.displayError in TempTableBuilder

Constantin Asofiei

06:28 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Constantin Asofiei wrote:
> > Igor, you need to use @ErrorManager.recordOrThrow@ most like... Igor Skornyakov

05:10 AM Feature #6453: temp-table features

Fixed DATASET (un)marshaling. See the attached .diff file.
Please review. There are many pended changes. I would pre... Igor Skornyakov

11/07/2022

12:03 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Very well. This is s strong reason to refactor the inter-process serialization (the @Extern... Igor Skornyakov

11:37 AM Feature #6453: temp-table features

> Thank you, Ovidiu. However, in my tests, there is no difference between a standalone TEMP-TABLE and one which is a ... Ovidiu Maxiniuc

11:23 AM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> Yes, this is true. Another reason (actually more important, at least at the beginning) was ... Igor Skornyakov

11:13 AM Feature #6453: temp-table features

Yes, this is true. Another reason (actually more important, at least at the beginning) was that, in the initial simpl... Ovidiu Maxiniuc

07:10 AM Feature #6453: temp-table features

> I think it will take a day or two.
Please do resolve these issues now. If it is going to take much longer than... Greg Shah

02:47 AM Feature #6453: temp-table features

In addition to the previous note.
In fact, there are many places when we work with a standalone temp table in a diff... Igor Skornyakov

02:35 AM Feature #6453: temp-table features

For some reason FWD processes @TEMP-TABLE@ which is part of the @DATASET@ input parameter differently than when it is... Igor Skornyakov

11/06/2022

12:54 PM Feature #6453: temp-table features

Greg Shah wrote:
> Optimally, yes. But I know Eric has other tasks for you as well. How long do you estimate the f... Igor Skornyakov

11:46 AM Feature #6453: temp-table features

Optimally, yes. But I know Eric has other tasks for you as well. How long do you estimate the fixes to take? Greg Shah

05:38 AM Feature #6453: temp-table features

I see a number of incompatibilities in the @NAMESPACE-URI@ and @NAMESPACE-PREFIX@ support in FWD for @DATASET@.
# FW... Igor Skornyakov

11/04/2022

04:58 PM Feature #6371: implement SAVE CACHE statement

Serialization Exception was a secondary issue. It was actually caused by unsuccessful class lookup. This is now fixed... Boris Schegolev

12:24 PM Feature #6453: temp-table features

I've fixed the issue with the indexes (un)marshaling by adding optional structured index data to the @TableWrapper@. ... Igor Skornyakov

04:03 AM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
>
> I'm still thinkin... Igor Skornyakov

11/03/2022

07:12 PM Feature #6371: implement SAVE CACHE statement

I pushed support for class unloading. I still need to do some refactoring and fix a @java.io.NotSerializableException... Boris Schegolev

10:29 AM Feature #6453: temp-table features

Fixed support for a binded table parameter.
Please review (see the attached .diff file).
Thank you.
Working ... Igor Skornyakov

10:18 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> In 4GL when @SCHEMA-MARCHAL@ is @NONE@ and the input parameter of the called procedure is d... Igor Skornyakov

05:24 AM Feature #6453: temp-table features

In 4GL when @SCHEMA-MARCHAL@ is @NONE@ and the input parameter of the called procedure is defined as @table FOR <loca... Igor Skornyakov

08:34 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

Committed 6708a/rev.14327. Added an initial version for tracing AdaptiveQuery and important data related to it.
Al... Dănuț Filimon

11/02/2022

03:52 PM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
Yes, this helps. Than... Igor Skornyakov

12:10 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> It seems now that the full set of knowledge about the indexes is needed. So I am definite... Igor Skornyakov

11:07 AM Feature #6453: temp-table features

Eric Faulhaber wrote:
> > Please let me know what you think about my suggestion #6453-120 and related idea about the... Ovidiu Maxiniuc

10:37 AM Feature #6453: temp-table features

Igor, you need to use @ErrorManager.recordOrThrow@ most likely.
I'm still thinking about the index issue. Constantin Asofiei

09:39 AM Feature #6453: temp-table features

I've added reporting @12323@ error reporting ('A NO-SCHEMA-MARSHAL table cannot be used as a parameter where the rece... Igor Skornyakov

01:05 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Please let me know what you think about my suggestion #6453-120 and related idea about th... Eric Faulhaber

02:23 PM Feature #4056: instrument FWD and supporting libraries to measure database performance

I've committed to 6129b/14318 the changes in DatasetWrapper (serialization ID) and SortIndex (made the class public),... Constantin Asofiei

11/01/2022

04:10 PM Feature #6453: temp-table features

I'm working now on table parameter issues described in #6453-133 and #6453-134.
After that I plan to return to work ... Igor Skornyakov

08:02 AM Feature #6453: temp-table features

One more problem with table parameter in remote call.
If the parameter is declared like this:... Igor Skornyakov

06:52 AM Feature #6453: temp-table features

Added 4GL test for the @NONE@ value of the @SCHEMA-MARSHAL@.
FWD results are different from the 4GL ones Igor Skornyakov

04:26 AM Feature #6453: temp-table features

Marian Edu wrote:
> [[https://docs.progress.com/bundle/openedge-develop-abl-applications-122/page/Using-a-temp-table... Igor Skornyakov

03:57 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Sorry for my ignorance. How the declaration of such parameter should look like?
[[https:... Marian Edu

03:47 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
>
> > Thank you Marian. But my question was about the situations when ... Igor Skornyakov

03:42 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Thank you Marian. But my question was about the situations when @NONE@ value of the @SCHE... Marian Edu

03:34 AM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor Skornyakov wrote:
> > At this moment we keep index data in the @TableParameter@ as... Igor Skornyakov

03:26 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> At this moment we keep index data in the @TableParameter@ as a string created by @TableMapp... Constantin Asofiei

11:11 AM Feature #6418: represent extent fields as arrays

Radu Apetrii wrote:
> Greg Shah wrote:
> > The above is my opinion. Eric makes the final decision here.
> Any dec... Eric Faulhaber

09:29 AM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

Please ensure that the solution is built such that all dialects (not just MariaDB) can leverage this same approach. ... Greg Shah

09:33 PM Bug #6837: prevent unnecessary upper/rtrim injection in queries if the data type doesn't require it

I committed an intermediary solution. The occurrences of properties are no longer wrapped in @upper()@ and/or @rtrim(... Ovidiu Maxiniuc

08:33 AM Feature #6898: PostgreSQL schema improvements

We have multiple requests for changes to the default approach to some PostgreSQL schema. We have a small window righ... Greg Shah

08:05 AM Feature #6898 (New): PostgreSQL schema improvements

Greg Shah

08:32 AM Support #4058: consider denormalizing tables as the default approach

We've discussed offline this many times recently. I want to record some thoughts from those discussions:
* The cu... Greg Shah

10/31/2022

06:14 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Radu, in my last post I started right into the code review comments, but I want to note that I appreciate your recogn... Eric Faulhaber

05:38 PM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

Code review @6695.patch@:
Interesting solution. Why is it only applicable for NO-UNDO temp-tables?
The patch do... Eric Faulhaber

07:16 AM Feature #6695: Multi-table preselect query may underperform due to repetitive fetching

I have attached a patch that includes the changes regarding preselect queries on temporary no-undo tables. They were ... Radu Apetrii

04:03 PM Feature #6453: temp-table features

Greg Shah wrote:
> > Sorry, I do not understand. I see both @commons-lang@ and @commons-lang3@.
> > Can I use them ... Igor Skornyakov

04:02 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Greg Shah wrote:
> > [[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|Li... Greg Shah

03:21 PM Feature #6453: temp-table features

Greg Shah wrote:
> [[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|List of Supported 3rd Party Li... Igor Skornyakov

02:46 PM Feature #6453: temp-table features

[[p2j::Dependency_Management#List-of-Supported-3rd-Party-Libraries|List of Supported 3rd Party Libraries]] Greg Shah

01:17 PM Feature #6453: temp-table features

Constantin Asofiei wrote:
> Igor, review for the #6453-117 patch:
> * @DynamicTablesHelper@ - please do not use @St... Igor Skornyakov

12:49 PM Feature #6453: temp-table features

Igor, review for the #6453-117 patch:
* @DynamicTablesHelper@ - please do not use @StringUtils.isEmpty@ in apache co... Constantin Asofiei

10:21 AM Feature #6453: temp-table features

At this moment we keep index data in the @TableParameter@ as a string created by @TableMapper.getLegacyIndexInfo@.
... Igor Skornyakov

02:55 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor, sorry for being late on this one... it just slipped my mind I guess. The use for that erro... Igor Skornyakov

02:49 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> The support for @SCHEMA-MARSHAL@ @NONE@ value is still missing (see #6453-116). In partic... Marian Edu

10/29/2022

12:49 PM Feature #6453: temp-table features

Fixed (un)marshalling of fields and @NAMESPACE-URI@/@NAMESPACE-PREFIX@ for table parameter.
However, indexes are sti... Igor Skornyakov

10/28/2022

06:06 PM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> I pushed support for indexes (revision 6371a/14117). There's an issue with detecting if the... Eric Faulhaber

05:15 PM Feature #6371: implement SAVE CACHE statement

I pushed support for indexes (revision 6371a/14117). There's an issue with detecting if the index is primary - this i... Boris Schegolev

03:42 PM Bug #6812: fix FastFindCache to use a shared instance per persistent database

Changed the issue description to match the point made in #6812-3 and #6812-4. Eric Faulhaber

04:36 AM Feature #6895: improve performance of BufferReference methods buffer() and definition()

@RecordBuffer.buffer()@ and @RecordBuffer.define()@ (declared by the @BufferReference@ interface, and implemented by ... Eric Faulhaber

04:35 AM Feature #6895 (New): improve performance of BufferReference methods buffer() and definition()

Eric Faulhaber

03:21 AM Feature #6453: temp-table features

Marian Edu wrote:
> I wouldn't bother too much with MIN/FULL differences... the real use case here was the 'NONE' va... Igor Skornyakov

10/27/2022

11:49 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

I've fixed the issue with the wrong times - the thread-local must be static, for it to work as 'thread-local'. See 6... Constantin Asofiei

10:26 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Extended the method tracing with a new file: @summary_trace.csv@ including
* Summary table of scenarios: root, cou... Alexandru Lungu

04:06 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

The aggregate times per bucket are not displayed yet; working on it.
I will check how the sums are computed (and th... Alexandru Lungu

02:55 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

There are also some cases where a method's aggregate and own times are reported as >100% of the bucket total:
!405... Eric Faulhaber

01:03 AM Feature #4056: instrument FWD and supporting libraries to measure database performance

Alexandru, I like the HTML UI, nice work! Is the Grand Totals by package breakdown somewhere in there?
For a large... Eric Faulhaber

08:03 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Ovidiu: Please review. Greg Shah

06:09 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Apart from the tests mentioned in #6714-4, I have also made tests in which the called external procedure passes the t... Dănuț Filimon

06:48 AM Feature #6453: temp-table features

Greg Shah wrote:
> Now is the time to get all details exactly correct. I'm glad it is less likely to be an issue, b... Igor Skornyakov

06:27 AM Feature #6453: temp-table features

Now is the time to get all details exactly correct. I'm glad it is less likely to be an issue, but please do impleme... Greg Shah

03:41 AM Feature #6453: temp-table features

Marian Edu wrote:
> I wouldn't bother too much with MIN/FULL differences... the real use case here was the 'NONE' va... Igor Skornyakov

03:39 AM Feature #6453: temp-table features

It looks that when @SCHEMA-MARSHAL@ is @FULL@, 4GL serializes all field attributes of the table parameter of the remo... Igor Skornyakov

01:59 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Ovidiu Maxiniuc wrote:
> > My surprise was to see the last htt-MIN.xsd and htt-FULL.xsd ar... Marian Edu

10/26/2022

03:22 PM Feature #6348: implement support for MariaDB

I have an update which passes my (rather simple) testcase (unlike the current implementation from 6129 branches). I r... Ovidiu Maxiniuc

06:48 AM Feature #6348: implement support for MariaDB

It seems like sequences are not implemented in MariaDB for general usage. There must be some kind of use case that a... Greg Shah

02:29 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> My surprise was to see the last htt-MIN.xsd and htt-FULL.xsd are pretty identical. Initiall... Igor Skornyakov

02:26 PM Feature #6453: temp-table features

When I initially added this attribute I used a possible older manuals. It seems it is unchanged, though.
I think you... Ovidiu Maxiniuc

01:39 PM Feature #6453: temp-table features

Ovidiu Maxiniuc wrote:
> A bit strange. The OE versions looks identical in both FULL and NONE case.
> https://docs.... Igor Skornyakov

01:24 PM Feature #6453: temp-table features

A bit strange. The OE versions looks identical in both FULL and NONE case.
https://docs.progress.com/ru-RU/bundle/op... Ovidiu Maxiniuc

12:02 PM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Constantin,
> If you have time can you please advise me how to run this test with FWD? I... Igor Skornyakov

06:08 AM Feature #6453: temp-table features

Please find the final version of my test and its results attached.
It looks that currently we serialize less fields'... Igor Skornyakov

05:59 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > When running attached test with an application server, I see the fol... Igor Skornyakov

05:41 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> When running attached test with an application server, I see the following in the server lo... Marian Edu

05:36 AM Feature #6453: temp-table features

When running attached test with an application server, I see the following in the server log:... Igor Skornyakov

02:40 AM Feature #6453: temp-table features

Marian Edu wrote:
> Igor Skornyakov wrote:
> > Marian,
> > Sorry for disturbing you, but I've stuck with the probl... Igor Skornyakov

02:02 AM Feature #6453: temp-table features

Igor Skornyakov wrote:
> Marian,
> Sorry for disturbing you, but I've stuck with the problem of running my test wit... Marian Edu

01:06 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> OK, I understand now. If the ASTs are found in cache, the time _"was 40-50ms for dynamic ca... Constantin Asofiei

12:59 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

OK, I understand now. If the ASTs are found in cache, the time _"was 40-50ms for dynamic case"_ which is pretty close... Ovidiu Maxiniuc

12:47 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> That is correct.
> I see that there are 2 buffers involved so there is a bit of initializa... Constantin Asofiei

12:46 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

That is correct.
I see that there are 2 buffers involved so there is a bit of initialization and maybe some optimiza... Ovidiu Maxiniuc

12:29 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu Maxiniuc wrote:
> The engine for dynamic queries will replace the inlined values with parameters. So all @con... Constantin Asofiei

12:07 PM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> The dynamic case is 3-6 times slower than OE, and the static case 2-5 times slower. I a... Ovidiu Maxiniuc

11:56 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

> I am thinking of a specialized AST (JavaAst) with runtime annotations as members instead. No more access using Map ... Greg Shah

11:48 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu, the problem is when the query predicate for the dynamic query is really unique (think a UUID is used as an op... Constantin Asofiei

11:35 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> Ovidiu, I don't see any calls originating from @interpret@, into the application (via Co... Ovidiu Maxiniuc

11:17 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Constantin Asofiei wrote:
> Improvements need to be made to dynamic query interpreter, too. It takes ~280ms for 650... Ovidiu Maxiniuc

11:16 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Ovidiu, I don't see any calls originating from @interpret@, into the application (via ControlFlowOps). Most of the t... Constantin Asofiei

11:04 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Do you have a statistic of where is the most time consumed?
Usually the AST tree is simple enough and there are on... Ovidiu Maxiniuc

09:56 AM Feature #6813: analyze and improve performance of dynamic conversion of temp-tables and queries

Improvements need to be made to dynamic query interpreter, too. It takes ~280ms for 6500 'interpret' calls. Constantin Asofiei

10:55 AM Bug #6887: improve performance of dataset/table[-handle] parameters

In a large application, there is this JMX reported times (count/millis/nanos):... Constantin Asofiei

10:48 AM Bug #6887 (Review): improve performance of dataset/table[-handle] parameters

Constantin Asofiei

08:38 AM Feature #6582: implement multi-table AdaptiveQuery

This is the latest comparison of the scenarios in the context of the new H2 feature: incremental indexed sorting (658... Alexandru Lungu

07:09 AM Support #6708: track/report at runtime when AdaptiveQuery shifts into dynamic mode

As I recently got time to work with our LTW engine, I suggest extending it to cover the run-time tracking / reporting... Alexandru Lungu

06:28 AM Bug #5306: AdaptiveQuery sets a found record into the buffer twice in dynamic mode

The same issue I encountered when implemented #6582. When @AdaptiveQuery@ was in dynamic mode, the record was fetched... Alexandru Lungu

05:53 AM Feature #6815: configure all cache sizes in the directory, and create documentation for them

This includes the @query_cache_size=1024@ for temp-tables in @H2Helper.setCommonInMemoryProperties@. Constantin Asofiei

05:01 AM Feature #6830: find and fix all SQL SELECT statements with inlined literal arguments, and rewrite...

I have applied the changes you suggested. Also, I have tested all of them against a large customer application and th... Radu Apetrii