FWD » Core Development » Database

01/03/2023

02:35 PM Feature #6371: implement SAVE CACHE statement

The changes to support the DMO interface/class unloading and re-use at the proxy level are turning into something mor... Eric Faulhaber

02:27 PM Feature #6371: implement SAVE CACHE statement

Greg Shah wrote:
> Let's assume that only the denormalized extents will be supported for now. It doesn't make sense... Eric Faulhaber

09:47 PM Feature #6371: implement SAVE CACHE statement

Let's assume that only the denormalized extents will be supported for now. It doesn't make sense to add the more com... Greg Shah

11:33 AM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> I've changed my test so that data sources used for FILL get data from the permanent tables.... Igor Skornyakov

11:09 AM Feature #6444: dataset improvements

I've changed my test so that data sources used for FILL get data from the permanent tables.
However, at runtime, I s... Igor Skornyakov

07:08 AM Bug #6829: H2 forces re-parse of all prepared statements when metadata is changed

Constantin Asofiei wrote:
> Alexandru, I have in a heap dump @UnclosablePreparedStatement@ instances with ~90KB reta... Alexandru Lungu

03:48 AM Feature #6720 (WIP): lazy hydration

Eric Faulhaber

01/02/2023

05:34 PM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> > If there are any other critical configuration values that cannot be determined by defau... Eric Faulhaber

12/31/2022

03:29 AM Feature #6444: dataset improvements

Eric Faulhaber wrote:
> If I understand the issue correctly, I respectfully disagree. In the event there is no expli... Igor Skornyakov

02:56 AM Feature #6444: dataset improvements

Greg Shah wrote:
> > Greg, please correct me if I am overly aggressive on this.
>
> If the application code can b... Eric Faulhaber

02:17 AM Feature #6444: dataset improvements

Greg Shah wrote:
> In this case, I suspect we should implement the application-visible behavior.
I see. In this c... Igor Skornyakov

02:15 AM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
> I understand that, ultimately, the content of the databases (ABL and FWD) are identically a... Igor Skornyakov

12/30/2022

06:11 PM Feature #6371: implement SAVE CACHE statement

I added an implementation for SchemaValidator + unit test for it. See revision 14442. Boris Schegolev

11:00 AM Feature #6371: implement SAVE CACHE statement

> Do we have an example of the table structure where such 2-field DTTZ storage is used? First column is DATETIME and ... Greg Shah

05:07 PM Feature #6444: dataset improvements

> Greg, please correct me if I am overly aggressive on this.
If the application code can be implemented to rely up... Greg Shah

03:19 PM Feature #6444: dataset improvements

Well, the _binary_ might not have been the best attribute here to describe the compatibility.
I understand that, u... Ovidiu Maxiniuc

02:35 PM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
> I understand that. Can we isolate this damage to recursive relations only? These cases are ... Igor Skornyakov

02:11 PM Feature #6444: dataset improvements

Igor Skornyakov wrote:
>
> > > Should we care about the order in which records are added to the target tables dur... Ovidiu Maxiniuc

01:59 PM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
> > I understand that refactoring from using a 'plain' query string (@hql@) to a query string... Igor Skornyakov

01:34 PM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> I understand that refactoring from using a 'plain' query string (@hql@) to a query string h... Ovidiu Maxiniuc

12/29/2022

03:23 PM Feature #6371: implement SAVE CACHE statement

I pushed an update for error handling and other minor changes - rev. 14441.
Greg Shah wrote:
> We have 2 customer... Boris Schegolev

12:14 AM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> Also, for the primary key: I guess we need to add a validation before registering a new tab... Eric Faulhaber

07:57 PM Feature #6371: implement SAVE CACHE statement

>> 2 What is the issue with timezoned types (timestamp with time zone)?
>
> We need to support them ultimately, but... Greg Shah

12/28/2022

04:47 PM Feature #6371: implement SAVE CACHE statement

Also, for the primary key: I guess we need to add a validation before registering a new table to check that the table... Boris Schegolev

04:41 PM Feature #6371: implement SAVE CACHE statement

Eric Faulhaber wrote:
> Before you changed it to "TODO: Surrogate Primary Key", the note for 8-byte integer read, "T... Boris Schegolev

02:09 PM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> I updated the documentation / WIKI page. A few questions here:
>
> 1 What's the TODO for... Eric Faulhaber

01:59 PM Feature #6444: dataset improvements

> If this blocks you from making headway on the recursive FILL operation, please work on another item that needs atte... Igor Skornyakov

01:51 PM Feature #6444: dataset improvements

Eric Faulhaber wrote:
> I think we should wait for Ovidiu's input when he returns before undertaking any massive cha... Igor Skornyakov

01:43 PM Feature #6444: dataset improvements

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
> > [...]
> > I understand that refactoring from using a 'plain' qu... Eric Faulhaber

01:40 PM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> [...]
> I understand that refactoring from using a 'plain' query string (@hql@) to a query... Eric Faulhaber

01:38 PM Feature #6444: dataset improvements

Eric Faulhaber wrote:
> I don't know what the intent of this variable was, maybe something that is only changed for ... Igor Skornyakov

01:37 PM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> At this moment it looks that @FqlToSqlConverter.generateUniqueSqlColumnNames@ is always @tr... Eric Faulhaber

05:30 AM Feature #6444: dataset improvements

At this moment it looks that @FqlToSqlConverter.generateUniqueSqlColumnNames@ is always @true@.
Can I rely on this?
... Igor Skornyakov

02:56 AM Feature #6444: dataset improvements

Working on the second pass query for the recursive data-relation FILL support I have to deal with the situation when ... Igor Skornyakov

12/27/2022

05:28 PM Feature #6371: implement SAVE CACHE statement

I updated the documentation / WIKI page. A few questions here:
# What's the TODO for Surrogate Primary Key about?
... Boris Schegolev

12/25/2022

04:20 AM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> I think that we also can avoid it if the DATA-SOURCE QUERY uses a single table but at this ... Igor Skornyakov

12/23/2022

11:15 AM Feature #6371: implement SAVE CACHE statement

Eric Faulhaber wrote:
> That seems like a great idea, as long as column-level comments are supported in @create tabl... Boris Schegolev

09:45 AM Feature #6371: implement SAVE CACHE statement

> As for branching, in git I would rebase 6371a onto 6129b. Not sure how to do that in Bazaar and don't want to break... Greg Shah

01:32 AM Feature #6371: implement SAVE CACHE statement

Boris Schegolev wrote:
> I am implementing one-to-many SQL->4GL type mappings. I would only suggest to use column's ... Eric Faulhaber

09:43 AM Feature #6444: dataset improvements

In the relation model, the physical order or the records in the table is irrelevant and the order of records in the r... Igor Skornyakov

12/22/2022

05:09 PM Feature #6371: implement SAVE CACHE statement

I am implementing one-to-many SQL->4GL type mappings. I would only suggest to use column's comment property for 4GL t... Boris Schegolev

04:26 AM Feature #6371: implement SAVE CACHE statement

Boris, please work on the validation of the mutable database requirements described in the [[p2j:Non-Standard_Save_Ca... Eric Faulhaber

09:51 AM Support #6709: track nested FIND inside a related FOR loop

Eric: Please review and advise. Greg Shah

09:49 AM Support #6709: track nested FIND inside a related FOR loop

Code Review 6709-20221222.diff
The conversion changes seem reasonable.
Do we really need to implement this as p... Greg Shah

08:40 AM Support #6709: track nested FIND inside a related FOR loop

I will try to patch a customer application I have on my development environment to test the changes. If everything is... Alexandru Lungu

07:02 AM Support #6709: track nested FIND inside a related FOR loop

I managed to convert Hotel GUI and a few smaller tests and use FieldReference. I actually didn't make use of @convert... Dănuț Filimon

09:50 AM Support #7004 (New): eliminate the off-end processing in the TransactionManager and minimize the ...

Greg Shah

12/21/2022

11:30 AM Feature #6371: implement SAVE CACHE statement

I pushed a cleanup for PostgreSQL dialect - there were MySQL types mixed in. Boris Schegolev

02:11 AM Feature #6371: implement SAVE CACHE statement

Revision 6371a/14438 works around the @SchemaDictionary@ issue described in #6371-80, but the workaround is limited, ... Eric Faulhaber

12/20/2022

05:42 AM Support #6709: track nested FIND inside a related FOR loop

Committed 3821c/rev.14460. Added nested find annotation and report. Any annotated @FIND@ that is nested in a @FOR EAC... Dănuț Filimon

12/19/2022

06:09 PM Support #6709: track nested FIND inside a related FOR loop

Yes. Greg Shah

03:41 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> You've tested conversion for both the large customer application and also Hotel GUI?
Yes, both... Dănuț Filimon

01:32 PM Bug #6834: RAQ navigation on records having NULL values on same index components

The issue is described with details and example in #6755. I will copy it here because that task contains sensitive da... Ovidiu Maxiniuc

10:28 AM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
> I am not very happy with this double iteration. It implies a performance penalty. Ultimatel... Igor Skornyakov

10:12 AM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> I need to iterate over a DATA-SOURCE twice - first with the original QUERY and then with th... Ovidiu Maxiniuc

12/16/2022

02:59 PM Feature #6444: dataset improvements

Something is wrong in the @BufferImpl.fill()@ logic regarding child relations. In the @data_relation/recursive_relati... Igor Skornyakov

03:48 AM Feature #6444: dataset improvements

I need to iterate over a DATA-SOURCE twice - first with the original QUERY and then with the 'relaxed' one. What is t... Igor Skornyakov

10:59 AM Support #6709: track nested FIND inside a related FOR loop

I'm OK with the @6709-20221216.patch@.
You've tested conversion for both the large customer application and also H... Greg Shah

09:53 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> Please post an updated patch for review.
> We emit the getter or the @FieldReference@ in @conv... Dănuț Filimon

08:30 AM Support #6709: track nested FIND inside a related FOR loop

> The bug is fixed and the conversion was successful. Should I commit the changes?
Please post an updated patch fo... Greg Shah

03:31 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> Danut, please address the bug and make sure to test conversion with that application.
The bug is... Dănuț Filimon

08:06 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

I've made a mistake and used the wrong patch when testing previously. The second @removeRecords@ call didn't execute ... Dănuț Filimon

12/15/2022

02:11 PM Bug #6997: OUTPUT BY-REFERENCE records are not cleared

I returned to this testcase. I altered the original code a bit and I found more evidences in favour of confirming the... Ovidiu Maxiniuc

08:36 AM Feature #6816: improve PreselectQuery.assembleHQL

Another update: after the last post I started working on a Factory design pattern for @SortCriterion@. This patch inc... Radu Apetrii

12/14/2022

02:07 PM Feature #6444: dataset improvements

I've realized that we do not really need an auxiliary temporary table to use recursive CTE instead of iteration (see ... Igor Skornyakov

02:31 AM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
> The FILL operation is QUERY driven. And that happens for each buffer affected (that is, aft... Igor Skornyakov

11:46 AM Bug #6997: OUTPUT BY-REFERENCE records are not cleared

It does not make full sense. The BY-REFERENCE should be bound to the target table temporarily, hiding that table. As ... Ovidiu Maxiniuc

05:46 AM Bug #6997: OUTPUT BY-REFERENCE records are not cleared

In start.p, a temporary table @main@ is created which is assigned 1 element, then that table is used as an output par... Dănuț Filimon

05:44 AM Bug #6997 (New): OUTPUT BY-REFERENCE records are not cleared

Dănuț Filimon

10:01 AM Support #6709: track nested FIND inside a related FOR loop

Looks like the problem was caused by a @FOR FIRST table@ statement. Fixed by checking if the previous sibling of the ... Dănuț Filimon

09:23 AM Feature #6995: possibly support NO-UNDO temporary tables directly in H2

I am aware of the @force-no-undo@ setting which forces all tables into @no-undo@. If this flag is set, no @SavepointM... Alexandru Lungu

06:27 AM Feature #6582: implement multi-table AdaptiveQuery

I've wrapped my head around this topic (@ProgressiveResults@ vs @ScrollingResults@) for a while now, so I can provide... Alexandru Lungu

05:55 AM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Originally, my intention was to delete the records from a table based on the @targetMultiplex@ when the @outputParame... Dănuț Filimon

04:22 AM Feature #6816: improve PreselectQuery.assembleHQL

Update: I noticed that @SortCriterion.toString@ gets called quite often so I had a look in that function to see if so... Radu Apetrii

12/13/2022

06:48 PM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> BTW: it looks that @FILL@ for recursive dataset can be very expensive since it can require ... Ovidiu Maxiniuc

11:56 AM Feature #6995: possibly support NO-UNDO temporary tables directly in H2

I think @Session.log@ is responsible for logging the sub(tx) details needed for savepoints or rollback. This is enab... Constantin Asofiei

11:13 AM Feature #6995: possibly support NO-UNDO temporary tables directly in H2

I guess a simple first step is to profile typical flows in real applications and see if we can associate any bottlene... Eric Faulhaber

11:08 AM Feature #6995: possibly support NO-UNDO temporary tables directly in H2

NO-UNDO temp-tables are used heavily in many 4GL applications. Typically, regular temp-tables are used as well, such ... Eric Faulhaber

10:41 AM Feature #6995 (Closed): possibly support NO-UNDO temporary tables directly in H2

Eric Faulhaber

10:37 AM Feature #6582: implement multi-table AdaptiveQuery

Greg Shah wrote:
> > consider way stronger optimizations directly in the H2 engine: this involves some extended work... Eric Faulhaber

07:47 AM Feature #6928: H2 UPDATE performance improvement

Alexandru Lungu wrote:
> As a side question, I see some changes from for-each structure into index based iteration: ... Constantin Asofiei

06:58 AM Feature #6928: H2 UPDATE performance improvement

I got deep in most of the changes in rev. 8 - they seem reasonable. As a side question, I see some changes from for-e... Alexandru Lungu

12/12/2022

02:29 PM Feature #6444: dataset improvements

Based on my tests the @FILL@ for recursive DATA-RELATION add records which are not in the DATA-SOURCE QUERY only if b... Igor Skornyakov

12:19 PM Feature #6928 (Review): H2 UPDATE performance improvement

The changes are in fwd-h2 project rev 8.
Alexandru: please review. Constantin Asofiei

08:17 AM Support #6709: track nested FIND inside a related FOR loop

I've uncommitted 3821c rev 14436. It causes the following regression in conversion for a large customer application.... Greg Shah

04:29 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> You can just do this:
> ... Dănuț Filimon

12/10/2022

11:52 PM Support #6714: check if there is an optimization opportunity when copying temp-tables based on mu...

Dănuț,
I am sorry for the delay, somehow the notification emails got lost at the bottom of the list and I only saw... Ovidiu Maxiniuc

12/09/2022

10:30 AM Bug #6767: Possible name clashes in DMO classes

Eric: This is still waiting for review. Greg Shah

08:39 AM Feature #6444: dataset improvements

Marian Edu wrote:
> Igor Skornyakov wrote:
> > What puzzles me is the logic for 'relaxing' the @qEmp@ conditions to... Igor Skornyakov

08:19 AM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> What puzzles me is the logic for 'relaxing' the @qEmp@ conditions to find the parent record... Marian Edu

07:45 AM Feature #6444: dataset improvements

Marian Edu wrote:
> Please not that while the recursive relation does load the FK parent in the parent temp-table (s... Igor Skornyakov

05:32 AM Feature #6444: dataset improvements

The @data_relation/recursive_relation_fill.p@ test fails with FWD with @Cannot run GET methods on query qEmp until it... Igor Skornyakov

04:51 AM Feature #6444: dataset improvements

Marian Edu wrote:
> Ovidiu Maxiniuc wrote:
> >
> > > Marian,
> > > Is it correct that the DATA-SOURCE query shou... Igor Skornyakov

04:46 AM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
>
> > Marian,
> > Is it correct that the DATA-SOURCE query should select a subset of a sin... Marian Edu

08:16 AM Feature #6371: implement SAVE CACHE statement

Regression is fixed, current revision number is 14434. Boris Schegolev

12/08/2022

04:40 PM Support #6709: track nested FIND inside a related FOR loop

> > 1. I don't think we should limit the for each processing to only match temp-tables and buffers. All record phrase... Greg Shah

04:32 PM Support #6709: track nested FIND inside a related FOR loop

I think @FieldReference@ is less clean looking and it will be slower. In #4928, we do have some ideas to improve it ... Greg Shah

08:45 AM Support #6709: track nested FIND inside a related FOR loop

... Dănuț Filimon

03:24 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> 1. I don't think we should limit the for each processing to only match temp-tables and buffers. A... Dănuț Filimon

04:33 PM Feature #6371: implement SAVE CACHE statement

Boris, if you have a fix for the conversion regression, please commit it. Eric Faulhaber

01:06 PM Feature #6371: implement SAVE CACHE statement

Revision 6371a/14431 fixes some issues, including the following with the DMO interface assembly:
* a package path ... Eric Faulhaber

03:45 PM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
>More than that, I understand that you aim for a 'fast-copy' mechanism where the rows are cop... Igor Skornyakov

03:27 PM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > > Oh, I see. Indeed, in this case we cannot be sure which versi... Igor Skornyakov

03:02 PM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> > Oh, I see. Indeed, in this case we cannot be sure which version of the CTE syntax to use.... Ovidiu Maxiniuc

05:59 AM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> Oh, I see. Indeed, in this case we cannot be sure which version of the CTE syntax to use.
... Igor Skornyakov

01:31 AM Feature #6444: dataset improvements

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > It appears that DATASET:FILL() is not working at all for RECURS... Igor Skornyakov

05:57 AM Support #6708 (Review): track/report at runtime when AdaptiveQuery shifts into dynamic mode

Committed 3821c/rev.14425. Small fix. The number of invalidated queries displayed in the Overview was wrong when @OUT... Dănuț Filimon

12/07/2022

04:42 PM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> It appears that DATASET:FILL() is not working at all for RECURSIVE DATA-RELATION (the @Data... Ovidiu Maxiniuc

11:56 AM Feature #6444: dataset improvements

Igor Skornyakov wrote:
> I'm thinking about adding support for a RECURSIVE DATA-RELATION by re-writing the DATA-SOUR... Igor Skornyakov

10:45 AM Feature #6444: dataset improvements

The @BufferImpl.fill()@ method is pretty long (400 lines of code) and complicated.
I'm thinking about adding support... Igor Skornyakov

12/05/2022

03:01 PM Feature #6444: dataset improvements

It appears that DATASET:FILL() is not working at all for RECURSIVE DATA-RELATION (the @DataSet.getTopBufferList()@ re... Igor Skornyakov

07:34 AM Support #6709: track nested FIND inside a related FOR loop

Code Review 6709-rpt-20221205.diff
Good, it is getting closer.
1. I don't think we should limit the for each pr... Greg Shah

03:51 AM Support #6709: track nested FIND inside a related FOR loop

Greg Shah wrote:
> 1. Please switch out the use of a @Stack@ for the @ScopedSymbolDictionary@.
I made the @FOR EACH... Dănuț Filimon

06:07 AM Feature #6371: implement SAVE CACHE statement

My suspicion is the logical change in @XmlHelper.setAttribute()@ - I added a check for @value.isEmpty()@ and it proba... Boris Schegolev

12:51 AM Feature #6371: implement SAVE CACHE statement

Eric Faulhaber wrote:
> I found that I was unable to convert test cases using 6371a/14119, with an error during anno... Eric Faulhaber

02:50 AM Feature #6816: improve PreselectQuery.assembleHQL

After I did some investigation upon the @PreselectQuery.assembleHQL@ function, two types of optimization occurred to ... Radu Apetrii

02:13 AM Feature #6928: H2 UPDATE performance improvement

|_. version |_. server run |_. client run |_. indexed (ms) |_. unindexed(ms) |
|h2-1.4.200.jar | 1... Alexandru Lungu