FWD » Core Development » Database

05/30/2020

09:59 PM Feature #4011: database/persistence layer performance improvements

I committed r11447. Changes include:
* unique index validation: primarily a no 1 implementation (see entry 127), but... Ovidiu Maxiniuc

05/29/2020

05:28 PM Feature #4011: database/persistence layer performance improvements

Nice find. I wonder why we weren't hitting this before. I was not doing anything special to do the temp-table DDL out... Eric Faulhaber

04:21 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> * @transactional@ - the H2 manual says that the @CREATE TABLE@ _command commits an open tra... Constantin Asofiei

03:58 PM Feature #4011: database/persistence layer performance improvements

The root cause for the @Reservations/Check-in/Delete guest/Cancel reservation@ abend related to invalid savepoint is ... Constantin Asofiei

01:22 PM Feature #4011: database/persistence layer performance improvements

OK, so it probably is an issue with the timing of the record flush. Eric Faulhaber

01:16 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Constantin Asofiei wrote:
> > OK, I'll switch to the savepoint problem when using the 'good... Constantin Asofiei

01:03 PM Feature #4011: database/persistence layer performance improvements

Constantin Asofiei wrote:
> OK, I'll switch to the savepoint problem when using the 'good data'.
Since you have t... Eric Faulhaber

01:00 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Constantin,
> I am working on getting these messages parsed and the proper 4GL error issue... Constantin Asofiei

12:58 PM Feature #4011: database/persistence layer performance improvements

This is one of the open items. See #4011-127. Eric Faulhaber

12:58 PM Feature #4011: database/persistence layer performance improvements

Constantin,
I am working on getting these messages parsed and the proper 4GL error issued. I estimate they will be c... Ovidiu Maxiniuc

12:55 PM Feature #4011: database/persistence layer performance improvements

OK, that makes sense. My problem is this: in the @validateUniqueCommitted@, we don't know which unique index failed v... Constantin Asofiei

12:53 PM Feature #4011: database/persistence layer performance improvements

This is interesting:... Eric Faulhaber

12:48 PM Feature #4011: database/persistence layer performance improvements

Constantin Asofiei wrote:
> Why is @records@ empty? Shouldn't this be loaded with all the unique records from the ... Eric Faulhaber

12:37 PM Feature #4011: database/persistence layer performance improvements

Eric/Ovidiu, about the 'bad data' case; there is this class @orm.UniqueTracker@:... Constantin Asofiei

12:26 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Constantin Asofiei wrote:
> > There is another issue, and please let me know if either of ... Constantin Asofiei

12:08 PM Feature #4011: database/persistence layer performance improvements

Constantin Asofiei wrote:
> There is another issue, and please let me know if either of you tracks it. If you go to... Ovidiu Maxiniuc

12:04 PM Feature #4011: database/persistence layer performance improvements

There is another issue, and please let me know if either of you tracks it. If you go to @Reservations@, and click @C... Constantin Asofiei

11:54 AM Feature #4011: database/persistence layer performance improvements

Ovidiu, I can't duplicate this anymore - yesterday I saw this issue, but it seems to be fixed in 4011a r11446. Constantin Asofiei

11:51 AM Feature #4011: database/persistence layer performance improvements

From Ovidiu:
> I understand that you managed to get the hotel GUI application up and running (including the database... Constantin Asofiei

08:30 AM Feature #4011: database/persistence layer performance improvements

Committed rev 11446, which fixes a flushing issue found in testing, adds stricter caching logic to the ORM session to... Eric Faulhaber

05/28/2020

07:36 PM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> The leaking of these malformed records seems less likely to be useful than the one ChUI applicati... Eric Faulhaber

05:37 PM Feature #4011: database/persistence layer performance improvements

The leaking of these malformed records seems less likely to be useful than the one ChUI application case where the co... Greg Shah

04:50 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Eric Faulhaber wrote:
> > Constantin Asofiei wrote:
> > > Do you have any unofficial/uncom... Constantin Asofiei

04:43 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Note regarding the check for null mandatory fields (@checkNotNull@/@failNotNull@) already c... Eric Faulhaber

04:33 PM Feature #4011: database/persistence layer performance improvements

Constantin,
I think the @AsmClassLoader@ is missing from the command line (see the @-Djava.system.class.loader@).
L... Ovidiu Maxiniuc

04:31 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Constantin Asofiei wrote:
> > Do you have any unofficial/uncommitted updates to Hotel GUI, ... Eric Faulhaber

04:28 PM Feature #4011: database/persistence layer performance improvements

Constantin Asofiei wrote:
> Do you have any unofficial/uncommitted updates to Hotel GUI, in respect to #4011? I can... Eric Faulhaber

04:21 PM Feature #4011: database/persistence layer performance improvements

Do you have any unofficial/uncommitted updates to Hotel GUI, in respect to #4011? I can't import the data in H2, I g... Constantin Asofiei

03:58 PM Feature #4011: database/persistence layer performance improvements

Note regarding the check for null mandatory fields (@checkNotNull@/@failNotNull@) already committed in 4011.
Ther... Ovidiu Maxiniuc

05/27/2020

03:52 PM Feature #4011: database/persistence layer performance improvements

The revision 11445 contains the fix for conversion of FQL statements when denormalized extent fields are present. Ovidiu Maxiniuc

10:55 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> I've committed rev 11443.
> Ovidiu, please review, as these are largely in areas of the cod... Ovidiu Maxiniuc

04:29 AM Feature #4011: database/persistence layer performance improvements

Something in r11440 caused the following regression in Hotel GUI (virtual desktop). Recreate is to check in a guest f... Eric Faulhaber

05/26/2020

08:12 AM Feature #4011: database/persistence layer performance improvements

I've committed rev 11443.
* temp-table copying fixes;
* @ScrollableResults@ fix to not skip first record;
* @SQL... Eric Faulhaber

08:04 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Ovidiu, please revert the changes in rev 11441 and commit a new revision to bring the branch... Ovidiu Maxiniuc

08:03 AM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> > the pdf is generated but my pdf application is not fast enough to display it. The file is delet... Ovidiu Maxiniuc

07:39 AM Feature #4011: database/persistence layer performance improvements

Ovidiu, please revert the changes in rev 11441 and commit a new revision to bring the branch back to where it was. Do... Eric Faulhaber

06:06 AM Feature #4011: database/persistence layer performance improvements

> the pdf is generated but my pdf application is not fast enough to display it. The file is deleted too fast so a mis... Greg Shah

04:25 AM Feature #4011: database/persistence layer performance improvements

Thanks, but this breaks things the same way as I described. Here's an example stack trace. There are many like this, ... Eric Faulhaber

10:20 PM Feature #4011: database/persistence layer performance improvements

Eric,
I just committed r11441 with some adjustments in @Sql/Query.uniqueResult()@ implementation. I do not know th... Ovidiu Maxiniuc

09:00 PM Feature #4011: database/persistence layer performance improvements

Ovidiu, can you help me understand the idea behind the @Query.uniqueResult@ implementation? The object ignores @rowSt... Eric Faulhaber

08:39 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> it seems to me that the @PropertyMeta@ and @Property@ like the @DmoMeta@ and @RecordMeta@... Eric Faulhaber

05/25/2020

06:19 PM Feature #4011: database/persistence layer performance improvements

Eric,
please find r11440 of 4011. Some additional notes:
* the broken SELECT from note 258 was easy to fix, there ... Ovidiu Maxiniuc

04:51 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Constantin or Greg: is that invalid handle error in #4011-358 something to be concerned with... Constantin Asofiei

04:42 PM Feature #4011: database/persistence layer performance improvements

Constantin or Greg: is that invalid handle error in #4011-358 something to be concerned with? I want to avoid chasing... Eric Faulhaber

07:58 AM Feature #4011: database/persistence layer performance improvements

Eric,
I finished reviewing your latest changes. They are sound. I added the constant and replaced the related empt... Ovidiu Maxiniuc

06:41 AM Feature #4011: database/persistence layer performance improvements

I just realized I forgot to add a @NO_ALIAS@ constant to the FQL converter instead of using the empty string literal ... Eric Faulhaber

06:03 AM Feature #4011: database/persistence layer performance improvements

I've committed rev. 11439, which fixes several issues:
* the previously reported @ClassCastException@, which was c... Eric Faulhaber

05:00 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Consider the @_lock@ table as an example. With the current implementation, the @meta_lock@ t... Igor Skornyakov

04:54 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Eric Faulhaber wrote:
> > Regarding enabling all metadata in all applications, I have no p... Eric Faulhaber

04:46 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Regarding enabling all metadata in all applications, I have no problem doing this eventually... Igor Skornyakov

04:32 PM Feature #3814: more schema metadata

Regarding enabling all metadata in all applications, I have no problem doing this eventually, but we do not yet have ... Eric Faulhaber

05/24/2020

04:32 PM Feature #3814: more schema metadata

Greg Shah wrote:
> Please post the detailed status of this task. I would like to know:
>
> * In regard to the li... Igor Skornyakov

05/23/2020

07:51 AM Feature #4011: database/persistence layer performance improvements

In #4011-355, your references to @no-undo@ are probably meant to be @no-error@?
> This is not the problem to solve... Greg Shah

07:35 AM Feature #3814: more schema metadata

Please post the detailed status of this task. I would like to know:
* In regard to the list of work in #3814-1 an... Greg Shah

05/22/2020

06:29 PM Feature #4011: database/persistence layer performance improvements

I've committed rev. 11438, which fixes both the @UniqueTracker@ empty scopes issue and the missing extent field metho... Eric Faulhaber

06:22 PM Feature #4011: database/persistence layer performance improvements

There are a couple of days since I have started thinking of the following issue related to validation.
As I inform... Ovidiu Maxiniuc

05:33 AM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Do you have enough information in DMO annotations to generate them?
Yes, but I've had to... Eric Faulhaber

10:04 AM Feature #3814: more schema metadata

Constantin Asofiei wrote:
> Igor Skornyakov wrote:
> > This can be done of course if you believe that it makes sens... Igor Skornyakov

09:57 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> This can be done of course if you believe that it makes sense to complicate the conversion/... Constantin Asofiei

09:55 AM Feature #3814: more schema metadata

Constantin Asofiei wrote:
> What about the @_lock@ meta-table? Currently FWD enables this only if @_lock@ is explici... Igor Skornyakov

09:32 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I totally agree with the idea to convert all system tables. After all, in 4GL they exist in... Constantin Asofiei

09:22 AM Feature #3814: more schema metadata

Greg Shah wrote:
> At this point I see no reason to continue with the configuration at all. Why are we not just con... Igor Skornyakov

09:07 AM Feature #3814: more schema metadata

> My idea was for the conversion rules to do some sanity checks: if you specified one or more table which is part of ... Greg Shah

09:05 AM Feature #3814: more schema metadata

We really only have one project where we made the effort to eliminate all metadata usage. That may not ever happen a... Greg Shah

09:03 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> The overhead with the creation/population of the "minimal" set of meta-tables is negligible... Constantin Asofiei

08:58 AM Feature #3814: more schema metadata

Constantin Asofiei wrote:
> I'm not OK with this. Meta-tables should be optional. It adds overhead to other projec... Igor Skornyakov

08:28 AM Feature #3814: more schema metadata

From an email discussion:
Constantin:
> BTW, leaving an empty metadata in p2j.cfg.xml works in FWD, so the meta t... Constantin Asofiei

05/21/2020

06:02 PM Feature #4011: database/persistence layer performance improvements

My recent changes only involved the conversion. I do not have changes related to @DmoClass@. You can safely work in t... Ovidiu Maxiniuc

02:57 PM Feature #4011: database/persistence layer performance improvements

We are not generating implementations for some methods declared in DMO interfaces, such as indexed getters and setter... Eric Faulhaber

05/20/2020

07:00 AM Feature #3814: more schema metadata

Greg Shah wrote:
> I think 4231b is fine.
Committed to 4231b revision 11549. ... Igor Skornyakov

06:35 AM Feature #3814: more schema metadata

I think 4231b is fine. Greg Shah

05/19/2020

04:19 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I've finished the population of the @_File@, @_Field@, @_Index@, and @_Index-Field@ system ... Igor Skornyakov

02:58 PM Feature #4011: database/persistence layer performance improvements

I've just committed my latest changes to 4011 as r11437.
The most important thing here is the fix for the endless qu... Ovidiu Maxiniuc

10:00 AM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> > but in 4011 this is automatically done by DmoMetadataManager. The 4011 uses the new syntax supp... Ovidiu Maxiniuc

05/18/2020

06:53 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> OTOH, the @ProgressiveResults@ have @sessionClosing()@ method which will duplicate the un... Eric Faulhaber

06:27 PM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> > but in 4011 this is automatically done by DmoMetadataManager. The 4011 uses the new syntax supp... Eric Faulhaber

06:04 PM Feature #4011: database/persistence layer performance improvements

> but in 4011 this is automatically done by DmoMetadataManager. The 4011 uses the new syntax supported by H2 (and PSQ... Greg Shah

05:49 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> [..] when a query is associated with a browse, apparently we need some special treatment, be... Ovidiu Maxiniuc

04:00 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Separately, I committed rev 11436, which completes the fix to the server startup performance... Ovidiu Maxiniuc

05:02 AM Feature #4011: database/persistence layer performance improvements

The @UniqueTracker@ empty scopes issue seems to be caused by the fact that we aren't registering the tracker for @Fin... Eric Faulhaber

05:44 PM Feature #3814: more schema metadata

I've finished the population of the @_File@, @_Field@, @_Index@, and @_Index-Field@ system tables base on DMO reflect... Igor Skornyakov

05/16/2020

01:53 AM Feature #4011: database/persistence layer performance improvements

Running a single ETF test results in some errors...... Eric Faulhaber

05/15/2020

07:12 PM Feature #4011: database/persistence layer performance improvements

The long delay during ETF server startup while processing index metadata appears to be due to the fact that we swappe... Eric Faulhaber

06:57 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Eric Faulhaber wrote:
> > * Using the mouse wheel to scroll down in the "Available Rooms" ... Eric Faulhaber

06:32 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> * Using the mouse wheel to scroll down in the "Available Rooms" tab browse results in an end... Ovidiu Maxiniuc

05:07 PM Feature #3814: more schema metadata

The @_File@ table population for a large customer application (both system and user tables) takes ~100 ms in my envir... Igor Skornyakov

12:55 PM Feature #3814: more schema metadata

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > I'm a little bit confused with the discussion in #3145. At the ... Igor Skornyakov

12:07 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I'm a little bit confused with the discussion in #3145. At the bottom line - is it OK to se... Ovidiu Maxiniuc

06:18 AM Feature #3814: more schema metadata

I'm a little bit confused with the discussion in #3145. At the bottom line - is it OK to set the value of the @_Templ... Igor Skornyakov

02:55 AM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
>
> > [...]
> > I've noticed that the current data for the @_User... Igor Skornyakov

10:04 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> [...]
> I've noticed that the current data for the @_User@ table in the @xxx.meta.xml@ f... Eric Faulhaber

05/14/2020

05:45 PM Feature #3814: more schema metadata

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > [...] and @_Template@ is always zero (not a negative number).
... Igor Skornyakov

05:15 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> [...] and @_Template@ is always zero (not a negative number).
I do not remember exactly, b... Ovidiu Maxiniuc

05:04 PM Feature #3814: more schema metadata

Populating @_File@ system table based *only* on database metadata (provided by @Connection.getMetaData()@) looks to b... Igor Skornyakov

03:55 PM Feature #4011: database/persistence layer performance improvements

Hotel GUI r220 issues with 4011a:
* Adding a guest to the "Check-In" dialog replaces the existing guest instead of... Eric Faulhaber

03:26 PM Feature #4011: database/persistence layer performance improvements

I noticed something else related to assigns, which will probably not be handled in this task: the way the ASSIGN trig... Ovidiu Maxiniuc

03:12 PM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> We've never seen this behavior before. I think it can only be seen in a @NO-ERROR@ case because ... Ovidiu Maxiniuc

07:24 AM Feature #4011: database/persistence layer performance improvements

> Next, if we enable the two commented lines the program will print 0 ? because the values of k and f11 will be rolle... Greg Shah

09:44 PM Feature #4011: database/persistence layer performance improvements

Indeed, to observe the null-mandatory-nonValidated field case the field initial value must be declared @?@, like:
<p... Ovidiu Maxiniuc

05/13/2020

07:08 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Related to not-null/mandatory validation. There are some issues with current implementation... Eric Faulhaber

04:28 PM Feature #4011: database/persistence layer performance improvements

Related to not-null/mandatory validation. There are some issues with current implementation.
* 1st: there is a two... Ovidiu Maxiniuc

03:12 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> I will do a test on my HDD to give as a base for comparing future work. How many threads ... Eric Faulhaber

06:19 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Revision 11434 implements bulk insert (i.e., broader SQL batching) and a few other optimizat... Ovidiu Maxiniuc

04:05 AM Feature #4011: database/persistence layer performance improvements

Revision 11434 implements bulk insert (i.e., broader SQL batching) and a few other optimizations for the database imp... Eric Faulhaber

09:33 AM Bug #3471: incremental conversion is regressed

Some other parts which are required for incremental conversion:
* the @.wsdl@ generation for exported SOAP services... Constantin Asofiei

05/12/2020

08:07 AM Feature #3814: more schema metadata

Greg Shah wrote:
> > This can result in a performance drag on the system, in that the maintenance of the state of th... Igor Skornyakov

08:03 AM Feature #3814: more schema metadata

> do we already have an issue for this?
Not that I recall.
> I do not want to include all system tables support... Greg Shah

05/11/2020

07:28 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I do not see any potential conflicts between what I see in 4011a and my changes in 4231b. S... Eric Faulhaber

06:37 PM Feature #3814: more schema metadata

I've made @MetadataManager.initialize()@ implementation as close to one in 4011a as possible. Both implementations a... Igor Skornyakov

06:52 AM Feature #3814: more schema metadata

I do not see any potential conflicts between what I see in 4011a and my changes in 4231b. Some changes will be requir... Igor Skornyakov

02:01 PM Feature #4011: database/persistence layer performance improvements

Thanks for the batch size fix, I hadn't noticed that. Yes, the import organizes the records into batches, but we don'... Eric Faulhaber

12:13 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> I've been running database import and I noticed something surprising in the import worker co... Ovidiu Maxiniuc

05:42 AM Feature #4011: database/persistence layer performance improvements

I'm trying to run my standalone test with H2 database. It runs fine with 4231b, but with 4011a the server fails at st... Igor Skornyakov

04:32 AM Feature #4011: database/persistence layer performance improvements

I've been running database import and I noticed something surprising in the import worker code:... Eric Faulhaber

05/10/2020

03:31 AM Feature #4011: database/persistence layer performance improvements

To clarify, that last line refers to deleting DMOs one at a time using @Persistence.delete@, as is done in the persis... Eric Faulhaber

03:27 AM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Eric Faulhaber wrote:
> > Ovidiu Maxiniuc wrote:
> > > [...]
> > > I think this region o... Eric Faulhaber

05/09/2020

09:41 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Ovidiu Maxiniuc wrote:
> > [...]
> > I think this region of the persistence was not altere... Ovidiu Maxiniuc

06:38 AM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> [...]
> I think this region of the persistence was not altered. I wonder how this was work... Eric Faulhaber

05/08/2020

07:55 PM Feature #4011: database/persistence layer performance improvements

Please find the TempTable DataSource management patch as r11432. I tested the web client and multiple clients connect... Ovidiu Maxiniuc

07:09 PM Feature #4011: database/persistence layer performance improvements

I was testing the new datasource/connection when I encounter the following issue: once a dynamic table (probably not ... Ovidiu Maxiniuc

06:08 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Ovidiu, w.r.t. the ETF import taking almost 17 hours, is that normal for that machine? It se... Eric Faulhaber

05:59 PM Feature #4011: database/persistence layer performance improvements

Agreed. That's what I was trying to suggest, but you summarized it better.
Also, looking back at how it ended up, ... Eric Faulhaber

05:54 PM Feature #4011: database/persistence layer performance improvements

I think the framework might work with this idea: we just move the context from the @TempTableDataSourceProvider@, to ... Ovidiu Maxiniuc

05:46 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> The problem is in @JdbcDataSource@. For temp-tables, it is built from the @DataSourceImpl@ ... Eric Faulhaber

05:04 PM Feature #4011: database/persistence layer performance improvements

The problem is in @JdbcDataSource@. For temp-tables, it is built from the @DataSourceImpl@ returned by the @TempTable... Ovidiu Maxiniuc

04:25 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Ovidiu Maxiniuc wrote:
>
> > I investigated the connections and they seem to share the sa... Eric Faulhaber

12:37 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> I investigated the connections and they seem to share the same connection. I will investi... Eric Faulhaber

06:18 AM Feature #4011: database/persistence layer performance improvements

I tried connecting with two clients at once: the Swing client first (only connect, no other clicks) and then the embe... Ovidiu Maxiniuc

04:32 AM Feature #4011: database/persistence layer performance improvements

Ovidiu, w.r.t. the ETF import taking almost 17 hours, is that normal for that machine? It seems really long to me. Wi... Eric Faulhaber

04:08 AM Feature #4011: database/persistence layer performance improvements

Thanks for the fixes.
I rebuilt and converted Hotel GUI from scratch. The import worked, but unfortunately, I stil... Eric Faulhaber

09:59 PM Feature #4011: database/persistence layer performance improvements

New update, r11431.
There was an issue with _temp records fetched by loader. The hydration wasn't setting up the @_m... Ovidiu Maxiniuc

03:20 AM Feature #3757: add metadata support

Ovidiu Maxiniuc wrote:
> Igor,
> I do not remember exactly why the @ _database-feature@ was removed. It seems to me... Igor Skornyakov

05/07/2020

07:24 PM Feature #4011: database/persistence layer performance improvements

Eric,
Please grab -r11429- LE:r11430. I have a separate directory for commits and I forgot to compare and merge in t... Ovidiu Maxiniuc

06:38 PM Feature #4011: database/persistence layer performance improvements

Ovidiu, I'm getting this during the import step:... Eric Faulhaber

04:40 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> The standalone clients now can sequentially connect to server and the table/index cleanup... Eric Faulhaber

04:15 PM Feature #4011: database/persistence layer performance improvements

I have just committed r11429. Changes include:
* moved DDL generation from @DmoMetadataManager@ to @TempTableHelper@... Ovidiu Maxiniuc

04:27 AM Feature #4011: database/persistence layer performance improvements

I committed rev 11427, which adds the placeholder suffix at the end of temp-table index names. Note that this short t... Eric Faulhaber

04:19 AM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> This must be done as some things changed: @id@/@recid@ as PK, @identifier@/@id@, datetime... Eric Faulhaber

10:43 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Please prioritize the temp-table name suffix fix. I'd like to see this go in as soon as poss... Eric Faulhaber

08:38 PM Feature #4011: database/persistence layer performance improvements

Please prioritize the temp-table name suffix fix. I'd like to see this go in as soon as possible, even if nothing els... Eric Faulhaber

07:17 PM Feature #3757: add metadata support

Igor,
I do not remember exactly why the @ _database-feature@ was removed. It seems to me that, since work on @_ActLo... Ovidiu Maxiniuc

06:24 PM Feature #3757: add metadata support

Greg Shah wrote:
> I had not planned for you to make everything "live" yet. For this current task, it is OK to exte... Igor Skornyakov

06:16 PM Feature #3757: add metadata support

I had not planned for you to make everything "live" yet. For this current task, it is OK to extend the current stati... Greg Shah

04:59 PM Feature #3757: add metadata support

Ovidiu Maxiniuc wrote:
> The @_database-feature@ has the runtime support in @MetadataManager.populateDatabaseFeature... Igor Skornyakov

05/06/2020

07:52 PM Feature #4011: database/persistence layer performance improvements

The following is a short discussion from email, with my last replies:
Eric Faulhaber wrote:
> On 5/6/20 4:33 PM, Ov... Ovidiu Maxiniuc

05:56 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> ... the @TempTableHelper@ also keeps a static @cache@ in a form of a @HashMap@ which is nev... Eric Faulhaber

03:51 PM Feature #4011: database/persistence layer performance improvements

It doesn't matter much to me where we keep the cache... wherever you think it makes the most sense is fine.
As to... Eric Faulhaber

01:59 PM Feature #4011: database/persistence layer performance improvements

Eric,
Your statements are correct. I added the code exactly as you described them an continued some investigations. ... Ovidiu Maxiniuc

08:25 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Constantin, I am hitting a resource cleanup problem with a persistent procedure in Hotel GUI... Constantin Asofiei

07:33 AM Feature #4011: database/persistence layer performance improvements

Ovidiu, I've committed rev 11427, which partially fixes persistence context cleanup. However, I found a problem with ... Eric Faulhaber

04:22 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Constantin, I am hitting a resource cleanup problem with a persistent procedure in Hotel GUI... Constantin Asofiei

04:12 AM Feature #4011: database/persistence layer performance improvements

Constantin, I am hitting a resource cleanup problem with a persistent procedure in Hotel GUI. The procedure is the co... Eric Faulhaber

01:09 PM Feature #3814: more schema metadata

It's #4155.
I was looking for #3757 several days ago and for some reasons I could not reach it. Thanks for finding i... Ovidiu Maxiniuc

01:00 PM Feature #3814: more schema metadata

Ovidiu Maxiniuc wrote:
> I will answer this:
> there are a few versions of @standard.df@ in use. Depending on the v... Roger Borrello

12:46 PM Feature #3814: more schema metadata

Roger Borrello wrote:
> Igor,
> On a project, we had to include @_connect@ in the @metadata@. At that point, we rec... Ovidiu Maxiniuc

12:38 PM Feature #3814: more schema metadata

Igor,
On a project, we had to include @_connect@ in the @metadata@. At that point, we received @java.lang.ClassNot... Roger Borrello

05/05/2020

08:12 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Constantin, can you please point me at some test cases which use buffers in persistent proce... Constantin Asofiei

05:45 AM Feature #4011: database/persistence layer performance improvements

I committed rev 11426, which attempts to fix the session closing issue. However, there is still a problem, which I am... Eric Faulhaber

05:42 AM Feature #4011: database/persistence layer performance improvements

Tried an ETF run. Server startup reported these errors:... Eric Faulhaber

03:44 AM Bug #3471: incremental conversion is regressed

Eric, I've noticed something while looking at #4633. The @SchemaWorker.joins@ does not survive for incremental conve... Constantin Asofiei

05/01/2020

04:11 PM Feature #4011: database/persistence layer performance improvements

Constantin, can you please point me at some test cases which use buffers in persistent procedures? Eric Faulhaber

01:40 PM Feature #4011: database/persistence layer performance improvements

Committed 11423. Changes:
* fixed regression in _meta tables;
* skip re-hydration of records if a non STALE copy is... Ovidiu Maxiniuc

07:31 AM Feature #4011: database/persistence layer performance improvements

I do not know the exact H2 implementation, but it seems to me logical that once the table is LOCAL, its indexes will ... Ovidiu Maxiniuc

06:44 AM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> At this moment we have some @Session@ -related events: SESSION_CLEANUP, SESSION_CLOSING_N... Eric Faulhaber

06:11 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> We have not changed the approach to "static" metadata tables in this branch, and we're makin... Igor Skornyakov

06:06 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> We have not changed the approach to "static" metadata tables in this branch, and we're makin... Ovidiu Maxiniuc

06:03 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Ovidiu, AFAICT, we do not do any explicit closing of resources associated with converted, le... Ovidiu Maxiniuc

06:01 AM Feature #4011: database/persistence layer performance improvements

We have not changed the approach to "static" metadata tables in this branch, and we're making only minimal changes to... Eric Faulhaber

05:52 AM Feature #4011: database/persistence layer performance improvements

Gentlemen,
I'm working on metatables support right now. Maybe it makes sense for me to use your branch?
Thank you. Igor Skornyakov

05:48 AM Feature #4011: database/persistence layer performance improvements

No worries. The patch I used to get past it is attached. If you're ok with it (I didn't test the case where the @Meta... Eric Faulhaber

05:43 AM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> The discussion of modifying the approach on the metadata DMOs notwithstanding, the current a... Ovidiu Maxiniuc

04:30 AM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> Last night I committed r11422.
> This revision re-enables @LockTableUpdater@, @ConnectTab... Eric Faulhaber

04/30/2020

06:56 PM Feature #4011: database/persistence layer performance improvements

Constantin Asofiei wrote:
> For persistent procedures, a buffer will live as long as that program lives.
Hm, ri... Eric Faulhaber

06:24 PM Feature #4011: database/persistence layer performance improvements

Eric Faulhaber wrote:
> Specifically, I provisionally have decided on the approach where we tie the database session... Constantin Asofiei

05:55 PM Feature #4011: database/persistence layer performance improvements

Constantin, I need some guidance on my previous entry.
Specifically, I provisionally have decided on the approach ... Eric Faulhaber

03:50 PM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> If @Persistence.closeSession(false)@ is called only from @Persistence$Context.cleanup()@. T... Eric Faulhaber

10:08 AM Feature #4011: database/persistence layer performance improvements

I agree that refactoring and using modules are both very good things. And we are definitely wanting to do some of th... Greg Shah

09:34 AM Feature #4011: database/persistence layer performance improvements

Ovidiu Maxiniuc wrote:
> When we upgrade to Java9 we could make use of the new modules and compile them independentl... Igor Skornyakov

09:24 AM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> In #4155 we are working to eliminate use of the @.df@. And converting as part of the build proce... Ovidiu Maxiniuc

09:10 AM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> In #4155 we are working to eliminate use of the @.df@. And converting as part of the build proce... Igor Skornyakov

08:58 AM Feature #4011: database/persistence layer performance improvements

In #4155 we are working to eliminate use of the @.df@. And converting as part of the build process leads to a bootst... Greg Shah

08:50 AM Feature #4011: database/persistence layer performance improvements

Greg Shah wrote:
> > taking in consideration that we do not have the class at the moment FWD is compiled
>
> In m... Ovidiu Maxiniuc

08:44 AM Feature #4011: database/persistence layer performance improvements

> taking in consideration that we do not have the class at the moment FWD is compiled
In my opinion, it makes no s... Greg Shah