FWD » Core Development » Database

04/15/2021

03:16 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> If you think (a) you are not close to finished with the changes; or (b) they are especially ... Igor Skornyakov

03:01 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> BTW: I already have a lot of uncommitted changes for the 3821c. Maybe it makes sense to c... Eric Faulhaber

07:23 AM Feature #3814: more schema metadata

The problem with @InterruptedException@ is resolved. See attached .diff file.
The reason was the following. @UserT... Igor Skornyakov

03:45 AM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Please see #3896. We need to make much more substantial changes to the implementation suppor... Igor Skornyakov

08:08 AM Bug #3896: improve performance of the _lock metadata implementation

Greg Shah wrote:
> This approach should be generalized. I think there are many other tables that would be better im... Igor Skornyakov

07:49 AM Bug #3896: improve performance of the _lock metadata implementation

This approach should be generalized. I think there are many other tables that would be better implemented this way. Greg Shah

07:38 AM Bug #3896: improve performance of the _lock metadata implementation

Greg Shah wrote:
> I don't think this is a radical idea. It is exactly what I've been suggesting for years. See #3... Igor Skornyakov

07:31 AM Bug #3896: improve performance of the _lock metadata implementation

I don't think this is a radical idea. It is exactly what I've been suggesting for years. See #3896-3. Greg Shah

03:39 AM Bug #3896: improve performance of the _lock metadata implementation

Maybe it makes sense to implement the @_Lock@ support in the same way it is done for the @_UserTableStat@? I mean ke... Igor Skornyakov

04/14/2021

06:03 PM Feature #3814: more schema metadata

Please see #3896. We need to make much more substantial changes to the implementation supporting @_Lock@ metadata, th... Eric Faulhaber

05:16 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> One more difference between the two tests.
> With @stat.p@ I see a normal decorated window... Igor Skornyakov

04:45 PM Feature #3814: more schema metadata

One more difference between the two tests.
With @stat.p@ I see a normal decorated window from the very beginning. Wi... Igor Skornyakov

04:21 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I've noticed a strange thing. As I wrote before, I have two tests @stat.p@ and @stat1.p@. I... Igor Skornyakov

04:13 PM Feature #3814: more schema metadata

I've noticed a strange thing. As I wrote before, I have two tests @stat.p@ and @stat1.p@. I convert both of them.
If... Igor Skornyakov

02:59 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> To the degree a metadata table is writable and is updated from converted application logic, ... Igor Skornyakov

02:33 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> More about the issue with @InterruptedExcception@.
> I have two tests in @testcases/meta@:... Eric Faulhaber

02:04 PM Feature #3814: more schema metadata

More about the issue with @InterruptedExcception@.
I have two tests in @testcases/meta@: @stat.p@ and @stat1.p@. I s... Igor Skornyakov

07:50 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Sometimes I get the following exception:
> [...]
> I've never seen that before. Please no... Igor Skornyakov

07:18 AM Feature #3814: more schema metadata

Sometimes I get the following exception:... Igor Skornyakov

04/13/2021

04:48 PM Feature #3814: more schema metadata

Ovidiu Maxiniuc wrote:
> BTW, to see what exactly there is stored in a table at a given moment you can put a breakpo... Igor Skornyakov

04:33 PM Feature #3814: more schema metadata

BTW, to see what exactly there is stored in a table at a given moment you can put a breakpoint in your IDE and execut... Ovidiu Maxiniuc

04:33 PM Feature #3814: more schema metadata

Ovidiu Maxiniuc wrote:
> I did a quick conversion and run your testcase with @show_sql=true@ for persistent database... Igor Skornyakov

04:29 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Without debugging, I do not know. What does the program do in FWD? Are you seeing the error ... Igor Skornyakov

04:27 PM Feature #3814: more schema metadata

I did a quick conversion and run your testcase with @show_sql=true@ for persistent database in my @directory.xml@.
H... Ovidiu Maxiniuc

04:25 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I'm running the following test program:
> [...]
>
> With 4GL I see 1 creates, 5 reads, ... Igor Skornyakov

04:13 PM Feature #3814: more schema metadata

Without debugging, I do not know. What does the program do in FWD? Are you seeing the error message after the @BUFFER... Eric Faulhaber

03:30 PM Feature #3814: more schema metadata

I'm running the following test program:... Igor Skornyakov

01:32 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> See @UserTableStatUpdater.updateRecord@ at line 225:
>
> [...]
>
> This file needs a f... Igor Skornyakov

01:28 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I've tested my (not yet committed) changes with a large customer app.
> I've noticed a pro... Eric Faulhaber

01:22 PM Feature #3814: more schema metadata

I've tested my (not yet committed) changes with a large customer app.
I've noticed a problem with @_UserTableStat@ o... Igor Skornyakov

04/12/2021

09:12 AM Feature #4176: make certain metadata tables aware of live runtime state

How much of this task is already complete from the work done in #3814? Greg Shah

06:40 AM Feature #3813 (Closed): misc DB features part deux

Greg Shah

05:40 AM Feature #3813: misc DB features part deux

The @BUFFER-FIELD:DECIMALS@ support looks correct with 3821c/12264. Igor Skornyakov

04/09/2021

12:05 PM Feature #3814: more schema metadata

I am ok with disabling the caching for metadata tables, for the reasons already discussed above. If we find some perf... Eric Faulhaber

07:20 AM Feature #3814: more schema metadata

Added additional configuration options for the 'remote-meta' mode to simplify the development/debugging of the metada... Igor Skornyakov

07:15 AM Feature #3814: more schema metadata

Greg Shah wrote:
> Eric will respond more fully to your concerns. But one thing I can point out here is that the co... Igor Skornyakov

07:00 AM Feature #3814: more schema metadata

Eric will respond more fully to your concerns. But one thing I can point out here is that the converted 4GL code is ... Greg Shah

06:15 AM Feature #3814: more schema metadata

In addition to a previous note.
In contrast with the situation with customer tables (both primary and temporary), ... Igor Skornyakov

04/08/2021

05:18 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
>
> > I do not think that it makes much sense to use cache with me... Igor Skornyakov

04:22 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I do not think that it makes much sense to use cache with metatables, at least for reques... Eric Faulhaber

01:19 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Yes, we probably don't have invalidation logic for places where we update/insert/delete meta... Igor Skornyakov

12:42 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I've disabled the @RandomAccessQuery.ffCache@ usage for meta tables and now @MyConnction@... Eric Faulhaber

10:59 AM Feature #3814: more schema metadata

The @_MyConnection@ support is fixed (see however #3814-315).
Please review the changes vs. 3821c/12259.
Thank yo... Igor Skornyakov

07:54 AM Feature #3814: more schema metadata

Committed trigger function for @meta_myconnection@ view to 3821c/12258. It doesn't have any effect until other change... Igor Skornyakov

06:33 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> The cache logic in the @RandomAccessQuery.execute()@ seems to be not 100% correct. For @_My... Igor Skornyakov

06:17 AM Feature #3814: more schema metadata

The cache logic in the @RandomAccessQuery.execute()@ seems to be not 100% correct. For @_MyConnection@ it finds the @... Igor Skornyakov

04:49 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> As I wrote, H2 seems not to understand a UDF definition that I've provided. This means that... Igor Skornyakov

04/07/2021

05:22 PM Feature #3814: more schema metadata

> I think the UDF definition is correct, but please remove the @session_id@ column and the use of the UDF in the @cre... Igor Skornyakov

05:00 PM Feature #3814: more schema metadata

I think the UDF definition is correct, but please remove the @session_id@ column and the use of the UDF in the @creat... Eric Faulhaber

03:34 PM Feature #3814: more schema metadata

I've created a function... Igor Skornyakov

03:20 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> The context-local @_MyConn-Id@ will have to be the same value across connected, primary data... Igor Skornyakov

02:50 PM Feature #3814: more schema metadata

There is no physically separate "server-side" in this case. The @_meta@ database is an embedded, in-memory database, ... Eric Faulhaber

02:08 PM Feature #3814: more schema metadata

Unfortunately, the trick with view/trigger doesn't for the same reason which complicated the approach bases on the te... Igor Skornyakov

01:43 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Sorry, I'm not following... do you mean the FWD runtime code, or the converted business logi... Igor Skornyakov

01:40 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> It seems like the view could be filtered on @_MyConn-Id@, rather than introducing another un... Igor Skornyakov

01:30 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> The reason is to avoid *any* changes in the Java code. All we need is a new trigger class... Eric Faulhaber

01:24 PM Feature #3814: more schema metadata

Another question...isn't... Eric Faulhaber

01:21 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> I'm not sure why the trigger would be needed in this case. The @_Myconnection@ VST is read-o... Igor Skornyakov

01:15 PM Feature #3814: more schema metadata

I'm not sure why the trigger would be needed in this case. The @_Myconnection@ VST is read-only for general use. The ... Eric Faulhaber

11:51 AM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Please illustrate this idea further. I have not used INSTEAD OF triggers, but I understand s... Igor Skornyakov

11:46 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> If we create a corresponding INSTEAD OF triggers for the view, then only changes in the D... Eric Faulhaber

10:58 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Igor Skornyakov wrote:
> > I suggested adding an additional condition to all SELECTs from ... Igor Skornyakov

10:42 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I suggested adding an additional condition to all SELECTs from @_MyConnection@ so that only... Igor Skornyakov

09:56 AM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> I am not suggesting any change to your planned support for the @_UserTableStat@ table. But f... Igor Skornyakov

09:44 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> I understand that using a local temporary table for the @_UserTableStat@ requires a separat... Eric Faulhaber

05:47 AM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Eric Faulhaber wrote:
> > Sorry, I should have used the term temporary table, so as not to... Igor Skornyakov

04/02/2021

02:21 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Sorry, I should have used the term temporary table, so as not to confuse with temp-table, wh... Igor Skornyakov

02:20 PM Feature #3814: more schema metadata

Sorry, I should have used the term temporary table, so as not to confuse with temp-table, which is the 4GL term.
T... Eric Faulhaber

01:58 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> It seems to me this can be driven primarily from the @ConnectionManager@. Where are the ma... Igor Skornyakov

01:49 PM Feature #3814: more schema metadata

I've finished with persisting of the @_UserTableStat@ data and update of the @_UserTableStat._UserTableStat-read@ cou... Igor Skornyakov

01:47 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Your suggestion with temp tables looks cleaner than mine but seems to be more complicated... Eric Faulhaber

12:56 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> I do think we need to work on correcting the @_myconnection@ implementation sooner rather th... Igor Skornyakov

12:25 PM Feature #3814: more schema metadata

I do think we need to work on correcting the @_myconnection@ implementation sooner rather than later.
The implemen... Eric Faulhaber

03:35 AM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Greg, is right, we did not implement the @_myconnection@ table correctly. For each context/s... Igor Skornyakov

12:48 AM Feature #3814: more schema metadata

Greg, is right, we did not implement the @_myconnection@ table correctly. For each context/session, there should only... Eric Faulhaber

04/01/2021

05:34 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> The first and second inserted records from #3814-282 are a result of the @ConnectManager.re... Igor Skornyakov

05:02 PM Feature #3814: more schema metadata

The first and second inserted records from #3814-282 are a result of the @ConnectManager.register()@ call at the serv... Igor Skornyakov

04:51 PM Feature #3814: more schema metadata

What I see in the H2 trace log is that we add two records to this table and never delete them.
inserts:... Igor Skornyakov

04:34 PM Feature #3814: more schema metadata

I recall that @_myconnection@ was added during #3293 and that there is some kind of "singleton" problem with it. But... Greg Shah

04:26 PM Feature #3814: more schema metadata

It seems that we have problems with @_MyConnection@ VST support.
With 4GL the statement @find first _MyConnection no... Igor Skornyakov

05:34 AM Feature #3814: more schema metadata

Greg Shah wrote:
> This would never be seen in a production application which is compiled in advance. That means th... Igor Skornyakov

05:33 AM Feature #3814: more schema metadata

This would never be seen in a production application which is compiled in advance. That means that the compilation i... Greg Shah

04:15 AM Feature #3814: more schema metadata

I've noticed an interesting thing about @_usertablestat@.
Consider the following program:... Igor Skornyakov

03/31/2021

05:35 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> At some point before @Class.forName@ is called, @DmoMeta.registerDmo@ must have been called ... Igor Skornyakov

05:34 PM Feature #3814: more schema metadata

At some point before @Class.forName@ is called, @DmoMeta.registerDmo@ must have been called for the DMO implementatio... Eric Faulhaber

04:09 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Eric Faulhaber wrote:
> > Something doesn't seem right. Why @MetaLock@?
>
> I do not kn... Igor Skornyakov

04:02 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Something doesn't seem right. Why @MetaLock@?
I do not know. The constructor is just
<pr... Igor Skornyakov

03:54 PM Feature #3814: more schema metadata

> ... Eric Faulhaber

03:13 PM Feature #3814: more schema metadata

There was a bug in the @ConnetionManager@ which prevented the creation of the @UserTableStatUpdater@ instances. After... Igor Skornyakov

03:00 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Thinking more on this, if there is commonality in the changes for @RandomAccessQuery@ and @P... Igor Skornyakov

02:59 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Eric Faulhaber wrote:
> > The fact that there is a "meta" database is only a FWD implement... Igor Skornyakov

02:35 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> Eric Faulhaber wrote:
> > @PreselectQuery@ (the superclass of @AdaptiveQuery@) is the bett... Eric Faulhaber

02:29 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> The fact that there is a "meta" database is only a FWD implementation detail. The metadata a... Igor Skornyakov

02:27 PM Feature #3814: more schema metadata

The fact that there is a "meta" database is only a FWD implementation detail. The metadata always should be about a p... Eric Faulhaber

02:14 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> You have access to the @RecordBuffer@ object(s) for every query. @RecordBuffer.getDatabase@ ... Igor Skornyakov

02:11 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> 1. We have one instance of the @UserTableStatUpdater@ per primary database in the @Connec... Eric Faulhaber

02:10 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> @PreselectQuery@ (the superclass of @AdaptiveQuery@) is the better place for this than @Adap... Igor Skornyakov

02:01 PM Feature #3814: more schema metadata

Igor Skornyakov wrote:
> 2. I understand that there are at least two different types of queries where I should add... Eric Faulhaber

01:15 PM Feature #3814: more schema metadata

I'm looking for the right place for the @UserTableStatUpdater.persist()@ call which flashes collected statistics from... Igor Skornyakov

08:55 AM Feature #3814: more schema metadata

Greg Shah wrote:
> Please post it now, so that it is early enough to let Eric provide feedback. It may save time.
... Igor Skornyakov

08:52 AM Feature #3814: more schema metadata

Please post it now, so that it is early enough to let Eric provide feedback. It may save time. Greg Shah

08:32 AM Feature #3814: more schema metadata

Greg Shah wrote:
> Please post an example of the changes you are making (as a diff). I want Eric to have a chance t... Igor Skornyakov

07:28 AM Feature #3814: more schema metadata

Please post an example of the changes you are making (as a diff). I want Eric to have a chance to review before you ... Greg Shah

09:40 AM Bug #5224: ROWID is a variable length binary value (4GL quirk)

At this time we have no reason to expect that real applications will depend upon these differences. In FWD code can ... Greg Shah

09:23 AM Bug #5224: ROWID is a variable length binary value (4GL quirk)

Parsing, formatting and comparing of @ROWID@ values works differently in FWD:
In OE:
# @ROWID@ is a string of b... Vladimir Tsichevski

08:51 AM Bug #5224 (New): ROWID is a variable length binary value (4GL quirk)

Vladimir Tsichevski

03/30/2021

01:37 PM Feature #4011 (Closed): database/persistence layer performance improvements

Eric Faulhaber

03/29/2021

02:46 PM Feature #5219: implement native full-text search for word index and CONTAINS support

Eric Faulhaber wrote:
> We also need to review edit 3821c/12004, which changed the index selection portion of query ... Igor Skornyakov

12:04 AM Feature #5219: implement native full-text search for word index and CONTAINS support

With #1587, we have provided first class support for the word index and CONTAINS operator, including parsing/indexing... Eric Faulhaber

11:54 PM Feature #5219 (New): implement native full-text search for word index and CONTAINS support

Eric Faulhaber

12:49 PM Feature #3814: more schema metadata

Greg Shah wrote:
> Yes, now is the time to complete this task.
Thank you. Working in this. Igor Skornyakov

12:28 PM Feature #3814: more schema metadata

Yes, now is the time to complete this task. Greg Shah

12:05 PM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Sorry, please ignore my last post. I was going from memory, but upon rereading this issue's ... Igor Skornyakov

11:56 AM Feature #3814: more schema metadata

Sorry, please ignore my last post. I was going from memory, but upon rereading this issue's history, I see we already... Eric Faulhaber

03:26 AM Feature #3814: more schema metadata

Eric Faulhaber wrote:
> Assuming @_usertablestat@ is frequently updated, we do not want to take an approach like we ... Igor Skornyakov

12:30 AM Feature #3814: more schema metadata

Assuming @_usertablestat@ is frequently updated, we do not want to take an approach like we have now with the @_lock@... Eric Faulhaber

06:23 AM Feature #1587 (Closed): implement full support for word indexes

Greg Shah

12:05 AM Feature #1587 (Test): implement full support for word indexes

Eric Faulhaber

12:10 AM Feature #5220: extend word index support to SQL Server

#1587 implemented word index support and better implementation of CONTAINS-based queries for PostgreSQL and H2 databa... Eric Faulhaber

12:06 AM Feature #5220 (Closed): extend word index support to SQL Server

Eric Faulhaber

03/28/2021

11:33 AM Feature #1587: implement full support for word indexes

For the performance testing (@words/words-perf.p@) I use the following data:
The test table definition is:... Igor Skornyakov

04:56 AM Feature #1587: implement full support for word indexes

Added documentation for the new @CONTAINS@ support to the 'Internals' section of Wiki. See [[p2j:Word Index and CONTA... Igor Skornyakov

03/26/2021

05:04 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> For the "how it works" documentation you just posted, you could add a page to the @Internals... Igor Skornyakov

05:02 PM Feature #1587: implement full support for word indexes

Igor Skornyakov wrote:
> BTW: maybe it makes sense to add documentation to the Wiki? If so, then to what section?
... Eric Faulhaber

04:36 PM Feature #1587: implement full support for word indexes

h1. Word tables and related database objects
For the table *{T}* and field *{f}* with word index on it the word ta... Igor Skornyakov

02:13 PM Feature #1587: implement full support for word indexes

Merged wors tables tests' data to the @fwd@ test database. Performed cleanup of the tests.
Committed to @xfer.gold... Igor Skornyakov

01:33 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> What is the problem and do you have an idea about the level of effort required to fix it?
... Igor Skornyakov

01:22 PM Feature #1587: implement full support for word indexes

What is the problem and do you have an idea about the level of effort required to fix it? Eric Faulhaber

08:14 AM Feature #1587: implement full support for word indexes

Found a regression in the denormalized extents' support.
Working on this. Igor Skornyakov

07:23 AM Feature #1587: implement full support for word indexes

I've just found a stupid bug in the @schema_word_tables_<db>_<dialect>.sql@ script generation. It is not a problem fo... Igor Skornyakov

03/23/2021

12:17 PM Bug #4163 (Closed): SHARED non-NEW buffer definitions must act as the equivalent of FIND or CREATE

Greg Shah

03/22/2021

10:55 AM Support #5183: database collation

I've committed a newer version of the collation data collector program to 3821c/12160.
The @locale/cp_coll/@ direc... Eric Faulhaber

05:26 AM Feature #1587: implement full support for word indexes

Re-worked ordering looks a little slower than before. However, it is still much faster than with UDF.
| expr ... Igor Skornyakov

03/21/2021

05:16 PM Feature #1587: implement full support for word indexes

Implicit sorting for @_temp@ and 'dirty' databases is implemented. Will be committed tomorrow after additional testin... Igor Skornyakov

09:04 AM Feature #1587: implement full support for word indexes

Re-worked implicit ordering for word tables. See #1587-462.
Committed to 3821c/12157. Igor Skornyakov

03/19/2021

01:18 PM Feature #1587: implement full support for word indexes

I've realized that the implicit ordering described in #1587-407 is not correct and can result in the repeated records... Igor Skornyakov