FWD » Core Development » Database

03/06/2021

02:41 PM Feature #1587: implement full support for word indexes

The problem with ... Igor Skornyakov

08:21 AM Feature #1587: implement full support for word indexes

Added CNF ordering (see #1576-386).
Committed to 3821c/12089 Igor Skornyakov

05:10 AM Feature #1587: implement full support for word indexes

I understand that the order of terms in the @CONTAINS@@ expression is important for implicit sorting. So I'm going to... Igor Skornyakov

11:24 AM Support #5183: database collation

In the "Internationalize ABL Application" docs there are good details about codepages and all the character configura... Greg Shah

03/05/2021

05:17 PM Support #5183: database collation

The attached program (@cp_coll.p@) was written as a starting point to review the 4GL's collation, inspired by the exi... Eric Faulhaber

05:05 PM Support #5183: database collation

A review of how FWD collates its database data has raised some questions as to whether we are doing it correctly for ... Eric Faulhaber

05:00 PM Support #5183 (WIP): database collation

Eric Faulhaber

03/04/2021

08:02 AM Bug #5176: Buffer field attribute leak

My understanding is that in the 4GL every temp-table *instance* is a completely separate table. Multiple table insta... Greg Shah

08:10 PM Bug #5176: Buffer field attribute leak

I will try to summarize an ugly issue I am investigating lately.
I was working on a issue related to #3574-11 and ... Ovidiu Maxiniuc

07:07 PM Bug #5176 (Closed): Buffer field attribute leak

Ovidiu Maxiniuc

06:29 AM Feature #1587: implement full support for word indexes

I've noticed that queries with @CONTAINS@ run faster with CTE when the expression is 'simple' (its CNF contains only ... Igor Skornyakov

03/03/2021

03:46 PM Feature #1587: implement full support for word indexes

Added SQL re-writing using common table expressions (WITH clause).
Committed to 3821c/12077.
For PostgreSQL, it i... Igor Skornyakov

03/02/2021

02:41 AM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Well, it is extra work to add it in @HQLPreprocessor@ and I imagine there probably is some w... Igor Skornyakov

03/01/2021

05:59 PM Feature #1587: implement full support for word indexes

Well, it is extra work to add it in @HQLPreprocessor@ and I imagine there probably is some work to ignore it when gen... Eric Faulhaber

05:19 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> The RTRIM is injected for text property references by the @HQLPreprocessor@. It is needed fo... Igor Skornyakov

05:09 PM Feature #1587: implement full support for word indexes

The RTRIM is injected for text property references by the @HQLPreprocessor@. It is needed for normal text property eq... Eric Faulhaber

04:44 PM Feature #1587: implement full support for word indexes

I've noticed some strange thing.
Consider the following 4GL code:... Igor Skornyakov

02/26/2021

05:42 PM Bug #4994 (WIP): NPE in AdaptiveQuery when loading null data for an outer join

In case of OUTER JOIN tables when there is no matching record, @data[i]@ is @null@. Rewriting the problematic line as... Ovidiu Maxiniuc

02:47 PM Support #4550: eliminate the need to place the p2jspi.jar in the JVM extension directory

It is good to know that the bug is just a configuration issue. At a minimum we need to improve our documentation. F... Greg Shah

02:28 PM Support #4550: eliminate the need to place the p2jspi.jar in the JVM extension directory

Greg Shah wrote:
> If I understand correctly, the @p2jspi.jar @ is hard coded to the equivalent of the 4GL 8859-1.
... Eric Faulhaber

07:53 AM Support #4550: eliminate the need to place the p2jspi.jar in the JVM extension directory

We also need to consider that the effective codepage for H2 temp-tables must be something that can be overridden. In... Greg Shah

07:49 AM Feature #4722: data import should be able to run with only the converted application jar file (an...

> Both of these dependencies could be specified as parameters to a new command line program
I don't understand why... Greg Shah

02:11 AM Bug #5056: Temp-tables and unique indexes

Can this issue be closed? Eric Faulhaber

02/25/2021

03:07 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> A separate thought: what if we remove the @distinct@ keyword? Unless I'm reading the plan wr... Igor Skornyakov

03:03 PM Feature #1587: implement full support for word indexes

I'm trying to figure out whether there is a way to express these search conditions without the hash joins (and someti... Eric Faulhaber

02:45 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Thank you.
>
> Are these two statements functionally equivalent?
>
> 1)
>
> [...]
... Igor Skornyakov

02:37 PM Feature #1587: implement full support for word indexes

As I re-read that last post, I'm not sure my where clause makes sense in that second query...rethinking... Eric Faulhaber

02:35 PM Feature #1587: implement full support for word indexes

Thank you.
Are these two statements functionally equivalent?
1)... Eric Faulhaber

02:06 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> What is the SQL for the expression @"s0 | s1"@?
>
> For @"(s0 | s1) & s5*"@?
>
> (LE: ... Igor Skornyakov

01:59 PM Feature #1587: implement full support for word indexes

What is the SQL for the expression @"s0 | s1"@?
For @"(s0 | s1) & s5*"@?
(LE: fixed syntax error in original post) Eric Faulhaber

01:53 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Igor, can't the first form of the SQL be simplified as follows?
> In other words, why is th... Igor Skornyakov

01:39 PM Feature #1587: implement full support for word indexes

Igor, can't the first form of the SQL be simplified as follows?
From:... Eric Faulhaber

11:02 AM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Yes, I was looking at the sort part of the plan, but you are right that the parts below do l... Igor Skornyakov

11:01 AM Feature #1587: implement full support for word indexes

Yes, I was looking at the sort part of the plan, but you are right that the parts below do look somewhat faster. Plea... Eric Faulhaber

10:54 AM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
>
> > At this moment I'm looking how to change SQL re-writing usin... Igor Skornyakov

10:43 AM Feature #1587: implement full support for word indexes

Igor Skornyakov wrote:
> At this moment I'm looking how to change SQL re-writing using @WITH@ clause (see #1587-36... Eric Faulhaber

10:30 AM Feature #1587: implement full support for word indexes

I've added two things to the word tables support.
# The re-writing of the SQL queries with @CONTAINS@ re-writing can... Igor Skornyakov

04:27 AM Feature #1587: implement full support for word indexes

With a logically equivalent but different re-written SQL statement using @WITH@ clause:... Igor Skornyakov

03:46 AM Feature #1587: implement full support for word indexes

Adding index on the 'word' field in the word tables helps. The times are (see #1587-357)
|expr |records|e... Igor Skornyakov

03:09 AM Feature #1587: implement full support for word indexes

Word tables' tests to @sftp://xfer.goldencode.com/opt/testcases/@ revision 1008.
Tests (including Java project for g... Igor Skornyakov

02:07 AM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Igor, is this test code (including the code which generates the test data) checked in?
No... Igor Skornyakov

02:04 AM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Igor, while the improvement over the UDF approach is considerable in some cases, the new app... Igor Skornyakov

01:21 PM Bug #5056 (Test): Temp-tables and unique indexes

Patch committed as 3821c/12065. Eric Faulhaber

09:25 PM Bug #5056: Temp-tables and unique indexes

Eric Faulhaber wrote:
> Ovidiu, please test whether the attached patch resolves your original issue(s). It gets both... Ovidiu Maxiniuc

09:42 AM Feature #5165: disallow usage of an incompatible database instance

We can also validate the schema specified at the permament DMOs with the database tables/indexes. Constantin Asofiei

05:48 AM Feature #5165: disallow usage of an incompatible database instance

In OE, the database metaschema has CRC values stored for each table, index and schema trigger. When compiling 4GL to... Greg Shah

05:08 AM Feature #5165 (New): disallow usage of an incompatible database instance

Greg Shah

02/24/2021

06:35 PM Feature #1587: implement full support for word indexes

Igor, is this test code (including the code which generates the test data) checked in? Eric Faulhaber

05:22 PM Feature #1587: implement full support for word indexes

Igor, while the improvement over the UDF approach is considerable in some cases, the new approach is still not as per... Eric Faulhaber

02:33 PM Feature #1587: implement full support for word indexes

For the performance testing, I've used a modified approach described at #1587-340. Instead of generated "word" as all... Igor Skornyakov

02/23/2021

06:34 PM Bug #5056: Temp-tables and unique indexes

Similar to the patch proposed in #4884-15, I have fixed the cases identified above. The three main xfer testcases are... Ovidiu Maxiniuc

07:43 PM Bug #5056: Temp-tables and unique indexes

I found the cause for the 'regression' of your patch. It is probably not a single thing, but a couple of issues which... Ovidiu Maxiniuc

02/20/2021

11:26 AM Feature #1587: implement full support for word indexes

Finished with word tables support for denormalized extent fields. Tested with a large customer app. and with test app... Igor Skornyakov

07:46 PM Bug #5056: Temp-tables and unique indexes

I used your patch to run the xfer tests and things have changed. It is not perfect, but I cannot tell exactly why.
... Ovidiu Maxiniuc

02/19/2021

11:31 AM Feature #1587: implement full support for word indexes

Added word tables' support for custom extents (conversion and database import).
Committed to 3821c/12031 Igor Skornyakov

10:48 AM Feature #4397 (Closed): add database attrs, methods and options

Ovidiu has recently confirmed that testing passed for the features in this task. Greg Shah

02/18/2021

06:27 PM Bug #5056: Temp-tables and unique indexes

Eric,
I looked deeper into the server's log (there are a lot of @java.lang.IllegalStateException: Empty scopes in Un... Ovidiu Maxiniuc

06:19 PM Bug #5056: Temp-tables and unique indexes

Eric,
I am sorry to inform you that the patch is not making the testsets behave better. In fact, the test stops much... Ovidiu Maxiniuc

05:10 PM Bug #5056: Temp-tables and unique indexes

Ovidiu, please test whether the attached patch resolves your original issue(s). It gets both of the test cases you pr... Eric Faulhaber

05:00 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Please don't. If we were starting from scratch, we could consider it, but the changes you sh... Igor Skornyakov

04:44 PM Feature #1587: implement full support for word indexes

Please don't. If we were starting from scratch, we could consider it, but the changes you should be making to DDL gen... Eric Faulhaber

04:15 PM Feature #1587: implement full support for word indexes

As I can see at this moment FWD has a dependency on Apache Velocity. Can I use it for DDL generation? This will make ... Igor Skornyakov

03:44 PM Feature #1587: implement full support for word indexes

I've resolved the issues described in #1587-349, implemented the generation of the word tables with indices/constrain... Igor Skornyakov

03:13 PM Feature #4022: upgrade to a newer PostgreSQL release

> * Did the package come out of the standard Ubuntu repos? Or did you have to add a PPA to the system?
Ubuntu.
... Stanislav Lomany

01:55 PM Feature #4022: upgrade to a newer PostgreSQL release

>> Were you able to leverage the pre-built PL/Java binaries?
>
> I installed PL/Java package using apt, along with ... Greg Shah

02/17/2021

04:02 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> Igor Skornyakov wrote:
> > I've noticed two things regarding custom extends.
> > # If the ... Igor Skornyakov

03:58 PM Feature #1587: implement full support for word indexes

Igor Skornyakov wrote:
> I've noticed two things regarding custom extends.
> # If the number of the @<extent-field@... Eric Faulhaber

02:57 PM Feature #1587: implement full support for word indexes

I've noticed two things regarding custom extends.
# If the number of the @<extent-field@ subnodes on the .hints file... Igor Skornyakov

02/16/2021

01:57 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> No problem. The annotation for the @wiext@ index looks correct to me as is.
Well, formally ... Igor Skornyakov

01:50 PM Feature #1587: implement full support for word indexes

Igor Skornyakov wrote:
> I'm talking about @name = "fextFirst", legacy = "f-ext"@. The word table name was also ba... Eric Faulhaber

01:21 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> I'm having trouble understanding the question. Please provide an example.
I mean the follow... Igor Skornyakov

01:10 PM Feature #1587: implement full support for word indexes

Igor Skornyakov wrote:
> I've noticed that for word index with customized extent fields the value of the @name@ attr... Eric Faulhaber

01:05 PM Feature #1587: implement full support for word indexes

I've noticed that for word index with customized extent fields the value of the @name@ attribute of the @IndexCompone... Igor Skornyakov

02/12/2021

01:11 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> There are two aspects of performance we need to understand and measure from this testing:
>... Igor Skornyakov

12:44 PM Feature #1587: implement full support for word indexes

There are two aspects of performance we need to understand and measure from this testing:
* read (upon which the c... Eric Faulhaber

11:01 AM Feature #1587: implement full support for word indexes

Here is the suggested structure of the test table for performance testing of the new @CONTAINS@ support:
The test ta... Igor Skornyakov

02/10/2021

03:28 AM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> I understand we need to correct the sort behavior for queries which use word indices, but I ... Igor Skornyakov

01:39 AM Feature #1587: implement full support for word indexes

Ovidiu Maxiniuc wrote:
> Igor Skornyakov wrote:
> > I have problems with removing @CONTAINS@ field from the @genera... Eric Faulhaber

02/09/2021

03:27 PM Feature #1587: implement full support for word indexes

Suppressed using word index field for sorting.
Committed to 3821c revision 12004. Igor Skornyakov

01:29 PM Feature #1587: implement full support for word indexes

Ovidiu Maxiniuc wrote:
> The @sort@ parameter is computed in @annotations/index_selection.rules@. Look for the @orde... Igor Skornyakov

01:18 PM Feature #1587: implement full support for word indexes

Igor Skornyakov wrote:
> I have problems with removing @CONTAINS@ field from the @generated@ 'sort' parameter of the... Ovidiu Maxiniuc

02/08/2021

04:51 PM Feature #1587: implement full support for word indexes

Added runtime uppercasing of the CONTAINS UDF arguments (for temp-tables).
Committed to 3821c/11999. Igor Skornyakov

03:06 PM Feature #1587: implement full support for word indexes

I have problems with removing @CONTAINS@ field from the @generated@ 'sort' parameter of the @AdaptiveQuery.initialize... Igor Skornyakov

03:00 PM Feature #1587: implement full support for word indexes

Uppercasing the @CONTAINS@ arguments at conversion time removed in #3821/11998. Igor Skornyakov

09:26 AM Feature #1587: implement full support for word indexes

Added a subtask to #1587-327 which I've forgot to mention (Implement CONTAINS support for denormalized extent fields). Igor Skornyakov

04:28 PM Feature #5130: improve management of p2jpl.jar in deployment scripts

Eric Faulhaber wrote:
> Why do we use the most current @p2jpl.jar@ version from @${deploy.home.abs}/lib/@ only in th... Constantin Asofiei

03:49 PM Feature #5130: improve management of p2jpl.jar in deployment scripts

Does anyone know why we have this in the "template" @build_db.xml@ (not really a template, but it tends to get copied... Eric Faulhaber

02/05/2021

12:06 PM Feature #1587: implement full support for word indexes

This isn't about the word tables. I have to determine whether a word index comes into play with DMO validation and fl... Eric Faulhaber

12:00 PM Feature #1587: implement full support for word indexes

Eric Faulhaber wrote:
> One additional item, which I'll take as my responsibility, is to test how a word index plays... Igor Skornyakov

11:30 AM Feature #1587: implement full support for word indexes

One additional item, which I'll take as my responsibility, is to test how a word index plays into validation/flushing... Eric Faulhaber

03:32 AM Feature #1587: implement full support for word indexes

Remaining work for the word indices support:
# Fix regressions introduced by word tables' support (if found).
# Rem... Igor Skornyakov