FWD » Core Development » Database

10/11/2019

06:32 AM Support #3871: determine how to change codepages/locales during import

3871a was rebased up to revision 11346 over 11337 trunc. Sergey Ivanovskiy

10/10/2019

04:40 PM Feature #4323: parse full FQL statements and convert them to SQL

Ovidiu Maxiniuc wrote:
> I am trying to clean-up the code after DDL changes and I am not sure whether we need the @m... Eric Faulhaber

02:50 PM Feature #4323: parse full FQL statements and convert them to SQL

Eric,
I am trying to clean-up the code after DDL changes and I am not sure whether we need the @merge-dmo-root@ se... Ovidiu Maxiniuc

04:04 PM Feature #3815 (Review): more temp-table options

3816a has been merged into the trunk as bzr revision 11337. Stanislav Lomany

03:10 PM Support #3871: determine how to change codepages/locales during import

Committed revision 11344 (3871a) fixed missed comment. Sergey Ivanovskiy

03:01 PM Support #3871 (Review): determine how to change codepages/locales during import

The committed revision 11343 (3871a) should fix the cases when the source code page is UTF8 or a different code page ... Sergey Ivanovskiy

03:53 AM Support #3871: determine how to change codepages/locales during import

This it the test example. The question for me is that BOM (https://en.wikipedia.org/wiki/Byte_order_mark) is generate... Sergey Ivanovskiy

03:46 AM Support #3871: determine how to change codepages/locales during import

I found that the UTF-8 source dump file is imported incorrectly now. The problem here is @FileStream@ is supposed to ... Sergey Ivanovskiy

10/09/2019

08:49 AM Support #3871: determine how to change codepages/locales during import

Greg, please review the committed rev 11338 (3871a). It was tested with UTF-8 PostgresSql database. Sergey Ivanovskiy

10/08/2019

03:00 PM Feature #2156: datetime-tz and hibernate

Constantin Asofiei wrote:
> There's one other thing to add for @datetime-tz@ - its usage in indexes and sort clauses... Constantin Asofiei

01:47 PM Feature #2156: datetime-tz and hibernate

Stanislav Lomany wrote:
> Could you please elaborate what should be done for this task? datetime-tz seems to be work... Constantin Asofiei

12:21 PM Feature #2156: datetime-tz and hibernate

Could you please elaborate what should be done for this task? datetime-tz seems to be working in general. Stanislav Lomany

10:37 AM Support #3871: determine how to change codepages/locales during import

I doesn't matter what one byte encoding is used. If it is internally mapped into UTF-8, then this issue can be observ... Sergey Ivanovskiy

10:32 AM Support #3871: determine how to change codepages/locales during import

Sergey Ivanovskiy wrote:
> OK, it seems that the Hibernate uses UTF-8 internally because I experimented with US-ASCI... Sergey Ivanovskiy

06:52 AM Support #3871: determine how to change codepages/locales during import

OK, it seems that the Hibernate uses UTF-8 internally because I experimented with US-ASCII encoding as a default enco... Sergey Ivanovskiy

06:42 AM Support #3871: determine how to change codepages/locales during import

If there is no valid codepage conversion for the INPUT_ENCODING to the DATABASE_ENCODING, then you should throw an ex... Greg Shah

06:26 AM Support #3871: determine how to change codepages/locales during import

This is that imported file encoded as WINDOWS-1251. Sergey Ivanovskiy

06:24 AM Support #3871: determine how to change codepages/locales during import

I don't know the way how to convert parameters for HSQL queries like this one... Sergey Ivanovskiy

06:16 AM Support #3871: determine how to change codepages/locales during import

Sergey Ivanovskiy wrote:
> Greg Shah wrote:
> > > Then the client will read the imported data from this database as... Greg Shah

06:10 AM Support #3871: determine how to change codepages/locales during import

>Where are the UTF-8 strings coming from? Are they from the import data?
UTF-8 is the default encoding for my java e... Sergey Ivanovskiy

05:59 AM Support #3871: determine how to change codepages/locales during import

Greg Shah wrote:
> > Then the client will read the imported data from this database as LATIN-1 strings because the d... Sergey Ivanovskiy

05:47 AM Support #3871: determine how to change codepages/locales during import

> Then the client will read the imported data from this database as LATIN-1 strings because the database has LATIN-1 ... Greg Shah

05:25 AM Support #3871: determine how to change codepages/locales during import

Sergey Ivanovskiy wrote:
> I think that there is only one way to save UTF-8 strings into LATIN-1 database is to conv... Sergey Ivanovskiy

05:15 AM Support #3871: determine how to change codepages/locales during import

I think that there is only one way to save UTF-8/Unicode strings into LATIN-1 database that is to convert UTF-8 strin... Sergey Ivanovskiy

10/07/2019

02:17 PM Support #3871: determine how to change codepages/locales during import

I encountered the following issue that the WINDOWS-1251 string decoded into the UTF-8 string (if UTF-8 is a default j... Sergey Ivanovskiy

06:26 AM Support #3871: determine how to change codepages/locales during import

Let us consider this case in which the database has LATIN-1 encoding and the import table has WINDOWS-1251 encoding. ... Sergey Ivanovskiy

10:21 AM Feature #4323: parse full FQL statements and convert them to SQL

These are generateDDL() calls for the metadata database. I am working on it right now. Ovidiu Maxiniuc

10/04/2019

05:35 PM Feature #4323: parse full FQL statements and convert them to SQL

Are there any other project-level changes that need to be made, beside those in your last note? I'm trying to convert... Eric Faulhaber

04:01 PM Feature #4323: parse full FQL statements and convert them to SQL

I committed latest changes to 4011a/rr11344.
If you have an old ddl directory, please backup it. The conversion wi... Ovidiu Maxiniuc

02:16 PM Feature #4323: parse full FQL statements and convert them to SQL

Is there a new class @DDLGeneratorWorker@ that was supposed to be checked in?... Eric Faulhaber

01:59 PM Feature #4323: parse full FQL statements and convert them to SQL

I forgot to CTRL+F5 :(. Ovidiu Maxiniuc

01:57 PM Feature #4323: parse full FQL statements and convert them to SQL

Sorry, Eric. I cleaned the .g file only looking at the @ExpressionParser@. Indeed, I was too eager to drop the @java.... Ovidiu Maxiniuc

01:54 PM Feature #4323: parse full FQL statements and convert them to SQL

BTW, Greg has decreed that "FWDQL" should be "FQL" :)
Please work that name change back into the code, file names,... Eric Faulhaber

01:50 PM Feature #4323: parse full FQL statements and convert them to SQL

Ovidiu, I'm getting the following compile errors with 4011a/11342:... Eric Faulhaber

07:02 AM Bug #3471: incremental conversion is regressed

> for OO, the tricky part is keeping the Java converted method name consistent. If you recall, we need to keep this c... Greg Shah

10/03/2019

05:34 PM Bug #3471: incremental conversion is regressed

I'm still checking, but the current issues are:
* for OO, the tricky part is keeping the Java converted method name ... Constantin Asofiei

10/02/2019

01:54 PM Support #3871: determine how to change codepages/locales during import

Created task branch 3871a. Sergey Ivanovskiy

01:09 PM Bug #3471 (WIP): incremental conversion is regressed

Constantin Asofiei

01:09 PM Bug #3471: incremental conversion is regressed

Full convert/compile will always be required when someones updates FWD, and there are parser changes where the token ... Constantin Asofiei

09/27/2019

03:15 PM Bug #3379 (Closed): do not copy index definitions when defining WORKFILE like TABLE

I'm ok with the changes. Eric Faulhaber

02:41 PM Bug #3379: do not copy index definitions when defining WORKFILE like TABLE

Greg Shah wrote:
> 4103a was merged to trunk as revision 11331. Can this task be closed?
I think yes. Vladimir Tsichevski

02:37 PM Bug #3379: do not copy index definitions when defining WORKFILE like TABLE

4103a was merged to trunk as revision 11331. Can this task be closed? Greg Shah

09/25/2019

09:05 AM Support #3871: determine how to change codepages/locales during import

Google helps that this query follows ANSI standard schema view https://en.wikipedia.org/wiki/Information_schema
<pre... Sergey Ivanovskiy

09:00 AM Support #3871: determine how to change codepages/locales during import

I don't know a database-independent way to query a database encoding. For PosgreSQL this query returns its encoding
... Sergey Ivanovskiy

09/24/2019

10:22 AM Support #3871: determine how to change codepages/locales during import

Please read #3871-7 for the details on what you are meant to implement.
* For the input encoding,tThe @DataFileR... Greg Shah

10:15 AM Support #3871: determine how to change codepages/locales during import

Greg, I imported UTF-8 data into LATIN-1 PostgresSQL database using @-Dfile.encoding=UTF-8@ input parameter. I am not... Sergey Ivanovskiy

09:52 AM Support #3871: determine how to change codepages/locales during import

It seems that the client can use different encodings https://www.postgresql.org/docs/9.6/multibyte.html and hence the... Sergey Ivanovskiy

09:50 AM Support #3871: determine how to change codepages/locales during import

> What is the proper database encoding if data dump files are UTF-8? Should not it be UTF-8? Can it be LATIN-1 if ano... Greg Shah

09:46 AM Support #3871: determine how to change codepages/locales during import

What is the proper database encoding if data dump files are UTF-8? Should not it be UTF-8? Can it be LATIN-1 if anoth... Sergey Ivanovskiy

09:15 AM Support #3871: determine how to change codepages/locales during import

Ovidiu, thank you.
Now the db import scripts work properly, although the import of UTF-8 data
1 "cat"
2 "КОШК... Sergey Ivanovskiy

08:55 AM Support #3871: determine how to change codepages/locales during import

This is normal. The database was already imported at least once. Remove the sequence manually.
Note that if a table... Ovidiu Maxiniuc

08:50 AM Support #3871: determine how to change codepages/locales during import

Running this script for PostgreSQL... Sergey Ivanovskiy

09/23/2019

04:22 PM Support #3871: determine how to change codepages/locales during import

It seems that H2 or PostgreSQL should have the same issue when importing UTF-8 dumps. OK. I will test PostgreSQL. It ... Sergey Ivanovskiy

03:29 PM Support #3871: determine how to change codepages/locales during import

Sergey Ivanovskiy wrote:
> This configuration
> [...]
> doesn't work too.
What exactly is not working? If ther... Eric Faulhaber

03:27 PM Support #3871: determine how to change codepages/locales during import

Debug into the @PatternEngine@ to see why it is failing. Greg Shah

03:16 PM Support #3871: determine how to change codepages/locales during import

This configuration ... Sergey Ivanovskiy

02:48 PM Support #3871: determine how to change codepages/locales during import

This ant task works properly for this project:... Sergey Ivanovskiy

02:39 PM Support #3871: determine how to change codepages/locales during import

Yes, command line is wrong. It seems like @data/namespace encode2.p2o@ should not have a space in it. Greg Shah

02:11 PM Support #3871: determine how to change codepages/locales during import

Greg Shah wrote:
> Please see #4279-91 for how to fix the unicode character escape sequence. I don't know what is w... Sergey Ivanovskiy

09/22/2019

04:30 PM Support #3871: determine how to change codepages/locales during import

Please see #4279-91 for how to fix the unicode character escape sequence. I don't know what is wrong with the other ... Greg Shah

03:03 AM Support #3871: determine how to change codepages/locales during import

The fragment of the converted program looks incorrect too... Sergey Ivanovskiy

02:20 AM Support #3871: determine how to change codepages/locales during import

I created a 4GL databases @encode2.db@ using @UTF-8@ encoding based on @C:\Progress\OE116_64\prolang\utf\ICU-ru.df@. ... Sergey Ivanovskiy

09/20/2019

02:48 PM Support #3871: determine how to change codepages/locales during import

Sergey Ivanovskiy wrote:
> Are these statements correct that @-cpinternal@ code page is used for character variables... Greg Shah

02:00 PM Support #3871: determine how to change codepages/locales during import

and @proutil encode1.db -C idxbuild all -cpinternal UTF-8@ was successful. It seems that for UTF-8 database encoding ... Sergey Ivanovskiy

01:50 PM Support #3871: determine how to change codepages/locales during import

I used this command to rebuild indices for UTF-8 database and please look what was the output
@proutil encode1.db -C... Sergey Ivanovskiy

01:45 PM Support #3871: determine how to change codepages/locales during import

Are these statements correct that @-cpinternal@ code page is used for character variables and fields and @-cpstream@ ... Sergey Ivanovskiy

07:05 AM Support #3871: determine how to change codepages/locales during import

Some other things to check:
* to access the data dump, you need to start the procedure editor; what happens if you ... Constantin Asofiei

06:18 AM Support #3871: determine how to change codepages/locales during import

First step I think is to create a database in 4GL with non-default codepage/locale, add some records with non-ASCII c... Constantin Asofiei

06:09 AM Support #3871: determine how to change codepages/locales during import

What should be done in order to test solutions for this task? Sergey Ivanovskiy

06:07 AM Support #3871 (WIP): determine how to change codepages/locales during import

Sergey Ivanovskiy

05:54 AM Support #3871: determine how to change codepages/locales during import

It follows that within @ImportWorker@ namespace @DataFileReader.processPscHeader()@ fills @encoding@ field and this f... Sergey Ivanovskiy

09/19/2019

06:26 PM Feature #4323: parse full FQL statements and convert them to SQL

In a recent email, you noted you had added support for field/column aliases. I'm not sure whether you were referring ... Eric Faulhaber

06:13 PM Feature #4323: parse full FQL statements and convert them to SQL

Ovidiu Maxiniuc wrote:
> Two questions I thought about but I do not have the answer yet:
> * how to handle EXTENTS ... Eric Faulhaber

10:44 AM Feature #4307: DMO conversion changes

The @TempTableRecord@ methods are not directly accessed by the business logic. Instead they are accessed only interna... Ovidiu Maxiniuc

09/18/2019

04:33 PM Feature #4307: DMO conversion changes

Let me ask what I'm really trying to understand... Do the temp-table DMO proxies used by business logic ever need to ... Eric Faulhaber

04:15 PM Feature #4307: DMO conversion changes

Yes, you deduce correctly. @TempTableRecord@ adds 5 new fields to standard record. You need to look at 4124a to see a... Ovidiu Maxiniuc

03:31 PM Feature #4307: DMO conversion changes

Ovidiu, I am trying to rationalize the DMO class/interface hierarchy to eliminate casting and differential logic to h... Eric Faulhaber

09/17/2019

05:18 PM Feature #4323: parse full FQL statements and convert them to SQL

Two questions I thought about but I do not have the answer yet:
* how to handle EXTENTS (in denormalized case things... Ovidiu Maxiniuc

12:24 AM Feature #4015 (Rejected): upgrade to a newer Hibernate release

We have decided to replace Hibernate with a bespoke ORM implementation instead. Eric Faulhaber

09/16/2019

05:54 PM Feature #2156: datetime-tz and hibernate

Greg Shah wrote:
> Ovidiu Maxiniuc wrote:
> > I would go for an unified solution for all dialects, even this duplic... Eric Faulhaber

05:31 PM Feature #2156: datetime-tz and hibernate

Stanislav, please create a branch from 4124a and work on that. Don't commit to 4124a (this is expected to be in trun... Constantin Asofiei

05:30 PM Feature #2156 (WIP): datetime-tz and hibernate

Ovidiu Maxiniuc wrote:
> I would go for an unified solution for all dialects, even this duplicating the offset infor... Greg Shah

03:12 PM Feature #4323: parse full FQL statements and convert them to SQL

Ovidiu Maxiniuc wrote:
> Eric Faulhaber wrote:
> > No, I'm talking about true [OUTER] JOIN ... ON ... syntax. This ... Eric Faulhaber

02:44 PM Feature #4323 (WIP): parse full FQL statements and convert them to SQL

Eric Faulhaber wrote:
> No, I'm talking about true [OUTER] JOIN ... ON ... syntax. This is a hole we have today, in ... Ovidiu Maxiniuc

02:01 PM Feature #4323: parse full FQL statements and convert them to SQL

Ovidiu Maxiniuc wrote:
> I was working with @join@ ing tables. I have the following issue: there is no @on@ keyword ... Eric Faulhaber

01:10 PM Feature #4323: parse full FQL statements and convert them to SQL

I was working with @join@ ing tables. I have the following issue: there is no @on@ keyword in HQL. In fact, it is not... Ovidiu Maxiniuc

09/14/2019

05:39 PM Bug #4325: Not flushing the newly created record to the database

In the following sequence ... Stanislav Lomany

04:59 PM Bug #4325 (New): Not flushing the newly created record to the database

Stanislav Lomany

11:59 PM Feature #4307: DMO conversion changes

Well done.
As noted originally, however, the DMO implementation classes should _not_ be emitted by default. The de... Eric Faulhaber

09/13/2019

07:35 PM Feature #4323: parse full FQL statements and convert them to SQL

An intermediary status. I added support for gathering data from tables/fields annotations in a cached data structure ... Ovidiu Maxiniuc