FWD » Core Development » Database

• always must start with 0x' (lowercase 'x'). 0X is invalid
• hexadecimal digits 'a' to 'f' must be also in lowercase. 0x0A is invalid.
• must have at least 2 hexadecimal digits after 0x. 0x0 to 0xa are invalid.
• must have maximum 6 hexadecimal digits, the largest ROWID is 0xffffff. 0x1000000 is invalid.

Implementation on P2J should respect these, isn't it ?

Ovidiu Maxiniuc wrote:

After some investigation on the OpenEdge installation on windev01 I observed that string representation of @ROWID@s:

• always must start with 0x' (lowercase 'x'). 0X is invalid
• hexadecimal digits 'a' to 'f' must be also in lowercase. 0x0A is invalid.
• must have at least 2 hexadecimal digits after 0x. 0x0 to 0xa are invalid.
• must have maximum 6 hexadecimal digits, the largest ROWID is 0xffffff. 0x1000000 is invalid.

Implementation on P2J should respect these, isn't it ?

The first 3 bullets make clear sense, and our implementation should match these requirements.

The 4th, however, seems odd. How did you determine the 6 digit limit? If it is correct, it suggests some sort of temporary mapping of string values to rowids (similar to what we do with the handle class), rather than a "pure" string representation of the underlying rowid value. But could it be that your test was simply limited to a number of records that did not require more than 3 bytes in order to represent the largest rowid internal value? Can you produce a test with more than 2^24 (very simple/small) records in a temp table to confirm this hypothesis?

Bullet 4 is datailed in my next note (4).
0x1000000 is invalid just because the number of digits is odd. However, if you

DISPLAY STRING(TO-ROWID('0x12345678')).

2^24 is about 16 millions. I will try to create a small table with such record population.

Ovidiu Maxiniuc wrote:

Bullet 4 is datailed in my next note (4).
0x1000000 is invalid just because the number of digits is odd. However, if you
[...]
you will obtain a 0x123456. Just tested on windev1.

2^24 is about 16 millions. I will try to create a small table with such record population.

The odd vs. even digit constraint makes sense, since each pair of digits represents a byte.

I think the issue here with the perceived 6 digit limit is that the default format string for a character displayed in a frame is "x(8)", which means you will see 8 characters (i.e. "0x" + six hex digits) in your output, regardless of the actual string length. It will truncate any characters after the left-most 8.

Try instead something like:

DISPLAY STRING(TO-ROWID("0x12345678")) format "x(50)".

I think that should work, but if not, try adding a format phrase to the STRING function as well:

DISPLAY STRING(TO-ROWID("0x12345678"), "x(50)") format "x(50)".

Try it with wider rowid strings, too (e.g., "0x123456789abcdef123456789abcdef") and see if there is a practical limit. Our practical limit in P2J is that of a 64 bit integer.

You are right, the cutting is caused by default print format, in fact I could not find a limit for the size of this type of data.
So, to keep compatibility, a bigger datatype should be used like BigInteger. I'm pretty sure this does't make sense because of the performance penalty.

At this point, the 16M record count experiment will prove nothing, I guess.

On the other side, I observed that Progress stores the ROWID s internally as string. This is proved by the fact that STRING(TO-ROWID("0x000000") == "0x000000", for any even numbers of 0 s. This means that OpenEdge stores the actual string internally (?) and maybe perform a lazy-conversion only when needed (?).

I will adjust the implementation to support our internal 63 bit integer, as this is a fair record size for a table, but don't know how to handle the (rare but possible) overflow case (throw an exception or silently trim the extra digits ?).

Ovidiu Maxiniuc wrote:

You are right, the cutting is caused by default print format, in fact I could not find a limit for the size of this type of data.
So, to keep compatibility, a bigger datatype should be used like BigInteger. I'm pretty sure this does't make sense because of the performance penalty.

No, I don't see a reason to use BigInteger, since our internal data type is a long.

On the other side, I observed that Progress stores the ROWID s internally as string. This is proved by the fact that STRING(TO-ROWID("0x000000") == "0x000000", for any even numbers of 0 s. This means that OpenEdge stores the actual string internally (?) and maybe perform a lazy-conversion only when needed (?).

That is a fair guess, though their exact, internal representation is unknown. In any case, this is where our implementation diverges, since we use a long, and we do not expect to be able to convert an existing, 4GL rowid string to a meaningful P2J rowid at runtime. We will have to be aware of this difference if we come across a dependence on persisted, existing, 4GL rowid strings, but it is not something we can deal with in the implementation of this built-in function. Rather, it is an application issue we would need to correct/change before conversion.

I will adjust the implementation to support our internal 63 bit integer, as this is a fair record size for a table, but don't know how to handle the (rare but possible) overflow case (throw an exception or silently trim the extra digits ?).

In the case of overflow, let's throw an ErrorConditionException, but not directly; use ErrorManager.recordOrThrowError. While this diverges from the 4GL behavior, in that we won't be able to support conversion to/from very large rowid strings, I think that sort of use is largely academic, and is not particularly pragmatic. I think we need to bias the implementation toward the more likely, production uses of this feature, which should normally be constrained to "real" rowids that actually represent records in the database.

ErrorManager.recordOrThrowError() methods require a number or a NumberedException. I understand that this is in fact an error code of 4GL. What number should I use as long as Progress does not generate any errors/warnings here ? Should we use 0, -1 or other negative number as a special case of P2J code?

Ovidiu Maxiniuc wrote:

Should we use 0, -1 or other negative number as a special case of P2J code?

-1.

• Sorry, I was mistaken with respect to the format I originally specified for the rowid string format (beside the uppercase mistake). Since it must represent a 64 bit number, it must have up to 16 hex digits after the "0x" prefix, not 8. The implementation in rowid.java needs to be changed to reflect this (including in the JavaDoc).
• Please don't put a double-slash comment under the line of code it describes, but rather above or (for very short, targeted comments) to the right.
• In the event of any parsing or format problems in the rowid constructor, we effectively create a rowid instance which represents the unknown value (by setting the internal value member to null). Is this how TO-ROWID reacts to the same errors in Progress?
• Please fix the indent in builtin_functions.rules (should be 3 instead of 6 spaces).

Where are the test cases you wrote for the wider rowid strings? I didn't see that aspect of your research reflected in the test case x2rowid.p.

Feedback from code review (1030a):

• Please confirm that in the event of a failure in converting a rowid string to a rowid, Progress creates a rowid representing unknown value (rather than raising an error). This is what our implementation of rowid.parseRowidString() does in most failure cases (by returning null) and I want to be sure it is the correct behavior.
• The first overflow test in rowid.parseRowidString() is good, but the second one should not be necessary. Overflow of the sign bit is OK, it will just result in a negative long, which will still represent a unique primary key. So, it is not necessary to treat the rowid as a 63-bit long, but rather as a 64-bit long.
• Please fix the references in the JavaDoc to 0xhhhhhhhh to reflect the 16-digit hex format supported (0xhhhhhhhhhhhhhhhh).
• There are still some small format issues (opening curly braces in parseRowidString on wrong line, no space between // and comment in one case). Please fix these.

• yes, OE displays [yes] when running the following code: display to-rowid("dksajdhk") = ?.
• ok, I change the code to use the sign bit, too, but this seems to me a patch rather than natural
• the string representation of rowid are not mandatory 16 digits long, any even length is legal. Maybe this should be clarified.
• I found the missing space after double slash, but the { was harder to spot. It's difficult to get rid of old habits.

Ovidiu Maxiniuc wrote:

• yes, OE displays [yes] when running the following code: display to-rowid("dksajdhk") = ?.

OK, thanks.

• ok, I change the code to use the sign bit, too, but this seems to me a patch rather than natural

Since we support any signed Long as a rowid, I think we have to do it this way to be consistent.

• the string representation of rowid are not mandatory 16 digits long, any even length is legal. Maybe this should be clarified.

Good idea.

• I found the missing space after double slash, but the { was harder to spot. It's difficult to get rid of old habits.

Sorry, I know our choice here is uncommon in the Java world.

Please update the Conversion Reference (builtin_database_functions_methods_attributes.odt chapter) to reflect the implementation of this built-in function.

Don't forget that there is also the "master" table of all supported built-in functions in the "Functions" section of the "Expressions" chapter in that same book. Please update that table too.