Project

General

Profile

Bug #11149

DO i = 1 TO LENGTH(r, "RAW") converts to incompatible type expression

Added by Paul Bodale 6 months ago. Updated 6 months ago.

Status:
Review
Priority:
Normal
Assignee:
Target version:
-
Start date:
Due date:
% Done:

100%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:
production:
No
env_name:
topics:

Related issues

Related to Base Language - Bug #4766: fix CHR and ASC WIP
Related to Base Language - Feature #9448: compatibility test security features Closed
Related to Base Language - Bug #2657: appserver agent context reset does not deallocate currently allocated memptr buffers Closed

History

#1 Updated by Paul Bodale 6 months ago

I found the following conversion problem:

FUNCTION rawToHex RETURNS CHARACTER (INPUT ch AS RAW):
   DEFINE VARIABLE res AS CHARACTER INITIAL "" NO-UNDO.
   DEFINE VARIABLE i AS INTEGER NO-UNDO.
   DO i = 1 TO LENGTH(ch, "RAW"):
      res = res + intToHex(GET-BYTE(ch, i)) + "  ".
   END.

   RETURN res.
END FUNCTION.

The stop condition converts to:

         IntegerExpression resExpr0 = new IntegerExpression()
         {
            public integer execute()
            {
               return BinaryData.length(ch);
            }
         };
         ToClause toClause0 = new ToClause(i, 1, resExpr0);

Which doesn't compile in Java as BinaryData.length(ch) returns an int64 value.

Perhaps it should instead convert to:

         Int64Expression resExpr0 = new Int64Expression()
         {
            public int64 execute()
            {
               return BinaryData.length(ch);
            }
         };
         ToClause toClause0 = new ToClause(i, 1, resExpr0);

#2 Updated by Teodor Gorghe 6 months ago

  • Status changed from New to WIP
  • Assignee set to Teodor Gorghe

I don't think it needs to be int64 because in 4GL, the return type of LENGTH is INTEGER.
I will take care of that today.

#3 Updated by Paul Bodale 6 months ago

  • Related to Bug #4766: fix CHR and ASC added

#4 Updated by Teodor Gorghe 6 months ago

  • Related to Feature #9448: compatibility test security features added

#5 Updated by Teodor Gorghe 6 months ago

  • Related to Bug #2657: appserver agent context reset does not deallocate currently allocated memptr buffers added

#6 Updated by Constantin Asofiei 6 months ago

Teodor, please see if changing the function's return type in SignatureHelper solves the issue - that is still marked as integer.

#7 Updated by Teodor Gorghe 6 months ago

Constantin Asofiei wrote:

Teodor, please see if changing the function's return type in SignatureHelper solves the issue - that is still marked as integer.

It does not fix the issue.

#8 Updated by Constantin Asofiei 6 months ago

progress.g needs to also be changed: sym.addBuiltinFunction("length" , FUNC_INT64 , true );

#9 Updated by Teodor Gorghe 6 months ago

Yes, but this is not right. The return type of LENGTH should be INTEGER.

#10 Updated by Constantin Asofiei 6 months ago

Teodor Gorghe wrote:

Yes, but this is not right. The return type of LENGTH should be INTEGER.

Please find the revision when it was decided to use INT64 and ask the original writer the reason behind it.

#11 Updated by Teodor Gorghe 6 months ago

+ Ovidiu

tg@tg-Yoga-Pro-7-14APH8:~/gcd/testcases/p2j$ bzr log -r 11069.1.1
------------------------------------------------------------
revno: 11069.1.1 [merge]
committer: Ovidiu Maxiniuc <om@goldencode.com>
branch nick: 2657b
timestamp: Tue 2016-07-12 00:29:30 +0300
message:
  Implemented OS-sensitive memptr behaviour. Added memptr/raw parameter support (in SourceNameMapper). Switch to int64 as return type of DynamicOps.length()
------------------------------------------------------------
Use --include-merged or -n0 to see merged revisions.

I think the reason is that GET-SIZE uses BinaryData.length and the return type needs to be INT64.
I already have a quick fix for this, which is to wrap the return value to integer, like how DynamicOps.length does:

=== modified file 'rules/annotations/functions.rules'
--- old/rules/annotations/functions.rules    2025-08-28 08:13:05 +0000
+++ new/rules/annotations/functions.rules    2026-01-27 07:57:17 +0000
@@ -430,12 +430,6 @@
             <rule>arg_type != null and arg_type.equalsIgnoreCase("memptr")
                <action>arg_type = "raw"</action>
             </rule>
-            <rule>arg_type != null and arg_type.equals("unknown")
-               <action>
-                  throwException("Incompatible data types in expression or assignment. (223).",
-                                 this)
-               </action>
-            </rule>

             <!-- if present, detect the second parameter's value, break in invalid literals -->
             <rule>this.getNumImmediateChildren() > 1
@@ -496,6 +490,15 @@
                (arg_type.equalsIgnoreCase("raw") or arg_type.equalsIgnoreCase("blob"))
                <action>putNote("methodText", "BinaryData.length")</action>
             </rule>
+
+            <rule>arg_type != null and arg_type.equals("unknown")
+               <!-- Replace with unknown literal -->
+               <action>ref = createProgressAst(prog.unknown_val, copy.parent, copy.getIndexPos())</action>
+               <action>ref.putAnnotation("classname", "integer")</action>
+         
+               <!-- delete myself, the new node was inserted in my place -->
+               <action>copy.remove()</action>
+            </rule>

             <!-- if not processed above, wire the dynamic form -->
             <rule>isNote("methodText")

=== modified file 'rules/convert/builtin_functions.rules'
--- old/rules/convert/builtin_functions.rules    2025-09-15 11:13:00 +0000
+++ new/rules/convert/builtin_functions.rules    2026-01-27 08:34:25 +0000
@@ -1326,6 +1326,10 @@
                   <rule on="false">methodText.equals("TextOps.columnLength")
                      <action>methodText = "columnLength"</action>
                      <action>textopsimp = true</action>
+
+                     <rule on="false">methodText.equals("BinaryData.length")
+                        <action>putNote("chp_wrapper", "integer")</action>
+                     </rule>
                   </rule>
                </rule>
             </rule>

The fix in functions.rules is just for the case when the parameter is unknown, eg. LENGTH(?).

#12 Updated by Constantin Asofiei 6 months ago

It's either the LENGTH function is documented wrong in 4GL docs and we need it to be INT64 in FWD or the BinaryData.length return type must be integer instead of int64 in FWD. Lets wait for an answer from Ovidiu.

#13 Updated by Ovidiu Maxiniuc 6 months ago

  • reviewer Ovidiu Maxiniuc added

Definitely, the stop condition should be Int64Expression. This is more evident when you change the iteration to:

DO i = 1 TO 1 + LENGTH(ch, "RAW"):

In earlier 4GL there was no int64 datatype. It was added at 9.x or something similar. Starting with that point, all operations are internally performed on 64 bits integers. From my experiments, only when the final storage is performed the resulting values are truncated to fit into the destination location (variable or field).

I think the documentation is still not updated. length should be a int64 function. Indeed, ATM, there are no cases when this function will return a value which will not fit in int to prove it because the larger datatype it can be applied to is the longchar and it is limited to2 32-1 chars. But in practice the 4GL programmer does not care. The problem only occurs in FWD because we have wrappers instead of binary representations.

#14 Updated by Constantin Asofiei 6 months ago

Ovidiu Maxiniuc wrote:

Definitely, the stop condition should be Int64Expression. This is more evident when you change the iteration to:[...]

In earlier 4GL there was no int64 datatype. It was added at 9.x or something similar. Starting with that point, all operations are internally performed on 64 bits integers. From my experiments, only when the final storage is performed the resulting values are truncated to fit into the destination location (variable or field).

I think the documentation is still not updated. length should be a int64 function.

I did a test with two overloaded methods in a 4GL class, with a single INTEGER and INT64, and using DYNAMIC-INVOKE or direct call, with a LENGTH function argument, the INTEGER one is always executed. So, Ovidiu, here we are not talking about complex expressions which are always evaluated to int64, but about the return type of the LENGTH function.

#15 Updated by Ovidiu Maxiniuc 6 months ago

That's a valuable tool we did not have a decade+ ago, when int64 support was added in FWD.

We need to get the type correctly, or else overload methods will not be called as in 4GL. IMO, the converted code should be:

         Int64Expression resExpr0 = new Int64Expression()
         {
            public int64 execute()
            {
               return BinaryData.length(ch);
            }
         };
         ToClause toClause0 = new ToClause(i, 1, resExpr0);

         doTo("loopLabel0", toClause0, new Block((Body) () ->
regardless of the result type of BinaryData.length(...).

However, I recommend using a local variable to store the LENGTH(ch, "RAW") in ABL code, as an optimisation (avoiding re-evaluating it N times).

#16 Updated by Constantin Asofiei 6 months ago

Ovidiu Maxiniuc wrote:

We need to get the type correctly, or else overload methods will not be called as in 4GL. IMO, the converted code should be:[...]regardless of the result type of BinaryData.length(...).

Why do you say we need Int64Expression and not IntegerExpression and the return type of BinaryData.length(ch); fixed to integer?

However, I recommend using a local variable to store the LENGTH(ch, "RAW") in ABL code, as an optimisation (avoiding re-evaluating it N times).

The termination expression is already evaluated only once - see ToClause in FWD, in incrementAndTest.

#17 Updated by Ovidiu Maxiniuc 6 months ago

Constantin Asofiei wrote:

Ovidiu Maxiniuc wrote:
Why do you say we need Int64Expression and not IntegerExpression and the return type of BinaryData.length(ch); fixed to integer?

If the code is changed to

DO i = 1 TO 1 + LENGTH(ch, "RAW"):
the converted expression is
         Int64Expression resExpr0 = new Int64Expression()
         {
            public int64 execute()
            {
               return plus(1, BinaryData.length(ch));
            }
         };
This is debatable, and as I said, this is my opinion. But will I agree with the integer variant if that doesn't break the conversions of existing projects.

However, I recommend using a local variable to store the LENGTH(ch, "RAW") in ABL code, as an optimisation (avoiding re-evaluating it N times).

The termination expression is already evaluated only once - see ToClause in FWD, in incrementAndTest.

Is it? I see end = term.resolve(); is evaluated on each increment, but end is a local variable and not saved for next call.
Beside that, the ch variable can change its length changes during the loop, affecting the number of iterations, as well. But this is out of the scope of this task.

#18 Updated by Teodor Gorghe 6 months ago

On 4GL, it is not possible to achieve a return value of LENGTH to be greater than int limit (2147483647) because the limit of LONGCHAR length is actually the max integer value (2GiB - 1).
I think the only issue of having LENGTH to return a INT64 is on DYNAMIC-INVOKE, as Constantin told on #11149-14.

#19 Updated by Teodor Gorghe 6 months ago

  • Status changed from WIP to Review
  • % Done changed from 0 to 100

Committed revision 16368 on task branch 11149a:
- LENGTH(RAW) return type should be INTEGER.

#20 Updated by Paul Bodale 6 months ago

Teodor Gorghe wrote:

On 4GL, it is not possible to achieve a return value of LENGTH to be greater than int limit (2147483647) because the limit of LONGCHAR length is actually the max integer value (2GiB - 1).

No but it is possible to have an expression that does so. For example, DO i = 1 TO maxInt + LENGTH(...)

#21 Updated by Teodor Gorghe 6 months ago

Paul Bodale wrote:

Teodor Gorghe wrote:

On 4GL, it is not possible to achieve a return value of LENGTH to be greater than int limit (2147483647) because the limit of LONGCHAR length is actually the max integer value (2GiB - 1).

No but it is possible to have an expression that does so. For example, DO i = 1 TO maxInt + LENGTH(...)

Yes, but 4GL and FWD already treats this case and extends the INTEGER to INT64.

Also available in: Atom PDF