FWD » Core Development » Base Language

Check the FIX-CODEPAGE tests in Testcases to ensure they are complete and comprehensive. Add tests if needed. Convert and run the tests. Fix any issues as needed until the test pass completely.

I modified the test which was failing to take into account the cpinternal startup parameter. Because in PROGRESS if we try to assign a codepage to a longchar which is not empty an error will be thrown but the codepage value will be the same as specified at the beginning of the program and this was reason for which the test was failing.

I added 2 more tests for a comprehensive coverage and also modified the structure of class CodepageFix so that each test has it's own method and for the class itself to be more clean and I tested will all the values of cpinternal found.

There are some adjacent changes in the commit:

=== modified file 'udfs/udf.test.gen/.settings/org.eclipse.buildship.core.prefs'
--- old/udfs/udf.test.gen/.settings/org.eclipse.buildship.core.prefs    2021-09-21 18:09:52 +0000
+++ new/udfs/udf.test.gen/.settings/org.eclipse.buildship.core.prefs    2025-01-20 15:14:32 +0000
@@ -1,2 +1,13 @@
+arguments=--init-script /home/pbb/.config/Code/User/globalStorage/redhat.java/1.38.0/config_linux/org.eclipse.osgi/58/0/.cp/gradle/init/init.gradle --init-script /home/pbb/.config/Code/User/globalStorage/redhat.java/1.38.0/config_linux/org.eclipse.osgi/58/0/.cp/gradle/protobuf/init.gradle
+auto.sync=false
+build.scans.enabled=false
+connection.gradle.distribution=GRADLE_DISTRIBUTION(WRAPPER)
 connection.project.dir=
 eclipse.preferences.version=1
+gradle.user.home=
+java.home=/usr/lib/jvm/java-17-openjdk-amd64
+jvm.arguments=
+offline.mode=false
+override.workspace.settings=true
+show.console.view=true
+show.executions.view=true

=== modified file 'udfs/udf.test.gen/.settings/org.eclipse.jdt.core.prefs'
--- old/udfs/udf.test.gen/.settings/org.eclipse.jdt.core.prefs    2021-11-15 19:20:05 +0000
+++ new/udfs/udf.test.gen/.settings/org.eclipse.jdt.core.prefs    2025-01-20 15:14:32 +0000
@@ -1,4 +1,7 @@
 eclipse.preferences.version=1
+org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.8
+org.eclipse.jdt.core.compiler.compliance=1.8
+org.eclipse.jdt.core.compiler.source=1.8
 org.eclipse.jdt.core.formatter.align_assignment_statements_on_columns=false
 org.eclipse.jdt.core.formatter.align_fields_grouping_blank_lines=2147483647
 org.eclipse.jdt.core.formatter.align_type_members_on_columns=false

=== modified file 'words/java/data-gen/tests/.project'
--- old/words/java/data-gen/tests/.project    2021-02-25 08:08:41 +0000
+++ new/words/java/data-gen/tests/.project    2025-01-20 15:14:32 +0000
@@ -14,4 +14,15 @@
     <natures>
         <nature>org.eclipse.jdt.core.javanature</nature>
     </natures>
+    <filteredResources>
+        <filter>
+            <id>1737039660994</id>
+            <name></name>
+            <type>30</type>
+            <matcher>
+                <id>org.eclipse.core.resources.regexFilterMatcher</id>
+                <arguments>node_modules|\.git|__CREATED_BY_JAVA_LANGUAGE_SERVER__</arguments>
+            </matcher>
+        </filter>
+    </filteredResources>
 </projectDescription>

• I think convmap.cp is quite standard so FWD should support all things from that. Most of the tested codepages seem to be utf-8 and iso8859-1. Can we have some test to approve all cp to be supported are actually supported?
• Please test how this statement integrates with other statements like stated: COPY-LOB, OVERLAY and SUBSTRING. Take the extra mile and check GET-CODEPAGES, GET-CODEPAGE, IS-CODEPAGE-FIXED.
• check that the codepage is case-insensitive.

I have included tests for all the possible codepage values and even for the ones not yet supported by FWD.
As far as those tests go, there was only one specific codepage I found that it cannot be used for LONGCHAR (or CLOB) variables and that is "GB18030" . I modified the FWD's code so that the behavior will be the same as in PROGRESS where an error gets thrown if trying to assign this encoding to a LONGCHAR variable. For the rest of the supported encodings, all the tests passed.

Next, I've added a few tests to see how the FIX-CODEPAGE statement integrates with COPY-LOB, OVERLAY, SUBSTRING statements and the IS-CODEPAGE-FIXED function. I have to mention that these tests are not comprehensive as I only wanted to minimally test how those statements combine together and if one correctly affects the other. To be 100% sure that those statements behave the same as in progress, a lot more tests need to be written but I assume that those are not included in the scope of this task.
As far as those tests go, I haven't found any problem and everything seems to be working smoothly.

I also added some tests to check if the FIX-CODEPAGE is case-insensitive in FWD. From what I found it looks like this works solely based on the fact that the GET-CODEPAGE returns a CHARACTER type and in progress a variable containing "abc" is equal another variable containing "ABC".

• UTF-16
• UTF-16LE
• UTF-16BE
• UTF-32
• UTF-32LE
• UTF-32BE

For variables with this encodings, the statements return wrong results. For UTF-32BE I even encountered a case where PROGRESS would simply refuse to display it on a screen for no apparent reason.

One small thing to note is that there is a weird behavior I noticed in PROGRESS with the following encodings for the OVERLAY and SUBSTRING statements:

This is out of scope.

• I am OK with the changes.
• Please add a history entry and change the copyright year accordingly.

Please merge 9445a now.

Branch 9445a was merged into trunk as rev. 16342 and archived.