FWD » Core Development » Conversion Tools

Filesets are not finding files when case-insensitivity is true and -Z is used. This behaves in the same manner as file-cvt-list.txt when -F is used, although we aren't too concerned with that, since it is deprecated.

I have been debugging, and when FileListFactory.processFileSet creates a new ExplicitFileList reference = new ExplicitFileList(caseSens);, which is step 1 for the filelist, the FileList comparator is created, but subsequently is never used. It looks like it's is supposed to be used, to allow for ignoring case when filenames are compared. We never build a FileList because the first file listed in my zset_file.txt doesn't match in case:

            ExplicitFileList l1 = new ExplicitFileList(new String[] { path });
            String[] filenames = l1.listFilenames();

   protected void listImpl(List<File> results)
   {
      // separate file and directory results
      for (String filename : filenames)
      {
         File next = new File(filename);

         if (next.exists() && next.isFile())
         {
            // add the file to the results list
            results.add(next);
         }
      }
   }

Somehow the compare method should be used to determine if the file exists in a case-insensitive manner.

Below is a recreate:

On Ubuntu, create ./abl/test.p (just message "Hello world!" and lowercase filename) and zfile_set.txt containing:

F ./abl/Test.p

      <parameter name="opsys"            value="WIN32" />  
      <parameter name="winsys"           value="MS-WINDOWS" /> 
      <parameter name="case-sensitive"   value="false" />

I committed 9917a/15946 to address #9917-7. I will do a few more tests.

I tested and didn't find a scenario where this breaks. The customer application I'm working on uses uses the default value (false), so I'll just convert that app.

ETF and ChUI projects use case-sensitive = true in p2j.cfg.xml.

I ran a conversion for a customer application that uses case-sensitive = false and it failed.

• 7156e+9917a conversion only finds a class named (not the actual name): ClassnameTestTest.java
• Baseline conversion should the following two classes in the same folder: ClassnameTestTest.java and ClassnameTesttest.java

The reproduction involves frames:

form "test1" with frame testform.
form "test2" with frame test-form.

compile:
    [javac] Compiling 33 source files to /home/ddf/gcd/branches/hotel_gui/build/classes
    [javac] Compiling 3 source files to /home/ddf/gcd/branches/hotel_gui/build/classes
    [javac] /home/ddf/gcd/branches/hotel_gui/src/com/goldencode/hotel/Start.java:18: error: cannot find symbol
    [javac]    StartTestForm testFormFrame = GenericFrame.createFrame(StartTestForm.class, "test-form");
    [javac]    ^
    [javac]   symbol:   class StartTestForm
    [javac]   location: class Start
    [javac] /home/ddf/gcd/branches/hotel_gui/src/com/goldencode/hotel/Start.java:18: error: cannot find symbol
    [javac]    StartTestForm testFormFrame = GenericFrame.createFrame(StartTestForm.class, "test-form");
    [javac]                                                           ^
    [javac]   symbol:   class StartTestForm
    [javac]   location: class Start
    [javac] 2 errors

BUILD FAILED
/home/ddf/gcd/branches/hotel_gui/build.xml:477: Compile failed; see the compiler error output for details.

Currently investigating.

I think the naming convention for frames should be changed. When handling a frame with the name test-form, the name will be TestForm (uppercasing the first letter and the letter after -), while testform will be a simple Testform (uppercasing the first letter).

=== modified file 'src/com/goldencode/p2j/convert/ConversionDriver.java'
--- old/src/com/goldencode/p2j/convert/ConversionDriver.java    2025-01-02 14:26:27 +0000
+++ new/src/com/goldencode/p2j/convert/ConversionDriver.java    2025-06-04 13:08:51 +0000
@@ -1095,7 +1095,7 @@
       oldjasts.addAll(Arrays.asList(menus));

       // save the results
-      jasts = new ExplicitFileList(oldjasts.toArray(new String[0]));
+      jasts = new ExplicitFileList(oldjasts.toArray(new String[0]), true);
    }

    /**

This fixes the issue, but I've just found an issue.

1. zfile_set.txt contains F .abl/Start.p, then it is converted correctly.
2. zfile_set.txt contains F .abl/start.p, then it converts ./abl/Start.p. I'd expect the right start.p to be selected here. What do you think Greg?

There are also 2 ./abl/start.p.jast entries in oldjasts when calling convertSourceNamesToJavaAstNames() in this case, this is because the source is using an ExplicitFileList with caseSens=false. I think that listImpl() should actually operate on a Set<File> instead of a List<File>.

Dănuț Filimon wrote:

[...]

This fixes the issue, but I've just found an issue.

Scenarios:

1. zfile_set.txt contains F .abl/Start.p, then it is converted correctly.
2. zfile_set.txt contains F .abl/start.p, then it converts ./abl/Start.p. I'd expect the right start.p to be selected here. What do you think Greg?

There are also 2 ./abl/start.p.jast entries in oldjasts when calling convertSourceNamesToJavaAstNames() in this case, this is because the source is using an ExplicitFileList with caseSens=false. I think that listImpl() should actually operate on a Set<File> instead of a List<File>.

That is an impossible situation on a system such as Windows (having both start.p and Start.p). It really shouldn't happen when a project is handed over to us, but if it did, I don't think there's anything wrong with taking one that matches case-sensitively, otherwise just the first one if there isn't a case-sensitive match.

I'd expect the right start.p to be selected here. What do you think Greg?

Yes. On a case-insensitive approach, only one can be present in the list. The first found should "win". Any other one should be ignored.

I think that listImpl() should actually operate on a Set<File> instead of a List<File>.

Agreed

I committed 9917a/15947. This only contains changes for the frame and menu issue, the idea is that in convertSourceNamesToJavaAstNames() we've already gotten the sources from a FileList that uses the configured case sensitive value, but frames and menus are not subjected to this property when being created. In this case, the ExplicitFileList should set caseSens to true.

As for the Set usage for listImpl, it comes with issues when sorting. I am not sure if using a TreeSet and removing the sortFiles entirely is ok because we have ExplicitFileList.sort being used in a few cases.

I rebased 9917a to latest trunk/15967, the branch is now at revision 15970.

I plan to commit the change for listImpl() today.

I will be running a conversion today with the same customer application that failed previously.

Dănuț Filimon wrote:

I will be running a conversion today with the same customer application that failed previously.

Conversion was successful, I will need a review before I proceed with other tests.

Code Review Task Branch 9917a Revisions 15969 through 15971

Overall, the changes seem reasonable.

My only question is why we hard code the case-sensitive flag to true in use of the ExplicitFileList constructor in ConversionDriver.convertSourceNamesToJavaAstNames(). That seems like it may bring unwanted behavior on systems with case-insensitive filessystems. Shouldn't that be driven by our configuration in p2j.cf.xml? If that isn't appropriate, then we probably need to fix the frame/menu issue in a different way.

Greg Shah wrote:

Code Review Task Branch 9917a Revisions 15969 through 15971

Overall, the changes seem reasonable.

My only question is why we hard code the case-sensitive flag to true in use of the ExplicitFileList constructor in ConversionDriver.convertSourceNamesToJavaAstNames(). That seems like it may bring unwanted behavior on systems with case-insensitive filessystems. Shouldn't that be driven by our configuration in p2j.cf.xml? If that isn't appropriate, then we probably need to fix the frame/menu issue in a different way.

The issue is that the already scanned in front(), while the ExplicitFileList in this case is created in back(). This means that the sources we have should not be affected. The problem is that we can't create an ExplicitFileList for frame and menu names that is case sensitive, but not case sensitive for the sources. For your information, this is no longer a problem in #7180 because the ArtifactCollection instances are merged as is and ExplicitFileList is not used.

There's a single customer application left to test to get 9917a into the merge queue and tomorrow I expect to get the conversion done.

The customer application converted, but the build failed. Some temp-table dmos were missing and might be caused by something else, so a new baseline will have to be converted because I used a newer trunk revision.

--- deleted, the comment was for another issue

Dănuț Filimon wrote:

The customer application converted, but the build failed. Some temp-table dmos were missing and might be caused by something else, so a new baseline will have to be converted because I used a newer trunk revision.

The conversion failed again. I thought that the problem is two temp-tables (tt-example and ttexample) being defined, but the problem looks to be something else. Andrei is currently running the conversion baseline for the application.

• 9917a
  

       [java] ./app/dmo/_temp/Ttexample_1.java
       [java] ./app/dmo/_temp/TtSingleTable_1.java
  ...
       [java] ./app/dmo/_temp/TtTest_3.java
       [java] ./app/dmo/_temp/TtDemo_1.java
  ...
       [java] ./app/dmo/_temp/TtOther_2.java
       [java] ./app/dmo/_temp/TmpCheck_3_1.java
  ...
       [java] ./app/dmo/_temp/TtProblem_1.java
       [java] ./app/dmo/_temp/TtProblem_1_1.java
       [java] ./app/dmo/_temp/TmpNormal_1.java
       [java] ./app/dmo/_temp/TmpSimple_1.java
  

• trunk
  

       [java] ./app/dmo/_temp/Ttexample_1.java
       [java] ./app/dmo/_temp/Ttexample_1_1.java
  ...
       [java] ./app/dmo/_temp/TtTest_3.java
       [java] ./app/dmo/_temp/TtTest_3_1.java
  ...
       [java] ./app/dmo/_temp/TtOther_2.java
       [java] ./app/dmo/_temp/TtOther_2_1.java
  ...
       [java] ./app/dmo/_temp/TtProblem_1.java
       [java] ./app/dmo/_temp/TtProblem_1_1.java
       [java] ./app/dmo/_temp/Ttproblem_2.java
       [java] ./app/dmo/_temp/Ttproblem_2_1.java
  

• ./app/dmo/_temp/TtDemo_1.java is replaced by [java] ./app/dmo/_temp/TtTest_3_1.java in trunk, but when checking the 9917a log, the ./app/dmo/_temp/TtDemo_1.java appears two times which is wrong.
• Main problem lies in

       [java] ./app/dmo/_temp/TtProblem_1.java
       [java] ./app/dmo/_temp/TtProblem_1_1.java
       [java] ./app/dmo/_temp/Ttproblem_2.java
       [java] ./app/dmo/_temp/Ttproblem_2_1.java
  

  where the same table appears.

I wonder if Octavian's work in #3211 is related?

Greg Shah wrote:

I wonder if Octavian's work in #3211 is related?

I'll discuss with Octavian. My changes make this problem a lot more visible, so if I can get some specific debug points/test case from #3211, I might be able to reproduce this without converting the customer application.

• TtExample_2.java and TtExample2_1.java
• Ttexample_1.java and Ttexample1_1.java

I will revert the Set changes and see if the problem persists. If this is the cause, I will create a separate issue to work on replacing the List with Set.

The regression test plan passed with 9917a, the branch can be merged.

You can control the merge.

I prefer to defer the merging after customer deliveries happening tomorrow evening.

Alexandru Lungu wrote:

Danut, please merge 9917a to trunk after 10020a. There won't be any delivery this week - the next scheduled delivery is next week.

Danut, you can merge now.