FWD » Build and Source Control

Currently our FWD build logic is split in two build files, Gradle and Ant. Gradle manages the dependencies and drivers the Ant tasks. Ant contains the core build logic. All the Ant logic must be ported and the whole build must be handled by Gradle.

Marcin has done some exploration on how to accomplish this migration.

Marcin: Please post a diff with any useful results you have. It would be best to build on top of branch 3821c which is effectively our "trunk" at the current time. Post any issues you see and we can discuss it.

At this point I was attempting to move Ant tasks into Gradle to learn the build process, get familiar with code, this sort of thing.

Successes: IDE (IntelliJ) correctly recognizes Java sources location and Java dependencies.

The fundamental issue: src directory contains not only Java sources, but also other files (Antlr grammars, Antlr generated code, UI graphical files, etc)
which leads to large number of manual task definitions (cleanup, copying, manual VSC configs, etc). It's more difficult to take advantage of tooling (Gradle, IDEs, etc)

I do not know how much wiggle room is allowed here.
For example, I'd probably start by moving ANTLR grammars to a separate dir, the same goes for UI elements.

Greg, if you want me to work on this issue - great - can we define the success criteria and smaller steps (versioning - probably no one wants to end up with huge merge) ?

In regard to project import for an IDE, we have had good results in Eclipse by importing from the ant build.xml. I have not tried from gradle, but since there is so much in the ant build, the import would have to traverse that anyway.

We have not planned to work on this task right now, but if you have some safe changes that move us ahead we can consider merging them into a branch.

At a minimum, I want to capture the list of issues to be resolved.

The fundamental issue: src directory contains not only Java sources, but also other files (Antlr grammars, Antlr generated code, UI graphical files, etc) which leads to large number of manual task definitions (cleanup, copying, manual VSC configs, etc). It's more difficult to take advantage of tooling (Gradle, IDEs, etc)

I do a fair amount of the ANTLR work in the project. I do not use Eclipse for that. Nor do I use Eclipse for any UI resources. So I admit that I have not considered this aspect of our project structure. On the other hand, I am somewhat resistent to the idea that we must separate our project into top level directories based on the type of resource. I prefer grouping things by meaning/purpose instead of source language. When the same people work on related code, the source language doesn't matter too much IMO. This requirement is something I dislike about the "standard" maven project structure.

Anyway, let's discuss it here to inform our future work.

If you do not plan any work on this task now, I do not have anything safe to deploy. It was exploratory.

The questions of build scripts and sources layout are rather philosophical, not sure if you are interested in discussing them here ?

The questions of build scripts and sources layout are rather philosophical, not sure if you are interested in discussing them here ?

The advantages of moving to a standard Maven project structure:

• People are familiar with it.
• Some tools may depend upon it. Maven does, though we have no interest or plans to use Maven since we use Gradle for dependency processing.

Disadvantages:

• It is extra work for little benefit.
• Some people (i.e. me) consider the Maven standard project structure to be poorly designed.

Unless I've missed something important, we can probably leave the discussion there. :)

I agree with Greg that if you don't work in an IDE, having the standard, more complex and deeper (Maven, Gradle, etc.) directory structure is a burden. In an IDE you typically don't traverse directories as this is either automated or hidden to you. Also the majority of the code is kept in a single project, which also doesn't require the more complex directory structures.

Tooling is familiar with standard structure.

Take automated tests - the standard Maven way really boils down to "add Surefire plugin, add JUnit dependency" - boom, you're done. Maven knows what to do, IDE knows what to do - hey, you can get even code coverage report with a click of a button. CI/CD is happy.

Automation - each manual task is error prone. People are horrible at remembering small menial things. I know I am horrible at them ;-)
Take VCS - with "standard" approach - "Hey, VCS - just take care of the src directory and don't touch target, thank you". Pretty much done.
Take JAR builds - again - std tools just know "compile java srcs, copy resources, maybe add deps"

I'd say these are powerful tools at developer's hands, but I have to agree - it would be a major effort.

Hynek Cihlar wrote:

I agree with Greg that if you don't work in an IDE, having the standard, more complex and deeper (Maven, Gradle, etc.) directory structure is a burden. In an IDE you typically don't traverse directories as this is either automated or hidden to you. Also the majority of the code is kept in a single project, which also doesn't require the more complex directory structures.

Right, well, if you don't use IDE this might be the case.

Marcin Jaskolski wrote:

Tooling is familiar with standard structure.

Take automated tests - the standard Maven way really boils down to "add Surefire plugin, add JUnit dependency" - boom, you're done. Maven knows what to do, IDE knows what to do - hey, you can get even code coverage report with a click of a button. CI/CD is happy.

Automation - each manual task is error prone. People are horrible at remembering small menial things. I know I am horrible at them ;-)
Take VCS - with "standard" approach - "Hey, VCS - just take care of the src directory and don't touch target, thank you". Pretty much done.
Take JAR builds - again - std tools just know "compile java srcs, copy resources, maybe add deps"

I'd say these are powerful tools at developer's hands, but I have to agree - it would be a major effort.

To be fair all the benefits you mention above are one-time only. While setting them up in non-standard project layout does take time, it is only the matter of making it work, once. On the other hand interacting with the project on daily basis is a continuous kind of effort. Thus having a project layout that is easy to work with for all the members of the team has a higher priority IMO.

Btw I use IntelliJ IDEA myself and I also used to be a happy user of Eclipse.

Thanks for the thoughts, Guys. I agree that the project has a ways to go to integrate nicely into an IDE environment. We'll have to consider the improvements that can be made when we get back to this task.

OK, I see. What about the unit tests ? Shouldn't we integrate them into the build ?

Our unit tests are limited. We intend to add more, but the Testing Process for 4GL support is really more important and is getting our investment of time. Regardless, we intend to implement CI/CD for running these nightly. For our main testing, there are too many tests and too much conversion/setup to run integrated with the build.

I'm extending the scope of the issue to also add the standard Gradle Java plugin in order to allow the build to be used in some todays popular IDEs like VS Code.

Without the Java plugin VS Code will not be able to build the project (or at least I couldn't find a way around it).

Note that Java plugin injects its own build targets. Some of them conflict with existing targets, like jar. The idea is to unify and standardize the build targets. I.e. to use the standard Java plugin targets for compiling and building the project.

Hynek Cihlar wrote:

Paula, I didn't test ANTLR 4, can you please do that?

Sure!

All ANTLR4 grammars are correctly generated with 4645a, including the schema parser.
EDIT: demo grammars :)

Hynek Cihlar wrote:

The Gradle-only build is now checked in 4645a. This removes the hybrid Gradle + Ant approach. Besides, it also addresses the issues we've had with incremental building due to the ASM version conflicts.

I spent a good deal of testing on it and I also matched all the build targets against the original build and achieved 1:1 match. There are some small technical differences but these are benign.

This improvement brings the following benefits:

• decreased technical dept
• incremental builds
• with some overhead gone and with some of the optimizations Gradle can now utilize the build times decreased a bit
• FWD can now be deployed with standard dev tools like VS Code (with VS Code in particular this means zero config: cd <fwd root> && code .. It will just pick up the Gradle build and thanks to the Java plugin it will properly feed the language server assuming the Java and Gradle plugins are installed).

Tomasz, please put your critical eye on the changes and do a review.

Eugenie, I didn't test Windows build, can you please give it a good shake?

OK. It's time to update my Windows VM to use Java 17.

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

The Gradle-only build is now checked in 4645a. This removes the hybrid Gradle + Ant approach. Besides, it also addresses the issues we've had with incremental building due to the ASM version conflicts.

I spent a good deal of testing on it and I also matched all the build targets against the original build and achieved 1:1 match. There are some small technical differences but these are benign.

This improvement brings the following benefits:

• decreased technical dept
• incremental builds
• with some overhead gone and with some of the optimizations Gradle can now utilize the build times decreased a bit
• FWD can now be deployed with standard dev tools like VS Code (with VS Code in particular this means zero config: cd <fwd root> && code .. It will just pick up the Gradle build and thanks to the Java plugin it will properly feed the language server assuming the Java and Gradle plugins are installed).

Tomasz, please put your critical eye on the changes and do a review.

Eugenie, I didn't test Windows build, can you please give it a good shake?

OK. It's time to update my Windows VM to use Java 17.

Unfortunately gradlew does not work anymore on Windows. Even with trunk:

...
An attempt to start the daemon took 0.625 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Unable to start the daemon process.
This problem might be caused by incorrect configuration of the daemon.
For example, an unrecognized jvm option is used.For more details on the daemon,
please refer to https://docs.gradle.org/9.2.1/userguide/gradle_daemon.html in the Gradle documentation.
Process command line: C:\jdk17\bin\java.exe --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED
--add-opens=java.base/java.time=ALL-UNNAMED -XX:MaxMetaspaceSize=384m -XX:+HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=windows-1252 -Duser.country=US -Duser.language=en -Duser.variant -cp C:\Users\Administrator\.gradle\wrapper\dists\gradle-9.2.1-bin\2t0n5ozlw9xmuyvbp7dnzaxug\gradle-9.2.1\lib\gradle-daemon-main-9.2.1.jar -javaagent:C:\Users\Administrator\.gradle\wrapper\dists\gradle-9.2.1-bin\2t0n5ozlw9xmuyvbp7dnzaxug\gradle-9.2.1\lib\agents\gradle-instrumentation-agent-9.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 9.2.1
Please read the following process output to find out more:
-----------------------

FAILURE: Build failed with an exception.

* What went wrong:
Could not initialize native services.

* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to generate a Build Scan (powered by Develocity).
> Get more help at https://help.gradle.org.

* Exception is:
org.gradle.internal.service.ServiceCreationException: Could not initialize native services.
        at org.gradle.internal.nativeintegration.services.NativeServices.initialize(NativeServices.java:290)
        at org.gradle.internal.nativeintegration.services.NativeServices.initializeOnDaemon(NativeServices.java:250)
        at org.gradle.launcher.daemon.bootstrap.DaemonMain.doAction(DaemonMain.java:114)
        at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:53)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:72)
        at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:39)
        at org.gradle.launcher.daemon.bootstrap.GradleDaemon.main(GradleDaemon.java:22)
Caused by: net.rubygrapefruit.platform.NativeException: Failed to initialise native integration.
        at org.gradle.fileevents.FileEvents.init(FileEvents.java:71)
        at org.gradle.internal.nativeintegration.services.NativeServices$NativeFeatures$1.initialize(NativeServices.java:118)
        at org.gradle.internal.nativeintegration.services.NativeServices.<init>(NativeServices.java:343)
        at org.gradle.internal.nativeintegration.services.NativeServices.initialize(NativeServices.java:288)
        ... 10 more
Caused by: java.lang.UnsatisfiedLinkError: C:\Users\Administrator\.gradle\native\0.2.8\x86_64-windows-gnu\gradle-fileevents.dll: The specified procedure could not be found
        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2394)
        at java.base/java.lang.Runtime.load0(Runtime.java:755)
        at java.base/java.lang.System.load(System.java:1970)
        at org.gradle.fileevents.FileEvents.init(FileEvents.java:55)
        ... 13 more...

If someone has an idea how to fix this please share. I'm in stuck. Used Java 17 from Oracle for this run.

Eugenie Lyzenko wrote:

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

The Gradle-only build is now checked in 4645a. This removes the hybrid Gradle + Ant approach. Besides, it also addresses the issues we've had with incremental building due to the ASM version conflicts.

I spent a good deal of testing on it and I also matched all the build targets against the original build and achieved 1:1 match. There are some small technical differences but these are benign.

This improvement brings the following benefits:

• decreased technical dept
• incremental builds
• with some overhead gone and with some of the optimizations Gradle can now utilize the build times decreased a bit
• FWD can now be deployed with standard dev tools like VS Code (with VS Code in particular this means zero config: cd <fwd root> && code .. It will just pick up the Gradle build and thanks to the Java plugin it will properly feed the language server assuming the Java and Gradle plugins are installed).

Tomasz, please put your critical eye on the changes and do a review.

Eugenie, I didn't test Windows build, can you please give it a good shake?

OK. It's time to update my Windows VM to use Java 17.

Unfortunately gradlew does not work anymore on Windows. Even with trunk:

[...]

If someone has an idea how to fix this please share. I'm in stuck. Used Java 17 from Oracle for this run.

Seems like I found the reason. New gradle is not compatible with my Server 2012 windows set up. Now trying on another one - Windows 10. The gradlew started but I can not download common-2.4.0.jar to finish config process. As soon as it is finished I get building results.

Did you ever see this file download issue?

Eugenie Lyzenko wrote:

Did you ever see this file download issue?

Ocasionally I saw download issues, but those just went away quickly on its own.

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Did you ever see this file download issue?

Ocasionally I saw download issues, but those just went away quickly on its own.

Yes. for Ubuntu I had the same case.

But for Window not. Does someone have completely downloaded .gradle subdirectory for Windows 10? If I could apply ready to use setup I can continue with FWD building.

Roger, do you recall any case where Ant tasks from build.xml would be called directly from any customer project or during a project setup? The ant task would have a pattern like ant-*.

Eugenie Lyzenko wrote:

But for Window not. Does someone have completely downloaded .gradle subdirectory for Windows 10? If I could apply ready to use setup I can continue with FWD building.

Please post your build output.

Hynek Cihlar wrote:

Roger, do you recall any case where Ant tasks from build.xml would be called directly from any customer project or during a project setup? The ant task would have a pattern like ant-*.

There's nothing like that in the Hotel projects, and none of the projects I am familiar with.

Eugenie Lyzenko wrote:

Here it is.

I can download the file directly just fine. Try to fetch it with your browswer or with wget.

Some projects may have install_spawner.sh (or .cmd) still dependent on ant jar or something like this.

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Here it is.

I can download the file directly just fine. Try to fetch it with your browswer or with wget.

Do you know what is the target directory on local system this file should be put in?

Constantin Asofiei wrote:

Some projects may have install_spawner.sh (or .cmd) still dependent on ant jar or something like this.

Good point and indeed. I can confirm Ant is used in some of the active projects in install_spawner.sh.

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Here it is.

I can download the file directly just fine. Try to fetch it with your browswer or with wget.

Do you know what is the target directory on local system this file should be put in?

See ~/.gradle/caches. On Windows probably C:\Users\<YourUsername>\.gradle\caches.

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Here it is.

I can download the file directly just fine. Try to fetch it with your browswer or with wget.

Do you know what is the target directory on local system this file should be put in?

See ~/.gradle/caches. On Windows probably C:\Users\<YourUsername>\.gradle\caches.

Yes, thank you! I already found it. Also copied some other files from working gradle dir on Ubuntu. The trunk was finally built.

Now running the build script for 4645a_16431.

Hynek Cihlar wrote:

Constantin Asofiei wrote:

Some projects may have install_spawner.sh (or .cmd) still dependent on ant jar or something like this.

Good point and indeed. I can confirm Ant is used in some of the active projects in install_spawner.sh.

FYI, any projects based off recent Hotel samples wouldn't use ant in the install_spawner.sh, so you could consider it deprecated. Instead it calls the postbuild.sh script directly.

Hynek Cihlar wrote:

Tomasz, please put your critical eye on the changes and do a review.

Code review revision 16431: There is nothing to put any critique on - love those changes. Maybe except missing file headers :)

And some not really important remarks:
I am not a fan of listing single files in copyResources and cleanGenerated, sooner or later someone will forget add/remove a file from there, but I guess that needs to stay that way as resources are taken from different places.
Did you tests if patching terminals really works ?
After importing project to Eclipse I got and i need to remove build/gen/src/java/ from build path manually.

Description    Resource    Path    Location    Type
The type SpiVersion is already defined    SpiVersion.java    /4645a/build/gen/src/java/com/goldencode/p2j    Unknown    Java Problem
The type Version is already defined    Version.java    /4645a/build/gen/src/java/com/goldencode/p2j    Unknown    Java Problem

@@ -687,7 +1628,7 @@
     // upgrade to org.apache.tika 3.0+ requires migration
     fwdClient (group: 'org.apache.tika', name: 'tika-core', version: '2.9.3') {
         exclude group: 'org.slf4j', module: 'slf4j-api'
-    }
+    }version

Roger Borrello wrote:

FYI, any projects based off recent Hotel samples wouldn't use ant in the install_spawner.sh, so you could consider it deprecated. Instead it calls the postbuild.sh script directly.

Thanks for the heads up! Which projects still rely on install_spawner.sh?

Tomasz Domin wrote:

Hynek Cihlar wrote:

Tomasz, please put your critical eye on the changes and do a review.

Code review revision 16431: There is nothing to put any critique on - love those changes. Maybe except missing file headers :)

And some not really important remarks:
I am not a fan of listing single files in copyResources and cleanGenerated, sooner or later someone will forget add/remove a file from there, but I guess that needs to stay that way as resources are taken from different places.

Yes, this is good point. I don't like having to define the individual files there either. I had to put some new in in order to achieve 1:1 build outputs. Certainly a point of improvement.

Did you tests if patching terminals really works ?

I ran the build tasks and compared the outputs, but I didn't do a deep inspection. May I ask you to check this?

After importing project to Eclipse I got and i need to remove build/gen/src/java/ from build path manually.

Are gen/src/java not being output by trunk build? If so this should be addressed.

Forcing ASM 9.8 to match Gradle's internal ASM version didnt solve incremental compiler error in sheet, did it ? I see you've left it disabled.

An oversight. I expect the incremental build should work also for sheet.

Is this a leftover version - here ?

Right, a typo.

Hynek Cihlar wrote:

Roger Borrello wrote:

FYI, any projects based off recent Hotel samples wouldn't use ant in the install_spawner.sh, so you could consider it deprecated. Instead it calls the postbuild.sh script directly.

Thanks for the heads up! Which projects still rely on install_spawner.sh?

It is still included with the projects, but the Dockerfile goes directly to postbuild.sh. All the more recent customer projects build off that model.

Eugenie Lyzenko wrote:

The 4645a building failed on Windows. Please see the attached logs.

This looks like the known max command line length issue. Likely the manifest-only jars with dependencies are not properly applied for Windows build.

Can you please rerun the build with --info and --stacktrace arguments?

Eugenie Lyzenko wrote:

Done. Log attached.

Please test with the latest 4645a.

Tomasz, would you have any suggestions how to address the build/gen/*.java conflict?

Hynek Cihlar wrote:

Tomasz, would you have any suggestions how to address the build/gen/*.java conflict?

An attempt to fix this is in the latest 4645a. Tomasz, please test.

Eugenie Lyzenko wrote:

Failed again. But on Javadoc stage. I guess : char cannot be inside path, or missed path separator.

Fixed in the latest 4645a. Please retest.

Tomasz Domin wrote:

Forcing ASM 9.8 to match Gradle's internal ASM version didnt solve incremental compiler error in sheet, did it ? I see you've left it disabled.

Disabling incremental build for ext/sheet removed in the latest 4645a.

Eugenie Lyzenko wrote:

Building successful.

There is a problem report generated. However I'm not sure how sever it is.

Please smoke test the built jar by converting and running a project with it. The report only shows the build warnings, we can ignore those and also happen in trunk.

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Building successful.

There is a problem report generated. However I'm not sure how sever it is.

Please smoke test the built jar by converting and running a project with it.

What project do you mean? Hotel GUI is OK?

Do you want the conversion and run to be done completely on Windows?

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Building successful.

There is a problem report generated. However I'm not sure how sever it is.

Please smoke test the built jar by converting and running a project with it.

What project do you mean? Hotel GUI is OK?

Do you want the conversion and run to be done completely on Windows?

Yes, Hotel GUI on Windows will be fine. Yes, please include conversion, too.

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Building successful.

There is a problem report generated. However I'm not sure how sever it is.

Please smoke test the built jar by converting and running a project with it.

What project do you mean? Hotel GUI is OK?

Do you want the conversion and run to be done completely on Windows?

Yes, Hotel GUI on Windows will be fine. Yes, please include conversion, too.

The conversion looks fine.

But runtime is problematic. Seems like the Hotel GUI is not compatible with Windows at this time. Too many changes required to be used with Java 17. Some new features miss Windows version in build.xml, ANT to be updated ... and so on.

I'm struggling to start using Converted Hotel GUI on Windows.

Is fwdspi.jar next to hotel.jar? Build may be placing it incorrectly on Windows for Java 17. Java 8 needed in in a ./spi directory.

26/02/20 12:28:01.463-0800 |  SEVERE | com.goldencode.p2j.persist.dialect.H2Helper | ThreadName:main, ThreadId:00000003, User:standard | Error setting FWD-specific collation;  please ensure 'fwdspi.jar' is installed in Java extension directory

Roger Borrello wrote:

Is fwdspi.jar next to hotel.jar? Build may be placing it incorrectly on Windows for Java 17. Java 8 needed in in a ./spi directory.
[...]

Yes, both hotel.jar and fwdspi.jar in same directory on Windows. I tried to add the fwdspi.jar to classpath explicitly with relative and absolute path, no effect.

Eugenie Lyzenko wrote:

I'm struggling to start using Converted Hotel GUI on Windows.

Is this problem already in trunk?

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Tomasz, would you have any suggestions how to address the build/gen/*.java conflict?

An attempt to fix this is in the latest 4645a. Tomasz, please test.

I confirm the issue is fixed.
But I have a question - the solution like that hides generated files from developer and we still have two copies of some source files (SpiVersion.java, Version.java).
And we have build/generated/ and build/gen

Why not creating a separate directory in project tree like generated and put all generated files in there ? (and add its contents to .bzrignore) and including it explicitly in source path ?

Tomasz Domin wrote:

But I have a question - the solution like that hides generated files from developer and we still have two copies of some source files (SpiVersion.java, Version.java).
And we have build/generated/ and build/gen

The thing is that Gradle connects task dependencies and output dirs. So you can't simply use a task's output without declaring depdnency on the task itself. I'm not sure if this was the same case why I needed a new gen output dir, but I will double check.

Why not creating a separate directory in project tree like generated and put all generated files in there ? (and add its contents to .bzrignore) and including it explicitly in source path ?

build is already part of .bzrignore, so it won't be checked in. Plus we still have the potential issue with Gradle checking the output dir referencs.

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

I'm struggling to start using Converted Hotel GUI on Windows.

Is this problem already in trunk?

Yes, the same issue.

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

I'm struggling to start using Converted Hotel GUI on Windows.

Is this problem already in trunk?

Yes, the same issue.

Hynek,

Do you know what collation is to be used for DB in hotel GUI application on Windows? Is it en_US@CP1252? Or en_US@iso88591_fwd_basic? Or other one.

Other question: How FWD looks for fwdspi.jar? Only in classpath?

The other question.

Can somebody clarify what is the Windows version the Hotel GUI setup scripts were designed/tested for? I currently have Windows 10 virtual system. May be I just need to have Windows 11 to make them work properly?

Eugenie Lyzenko wrote:

Hynek,

Do you know what collation is to be used for DB in hotel GUI application on Windows? Is it en_US@CP1252? Or en_US@iso88591_fwd_basic? Or other one.

I don't know what the intention was, but currently only en_US@iso88591_fwd_basic is used in the project.

Other question: How FWD looks for fwdspi.jar? Only in classpath?

Yes (Java extension dir was removed with Java 9), together with java.locale.providers Java property.

My regression tests for the Gradle build passed. I tested Hotel GUI, ran ChUI regressions, I smoke tested a large customer app and also compared the build outputs.

The branch is rebased against latest trunk and I believe it is ready to be merged. Costantin/Greg please approve the merge first.

Can somebody clarify what is the Windows version the Hotel GUI setup scripts were designed/tested for? I currently have Windows 10 virtual system. May be I just need to have Windows 11 to make them work properly?

Roger?

Eugenie: Are the Windows problems already present in trunk?

My regression tests for the Gradle build passed. I tested Hotel GUI, ran ChUI regressions, I smoke tested a large customer app and also compared the build outputs.

Is there anything that one needs to do to make cfg or script changes when this merges? In other words, is this a breaking change?

At a minimum, developers will need to recreate their projects I guess.

Greg Shah wrote:

Can somebody clarify what is the Windows version the Hotel GUI setup scripts were designed/tested for? I currently have Windows 10 virtual system. May be I just need to have Windows 11 to make them work properly?

Roger?

Eugenie: Are the Windows problems already present in trunk?

Yes, it works the same here in trunk too.

Eugenie Lyzenko wrote:

Greg Shah wrote:

Can somebody clarify what is the Windows version the Hotel GUI setup scripts were designed/tested for? I currently have Windows 10 virtual system. May be I just need to have Windows 11 to make them work properly?

Roger?

Eugenie: Are the Windows problems already present in trunk?

Yes, it works the same here in trunk too.

Please create a separate task for that work. Hotel GUI does need to be functional on Windows but we don't need to hold up this merge for the problem.

Greg Shah wrote:

Eugenie Lyzenko wrote:

Greg Shah wrote:

Can somebody clarify what is the Windows version the Hotel GUI setup scripts were designed/tested for? I currently have Windows 10 virtual system. May be I just need to have Windows 11 to make them work properly?

Roger?

Eugenie: Are the Windows problems already present in trunk?

Yes, it works the same here in trunk too.

Please create a separate task for that work. Hotel GUI does need to be functional on Windows but we don't need to hold up this merge for the problem.

OK. Under FWD -> Build and Source Control -> Support?

No, please use the Regression Testing project.

Greg Shah wrote:

No, please use the Regression Testing project.

OK. Support branches?

Greg Shah wrote:

My regression tests for the Gradle build passed. I tested Hotel GUI, ran ChUI regressions, I smoke tested a large customer app and also compared the build outputs.

Is there anything that one needs to do to make cfg or script changes when this merges? In other words, is this a breaking change?

I checked and asked, but I couldn't find such a case. Except some older projects being on hold currently.

At a minimum, developers will need to recreate their projects I guess.

They shouldn't unless they directly depend on the removed ant tasks, which is unlikely.

Eugenie Lyzenko wrote:

Greg Shah wrote:

No, please use the Regression Testing project.

OK. Support branches?

Yes

Greg Shah wrote:

Eugenie Lyzenko wrote:

Greg Shah wrote:

No, please use the Regression Testing project.

OK. Support branches?

Yes

Eugenie, relate it to this task, so it's easy to track.

Greg, do we have go for merge?

• the p2j.jar content: terminfo folder is missing, the others look OK
  

  Binary files ../p2j/1/com/goldencode/asm/package-info.class and 4645a/1/com/goldencode/asm/package-info.class differ
  Binary files ../p2j/1/com/goldencode/p2j/Version.class and 4645a/1/com/goldencode/p2j/Version.class differ
  diff -r ../p2j/1/META-INF/MANIFEST.MF 4645a/1/META-INF/MANIFEST.MF
  Only in ../p2j/1: terminfo
  diff -r ../p2j/1/udf/mariadb/udfs.sql 4645a/1/udf/mariadb/udfs.sql
  diff -r ../p2j/1/udf/postgresql/udfs.sql 4645a/1/udf/postgresql/udfs.sql
  diff -r ../p2j/1/udf/sqlserver/udfs.sql 4645a/1/udf/sqlserver/udfs.sql
  

• the content of fwdspi.jar and fwdaopltw.jar - I haven't checked these
• a jar version has been upgraded - was this by intent?
  

  Only in ../p2j/build/lib: jackson-dataformat-xml-2.17.1.jar
  Only in 4645a/build/lib: jackson-dataformat-xml-2.18.3.jar
  

Constantin Asofiei wrote:

Hynek, please compare with trunk/16442 (what 4645a has now):

• the p2j.jar content: terminfo folder is missing, the others look OK

Good catch, I will look at this.

[...]

• a jar version has been upgraded - was this by intent?

This was a latent issue resolved by the branch. In trunk while 2.18.3 is declared in the dependencies 2.17.1 is used.

I've ran ./gradlew all and I think there might be more missing from p2j.jar - check the material folder and 3pl:

$ diff -r ../p2j/1/ 4645a/1 | grep -Ei "^Only|^diff|^Binary" 
Binary files ../p2j/1/com/goldencode/asm/package-info.class and 4645a/1/com/goldencode/asm/package-info.class differ
Only in ../p2j/1/com/goldencode/expr: doc-files
Only in ../p2j/1/com/goldencode/p2j/net: doc-files
Only in ../p2j/1/com/goldencode/p2j/pattern: doc-files
Only in ../p2j/1/com/goldencode/p2j/preproc: doc-files
Only in ../p2j/1/com/goldencode/p2j/report/web/res: 3pl
Only in ../p2j/1/com/goldencode/p2j/schema: doc-files
Only in ../p2j/1/com/goldencode/p2j/uast: antlr4
Only in ../p2j/1/com/goldencode/p2j/uast: doc-files
Only in ../p2j/1/com/goldencode/p2j/ui/client/gui/theme: material
Binary files ../p2j/1/com/goldencode/p2j/Version.class and 4645a/1/com/goldencode/p2j/Version.class differ
diff -r ../p2j/1/META-INF/MANIFEST.MF 4645a/1/META-INF/MANIFEST.MF
Only in ../p2j/1/: terminfo
diff -r ../p2j/1/udf/mariadb/udfs.sql 4645a/1/udf/mariadb/udfs.sql
diff -r ../p2j/1/udf/postgresql/udfs.sql 4645a/1/udf/postgresql/udfs.sql
diff -r ../p2j/1/udf/sqlserver/udfs.sql 4645a/1/udf/sqlserver/udfs.sql

This is what I'm using to check diff -r ../p2j/build/ 4645a/build | grep -iE "^Only|^diff|^Binary"

The point is: we need to ensure we are not missing some resources from the .jar files.

Constantin Asofiei wrote:

I've ran ./gradlew all and I think there might be more missing from p2j.jar - check the material folder and 3pl:

Those should be empty dirs. I suppose you built with all?

Hynek Cihlar wrote:

Constantin Asofiei wrote:

Hynek, please compare with trunk/16442 (what 4645a has now):

• the p2j.jar content: terminfo folder is missing, the others look OK

Good catch, I will look at this.

Constantin, I don't see this. Or I should say I see it, terminfo with valid content is in the built jar. I tried gw clear all.

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Constantin Asofiei wrote:

Hynek, please compare with trunk/16442 (what 4645a has now):

• the p2j.jar content: terminfo folder is missing, the others look OK

Good catch, I will look at this.

Constantin, I don't see this. Or I should say I see it, terminfo with valid content is in the built jar. I tried gw clear all.

p2j.jar:terminfo/ folder is actually empty with trunk, didn't think to check its content. So is not an issue, sorry for false alarm.

3pl and material are empty, too. If the content for the other fwdspi and fwdaopltw jars is OK, then I'm good.

Constantin Asofiei wrote:

• the content of fwdspi.jar and fwdaopltw.jar - I haven't checked these

I checked them again, their files match.

So is good to merge. We need to ensure customers are aware that for FWD build ant is no longer used, and also any docker scripts do not use it - Roger, I think you are using gradle everywhere, right?

Constantin Asofiei wrote:

So is good to merge. We need to ensure customers are aware that for FWD build ant is no longer used, and also any docker scripts do not use it - Roger, I think you are using gradle everywhere, right?

Correct. I only direct users to use gradle. In fact, the tools that I built to build typically use ./gradlew all sheet:war archive and then use the archives for runtime distribution.

Greg Shah wrote:

Hynek Cihlar wrote:

Greg, do we have go for merge?

You have a GO.

Merging now.

Hynek Cihlar wrote:

4645a merged to trunk revision 16447 and archived.

Do we have to modify install_spawner.sh scripts for projects because we now have no build.xml inside FWD and ant command should be replaced with respective gradlew? Correct?

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

4645a merged to trunk revision 16447 and archived.

Do we have to modify install_spawner.sh scripts for projects because we now have no build.xml inside FWD and ant command should be replaced with respective gradlew? Correct?

Follow the example in Hotel GUI, as there isn't any use of ant in the install_spawner.sh. It calls postbuild.sh directly.

Roger Borrello wrote:

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

4645a merged to trunk revision 16447 and archived.

Do we have to modify install_spawner.sh scripts for projects because we now have no build.xml inside FWD and ant command should be replaced with respective gradlew? Correct?

Follow the example in Hotel GUI, as there isn't any use of ant in the install_spawner.sh. It calls postbuild.sh directly.

No. Don't "follow the example". Use that file verbetim.

The Hotel GUI install_spawner.sh should be copied into every project that needs spawning.