FWD » Deployment

I've posted some thoughts in #9597-101 related to factoring the images and scripts.

Greg Shah wrote:

1. Build FWD

• This would include enough bootstrapping of Java, Ubuntu, gcc, bzr and so forth to be able to checkout FWD and build a full binary distribution of it, which would be stored on the host filesystem in a bind mount. It should have a single command line to checkout and build and it should require nothing on the host except enough disk space to operate and leave the results in the current directory.
• Optionally, one should be able to get a shell in the container and "manually work with" the FWD source tree and build process.
• It has no FWD installation built in so it is different from the Development case below.
• This should also be a very thin image.
• I expect this to be used on build systems that take FWD source distributions and create a binary archive as the output. That is all this image should do.

There are 2 "base" images that support this requirement.

The base_ubuntu images have user ID "fwd" as the primary user. If you map your local setup into the container, you should have all the proper rights. To illustrate, these were run on cust002:

rfb@cust002:~$ docker run --name jdk8_container -h fwddev1-jdk8 --rm -it -v ${HOME}:${HOME} -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -u 1000:1000 -e HOME=${HOME} -w ${HOME} goldencode/base_ubuntu_22.04:latest sh -c "whoami" 
ges
rfb@cust002:~$ docker run --name jdk8_container -h fwddev1-jdk8 --rm -it -v ${HOME}:${HOME} -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -u 1001:1001 -e HOME=${HOME} -w ${HOME} goldencode/base_ubuntu_22.04:latest sh -c "whoami" 
rfb

Much of the requirements are really a matter of how you start a container using the image. The most basic way to give full access to your homespace would be:

docker run --name jdk8_container -h fwddev1-jdk8 --rm -it -v ${HOME}:${HOME} -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -u $(id -u):$(id -g) -e HOME=${HOME} -w ${HOME} goldencode/base_ubuntu_22.04:latest bash

docker run --name jdk8_container -h fwddev1-jdk8 --rm -it -v ${HOME}:${HOME} -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -u $(id -u):$(id -g) -e HOME=${HOME} -w ${HOME}/projects/fwd goldencode/base_ubuntu_22.04:latest sh -c "bzr co ~/secure/code/p2j_repo/p2j/trunk" 

2. Build 4GL Application

• The idea here is a slim image that has enough of a FWD installation to automate the project conversion, running reports and building a binary version of the converted code.
• This is NOT a full developer's workstation (see below).
• This has no runtime elements. You cannot run FWD server/client/database on this.
• Builds on an image that provides a binary FWD installation for conversion.
• Use Cases
  • Conversion and Conversion Regression Testing
  • Analytics
  • Build/Archive of the Converted Code

There are 2 "fwd" images that support this requirement. The version of FWD within the image is indicated by the tagging that takes place. Take a look at fwd_4.0_ubuntu_22.04 repository on Docker Hub at them. Each of the base images has FWD added to them, linked at /opt/fwd for conversion and runtime.

Again, the requirements are met simply by starting a container (bash):

docker run --name jdk8_container -h fwddev1-jdk8 --rm -it -v ${HOME}:${HOME} -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -u $(id -u):$(id -g) -e HOME=${HOME} -w ${HOME} goldencode/fwd_4.0_ubuntu_22.04:latest bash

docker run --name jdk8_container -h fwddev1-jdk8 --rm -it -v ${HOME}:${HOME} -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -u $(id -u):$(id -g) -e HOME=${HOME} -w ${HOME} goldencode/fwd_4.0_ubuntu_22.04:latest sh -c "echo $FWD_LIB; java -jar $FWD_LIB/build/lib/p2j.jar" 
/opt/fwd
FWD v4.0.0_p2j_trunk_15770

3. Development

• Think of this as a fully working developer's workstation. The only thing missing is a GUI desktop and IDE. But a developer using this should be able to connect their IDE to this container to do the full range of things developmers must do.
• Use cases:
  • Conversion and Conversion Regression Testing
  • Analytics
  • Build/Archive of the Converted Code
  • FWD Development (FWD source code is present, helps with source debugging of converted 4GL code AND allows changes to FWD source code on the fly which should be buildable)
  • 4GL Development (interactive and iterative direct development of a 4GL application, including edits to code/schemata and runs of conversion/builds)
  • Debugging and Testing of the converted results
  • Running of ABLUnit and the Harness.
• This will have the full dev toolchain AND runtime support for the server/client/database.
• A key point here is that it doesn't build upon an image with FWD in binary-only form. The development process naturally needs a source installation of FWD.
• This is the "heaviest" of the images.
• There may be multiple of these based on different databases, or maybe just one monster with multiple databases.

Here's where you are getting into something that doesn't make sense. #2 is a development environment. If you can build a 4GL application, isn't that development? When you start the container just opening appropriate with your docker run command so that you can attach to the container. The _pg14 images contain PostgreSQL so you can have a database. I might need to be schooled on this. I believe #2 and #3 are the same thing.

4. Runtime

• Provides a production deployment of FWD without the development baggage.
• Will be multiple images but designed for security, supportability, reduced size.
• There will be options for a single container approach (server/clients/database in one container). This would mean different images for different databases.
• There must be a tiered (a.k.a. complex) container approach (server in one container, clients in another, database in yet another).
• In the tiered case, the database containers would be db-specific (different ones for PostgreSQL, MariaDB, SQLServer...) while the server and client containers can be simpler.
• No source code from the application.
• No source code from FWD. It should build on a
• No extraneous crap that increases the "surface area" for security vulnerabilities.
• Tools for runtime support/diagnosis/monitoring.

At the current time, the fwd image contains the base_ubuntu image. The converted application is really up to the customer. They can gorp it up however they see fit. If we take the Hotel samples as a bases, when you build hotel, you can build a client image, a server image, and a database image. The things in the base image are broken down as:

• build-essential
• apt-utils
• wget
• curl
• dos2unix
• gcc
• jq
• locales
• sudo
• unzip
• vim
• zip
• bzr
• python3-paramiko
• openssh-server
• openssh-client
• gosu
• iproute2
• iputils-ping
• dnsutils
• iproute2
• rsync
• less
• p7zip-full

• openjdk-8-jdk
• openjdk-8-source
• openjdk-17-jdk
• openjdk-17-source

• dpkg-dev
• libffi-dev
• libncurses5-dev
• ncurses-term
• ant
• libpam0g-dev
• libc-bin

So this is a case-by-case requirement. Suffice it to say, the fwd_4.0_ubuntu_22.04 image can be stripped of whatever when the application's image is built. It does not have any source code from either FWD or hotel. Take a look at the 3 varieties of Docker images:

docker run -it --rm goldencode/hotel_chui_client:latest bash
docker run -it --rm goldencode/hotel_chui_server:latest bash
docker run -it --rm goldencode/hotel_chui_pg14_server:latest bash

The most glaring thing (besides stripping out some of the development tools in the list) would be the fact that I place all varieties of FWD runtime into the image:

fwd@10ea284d76c6:/opt/hotel$ ls /opt/fwd-deploy -l
total 56
drwxr-xr-x 1 fwd fwd 4096 Sep 18 08:05 admin
drwxr-xr-x 1 fwd fwd 4096 Sep 18 08:05 client
drwxr-xr-x 1 fwd fwd 4096 Sep 18 08:05 convert
drwxr-xr-x 1 fwd fwd 4096 Sep 18 08:05 docs
drwxr-xr-x 1 fwd fwd 4096 Sep 18 08:05 native
drwxr-xr-x 1 fwd fwd 4096 Sep 18 08:05 server
drwxr-xr-x 1 fwd fwd 4096 Sep 18 08:05 spawner

They are there only because of #6671 and the need to understand better what goes into a client image versus a server image. This is also tied to the multi-container task as well.

1. Build FWD

• This would include enough bootstrapping of Java, Ubuntu, gcc, bzr and so forth to be able to checkout FWD and build a full binary distribution of it, which would be stored on the host filesystem in a bind mount. It should have a single command line to checkout and build and it should require nothing on the host except enough disk space to operate and leave the results in the current directory.
• Optionally, one should be able to get a shell in the container and "manually work with" the FWD source tree and build process.
• It has no FWD installation built in so it is different from the Development case below.
• This should also be a very thin image.
• I expect this to be used on build systems that take FWD source distributions and create a binary archive as the output. That is all this image should do.

There are 6 "base" images that support this requirement, although the pg14, postgres, and maria images are really a baseline OS used for building the runtime versions with either a separate engine (as in the case of postgres and maria) or built into the server's image (as in the case of pg14). The requirement necessitating their creation was the need for custom locales within the image.

I think you are focusing on closing the task rather than actually meeting the stated requirements. Now is the time to get this right. That is the ENTIRE POINT of this task.

Type 1 has only the purpose of building FWD. In what world does that mean larding the image with database code and God only knows what other drek? At most, we need 2 images: ubuntu + JDK 8 (for a little while longer) and ubuntu + JDK 17. In a few months, we probably only need 1 image.

Please go back and reimplement images to meet this requirement. Make them very thin. Remove all the drek. This is a "build FWD" image. That is all.

2. Build 4GL Application

• The idea here is a slim image that has enough of a FWD installation to automate the project conversion, running reports and building a binary version of the converted code.
• This is NOT a full developer's workstation (see below).
• This has no runtime elements. You cannot run FWD server/client/database on this.
• Builds on an image that provides a binary FWD installation for conversion.
• Use Cases
  • Conversion and Conversion Regression Testing
  • Analytics
  • Build/Archive of the Converted Code

There are 6 "fwd" images that support this requirement. The version of FWD within the image is indicated by the tagging that takes place. Take a look at fwd_4.0_ubuntu_22.04 repository on Docker Hub at them. Each of the base images has FWD added to them, linked at /opt/fwd for conversion and runtime.

Image	Size
fwd_4.0_ubuntu_22.04	5.38GB
fwd_4.0_ubuntu_22.04_jdk17	5.42GB
fwd_4.0_ubuntu_22.04_pg14	6.19GB
fwd_4.0_ubuntu_22.04_pg14_jdk17	6.23GB
fwd_4.0_postgres_14	2.89GB
fwd_4.0_mariadb_11	2.94GB

My whining for type 1 is the same for type 2. In 0 universes (assuming we live in a many universes multi-verse) do we need databases and other runtime crap in this image. Please follow the requirements EXACTLY.

It is to give an thin option for conversion and runnng reports. That is it. This is NOT a FULL DEVELOPMENT SYSTEM and there are NO runtime requirements. Again, we need 2 images max (JDK 8 and JDK 17).

3. Development

• Think of this as a fully working developer's workstation. The only thing missing is a GUI desktop and IDE. But a developer using this should be able to connect their IDE to this container to do the full range of things developmers must do.
• Use cases:
  • Conversion and Conversion Regression Testing
  • Analytics
  • Build/Archive of the Converted Code
  • FWD Development (FWD source code is present, helps with source debugging of converted 4GL code AND allows changes to FWD source code on the fly which should be buildable)
  • 4GL Development (interactive and iterative direct development of a 4GL application, including edits to code/schemata and runs of conversion/builds)
  • Debugging and Testing of the converted results
  • Running of ABLUnit and the Harness.
• This will have the full dev toolchain AND runtime support for the server/client/database.
• A key point here is that it doesn't build upon an image with FWD in binary-only form. The development process naturally needs a source installation of FWD.
• This is the "heaviest" of the images.
• There may be multiple of these based on different databases, or maybe just one monster with multiple databases.

Here's where you are getting into something that doesn't make sense. #2 is a development environment.

No, it isn't. It is a tool for running conversion.

If you can build a 4GL application, isn't that development?

No, type 2 is not the same thing as a full development workstation. Who cares if it is a task that is done during development?

When you start the container just opening appropriate with your docker run command so that you can attach to the container. The _pg14 images contain PostgreSQL so you can have a database. I might need to be schooled on this. I believe #2 and #3 are the same thing.

In what world do we need PostgreSQL for converting a 4GL application? It is RUNTIME ONLY. It would be needed for type 3 and type 4, but not for type 2 or type 1.

For type 4, you need to remove development gorp. Things like gcc are not needed at runtime.

The previous post was amended to indicate the actual images that more closely match up with Build FWD and Build 4GL. The now "unlisted" images are required for downstream runtime support. The will be renamed to include _dev in the image names.

What are the software elements that should be removed from the BASE_OS_TOOLS and RUNTIME_TOOLS for strictly runtime?

For the the development environment, what is missing from the software list? Also, if you are developing for a PostgreSQL environment, what is required for that?

What are the software elements that should be removed from the BASE_OS_TOOLS and RUNTIME_TOOLS for strictly runtime?

That is your job to determine. Our installation documentation for the client and server would probably be helpful.

For the the development environment, what is missing from the software list? Also, if you are developing for a PostgreSQL environment, what is required for that?

The standard workstation is a good starting point.

The current container project contains dbash.sh which is setup to use the basedev_ubuntu_24.04 image. That image already has a lot of development support within it. What is missing from the actual image? Do you propose to place FWD code, application code, testing code within the actual image? That would make them tied to a particular version and become outdated very quickly.

There is a lot to how you run it that determines what it can do. If you don't map your own userspace into the container, you will only have access to what is baked into the image. If you map certain directories into the container, you would have access. This would include the ability to checkout projects, build them, and run them. It may need some additional port openings so that tools running on the host (like Eclipse) can access the processes running in the container.

The current container project contains dbash.sh which is setup to use the basedev_ubuntu_24.04 image. That image already has a lot of development support within it. What is missing from the actual image? Do you propose to place FWD code, application code, testing code within the actual image? That would make them tied to a particular version and become outdated very quickly.

It isn't about what is in the container. The customer using this approach in #10149 has found the design to be painful. Common development tasks are fragile, cumbersome and/or non-persistent. We will fix this in #10727.