Building and Using Docker Images¶

Building and Using Docker Images
- Base Images
  - Under the Hood
  - Base Image Creation
- FWD Images
  - Under the Hood
    - Dockerfile details
  - FWD Image Creation
    - Examples

This document outlines how to create and use the FWD docker image in containers. You can start with the standard images available in the goldencode repository on Docker Hub for use in conversion and runtime of other projects.

The only configuration option for Docker that is required, aside from the defaults, is the buildkit feature should be enabled in the /etc/docker/daemon.json:

{
  "features": {
    "buildkit": true
  }
}

To get the container project:

bzr co sftp://<user>@xfer.goldencode.com/opt/fwd/container/
cd container/

Base Images¶

The term "base" is used in the context of all the subsequent images are based from these images. There are 2 base Ubuntu images created using the ubuntu:22.04 image as the base. One containing PostgreSQL 14 client-only components, and one containing the full PostgreSQL 14. The latter is intended for a more standalone approach without the requirement of a separate postgres image. These are used as a base for the images containing FWD with the branch and revision specified in the build process (with and without full PostgreSQL), discussed later.

The base_postgres_14 image is created using the postgres:14 image as the base and the base_mariadb_11 image is created using the maria:11 image as a base. They contains all the FWD custom locales that are currently included in the FWD project, as well as the unzip utility for ease of downstream database creation. They both expect initdb components to exist in the /docker-entrypoint-initdb.d directory (such as SQL and bash scripts). Postgres uses the custom locale specified via POSTGRES_INITDB_ARGS environment variable. For example: POSTGRES_INITDB_ARGS=--locale=en_US@iso88591_fwd_basic

These 6 "base" images are stored in the goldencode repository as:

REPOSITORY	TAG
goldencode/base_ubuntu_22.04	latest
goldencode/base_ubuntu_22.04_pg14	latest
goldencode/base_ubuntu_22.04_jdk17	latest
goldencode/base_ubuntu_22.04_pg14_jdk17	latest
goldencode/base_postgres_14	latest
goldencode/base_mariadb_11	latest

The steps for the creation of those images are outlined in the Base Image Creation section below.

Under the Hood¶

This section details what is contained within the images, describing highlights from the Dockerfiles.

base_ubuntu_22.04 and base_ubuntu_22.04_jdk17¶

The ubuntu:22.04 is the root image from which base_ubuntu_22.04 starts:

FROM ubuntu:22.04

Because of the complexities of some of the steps within the dockerfile, we specify bash as the shell to use for all commands:

SHELL ["/bin/bash", "-c"]

The build_base_ubuntu.sh script that builds the docker images allows certain build-arg key/values to be passed to the docker build. These are defined in the Dockerfile using ARG, which also allows the specification of a default value. These are:

ARG LANGUAGE_VALUE=en_US
ARG ENCODING_VALUE=UTF-8
ENV LANG=${LANGUAGE_VALUE}.${ENCODING_VALUE}
ENV LANGUAGE=${LANGUAGE_VALUE}

ARG TZDATA_AREA=America
ARG TZDATA_ZONE=New_York
ARG BASE_OS_TOOLS="build-essential apt-utils wget curl dos2unix gcc jq locales sudo unzip vim zip bzr python3-paramiko openssh-server openssh-client gosu iproute2 iputils-ping dnsutils iproute2 rsync less" 
ARG JDK_INSTALL="openjdk-8-jdk openjdk-8-source openjdk-17-jdk openjdk-17-source" 
ARG RUNTIME_TOOLS="dpkg-dev libffi-dev libncurses5-dev ncurses-term ant libpam0g-dev"

There are some Debian configuration items set so as to prevent manual intervention to confirm package installations, select options, or provide input. The debconf frontend is set to noninteractive mode (DEBIAN_FRONTEND=noninteractive), indicating that package configuration prompts should be automatically answered without user intervention during the execution of subsequent commands or scripts. DEBCONF_NOWARNINGS="yes" is so that commands aren't interrupted or prompted by non-fatal warning messages.

Base OS Tools
These base OS tools are installed by default using apt-get install:

build-essential
apt-utils
wget
curl
dos2unix
gcc
jq
locales
sudo
unzip
vim
zip
bzr
python3-paramiko
openssh-server
openssh-client
gosu
iproute2
iputils-ping
dnsutils
iproute2
rsync
less

They can be overridden using BASE_OS_TOOLS build-arg: --base_os_tools="..."

Language and Encoding
The localedef and language decoding are set to en_US.UTF-8 by default. These can be overridden using the LANGUAGE_VALUE and ENCODING_VALUE build-args: --language=en_GB --encoding=ISO-8859-1. We also include all the custom locales in the base image, so they are available to any subsequent images the build from the base image.

# Copy custom locale files to the image
COPY ./db/locale/* /usr/share/i18n/locales/

# Step 5/25
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    locales-all && \
    rm -rf /var/lib/apt/lists/*

Custom Locales
The custom locales are placed in the context directory base_ubuntu/postgres/locale by the build_base_ubuntu.sh script and are then positioned in the /usr/share/i18n/locales/ directory in the image. The locales-all package is installed, then the custom locales are built with this command (which is why bash is required):

# Build the custom locales
RUN cd /usr/share/i18n/charmaps/; \
    declare -A char_encoding_file; \
    char_encoding_file["cp1252"]="CP1252"; \
    char_encoding_file["iso885915"]="ISO-8859-15"; \
    char_encoding_file["iso88591"]="ISO-8859-1"; \
    for locale_file in /usr/share/i18n/locales/*_fwd_basic; do \
        cust_locale=$(basename ${locale_file}); \
        [[ $cust_locale =~ @([^_]*) ]] && char_encoding=${char_encoding_file["${BASH_REMATCH[1]}"]}; \
        chmod 644 -R /usr/share/i18n/locales/${cust_locale}; \
        ([[ -f "${char_encoding}.gz" ]] && gunzip -kf ${char_encoding}.gz); \
        ([[ ! -z "$char_encoding" ]] && echo ${cust_locale} ${char_encoding} >> /etc/locale.gen); \
    done; \
    locale-gen --no-archive

# needed to handle special characters in Java comments - https://proj.goldencode.com/issues/4127
RUN localedef --no-archive -f $ENCODING_VALUE -i $LANGUAGE_VALUE ${LANGUAGE_VALUE}.${ENCODING_VALUE}

/etc/ssh/sshd_config
While OpenSSH is installed, it is not set on by default. In addition, certain security settings are enforced in /etc/ssh/sshd_config as well as base configuration for optimization via sed edits:

Set AllowAgentForwarding to no
Set AllowTcpForwarding to no
Set X11Forwarding to no
ClientAliveInterval to 30
ClientAliveCountMax to 99999
MaxStartups to 100:100:100
Disabling AcceptEnv LANG LC_*

Timezone
Timezone data is installed and Areas/Zones is set to America/NewYork by default. These can be overridden using the TZDATA_AREA and TZDATA_ZONE build-args: --tzdata_area=Europe --tzdata_zone=London

RUN truncate -s0 /tmp/preseed.cfg && \
    (echo "tzdata tzdata/Areas select $TZDATA_AREA" >> /tmp/preseed.cfg) && \
    (echo "tzdata tzdata/Zones/${TZDATA_AREA} select $TZDATA_ZONE" >> /tmp/preseed.cfg) && \
    debconf-set-selections /tmp/preseed.cfg && \
    rm -f /etc/timezone /etc/localtime && \
    apt-get update && \
    DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true \
    apt-get install -y tzdata

Java
Java (openjdk-8-jdk openjdk-8-source openjdk-17-jdk openjdk-17-source) are installed by default using apt-get install. This can be overridden using the JDK_INSTALL build-arg: --jdk_install="openjdk-8-jdk openjdk-8-source" but both JDKs are available to choose as the default by setting up alternatives:

# Set up alternatives for JDK
# Step 12/34
RUN update-alternatives --install /usr/bin/java java /usr/lib/jvm/java-8-openjdk-amd64/bin/java 1081 && \
    update-alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk-amd64/bin/java 1082 && \
    update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/java-8-openjdk-amd64/bin/javac 1081 && \
    update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/java-17-openjdk-amd64/bin/javac 1082

The default is selected by use of the --build-arg JDK_VERSION=17 because the /usr/local/bin/switch-java.sh utilizes the environment variable to either set the default to JDK 17 if JDK_VERSION=17 or JDK 8 for any other value, including not being set. The JNI symbolic is created for /usr/lib/libjvm.so to the correct JDK version by this script, as well. In the future, this can be set to default to JDK 17.

Runtime Tools
These runtime OS tools are installed by default using apt-get install. They can be overridden using RUNTIME_TOOLS build-arg: --runtime_tools="..."

dpkg-dev
libffi-dev
libncurses5-dev
ncurses-term
ant
libpam0g-dev

NCurses and Terminfo Patching
The script used to perform ncurses and terminfo patching is positioned in the /tmp directory. The patching is then executed for the first time. The ZIP file can be overridden using FIX_NCURSES_FILE build-arg: --fix_ncurses_file=... and would need to be placed in the base_ubuntu context directory.

# Setup ncurses patching and perform it
COPY setup_ncurses6x.sh /tmp
RUN cd /tmp && /tmp/setup_ncurses6x.sh -sap /opt/ncurses
ENV NCURSES_FWD_STATIC=/opt/ncurses
RUN rm -fr /tmp/ncurses* /tmp/setup_ncurses6x.sh

# Setup terminfo patching and perform it
COPY 95patch-ncurses /root
COPY manage_term_patches.sh /root
COPY patch_terminfo.sh /root
RUN cd /root && /root/manage_term_patches.sh

Install the C Library
libc-bin is installed for runtime support using apt-get install.

PostgreSQL 14 Client
The postgresql-client package is installed to setup access to psql:

ENV PG_VERSION 14
RUN apt-get update && \
    locale-gen $LANG && update-locale LANG=$LANG && \
    apt-get install -y --no-install-recommends \
       postgresql-client && \
    apt-get clean  && \
    rm -rf /var/lib/apt/lists/*

# Update the PATH for PostgreSQL
RUN echo "export PATH=\"\$PATH:/usr/lib/postgresql/${PG_VERSION}/bin\"" > /etc/profile.d/02-pg${PG_VERSION}.sh && \
    chmod 644 /etc/profile.d/02-pg${PG_VERSION}.sh && \
    . /etc/profile.d/02-pg${PG_VERSION}.sh

Runtime setup of sshd

RUN mkdir /var/run/sshd

EXPOSE 22

Setup service account and groups

RUN groupadd -g ${FWD_GID} fwd && \
    useradd -r -m -u ${FWD_UID} -g ${FWD_GID} -G sudo -s /bin/bash -p $(openssl passwd fwd) fwd

The build script will create the base_ubuntu_22.04 image with JDK 8 as the default. It will also create base_ubuntu_22.04_jdk17 that will have JDK 17 as the default.

base_ubuntu_22.04_pg14 and base_ubuntu_22.04_pg14_jdk17¶

The base_ubuntu_22.04_pg14 comes from the base_ubuntu_22.04 image. Likewise, the The base_ubuntu_22.04_jdk17_pg14 comes from the base_ubuntu_22.04_jdk17 image, with the default JDK set to 17. The build utilizes a template Dockerfile, where the _jdk17 will be added to the FROM for the creation of the image with JDK 17, and removed with JDK 8.

FROM base_ubuntu_22.04@jdk_version@

The language is setup as en_US.UTF-8 (which can overwritten) and the full PostgreSQL server (postgresql-common postgresql-contrib postgresql-server-dev-14) are installed using apt-get install.

ENV PG_VERSION 14
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
       postgresql                        \
       postgresql-client                 \
       postgresql-common                 \
       postgresql-contrib                \
       postgresql-server-dev-$PG_VERSION  && \
     apt-get clean  && \
     rm -rf /var/lib/apt/lists/*

Position a Docker Entrypoint
An optional entrypoint that is very similar to the postgres:14 standard entrypoint is placed in the image:

# Setup the location for pg initialization routine
ADD ./fwdpg-docker-entrypoint.sh /usr/local/bin/fwdpg-docker-entrypoint.sh
RUN chmod 755 /usr/local/bin/fwdpg-docker-entrypoint.sh

Position startup/shutdown Scripts
This script handles startup similar to postgres:14:

ADD ./init-user-db.sh /docker-entrypoint-initdb.d/
RUN chmod 755 /docker-entrypoint-initdb.d/init-user-db.sh
ADD ./shutdown-db.sh /docker-entrypoint-initdb.d/
RUN chmod 755 /docker-entrypoint-initdb.d/shutdown-db.sh

The fwdpg-docker-entrypoint.sh determines whether or not to create the database based upon the directories existence, creates fwd_admin and fwd_user roles, sets up security, and starts postgres:

pgdata=/var/lib/postgresql/data/14/fwdcluster
pgbin=/usr/lib/postgresql/14/bin
user_role="nosuperuser nocreatedb nocreaterole" 
admin_role="superuser createdb createrole" 

if [ ! -d "$pgdata" ]; then
   gosu postgres $pgbin/initdb -D $pgdata --locale=en_US@iso88591_fwd_basic
   gosu postgres $pgbin/pg_ctl -D $pgdata start
   gosu postgres psql -c "create role fwd_user  with $user_role login password 'user';" 
   gosu postgres psql -c "create role fwd_admin with $admin_role login password 'admin';" 
   gosu postgres $pgbin/pg_ctl -D $pgdata stop
   sed -i -e 's/\(^local\s\+all\s\+all\s\+\)trust$/\1password/g' \
          -e 's/\(^host\s\+all\s\+all\s\+127\.0\.0\.1\/32\s\+\)trust$/\1password/g' \
          -e 's/\(^host\s\+all\s\+all\s\+\:\:1\/128\s\+\)trust$/\1password/g' \
          -e 's/\(^local\s\+replication\s\+all\s\+\)trust$/\1md5/g' \
          -e 's/\(^host\s\+replication\s\+all\s\+127\.0\.0\.1\/32\s\+\)trust$/\1md5/g' \
          -e 's/\(^host\s\+replication\s\+all\s\+\:\:1\/128\s\+\)trust$/\1md5/g' \
          -e 's/\(^host all all all scram-sha-256$\)/\#\1/g' \
           $pgdata/pg_hba.conf
   sed -i -e 's/\^#log_destination(.*)$/log_destination\1/g' \
          -e 's/\^#logging_collector(.*)$/logging_collector\1/g' \
          -e 's/\^#log_directory(.*)$/log_directory\1/g' \
          -e 's/\^#log_filename(.*)$/log_filename\1/g' \
          -e 's/\^#log_file_mode(.*)$/log_file_mode\1/g' \
          -e 's/\^#log_rotation_age(.*)$/log_rotation_age\1/g' \
          -e 's/\^#log_rotation_size(.*)$/log_rotation_size\1/g' \
           $pgdata/postgresql.conf
else
   echo "Directory $pgdata exists already." 
fi
gosu postgres $pgbin/pg_ctl -D $pgdata start

Final Cleanup and Setup
The fwd_pg14_admin sudoers file allows startup/shutdown scripts to be run by fwd without a sudo password. The pg_dropcluster cleans up the main cluster.

ADD fwd_pg14_admin /etc/sudoers.d/
RUN gosu postgres pg_dropcluster --stop $PG_VERSION main

EXPOSE 5432/tcp

base_postgres_14¶

This image is built at the same time as the base_ubuntu_22.04 and base_ubuntu_22.04_pg14, but is not derived from them. It is small, and comes from the postgres:14 image.

FROM postgres:14

Install utilities
The unzip, C Library, locales and Java 17 are installed using apt-get install. These facilitate utilities required by FWD.

Custom Locales
The custom locales are placed in the context directory base_ubuntu/postgres/locale by the build_base_ubuntu.sh script, but this image build used the base_ubuntu/postgres as the context directory, so other files outside of the postgres directory are not included during the build (for speed). During the build they are then positioned in the /usr/share/i18n/locales/ directory in the image. The locales-all package is installed, then the custom locales are built with this command (which is why bash is required):

# Copy custom locale files to the image
COPY ./locale/* /usr/share/i18n/locales/

# Build the custom locales
# Step 4/5
RUN cd /usr/share/i18n/charmaps/; \
    declare -A char_encoding_file; \
    char_encoding_file["cp1252"]="CP1252"; \
    char_encoding_file["iso885915"]="ISO-8859-15"; \
    char_encoding_file["iso88591"]="ISO-8859-1"; \
    for locale_file in /usr/share/i18n/locales/*_fwd_basic; do \
        cust_locale=$(basename ${locale_file}); \
        [[ $cust_locale =~ @([^_]*) ]] && char_encoding=${char_encoding_file["${BASH_REMATCH[1]}"]}; \
        chmod 644 -R /usr/share/i18n/locales/${cust_locale}; \
        ([[ -f "${char_encoding}.gz" ]] && gunzip -kf ${char_encoding}.gz); \
        ([[ ! -z "$char_encoding" ]] && echo ${cust_locale} ${char_encoding} >> /etc/locale.gen); \
    done; \
    locale-gen --no-archive

Proper Ownership Set
The ownership of the entrypoint directory is set:

RUN chown -R postgres:postgres /docker-entrypoint-initdb.d/

base_mariadb_11¶

This image is built at the same time as the base_ubuntu_22.04 and base_ubuntu_22.04_pg14, but is not derived from them. It is small, and comes from the mariadb:11 image.

FROM mariadb:11

Install utilities
The unzip, C Library, locales and Java 17 are installed using apt-get install. These facilitate utilities required by FWD.

# Copy custom locale files to the image
COPY ./locale/* /usr/share/i18n/locales/

# Build the custom locales
RUN cd /usr/share/i18n/charmaps/; \
    declare -A char_encoding_file; \
    char_encoding_file["cp1252"]="CP1252"; \
    char_encoding_file["iso885915"]="ISO-8859-15"; \
    char_encoding_file["iso88591"]="ISO-8859-1"; \
    for locale_file in /usr/share/i18n/locales/*_fwd_basic; do \
        cust_locale=$(basename ${locale_file}); \
        [[ $cust_locale =~ @([^_]*) ]] && char_encoding=${char_encoding_file["${BASH_REMATCH[1]}"]}; \
        chmod 644 -R /usr/share/i18n/locales/${cust_locale}; \
        ([[ -f "${char_encoding}.gz" ]] && gunzip -kf ${char_encoding}.gz); \
        ([[ ! -z "$char_encoding" ]] && echo ${cust_locale} ${char_encoding} >> /etc/locale.gen); \
    done; \
    locale-gen --no-archive

Proper Ownership Set
The ownership of the entrypoint directory is set:

RUN chown -R mysql:mysql /docker-entrypoint-initdb.d/

Base Image Creation¶

The build_base_ubuntu.sh script in the project is the tool that creates the 6 base images:

Usage: ./build_base_ubuntu.sh [-qpt] [-c<context_dir>] [-l<locale_dir>] [-r<docker_repo>] [-t<image_tag>][--language=<language>] [--encoding=<encoding>] [--tzdata_area=<area>] [--tzdata_zone=<zone>] [--base_os_tools=<base_os_tools>] [--runtime_tools=<runtime_tools>] [--fwd_uid=<fwd_uid>] [--fwd_gid=<fwd_gid>] [--pg_uid=<pg_uid>] [--pg_gid=<pg_gid>]

Build the base Ubuntu images with/without PostgreSQL 14 support, JDK 8 and JDK 17 support, and PostgreSQL and MariaDB docker images and optionally push.

Where:
    -q suppress image builds
    -p push images to repository
    -d just display commands (dry-run)
    -c<context_dir> Use <context_dir> as the context directory (default=/home/rfb/projects/fwd/container_20240603/base_ubuntu)
    -l<locale_dir> Directory to root of FWD for pulling over custom locales (default=/opt/fwd)
    -r<docker_repo> use specified docker repository (default=goldencode)
    -t<image_tag> use specified image_tag (default=latest)
    --language=<language> use specified language (default=en_US)
    --encoding=<encoding> use specified encoding (default=UTF-8)
    --tzdata_area=<area> use specified tzdata/areas (default=America)
    --tzdata_zone=<zone> use specified tzdata/areas/America zones (default=New_York)
    --base_os_tools=<base_os_tools> install specified base os tools (default=build-essential apt-utils wget curl dos2unix gcc jq locales sudo unzip vim zip bzr python3-paramiko openssh-server openssh-client gosu iproute2 iputils-ping dnsutils iproute2 rsync less)
    --runtime_tools=<runtime_tools> install specified runtime tools (default=dpkg-dev libffi-dev libncurses5-dev ncurses-term ant libpam0g-dev)
    --fwd_uid=<fwd_uid> UID of fwd user ID (default=1000)
    --fwd_gid=<fwd_gid> GID of fwd user ID (default=1000)
    --pg_uid=<pg_uid> UID of postgres user ID (default=1002)
    --pg_uid=<pg_gid> GID of postgres user ID (default=1003)
     <file> should include the tools going into /root and the /etc/apt/apt.conf.d directories.

 -? or -h or --help = show usage.

Option Details
-q If you only want to push existing images and not recreate them
-p Push the images to the repository
-t Perform a dry-run
-c If you want to specify a directory other than fwd_ubuntu for the context directory. There are some files in the context directory that are required to build the image.
-l This indicates where you would get the locale files from, typically copied from a FWD location. The directory will be copied into the context.
-r The repository you want to push to instead of goldencode.
Some variables specified in the Dockerfile can be overridden via command line options that begin with --. For example, you may not want to include dos2unix so you would specify: --base_os_tools=build-essential apt-utils wget curl gcc jq locales sudo unzip vim zip bzr python3-paramiko openssh-server openssh-client gosu iproute2 iputils-ping dnsutils iproute2 rsync

In the above usage example, the context directory is /home/rfb/projects/fwd/container/fwd_ubuntu. Here is a simple example taking the locale files from your ~/projects/fwd/p2j directory:

./build_base_ubuntu.sh -l ~/projects/fwd/p2j/

This will generate these images local to your system, unless you use -p to push them to a repository -r<docker_repo>:

base_ubuntu_22.04:latest
base_ubuntu_22.04_pg14:latest
base_ubuntu_22.04_jdk17:latest
base_ubuntu_22.04_pg14_jdk17:latest
base_postgres_14:latest
base_mariadb_11:latest

FWD Images¶

Assuming the base images are available either locally or in a repository like goldencode, docker images with FWD built-in can be generated. The container project creates the 4 FWD images using the base images. The Dockerfiles add the FWD deployment parts, as built by a gradlew all sheet:war archive command. They are placed into the image with the ./deploy_fwd.sh /opt/fwd-deploy all -f command.

The images are stored in the goldencode repository. The images are also tagged with the branch and revision of FWD they contain:

REPOSITORY	TAGS	CONTENTS
goldencode/fwd_4.0_ubuntu_22.04	latest and trunk_<rev>	FWD with only necessary PostgreSQL 14 client components, JDK 8
goldencode/fwd_4.0_ubuntu_22.04_pg14	latest and trunk_<rev>	FWD with fully capable PostgreSQL 14, JDK 8
goldencode/fwd_4.0_ubuntu_22.04_jdk17	latest and trunk_<rev>	FWD with only necessary PostgreSQL 14 client components, JDK 17
goldencode/fwd_4.0_ubuntu_22.04_pg14_jdk17	latest and trunk_<rev>	FWD with fully capable PostgreSQL 14, JDK 17
goldencode/fwd_4.0_postgres_14	latest and trunk_<rev>	PostgreSQL containing portions of FWD
goldencode/fwd_4.0_mariadb_11	latest and trunk_<rev>	MariaDB containing portions of FWD

Typically you would create your images without any repository, and just store them locally. Here are examples of branch 7277a revision 14573 (tagged with --latest as well):

REPOSITORY	TAGS
fwd_4.0_ubuntu_22.04	7277a_latest and 7277a_14573
fwd_4.0_ubuntu_22.04_pg14	7277a_latest and 7277a_14573
fwd_4.0_ubuntu_22.04_jdk17	7277a_latest and 7277a_14573
fwd_4.0_ubuntu_22.04_pg14_jdk17	7277a_latest and 7277a_14573
goldencode/fwd_4.0_postgres_14	7277a_latest and 7277a_14573
goldencode/fwd_4.0_mariadb_11	7277a_latest and 7277a_14573

The steps for the creation of those images are outlined in the FWD Image Creation section below. Note that the JDK 17 images are not created by default as they may not be desired, as well as you may not want to build both versions in the same space.

Under the Hood¶

This section details what is contained within the images, describing highlights from the Dockerfiles. Note that both images utilize the same Dockerfile contents, with the exception of the FROM image, one being the base_ubuntu_22.04 image, the other being base_ubuntu_22.04_pg14.

Dockerfile details¶

FWD_DIR build-arg
The directory where FWD resides in the context directory is passed into the Dockerfile using this argument.

ARG FWD_DIR

Prepare fwd service account directory
The service account will have a home directory, and any use of sudo will have the warning message suppressed.

ENV HOME /home/fwd
WORKDIR ${HOME}
RUN touch ${HOME}/.sudo_as_admin_successful

Deploy FWD that was placed in the context directory
The ZIP files created by the archive task will be placed in the context directory given in the FWD_DIR build-arg, then the deploy_fwd.sh script will placed FWD into the directories under /opt/fwd-deploy in the image. These are:

admin
client
convert
docs
native
server

spawner

COPY --chown=fwd:fwd ${FWD_DIR}/ /tmp/
RUN chmod 755 ${HOME}
WORKDIR /tmp
RUN ./deploy_fwd.sh /opt/fwd-deploy all -f

Cleanup and Setup the /opt/spawner directory
The temporary location for the zip archives is cleaned up, permissions of the /opt/fwd-deploy directory are set, the p2j.jar and spawn files are placed in /opt/spawner, and permissions set.

RUN rm -fr /tmp/* && \
    chown -R fwd:fwd /opt/fwd-deploy && \
    mkdir -p /opt/spawner && \
    cp /opt/fwd-deploy/native/spawn /opt/spawner/ && \
    chmod -R gou+rx /opt/spawner/ && \
    chmod u+s /opt/spawner/spawn && \
    cp /opt/fwd-deploy/convert/build/lib/p2j.jar /opt/spawner/ && \
    chmod 0550 /opt/spawner/p2j.jar

Final Steps
The current user is switched to the fwd service account and the working directory is set to the home directory.

USER fwd
WORKDIR /home/fwd

EXPOSE 22

FWD Image Creation¶

The build_fwd.sh script in the project is the tool that creates the Ubuntu images containing FWD that can be used for conversion and runtime:

Usage: ./build_fwd.sh [-b<branch>] [-r<revision>] [-dpqwit] [-x<user>] [-l<build_location>] [-c<context_dir>] [--build_image=<build_image>] [--repo=<docker_repo>] [--from_repo=<from_repo>] [--jdk_version=<jdk8|jdk17>] [--build_db] [--latest]

Build the FWD and DB Docker docker images. Tag appropriately, and optionally push.

Where:
    -b<branch> = use this specific <branch> (default=trunk)
    -r<revision> = use this specific <revision> of the given branch (default=latest)
    -d delete any existing project directory. (default=/home/rfb/projects/fwd/container_20240603/projects/p2j)
    -p push image to repository
    -q suppress FWD checkout
    -w suppress FWD build
    -i suppress image build
    -t just display commands (test)
    -x use xfer.goldencode.com repository for FWD, with username <user>
    -l<build_location> Directory where FWD is (or will be) checked out to. (default=/home/rfb/projects/fwd/container_20240603/projects/p2j)
    -c<context_dir> Use <context_dir> as the context directory (default=/home/rfb/projects/fwd/container_20240603/fwd_bld/)
    --build_image=<build_image> use specified docker image for building FWD (default=base_ubuntu_22.04)
    --repo=<docker_repo> if pushing, push built images to specified docker repository (default=goldencode)
    --from_repo=<from_repo> specify a repo to pull base images from ('.' indicates local) (default=.)
    --jdk_version=<jdk8|jdk17> indicates which JDK to use for FWD build. (default=jdk8)
    --build_dbs build postgres and mariadb images
    --latest tag as 'latest' as well

 -? or -h or --help = show usage.

Option Details
-b / -r The script can pull various FWD branch/revisions and this is how you specify which to pull. By default, the latest trunk revision is pulled
-d Clean out any existing project directory. If you want to checkout a new FWD, and the project directory exists, you need to specify this option, or the script will halt so as not to overwrite.
-p Push the images to the repository
--repo The repository you want to push to instead of goldencode. Since -r option is used, we need to use an alternative method to specify the repository.
--from_repo The base images are typically contained locally, if you build them yourself. By including --from_repo=<from_repo>, the Dockerfiles will pull from the specified repository. For example, --from_repo=goldencode
-q If you already have a branch/revision in place, this will prevent checkout of a new copy.
-w This allows suppression of a build of FWD. It is expected that the ./dist directory of the build_location already contains built distribution zip files, which will be used to create the images.
-i The images held locally are to be left as-is. Typically this allows you to perform a push to a repository of the existing local image.
-t Perform a dry-run
-x Use the given username to access the xfer.goldencode.com server to access FWD. By default, the ssh connected repo on an internal secured server is used.
-l This specifies the location to which a checkout is done, or in the case of -q where a checkout has already been done.
-c The context directory location where files needing to be put into the image are placed. The default is ./fwd_bld.
--build_image By default the Ubuntu base image is used to perform the build of FWD, but can be specified with this option.
--repo If you are pushing to a repository, you can specify the docker repository, if it isn't goldencode
--from_repo You may want to base these off of images from a repository other than goldencode, or from a local image, which can be specified as .*. A local image is the default.
--jdk_version* By default, FWD is built using JDK 8. However, you can specify JDK 17 using this parameter. The image name will have _jdk17 appended.
--build_dbs The postgres and mariadb images don't need to be built, but can be built by adding this parameter.
--latest This allows you to tag the images with "latest" as well as the <branch>_<revision> tag. This should typically be reserved for trunk, if published on the goldencode repository.

If the parameters are set such that a build of FWD is performed, the script will position the FWD project as specified, and utilize the base_ubuntu_22.04 image to perform the ./gradlew all sheet:war archive build, unless the --build_image=<build_image> is specified. The generated archives are positioned in the context directory, and the images are built. If JDK 17 is requested, the base_ubuntu_22.04_jdk17 image will be used to perform the build.

Examples¶

To build the latest trunk and store as latest on the goldencode repository:

./build_fwd.sh -p --latest

To retrieve and build the latest revision of branch 7277a and leave it local, but utilize base images on the goldencode repository:

./build_fwd.sh -b 7277a --from_repo=goldencode

If a copy of the latest revision of branch 7277a is already built in the ~/projects/fwd/7277a directory (there needs to be ~/projects/fwd/7277a/dist/ zip files in place from the archive task), you can just create local images:

./build_fwd.sh -qw -b 7277a -l ~/projects/fwd/7277a

If a copy of the latest revision of branch 7277a needs to be built is already built (there needs to be ./dist/ zip files in place from the archive task), you can just create local images:

./build_fwd.sh -qw -b 7277a

To build the latest trunk using your ID to pull from xfer to ~/projects/fwd/trunk, and just keep the image local:

./build_fwd.sh -x$USER -l ~/projects/fwd/trunk

To build the latest trunk with JDK 17 and store as latest on the goldencode repository:

./build_fwd.sh -p --latest --jdk_version=jdk17

Project

General

Profile

FWD

Wiki

Building and Using Docker Images¶

Base Images¶

Under the Hood¶

base_ubuntu_22.04 and base_ubuntu_22.04_jdk17¶

base_ubuntu_22.04_pg14 and base_ubuntu_22.04_pg14_jdk17¶

base_postgres_14¶

base_mariadb_11¶

Base Image Creation¶

FWD Images¶

Under the Hood¶

Dockerfile details¶

FWD Image Creation¶

Examples¶