Project

General

Profile

Feature #10719

Base Docker images need version control

Added by Roger Borrello 9 months ago. Updated 26 days ago.

Status:
WIP
Priority:
Normal
Target version:
-
Start date:
Due date:
% Done:

90%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:
reviewer:
production:
No
env_name:
topics:

History

#2 Updated by Roger Borrello 9 months ago

When images are built off a base Docker image (the FWD images, and subsequently the customer Docker images) and "latest" tag is used, changes can be pulled into the layer above that may not be desired.

By tagging a base image with some sort of convention, the layer above can utilize that tag in it's FROM directive, and not pull in changes that may be pulled in if :latest is used.

#3 Updated by Roger Borrello 9 months ago

🧩 How Docker Determines if a Step Needs to Be Re-Executed or a Layer Recreated

When you run docker build, Docker executes your Dockerfile instruction by instruction, and after each instruction it decides whether it can reuse a cached layer or needs to recreate it.


1. Layer Cache is Keyed by Build Context and Instruction

For every Dockerfile instruction (FROM, RUN, COPY, etc.), Docker calculates a cache key that includes:

  • The instruction text itself
  • The context of all previous layers (the image it’s building on)
  • Any files referenced by that instruction (for example, from COPY or ADD)

If all of those inputs are byte-for-byte identical to a previous build, Docker reuses the cached layer.
If anything differs β€” even whitespace in the instruction or file modification β€” that step and all following steps are rebuilt.


2. How Each Instruction Affects Caching

FROM
  • Reused if the base image digest is identical.
  • If the tag points to a new digest β†’ cache is invalidated.
RUN
  • Cache key includes the command text and the filesystem snapshot from the previous step.
  • Any change in command, arguments, or environment causes rebuild.
COPY / ADD
  • Cache key includes the content hash of the copied files.
  • If any file content changes β†’ step and following steps are rebuilt. (Timestamps do not matter.)
ENV, ARG, WORKDIR, USER, EXPOSE, etc.
  • Cache depends on instruction text. Any change β†’ cache busts.
CMD, ENTRYPOINT, LABEL, etc.
  • Affects only metadata; reused if text is identical.

3. Once a Step Changes, All Following Layers Rebuild

Cache reuse is linear.
As soon as one step changes, all later steps must be re-executed because the filesystem changed.

Example of good caching structure:

FROM ubuntu:24.04
RUN apt-get update && apt-get install -y curl
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

If only your application source code changes, Docker will reuse all prior layers up to the final COPY . . step.


4. Build Arguments and Cache Busting

Build args (ARG) and environment variables (ENV) affect cache only if used in the same layer.

Example:

ARG BUILD_DATE
RUN echo $BUILD_DATE

Changing --build-arg BUILD_DATE forces that RUN step and all subsequent steps to rebuild.


5. Manual Cache Control

To force Docker to ignore cache:

docker build --no-cache .

Or add a dummy command that always changes:

RUN echo "Force rebuild $(date)" 

πŸ” Quick Summary

Instruction Type What Invalidates Cache
FROM Base image digest changes
RUN Command text or previous layer changes
COPY / ADD File contents differ
ENV, ARG, etc. Instruction text or value changes
Any Step After a Change Always rebuilt

πŸ’‘ Pro Tip: See Which Layers Were Reused

You can inspect caching behavior with:

docker build --progress=plain .

Each step will indicate whether it used cache:


# Using cache
#0  CACHED

or executed:


#0  RUN apt-get update && apt-get install ...

#4 Updated by Roger Borrello 26 days ago

  • % Done changed from 0 to 90
  • Status changed from New to WIP

I've got updates in the container project to stuff meta data into /etc/version.properties in the image's filesystem when building it. Retrieve with show_image_version.sh:

image.name=basedev_ubuntu_24.04
image.tag=latest
image.variant=basedev
build.script=build_base_ubuntu_24.04.sh
build.dockerfile=basedev_ubuntu_24.04_Dockerfile
build.date=2026-06-25T19:15:40Z
source.vcs=bzr
source.repo=container_working
source.branch=/home/rfb/secure/code/p2j_repo/container
source.revno=135
build.arg.from_image=base_ubuntu_24.04:latest
build.arg.ubuntu_version=24.04
build.arg.jdk_version=17
build.arg.jdk_install=openjdk-25-jdk openjdk-21-jdk openjdk-17-jdk
build.arg.jdk_sources=openjdk-25-source openjdk-21-source openjdk-17-source
build.arg.language_value=en_US
build.arg.encoding_value=UTF-8
build.arg.tzdata_area=America
build.arg.tzdata_zone=New_York
build.arg.base_tools=openssh-server gosu locales jq sed curl sudo wget unzip zip 7zip rsync less ca-certificates gnupg screen dos2unix vim ncurses-term sysstat iptables lsb-release sshfs redir lynx
build.arg.base_dev_tools=build-essential gcc brz python3-paramiko apt-utils libc-bin ant dpkg-dev libffi-dev libpam0g-dev iproute2 iputils-ping bind9-dnsutils software-properties-common libncurses5-dev
build.arg.code_url=bzr+ssh://rfb@xfer.goldencode.com/opt/fwd
build.arg.fwd_uid=1000
build.arg.fwd_gid=1000
build.arg.pg_uid=999
build.arg.pg_gid=999

I've also added OCI labels so you can directly inspect the image with docker inspect base_ubuntu_24.04:latest --format '{{json .Config.Labels}}' | jq:

{
  "com.goldencode.build.dockerfile": "base_ubuntu_24.04_Dockerfile",
  "com.goldencode.build.script": "build_base_ubuntu_24.04.sh",
  "com.goldencode.image.variant": "base",
  "com.goldencode.source.branch": "/home/rfb/secure/code/p2j_repo/container",
  "com.goldencode.source.vcs": "bzr",
  "org.opencontainers.image.created": "2026-06-25T19:51:25Z",
  "org.opencontainers.image.ref.name": "ubuntu",
  "org.opencontainers.image.revision": "135",
  "org.opencontainers.image.source": "container_working",
  "org.opencontainers.image.title": "base_ubuntu_24.04",
  "org.opencontainers.image.version": "latest" 
}

Also available in: Atom PDF