Build smaller containers

build smaller containers

Otter image excerpted from photo by Dele Oluwayomi on Unsplash

Working with containers is a daily task for many users and developers. Container developers often need to (re)build container images frequently. If you develop containers, have you ever thought about reducing the image size? Smaller images have several benefits. They require less bandwidth to download and they save costs when run in cloud environments. Also, using smaller container images on Fedora CoreOS, IoT and Silverblue improves overall system performance because those operating systems rely heavily on container workflows. This article will provide a few tips for reducing the size of container images.

The tools

The host operating system in the following examples is Fedora Linux 33. The examples use Podman 3.1.0 and Buildah 1.2.0. Podman and Buildah are pre-installed in most Fedora Linux variants. If you don’t have Podman or Buildah installed, run the following command to install them.

$ sudo dnf install -y podman buildah

The task

Begin with a basic example. Build a web container meeting the following requirements.

  • The container must be based on Fedora Linux
  • Use the Apache httpd web server
  • Include a custom website
  • The container should be relatively small

The following steps will also work on more complex images.

The setup

First, create a project directory. This directory will include your website and container file.

$ mkdir smallerContainer
$ cd smallerContainer
$ mkdir files
$ touch files/index.html

Make a simple landing page. For this demonstration, you may copy the below HTML into the index.html file.

<!doctype html>

<html lang="de">
<head>
  <title>Container Page</title>
</head>

<body>
  <header>
    <h1>Container Page</h1>
  </header>
  <main>
    <h2>Fedora</h2>
    <ul>
      <li><a href="https://getfedora.org">Fedora Project</a></li>
      <li><a href="https://docs.fedoraproject.org/">Fedora Documentation</a></li>
      <li><a href="https://fedoramagazine.org">Fedora Magazine</a></li>
      <li><a href="https://communityblog.fedoraproject.org/">Fedora Community Blog</a></li>
    </ul>
    <h2>Podman</h2>
    <ul>
      <li><a href="https://podman.io">Podman</a></li>
      <li><a href="https://docs.podman.io/">Podman Documentation</a></li>
      <li><a href="https://github.com/containers/podman">Podman Code</a></li>
      <li><a href="https://podman.io/blogs/">Podman Blog</a></li>
    </ul>
    <h2>Buildah</h2>
    <ul>
      <li><a href="https://buildah.io">Buildah</a></li>
      <li><a href="https://github.com/containers/buildah">Buildah Code</a></li>
      <li><a href="https://buildah.io/blogs/">Buildah Blog</a></li>
    </ul>
    <h2>Skopeo</h2>
    <ul>
      <li><a href="https://github.com/containers/skopeo">skopeo Code</a></li>
    </ul>
    <h2>CRI-O</h2>
    <ul>
      <li><a href="https://cri-o.io/">CRI-O</a></li>
      <li><a href="https://github.com/cri-o/cri-o">CRI-O Code</a></li>
      <li><a href="https://medium.com/cri-o">CRI-O Blog</a></li>
    </ul>
  </main>
</body>

</html>

Optionally, test the above index.html file in your browser.

$ firefox files/index.html

Finally, create a container file. The file can be named either Dockerfile or Containerfile.

$ touch Containerfile

You should now have a project directory with a file system layout similar to what is shown in the below diagram.

smallerContainer/
|- files/
|    |- index.html
|
|- Containerfile

The build

Now make the image. Each of the below stages will add a layer of improvements to help reduce the size of the image. You will end up with a series of images, but only one Containerfile.

Stage 0: a baseline container image

Your new image will be very simple and it will only include the mandatory steps. Place the following text in Containerfile.

# Use Fedora 33 as base image
FROM registry.fedoraproject.org/fedora:33

# Install httpd
RUN dnf install -y httpd

# Copy the website
COPY files/* /var/www/html/

# Expose Port 80/tcp
EXPOSE 80

# Start httpd
CMD ["httpd", "-DFOREGROUND"]

In the above file there are some comments to indicate what is being done. More verbosely, the steps are:

  1. Create a build container with the base FROM registry.fedoraproject.org/fedora:33
  2. RUN the command: dnf install -y httpd
  3. COPY files relative to the Containerfile to the container
  4. Set EXPOSE 80 to indicate which port is auto-publishable
  5. Set a CMD to indicate what should be run if one creates a container from this image

Run the below command to create a new image from the project directory.

$ podman image build -f Containerfile -t localhost/web-base

Use the following command to examine your image’s attributes. Note in particular the size of your image (467 MB).

$ podman image ls
REPOSITORY                         TAG     IMAGE ID      CREATED        SIZE
localhost/web-base                 latest  ac8c5ed73bb5  5 minutes ago  467 MB
registry.fedoraproject.org/fedora  33      9f2a56037643  3 months ago   182 MB

The example image shown above is currently occupying 467 MB of storage. The remaining stages should reduce the size of the image significantly. But first, verify that the image works as intended.

Enter the following command to start the container.

$ podman container run -d --name web-base -P localhost/web-base

Enter the following command to list your containers.

$ podman container ls
CONTAINER ID  IMAGE               COMMAND               CREATED        STATUS            PORTS                  NAMES
d24063487f9f  localhost/web-base  httpd -DFOREGROUN...  2 seconds ago  Up 3 seconds ago  0.0.0.0:46191->80/tcp  web-base

The container shown above is running and it is listening on port 46191. Going to localhost:46191 from a web browser running on the host operating system should render your web page.

$ firefox localhost:46191

Stage 1: clear caches and remove other leftovers from the container

The first step one should always perform to optimize the size of their container image is “clean up”. This will ensure that leftovers from installations and packaging are removed. What exactly this process entails will vary depending on your container. For the above example you can just edit Containerfile to include the following lines.

[...]
# Install httpd
RUN dnf install -y httpd && \
    dnf clean all -y
[...]

Build the modified Containerfile to reduce the size of the image significantly (237 MB in this example).

$ podman image build -f Containerfile -t localhost/web-clean
$ podman image ls
REPOSITORY            TAG     IMAGE ID      CREATED        SIZE
localhost/web-clean   latest  f0f62aece028  6 seconds ago  237 MB

Stage 2: remove documentation and unneeded package dependencies

Many packages will pull in recommendations, weak dependencies and documentation when they are installed. These are often not needed in a container and can be excluded. The dnf command has options to indicate that it should not include weak dependencies or documentation.

Edit Containerfile again and add the options to exclude documentation and weak dependencies on the dnf install line:

[...]
# Install httpd
RUN dnf install -y httpd --nodocs --setopt install_weak_deps=False && \
    dnf clean all -y
[...]

Build Containerfile with the above modifications to achieve an even smaller image (231 MB).

$ podman image build -f Containerfile -t localhost/web-docs
$ podman image ls
REPOSITORY            TAG     IMAGE ID      CREATED        SIZE
localhost/web-docs    latest  8a76820cec2f  8 seconds ago  231 MB

Stage 3: use a smaller container base image

The prior stages, in combination, have reduced the size of the example image by half. But there is still one more thing that can be done to reduce the size of the image. The base image registry.fedoraproject.org/fedora:33 is meant for general purpose use. It provides a collection of packages that many people expect to be pre-installed in their Fedora Linux containers. The collection of packages provided in the general purpose Fedora Linux base image is often more extensive than needed, however. The Fedora Project also provides a fedora-minimal base image for those who wish to start with only the essential packages and then add only what they need to achieve a smaller total image size.

Use podman image search to search for the fedora-minimal image as shown below.

$ podman image search fedora-minimal
INDEX               NAME   DESCRIPTION   STARS   OFFICIAL   AUTOMATED
fedoraproject.org   registry.fedoraproject.org/fedora-minimal         0

The fedora-minimal base image excludes DNF in favor of the smaller microDNF which does not require Python. When registry.fedoraproject.org/fedora:33 is replaced with registry.fedoraproject.org/fedora-minimal:33, dnf needs to be replaced with microdnf.

# Use Fedora minimal 33 as base image
FROM registry.fedoraproject.org/fedora-minimal:33

# Install httpd
RUN microdnf install -y httpd --nodocs --setopt install_weak_deps=0 && \
    microdnf clean all -y
[...]

Rebuild the image to see how much storage space has been recovered by using fedora-minimal (169 MB).

$ podman image build -f Containerfile -t localhost/web-docs
$ podman image ls
REPOSITORY             TAG     IMAGE ID      CREATED        SIZE
localhost/web-minimal  latest  e1603bbb1097  7 minutes ago  169 MB

The initial image size was 467 MB. Combining the methods detailed in each of the above stages has resulted in a final image size of 169 MB. The final total image size is smaller than the original base image size of 182 MB!

Building containers from scratch

The previous section used a container file and Podman to build a new image. There is one last thing to demonstrate — building a container from scratch using Buildah. Podman uses the same libraries to build containers as Buildah. But Buildah is considered a pure build tool. Podman is designed to work as a replacement for Docker.

When building from scratch using Buildah, the container is empty — there is nothing in it. Everything needed must be installed or copied from outside the container. Fortunately, this is quite easy with Buildah. Below, a small Bash script is provided which will build the image from scratch. Instead of running the script, you can run each of the commands from the script individually in a terminal to better understand what is being done.

#!/usr/bin/env bash
set -o errexit

# Create a container
CONTAINER=$(buildah from scratch)

# Mount the container filesystem
MOUNTPOINT=$(buildah mount $CONTAINER)

# Install a basic filesystem and minimal set of packages, and httpd
dnf install -y --installroot $MOUNTPOINT  --releasever 33 glibc-minimal-langpack httpd --nodocs --setopt install_weak_deps=False

dnf clean all -y --installroot $MOUNTPOINT --releasever 33

# Cleanup
buildah unmount $CONTAINER

# Copy the website
buildah copy $CONTAINER 'files/*' '/var/www/html/'

# Expose Port 80/tcp
buildah config --port 80 $CONTAINER

# Start httpd
buildah config --cmd "httpd -DFOREGROUND" $CONTAINER

# Save the container to an image
buildah commit --squash $CONTAINER web-scratch

Alternatively, the image can be built by passing the above script to Buildah. Notice that root privileges are not required.

$ buildah unshare bash web-scratch.sh
$ podman image ls
REPOSITORY             TAG     IMAGE ID      CREATED        SIZE
localhost/web-scratch  latest  acca45fc9118  9 seconds ago  155 MB

The final image is only 155 MB! Also, the attack surface has been reduced. Not even DNF (or microDNF) is installed in the final image.

Conclusion

Building smaller container images has many advantages. Reducing the needed bandwidth, the disk footprint and attack surface will lead to better images overall. It is easy to reduce the footprint with just a few small changes. Many of the changes can be done without altering the functionality of the resulting image.

It is also possible to build very small images from scratch which will only hold the needed binaries and configuration files.

FAQs and Guides For Developers For System Administrators

26 Comments

  1. Thank you for the last part. “Building from scratch.”
    I’m woundering about SystemD. It must be installed as well? but not started?

    • Daniel Schier

      systemd is installed by dependencies, but is not needed in a container. In fact, it should be even avoided if necessary. Starting a service directly in foreground should be preferred.

  2. Rob Verduijn

    Removing the layers also reduces the overhead a bit.

    podman build –layers=false

    • Daniel Schier

      --layers=false

      will only avoid caching layers during the build. If you want to squash everything in one layer (which can be useful depending on your usecase), you can use

      --squash

      or

      --sqash-all

      .

  3. Rene Reichenbach

    And now try with a go or rust app doing the same 😉

    • Phozzy

      cargo install --root=${MOUNTPOINT} coreutils

      – will work well in the last scenario.

    • Daniel Schier

      The “from scratch” example will most likely work similar, but you can also have a look at multi-stage builds. 🙂

  4. rnx

    useful article. thanks.
    perhaps dnf should get a new subcommand specifically for installing with recommended defaults for container images and cache cleaning … would make things more cleaner and more readable.
    dnf containerinstall xy or something

  5. Mark

    I have had issues in the past with microdnf when a fedora-minimal image must have been built prior to a package update being removed from the main repository and it found a package to downgrade, microdnf will fail instead of downgrading. Infrequent but if you care about attack surfaces you should always ‘microdnf -y update’ when building a new container so it can be an issue. So some of my docker files use microdnf to install dnf and then remove that when done.

    I use docker and there is a utility ‘docker-squash’ (obtained simply with ‘pip3 install docker-squash’) which has made me a bit lazy; in that you can use clarity in Dockerfile usage by not bothering to use && to stack multiple commands to minimise layers but instead happily generate as many layers as you want and even have huge ones at the start. And docker-squash will happily compress 20-30 layers down to 2 for you which shrinks the container image size a lot. I also use that on images pulled from dockerhub where some application containers can be >1Gb in size when pulled from dockerhub and many of those can be shrunk as well. docker-squash does need docker though and won’t work with podman last time I checked.

    Having said that I was interested in the comment from Rob Verduijn about the ‘podman build –layers=false’ option I will have to look at to see what it does, as it is the layers that while useful for managing changes/versions take up needless space in a container image. But it’s use would be in containers personally built at build time rather than run against containers already built externally.

    I also appreciate the ‘building from scratch’ section, very easy to follow with the example you used.

    On layers, it should probably also be mentioned it is best to keep large layers toward the end of a Dockerfile or config file; don’t layer1 copy in a 100Mb datafile then layer2 install a package then layer3 add a user or all three layers would have that datafile. Add the user then the package then the 100Mb database and you have saved 200Mb in your final container image; order is important. Or you could docker-squash and as I am also going to do investigate the ‘podman build –layers=false’ option Rob mentioned.

    • Daniel Schier

      For squashing layers you can use the option

      --squash

      or

      --squash-all

      , which is available by default for podman build and buildah.

  6. Diogo Lopes

    Very Good

  7. langdon

    Minor complaint. I think you have a “typo” in the final script. You say “nginx” in the comments when I think you meant “httpd” (or apache). I was confused for a minute so others might be as well. Otherwise, nice article.

  8. laolux

    Thanks for the article. I really like the from scratch method using buildah. Now I would like to improve on that by reusing the system’s dnf metadata cache. Any idea of how to accomplish that? Otherwise any user on my system would have to download all the metadata just for himself.
    Another point: When using the btrfs backend for podman, then regular users need to execute the script with

    buildah unshare

    . Then it works fine though.

  9. Mohamed Amin

    Thanks for the article. I would think of using buildah for building containers.

    I pulled apache default and alpine image from docker hub and I found this:

    [ ~]$ podman images | grep httpd
    docker.io/library/httpd                                 alpine   a4d0bee07118  8 days ago     57 MB
    docker.io/library/httpd                                 latest   0b932df43057  13 days ago    142 MB

    Why the alpine image here is so small?

    • Daniel Schier

      Hi, this is an easy/not so easy to answer question.

      Alpine is very focused on minimalism. Therefore, it replaced and stripped important packages, which are common in other Linux systems. For example, you will not have gcc/glibc, but musl libc/gcc. But this also means, that you have to build software differently and may not be able to use certain software at all.

  10. xhtml

    useful article. thanks. 😀
    Podman tutorial – How to install XAMPP server in Podman container?

    • Daniel Schier

      Since XAMPP was the idea to get LAMP on Windows without knowing how to install Apache, MySQL, PHP, etc. I would suggest using each of these tools in a container is the proper way. At least I am considering XAMPP “a bunch of tools for mere testing only”. Using httpd or MariaDB in a container is a piece of cake with containers 🙂

  11. Shaun McFee

    Could multi-stage builds be used within the Containerfile itself to achieve what the bash script of Buildah commands is doing in order to keep the Containerfile as the single source of truth for how the container was built? Is there an advantage to the bash script with Buildah approach versus multi-stage builds?

  12. SergMx

    I’m sorry.
    Doesn’t Fedora have a similar project: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image
    ?

    • You can use UBI on Fedora. UBI is supposed to sort of be universal, can be used on Fedora or even other Linux distros.

      Also, UBI micro will come out with RHEL 8.4 and that is even smaller than the minimal image with microdnf

      • Daniel Schier

        Last time I used UBI on Fedora/CentOS (a year ago maybe), it requested a subscription. Is this still the case?

  13. Lee Whitty

    Great article! I just went through this exercise using Fedora 34 images. They must have reduced the minimal image size because the podman build using the minimal image was smaller (154 MB) than the buildah build (155 MB).

    • Lee Whitty

      Correction: Both the minimal image size build and buildah builds were smaller under Fedora 34, with the Buildah build being the smallest. I like the trend of the container image sizes shrinking!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

The opinions expressed on this website are those of each author, not of the author's employer or of Red Hat. Fedora Magazine aspires to publish all content under a Creative Commons license but may not be able to do so in all cases. You are responsible for ensuring that you have the necessary permission to reuse any work on this site. The Fedora logo is a trademark of Red Hat, Inc. Terms and Conditions

%d bloggers like this: