Docker Image Automated Tests

We are going to explore automated testing for docker images. I found this to be a bit more difficult than application testing because images are generally built with shell scripts and I couldn’t find a comprehensive framework in the likes of JUnit or RSpec that didn’t require a lot of dependencies. We will look at a few continuous delivery (CD) solutions so we can see the differences in implementation.

I assume a basic understanding of Docker images, what they are and how to build them using a Dockerfile.

Facilities for Automated Testing

We will show examples of automated testing for the following services:

Fortunately the same test code will be shared by all. Each service has it’s own configuration file for invoking the tests which we’ll look at to invoke the tests.

We’ll be using for our example. forked-daapd is a music server using the DAAP protocol recognized by iTunes and other software, and also using the Roku music protocol. Our test is simple, make sure the server is listening to the port. More tests can and should be written, but this will suffice for our purposes.

Docker Hub

Docker Hub provides facilities for building and publishing docker images. It is the default registry in docker installations and public image hosting is free.

Docker Hub provides automated testing facilities through Docker Compose files. Any files in the same folder as the Dockerfile that end with .test.yml will be considered a Compose file to be used for testing. Docker Hub will perform a one-off execution of the sut service. This is where your test scripts are executed. The return code from the sut service is used to determine if the tests passed or failed. A return value of zero is considering passing, non-zero is failing.

File Structure

Here is an overview of the files we’ll be creating. (The files needed to produce the application container are omitted.)

.travis.yml             # Config for TravisCI
circle.yml              # Config for CircleCI
docker-compose.test.yml # Config for shared test environment
Dockerfile              # Builds the images
  - Dockerfile     # Builds the image running the tests
  - results        # Mapped into the test container to hold the xunit results
  -       # Records test results in `xunit` (aka `JUnit` format)
  -        # Runs the tests
  - # Waits for the container under test to start up


docker-compose.test.yml defines the container environment for our tests and what to invoke. The sut service is what Docker Hub will invoke as a one-off command, equivalent to docker-compose -f docker-compose.test.yml run sut. We link the image under test, in this case daapd, and the container will be available to us with the host name daapd. The /results volume will be used to store test results using xunit/JUnit format so that the CI environment can report on our individual tests. Any number of necessary services, volumes, networks, etc. can be used to support the tests, they will not affect the published image.


version: '2'
    build: .
    build: test
      - daapd
      - ./test/results:/results

The alpine image being used for the tests is a very small image based on busybox. If you need more functionality, other images will work as well.


FROM alpine:3.4
COPY . .
CMD ["./"]

The tests can run any command as long as the exit status indicates pass/fail. The script will need to handle returning a failing exit code if any of the tests fail, otherwise the exit code of the last command is returned from

The script is used to wait for the container to be started. It has options for setting timeouts and requires very few commands to do it’s work.

The script reports command status using the xunit (aka JUnit) XML format. Most CI environments can consume this and it makes for nice reports.

/ daapd:3689
# Provides juLog and juLogClean
source /
# Remove any previous test results
# Run a test named 'opentcp' that ensures the server is listening
juLog -name="opentcp" nc -zv -w 10 daapd 3689

Test It!

With this repo configured in Docker Hub as an automated build, pushing will trigger the image to be built and tested. Only when the tests pass will the image be published.


Integration with CircleCI is straightforward. CircleCI uses the circle.yml file in the root of your repo to configure the pipeline.

Since docker-compose.test.yml is a standard Compose file, we can invoke Compose in our circle.yml file to run the tests. CircleCI wants test results in the directory referenced by CIRCLE_TEST_REPORTS, so we’ll copy them over.

You’ll want to configure your CircleCI project to run on the latest Ubuntu available (trusty at this time), and report on JUnit results. At the time of this writing the docker install is older, so we’ll install version 1.10.0.


    - curl -sSL | bash -s -- 1.10.0
    - docker
    - sudo pip install docker-compose
    - mkdir -p ./test/results
    - docker-compose -f docker-compose.test.yml build sut
    - docker-compose -f docker-compose.test.yml run sut
    - cp ./test/results/* $CIRCLE_TEST_REPORTS


Integration with TravisCI is simple as well. The pipeline configuration is stored in .travis.yml (don’t miss the leading dot). The docker version provided is most recent than CircleCI, so the configuration file is smaller.


sudo: required
  - docker
  - mkdir -p ./test/results
  - docker-compose -f docker-compose.test.yml build sut
  - docker-compose -f docker-compose.test.yml run out


Jenkins 2.0 (and earlier versions with the workflow plugin installed) support configuring a pipeline using a Jenkinsfile in the root of the repo. You must have docker and docker-compose installed on the Jenkins slave (or master) running the pipeline.

Create a new Multibranch Pipeline item and add your repo in the source code section. Run the pipeline and Jenkins will build and test your container.


node {
  checkout scm
  sh """
mkdir -p ./test/results
docker-compose -f docker-compose.test.yml build sut
docker-compose -f docker-compose.test.yml run sut
  step([$class: 'JUnitResultArchiver', testResults: '**/test/results/TEST*.xml'])


Docker containers are a popular packaging and deployment architecture. Images can and should be tested as with other software and there are straight-forward methods available. The approach described here uses simple shell commands to perform tests. The docker-compose.test.yml design allows a separate test image to be constructed, so more complex tests using other languages and frameworks can be used without increasing the size of the image being tested. CI tools that include Docker as part of the test infrastructure should be able to incorporate automated tests as part of the delivery pipeline.

About the Author

Patrick Double profile.

Patrick Double

Principal Technologist

I have been coding since 6th grade, circa 1986, professionally (i.e. college graduate) since 1998 when I graduated from the University of Nebraska-Lincoln. Most of my career has been in web applications using JEE. I work the entire stack from user interface to database.   I especially like solving application security and high availability problems.

Leave a Reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]