The previous Magazine article on container testing showed how to use Meta-Test-Family (MTF) to validate  standalone containers. The goal is to avoid shipping containers without proper testing, and to guarantee that a given service in a container works properly.

Another possible way to test containers is in the OpenShift environment. This article describes how to test container in the orchestrated OpenShift world. Fedora alone already includes a huge number of containers.

MTF installation

Before you can start with the testing itself, install MTF from the official Fedora repository using sudo:

sudo dnf install -y meta-test-family

A COPR repository contains a development version of MTF which should not be used on a production environment. However, you can install it with these commands:

sudo dnf copr enable phracek/meta-test-family
sudo dnf install -y meta-test-family

To install MTF directly from GitHub, run these commands:

git clone git@github.com:fedora-modularity/meta-test-family.git
cd meta-test-family
sudo python setup.py install

Now you can start testing containers in the OpenShift environment.

Prepare test for the OpenShift

Running your containers locally is dead-simple — just use the docker run command. But that’s not how you run your application in production — that’s OpenShift’s business. To make sure your containers are orchestrated well, you should test them in the same environment.

Bear in mind that standalone and orchestrated environments are different. Standalone containers can be executed easily with a single command. Managing such containers isn’t as easy: you need to figure out persistent storage, backups, updates, routing, scaling — all the things you get for free with orchestrators.

The OpenShift environment has its own characteristics: security restrictions, differences in persistent storage logic, expectation of stateless pods, support for updates, multi-node environment, native source-to-image support and much much more.  Deploying an orchestrator here is not an easy task. This is the reason why MTF now supports OpenShift, so you can easily test your containerized application in an orchestrated environment.

Before running and preparation the OpenShift environment, you have to create a test and a configuration file for MTF, in YAML format. These two files have to be in the same directory, and tests will be executed from the directory.

Configuration file for MTF

The configuration file looks like this:

document: modularity-testing
version: 1
name: memcached
service:
    port: 11211
module:
    openshift:
        container: docker.io/modularitycontainers/memcached

Here’s an explanation of each field in the YAML config file for MTF:

  • service.port – port, where the service is available.
  • module.openshift – configuration part relevant only for OpenShift environment
  • module.openshift.container – Reference to container, which will be used for testing in OpenShift.

Test for memcached container

Here’s an example of a memcached test for a container:

$ cat sanity1.py
import pexpect
from avocado import main
from avocado.core import exceptions
from moduleframework import module_framework
from moduleframework import common

class SanityCheck1(module_framework.AvocadoTest):

"""
:avocado: enable
"""
def test_smoke(self):
    self.start()
    session = pexpect.spawn("telnet %s %s " % (self.ip_address,self.getConfig()['service']['port']))
    session.sendline('set Test 0 100 4\r\n\n')
    session.sendline('JournalDev\r\n\n')
    common.print_info("Expecting STORED")
    session.expect('STORED')
    common.print_info("STORED was catched")
    session.close()

if __name__ == '__main__':
    main()

This test connects to memcached via telnet on the given IP address and port. The port is specified in the MTF configuration file. The following sections speak more about the IP address.

Prepare OpenShift for container testing

MTF can install the OpenShift environment on your local system with the mtf-env-set command.

$ sudo MODULE=openshift OPENSHIFT_LOCAL=yes mtf-env-set
Setting environment for module: openshift
Preparing environment ...
Loaded config for name: memcached
Starting OpenShift
Starting OpenShift using openshift/origin:v3.6.0 ...
OpenShift server started.

The server is accessible via web console at:
https://127.0.0.1:8443

You are logged in as:
User: developer
Password: <any value>

To login as administrator:
oc login -u system:admin

The mtf-env-set command checks for a shell variable OPENSHIFT_LOCAL — if specified, it checks if the origin and origin-clients packages are installed. If not, then it installs them.

In this case, a local machine performs the container testing. If you test containers on a remote OpenShift instance, you can ignore this step. If the OPENSHIFT_LOCAL variable is missing, tests are executed on the remote OpenShift instance specified by the OPENSHIFT_IP parameter (see below).

Container testing

Now you can test your container either on a local or remote OpenShift instance with the mtf command. The only difference is the command parameters:

sudo MODULE=openshift OPENSHIFT_USER=developer OPENSHIFT_PASSWORD=developer mtf sanity1.py

In this local testing case, sanity1.py uses 127.0.0.1 as the value for self.ip_address.

sudo OPENSHIFT_IP=<ip_address> OPENSHIFT_USER=<username> OPENSHIFT_PASSWD=<passwd> mtf sanity1.py

In this remote testing case, sanity1.py uses OPENSHIFT_IP as the value for self.ip_address.

Tests are then executed from the environment where you store configuration file and tests for the given OpenShift instance. The output looks like this:

JOB ID : c2b0877ca52a14c6c740582c76f60d4f19eb2d4d
JOB LOG : /root/avocado/job-results/job-2017-12-18T12.32-c2b0877/job.log
(1/1) sanity1.py:SanityCheck1.test_smoke: PASS (13.19 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 13.74 s
JOB HTML : /root/avocado/job-results/job-2017-12-18T12.32-c2b0877/results.html
$

If you open the log file marked in boldface above, you’ll see contents similar to the example below.

[...snip...]
['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
2017-12-18 14:29:36,208 job L0321 INFO | Command line: /bin/avocado run --json /tmp/tmppfZpNe sanity1.py
2017-12-18 14:29:36,208 job L0322 INFO |
2017-12-18 14:29:36,208 job L0326 INFO | Avocado version: 55.0
2017-12-18 14:29:36,208 job L0342 INFO |
2017-12-18 14:29:36,208 job L0346 INFO | Config files read (in order):
2017-12-18 14:29:36,208 job L0348 INFO | /etc/avocado/avocado.conf
2017-12-18 14:29:36,208 job L0348 INFO | /etc/avocado/conf.d/gdb.conf
2017-12-18 14:29:36,208 job L0348 INFO | /root/.config/avocado/avocado.conf
2017-12-18 14:29:36,208 job L0353 INFO |
2017-12-18 14:29:36,208 job L0355 INFO | Avocado config:
2017-12-18 14:29:36,209 job L0364 INFO | Section.Key 
[...snip...]

:::::::::::::::::::::::: SETUP ::::::::::::::::::::::::

2017-12-18 14:29:36,629 avocado_test L0069 DEBUG|

:::::::::::::::::::::::: START MODULE ::::::::::::::::::::::::

MTF verifies whether the application exists in the OpenShift environment:

2017-12-18 14:29:36,629 process L0389 INFO | Running 'oc get dc memcached -o json'
2017-12-18 14:29:36,842 process L0479 DEBUG| [stderr] Error from server (NotFound): deploymentconfigs.apps.openshift.io "memcached" not found
2017-12-18 14:29:36,846 process L0499 INFO | Command 'oc get dc memcached -o json' finished with 1 after 0.213222980499s

In the next step MTF verifies whether the pod exists in OpenShift:

2017-12-18 14:29:36,847 process L0389 INFO | Running 'oc get pods -o json'
2017-12-18 14:29:37,058 process L0479 DEBUG| [stdout] {
2017-12-18 14:29:37,059 process L0479 DEBUG| [stdout] "apiVersion": "v1",
2017-12-18 14:29:37,059 process L0479 DEBUG| [stdout] "items": [],
2017-12-18 14:29:37,059 process L0479 DEBUG| [stdout] "kind": "List",
2017-12-18 14:29:37,059 process L0479 DEBUG| [stdout] "metadata": {},
2017-12-18 14:29:37,059 process L0479 DEBUG| [stdout] "resourceVersion": "",
2017-12-18 14:29:37,059 process L0479 DEBUG| [stdout] "selfLink": ""
2017-12-18 14:29:37,060 process L0479 DEBUG| [stdout] }
2017-12-18 14:29:37,064 process L0499 INFO | Command 'oc get pods -o json' finished with 0 after 0.211796045303s

The next step creates an application with the given label mtf_testing and with the name taken from the config.yaml file in the container tag.

2017-12-18 14:29:37,064 process L0389 INFO | Running 'oc new-app -l mtf_testing=true docker.io/modularitycontainers/memcached --name=memcached'
2017-12-18 14:29:39,022 process L0479 DEBUG| [stdout] --> Found Docker image bbc8bba (5 weeks old) from docker.io for "docker.io/modularitycontainers/memcached"
2017-12-18 14:29:39,022 process L0479 DEBUG| [stdout]
2017-12-18 14:29:39,022 process L0479 DEBUG| [stdout] memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
2017-12-18 14:29:39,022 process L0479 DEBUG| [stdout]
2017-12-18 14:29:39,022 process L0479 DEBUG| [stdout] Tags: memcached
2017-12-18 14:29:39,023 process L0479 DEBUG| [stdout]
2017-12-18 14:29:39,023 process L0479 DEBUG| [stdout] * An image stream will be created as "memcached:latest" that will track this image
2017-12-18 14:29:39,023 process L0479 DEBUG| [stdout] * This image will be deployed in deployment config "memcached"
2017-12-18 14:29:39,023 process L0479 DEBUG| [stdout] * Port 11211/tcp will be load balanced by service "memcached"
2017-12-18 14:29:39,023 process L0479 DEBUG| [stdout] * Other containers can access this service through the hostname "memcached"
2017-12-18 14:29:39,023 process L0479 DEBUG| [stdout]
2017-12-18 14:29:39,023 process L0479 DEBUG| [stdout] --> Creating resources with label mtf_testing=true ...
2017-12-18 14:29:39,032 process L0479 DEBUG| [stdout] imagestream "memcached" created
2017-12-18 14:29:39,043 process L0479 DEBUG| [stdout] deploymentconfig "memcached" created
2017-12-18 14:29:39,063 process L0479 DEBUG| [stdout] service "memcached" created
2017-12-18 14:29:39,064 process L0479 DEBUG| [stdout] --> Success
2017-12-18 14:29:39,064 process L0479 DEBUG| [stdout] Run 'oc status' to view your app.
2017-12-18 14:29:39,069 process L0499 INFO | Command 'oc new-app -l mtf_testing=true docker.io/modularitycontainers/memcached --name=memcached' finished with 0 after 2.00025391579s

The next step verifies whether the application is really running and on which IP address it’s reachable:

2017-12-18 14:29:46,201 process L0389 INFO | Running 'oc get service -o json'
2017-12-18 14:29:46,416 process L0479 DEBUG| [stdout] {
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] "apiVersion": "v1",
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] "items": [
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] {
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] "apiVersion": "v1",
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] "kind": "Service",
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] "metadata": {
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] "annotations": {
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] "openshift.io/generated-by": "OpenShiftNewApp"
2017-12-18 14:29:46,417 process L0479 DEBUG| [stdout] },
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] "creationTimestamp": "2017-12-18T13:29:39Z",
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] "labels": {
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] "app": "memcached",
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] "mtf_testing": "true"
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] },
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] "name": "memcached",
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] "namespace": "myproject",
2017-12-18 14:29:46,418 process L0479 DEBUG| [stdout] "resourceVersion": "2121",
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] "selfLink": "/api/v1/namespaces/myproject/services/memcached",
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] "uid": "7f50823d-e3f7-11e7-be28-507b9d4150cb"
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] },
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] "spec": {
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] "clusterIP": "172.30.255.42",
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] "ports": [
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] {
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] "name": "11211-tcp",
2017-12-18 14:29:46,419 process L0479 DEBUG| [stdout] "port": 11211,
2017-12-18 14:29:46,420 process L0479 DEBUG| [stdout] "protocol": "TCP",
2017-12-18 14:29:46,420 process L0479 DEBUG| [stdout] "targetPort": 11211
2017-12-18 14:29:46,420 process L0499 INFO | Command 'oc get service -o json' finished with 0 after 0.213701963425s
2017-12-18 14:29:46,420 process L0479 DEBUG| [stdout] }
2017-12-18 14:29:46,420 process L0479 DEBUG| [stdout] ],
2017-12-18 14:29:46,420 process L0479 DEBUG| [stdout] "selector": {
2017-12-18 14:29:46,420 process L0479 DEBUG| [stdout] "app": "memcached",
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] "deploymentconfig": "memcached",
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] "mtf_testing": "true"
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] },
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] "sessionAffinity": "None",
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] "type": "ClusterIP"
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] },
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] "status": {
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] "loadBalancer": {}
2017-12-18 14:29:46,421 process L0479 DEBUG| [stdout] }
2017-12-18 14:29:46,422 process L0479 DEBUG| [stdout] }
2017-12-18 14:29:46,422 process L0479 DEBUG| [stdout] ],
2017-12-18 14:29:46,422 process L0479 DEBUG| [stdout] "kind": "List",
2017-12-18 14:29:46,422 process L0479 DEBUG| [stdout] "metadata": {},
2017-12-18 14:29:46,422 process L0479 DEBUG| [stdout] "resourceVersion": "",
2017-12-18 14:29:46,422 process L0479 DEBUG| [stdout] "selfLink": ""
2017-12-18 14:29:46,422 process L0479 DEBUG| [stdout] }

In the last phase, tests are executed.

2017-12-18 14:29:46,530 output L0655 DEBUG| Expecting STORED
2017-12-18 14:29:46,531 output L0655 DEBUG| STORED was catched
2017-12-18 14:29:46,632 avocado_test L0069 DEBUG|

:::::::::::::::::::::::: TEARDOWN ::::::::::::::::::::::::

2017-12-18 14:29:46,632 process L0389 INFO | Running 'oc get dc memcached -o json'
2017-12-18 14:29:46,841 process L0479 DEBUG| [stdout] {
2017-12-18 14:29:46,841 process L0479 DEBUG| [stdout] "apiVersion": "v1",
2017-12-18 14:29:46,841 process L0479 DEBUG| [stdout] "kind": "DeploymentConfig",
2017-12-18 14:29:46,841 process L0479 DEBUG| [stdout] "metadata": {

At the end of the tests, you can verify whether the service is running in OpenShift environment using the command oc status:

$ sudo oc status
In project My Project (myproject) on server https://127.0.0.1:8443

You have no services, deployment configs, or build configs.
Run 'oc new-app' to create an application.

From this output, you can see that you can test an arbitrary container and afterward, the OpenShift environment is cleared.

Summary

As you may have seen in this aricle, writing tests for containers is really easy. This helps you guarantee that a container is working properly just as an RPM package would. In the near future, MTF plans to extend its capabilities with S2I testing, and testing containers with OpenShift templates. You can read more in the MTF documentation.