Concourse CI - how to run functional tests?

Question

we are in the middle of process migrating from Jenkins to Concourse CI and everything was pretty smooth so far. But now I have the issue, that I don't know how to solve. I would like to get any advices from the community.

What I am trying to do is a job that can run integrational or functional (web) tests using Selenium. There are few issues for us:

1. To run web tests I need to set up the database (and optionally, the searching engine, proxy and etc...) proxy to imitate the production environment as close at possible. Ideally, it should be set up by docker-compose.
2. This database service should run in parallel of my tests
3. This database service should not return anything, neither error or success, because it only starts the database and nothing else
4. My web-tests should not be started until the database is ready
5. This database service should be stopped when all the web-tests were finished

As you can see, it's pretty non-trivial task. Of course, I can create an big uber-container that contains everything I need, but this is bad solution. Another option is to create a shell-script for that, but this is not flexible enough.

Is there any example how I could implement that or good practices for this issue?

Thanks!

Answer 1

Since version 1.3.0 it appears you can run Docker-compose in a task: https://github.com/concourse/concourse/issues/324

This appears to work:

jobs:
  - name: docker-compose
    public: true
    serial: true
    plan:
      - do:
        - task: docker-compose
          timeout: 20m
          privileged: true
          config:
            platform: linux
            image_resource:
              type: docker-image
              source: {repository: "mumoshu/dcind", tag: "latest"}
            run:
              path: sh
              args:
                - -exc
                - |
                  source /docker-lib.sh
                  start_docker
                  docker ps
                  docker-compose version

Answer 2

This is comment from author of Concourse:

There is no Docker binary or socket on the host - they're just running a Garden backend (probably Guardian). Concourse runs at an abstraction layer above Docker, so providing any sort of magic there doesn't really make sense.

The one thing missing post-1.3 is that Docker requires you to set up cgroups yourself. I forgot how annoying that is. I wish they did what Guardian does and auto-configure it, but what can ya do.

So, the full set of instructions is:

Use or a build an image with docker in it, e.g. docker:dind. Run the following at the start of your task: https://github.com/concourse/docker-image-resource/blob/master/assets/common.sh#L1-L40 Spin up Docker with docker daemon &.

Then you can run docker-compose and friends as normal.

The downside of this is that you'll be fetching the images every time. #230 will address that.

In the long run, #324 (comment) is the direction I want to go.

See here https://github.com/concourse/concourse/issues/324

as in the accepted answer, the Slack archive data is deleted (due to Slack limit)

The docker image specialized for the usecase: https://github.com/meAmidos/dcind

Answer 3

It does not sound that complicated to me. I wrote a post on how to get something similar up and running here. I use some different containers for the stack and the test runner and fire up everything from an official docker:dind image with docker-compose installed on it...

Beyond the usual concourse CI stuff of fetching resources etc. Performing a test-run would consist of :

1. Starting the web,rest, and other services with docker-compose up.
2. Starting the Testrunner service and fire the test-suites on the web-page which communicates with the rest layer, which in turn is dependent on the other services for responses.
3. Performing docker-compose down when the test-runner completes and deciding the return-code of the task (0=fail, 1=success) based upon the return code of the test-suite.

To cleanly setup and tear down the stack and test runner you could do something like the below, ( maybe you could use depends if your service is not started when the test begins, for me it works without)

# Setup the SUT stack:
docker-compose up -d
‌‌
# Run the test-runner container outside of the SUT to be able to teardown the SUT when testing is completed:
docker-compose run --rm test-runner --entrypoint '/entrypoint.sh /protractor/project/conf-dev.js --baseUrl=http://web:9000/dist/ --suite=my_suite'
‌‌
# Store the return-code from the tests and teardown:
rc=$?
docker-compose down
echo "exit code = $rc "
kill %1
exit $rc