11

I want to build a docker image in my pipeline and then run a job inside it, without pushing or pulling the image.

Is this possible?

Robert Columbia
  • 6,012
  • 14
  • 28
  • 36
Shawabawa
  • 2,279
  • 1
  • 13
  • 16
  • It is not possible at present. I have created an image repository in ECR to which I push images which then get ignored :-( – Rich Feb 17 '21 at 09:24

2 Answers2

3

It's by design that you can't pass artifacts between jobs in a pipeline without using some kind of external resource to store it. However, you can pass between tasks in a single job. Also, you specify images on a per-task level rather than a per-job level. Ergo, the simplest way to do what you want may be to have a single job that has a first task to generate the docker-image, and a second task which consumes it as the container image.

In your case, you would build the docker image in the build task and use docker export to export the image's filesystem to a rootfs which you can put into the output (my-task-image). Keep in mind the particular schema to the rootfs output that it needs to match. You will need rootfs/... (the extracted 'docker export') and metadata.json which can just contain an empty json object. You can look at the in script within the docker-image-resource for more information on how to make it match the schema : https://github.com/concourse/docker-image-resource/blob/master/assets/in. Then in the subsequent task, you can add the image parameter in your pipeline yml as such:

- task: use-task-image
  image: my-task-image
  file: my-project/ci/tasks/my-task.yml

in order to use the built image in the task.

Clara Fu
  • 130
  • 1
  • I've tried doing this but I cannot get a task to successfully build the image. As far as I know tasks have to run in docker, which means it will have to run docker build within docker which I haven't managed to work out – Shawabawa Dec 13 '16 at 14:52
  • You might want to check out a [tutorial](http://engineering.pivotal.io/post/concourse-docker-image-resource-cache-tutorial/) we wrote on using the `docker-image-resource` – materialdesigner Dec 15 '16 at 23:28
  • 4
    Again that pushes the image. I want to be able to create a docker container and use it *without* pushing – Shawabawa Dec 20 '16 at 13:56
  • This answer is very nearly super useful. If you could add some detail or link to some detail around how to use "docker export" it would be complete and fantastic – The Trav May 29 '17 at 05:41
  • 1
    @materialdesigner: The OP is spot on here, you're pushing the image - the whole question is about how to _not_ push the image, but build it and use it for testing. (Why push it if it fails tests?) -- Unless we're misunderstanding you, but your tutorial uses a `put`, - which pushes it to the registry. – damick Aug 26 '17 at 00:14
0

UDPATE: the PR was rejected

This answer doesn't currently work, as the "dry_run" PR was rejected. See https://github.com/concourse/docker-image-resource/pull/185

I will update here if I find an approach which does work.

The "dry_run" parameter which was added to the docker resource in Oct 2017 now allows this (github pr)

You need to add a dummy docker resource like:

resources:
  - name: dummy-docker-image
    type: docker-image
    icon: docker
    source:
      repository: example.com
      tag: latest
  - name: my-source
    type: git
    source:
      uri: git@github.com:me/my-source.git

Then add a build step which pushes to that docker resource but with "dry_run" set so that nothing actually gets pushed:

jobs:
  - name: My Job
    plan:
      - get: my-source
        trigger: true
      - put: dummy-docker-image
        params:
          dry_run: true
          build: path/to/build/scope
          dockerfile: path/to/build/scope/path/to/Dockerfile
Rich
  • 13,254
  • 1
  • 56
  • 102