Why does Concourse `get` a resource after `put`ing it?

Question

When I configure the following pipeline:

resources:
- name: my-image-src
  type: git
  source:
    uri: https://github.com/concourse/static-golang
- name: my-image
  type: docker-image
  source:
    repository: concourse/static-golang
    username: {{username}}
    password: {{password}}

jobs:
- name: "my-job"
  plan:
  - get: my-image-src
  - put: my-image

After building and pushing the image to the Docker registry, it subsequently fetches the image. This can take some time and ultimately doesn't really add anything to the build. Is there a way to disable it?

score 14 · Accepted Answer · edited Mar 13 '18 at 07:26

Every put implies a get of the version that was created. There are a few reasons for this:

The primary reason for this is so that the newly created resource can be used by later steps in the build plan. Without the get there is no way to introduce "new" resources during a build's execution, as they're all resolved to a particular version to fetch when the build starts.

There are some side-benefits to doing this as well. For one, it immediately warms the cache on one worker. So it's at least not totally worthless; later jobs won't have to fetch it. It also acts as validation that the put actually had the desired effect.

In this particular case, as it's the last step in the build plan, the primary reason doesn't really apply. But we didn't bother optimizing it away since in most cases the side benefits make it worth not having the secondary question arise ("why do only SOME put steps imply a get?").

It also cannot be disabled as we resist adding so many knobs that you'll want to turn one day and then have to go back and turn back off once you actually do need it back to the default.

Docs: https://concourse-ci.org/put-step.html

Isn't the `skip_download` parameter already a "knob" for this? Does it avoid the redundant fetch? If not, could it? — timbod, Sep 12 '16 at 08:28
Yeah. Difference being that `skip_download` is implemented by the Docker Image resource in particular, and isn't a general concept across all resources. It was also added for a different reason: sometimes folks are just propagating an image ID through. It happens to be useful in this case, if you're familiar with that resource, but it's not something to worry about constantly in all of Concourse. — Alex Suraci, Sep 24 '16 at 00:03

Why does Concourse `get` a resource after `put`ing it?

1 Answers1