CODEWITHSUNDEEP
concourse
JointEffort
JointEffort

Reputation: 643

How to parameterise concourse task files

I'm pretty impressed by the power and simplicity of Concourse. Since my pipelines keep growing I decided to move the tasks to separate files. One of the tasks use a custom Docker image from our own private registry. So, in that task file I have:

image_resource:
  type: docker-image
  source:
    repository: docker.mycomp.com:443/app-builder
    tag: latest
    username: {{dckr-user}}
    password: {{dckr-pass}}

When I do a set-pipeline, I pass the --load-from-vars argument to load credentials etc from a seperate file.

Now here's my problem: I notice that the vars in my pipeline files are replaced with the actual correct values, but once the task runs, the afore mentioned {{dckr-user}} and {{dckr-pass}} are not replaced.

How do I achieve this?

Upvotes: 3

Views: 3416

Answers (3)

sercanturkmen
sercanturkmen

Reputation: 127

As of concourse v3.3.0, you can set up Credential Management in order to use variables from one of the supported credential managers which are currently Vault, Credhub, Amazon SSM, and Amazon Secrets Manager. So you don't have to separate your task files partially in the pipeline.yml anymore. The values you set in the Vault will be also accessible from the task.yml files.

And since v3.2.0 {{foo}} is deprecated in favor of ((foo)).

Using the Credential Manager you can parameterize:

  • source under resources in a pipeline
  • source under resource_types in a pipeline
  • webhook_token under resources in a pipeline
  • image_resource.source under image_resource in a task config
  • params in a pipeline
  • params in a task config

For setting up vault with concourse you can refer to:

https://concourse-ci.org/creds.html

Upvotes: 2

materialdesigner
materialdesigner

Reputation: 1522

In addition to what was provided in this answer

If specifically you are looking to use private images in a task, you can do the following in your pipeline.yml:

resources:
- name: some-private-image
  type: docker
  params:
    repository: ...
    username: {{my-username}}
    password: {{my-password}}

jobs:
- name: foo
  plan:
  - get: some-private-image
  - task: some-task
    image: some-private-image

Because this is your pipeline, you can use --load-vars-from, which will first get your image as a resource and then use it for the subsequent task.

You can also see this article on pre-fetching ruby gems in test containers on Concourse

The only downside to this is you cannot use this technique when running a fly execute.

Upvotes: 6

Josh Zarrabi
Josh Zarrabi

Reputation: 1064

You can always define tasks in a pipeline.yml... For example:

jobs:
- name: dotpersecond
  plan:
  - task: dotpersecond
    config:
      image_resource:
        type: docker-image
        source:
          repository: docker.mycomp.com:443/app-builder
          tag: latest
          username: {{dckr-user}}
          password: {{dckr-pass}}
      run:
        path: sh
        args:
          - "-c"
          - |
            for i in `seq 1000`; do echo hi; sleep 2; done

Upvotes: 1

Related Questions