92

I'm trying to write a gitlab-ci.yml file which uses a multi-line string for the command. However, it seems like it is not being parsed. I've tried both the - | and - > with identical results.

stages:
  - mystage

Build:
  stage: mystage
  script:
    - |
        echo -e "
            echo 'hi';
            echo 'bye';
        "

When it tries to run, it only shows echo -e ' as the script to run, and not the whole multiline string. This causes issues for me.

What would be the correct syntax to write something like this?

Anthon
  • 51,019
  • 25
  • 150
  • 211
samanime
  • 21,211
  • 7
  • 69
  • 122
  • There's an issue for this: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/166 It's not clear to me what the problem is, since your code *should* be equivalent (enough) YAML to the solutions proposed there. You might try appending ``\`` to your lines, but I can't say if that'll work or not. – Jordan Running Mar 02 '17 at 17:48

6 Answers6

120

I came here preemptively expecting this would be an issue but the following "multi-line" command for readability is working for me:

Gitlab Runner: Shell Runner version 1.11.0 / Gitlab version: 8.17.2

myjob:
stage: deploy
script:
  # Single line command
  - az component update --add sql

  # Multi-line command
  - az sql server create -n ${variable} -g ${variable} -l ${variable}
    --administrator-login ${variable} --administrator-login-password ${variable}
PotatoFarmer
  • 1,946
  • 2
  • 10
  • 18
  • 3
    What's the trick here? Did you indent the second line to the same level as the first line? – Victor Grazi Feb 08 '19 at 03:00
  • 6
    @victor-grazi As I understand it: In plain YAML (plain flow scalar), escapes (such as newline `\n`) don't do anything, and leading white space is ignored - it appears Gitlab YAML parses script blocks in this fashion. On indentation: YAML spec says `In YAML block styles, structure is determined by indentation` and so the second line is indented as much as required for YAML spec (one space relative to parent indent), and one more for readability (which is technically superfluous but prettier). – PotatoFarmer Feb 09 '19 at 08:25
  • Works like a charm. Also works with all params in new line – bodolsog Aug 21 '19 at 09:45
47

TL;DR; You want to use a multi-line YAML scalar (for readability) that is loaded as a single line string that can be issued as a command by Gitlab-CI. To do so use a plain (without quotes) scalar in YAML that is spread out over multiple lines:

script:
- echo -e 
   "echo 'hi';
    echo 'bye';"

Please be aware that there are some restrictions imposed by YAML on such scalars. What you certainly need to know is that each following line is indented at least one more position than echo -e (which is indented two positions relative to its collection node, which is not indented at all), and that every new-line is replaced by a space when loaded (so you need to take a bit care of where to put newlines).


There are multiple misconceptions in your post, that lead to you asking the wrong question.

There is no such thing as a multi-line YAML string. YAML has scalars and some of these scalars can be loaded by a program as strings, while some others will be loaded as integers, floats, etc.

You are obviously interested in scalar nodes that are being loaded as a string, since that string can be then be interpreted as a command-line. But you don't want to have multi-line command-line (i.e. with embedded newlines), since multi-line scripts are not supported in Gitlab CI (as @Jordan indicated).

For readability you want to use the, standard, capability of YAML to load multi-line scalars as single line string.

If you wouldn't care about readability you could use:

- echo -e "\n    echo 'hi';\n    echo 'bye';\n"

and since your scalar is not quoted (i.e. it starts with echo) you don't need to do anything special in YAML for the backslashes or quotes.

The result of the script is the same (print an empty line, print echo 'hi'; on a line indented four spaces, print echo 'bye'; on a line indented four spaces.)

If you want to use the multi-line input for readability, that are loaded as a single line, there are essentially two options: use a multi-line plane scalar or use a folded scalar in your YAML.

multi-line plain scalar

Plain means the scalar is non-quoted, and as with any multi-line thing in YAML multi-line means following lines need to be indented appropriately, in this case further than the initial line

script:
- echo -e 
   "echo 'hi';
    echo 'bye';"

newlines are replaced by spaces so don't do:

script:
- echo -e 
   "echo 'hi';
    echo '
   bye';"

as you will get a visible space before bye.

There are some restrictions like that you cannot have a colon followed by a space within such a scalar (which would make it look like key-value pair).

There is no need to escape backslashes in plain scalars, as you cannot escape any characters in a plain scalar, but of course you can include a backslash, which will end up in the string loaded from the YAML and can have meaning for the command executed from that string.

folded scalar

A folded scalar is similar to a plain scalar in that all (single) newlines are substituted by a space during loading:

script:
- >
  echo -e 
  "echo 'hi';
  echo 'bye';"

You need to indent the actual command information at least as much as the folded scalar indicator (>).

Contrary to plain scalars things like : have no special meaning. So if plain scalars fail by throwing a YAML error, similar folded scalars most likely won't.

Anthon
  • 51,019
  • 25
  • 150
  • 211
  • I want to write it multi-line for clarity and maintainability. While my example is trivial, the real scripts are decidedly not. – samanime Mar 08 '17 at 15:06
  • I can understand that. Would it be acceptable to preprocess your readable YAML file before it is being processed by GitLab CI? – Anthon Mar 08 '17 at 15:59
  • I've considered that. It's an extra step and a bit added complexity, but may be worth it. – samanime Mar 08 '17 at 16:02
  • I added a possible solution. – Anthon Mar 08 '17 at 22:08
  • 4
    **Oh, boy.** While technically correct, this answer is ludicrously verbose to the point of unreadability. Everyone not writing a YAML parser probably just wants [PotatoFarmer](https://stackoverflow.com/users/7650275/potatofarmer)'s [highly upvoted and *much* terser answer](https://stackoverflow.com/a/42705098/2809027), instead. – Cecil Curry Nov 29 '19 at 07:15
26

You can use any multiline scripts/commands via yaml literal_block and anchors feature. Example:

.build: &build |
    echo -e "\n$hl Building $green$build_path/$build_assets_dir/*.js $nl\n"
    echo -e "javascript-obfuscator $build_path/$build_assets_dir/*.js"
[...]

build:master: 
  stage: build
  script:
    - *rsync
    - *build
[...]
Benny K
  • 859
  • 9
  • 8
  • Thank you for sharing - this more advanced functionality will be particularly useful for readability of the job/being able to reuse code chunks throughout the recipe. – PotatoFarmer Dec 19 '17 at 17:37
  • 5
    This is a great example, but it would be clearer if you define .rsync – Victor Grazi Jan 30 '19 at 19:25
13

The wp config create command was pretty finicky... from the .gitlab-ci...

build:
  stage: build
  script:
    - echo "Building the app"
    - |
        wp config create --dbname=$vardb --dbhost=$varhost --dbuser=$varusr --dbpass=$varpas --extra-php <<PHP
            define( 'WP_DEBUG', false );
            define( 'FS_METHOD', 'direct' );
            define( 'WP_POST_REVISIONS', 5 );
            define( 'AUTOSAVE_INTERVAL', 600 );
        PHP
    - scp ./wp-config.php continued...
  allow_failure: true
mal
  • 171
  • 1
  • 4
4

This working for me in Travis CI

before_install:
  - set -e
  - |
    echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <settings xmlns=\"http://maven.apache.org/SETTINGS/1.0.0\"
              xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"
              xsi:schemaLocation=\"http://maven.apache.org/SETTINGS/1.0.0
                                   http://maven.apache.org/xsd/settings-1.0.0.xsd\">
      <servers>
        <server>
          <id>github</id>
          <username>${GITHUB_USERNAME}</username>
          <password>${GITHUB_PASSWORD}</password>
        </server>
      </servers>
    </settings>
    " >  ${HOME}/.m2/settings.xml

Here two env variables (${GITHUB_USERNAME} and ${GITHUB_PASSWORD}) will be also interpolated

Maksim Kostromin
  • 2,243
  • 1
  • 23
  • 21
0

This format will work. use a plain (without quotes) scalar in YAML. Eg script used for initializing terraform backend

  before_script:
    - cd ${TF_ROOT}
    - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}"
      -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock"
      -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock"
      -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_ACCESS_TOKEN}"
      -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE"
      -backend-config="retry_wait_min=5"
Jobin James
  • 522
  • 4
  • 10