CODEWITHSUNDEEP
jsonbashyamlgithub-actionsjsonparser
WubbaLubbaDubbDubb
WubbaLubbaDubbDubb

Reputation: 301

GitHub Actions: How to pass toJSON() result to shell commands

So, I'm working with Github Actions on end-to-end testing. The setup I'm looking at is having one job retrieve a list of urls to be tested, and my second job creates a matrix with that list and tests them all. My problem here is that when I actually run my testing script, it has to be done from the command line, because I'm using Playwright. Therefore I can't use my matrix object directly; I have to output it to a JSON file. The problem is that toJSON creates invalid pretty-printed JSON when I output it to my file, which breaks my script. Here's my code:

name: <name>

on:
    push:
    workflow_dispatch:

jobs:
    fetch_strategic_urls:
        runs-on: ubuntu-latest
        outputs:
            urls: ${{ steps.req-urls.outputs.urls }}
        steps:
            - name: Request Urls
              id: req-urls
              run: |
                  export RESPONSE=$(
                    curl -X GET -H "Accept: application/json" <api-endpoint>)
                  echo "::set-output name=urls::$RESPONSE"
    run_tests:
        runs-on: ubuntu-latest
        strategy:
            matrix:
                url: ${{needs.fetch_strategic_urls.outputs.urls}}
        needs: fetch_strategic_urls
        steps:
            - ...
            - ...
            - run: |
                  ls
                  echo '${{ toJSON(matrix.url) }}' >> props.json
                  cat props.json
                  npm test
              working-directory: E2E.Tests

No matter which configuration of echo ${{matrix.url}} >> props.json I've tried (cat <<'EOF' > props.json ${{matrix.url}}, adding and removing quotes), it always produced JSON files that have no quotes, i.e.: { url: string } instead of {"url": "string"}, which is invalid. This is obviously pretty breaking behavior. I've seen a lot of people online recommending jq, but I don't see how I would use it in this case, since I doubt jq can parse a GitHub-type JSON object, which is necessary for me to use when sharding my jobs. Any help is greatly appreciated!

Upvotes: 6

Views: 20443

Answers (6)

glandium
glandium

Reputation: 1

Here is a version that should work whatever the serialized contents are (whether they contain quotes, shell special characters or whatever):

  - name: Dump GitHub context
    run: |
      cat << 'EOF'
      ${{ toJSON(github) }}
      EOF

Upvotes: 0

Max Cascone
Max Cascone

Reputation: 833

I've found that the output of toJSON() is parseable as YAML.

jobs:
  job1:
    runs-on: ubuntu-latest

    outputs:
      my_output: ${{ steps.out1.outputs.my_output }}
      output_2:  ${{ steps.out2.outputs.output_2  }}

    steps:
      - name: output one line
        id: out1
        run: echo 'my_output="Hello, world!"' >> $GITHUB_OUTPUT

      - name: output two line
        id: out2
        run: |
          echo 'output_2="Hello, chicago!"' >> $GITHUB_OUTPUT

     
  job2:
    needs: job1
    runs-on: ubuntu-latest

    steps:
      - name: get all outputs from prior job
        run: |
          echo "The outputs are ${{ toJSON(needs.job1.outputs) }}"
          echo
          jobjson=$(echo "${{ toJSON(needs.job1.outputs) }}" )
          echo $jobjson | yq
          echo
          echo "my_output is $(echo $jobjson | yq .my_output)"

In the console:

| The outputs are {
|   my_output: "Hello, world!",
|   output_2: "Hello, chicago!"
| }
| 
| {my_output: "Hello, world!", output_2: "Hello, chicago!"}
| 
| my_output is Hello, world!

Upvotes: 0

pynexj
pynexj

Reputation: 20698

It's not easy to put a JSON doc directly in the command line. You can use env vars.

- shell: bash
  env:
    JSON_DOC: ${{ toJSON(some.var) }}
  run: |
    printf '%s\n' "$JSON_DOC" > foo.json
    cat foo.json
    ...

Upvotes: 8

Marco Di Croce
Marco Di Croce

Reputation: 1

You can use echo with single quotes:

- name: Print inputs
  run: |
  echo 'Inputs: ${{ toJson(inputs) }}'

Upvotes: 0

Bruce Edge
Bruce Edge

Reputation: 2189

This isn't an answer, or even an attempt, more a place to post about a related GH actions oddity.

I thought I had the above solved with:

- name: step-name
  run: | # shell
    echo "${{ toJson(github) }}"

which, worked fine, until I merged to main, the default branch, at which point it generates this error:

##[debug]/usr/bin/bash -e /home/runner/work/_temp/11b0a2ad-c8d6-4251-8b4a-6ab18d3de45f.sh
/home/runner/work/_temp/11b0a2ad-c8d6-4251-8b4a-6ab18d3de45f.sh: line 58: syntax error near unexpected token `('
Error: Process completed with exit code 2.

Despite that offending statment ( ${{ toJson(github) }} ) being no where near line 58.

Note that one can't even have this as a comment:

# echo ${{ toJson(github) }}

as that still fails the job, apparently with something in the github object, that is only present in the default branch.

All I can think of is that the returned string from toJSON() isn't always correctly escaped, so it is breaking the YAML parser's ability to extract a legal shell script from the action yaml.

Upvotes: 1

alliannas
alliannas

Reputation: 334

Github actions documenation has a solution here:

name: Context testing
on: push

jobs:
  dump_contexts_to_log:
    runs-on: ubuntu-latest
    steps:
      - name: Dump GitHub context
        id: github_context_step
        run: echo '${{ toJSON(github) }}'

From https://docs.github.com/en/actions/learn-github-actions/contexts#example-printing-context-information-to-the-log

Upvotes: 3

Related Questions