Stuctured logs in shell script using JQ

Question

I've been wanting to logs some shells scripts to a file to be picked up by an agent like Fluentbit and shipped off to Cloudwatch and Datadog.

I found this example on the web which works like a charm using jq.

__timestamp(){
  date "+%Y%m%dT%H%M%S"
}
__log(){
  log_level="$1"
  message="$2"
  echo '{}' | jq \
    --arg timestamp "$(__timestamp)"
    --arg log_level "$log_level" \
    --arg message "$message" \
    '.timestamp=$timestamp|.log_level=$log_level|.message=$message' >> logs.log
}
__log "INFO" "Hello, World!"

The one issue is that the output is rendering a full json with newlines and tabs. Easy on the eyes, but not great for cloud logging services.

{
  "timestamp": "20220203T162320",
  "log_level": "INFO",
  "message": "Hello, World!"
}

How would I modify to render the output on one line like so?

{"timestamp": "20220203T171908","log_level": "INFO","message": "Hello, World!"}

Answer 1

Use the --compact-output or -c option: jq -c --arg … '…' >> logs.log

From the manual:

--compact-output / -c:
    By default, jq pretty-prints JSON output. Using this option will
    result in more compact output by instead putting  each JSON object
    on a single line.

---

After having read your linked article about "Structured Logging", this is how I would have implemented said __log function:

__log(){
  jq -Mcn --arg log_level "$1" --arg message "$2" \
    '{timestamp: now|todate, $log_level, $message}'
}
__log "INFO" "Hello, World!"

Answer 2

🤦 Somehow I completely missed Step 3.0 The Finishing Touches of this terrific guide which answers my question perfectly.

In case anyone else was curious, the solution is here:

__timestamp(){
  date "+%Y%m%dT%H%M%S"
}
__log(){
  log_level="$1"
  message="$2"
  echo '{}' | \
  jq  --monochrome-output \
      --compact-output \
      --raw-output \
      --arg timestamp "$(__timestamp)" \
      --arg log_level "$log_level" \
      --arg message "$message" \
      '.timestamp=$timestamp|.log_level=$log_level|.message=$message'
}
__log "INFO" "Hello, World!"

Which outputs:

{"timestamp":"20210812T191730","log_level":"INFO","message":"Hello, World!"}