CODEWITHSUNDEEP
jsonnode.jsgoconfiginject
JSmith
JSmith

Reputation: 143

Inject variables into JSON config file

We have an AWS Lambda function in place that dynamically builds test environments for our engineers to test their code through GitHub pull requests.

This Lambda function is called using GitHub webhooks, whereby Github POSTs across all the information needed to configure the test environment for that specific product.

At the moment, the AWS Lambda function is hard coded to parse the POST data and build the test environment. However, as the range of products being tested increases and the range of test environments become more diverse, we are wanting to move away from the hard coded approach; for manageability sake too.

What I want to be able to do, is load a configuration file, preferably in JSON and apply some of the data in the GitHub POST data to the configuration file, essentially injecting the data into the JSON config.

I don't know how to approach this. I've seen a question being asked on here wanting to do something similar, but in Java: Inject dynamically generated data into JSON

Originally our Lambda function was written in Node.js, but we've started to move across to Go - mainly because it was an interesting new challenge. If there is a solution to this in Node.js I'll take that, but if there is a solution available in Go, that would be preferable.

Edit:

The config file and the GitHub POST data have two different structures.

GitHub POST data (heavily stripped down):

{
 "action": "opened",
  "number": 89,
  "pull_request": {
    "url": "https://api.github.com/repos/Owner/ExampleRepository/pulls/89",
    "head": {
      "repo": {
        "id": 123454678,
         "name": "ExampleRepository"
        }
      }
  }
}

Example Config File (also stripped down):

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": [GitHub Post data].pull_request.url,
  "Resources": {
      "ElasticBeanstalk": {
         "Type": "AWS::ElasticBeanstalk::Environment",
         "Properties": {
             "ApplicationName": [GitHub Post data].pull_request.head.repo + [GitHub Post data].number,
             "Description": [GitHub Post data].pull_request.url
          }
     }
  }
}

Both the config file and the GitHub POST data is much more complex than this. Also, the config file will reference various parts of the POST data multiple times and sometimes would require the concatenation of multiple values.

Upvotes: 3

Views: 1655

Answers (3)

JSmith
JSmith

Reputation: 143

I appreciate all the responses, each of which has helped me further increase my knowledge of Go.

The best way of solving this issue for me was to use the text/template library and create the JSON configuration as a text template, which allowed me to easily apply the variables from the GitHub post request.

https://golang.org/pkg/text/template/

Upvotes: 0

Elias Van Ootegem
Elias Van Ootegem

Reputation: 76395

Why not use the standard packages? The os package has ExpandEnv to expand ${VARNAME} and $VARNAME style substrings in a given string with the environment variable values. Just pass the values through as environment values, read the file and do:

replacedVars := os.ExpandEnv(fContents)

Check os.ExpandEnv docs

Basically, I'd write something like:

package main

import (
    "fmt"
    "io/ioutil"
    "os"
)

func main() {
    fname = "foobar.json" // wherever this comes from
    buf, err := ioutil.ReadFile(fname)
    if err != nil {
        fmt.Printf("Failed to read file %s - %+v", fname, err)
        return
    }
    conf := os.ExpandEnv(string(buf))
    // do whatever you need to do with conf string...
}

Upvotes: 0

icza
icza

Reputation: 417422

Attempting to text-manipulate JSON-text can be error-prone (think about escapes and unicode sequences, e.g. this is valid JSON: {"a":"\"b"}). It's much easier and safer to unmarshal the JSON config into a Go value, perform the modifications on it, then marshal to JSON.

If the config structure is known, create matching Go structs to model it exactly.

If the config structure is unknown or you don't want to hassle with it, you may unmarshal into a value of type interface{}, and perform the modification on it. This will be more verbose as you have to perform type assertions for each property and indices.

To ease the pain of the latter, you may use 3rd party libs such as github.com/icza/dyno (disclosure: I'm the author).

For "merging" your existing config and data from the POST request, check out this answer: "Merge" fields two structs of same type

Upvotes: 2

Related Questions