Asynchronous Testing With Stream Processing

Question

I'm very new to Go, so I may be misunderstanding something foundational about Go's async/stream handling, but here goes...

I'm trying to write some tests using ginkgo on a function I wrote that processes streams.

The processing side reads in newline-delimited text from a File until it encounters a special delimiter line at which point it tries to parse the text as JSON. The code looks like this:

func ParseConfig(inStream *os.File) (Config, error){
  var header string

  var stdin = bufio.NewScanner(inStream)
  for stdin.Scan() {
    line := stdin.Text()

    if line == "|||" {
      break;
    }

    header += line
  }

  // parse JSON here and return
}

My test looks something like this

Describe("ParseConfig()", func() {
  It("should pass for a valid header", func(){
    _, err := io.WriteString(stream, "{\"Key\": \"key\", \"File\": \"file\"}\n|||\n")
    Expect(err).NotTo(HaveOccurred())

    conf, err := parser.ParseConfig(stream)
    Expect(err).NotTo(HaveOccurred())

    Expect(conf.Key).To(Equal("key"))
  })
})

Unfortunately, this yields a JSON parsing error, as it's trying to parse an empty string. I'm assuming that my problem is that I'm sending the string on the stream before I've told the ParseConfig() function to listen on that string for data? But I'm not entirely clear how I could refactor this to use proper go routines to first listen for data then send it.

Some of the potential solutions I saw were around the use of "channels" (with which I'm unfamiliar) but I was worried that this one need might not be worth a major refactor to introduce a whole new paradigm of concurrency.

Thanks!

Answer 1

Not sure if I understood correctly, but your ParseConfig should probably take an io.Reader instead of a *os.File. That way you can test it directly without worrying about concurrency.

file t_test.go:

package main

import (
        "strings"
        "testing"

        "github.com/onsi/ginkgo"
        "github.com/onsi/gomega"
)

var _ = ginkgo.Describe("ParseConfig()", func() {
        ginkgo.It("should pass for a valid header", func() {
                // really don't know what you were doing with your 'stream' variable
                // This is a test, you should forge a test scenario and pass it to your config function
                stream := strings.NewReader(`{"Key": "key", "File": "file"}` + "\n|||\n")

                conf, err := ParseConfig(stream)
                gomega.Expect(err).NotTo(gomega.HaveOccurred())
                gomega.Expect(conf.Key).To(gomega.Equal("key"))
        })
})

func TestParseConfig(t *testing.T) {
        ginkgo.RunSpecs(t, "Parse Config")
}

file main.go

package main

import (
        "bufio"
        "encoding/json"
        "io"
        "log"
        "os"
)

type Config struct {
        Key  string
        File string
}

func ParseConfig(inStream io.Reader) (*Config, error) {
        var header string

        var stdin = bufio.NewScanner(inStream)
        for stdin.Scan() {
                line := stdin.Text()

                if line == "|||" {
                        break
                }
                header += line
        }

        c := &Config{}

        // parse JSON here and return
        if err := json.Unmarshal([]byte(header), c); err != nil {
                return nil, err
        }
        return c, nil
}

func main() {
        f, err := os.Open("config.json")
        if err != nil {
                log.Fatal(err)
        }
        ParseConfig(f)
}