How do I unit test command line flags in Go?

Question

I would like a unit test that verifies a particular command line flag is within an enumeration.

Here is the code I would like to write tests against:

var formatType string

const (
    text = "text"
    json = "json"
    hash = "hash"
)

func init() {
    const (
        defaultFormat = "text"
        formatUsage   = "desired output format"
    )

    flag.StringVar(&formatType, "format", defaultFormat, formatUsage)
    flag.StringVar(&formatType, "f", defaultFormat, formatUsage+" (shorthand)")

}

func main() {
    flag.Parse()
}

The desired test would pass only if -format equalled one of the const values given above. This value would be available in formatType. An example correct call would be: program -format text

What is the best way to test the desired behaviors?

Note: Perhaps I have phrased this poorly, but the displayed code it not the unit test itself, but the code I want to write unit tests against. This is a simple example from the tool I am writing and wanted to ask if there were a good way to test valid inputs to the tool.

Answer 1

You can test main() by:

1. Making a test that runs a command
2. Which then calls the app test binary, built from go test, directly
3. Passing the desired flags you want to test
4. Passing back the exit code, stdout, and stderr which you can assert on.

NOTE This only works when main exits, so that the test does not run infinitely, or gets caught in a recursive loop.

Given your main.go looks like:

package main

import (
    "flag"
    "fmt"
    "os"
)

var formatType string

const (
    text = "text"
    json = "json"
    hash = "hash"
)

func init() {
    const (
        defaultFormat = "text"
        formatUsage   = "desired output format"
    )

    flag.StringVar(&formatType, "format", defaultFormat, formatUsage)
    flag.StringVar(&formatType, "f", defaultFormat, formatUsage+" (shorthand)")
}

func main() {
    flag.Parse()
    fmt.Printf("format type = %v\n", formatType)
    os.Exit(0)
}

Your main_test.go may then look something like:

package main

import (
    "fmt"
    "os"
    "os/exec"
    "path"
    "runtime"
    "strings"
    "testing"
)

// This will be used to pass args to app and keep the test framework from looping
const subCmdFlags = "FLAGS_FOR_MAIN"

func TestMain(m *testing.M) {
    // Only runs when this environment variable is set.
    if os.Getenv(subCmdFlags) != "" {
        runAppMain()
    }

    // Run all tests
    exitCode := m.Run()
    // Clean up
    os.Exit(exitCode)
}

func TestMainForCorrectness(tester *testing.T) {
    var tests = []struct {
        name     string
        wantCode int
        args     []string
    }{
        {"formatTypeJson", 0, []string{"-format", "json"}},
    }

    for _, test := range tests {
        tester.Run(test.name, func(t *testing.T) {
            cmd := getTestBinCmd(test.args)

            cmdOut, cmdErr := cmd.CombinedOutput()

            got := cmd.ProcessState.ExitCode()

            // Debug
            showCmdOutput(cmdOut, cmdErr)

            if got != test.wantCode {
                t.Errorf("unexpected error on exit. want %q, got %q", test.wantCode, got)
            }
        })
    }
}

// private helper methods.

// Used for running the application's main function from other test.
func runAppMain() {
    // the test framework has process its flags,
    // so now we can remove them and replace them with the flags we want to pass to main.
    // we are pulling them out of the environment var we set.
    args := strings.Split(os.Getenv(subCmdFlags), " ")
    os.Args = append([]string{os.Args[0]}, args...)

    // Debug stmt, can be removed
    fmt.Printf("\nos args = %v\n", os.Args)

    main() // will run and exit, signaling the test framework to stop and return the exit code.
}

// getTestBinCmd return a command to run your app (test) binary directly; `TestMain`, will be run automatically.
func getTestBinCmd(args []string) *exec.Cmd {
    // call the generated test binary directly
    // Have it the function runAppMain.
    cmd := exec.Command(os.Args[0], "-args", strings.Join(args, " "))
    // Run in the context of the source directory.
    _, filename, _, _ := runtime.Caller(0)
    cmd.Dir = path.Dir(filename)
    // Set an environment variable
    // 1. Only exist for the life of the test that calls this function.
    // 2. Passes arguments/flag to your app
    // 3. Lets TestMain know when to run the main function.
    subEnvVar := subCmdFlags + "=" + strings.Join(args, " ")
    cmd.Env = append(os.Environ(), subEnvVar)

    return cmd
}

func showCmdOutput(cmdOut []byte, cmdErr error) {
    if cmdOut != nil {
        fmt.Printf("\nBEGIN sub-command out:\n%v", string(cmdOut))
        fmt.Print("END sub-command\n")
    }

    if cmdErr != nil {
        fmt.Printf("\nBEGIN sub-command stderr:\n%v", cmdErr.Error())
        fmt.Print("END sub-command\n")
    }
}

Answer 2

You can do this

func main() {
    var name string
    var password string
    flag.StringVar(&name, "name", "", "")
    flag.StringVar(&password, "password", "", "")
    flag.Parse()
    for _, v := range os.Args {
        fmt.Println(v)
    }
    if len(strings.TrimSpace(name)) == 0 || len(strings.TrimSpace(password)) == 0 {
        log.Panicln("no name or no passward")
    }
    fmt.Printf("name:%s\n", name)
    fmt.Printf("password:%s\n", password)
}

func TestMainApp(t *testing.T) {
    os.Args = []string{"test", "-name", "Hello", "-password", "World"}
    main()
}

Answer 3

Custom testing and processing of flags can be achieved with the flag.Var function in the flag package.

Flag.Var "defines a flag with the specified name and usage string. The type and value of the flag are represented by the first argument, of type Value, which typically holds a user-defined implementation of Value."

A flag.Value is any type that satisfies the Value interface, defined as:

type Value interface {
    String() string
    Set(string) error
}

There is a good example in the example_test.go file in the flag package source

For your use case you could use something like:

package main

import (
    "errors"
    "flag"
    "fmt"
)

type formatType string

func (f *formatType) String() string {
    return fmt.Sprint(*f)
}

func (f *formatType) Set(value string) error {
    if len(*f) > 0 && *f != "text" {
        return errors.New("format flag already set")
    }
    if value != "text" && value != "json" && value != "hash" {
        return errors.New("Invalid Format Type")
    }
    *f = formatType(value)
    return nil
}

var typeFlag formatType

func init() {
    typeFlag = "text"
    usage := `Format type. Must be "text", "json" or "hash". Defaults to "text".`
    flag.Var(&typeFlag, "format", usage)
    flag.Var(&typeFlag, "f", usage+" (shorthand)")
}

func main() {
    flag.Parse()
    fmt.Println("Format type is", typeFlag)
}

This is probably overkill for such a simple example, but may be very useful when defining more complex flag types (The linked example converts a comma separated list of intervals into a slice of a custom type based on time.Duration).

EDIT: In answer to how to run unit tests against flags, the most canonical example is flag_test.go in the flag package source. The section related to testing custom flag variables starts at Line 181.

Answer 4

I'm not sure whether we agree on the term 'unit test'. What you want to achieve seems to me more like a pretty normal test in a program. You probably want to do something like this:

func main() {
    flag.Parse()

    if formatType != text || formatType != json || formatType != hash {
        flag.Usage()
        return
    }

    // ...
}

Sadly, it is not easily possible to extend the flag Parser with own value verifiers so you have to stick with this for now.

See Intermernet for a solution which defines a custom format type and its validator.