golang project can't find dependencies when run inside a docker container

Question

I have this golang sandbox project: https://github.com/cflynn07/golang-db-gateway-example

When I try to run gateway/gateway.go inside a golang:1.6.0-alpine

~/g/s/g/c/golang-db-gateway-example git:master ❯❯❯ docker-compose up gateway
mysql_server is up-to-date
Starting gateway
Attaching to gateway
gateway | gateway.go:7:2: cannot find package "github.com/go-sql-driver/mysql" in any of:
gateway |   /usr/local/go/src/github.com/go-sql-driver/mysql (from $GOROOT)
gateway |   /go/src/github.com/go-sql-driver/mysql (from $GOPATH)
gateway | gateway.go:8:2: cannot find package "github.com/gorilla/mux" in any of:
gateway |   /usr/local/go/src/github.com/gorilla/mux (from $GOROOT)
gateway |   /go/src/github.com/gorilla/mux (from $GOPATH)
gateway exited with code 1

Why isn't the build step detecting my project's dependencies inside the /example/vendor folder?

When I run go run gateway/gateway.go from my host OS, the command works.

Directory structure (mounted inside container at /example)

~/g/s/g/c/golang-db-gateway-example git:master ❯❯❯ tree -L 3
.
├── README.md
├── client
│   └── client.go
├── docker-compose.yml
├── gateway
│   └── gateway.go
├── glide.lock
├── glide.yaml
├── tmp
└── vendor
    └── github.com
        ├── go-sql-driver
        └── gorilla

---

Relevant files:

docker-compose.yml

mysql:
  container_name: mysql_server
  image: mysql:5.7.11
  environment:
    - MYSQL_ROOT_PASSWORD=root
  ports:
    - 3306
gateway:
  container_name: gateway
  image: golang:1.6.0-alpine
  volumes:
    - ./:/example
  working_dir: /example/gateway
  command: go run gateway.go
  environment:
    - MYSQL_ROOT_PASSWORD=root
    - MYSQL_DATABASE=sandbox
  links:
    - mysql

gateway/gateway.go

package main

import (
    "database/sql"
    "encoding/json"
    "fmt"
    _ "github.com/go-sql-driver/mysql"
    "github.com/gorilla/mux"
    "net/http"
    "os"
)

var db *sql.DB

func main() {
    r := mux.NewRouter()

    var e error
    db, e = sql.Open(
        "mysql", os.ExpandEnv("root:${MYSQL_SERVER_PASSWORD}@mysql_server:3306/${MYSQL_DATABASE}"))
    fmt.Print("error is", e)

    r.HandleFunc("/todos", getTodos).Methods("GET")

    http.ListenAndServe(":8080", r)
    fmt.Printf("gateway")
}

type todo struct{}

func getTodos(w http.ResponseWriter, r *http.Request) {
    t := new(todo)
    s, _ := json.Marshal(t)
    w.Header().Set("Content-Type", "application/json; charset=UTF-8")
    fmt.Fprint(w, string(s))
}

Update 1 I changed my data-volume mount path inside the container to mount the project under the containers $GOPATH

mysql:
  container_name: mysql_server
  image: mysql:5.7.11
  environment:
    - MYSQL_ROOT_PASSWORD=root
  ports:
    - 3306
gateway:
  container_name: gateway
  image: golang:1.6.0-alpine
  volumes:
    - ./:/go/src/github.com/cflynn07/golang-db-gateway-example
  working_dir: /go/src/github.com/cflynn07/golang-db-gateway-example
  command: go run gateway/gateway.go
  environment:
    - MYSQL_ROOT_PASSWORD=root
    - MYSQL_DATABASE=sandbox
  links:
    - mysql

However now docker appears to hang:

~/g/s/g/c/golang-db-gateway-example git:master ❯❯❯ docker-compose up gateway                                                         ✱
mysql_server is up-to-date
Recreating gateway
Attaching to gateway

Answer 1

First - need to change ${MYSQL_SERVER_PASSWORD} on ${MYSQL_ROOT_PASSWORD}, because environment only has MYSQL_ROOT_PASSWORD variable.

Second - this is wrong @mysql_server:3306, correct one is @tcp(mysql_server:3306) to be able to connect to MySQL via TCP.

conn := os.ExpandEnv("root:${MYSQL_ROOT_PASSWORD}@tcp(mysql_server:3306)/${MYSQL_DATABASE}")

All will work fine, the only problem can be while db initializing need to make pause before run main program. There are sever ways to solve this problem here are few methods https://docs.docker.com/compose/startup-order/

Answer 2

It looks like, in my opinion, that the primary problem here is that you aren't building your Go program ahead of time. It looks like you've put the Go source files in the Docker container and you're relying on go run to build and then run the program.

I guess you can do that? It's very script language style that way.

However, the way that I think works best is to build the Go application ahead of time.

(Note, for the following I adapted existing makefile code but I didn't actually run this.)
For example, you could build it like this:

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo -installsuffix netgo -o ./gateway/gateway ./gateway

Then, assuming that you don't need other stuff in that container, you can build the Docker image with a Dockerfile like:

FROM scratch
ENTRYPOINT ["/gateway"]
ADD ./gateway/gateway /gateway

Resulting in a simple, small (about 8 MB) container with one static linked executable in it.

Answer 3

Actually you successfully got the Go server running. It wasn't hanging, just waiting for connections. Due to some quirks, there's no output: it didn't attempt to connect to the database, and logging statements were buffered.

Try modifying gateway.go main:

func main() {
    log.Println("Starting main...")

    conn := os.ExpandEnv("root:${MYSQL_SERVER_PASSWORD}@mysql_server:3306/${MYSQL_DATABASE}")

    var err error
    db, err = sql.Open("mysql", conn)
    if err != nil {
        log.Fatal(err)
    }

    log.Println("pinging", conn)
    if err := db.Ping(); err != nil {
        log.Fatal(err)
    }

    r := mux.NewRouter()
    r.HandleFunc("/todos", getTodos).Methods("GET")

    listen := ":8080"
    log.Printf("Listening on %s\n", listen)
    log.Fatal(http.ListenAndServe(listen, r))
}

Running this version gives:

$ docker-compose up gateway
mysql_server is up-to-date
Starting gateway
Attaching to gateway
gateway | 2016/03/15 10:58:05 Starting main...
gateway | 2016/03/15 10:58:05 pinging root:@mysql_server:3306/sandbox
gateway | 2016/03/15 10:58:05 default addr for network 'mysql_server:3306' unknown
gateway | exit status 1
gateway exited with code 1

You should be good to go from there. Note:

• docker-compose seems to buffer standard output until newline
• log functions such as log.Print add newlines, fmt.Print doesn't
• sql.Open doesn't connect to the database, use sql.Ping (see wiki)
• MYSQL_SERVER_PASSWORD is missing
• network type of the mysql connection string is missing (see examples)
• start mysql server too
• need to create new or mount existing database 'sandbox'

Hope that helps.