How to run a Google Cloud Function using Go with dependencies in private repositories?

Question

I'm trying to adapt the Go example from this tutorial to something that uses dependencies in a private repository. Here is the Cloud Function sample code:

package helloworld

import (
    "context"
    "log"

    "github.com/kurtpeek/my-private-repo/mypackage"
)

// PubSubMessage is the payload of a Pub/Sub event.
type PubSubMessage struct {
    Data []byte `json:"data"`
}

// HelloPubSub2 consumes a Pub/Sub message.
func HelloPubSub2(ctx context.Context, m PubSubMessage) error {
    name := string(m.Data)
    if name == "" {
        name = "World"
    }
    log.Printf("Hello, %s!", name)
    log.Println(mypackage.SayHello())
    return nil
}

where SayHello() is defined in the private repo github.com/kurtpeek/my-private-repo as

package mypackage

// SayHello says hello
func SayHello() string {
    return "Hello, world!"
}

Without the invocation of mypackage.SayHello(), the Cloud Function deploys and runs as expected. I'm also able to run HelloPubSub2 in a main.go after running

git config url."git@github.com".insteadOf "https://github.com"

and adding my SSH key to the repo (cf. this Medium article).

However, if I try it with the mypackage.SayHello(), I get the following error:

could not read Username for 'https://github.com'

Here is the full terminal output:

> 
gcloud functions deploy HelloPubSub2 --runtime go113 --trigger-topic mytopic
Created .gcloudignore file. See `gcloud topic gcloudignore` for details.
WARNING: Function created with limited-access IAM policy. To enable unauthorized access consider "gcloud alpha functions add-iam-policy-binding HelloPubSub2 --member=allUsers --role=roles/cloudfunctions.invoker"
Deploying function (may take a while - up to 2 minutes)...failed.                                              
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: {"error":{"buildpackId":"google.go.functions-framework","buildpackVersion":"0.9.0","errorType":13,"canonicalCode":13,"errorId":"03a1e2f7","errorMessage":"go: github.com/kurtpeek/my-private-repo@v0.0.0-20200508055124-8eb94cb388b2: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /layers/google.go.functions-framework/functions-framework/pkg/mod/cache/vcs/93c0b37d34e5cf0f9b9778b99e5968bf272c89cabd21f8c0d683212c8dd2ef89: exit status 128:\n\tfatal: could not read Username for 'https://github.com': terminal prompts disabled"},"stats":null}

What is the best way to deploy this Cloud Function? Should I run go mod vendor and then change the dependencies to reference the vendor/ directory? (This seems like a hassle to do every time, though).

Update

This documentation, https://cloud.google.com/functions/docs/writing/specifying-dependencies-go#using_private_dependencies, addresses exactly this use case. I wasn't quite able to get it to work yet, though. I've added

go.mod
go.sum

to my .gcloudignore and ran go mod vendor, thereby creating a vendor/ directory, but if I try to deploy it again, I now get

cannot find package \"github.com/kurtpeek/my-private-repo/mypackage\" in any of:\n\t/usr/local/go/src/github.com/kurtpeek/my-private-repo/mypackage (from $GOROOT)\n\t/workspace/src/github.com/kurtpeek/my-private-repo/mypackage (from $GOPATH)"

~/g/s/g/k/m/helloworld> 
gcloud functions deploy HelloPubSub2 --runtime go113 --trigger-topic mytopic
Deploying function (may take a while - up to 2 minutes)...failed.                                            
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: {"error":{"buildpackId":"google.go.build","buildpackVersion":"0.9.0","errorType":2,"canonicalCode":2,"errorId":"6191efcd","errorMessage":"src/helloworld/helloworld.go:7:2: cannot find package \"github.com/kurtpeek/my-private-repo/mypackage\" in any of:\n\t/usr/local/go/src/github.com/kurtpeek/my-private-repo/mypackage (from $GOROOT)\n\t/workspace/src/github.com/kurtpeek/my-private-repo/mypackage (from $GOPATH)"},"stats":null}

I'm running this inside my GOPATH:

kurt@Kurts-MacBook-Pro-13 ~/g/s/g/k/my-cloud-function> go env GOPATH
/Users/kurt/go
kurt@Kurts-MacBook-Pro-13 ~/g/s/g/k/my-cloud-function> pwd
/Users/kurt/go/src/github.com/kurtpeek/my-cloud-function

Any ideas how to fix this?

Answer 1

I came across the answer (by a Google engineer working on this product) here as well as in Error when trying to deploy Google cloud function in Go 1.11 using go modules. The key point is that gcloud functions deploy only copies the contents of the directory in which it is run into the Cloud Function's 'context', so the vendor/ directory has to be in that directory. At the same time, I noticed that the package cannot be main, so I took the somewhat unusual step of removing the go.mod from the repo's root directory and running go mod init and go mod vendor in the helloworld directory. Now it works:

> 
gcloud functions deploy HelloPubSub2 --runtime go113 --trigger-topic mytopic
Deploying function (may take a while - up to 2 minutes)...done.                                                         
availableMemoryMb: 256
entryPoint: HelloPubSub2
eventTrigger:
...