CODEWITHSUNDEEP
google-app-enginegogoogle-bigquery
N.F.
N.F.

Reputation: 4202

DEADLINE_EXCEEDED on Go Bigquery

Our application sometimes see DEADLINE_EXCEEDED when accessing to Big Query.

import (
    "cloud.google.com/go/bigquery"
    "golang.org/x/net/context"
    "google.golang.org/api/option"
)

func MyFunc(ctx context.Context) {
            :
    client, err := bigquery.NewClient(ctx, PROJECT_ID, option.WithServiceAccountFile(SERVICE_ACCOUNT_JSON_FILE_PATH))
    query := client.Query("SELECT * FROM ....")
    it, err := query.Read(ctx)
    var list []MyStruct
    for {
        var m MyStruct
        err := it.Next(&m)
        if err == iterator.Done {
            break
        }
        if err != nil {
           <Error Handling>
        }
        list = append(list, m)
    }
            :
}

Sometimes we see this error.

Get https://www.googleapis.com/bigquery/v2/projects/myproject/queries/job_zandIeLwH0s8f3FAQ_ORC0zau14?alt=json\u0026startIndex=0\u0026timeoutMs=60000: API error 5 (urlfetch: DEADLINE_EXCEEDED): ('The read operation timed out',)"

It looks timeout is 5 second, but I can' find how can I change timeout seconds.

I read this post, and I modified my source code as below.

ctx_with_deadline, _ := context.WithTimeout(ctx, 1*time.Minute)
httpClient := &http.Client{
    Transport: &oauth2.Transport{
        Base: &urlfetch.Transport{Context: ctx_with_deadline},
    },
}

client, err := bigquery.NewClient(ctx, PROJECT_ID, option.WithServiceAccountFile(SERVICE_ACCOUNT_JSON_FILE_PATH), option.WithHTTPClient(httpClient))

Then I met this error.

Post https://www.googleapis.com/bigquery/v2/projects/myproject/jobs?alt=json: oauth2: Transport's Source is nil

How can I change timeout in Go Bigquery?

Upvotes: 1

Views: 1513

Answers (1)

Alex Efimov
Alex Efimov

Reputation: 3703

Use ctx_with_deadline also when creating a new instance of the bigquery client:

client, err := bigquery.NewClient(ctx_with_deadline, PROJECT_ID, option.WithServiceAccountFile(SERVICE_ACCOUNT_JSON_FILE_PATH‌​), option.WithHTTPClient(httpClient))

Upvotes: 1

Related Questions