CODEWITHSUNDEEP
javagoogle-app-enginegoogle-bigquery
pgiecek
pgiecek

Reputation: 8200

How to set timeout for BigQuery API request in Java

Sometimes when we poll for a BigQuery job, our request ends up with SocketTimeoutException. You can see the code raising the exception below.

this.bigquery.jobs().get(projectNumber, jobId).execute();

And here is the error message we get.

...
Caused by: java.net.SocketTimeoutException:
Timeout while fetching URL: https://www.googleapis.com/bigquery/v2/projects/######/jobs/######
...

My question is if there is a way to extend the timeout. And does anyone know what the default timeout is?

Upvotes: 7

Views: 5042

Answers (3)

Denis
Denis

Reputation: 837

I had to increase HTTP timeout in BigQuery client. But I used newer version of BigQuery client (com.google.cloud.bigquery.BigQuery package) where it should be done in a different way (not like in old BigQuery client com.google.api.services.bigquery.Bigquery package)

BigQuery createBigQueryConnection(String projectId) {
            return BigQueryOptions.newBuilder()
                    .setProjectId(projectId)
                    .setCredentials(credentials)
                    .setTransportOptions(HttpTransportOptions.newBuilder()
                            .setConnectTimeout(100)
                            .setReadTimeout(100)
                            .build())
                    .build()
                    .getService();
}

Upvotes: 0

Jordan Tigani
Jordan Tigani

Reputation: 26617

You can wrap the credential object in a HTTP initializer that disables (or extends) timeouts. That is, where you currently have this:

Credential credential = ...
Bigquery bigquery = new Bigquery(HTTP_TRANSPORT, JSON_FACTORY, credential);

you could do

final Credential credential = ...
HttpRequestInitializer initializer = new HttpRequestInitializer() {
  public void initialize(HttpRequest request) {
    credential.initialize(request);
    request.connectTimeout = request.readTimeout = 0;
  }
}
Bigquery bigquery = new Bigquery(HTTP_TRANSPORT, JSON_FACTORY, initializer);

See this for BigQuery object javadoc, this for BigQuery object creation example, and this for HttpRequestInitializer overloading.

Upvotes: 7

auracool
auracool

Reputation: 172

If it is the request only that you want to set a timeout to then in request body:

query_config = { 'timeoutMs': 1000,

set timeoutsMs to whatever you like withing reason ;)

Hope that helps the documentation is here

Edit

To get the results of the query even if it hasn't obeyed the timeout Call jobs.getQueryResults taken from the site you must specify a start row, and this also takes a timeout that behaves the same as the jobs.query timeout to allow waiting if the job is not yet complete.

Upvotes: 1

Related Questions