CODEWITHSUNDEEP
jdbcgoogle-bigquery
mihai
mihai

Reputation: 53

Exceeded rate limits: too many table update operations for this table message when using simba jdbc driver

I am using the Simba Jdbc driver version 1.2.2.1004 to sequentially insert 6 rows of data into a BigQuery table. The table only has 3 columns. After 4 inserts I see:

Caused by: com.simba.googlebigquery.support.exceptions.GeneralException: [Simba]BigQueryJDBCDriver Error executing query job. Message: Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors

I've looked at the quotas on the google side but none of them seems to be reached. If I throttle the inserts, all the rows get inserted so it's not a day limit, it seems to be the number of inserts per second, but that should be at least 10 according to my quota...

I have also created a test class that insrts the same values using the google api, that works just fine..

Upvotes: 1

Views: 4236

Answers (1)

Yun Zhang
Yun Zhang

Reputation: 5518

You hit this quota, 5 ops / 10s. So, limiting your rate would help.

If possible, try to group smaller updates into big ones to avoid this quota.

https://cloud.google.com/bigquery/quotas#standard_tables

Maximum rate of table metadata update operations — 5 operations every 10 seconds per table

The table metadata update limit includes all metadata update operations performed by using the Cloud Console, the classic BigQuery web UI, the bq command-line tool, the client libraries, by calling the tables.insert, tables.patch, or tables.update API methods, or executing ALTER TABLE DDL statements. This limit also applies to job output.

Upvotes: 4

Related Questions