CODEWITHSUNDEEP
google-bigquery
Travis Webb
Travis Webb

Reputation: 15028

BiqQuery GEOGRAPHY clustering column - max limit 1024 bytes?

The BigQuery docs say:

If you store GEOGRAPHY data in a table and your queries filter data by using a spatial predicate, ensure that the table is clustered by the GEOGRAPHY column.

But only POINT types would be smaller than 1kb, most geographies are much larger. So when I go to insert a geography into a clustered column, I get this error:

Clustering encountered a key Msg_0_CLOUD_QUERY_TABLE.geom that is 1995 bytes long, which is more than the maximum allowed length of 1024 bytes.

Is the max size for a GEOGRAPHY clustered column really 1024 bytes? Do I need to insert my data differently somehow?

Upvotes: 1

Views: 227

Answers (1)

Travis Webb
Travis Webb

Reputation: 15028

UPDATE Jan 2021: I was informed by the Bigquery team that the 1kb limit was not intentional, and they are working on a fix so that arbitrary-size geometries can be clustered without generating a surrogate bounding box.

This looks like the real limit, so I pre-computed centroids for my polygons using GDAL in Dataflow, stored them in a separate column, and clustered on the centroid geography. Works great.

Upvotes: 1

Related Questions