CODEWITHSUNDEEP
google-bigquerydatabase-performance
Gustavo Concon
Gustavo Concon

Reputation: 53

BigQuery performance: Is this correct?

Folks, I'm using BigQuery as a superfast database for my analytics queries, but I'm very disappointed with its performance.

Let me show you the numbers:

When I execute this query on BigQuery's console, it takes about 1 minute to process. Is this ok for you? I was expecting that it will return in about 2 seconds... If I execute this query on a columnar database, like Sybase IQ, it takes less than 2 seconds.

Upvotes: 1

Views: 264

Answers (2)

DanielViveiros
DanielViveiros

Reputation: 179

Big Query is a highly scalable database, before being a "super fast" database. It's designed to process HUGE amount of data distributing the processing among several different machines using a technique named Dremel. Because it's designed to use several machines and parallel processing, you should expect to have super-scalability with a good performance.

For example: analyzing all the wikipedia revisions in 5-10 seconds isn't bad, is it? But even a much smaller table would take about the same time.

Sybase IQ is often installed in a single database and it doesn't use Dremel. That said, it's going to be faster than Big Query in many scenarios...as designed.

Cheers!

Upvotes: 0

encc
encc

Reputation: 1633

Since you are returning 830k rows and BQ is always creating a temporary result table, the creation is more than a small result.

Have you turned on large results?

We are working in a shared environment and sometime loads ( table creation ) takes a while. Certainly the performance differ from a dedicated environment. You get your dedicated environment for 20K$ a month.

Upvotes: 0

Related Questions