CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquerygoogle-cloud-functionscloudgoogle-compute-engine
Frank David
Frank David

Reputation: 317

How to create a private network between google compute engine and google bigquery?

Is it possible to access bigquery from the google compute engine or cloudrun or cloud functions via private network?

Upvotes: 0

Views: 1436

Answers (2)

guillaume blaquiere
guillaume blaquiere

Reputation: 75810

In cloud environment, the meaning of "private" is unclear. In fact, with a VM in your VPC, without public IP, you can't access directly BigQuery because the API http://bigquery.googleapis.com is public and you need to resolve it publicly.

However, you have an option at the Subnet level, to route the well known Google URL through a private/dedicated channel. It's named Private Google Access when you edit your VPC

enter image description here

Like this, the traffic stay internal. Private?? Hmmm what is private? If private means in your private addressing plan, it's not, because BigQuery is not in your project, it's elsewhere, it's serverless!!

BUT, the traffic stay internal to Google Cloud. Is it enough for you?

And in any cases, the traffic is encrypted (HTTPS), even if it goes on the Internet, it stay confidential; it's just a matter of latency, not of security.

Same thing with Cloud Function and Cloud Run. You need a serverless VPC connector with a egress control set to ALL, to route all the traffic, even this one for BigQuery, if you want it through your VPC. And then, reach BigQuery.

If you don't do this, the traffic leave Cloud Run/Cloud Functions from the serverless world of Google Cloud to reach BigQuery, in the serverless world of Google Cloud. Is it more or less internal than with a serverless VPC connector? Not sure.

We an higher point of view, when you pay online on internet, HTTPS is enough. When you create a VPN tunnel, it's similar to HTTPS feature. What's the difference with the BigQuery request?

Finally the real question is Why do you need to be sure that is private? What's your requirements? Your fears?

EDIT 1

According to your comment, your concert is to improve the latency when you reach BigQuery. But I'm not sure to understand your use case and blocking point

BigQuery is an analytics database, the incompressible warmup on each query is between 500ms and 1s. And then, the query run.

If you search the optimal network path, in the case of the default Google routing policies aren't the best, you will save few ms, something like 5 or 10ms in Google petabits network.

Is saving the 1% of latency is a real concern?

Same thing if your concern is the stream write. Is your realtime data need to be ingested 10ms faster when querying them take at least 1s? What's the advantage of this?

Upvotes: 3

al-dann
al-dann

Reputation: 2725

From the best of my knowledge, in order to access BigQuery from a compute engine, or app engine, or cloud run, or cloud function, etc. one does not need any private networks at all. Most likely a correct IAM configuration for the service account under which the process is running, should be enough.

Upvotes: 0

Related Questions