CODEWITHSUNDEEP
google-cloud-platformgoogle-bigqueryvpc
Ansh Jain
Ansh Jain

Reputation: 21

How can I get endpoint for Egress connection to Big Query

I am using google-cloud-bigquery SDK version 2.38.2 to create a BigQuery client in a Java project. I need to determines the exact endpoint URL when creating a BigQuery client using the following code snippet:

BigQueryOptions.newBuilder()
    .setProjectId(projectName)
    .setCredentials(credentials)
    .build().getService();

Context:

  1. I have created a VPC, and I'm using Private Service Connect to access BigQuery privately.
  2. I am working on a managed Kafka connector (Confluent Cloud) that needs to accurately identify and tag network traffic originating from BigQuery. Specifically, I need to distinguish between traffic going to public BigQuery endpoints and traffic going to private BigQuery endpoints within my VPC.
  3. I need to verify that my BigQuery client is using a VPC endpoint with a resolvable internal DNS hostname, rather than a public IP address.
  4. I need to understand if BigQuery provides a specific DNS hostname that resolves to an internal IP address when accessed from within a VPC using Private Service Connect.

Specific Questions:

  1. How does the Java SDK determine the final endpoint URL used for the BigQuery API?
  2. When accessing BigQuery from within a VPC using Private Service Connect, is there a specific DNS hostname that resolves to an internal IP address? If so, what is it?
  3. Is there any GCP API or SDK method to retrieve the exact endpoint URL that the BigQuery client is using during execution?

Upvotes: 2

Views: 31

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75950

You can't, because the client library always use bigquery.googleapis.com. It's a game of router, DNS, other network magic that route your traffic.

You have to trust Google for this, but you can also validate that it work correctly. Deploy a VM in a subnet without private service connect, and another with it. Do not add public IP on the VM. Then run your code.

If you have the private service connect, the traffic goes to BigQuery. If not, the VM can't resolve the public URL because it hasn't public IP.

Upvotes: 0

Related Questions