CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-dataflow
chiranjib ghatak
chiranjib ghatak

Reputation: 29

GCP dataflow and On-premDB

can we connect to On-Prem Sql DB and cloud Dataflow(GCP) without API? Our databases do not provide API’s for data extraction. Please help me in this , we are stuck on this for a quite sometime.

Upvotes: 0

Views: 1097

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75715

Yes, you can do this. If you have a look to the Beam documentation, you have several Database built-in connector, like a JDBC IO connector. So you can connect any database with an IP:PORT and the correct drivers.

Now, a security topic: You can choose to add a public IP to your on-prem database to access it with Dataflow. If you do this, it's secure (firstly), and then your dataflow worker node need to have a public IP (or you need to set up a Cloud NAT) to access to internet

A better solution is to create a VPN (or an interconnect) from the same VPC as this one where you run the dataflow worker. Like this, you can use the on-prem Database private IP address to reach it and it's more secure!

Upvotes: 1

Related Questions