Programatically download file from google cloud storage bucket

Question

I'm trying to figure out how to download a file from google cloud storage bucket.

My use-case is to run a scheduled script which downloads a .csv file once a day and save it to a SQL DB.

I considered doing it using python and the google SDK but got lost with all the options and which one is the right for me.

Could someone can explain the difference between cloud storage client, boto, gsutil, and google cloud SDK?

Thanks!

Answer 1

The Google Cloud SDK includes a gsutil program. It is based on the boto Python library. "Cloud storage client" is not a specific thing as far as I'm aware.

If you have some kind of scheduler like cron that can regularly run a command, all you need is

gsutil cp gs://mybucket/somefile.csv /tmp/

How to load that into a SQL DB is a separate problem.

Answer 2

Look into gcs-fuse: Makes like a lot easier since you then can use the GCS as just a standard file system.