CODEWITHSUNDEEP
pythonsqlpandasgoogle-bigquery
Matt Zerweck
Matt Zerweck

Reputation: 23

Google BigQuery Execute SQL file from Python Script

I have a python file that uses datalab.bigquery to execute a SQL query that is written out. However, I have this SQL query saved down in the same GitHub repository so am wondering if there is a way to run the github SQL file without having to copy and paste the query.

Currently it looks like this:

import datalab
import datalab.bigquery as bq
import pandas as pd 

df = bq.Query('''
                SELECT
                  CASE
                    WHEN advance_date IS NULL
                        AND release_date IS NULL
                        AND resale_close_date IS NULL
                            THEN TRUE
 ...
 '''_.to_dataframe()

This works fine, but when updates are made to the SQL script they are not reflected in this python script, which creates an issue. I would like it to call the SQL query that is saved in the Github repository. Is there a way to do this? Like df = bq.execute(sql_file.sql).to_dataframe().

Upvotes: 2

Views: 3286

Answers (1)

Iñigo
Iñigo

Reputation: 2670

You could use the "raw" page of that Github snippet containing the SQL script and check that page to get the script.

For example, let's say I want this script [1] (in your case it would be the SQL script), I click in the "RAW" button and save the URL. Then you can saw what it's inside that URL in Python by using requests:

import requests

raw=<URL OF YOUR SQL SCRIPT>
#In my case it would be
#raw="https://raw.githubusercontent.com/GoogleCloudPlatform/python-docs-samples/master/appengine/standard/bigquery/main.py"
r=requests.get(raw).text
df=bq.Query(r).to_dataframe()

If I understood correctly, that's what you want :D.

Upvotes: 4

Related Questions