CODEWITHSUNDEEP
google-api-python-clientgoogle-cloud-datalab
vvv
vvv

Reputation: 337

Reading in a file with Google datalab

I am trying to use Google Datalab to read in a file in ipython notebook, the basic pd.read_csv() as I can't find the path of the file. I have it locally and also uploaded it to google cloud storage in a bucket.

I ran the following commands to understand where I am

os.getcwd()

gives '/content/[email protected]'

os.listdir('/content/[email protected]')

gives ['.git', '.gitignore', 'datalab', 'Hello World.ipynb', '.ipynb_checkpoints']

Upvotes: 8

Views: 4724

Answers (3)

Kartik Podugu
Kartik Podugu

Reputation: 194

I uploaded my Iris.csv to my datalab root directory.

Then like you mentioned in your question ran the following code cell.

os.getcwd()

I got '/content/datalab/docs'

Then i ran the following code cell.

iris = pd.read_csv('/content/datalab/Iris.csv')
print(iris)

It worked for me.

Upvotes: 0

Graham Wheeler
Graham Wheeler

Reputation: 2814

You can also run BigQuery queries directly against CSV files in Cloud Storage by creating a FederatedTable wrapper object. That is described here:

https://github.com/GoogleCloudPlatform/datalab/blob/master/content/datalab/tutorials/BigQuery/Using%20External%20Tables%20from%20BigQuery.ipynb

Upvotes: 1

Chris
Chris

Reputation: 430

The following reads the contents of the object into a string variable called text:

%%storage read --object "gs://path/to/data.csv" --variable text

Then

from cStringIO import StringIO
mydata = pd.read_csv(StringIO(text)) 
mydata.head()

Hopefully Pandas will support "gs://" URLs (as it does for s3:// currently to allow reading directly from Google Cloud storage.

I have found the following docs really helpful:

https://github.com/GoogleCloudPlatform/datalab/tree/master/content/datalab/tutorials

Hope that helps (just getting started with Datalab too, so maybe someone will have a cleaner method soon).

Upvotes: 10

Related Questions