CODEWITHSUNDEEP
pythonpandasgoogle-apigoogle-colaboratory
Chelsea
Chelsea

Reputation: 335

Colab Python: Downloading/Uploading google files without mounting

I'm using Python in Colabs to access specific files with their IDs, analyze them, and record info on them. I need to save the recorded info in a team folder on Drive. I'm reluctant to mount because the team folder is deep in the architecture of the company drive folders. I don't know how to access it from the main drive, I have it stared in my drive for easy access. I also just feel weird mounting my entire drive with all the company info in it. I really wish you could mount a single folder (I know you can change the path after the mount, but that also feels weird).

I have found a ton of ways to download a file based on the file ID, but I can't find any to upload to that file ID or save to the file ID. I know there's also a way with Pandas to read info from a file ID and use that as a data frame which is an option, but can you save the new info to the file ID with Pandas? There also seems to be a way to easily download with the Google API with a file ID, but again, not an easy way to upload to a file ID or a folder ID and over write the file.

These files are going to be really big as time goes on; tens of thousands of lines, so it needs to be able to deal with that either by only uploading the new info or being able to handle long downloads.

Edit: I did also just now try using gspread, but I'm not able to share files with emails outside of our company domain, so I'm unable to use gspread. ):

Upvotes: 2

Views: 1633

Answers (1)

korakot
korakot

Reputation: 40838

You can use pydrive to read and write based on FILE_ID

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

# Authenticate and create the PyDrive client.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

fid = 'Your File ID'

# read it
f = drive.CreateFile({'id': fid})  # just open an existing file
f.FetchMetadata(fetch_all=True) 
text = f.GetContentString() # or f.GetContentFile('im.png') to save a local file

# or write it
f.SetContentString('Sample upload file content')  # or SetContentFile('im.png')
f.Upload()

Upvotes: 2

Related Questions