CODEWITHSUNDEEP
pythononedrive
Krystian
Krystian

Reputation: 53

Uploading huge files into OneDrive using SDK

I'd like to upload a huge (bigger than 2GB) file to OneDrive.

I have already tried using a code from the sdk webpage (https://github.com/OneDrive/onedrive-sdk-python)

returned_item = client.item(drive='me', path=backupPath).children['photos.tgz'].upload_async('/Users/koot/photos.tgz')

Although the code works for smaller files, when uploading a big file I got:

BrokenPipeError: [Errno 32] Broken pipe

requests.exceptions.ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

Upvotes: 3

Views: 1701

Answers (1)

Jason M
Jason M

Reputation: 41

SDK is now deprecated. https://learn.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_createuploadsession?view=odsp-graph-online You can use Microsoft graph and OneDriveAPI to upload files to OneDrive OneDriveAPI support simple upload for small file (<4MB) and resumable uploads for larger files where you basically create an upload session and the upload the large file, one chunk at a time.

Github repo

And here is a tutorial I wrote about to resolve this issue: https://dev.to/jsnmtr/automating-files-upload-to-microsoft-onedrive-unexpected-challenges-and-a-success-story-2ini

Below is the code for uploading large files:

#Creating an upload session
        upload_session = requests.post(onedrive_destination+"/"+file_name+":/createUploadSession", headers=headers).json()

        with open(file_path, 'rb') as f:
            total_file_size = os.path.getsize(file_path)
            chunk_size = 327680
            chunk_number = total_file_size//chunk_size
            chunk_leftover = total_file_size - chunk_size * chunk_number
            i = 0
            while True:
                chunk_data = f.read(chunk_size)
                start_index = i*chunk_size
                end_index = start_index + chunk_size
                #If end of file, break
                if not chunk_data:
                    break
                if i == chunk_number:
                    end_index = start_index + chunk_leftover
                #Setting the header with the appropriate chunk data location in the file
                headers = {'Content-Length':'{}'.format(chunk_size),'Content-Range':'bytes {}-{}/{}'.format(start_index, end_index-1, total_file_size)}
                #Upload one chunk at a time
                chunk_data_upload = requests.put(upload_session['uploadUrl'], data=chunk_data, headers=headers)
                print(chunk_data_upload)
                print(chunk_data_upload.json())
                i = i + 1

Upvotes: 4

Related Questions