How to upload PDF file using request POST method and requests must be formatted as multipart MIME using python for xeroAPI?

Question

I'm trying to upload PDF-file in the Xero account using the python request library (POST method) and Xeros FilesAPI said "Requests must be formatted as multipart MIME" and have some required fields (link) but I don't how to do that exactly...If I do GET-request I'm getting the list of files in the Xero account but having a problem while posting the file (POST-request)...

My Code:

post_url = 'https://api.xero.com/files.xro/1.0/Files/'
files = {'file': open('/home/mobin/PycharmProjects/s3toxero/data/in_test_upload.pdf', 'rb')}

response = requests.post(
    post_url,
    headers={
        'Authorization': 'Bearer ' + new_tokens[0],
        'Xero-tenant-id': xero_tenant_id,
        'Accept': 'application/json',
        'Content-type': 'multipart/form-data; boundary=JLQPFBPUP0',
        'Content-Length': '1068',
    },
    files=files,
)

json_response = response.json()

print(f'Uploading Responsoe ==> {json_response}')
print(f'Uploading Responsoe ==> {response}')

Error Mesage/Response:

Uploading Responsoe ==> [{'type': 'Validation', 'title': 'Validation failure', 'detail': 'No file is was attached'}]
Uploading Responsoe ==> <Response [400]>

Answer 1

looks like you got it solved. For reference and any future developers who are using the Xero supported package (https://github.com/XeroAPI/xero-python)

We just added the files_api example code to the sample app so the following would upload a file if you were using the Python SDK

https://github.com/XeroAPI/xero-python-oauth2-app/pull/29/files

name = "my-image"
filename= "my-image.jpg"
mime_type = "image/jpg"
with open('my-image.jpg', 'rb') as f:
    body = f.read()

try:
    file_object = files_api.upload_file(
        xero_tenant_id, 
        name = name, 
        filename= filename, 
        mime_type = mime_type,
        body=body
    )
except AccountingBadRequestException as exception:
    json = jsonify(exception.error_data)
else:
    json = serialize_model(file_object)

Answer 2

I've tested this with Xero's Files API to upload a file called "helloworld.rtf" in the same directory as my main app.py file.

var1 = "Bearer "
var2 = YOUR_ACCESS_TOKEN
access_token_header = var1 + var2
body = open('helloworld.rtf', 'rb')

mp_encoder = MultipartEncoder(
fields={
     'helloworld.rtf': ('helloworld.rtf', body),
     }
 )

r = requests.post(
    'https://api.xero.com/files.xro/1.0/Files',
    data=mp_encoder,  # The MultipartEncoder is posted as data
    # The MultipartEncoder provides the content-type header with the boundary:
    headers={
        'Content-Type': mp_encoder.content_type,
        'xero-tenant-id': YOUR_XERO_TENANT_ID,
        'Authorization': access_token_header
    }
)

Answer 3

As I see you're improperly set the boundary. You set it in the headers but not tell to requests library to use custom boundary. Let me show you an example:

>>> import requests
>>> post_url = 'https://api.xero.com/files.xro/1.0/Files/'
>>> files = {'file': open('/tmp/test.txt', 'rb')}
>>> headers = {
...    'Authorization': 'Bearer secret',
...    'Xero-tenant-id': '42',
...    'Accept': 'application/json',
...    'Content-type': 'multipart/form-data; boundary=JLQPFBPUP0',
...    'Content-Length': '1068',
... }
>>> print(requests.Request('POST', post_url, files=files, headers=headers).prepare().body.decode('utf8'))
--f3e21ca5e554dd96430f07bb7a0d0e77
Content-Disposition: form-data; name="file"; filename="test.txt"


--f3e21ca5e554dd96430f07bb7a0d0e77--

As you can see the real boundary (f3e21ca5e554dd96430f07bb7a0d0e77) is different from what was passed in the header (JLQPFBPUP0).

You can actually directly use the requests module to controll boundary like this:

Let's prepare a test file:

$ touch /tmp/test.txt
$ echo 'Hello, World!' > /tmp/test.txt 

Test it:

>>> import requests
>>> post_url = 'https://api.xero.com/files.xro/1.0/Files/'
>>> files = {'file': open('/tmp/test.txt', 'rb')}
>>> headers = {
...     'Authorization': 'Bearer secret',
...     'Xero-tenant-id': '42',
...     'Accept': 'application/json',
...     'Content-Length': '1068',
... }
>>> body, content_type = requests.models.RequestEncodingMixin._encode_files(files, {})
>>> headers['Content-type'] = content_type
>>> print(requests.Request('POST', post_url, data=body, headers=headers).prepare().body.decode('utf8'))
--db57d23ff5dee7dc8dbab418e4bcb6dc
Content-Disposition: form-data; name="file"; filename="test.txt"

Hello, World!

--db57d23ff5dee7dc8dbab418e4bcb6dc--

>>> headers['Content-type']
'multipart/form-data; boundary=db57d23ff5dee7dc8dbab418e4bcb6dc'

Here boundary is the same as in the header.

Another alternative is using requests-toolbelt; below example taken from this GitHub issue thread:

from requests_toolbelt import MultipartEncoder

fields = {
    # your multipart form fields
}

m = MultipartEncoder(fields, boundary='my_super_custom_header')
r = requests.post(url, headers={'Content-Type': m.content_type}, data=m.to_string())

But it is better not to pass bundary by hand at all and entrust this work to the requests library.

---

Update:

A minimal working example using Xero Files API and Python request:

from os.path import abspath
import requests

access_token = 'secret'
tenant_id = 'secret'

filename = abspath('./example.png')

post_url = 'https://api.xero.com/files.xro/1.0/Files'
files = {'filename': open(filename, 'rb')}
values = {'name': 'Xero'}

headers = {
    'Authorization': f'Bearer {access_token}',
    'Xero-tenant-id': f'{tenant_id}',
    'Accept': 'application/json',
}

response = requests.post(
    post_url,
    headers=headers,
    files=files,
    data=values
)

assert response.status_code == 201