CODEWITHSUNDEEP
pythonazureazure-blob-storage
David Kusner
David Kusner

Reputation: 93

python async upload_blob -- TypeError: object AccessToken can't be used in 'await' expression

I'm using Python 3.6.8 and the following packages:

azure-common 1.1.25
azure-core 1.3.0
azure-identity 1.3.0
azure-nspkg 3.0.2
azure-storage-blob 12.3.0

The following line in my code:

await blobClient.upload_blob(data, overwrite=True)

Gives me a stack trace with the message:

TypeError: object AccessToken can't be used in 'await' expression

If I take off the 'await' the code works fine, though it never surrenders control to other tasks, but it does upload the blobs perfectly. I'm using the asynchronous upload_blob() function. I can't understand why I'm getting this error, any ideas?

Here's the code minus some variable assignments:

from azure.identity import ClientSecretCredential
from azure.storage.blob.aio import BlobClient
import asyncio
import os

async def task(name, work_queue):
    """                                                                                                                                            
    """
    while not work_queue.empty():
        f = await work_queue.get()

        blobClient = BlobClient(
            "https://{}.blob.core.windows.net".format(accountName),
            credential = creds,
            container_name = container,
            blob_name = (f.replace(sourceDirPrefix, '')),
        )

        # Upload content to blob                                                                                                                   
        with open(f, "rb") as data:
            await blobClient.upload_blob(data, overwrite=True)


def main():
    """                                                                                                                                            
    This is the main entry point for the program                                                                                                   
    """
    # Create the queue of work                                                                                                                     
    work_queue = asyncio.Queue()

    # Put some work in the queue                                                                                                                   
    for (path, dirs, files) in os.walk(sourceDir):
        for f in files:
            work_queue.put_nowait(os.path.join(path, f))

    # Run the tasks                                                                                                                                
    loop = asyncio.get_event_loop()
    loop.run_until_complete(
        asyncio.gather(
            task("1", work_queue),
            task("2", work_queue),
            task("3", work_queue),
        )
    )
    loop.close()

Stack Trace:

Traceback (most recent call last):
  File "test04.py", line 77, in <module>
    main()
  File "test04.py", line 70, in main
    task("3", work_queue),
  File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
    return future.result()
  File "test04.py", line 44, in task
    await blobClient.upload_blob(data, overwrite=True, asyncio=True)
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/tracing/decorator_async.py", line 70, in wrapper_use_tracer
    return await func(*args, **kwargs)
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/storage/blob/aio/_blob_client_async.py", line 256, in upload_blob
    return await upload_block_blob(**options)
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/storage/blob/aio/_upload_helpers.py", line 79, in upload_block_blob
    **kwargs)
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/storage/blob/_generated/aio/operations_async/_block_blob_operations_async.py", line 207, in upload
    pipeline_response = await self._client._pipeline.run(request, stream=False, **kwargs)
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/pipeline/_base_async.py", line 213, in run
    return await first_node.send(pipeline_request)  # type: ignore
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/pipeline/_base_async.py", line 91, in send
    response = await self.next.send(request)  # type: ignore
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/pipeline/_base_async.py", line 91, in send
    response = await self.next.send(request)  # type: ignore
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/pipeline/_base_async.py", line 91, in send
    response = await self.next.send(request)  # type: ignore
  [Previous line repeated 3 more times]
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/pipeline/_base_async.py", line 89, in send
    await _await_result(self._policy.on_request, request)
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/pipeline/_base_async.py", line 62, in _await_result
    return await result  # type: ignore
  File "/work/python/venv_sm_36/lib/python3.6/site-packages/azure/core/pipeline/policies/_authentication_async.py", line 37, in on_request
    self._token = await self._credential.get_token(*self._scopes)  # type: ignore
TypeError: object AccessToken can't be used in 'await' expression

Upvotes: 2

Views: 2817

Answers (2)

rust4
rust4

Reputation: 136

The problem is that you are using the idenity class calling the ayncio version fo BlobClient. You have to use the async version of the idenity class.

So you have to change from this:

from azure.identity import ClientSecretCredential

To this:

from azure.identity.aio import ClientSecretCredential

Upvotes: 10

Ivan Glasenberg
Ivan Glasenberg

Reputation: 29985

In async def task(name, work_queue) method -> after this line of code blobClient = BlobClient(xxx), you should use the code below:

       async with blobClient:
            with open(f, "rb") as data:
                await blobClient.upload_blob(data, overwrite=True)

The completed method:

async def task(name, work_queue):
    """                                                                                                                                            
    """
    while not work_queue.empty():
        f = await work_queue.get()

        blobClient = BlobClient(
            "https://{}.blob.core.windows.net".format(accountName),
            credential = creds,
            container_name = container,
            blob_name = (f.replace(sourceDirPrefix, '')),
        )

        async with blobClient:
            with open(f, "rb") as data:
                await blobClient.upload_blob(data, overwrite=True)

Upvotes: 2

Related Questions