CODEWITHSUNDEEP
pythoncsvasync-awaitpython-asynciofastapi
Soumendra
Soumendra

Reputation: 1624

Write a CSV file asynchronously in Python

I am writing a CSV file with the following function:

import csv
import os
import aiofiles


async def write_extract_file(output_filename: str, csv_list: list):
    """
    Write the extracted content into the file
    """
    try:
        async with aiofiles.open(output_filename, "w+") as csv_file:
            writer = csv.DictWriter(csv_file, fieldnames=columns.keys())
            writer.writeheader()
            writer.writerows(csv_list)
    except FileNotFoundError:
        print("Output file not present", output_filename)
        print("Current dir: ", os.getcwd())
        raise FileNotFoundError

However, as there is no await allowed over writerows method, there are no rows being written into the CSV file.
How to resolve this issue? Is there any workaround available?
Thank you.
Entire code can be found here.

Upvotes: 11

Views: 13985

Answers (3)

Fomalhaut
Fomalhaut

Reputation: 9785

You can use aiocsv. Here is a quick example of writing a row to a CSV file asynchronously:

import asyncio
import aiofiles
from aiocsv import AsyncWriter

async def main():
    async with aiofiles.open('your-path.csv', 'w') as f:
        writer = AsyncWriter(f)
        await writer.writerow(['name', 'age'])
        await writer.writerow(['John', 25])

asyncio.run(main())

For more examples follow: https://pypi.org/project/aiocsv/

Upvotes: 8

CrookedCloud
CrookedCloud

Reputation: 124

You can use aiofiles, you just gotta convert the dict to a row :)

import aiofiles

async def write_extract_file(
    output_filename: str, csv_list: list
):

    cols = columns.keys()

    async with aiofiles.open(output_filename, mode='w+') as f_out:

        await f_out.write(','.join(cols)+'\n')

        for data in csv_list:

            line = []

            for c in cols:
                line.append(str(data[c]) if c in data else '')

            line = ','.join(line) + '\n'
    
            await f_out.write(line)

Upvotes: 0

alex_noname
alex_noname

Reputation: 32233

In my opinion it’s better not to try to use the aiofiles with the csv module and run the synchronous code using loop.run_in_executor and wait it asynchronously like below:

def write_extract_file(output_filename: str, csv_list: list):
    """
    Write the extracted content into the file
    """
    try:
        with open(output_filename, "w+") as csv_file:
            writer = csv.DictWriter(csv_file, fieldnames=columns.keys())
            writer.writeheader()
            writer.writerows(csv_list)
    except FileNotFoundError:
        print("Output file not present", output_filename)
        print("Current dir: ", os.getcwd())
        raise FileNotFoundError


async def main():
    loop = asyncio.get_running_loop()
    await loop.run_in_executor(None, write_extract_file, 'test.csv', csv_list)

Upvotes: 8

Related Questions