CODEWITHSUNDEEP
rust
ArtDen
ArtDen

Reputation: 263

Using synchronous file-IO library in asynchronous code

I want to use library with synchronous file IO in asynchronous application. I also want all file operations work asynchronously. Is that possible? Something like this:

// function in other crate with synchronous API
fn some_api_fun_with_sync_io(r: &impl std::io::Read) -> Result<(), std::io::Error> {
    // ...
}

async fn my_fun() -> anyhow::Result<()> {
    let mut async_file = async_std::fs::File::open("test.txt").await?;

    // I want some magic here ))
    let mut sync_file = magic_async_to_sync_converter(async_file);

    some_api_fun_with_sync_io(&mut sync_file)?;

    Ok(())
}

Upvotes: 1

Views: 637

Answers (2)

ArtDen
ArtDen

Reputation: 263

Benchmarking idea of @Caesar :

use async_std::prelude::*;
use std::time::*;

struct AsyncToSyncWriteCvt<T: async_std::io::Write + Unpin> (T);

impl<T: async_std::io::Write + Unpin> std::io::Write for AsyncToSyncWriteCvt<T> {
    fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
        async_std::task::block_on(self.0.write(buf))
    }
    fn flush(&mut self) -> std::io::Result<()> {
        async_std::task::block_on(self.0.flush())
    }
}

fn test_sync<W: std::io::Write>(mut w: W) -> Result<(), std::io::Error> {
    for _ in 0..1000000 { w.write("test test test test ".as_bytes())?; }
    Ok(())
}

async fn test_async<T: async_std::io::Write + Unpin>(mut w: T) -> Result<(), std::io::Error> {
    for _ in 0..1000000 { w.write("test test test test ".as_bytes()).await?; }
    Ok(())
}

fn main() -> anyhow::Result<()> {
    async_std::task::block_on(async {
        // bench async -> sync IO
        let now = Instant::now();
        let async_file = async_std::fs::File::create("test1.txt").await?;
        let sync_file = AsyncToSyncWriteCvt(async_file);
        test_sync(sync_file)?;
        println!("Async -> sync: {:.2}s", now.elapsed().as_secs_f32());

        // bench sync IO
        let now = Instant::now();
        let sync_file = std::fs::File::create("test2.txt")?;
        test_sync(sync_file)?;
        println!("Sync: {:.2}s", now.elapsed().as_secs_f32());

        // bench async IO
        let now = Instant::now();
        let async_file = async_std::fs::File::create("test3.txt").await?;
        test_async(async_file).await?;
        println!("Async: {:.2}s", now.elapsed().as_secs_f32());

        Ok(())
    })
}

This code shows "sync -> async" file writing as fast as "async" file writing but less fast then direct sync writing. BufWriter allow to speed up and to close the speed gap between sync and async

Upvotes: 0

Caesar
Caesar

Reputation: 8514

I don't think this magic exists yet, but you can conjure it up yourself with async_std::task::block_on:

fn magic_async_to_sync_converter(async_file: AsyncFile) -> Magic {
    Magic(async_file)
}

struct Magic(AsyncFile);

impl SyncRead for Magic {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        block_on(self.0.read(buf))
    }
}

use std::io::Read as SyncRead;

use async_std::{
    fs::File as AsyncFile,
    io::ReadExt,
    task::{block_on, spawn_blocking},
};

But since some_api_fun_with_sync_io is now doing blocking io, you'll have to shove it into a blocking io thread with spawn_blocking:

spawn_blocking(move || some_api_fun_with_sync_io(sync_file)).await?;

You might want to revise your design and see whether you can do without this though. spawn_blocking is still marked as unstable by async_std.

Upvotes: 2

Related Questions