CODEWITHSUNDEEP
asynchronoustaskelixir
Jason O.
Jason O.

Reputation: 3300

Asynchronously running long running tasks in Elixir

I have a module that saves data in csv format which takes a relative long time depending on the data size. What is the Elixir way to accomplish this asynchronously? I tried using Agent, but the process times out.

enter image description here

defmodule FinReporting.Export_CSV do
    alias FinReporting.DistributeRepo
    alias FinReporting.InterfaceMdl
    import Ecto.Query

    def start_link do
        Agent.start_link(fn -> HashDict.new end, name: __MODULE__)
    end

    def export do
        Agent.update(__MODULE__, fn dict -> 

                                    export_sub()    
                                end)
    end

    defp export_sub do
        file = File.open!("test.csv",[:write, :utf8])
        IO.puts("===> CSV export of NGInterface file started.")
        DistributeRepo.all(from entry in InterfaceMdl, limit: 100000, select: %{ field1: entry.field1, amount: entry.amount})
                |>Enum.map(fn(entry)-> %{entry|amount: Decimal.to_string(entry.amount)}end) 
                |> Enum.map(fn(m) -> [m.field1, m.amount] end) 
                |> CSV.encode 
                |> Enum.each(&IO.write(file, &1))

        IO.puts("===> CSV export of NGInterface file completed.")
        _ = File.close(file) 
    end
end

Upvotes: 2

Views: 642

Answers (1)

Dogbert
Dogbert

Reputation: 222188

You can specify a custom timeout using the third argument to Agent.update. You can pass an integer specifying the number of milliseconds, e.g. 60000 for one minute, or :infinity for infinite timeout.

Agent.update(__MODULE__, fn dict -> export_sub() end, 60000)

But, Agent.update waits for the function to complete executing, which is not what you want.

You want Task and specifically Task.async/1.

Task.async(fn -> export_sub() end)

This will return a Task struct that you can wait on later in your application using Task.await or ask for its status using Task.yield. All this and more is explained in great detail in the documentation of Task.

Upvotes: 5

Related Questions