Reputation: 63
Just messing with elixir result got an error that I cannot figure out.Here is my snippet,I implemented a simple Parallel map function for some api calls. func is the function where the actual calls are made and it returns {:ok,result} or {:error,reason} which I handle after the mapping in a different function
Originally
def pmap(collection,func,limit \\ 5000) do
collection
|> Enum.map(&Task.async(func.(&1)))
|> Enum.map(&Task.await(&1,limit))
Got the error so changed it to this for readability
def pmap(collection,func,limit) do
collection
|>Enum.map(fn(x) -> Task.async(func.(x)) end)
|>Enum.map(fn(task) -> Task.await(task,limit) end)
The error I am getting states
[error] Task #PID<0.197.0> started from #PID<0.187.0> terminating
** (BadFunctionError) expected a function, got: {:ok,result}
erlang.apply/2
(elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:36: Task.Supervised.reply/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Function: &:erlang.apply/2
From what I gather it is assuming that the actual task collection element is being passed as the function
So I modified the function to
IO.puts "PMAP BEGUN"
tasks = collection
|> Enum.map(fn(x) -> Task.async(func.(x)) end)
answer = Enum.map(tasks,fn(task) -> Task.await(task,limit) end)
IO.puts "PMAP DONE"
answer
The IO.puts were for debugging.So I guess the error happens on the second map as PMAP DONE is never displayed. I still have the same error.
What exactly is wrong here? I have written the same function before almost verbatim and it worked.
Upvotes: 2
Views: 1333
Reputation: 9639
The problem you're experiencing happens on line:
|> Enum.map(fn(x) -> Task.async(func.(x)) end)
to make it work you need to wrap function execution with anonymous function, like:
|> Enum.map(fn(x) -> Task.async(fn -> func.(x) end) end)
This due to fact if you run func.(x)
this will evaluate, and the result of evaluation will be passed to Task.async/1
, while when you wrap it with function, it will be up to Task
to execute it.
The hint is indicated in error message:
** (BadFunctionError) expected a function, got: {:ok,result}
Hope this helps!
Upvotes: 7