CODEWITHSUNDEEP
recursionstreamelixirgenerator
Alexey Orlov
Alexey Orlov

Reputation: 2804

Elixir: recursive generators

Is it possible to build a Python style recursive generator with Elixir? Something like this:

def traverse(parent_dir):
    dirs, files = get_dirs_and_files_as_lists(parent_dir)

    for d in dirs:
        yield from traverse(d)

    for f in files:
        yield f

For all the files to be processed linearly, without overhead implied by an eager list of indefinite length:

for f in traverse(dir):
    process(f)

This, or some working equivalent, should be possible using streams; unfortunately, I have no idea how.

I want something like this, just lazy:

def traverse_eagerly(parent_dir) do
  {dirs, files} = get_dirs_and_files_as_lists(parent_dir)

  for x <- dirs do
    traverse_eagerly(x)
  end
  |> Enum.concat()
  |> Enum.concat(files)
end

Upvotes: 1

Views: 217

Answers (2)

Alexey Orlov
Alexey Orlov

Reputation: 2804

The solution appears to be trivial: replace Enum with Stream.

def traverse_lazily(parent_dir) do
  {dirs, files} = get_dirs_and_files_as_lists(parent_dir)

  for x <- dirs do
    traverse_lazily(x)
  end
  |> Stream.concat()
  |> Stream.concat(files)
end

The following works as expected:

s = traverse_lazily(a_dir_of_choice)

for x <- s, do: whatever_you_please(x)

Very nice of the language. As fine a solution as you would wish for. Unless I'm missing something, that is :) . Comments are welcome!

Upvotes: 2

Aleksei Matiushkin
Aleksei Matiushkin

Reputation: 121000

You do not need Stream here, but if you want, here is it:

defmodule Traverse do
  @spec traverse(root :: binary(), yielder :: (binary() -> any())) ::
     :ok | {:error, posix()}
  def traverse(root, yielder) do
    # https://hexdocs.pm/elixir/master/File.html?#ls/1
    with {:ok, list} <- File.ls(root) do
      list
      |> Stream.each(fn file_or_dir ->
        if File.dir?(file_or_dir),
          do: traverse(file_or_dir, yielder), # TCO
          else: yielder.(file_or_dir)
      end)
      |> Stream.run()
    end
  end
end

And call it like:

Traverse.traverse(".", &IO.inspect/1)

Upvotes: 1

Related Questions