How to interpolate into a Julia "for" expression?

Question

I was writing a macro @vcomp (vector comprehension) based on Python's list comprehensions with a conditional clause to filter elements in a succinct way.

macro vcomp(comprehension::Expr, when::Symbol, condition)
    comp_head, comp_args = comprehension.head, comprehension.args
    comp_head ∉ [:comprehension, :typed_comprehension] && error("@vcomp not a comprehension")
    when ≠ :when && error("@vcomp expected `when`, got: `$when`")
    T = comp_head == :typed_comprehension ? comp_args[1] : nothing
    if VERSION < v"0.5-"
        element  = comp_head == :comprehension ? comp_args[1] : comp_args[2]
        sequence = comp_head == :comprehension ? comp_args[2] : comp_args[3]
    else
        element  = comp_head == :comprehension ? comp_args[1].args[1] : comp_args[2].args[1]
        sequence = comp_head == :comprehension ? comp_args[1].args[2] : comp_args[2].args[2]
    end
    result = T ≠ nothing ? :($T[]) : :([])
    block = Expr(:let, Expr(:block,
                Expr(:(=), :res, result),
                Expr(:for, sequence,
                    Expr(:if, condition,
                        Expr(:call, :push!, :res, element))),
                :res))
    return esc(block)
end

Used like this:

julia> @vcomp Int[i^3 for i in 1:10] when i % 2 == 0
5-element Array{Int64,1}:
    8
   64
  216
  512
 1000

Which expand to this:

julia> macroexpand(:(@vcomp Int[i^3 for i in 1:15] when i % 2 == 0))
:(let
        res = Int[]
        for i = 1:15
            if i % 2 == 0
                push!(res,i ^ 3)
            end
        end
        res
    end)

I was expecting to be able to write block like this:

block = quote
    let
        res = $result
        for $sequence
            if $condition
                push!(res, $element)
            end
        end
        res
    end
end

Which gives the following error:

• ERROR: syntax: invalid iteration specification

Instead of the way I came up with:

block = Expr(:let, Expr(:block,
            Expr(:(=), :res, result),
            Expr(:for, sequence,
                Expr(:if, condition,
                    Expr(:call, :push!, :res, element))),
            :res))

However I was able to do it using Expr(:for, ...) directly as shown above and as far as I understand this is a parser error (is this a bug?). I have also been unable to find examples of this kind of interpolation, this is what I've tried:

julia> ex₁ = :(i in 1:10)
:($(Expr(:in, :i, :(1:10))))

julia> ex₂ = :(i = 1:10)
:(i = 1:10)

julia> quote
           for $ex₁
ERROR: syntax: invalid iteration specification

julia> quote
           for $ex₂
ERROR: syntax: invalid iteration specification

Construct whole expression and inspect it:

julia> ex₃ = quote
           for i in 1:10
               print(i)
           end
       end
quote  # none, line 2:
    for i = 1:10 # none, line 3:
        print(i)
    end
end

julia> ex₃.args
2-element Array{Any,1}:
 :( # none, line 2:)
 :(for i = 1:10 # none, line 3:
        print(i)
    end)

julia> ex₃.args[2].args
2-element Array{Any,1}:
 :(i = 1:10)
 quote  # none, line 3:
    print(i)
end

julia> ex₃.args[2].args[1]
:(i = 1:10)

julia> ex₃.args[2].args[1] == ex₂    # what's the difference then?
true

This works but is less readable:

julia> ex₄ = Expr(:for, ex₁, :(print(i)))
:(for $(Expr(:in, :i, :(1:10)))
        print(i)
    end)

julia> ex₅ = Expr(:for, ex₂, :(print(i)))
:(for i = 1:10
        print(i)
    end)

julia> eval(ex₃)
12345678910
julia> eval(ex₄)
12345678910    
julia> eval(ex₅)
12345678910    

Is there a way I can use the more terse syntax instead? I find the current implementation difficult to read and reason about compared to what I was expecting to write.

Answer 1

First of all, I belive that comprehensions with guards are coming to Julia (in v0.5?).

To answer your question: The parser wants to be able to verify that its input is syntactically correct without looking into the actual value that is interpolated. Try eg

x, y = :i, :(1:10)
quote
    for $x = $y
    end
end

Now the parser can recognize the relevant parts of the syntax. (And you should get the same AST if you use for $x in $y instead.)