CODEWITHSUNDEEP
julia
Igor Rivin
Igor Rivin

Reputation: 4864

Julia automatic differentiation - is it completely busted?

Can Julia's ForwardDiff deal with closures? If not, it makes it not so very useful, but if yes, where did I go wrong below?

using ForwardDiff
function make_add(x)
   foo = y::Vector -> y+x
   return foo
end

zulu = make_add(17)

g = x-> ForwardDiff.gradient(zulu, x)

g([1, 2, 3])

MethodError: no method matching extract_gradient! 
  (::Type{ForwardDiff.Tag{##1#2{Int64},Int64}},  

 `::Array{Array{ForwardDiff.Dual{ForwardDiff.Tag{##1#2{Int64},Int64},Int64,3},1},1}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{##1#2{Int64},Int64},Int64,3},1})
Closest candidates are:
  extract_gradient!(::Type{T}, ::AbstractArray, ::ForwardDiff.Dual) where T at /home/jrun/.julia/v0.6/ForwardDiff/src/gradient.jl:76
  extract_gradient!(::Type{T}, ::AbstractArray, ::Real) where T at /home/jrun/.julia/v0.6/ForwardDiff/src/gradient.jl:75
  extract_gradient!(::Type{T}, ::DiffResults.DiffResult, ::ForwardDiff.Dual) where T at /home/jrun/.julia/v0.6/ForwardDiff/src/gradient.jl:70
  ...

Stacktrace:
 [1] gradient(::Function, ::Array{Int64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{##1#2{Int64},Int64},Int64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{##1#2{Int64},Int64},Int64,3},1}}, ::Val{true}) at /home/jrun/.julia/v0.6/ForwardDiff/src/gradient.jl:17
 [2] gradient(::Function, ::Array{Int64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{##1#2{Int64},Int64},Int64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{##1#2{Int64},Int64},Int64,3},1}}) at /home/jrun/.julia/v0.6/ForwardDiff/src/gradient.jl:15
 [3] (::##3#4)(::Array{Int64,1}) at ./In[8]:1`

EDIT In fact this has nothing to do with closures. Simply:

h = x-> ForwardDiff.gradient(x-> x+17.0, x)

bombs in exactly the same way

Upvotes: 2

Views: 570

Answers (2)

Bogumił Kamiński
Bogumił Kamiński

Reputation: 69939

The documentation of ForwardDiff.gadient states:

This method assumes that isa(f(x), Real).

The problem is that your function returns a vector not a scalar, so you need to use jacobian (which accepts arrays as return values):

julia> function make_add(x)
          foo = y::Vector -> y .+ x
             return foo
             end
make_add (generic function with 1 method)

julia> zulu = make_add(17)
#27 (generic function with 1 method)

julia> g = x-> ForwardDiff.jacobian(zulu, x)
#29 (generic function with 1 method)

julia> g([1, 2, 3])
3×3 Array{Int64,2}:
 1  0  0
 0  1  0
 0  0  1

Also note that I have added a dot before + (so it reads y .+ x), because on current release of Julia 1.0 you are not allowed to add a scalar to a vector without broadcasting.

Upvotes: 2

Chris Rackauckas
Chris Rackauckas

Reputation: 19152

gradient is defined for arrays. Use derivative on scalars.

Upvotes: 2

Related Questions