Can I write I Julia method that works "whenever possible" like a c++ template function?

Question

rand works with ranges:

rand(1:10)

I'd like to make rand work with Array, and anything that is indexable and has length:

import Base.Random
rand(thing) = thing[rand(1:length(thing))]

array = {1, 2, 3}
myRand(array)

range = 1:8
myRand(range)

tupple = (1, 2, 3, "a", "b", "c")
myRand(tupple)

… but if I try this, my implementation stack overflows, presumably because it is completely general and matches everything passed, so it ends up calling itself?

Is there a way to fix this? I want to better understand Julia's polymorphic functions rather than get a fix for this particular (probably silly) function specialisation.

Is there also a tool to discover the various implementations that are available, and debug which will be called with particular arguments?

---

Okay, some digging. This is interesting…

I'll start up a fresh REPL, and:

julia> import Base.Random

julia> rand(thing) = thing[rand(1:length(thing))]
rand (generic function with 1 method)

julia> rand({1,2,3})
ERROR: stack overflow
 in rand at none:1 (repeats 80000 times)

…Oh dear, that's the recursive call and stack overflow I was talking about.

But, watch this. I kill Julia and start the REPL again. This time I import Base.Random.rand:

julia> import Base.Random.rand

julia> rand(thing) = thing[rand(1:length(thing))]
rand (generic function with 33 methods)

julia> rand({1,2,3})
3

It works – it added my new implementation to all the others, and picked the right one.

So, the answer to my first question seems to be – "it just works". Which is amazing. How does that work?!

But there's a slightly less interesting sounding question about modules, and why import Base.Random doesn't pull in the rand method or give an error, but import Base.Random.rand does.

Answer 1

Method extension

As some have pointed, Julia let you extend functions: you can have functions that work differently for different types (see this part of the documentation).

For example:

f(x) = 2
f(x::Int) = x

In this example, we have a version (or method) of the function that is called if (and only if) the argument is of the type Int. The first one is called otherwise.

We say that we have extended the f function, and now it has 2 methods.

Your Problem

What you want, then, is to extend the rand function.

You want your rand function, if called with a argument that was not caught by other methods, to execute thing[rand(1:length(thing))]. If done correctly, you would call the rand method that is applied to a Range object, since you are passing 1:length(thing) as argument.

Although flawed (what if thing doesn't have a length, e.g. a complex number?), I would say your first attempt was very reasonable.

The problem: rand couldn't be extended on the first version of your program. According to this piece of documentation, writing import Base.Random doesn't make rand available for method extension.

While trying to extend rand, you actually overwrite the rand function. After this, when you call the function rand, there is only your method!

Remember, you were relying on the fact that a method for ranges (e.g. rand(1:10)) was defined elsewere, and that it gave the result you expected. What happened was that this method was overwritten by yours, so your method is called again (recursively).

The solution: import rand such as it is available to extension. You can see that on the table on the documentation.

Notice that your second program (the one with import Base.Random.rand) and Colin's program (the one with importall Base.Random) did exactly that. That's why they work.

Keep in mind what methods are or are not available for extension, and if the documentation isn't clear enough, a bug report (or maybe a fix) will be welcomed.

Answer 2

As another pointed out, explicitly specifying Base.rand= instead of rand= adds a method to the rand function definition in Base, instead of completely replacing it.

For example,

julia> Base.rand(x::Any)=x[rand(1:length(x)]
rand (generic function with 33 methods)

The reason this works has to do with Julia's abstract type system. When Julia is looking for a function with a matching method signature, it starts at the leaf node of the defined types, (e.g. whether or not x is an Int8), and then moves up the type hierarchy until it finds a function with a matching signature. At the top of the type hierarchy is the catchall type Any. If there is no matching function at any level of the hierarchy, then the function call fails.

This is best illustrated by a simple example. We'll create a function f that responds to several different types in the type hierarchy with the name of the type:

julia> f(x::Int)="Int"
f (generic function with 1 method)

julia> f(x::Real)="Real"
f (generic function with 2 methods)

julia> f(x::Number)="Number"
f (generic function with 3 methods)

julia> f(x::Any)="Any"
f (generic function with 4 methods)

julia> f(x::Array)="Array"
f (generic function with 5 methods)

julia> f(4) # typeof(4) isInt64
"Int"

julia> f(2.0) # typeof(2.0) is Float64
"Real"

julia> f(3im) # typeof(3im) is Complex{Int64}

"Number"

julia> f([1,2]) # typeof([1,2]) is Array{Int64, 1}
"Array"

julia> f(Dict(1=>3,4=>5)) # typeof(Dict(1=>3,4=>5)) is Dict{Int64, Int64}
"Any" 

Answer 3

I was going to make this a comment, but it ended up being too long:

Note two other approaches that will also "just work":

Base.rand(thing) = thing[rand(1:length(thing))]
rand({1,2,3})

Or you could also use:

importall Base.Random
rand(thing) = thing[rand(1:length(thing))]
rand({1,2,3})

When you just use import Base.Random that won't actually allow you to extend Base.rand with your locally defined rand. This is because the import statement should only be used with functions, not modules. If you want to import all the functions in a module, you need to use importall (as in my example above). Alternatively (as I have done above), you can just directly reference the module in your function definition Base.rand.

Admittedly, last time I looked through the docs, this point could definitely have been made clearer. It may be addressed in the latest version though.

As to the rest of your question (why it "just works"?), I'm not convinced I can give a smart concise answer to this so I might leave it for someone else.

Also, partial duplicate.