What is the error "A type instantiation involves a byref type." and what is a workaround in F#

Question

I have some code wrapping TA-Lib and a lot of the wrappers are very similar:

let sma (timePeriod: int) (data: float[]) =
    let mutable outStartIndex = 0
    let mutable outNbElement = 0

    let mutable smaData : float array = Array.zeroCreate (data.Length - timePeriod + 1)

    let retCode = Core.Sma(0, (data.Length - 1), data, timePeriod, &outStartIndex, &outNbElement, smaData)

    if retCode <> Core.RetCode.Success then
        invalidOp (sprintf "AssertRetCodeSuccess")

    let padding = Array.create (timePeriod - 1) System.Double.NaN
    Array.append padding smaData



let ema (timePeriod: int) (data: float[]) =
    let mutable outStartIndex = 0
    let mutable outNbElement = 0
    let mutable emaData : float array = Array.zeroCreate (data.Length - timePeriod + 1)

    let retCode = Core.Ema(0, (data.Length - 1), data, timePeriod, &outStartIndex, &outNbElement, emaData)

    if retCode <> Core.RetCode.Success then
        invalidOp (sprintf "AssertRetCodeSuccess")

    let padding = Array.create (timePeriod - 1) System.Double.NaN
    Array.append padding emaData

What I could like to do is create a generic function where I can just pass the TA-Lib function to call. Something like:

let myGenericFunction (timePeriod: int) (data: float[]) TALibFunc =
    let mutable outStartIndex = 0
    let mutable outNbElement = 0

    let mutable smaData : float array = Array.zeroCreate (data.Length - timePeriod + 1)

    let retCode = TALibFunc(0, (data.Length - 1), data, timePeriod, &outStartIndex, &outNbElement, smaData)

    if retCode <> Core.RetCode.Success then
        invalidOp (sprintf "AssertRetCodeSuccess")

    let padding = Array.create (timePeriod - 1) System.Double.NaN
    Array.append padding smaData

but the error I am getting is:

[FS0412] A type instantiation involves a byref type. This is not permitted by the rules of Common IL.

Is there a workaround for this? I am not familiar with this issue.

Answer 1

Short answer: replace your mutable parameters with ref.

---

TA-Lib has a very unfortunate API: those pesky out-parameters (known in F# as byref), they always make trouble. In this case, they cannot be part of a generic type instantiation.

Here's a much shorter example. Consider a good old list<T>. We can make an empty list<int>:

let noInts = [] : list<int>

But what if those ints are byref?

let noRefs = [] : list< byref<int> >

No can do - says the compiler. A type instantiation involves a byref type. This is not permitted by the rules of Common IL. Sorry.

---

In your case, the last parameter of myGenericFunction is an F# function. In F# functions are represented by the type FSharpFunc<T, R> (where T is argument and R is result). So the type of your last parameter is this:

FSharpFunc< int * int * float array * int * byref<int> * byref<int> * float array, int >

See those two byref<int>s in there? Those are &outStartIndex and &outNbElement. And they are forbidden in a generic instantiation. Tough luck.

---

But there is hope!

The mutable keyword is only one of two ways to make mutable cells in F#. The other way is ref:

let x = ref 0     // Initialization
printfn "%d" !x   // Prints "0"
x := 42           // Mutation
printfn "%d" !x   // Prints "42"

It's an old-school thing, predates mutable, is implemented as a library (as opposed to a language construct), and in most cases mutable is better. But this is not one of those cases!

It turns out that:

1. Unlike true .NET CIL out-parameters, ref cells can be part of generic instantiation just fine. Because, from .NET point of view, they're nothing special - just another class.
2. The F# compiler has special sauce for them: when the expected type is a ref, but you're trying to pass a function with an out-parameter in its place, the compiler will automatically generate some wrapping code for you.

So, armed with this knowledge, you can modify myGenericFunction like this:

let myGenericFunction (timePeriod: int) (data: float[]) TALibFunc =
    let outStartIndex = ref 0
    let outNbElement = ref 0

    let mutable smaData : float array = Array.zeroCreate (data.Length - timePeriod + 1)

    let retCode = TALibFunc(0, (data.Length - 1), data, timePeriod, outStartIndex, outNbElement, smaData)

    ...

And then the consumers can call it like this:

myGenericFunction 42 [|1; 2; 3|] Core.Sma // Wrapping code gets generated here