How to call a Variadic C function in Swift?

Question

I need help, I'm trying to call a C function in a Swift 5 function.

I have an error that I can't solve:

'evolis_get_devices' is unavailable: Variadic

function is unavailable

Do you have any idea how I can solve this?

The C function :

/// Get device list. Takes a list of model ended by zero.
EVOLIS_LIB int evolis_get_devices(evolis_device_t** devices, ...);

The swift function :

public static func getDevices() -> [Device] {
    var devices: UnsafeMutablePointer<evolis_device_t>?
    var count: Int32 = 0

    evolis_get_devices(&devices, count) // <- 'evolis_get_devices' is unavailable: Variadic

    var devicesList: [Device] = []

    if count > 0 {
        for i in 0...count - 1 {
            let device = Device(device_c: devices![Int(i)])
            devicesList.append(device)
        }
    }

    evolis_free_devices(devices)

    return devicesList
}

Answer 1

The Apple doc I linked in the comment specifically says:

Swift only imports C variadic functions that use a va_list for their arguments. C functions that use the ... syntax for variadic arguments are not imported, and therefore can’t be called using CVarArg arguments.

Hence the error you're seeing:

'evolis_get_devices' is unavailable: Variadic

Ideally your C function would have va_list equivalent similarly to how:

void NSLog(NSString *format, ...);

has a matching:

void NSLogv(NSString *format, va_list args);

But there's no guarantee it actually exists. Even in pure C a variadic function cannot call another variadic function by "passing through" its arguments. See the question here.

Let's consider possible workarounds:

1. C wrapping function(s) with predefined number of arguments which calls the variadic function. Those can be exposed to Swift. We can have multiple definitions with varying number of arguments, but this is not maintainable in the long run and pollutes the Swift symbol space with redundant entries. Meh.

2. Hypothetically we could come up with a C function wrapper with va_list arg as in NSLogv that would call the variadic genuine function. Let's explore this idea.

va_list is a dynamic collection where the original arguments are stored. The problem is, the information where each argument goes exactly is not stored in it, we only have the order. It is compiler responsibility to adhere to cpu platform and operating system's ABI rules when invoking calls to variadic functions. But there are even more challenges, the C language makes no assumption how those arguments will be handled or even when to stop the handling. To illustrate problem let's compare the signature of your C function:

/// Get device list. Takes a list of model ended by zero.
EVOLIS_LIB int evolis_get_devices(evolis_device_t** devices, ...);

vs

int printf ( const char * format, ... );

printf uses format string to know where to stop, but your function according to the doc expects a terminating NULL argument. Just by looking at va_list there is no way to tell, the information isn't there. Only based on the spec one can assume how the function works and how to use it, while C language compiler on its own has no way of figuring it out.

Which leaves us with important conclusion - our C function wrapper with va_list arg will need to make some assumptions.

Let's take another step back to analyze what exactly happens when a variadic function is called and how the argument handling actually works. For our analysis I came up with a function mimicking yours (kinda), I don't know what evolis_device_t** exactly is hence I went for void**.

int evolis_get_devices(void** devices, ...) {
    /* pointer to the variable arguments list */
    va_list pargs;

    /* Initialise pargs to point to the first optional argument */
    va_start(pargs, devices);

    int numberOfArgumentsProcessed = 0;
    void** currentVargsPointer = devices;
    while( currentVargsPointer ) {
        // there are probably ways to silence the warning here but I didn't bother
        printf("%lx\n", currentVargsPointer);
        // position next va_list entry
        currentVargsPointer = va_arg(pargs, void**);
        numberOfArgumentsProcessed++;

    }
    va_end(pargs);
    return numberOfArgumentsProcessed;
}

The variadic arguments void** devices, ... somehow become va_list by the means of va_start(pargs, devices). We actually do not need to go into details how that works. In my test function I print out the pointer variables arriving and return total handled number of those. I imagine your real function might actually return some status code as an int. However our test specimen is mostly about having a way to inspect what's going in.

Back in the day passing arguments on stack was the standard way of doing it on available hardware. The ancient C language did not specify argument signatures at all and as a side effect the number of arguments wasn't limited. Since every compiler could do it, it stuck, and later become a language feature known as variadic arguments.

We now arrive at present day where modern CPUs are packed with hardware registers and stack operations aren't that fast anymore in comparison. That is the reason modern ABIs define a hybrid approach - start with registers and use arguments pushed on stack when we run out of hardware registers.

The CPUs clearly differ between one another, a x86-64 has different register set and instructions compared to an Arm64 for example. Those nuances do not stop there, even operating systems occasionally introduce their own rules.

That leads us to another very important conclusion - whatever we do portability will suffer, since we are aiming at something even the C language can't express on its own. Hence it's accompanied by hardware and OS specific inline assembly.

I've written this lengthy introduction by stackoverflow standards, because I want to make it very clear what I'm giving you. Here's my shot at a C function wrapper with va_list of int evolis_get_devices(void** devices, ...) . I've made following assumptions:

• it has support for integer/pointer arguments and nothing else
• it currently only supports X86-64 & ARM64 as likely targets Swift is actually popular on
• Apple platforms take precedence as my target environments, Linux & Windows variants are a leap of faith, and are untested by me
• The last argument is assumed to be NULL e.g. evolis_get_devices(ptr, NULL)
• It used void** as the arg type, because ABI handling is exactly the same for all pointers in C, so evolis_device_t** is no different to void**

This C code is XCode compatible. This is the wrapper.h required eventually by the bridging header for Swift.

#ifndef wrapper_h
#define wrapper_h

#include <stdio.h>
#include <stdarg.h>

#endif /* wrapper_h */

int evolis_get_devices_using_va_list(va_list args);

Here's my int evolis_get_devices_using_va_list(va_list args) function in wrapper.c. The code is lengthy, but I did put a lot of comments with explanation what's going on.

#include "wrapper.h"
#include <stdbool.h>

#define STACK_NUMBER_OF_ARGS_MAXIMUM 256 // keep this multiple of 32 ( for arm 32 byte chunk processing)
#define STACK_ARGS_MAXIMUM_SIZE_IN_BYTES 8 * STACK_NUMBER_OF_ARGS_MAXIMUM

// from https://stackoverflow.com/a/5459929/5329717
#define STR_HELPER(x) #x
// c Preprocessor conversion from int to string
#define STR(x) STR_HELPER(x)

#if defined(__APPLE__)
#define C_SYMBOLS_LINKER_PREFIX "_"
#elif
#define C_SYMBOLS_LINKER_PREFIX ""
#endif
int evolis_get_devices_using_va_list(va_list args) {
    register int argsOnStackCounter
#if defined(__arm64__)
    asm("x8")
#elif __x86_64__
    asm("r11")
#endif
    = 0;

    void** stackVarsMemoryBuffer[STACK_NUMBER_OF_ARGS_MAXIMUM];
    register void* inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue
#if defined(__arm64__)
    asm("x9")
#elif __x86_64__
    asm("rax")
#endif
    = (void*)stackVarsMemoryBuffer;

    register void** arg1_arm64_outputReturnValue
#if defined(__arm64__)
    asm("x0")
#elif __x86_64__
    asm("rdi")
#endif
    = 0;
#if __x86_64__
    register void** arg2 asm("rsi") = 0;
#elif defined(__arm64__) && !defined(__APPLE__)
    register void** arg2 asm("x1") = 0;
#endif

#if __x86_64__
    register void** arg3 asm("rdx") = 0;
#elif defined(__arm64__) && !defined(__APPLE__)
    register void** arg3 asm("x2") = 0;
#endif

#if __x86_64__
    register void** arg4 asm("rcx") = 0;
#elif defined(__arm64__) && !defined(__APPLE__)
    register void** arg4 asm("x3") = 0;
#endif

#if __x86_64__
    register void** arg5 asm("r8") = 0;
#elif defined(__arm64__) && !defined(__APPLE__)
    register void** arg5 asm("x4") = 0;
#endif

#if __x86_64__
    register void** arg6 asm("r9") = 0;
#elif defined(__arm64__) && !defined(__APPLE__)
    register void** arg6 asm("x5") = 0;
#endif

#if defined(__arm64__) && !defined(__APPLE__)
    register void** arg7 asm("x6") = 0;
    register void** arg8 asm("x7") = 0;
#endif

    void** currentVargsPointer;
    int argsDepth = 0;
    do {
        // position on next va_list entry
        currentVargsPointer = va_arg(args, void**);
        switch (argsDepth) {
        case 0:
            arg1_arm64_outputReturnValue = currentVargsPointer;
            break;
#if __x86_64__ || (defined(__arm64__) && !defined(__APPLE__))
        case 1:
            arg2 = currentVargsPointer;
            break;
        case 2:
            arg3 = currentVargsPointer;
            break;
        case 3:
            arg4 = currentVargsPointer;
            break;
        case 4:
            arg5 = currentVargsPointer;
            break;
        case 5:
            arg6 = currentVargsPointer;
            break;
#endif
#if defined(__arm64__) && !defined(__APPLE__)
        case 6:
            arg7 = currentVargsPointer;
            break;
        case 7:
            arg8 = currentVargsPointer;
            break;
#endif
        default:
            // argument on stack prep
            stackVarsMemoryBuffer[argsOnStackCounter] = currentVargsPointer;
            argsOnStackCounter++;
            break;
        }
        argsDepth++;
    } while( currentVargsPointer );
#if defined(__arm64__)
    asm volatile(// reserve space on stack
                 "sub    sp, sp," STR(STACK_ARGS_MAXIMUM_SIZE_IN_BYTES) "\t\n"
                 // calculate how many 32 byte chunks we need for x8 amount of stack args
                 "lsr    x8,x8, #2 \t\n" 
                 //bit shift right equivalent to int division x8 / 8
                 "add    x8,x8, #1 \t\n"
                 // use another register for destination to keep the original sp intact
                 "mov    x10, sp      \t\n"
                 // copy in 32 byte chunks, after that shift source and destination +32
                 "1:ldp  q0, q1, [x9], #32 \t\n"
                 "stp    q0, q1, [x10], #32 \t\n"
                 // move loop iterator by 1
                 "sub    w8, w8, #1   \t\n"
                 // check if 32 byte chunks are left to handle
                 "cmp    w8, #0       \t\n"
                 // jmp backwards if there are chunks left to process
                 "b.hi   1b           \t\n"
                 // we are done, time to call
                 "2:     bl " C_SYMBOLS_LINKER_PREFIX "evolis_get_devices \t\n"
                 // restore stack to entry state
                 "add    sp, sp," STR(STACK_ARGS_MAXIMUM_SIZE_IN_BYTES) "\t\n"
                 // mark the outputs:
                 // x0     it's the return value of the function we call
                 // "dirty" outputs, not really needed for anything afterwards:
                 // x8     argsOnStackCounter
                 // x9     inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue
                 :"=r"(inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue), "=r"(argsOnStackCounter), "=r"(arg1_arm64_outputReturnValue)
    #if !defined(__APPLE__)
                 // potential callee outputs under AARCH64 ABI for Windows & Linux
                 //    x1        former arg2
                 //    x2        former arg3
                 //    x3        former arg4
                 //    x4        former arg5
                 //    x5        former arg6
                 //    x6        former arg7
                 //    x7        former arg8
                 , "=r"(arg2), "=r"(arg3), "=r"(arg4), "=r"(arg5), "=r"(arg6), "=r"(arg7), "=r"(arg8)
    #endif
                 // mark the inputs
                 // x0     arg1
                 // x8     argsOnStackCounter
                 // x9     inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue
                 :"r"(arg1_arm64_outputReturnValue),"r"(inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue),"r"(argsOnStackCounter)
    #if !defined(__APPLE__)
                 // more register input args on Windows & Linux Aarch64 ABIs
                 //    x1        arg2
                 //    x2        arg3
                 //    x3        arg4
                 //    x4        arg5
                 //    x5        arg6
                 //    x6        arg7
                 //    x7        arg8
                 , "r"(arg2), "r"(arg3), "r"(arg4), "r"(arg5), "r"(arg6), "r"(arg7), "r"(arg8)
    #endif
                 // clobbered registers
                 : // by us and potentially by callee
                 "x10","x11","x12",
                 // v0, v1 those are synonyms for q0,q1 that do work in XCode's clang inline assembly in the clobber list, see https://stackoverflow.com/a/19984325/5329717
                 "v0","v1",
                 // clobbered potentially by the callee
    #if defined(__APPLE__)
                 "x2","x3","x4","x5","x6","x7",
    #endif
                 "x13","x14","x15");
#elif __x86_64__
     asm volatile(".intel_syntax noprefix \t\n"
                  // reserve space on stack for the args of the call
                  "sub rsp," STR(STACK_ARGS_MAXIMUM_SIZE_IN_BYTES) "\t\n"
                  // make a copy on the stack of original values in rsi, rdi, rcx representing args so we can use movsq
                  "push rsi \t\n"
                  "push rdi \t\n"
                  "push rcx \t\n"
                  // position rdi on the original reserved space on stack
                  // adjusted +24 because stack grew "downwards" when rsi,rdi,rcx got pushed
                  "lea rdi, [rsp + 24] \t\n"
                  // position rsi on the inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue
                  "mov rsi, rax \t\n"
                  // put number of arg stack arguments into ecx
                  "mov ecx, r11d \t\n"
                  // copy (stack args number) * 8 bytes from rsi to rdi
                  "rep movsq \t\n"
                  // restore rsi,rdi,rcx representing args in registers
                  "pop rcx \t\n"
                  "pop rdi \t\n"
                  "pop rsi \t\n"
                  // args are prepared, we are almost ready to call
                  // by AMD64 ABI convention register al will denote the number of float arguments in variadic call, zero in our case
                  "xor eax,eax \t\n"
                  // make the actual call to evolis_get_devices(...)
                  "call " C_SYMBOLS_LINKER_PREFIX "evolis_get_devices \t\n"
                  // restore stack to entry state
                  "add rsp," STR(STACK_ARGS_MAXIMUM_SIZE_IN_BYTES) "\t\n"
                  // mark the outputs:
                  //    rax        it's the return value of the function we call
                  // "dirty" outputs, not really needed for anything afterwards:
                  //    r11        can be clobbered by the callee
                  // potential callee outputs under AMD64 ABI
                  //    rdi        former arg1
                  //    rsi        former arg2
                  //    rdx        former arg3
                  //    rcx        former arg4
                  //    r8         former arg5
                  //    r9         former arg6
                  :"=r"(inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue), "=r"(argsOnStackCounter), "=r"(arg1_arm64_outputReturnValue), "=r"(arg2),"=r"(arg3),"=r"(arg4),"=r"(arg5),"=r"(arg6)
                  // mark the inputs:
                  //    rdi        arg1
                  //    rsi        arg2
                  //    rdx        arg3
                  //    rcx        arg4
                  //    r8         arg5
                  //    r9         arg6
                  //    rax        inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue
                  //    r11        argsOnStackCounter
                  :"r"(arg1_arm64_outputReturnValue), "r"(arg2), "r"(arg3), "r"(arg4), "r"(arg5), "r"(arg6), "r"(inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue), "r"(argsOnStackCounter)
                  // clobbered registers (potentially by the callee)
                  : "r10"
                  );
#endif

// there are probably ways to silence the warning here but I didn't bother
#if __x86_64__
    return (int)inputStackVarsMemoryBufferPointer_x86_64_outputReturnValue;
#elif defined(__arm64__)
    return (int)arg1_arm64_outputReturnValue;
#endif
}

This is my test code in Swift to invoke this (notice those aren't valid pointers, I used consecutive integer values to make them stand out in debugging):

        let args: [OpaquePointer] = [ OpaquePointer.init(bitPattern: 1)!,
                                      OpaquePointer.init(bitPattern: 2)!,
                                      OpaquePointer.init(bitPattern: 3)!,
                                      OpaquePointer.init(bitPattern: 4)!,
                                      OpaquePointer.init(bitPattern: 5)!,
                                      OpaquePointer.init(bitPattern: 6)!,
                                      OpaquePointer.init(bitPattern: 7)!,
                                      OpaquePointer.init(bitPattern: 8)!,
                                      OpaquePointer.init(bitPattern: 9)!
        ]
        withVaList(args + [Int(0)], {
            print("Number of args processed:",evolis_get_devices_using_va_list($0))
        })

Also possible with Swift variadic function of course:

    func variadicSwiftFunction(devices: UnsafeMutablePointer<OpaquePointer>...) {
        withVaList(devices + [Int(0)], {
            print("Number of args processed:",evolis_get_devices_using_va_list($0))
        })
    }

In the Swift code notice the last argument NULL is passed as weirdly looking Int(0). It's a workaround to a Swift bug discussed here.