How can I dynamically create a function?

Question

I try to summarize my C problem as follows:

• I have a matrix and I want to be able to use some function to generate its elements.

• function has no arguments

Hence I define the following:

typedef double(function)(unsigned int,unsigned int);

/* writes f(x,y) to each element x,y of the matrix*/
void apply(double ** matrix, function * f);

Now I need to generate constant functions within the code. I thought about creating a nested function and returning its pointer, but GCC manual (which allows nested functions) says:

"If you try to call the nested function through its address after the containing function has exited, all hell will break loose."

which I would kind of expect from this code:

function * createConstantFunction(const double value){
 double function(unsigned int,unsigned int){
   return value;
 }
 return &function;
}

How can I get it to work?

Answer 1

I'm guessing that you are familiar with some programming language which supports closure mechanism? Unfortunately, C does not support closures like that itself.

You could find some useful libraries which simulate closures in C if you insisted on closure. But most of those libraries are complex and machine-dependent.
Alternatively, you can change your mind to agree with the C-style closure if you could change the signature of double ()(unsigned,unsigned);.

In C, a function has no data (or context) except the arguments passed to it and the static variables which it could access.
So the context must be passed by yourself. Here is a example using an extra argument:

// first, add one extra parameter in the signature of function.
typedef double(function)(double extra, unsigned int,unsigned int);

// second, add one extra parameter in the signature of apply
void apply(double* matrix,unsigned width,unsigned height, function* f, double extra)
{
        for (unsigned y=0; y< height; ++y)
            for (unsigned x=0; x< width ++x)
                    matrix[ y*width + x ] = f(x, y, extra);
        // apply will passing extra to f
}

// third, in constant_function, we could get the context: double extra, and return it
double constant_function(double value, unsigned x,unsigned y) { return value; }

void test(void)
{
        double* matrix = get_a_matrix();
        // fourth, passing the extra parameter to apply
        apply(matrix, w, h, &constant_function, 1212.0);
        // the matrix will be filled with 1212.0
}

Is a double extra enough? Yes, but only in this case.
What should we do if more context is required?
In C, the general purpose parameter is void*: we can pass any context though one void* parameter by passing the address of context.

Here is another example :

typedef double (function)(void* context, int, int );
void apply(double* matrix, int width,int height,function* f,void* context)
{
        for (int y=0; y< height; ++y)
            for (int x=0; x< width ++x)
                    matrix[ y*width + x ] = f(x, y, context); // passing the context
}
double constant_function(void* context,int x,int y)
{
        // this function use an extra double parameter \
        //    and context points to its address
        double* d = context;
        return *d;
}
void test(void)
{
        double* matrix = get_a_matrix();
        double context = 326.0;
        // fill matrix with 326.0
        apply( matrix, w, h, &constant_function, &context);
}

(function,context) pair like &constant_function,&context is the C-style closure.
Each function(F) that needs a closure must has one context parameter which will be passed to closure as its context. And the caller of F must use a correct (f,c) pair.

If you can change the signature of function to fit to C-style closure, your code will be simple and machine-independent.
If you can't (function and apply is not written by you), try to persuade the author to change that code.
If that's not possible, you have no choice but to use some closure library.

Answer 2

Usually functions are explained as binary trees for dynamic use, so you would probably need to create simple static functions (operations or objects) and then combine them as necessary, like LEGO blocks.

Answer 3

Since you want to generate a function that follows a simple recipe, this shouldn't be too tricky to do with some inline assembly and a block of executable/writable memory.

This approach feels a bit hacky so I wouldn't recommend it in production code. Due to the use of inline assembly this solution works only on Intel x86-64 / AMD64, and will need to be translated to work with other architectures.

You might prefer this to other JIT-based solutions as it does not depend on any external library.

If you would like a longer explanation of how the below code works, leave a comment and I'll add it.

For security reasons, the code page should be marked PROT_READ|PROT_EXEC after a function is generated (see mprotect).

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include <sys/mman.h>

int snippet_processor(char *buffer, double value, int action);

enum snippet_actions {
    S_CALC_SIZE,
    S_COPY,
};

typedef double (*callback_t) (unsigned int, unsigned int);

int main(int argc, char **argv) {

    unsigned int pagesize = 4096;
    char *codepage = 0;
    int snipsz = 0;

    callback_t f;

    /* allocate some readable, writable and executable memory */
    codepage = mmap(codepage,
        pagesize,
        PROT_READ | PROT_WRITE | PROT_EXEC,
        MAP_ANONYMOUS | MAP_PRIVATE,
        0,
        0);

    // generate one function at `codepage` and call it
    snipsz += snippet_processor(codepage, 12.55, S_COPY);
    f = (callback_t) (codepage);
    printf("result :: %f\n", f(1, 2));

    /* ensure the next code address is byte aligned
     * - add 7 bits to ensure an overflow to the next byte.
     *   If it doesn't overflow then it was already byte aligned.
     * - Next, throw away any of the "extra" bit from the overflow,
     *   by using the negative of the alignment value 
     *   (see how 2's complement works.
     */
    codepage += (snipsz + 7) & -8;

    // generate another function at `codepage` and call it
    snipsz += snippet_processor(codepage, 16.1234, S_COPY);
    f = (callback_t) (codepage);
    printf("result :: %f\n", f(1, 2));
}

int snippet_processor(char *buffer, double value, int action) {
    static void *snip_start = NULL; 
    static void *snip_end = NULL; 
    static void *double_start = NULL; 
    static int double_offset_start = 0;
    static int size;

    char *i, *j;
    int sz;

    char *func_start;
    func_start = buffer;

    if (snip_start == NULL) {
        asm volatile(
            // Don't actually execute the dynamic code snippet upon entry
            "jmp .snippet_end\n"

            /* BEGIN snippet */
            ".snippet_begin:\n"
            "movq .value_start(%%rip), %%rax\n"
            "movd %%rax, %%xmm0\n"
            "ret\n"

            /* this is where we store the value returned by this function */
            ".value_start:\n"
            ".double 1.34\n"
            ".snippet_end:\n"
            /* END snippet */

            "leaq .snippet_begin(%%rip), %0\n"
            "leaq .snippet_end(%%rip), %1\n"
            "leaq .value_start(%%rip), %2\n"
            : 
            "=r"(snip_start),
            "=r"(snip_end),
            "=r"(double_start)
        );
        double_offset_start = (double_start - snip_start);
        size = (snip_end - snip_start);
    }

    if (action == S_COPY) {
        /* copy the snippet value */
        i = snip_start;
        while (i != snip_end) *(buffer++) = *(i++); 

        /* copy the float value */
        sz = sizeof(double);
        i = func_start + double_offset_start; 
        j = (char *) &value;

        while (sz--) *(i++) = *(j++); 
    }

    return size;
}

Answer 4

Using FFCALL, which handles the platform-specific trickery to make this work:

#include <stdio.h>
#include <stdarg.h>
#include <callback.h>

static double internalDoubleFunction(const double value, ...) {
    return value;
}
double (*constDoubleFunction(const double value))() {
    return alloc_callback(&internalDoubleFunction, value);
}

main() {
    double (*fn)(unsigned int, unsigned int) = constDoubleFunction(5.0);
    printf("%g\n", (*fn)(3, 4));
    free_callback(fn);
    return 0;
}

(Untested since I don't have FFCALL currently installed, but I remember that it works something like this.)

Answer 5

If you really need to dynamically create the functions, maybe an embedded C interpreter could help. I've just googled for "embedded C interpreter" and got Ch as a result:

http://www.softintegration.com/

Never heard of it, so I don't know anything about it, but it seems to be worth a look.

Answer 6

It looks like you're coming from another language where you commonly use this type of code. C doesn't support it and it although you could certainly cook up something to dynamically generate code, it is very likely that this isn't worth the effort.

What you need to do instead is add an extra parameter to the function that references the matrix it is supposed to work on. This is most likely what a language supporting dynamic functions would do internally anyway.

Answer 7

This mechanism is called reflection where code modifies its own behavior at runtime. Java supports reflection api to do this job.
But I think this support is not available in C.

Sun web site says :

Reflection is powerful, but should not be used indiscriminately. If it is possible to perform an operation without using reflection, then it is preferable to avoid using it. The following concerns should be kept in mind when accessing code via reflection.

Drawbacks of Reflection

Performance Overhead Because reflection involves types that are dynamically resolved, certain Java virtual machine optimizations can not be performed. Consequently, reflective operations have slower performance than their non-reflective counterparts, and should be avoided in sections of code which are called frequently in performance-sensitive applications.

Security Restrictions

Reflection requires a runtime permission which may not be present when running under a security manager. This is in an important consideration for code which has to run in a restricted security context, such as in an Applet.

Exposure of Internals

Since reflection allows code to perform operations that would be illegal in non-reflective code, such as accessing private fields and methods, the use of reflection can result in unexpected side-effects, which may render code dysfunctional and may destroy portability. Reflective code breaks abstractions and therefore may change behavior with upgrades of the platform. .

Answer 8

As unwind already mentioned, "creating code at runtime" is not supported by the language and will be a lot of work.

I haven't used it myself, but one of my co-workers swears by Lua, an "embedded language". There is a Lua C API which will (theoretically, at least) allow you to perform dynamic (scripted) operations.

Of course, the downside would be that the end user may need some sort of training in Lua.

It may be a dumb question, but why does the function have to be generated within your application? Similarly what advantage does the end-user get from generating the function themselves (as opposed to selecting from one or more predefined functions that you provide)?

Answer 9

One way of doing would be to write a standard C file with the set of functions you want, compile it via gcc and the load it as a dynamic library to get pointers to the functions.

Ultimately, it probably would be better if you were able to specify your functions without having to define them on-the-fly (like via having a generic template function that takes arguments that define its specific behavior).

Answer 10

C is a compiled language. You can't create code at run-time "in C"; there is no specific C support to emit instructions to memory and so on. You can of course try just allocating memory, making sure it's executable, and emit raw machine code there. Then call it from C using a suitable function pointer.

You won't get any help from the language itself though, this is just like generating code and calling it in BASIC on an old 8-bit machine.

Answer 11

If you want to write code on the fly for execution, nanojit might be a good way to go.

In your code above, you're trying to create a closure. C doesn't support that. There are some heinous ways to fake it, but out of the box you're not going to be able to runtime bind a variable into your function.