CODEWITHSUNDEEP
pythoncnumpyctypes
Anna
Anna

Reputation: 95

Freeing memory when using ctypes

I am using ctypes to try and speed up my code.

My problem is similar to the one in this tutorial : https://cvstuff.wordpress.com/2014/11/27/wraping-c-code-with-python-ctypes-memory-and-pointers/

As pointed out in the tutorial I should free the memory after using the C function. Here is my C code

//C functions
double* getStuff(double *R_list, int items){

    double results[items];  
    double* results_p;   

    for(int i = 0; i < items; i++){ 
            res = calculation ; \\do some calculation
            results[i] = res;   }

        results_p = results;
        printf("C allocated address %p \n", results_p);

return results_p; }

void free_mem(double *a){
    printf("freeing address: %p\n", a);

    free(a);    }

Which I compile with gcc -shared -Wl,-lgsl,-soname, simps -o libsimps.so -fPIC simps.c

And python: 

//Python
from ctypes import *
import numpy as np


mydll = CDLL("libsimps.so")
mydll.getStuff.restype = POINTER(c_double)
mydll.getStuff.argtypes = [POINTER(c_double),c_int]
mydll.free_mem.restype = None
mydll.free_mem.argtypes = [POINTER(c_double)]

R = np.logspace(np.log10(0.011),1, 100, dtype = float) #input


tracers = c_int(len(R))

R_c = R.ctypes.data_as(POINTER(c_double))


for_list = mydll.getStuff(R_c,tracers)
print 'Python allocated', hex(for_list)
for_list_py = np.array(np.fromiter(for_list, dtype=np.float64, count=len(R)))
mydll.free_mem(for_list)

Up to the last line the code does what I want it to and the for_list_py values are correct. However, when I try to free the memory, I get a Segmentation fault and on closer inspection the address associated with for_list --> hex(for_list) is different to the one allocated to results_p within C part of the code.

As pointed out in this question, Python ctypes: how to free memory? Getting invalid pointer error , for_list will return the same address if mydll.getStuff.restype is set to c_void_p. But then I struggle to put the actual values I want into for_list_py. This is what I've tried: cast(for_list, POINTER(c_double) ) for_list_py = np.array(np.fromiter(for_list, dtype=np.float64, count=len(R))) mydll.free_mem(for_list) where the cast operation seems to change for_list into an integer. I'm fairly new to C and very confused. Do I need to free that chunk of memory? If so, how do I do that whilst also keeping the output in a numpy array? Thanks!

Edit: It appears that the address allocated in C and the one I'm trying to free are the same, though I still recieve a Segmentation fault. 

C allocated address 0x7ffe559a3960
freeing address: 0x7ffe559a3960
Segmentation fault

If I do print for_list I get <__main__.LP_c_double object at 0x7fe2fc93ab00>

Conclusion

Just to let everyone know, I've struggled with c_types for a bit.

I've ended up opting for SWIG instead of c_types. I've found that the code runs faster on the whole (compared to the version presented here). I found this documentation on dealing with memory deallocation in SWIG very useful https://scipy-cookbook.readthedocs.io/items/SWIG_Memory_Deallocation.html as well as the fact that SWIG gives you a very easy way of dealing with numpy n-dimensional arrays.

Upvotes: 3

Views: 2608

Answers (1)

Leo
Leo

Reputation: 751

After getStuff function exits, the memory allocated to results array is not available any more, so when you try to free it, it crashes the program.

Try this instead:

double* getStuff(double *R_list, int items)
{
  double* results_p = malloc(sizeof((*results_p) * (items + 1));
  if (results_p == NULL)
  {
    // handle error
  }

  for(int i = 0; i < items; i++)
  { 
    res = calculation ; \\do some calculation
    results_p[i] = res;
  }

  printf("C allocated address %p \n", results_p);

  return results_p;
}

Upvotes: 3

Related Questions