CODEWITHSUNDEEP
c++luatorch
Conchylicultor
Conchylicultor

Reputation: 5729

Does lua garbage collect torch tensors pushed from C?

According to this answer (https://groups.google.com/forum/#!msg/torch7/rmIcBYCiFG8/IC68Xzd3DgAJ), it seems that torch automatically frees tensor which are not used anymore. And I already had this issue were the tensor pointer where automatically free by lua, causing a segfault in the c program. However, from my experiments, it seems that not manually calling free on the C code leads to a memory leak, where the program memory is constantly increasing.

Here is my test code:

#include <iostream>
#include <chrono>
#include <thread>

#include <lua.hpp>
extern "C" {
#include <TH.h>
#include <luaT.h>
}

using namespace std;

int main(int argc, char** argv)
{
    lua_State* L = luaL_newstate();

    luaL_openlibs(L);

    lua_getglobal(L, "require");
    lua_pushstring(L, "torch");
    lua_pcall(L,1,0,0); // require "torch"

    int h=224, w=224, nb_channel=3;

    int len = h * w * nb_channel;
    long stride_1 = h * w;
    long stride_2 = w;
    long stride_3 = 1;

    for (size_t i = 0 ; i < 1000000 ; ++i)
    {
        float *tensorData = (float*)malloc(sizeof(float)*len); // Should be when the storage is freed
        THFloatStorage *storage = THFloatStorage_newWithData(tensorData, len); // Memory released when tensor input is freed by lua garbadge collector
        THFloatTensor* input = THFloatTensor_newWithStorage3d(storage,
                                                              0, // Offset
                                                              nb_channel, stride_1, // Channels
                                                              h,          stride_2, // Height
                                                              w,          stride_3); // Width
        // Do some initialisation of the tensor...

        luaT_pushudata(L, (void*) input, "torch.FloatTensor"); // Send the tensor to lua
        // Do some stuff with the input... (call lua torch scripts...)
        lua_pop(L, 1);

        // If those two lines is not set, we get a memory leak
        THFloatTensor_free(input); // Will  sometimes create a segfault as the tensor seems to be deleted by the lua garbadge collector
        THFloatStorage_free(storage);

        if (i%10000 == 0)
            std::cout << i << std::endl;
        std::this_thread::sleep_for(std::chrono::microseconds(1));
    }



    return 0;
}

How to correctly manage the memory with c and lua to avoid segfault without having memory leaks ?

Upvotes: 0

Views: 283

Answers (1)

James
James

Reputation: 1

I think you are getting the memory leak because of not deallocating memory for "tensordata" in this line float tensorData = (float)malloc(sizeof(float)*len);

Hope it helps

Upvotes: 0

Related Questions