Lua 5.2 - C++ object within an object (using lua_lightuserdata)

Question

edit: [SOLUTION IN ANSWER 2]

I am new to LUA and am having trouble trying to do what I want. I have a C++ object that looks like this:

C++ Object definitions

struct TLimit
{
    bool   enabled;
    double value;

    TLimit() : enabled(false), value(0.0) {}
    ~TLimit() {}
};

class TMeaurement
{
public:
    TMeasurement() : meas(0.0) {}
    ~TMeasurement() {}

    TLimit min;
    TLimit max;
    double meas;
};

I want to be able in LUA to access an object of type TMeasurement in the following form:

LUA desired use

-- objmeas is an instance of TMeasurement
objmeas.min.enabled = true
print(objmeas.min.value);

...etc

The other one thing, I do not want LUA to allocate memory for the instance of the object of type TMeasurement. That will be done in my C++ code. I have tried many different things, all unsuccessful. I will post now the last of my tries.

In my C++ code, I have defined the following:

TLimit - Get function that will be mapped to __index

#define LUA_MEAS_LIMIT    "itse.measurement.limit"

extern int llim_get(lua_State* L)
{
    TLimit*     lim = (TLimit*)lua_chekuserdata(L, 1, LUA_MEAS_LIMIT);
    std::string key = std::string(luaL_checkstring(L, 2));

    //-- this is only to check what is going on
    std::cout << "lim.get: " << key << std::endl;

    if(key.find("enabled") == 0)
        lua_pushboolean(L, lim->enabled);
    else if(key.find("value") == 0)
        lua_pushnumber(L, lim->value);
    else
        return 0;   //-- should return some sort of error, but let me get this working first

    return 1;
}

TLimit - Set function that will be mapped to __newindex

extern int llim_set(lua_State* L)
{
    TLimit*     lim = (TLimit*)lua_chekuserdata(L, 1, LUA_MEAS_LIMIT);
    std::string key = std::string(luaL_checkstring(L, 2));

    //-- this is only to check what is going on
    std::cout << "limit.set: " << key << " <-" << std::endl;

    if(key.find("enabled") == 0)
        lim->enabled = lua_toboolean(L, 3);
    else if(key.find("value") == 0)
        lim->value = lua_tonumber(L, 3);

    return 0;
}

Now, one more functions for the TMeasurement class. (I will not provide in this example the set function for member "meas").

TMeasurement - Get function for __index

#define LUA_MEASUREMENT    "itse.measurement"

extern int lmeas_get(lua_State* L)
{
    TMeasurement* test = (TMeasurement*)lua_checkuserdata(L, 1, LUA_MEASUREMENT);
    std::string   key  = std::string(luaL_checkstring(L, 2));

    //-- this is only to check what is going on
    std::cout << "meas." << key << " ->" << std::endl;

    if(key.find("meas") == 0)
        lua_pushinteger(L, test->meas);
    else if(key.find("min") == 0)
    {
        lua_pushlightuserdata(L, &test->min);
        luaL_getmetatable(L, LUA_MEAS_LIMIT);
        lua_setmetatable(L, -2);
    }
    else if(key.find("max") == 0)
    {
        lua_pushlightuserdata(L, &test->max);
        luaL_getmetatable(L, LUA_MEAS_LIMIT);
        lua_setmetatable(L, -2);
    }
    else
        return 0;  //-- should notify of some error... when I make it work

    return 1;
}

Now, the part in the code that creates the mettatables for these two objects:

C++ - Publish the metatables

(never mind the nsLUA::safeFunction<...> bit, it is just a template function that will execute the function within the < > in a "safe mode"... it will pop-up a MessaegBox when an error is encountered)

static const luaL_Reg lmeas_limit_f[] = { { NULL, NULL} };
static const luaL_Reg lmeas_limit[] =
{
        { "__index",    nsLUA::safeFunction<llim_get> },
        { "__newindex", nsLUA::safeFunction<lllim_set> },
        { NULL,      NULL }
};
//-----------------------------------------------------------------------------

static const luaL_Reg lmeas_f[] =  { { NULL, NULL} };
static const luaL_Reg lmeas[] =
{
        { "__index", nsLUA::safeFunction<lmeas_get> },
        { NULL,   NULL }
};
//-----------------------------------------------------------------------------

int luaopen_meas(lua_State* L)
{
    //-- Create Measurement Limit Table
    luaL_newmetatable(L, LUA_MEAS_LIMIT);
    luaL_setfuncs(L, lmeas_limit, 0);
    luaL_newlib(L, lmeas_limit_f);

    //-- Create Measurement Table
    luaL_newmetatable(L, LUA_MEASUREMENT);
    luaL_setfuncs(L, lmeas, 0);
    luaL_newlib(L, lmeas_f);

    return 1;
}

Finally, my main function in C++, initializes LUA, creates and instance of object TMeasurement, passes it to LUA as a global and executes a lua script. Most of this functionality is enclosed in another class named LEngine:

C++ - Main function

int main(int argc, char* argv[])
{
    if(argc < 2)
        return show_help();

    nsLUA::LEngine eng;

    eng.runScript(std::string(argv[1]));

    return 0;
}
//-----------------------------------------------------------------------------

int LEngine::runScript(std::string scrName)
{
    //-- This initialices LUA engine, openlibs, etc if not already done. It also
    //   registers whatever library I tell it so by calling appropriate "luaL_requiref"
    luaInit();

    if(m_lua)    //-- m_lua is the lua_State*, member of LEngine, and initialized in luaInit()
    {
        LMeasurement measurement;

        measurement.value = 4.5;   //-- for testing purposes

        lua_pushlightuserdata(m_lua, &tst);
        luaL_getmetatable(m_lua, LUA_MEASUREMENT);
        lua_setmetatable(m_lua, -2);
        lua_setglobal(m_lua, "step");

        if(luaL_loadfile(m_lua, scrName.c_str()) || lua_pcall(m_lua, 0, 0, 0))
            processLuaError();   //-- Pops-up a messagebox with the error
    }

    return 0;
}

Now, at last the problem. Whe I execute whatever lua script, I can access step no problem, but I can only access a memebr within "min" or "max" the first time... any subsequent access gives an error.

LUA - example one

print(step.meas);        -- Ok
print(step.min.enabled); -- Ok
print(step.min.enabled); -- Error: attempt to index field 'min' (a nil value)

The output generated by this script is:

                              first script line: print(step.meas);
meas.meas ->                     this comes from lmeas_get function
4.5                              this is the actual print from lua sentence
                              second script line: print(step.min.enabled)
meas.min ->                      accessing step.min, call to function lmeas_get
limit.get: enabled ->            accessing min.enabled, call to function llim_get
false                            actual print from script sentence
                              third script line: print(step.min.enabled)
limit.get: min ->                accessing min from limit object, call to llim_get ???????

So. After the first time I access the field 'min' (or 'max' for that matter), any subsequent attempts to acess it will return "attempt to access index..." error. It doesn't matter whether I access first the __index (local e = step.min.enabled) function or the __newindex function (step.min.enabled = true).

It seems that I mess up the LUA stack the first time I access the min metatble of object step. It somehow "replaces" the "pointer to step" from a LUA_MEASUREMENT metatable to a LUA_MEAS_LIMIT... and I simply don't know why.

Please help... what is it that I am messing up so much?

Thank you and sorry for the long post... I just don't know how to make it shorter.

Answer 1

First of all, thanks to @siffiejoe and @greatwolf for their posts. It's them who explained to me what I was doing wrong.

Now, my solution. I am pretty sure this solution is not AT ALL the best around, but it covers my needs so far. If anyone has any suggestions, see/find potential bugs, or simply ant to comment, please do so.

Solution - The idea

Since in LUA, all lightuserdata share the same metatable, I've decided to make all structs and classes that I want to pass a lightuserdata pointer to LUA share the same inherit from a common class I have called LMetaPointer. This class will publish a metatable and map the __index and __newindex to given static methods LMetaPointer::__index and LMetaPointer::__newindex. The class also contains a static std::map (list) of pointers to all instances of LMetaPointer that are ever created. The constructor of the class makes sure that the newly created instance is added to this map.

Whenever in lua, the metamethod __index or __newindex is called, the corresponding LMetaPointer::__index or LMetaPointer::__newindex is executed. This methods search the map for the corresponding pointer that is responsible for the method call, and calls its own get or set methods, which are defined as pure virtual in LMetaPointer class.

I know this is may be a little bit confusing, so I will now post the definition of class LMetaPointer

Solution - The framework: LMetaPointer class

//-----------------------------------------------------------------------------
#define LUA_METAPOINTER     "itse.metapointer"    //-- Name given to the metatable for all lightuserdata (instances of LMetaPointer in C++)
//-----------------------------------------------------------------------------

class LMetaPointer
{
private:
    static lua_State*                           m_lua;           //-- All LMetaPointers will share a common lua State
    static const luaL_Reg                       m_lmembers[];    //-- Member functions (for later expansion)
    static const luaL_Reg                       m_lfunctions[];  //-- Metamethods
    static std::map<LMetaPointer*, std::string> m_pointers;      //-- List of all LMetaPointer instances

    std::string m_name;                  //-- Name of LUA global variable pointing to me.

    static int __index(lua_State* L);    //-- Shall be mapped to __index metamethod of the metatable for all lightuserdata pointers
    static int __newindex(lua_State* L); //-- Shall be mapped to __newindex metamethod of the metatable for all lightuserdata pointers

    void initialize(lua_State* L);       //-- Creates the metatable (only once) and publishes it

protected:
public:
    LMetaPointer(lua_State* L);
    virtual ~LMetaPointer();

    inline lua_State*  lua()    { return m_lua;             }
    inline std::string global() { return m_name;            }
    inline size_t      size()   { return m_pointers.size(); }

    void setGlobal(std::string n);      //-- Shall make this pointer globally accessible to LUA

    virtual int get(lua_State* L) = 0;  //-- To be implemented by inherited classes
    virtual int set(lua_State* L) = 0;  //-- To be implemented by inherited classes

    LMetaPointer* operator [](std::string n);
};

Now follows the implementation of the class

//-----------------------------------------------------------------------------
#define lua_checkmpointer(L)    (LMetaPointer*)luaL_checkudata(L, 1, LUA_METAPOINTER)
//-----------------------------------------------------------------------------
lua_State* LMetaPointer::m_lua = NULL;
std::map<LMetaPointer*, std::string> LMetaPointer::m_pointers;
const luaL_Reg LMetaPointer::m_lmembers[]   = { { NULL, NULL } };
const luaL_Reg LMetaPointer::m_lfunctions[] =
{
        { "__index",    LMetaPointer::__index    },
        { "__newindex", LMetaPointer::__newindex },
        { NULL, NULL }
};
//-----------------------------------------------------------------------------

LMetaPointer::LMetaPointer(lua_State* L) : m_name("")
{
    //-- Make sure we have created the metatable
    initialize(L);

    //-- Add this pointer as of kind LUA_METAPOINTER metatable. This bit of code
    //   might not be necessary here. (To be removed)
    lua_pushlightuserdata(m_lua, this);
    luaL_getmetatable(m_lua, LUA_METAPOINTER);
    lua_setmetatable(m_lua, -2);

    //-- Add myself to the map of all metapointers
    m_pointers[this] = m_name;
}
//-----------------------------------------------------------------------------

LMetaPointer::~LMetaPointer()
{
    //-- Remove myself from the map of metapointers
    std::map<LMetaPointer*, std::string>::iterator found = m_pointers.find(this);

    if(found != m_pointers.end())
        m_pointers.erase(found);
}
//-----------------------------------------------------------------------------

int LMetaPointer::__index(lua_State* L)
{
    //-- Obtain the object that called us and call its get method.
    //   Since get and set are pure virtual, all inherited classes of LMetaPointer
    //   must implement it, and, upon the call from here, the correct 'get' method
    //   will be called.
    LMetaPointer* p = lua_checkmpointer(L);
    return p->get(L);
}
//-----------------------------------------------------------------------------

int LMetaPointer::__newindex(lua_State* L)
{
    //-- Obtain the object that called us and call its set method
    //   Since get and set are pure virtual, all inherited classes of LMetaPointer
    //   must implement it, and, upon the call from here, the correct 'get' method
    //   will be called.
    LMetaPointer* p = lua_checkmpointer(L);
    return p->set(L);
}
//-----------------------------------------------------------------------------

void LMetaPointer::initialize(lua_State* L)
{
    //-- Only create the metatable the first time and instance of LMetaPointer is created
    if(!m_lua)
    {
        m_lua = L;

        luaL_newmetatable(m_lua, LUA_METAPOINTER);
        luaL_setfuncs(L, m_lfunctions, 0);
        luaL_newlib(L, m_lmembers);
    }
}
//-----------------------------------------------------------------------------

void LMetaPointer::setGlobal(std::string n)
{
    //-- Make myself (this) a global variable in LUA with name given by 'n'
    std::map<LMetaPointer*, std::string>::iterator found = m_pointers.find(this);

    if(found != m_pointers.end())
    {
        m_name = n;
        found->second = m_name;

        lua_pushlightuserdata(m_lua, this);
        luaL_getmetatable(m_lua, LUA_METAPOINTER);
        lua_setmetatable(m_lua, -2);
        lua_setglobal(m_lua, m_name.c_str());
    }
}
//-----------------------------------------------------------------------------

LMetaPointer* LMetaPointer::operator [](std::string n)
{
    //-- Simply for completeness, allow all metapointer access all other by their
    //   name. (Notice though that since names are only assigned to instances made
    //   global, this operator will only work properly when searching for a pointer
    //   made global. ALl othe rpointers have an empty name.
    std::map<LMetaPointer*, std::string>::iterator iter = m_pointers.begin();

    while(iter != m_pointers.end())
    {
        if(iter->second == n)
            return iter->first;
        ++iter;
    }

    return NULL;
}

Now, this class will allow me to define any other structure or class and pass LUA a pointer (lightuserdata) to it witout mixing methods or names. For the example in my original question this means defining the following:

NOTE: I've expanded a little bit my example and, the now called LMeasLimit is the previous TLimit, LMeasurement is a new class altogether and LTest is the previous TMeaasurement

Solution - Implementation

//-------------------------------------------------------------------------

struct LMeasLimit : public LMetaPointer
{
    bool   enabled;     //-- Is the limit enabled?
    double value;       //-- Limit value;

    LMeasLimit(lua_State* L) : LMetaPointer(L), enabled(false), value(0.0) {}
    ~LMeasLimit() {}

    int get(lua_State* L);   //-- Implements LMetaPointer::get
    int set(lua_State* L);   //-- Implements LMetaPointer::set
};
//-------------------------------------------------------------------------

struct LMeasurement : public LMetaPointer
{
    double      value;      //-- Measurement
    LStepResult result;     //-- Result of test
    std::string message;    //-- Message to display

    LMeasurement(lua_State* L) : LMetaPointer(L), value(0.0), result(srNothing), message("") {}
    ~LMeasurement() {}

    int get(lua_State* L);   //-- Implements LMetaPointer::get
    int set(lua_State* L);   //-- Implements LMetaPointer::set
};
//-------------------------------------------------------------------------

struct LTest : public LMetaPointer
{
    int          id;    //-- ID of test
    std::string  name;  //-- Name of test
    LMeasLimit   max;   //-- Max limit for measure
    LMeasLimit   min;   //-- Min limit for measure
    LMeasurement meas;  //-- Measurement

    LTest(lua_State* L) : LMetaPointer(L), id(0), name(""), min(L), max(L), meas(L) {}
    ~LTest() {}

    int get(lua_State* L);   //-- Implements LMetaPointer::get
    int set(lua_State* L);   //-- Implements LMetaPointer::set
};

//-----------------------------------------------------------------------------

And the definition of the different methods for the different classes

int LMeasLimit::get(lua_State* L)
{
    std::string key = std::string(luaL_checkstring(L, 2));

    if(key.find("enabled") == 0)
        lua_pushboolean(L, enabled);
    else if(key.find("value") == 0)
        lua_pushnumber(L, value);
    else
        return 0;

    return 1;
}
//-----------------------------------------------------------------------------

int LMeasLimit::set(lua_State* L)
{
    std::string key = std::string(luaL_checkstring(L, 2));

    if(key.find("enabled") == 0)
        enabled = lua_toboolean(L, 3);
    else if(key.find("value") == 0)
        value = lua_tonumber(L, 3);

    return 0;
}
//-----------------------------------------------------------------------------




int LMeasurement::get(lua_State* L)
{
    std::string key = std::string(luaL_checkstring(L, 2));

    if(key.find("value") == 0)
        lua_pushnumber(L, value);
    else if(key.find("result") == 0)
        lua_pushunsigned(L, result);
    else if(key.find("message") == 0)
        lua_pushstring(L, message.c_str());
    else
        return 0;

    return 1;
}
//-----------------------------------------------------------------------------

int LMeasurement::set(lua_State* L)
{
    std::string key = std::string(luaL_checkstring(L, 2));

    if(key.find("value") == 0)
        value = lua_tonumber(L, 3);
    else if(key.find("result") == 0)
        result = LStepResult(lua_tounsigned(L, 3));
    else if(key.find("message") == 0)
        message = std::string(lua_tostring(L, 3));

    return 0;
}
//-----------------------------------------------------------------------------



int LTest::get(lua_State* L)
{
    std::string key = std::string(luaL_checkstring(L, 2));

    if(key.find("id") == 0)
        lua_pushinteger(L, id);
    else if(key.find("name") == 0)
        lua_pushstring(L, name.c_str());
    else if(key.find("min") == 0)
    {
        lua_pushlightuserdata(L, &min);
        luaL_getmetatable(L, LUA_METAPOINTER);
        lua_setmetatable(L, -2);
    }
    else if(key.find("max") == 0)
    {
        lua_pushlightuserdata(L, &max);
        luaL_getmetatable(L, LUA_METAPOINTER);
        lua_setmetatable(L, -2);
    }
    else if(key.find("meas") == 0)
    {
        lua_pushlightuserdata(L, &meas);
        luaL_getmetatable(L, LUA_METAPOINTER);
        lua_setmetatable(L, -2);
    }
    else
        return 0;

    return 1;
}
//-----------------------------------------------------------------------------

int LTest::set(lua_State* L)
{
    std::string key = std::string(luaL_checkstring(L, 2));

    if(key.find("id") == 0)
        id = lua_tointeger(L, 3);
    else if(key.find("name") == 0)
        name = std::string(lua_tostring(L, 3));

    return 0;
}

Solution - Putting all together The final modification is in the LEngine::runScript from our original question.

int LEngine::runScript(std::string scrName)
{
    luaInit();

    if(m_lua)
    {
        LTest tst(m_lua);

        tst.name = std::string("mierda_esta");
        tst.setGlobal("step");

        if(luaL_loadfile(m_lua, scrName.c_str()) || lua_pcall(m_lua, 0, 0, 0))
            processLuaError();
    }

    return 0;
}

Finally I shall show one of the LUA scripts I used for testing and its output.

Testing - LUA script

print("step.id          = " .. step.id)
print("step.name        = " .. step.name)
print(step.min.enabled)
print("step.min.value   = " .. step.min.value)


step.id = 1
step.name = "nombre del test";
step.min.enabled = true;
step.min.value   = 5.6;

print("step.id          = " .. step.id)
print("step.name        = " .. step.name)
print(step.min.enabled)
print("step.min.value   = " .. step.min.value)

Testing - Output

step.id          = 0
step.name        = mierda_esta
false
step.min.value   = 0
step.id          = 1
step.name        = nombre del test
true
step.min.value   = 5.6

So, it all seems to work now as I wanted it. I still have to modify this LMetaPointer to be able to call now member functions of any inherited class in a similar fashion as we do in C++. But that shall be another story.

Thank you again to @siffiejoe and @greatwolf for their time and replies.

Answer 2

As already mentioned in the comments, all lightuserdata share a single metatable (see here), so all lightuserdata values are treated exactly the same at all times. If you change the metatable for one lightuserdata then it changes for all of them. And this is what happens in your code:

1. In LEngine::runScript you make all lightuserdata behave like TMeasurement objects. This is ok for the value in the global variable step.
2. When you access step.min for the first time, you make all lightuserdata behave like TLimit objects (in lmeas_get). This is ok for the value pushed by step.min, but now the value in step also behaves like a TLimit, so
3. when you try to access step.min for the second time, step acts as a TLimit object, so it doesn't have a field min and returns nil.

Lightuserdata is simply not the right tool for the job. See e.g. this discussion for cases where lightuserdata can be used. For everything else you need full userdata. This will allocate some extra memory compared to lightuserdata (sorry, can't be helped), but you can do some caching to avoid generating too many temporaries.

So for your step value you use a full userdata holding a pointer to your TMeasurement object. You also set a new table as uservalue (see lua_setuservalue) which will act as a cache for the sub-userdata. When your lmeas_get is called with a "min"/"max" argument, you look in the uservalue table using the same key. If you don't find a pre-existing userdata for this field, you create a new full userdata holding a pointer to the TLimit sub-object (using an appropriate metatable), put it in the cache, and return it. If your object lifetimes get more complicated in the future, you should add a back reference from the TLimit sub-userdata to the parent TMeasurement userdata to ensure that the later isn't garbage-collected until all references to the former are gone as well. You can use uservalue tables for that, too.