How to extend SWIG's userdata in Lua code?

Question

I am using SWIG to bind C++ code to Lua. So far it looks good, but now I need to "cheat" and extend a single userdata from within Lua, adding custom fields and methods, etc.

I can't find a way to accomplish it while working within SWIG's directives. I know where in the wrapper code the magic takes place, but I don't fully understand how the __index and __newindex work. Also, SWIG uses __setitem and __getitem which is commented as "/* NEW: looks for the __setitem() fn this is a user provided set fn */" - don't know what that means either. Lastly, my environment automatically calls the script to bind SWIG directives to C++ wrappers before each build, so modifying the sources afterwards is very tedious if I ever choose to recompile or add more Lua bindings.

So far my only conclusion is to go with toLua++, which has this feature documented. Please help me avoid a lot of transition work!

EDIT: I also found that "lua_setuservalue" API call in 5.2 - it seems useful but I don't understand where I would call it among all SWIG binding code.

EDIT: To clear things up:

I create objects by having C function

SoundObj *loadSound(const char *name)
{
    return g_audio->loadSound(name); // Returns SoundObj pointer
}

This function is bound by SWIG by including it's prototype in the *.i swig definition.

In Lua I write code like this:

sound = audio.loadSound("beep.mp3")
sound.scale = 0.2
sound:play()

EDIT: Advancement! I followed Schollii's instructions and wrote the following Lua code. Here, "ground" is the userdata that was acquired earlier using bound C++ code. The code stores the swig's version of __index and __newindex, and then I re-create these functions that query another ("_other") table first.

My current problem here is that the new values stored in "_other" table are shared between all userdata objects of that type. In other words, if ground.nameFoo = "ha!", all other objects have nameFoo field storing "ha!". How do I fix this?

mt = getmetatable(ground)
mt._other = {}
mt.__oldindex = mt.__index
mt.__oldnewindex = mt.__newindex

mt.__index = function(tbl, key)
    if mt.__oldindex(tbl, key) == nil then
        if mt._other[key] ~= nil then
            return mt._other[key]
        end
    else
        return mt.__oldindex(tbl, key)
    end
end

mt.__newindex = function(tbl, key, val)
    if mt.__oldnewindex(tbl, key, val) == nil then
        mt._other[key] = val
    end
end

EDIT: I implemented the solution from this answer: Add members dynamically to a class using Lua + SWIG

The problem is that now my object type is no longer userdata, it is a table. Which means I can no longer organically pass it as an argument to other bound C++ functions that take userdata as argument. Any solution to this? AND I have to do this to every function that returns a userdata object.

Answer 1

So you have a SoundObject class in C++, you export it to Lua (doesn't matter whether it's via SWIG or tolua++ or manually). Your application runs a lua script that creates an instance of SoundObject and adds a property (in Lua) and a method (again in Lua), then you want to be able to use that property and call that method from C++. Something like:

-- Lua script: 
sound = audio.loadSound("beep.mp3")
sound.scale = 0.2   -- this property exported by SWIG
sound.loop = true   -- this is Lua only
sound.effect = function (self, a) print(a) end  -- this is Lua only
sound:play()        -- this method exported by SWIG

// C++ pseudocode:
bind SoundObject* load_sound(string) to "audio.loadSound"
run Lua script 

And you want the play() method to investigate additional properties and methods (like loop and effect) and do something. What, I can't imagine, hopefully this gives you idea how to more clearly express your problem.

Answer 2

I have crafted a solution, following Schollii's example... but fewer sacrifices. This does not turn your userdata into table, so you still have userdata type (I can pass it to other C++ functions that take userdata). Schollii's solution effectively turned userdata into a table wrapper.

So the first line uses the variable "ground" - this is a userdata instance wrapped by SWIG. Doing this at the beginning of the execution alters the metatable of "ground" userdata type for all instances of that type, but each instance retains a personal table indexed by userdata's memory location. This table lives in _G._udTableReg table.

-- Store the metatable and move the index and newindex elsewhere
mt = getmetatable(ground)
mt.__oldindex = mt.__index
mt.__oldnewindex = mt.__newindex

-- Store the global registry of tables associated with userdata, make a pointer to it in the metatable
_G._udTableReg = {}
mt._udTableReg = _G._udTableReg

-- Rewrite the new index function that looks in the udTableReg using 'self' as index before proceeding to use the old index as backup
mt.__index = function(self, key)
    local ret;
    local privateTable = mt._udTableReg[self]

    -- If the private table exists and has key, return key
    if privateTable ~= nil and privateTable[key] ~= nil then
        ret = privateTable[key]
    -- Use the old index to retrieve the original metatable value
    else ret = mt.__oldindex(self, key) end

    if ret == nil then return 0
    else return ret end
end

-- Try to assign value using the original newindex, and if that fails - store the value in
mt.__newindex = function(self, key, val)
    -- If old newindex assignment didn't work
    if mt.__oldnewindex(self, key, val) == nil then
        -- Check to see if the custom table for this userdata exists, and if not - create it
        if mt._udTableReg[self] == nil then
            mt._udTableReg[self] = {}
        end
        -- Perform the assignment
        mt._udTableReg[self][key] = val
    end
end

I still have not figured out the best place to put this Lua code, or how to get a more elegant way of grabbing the userdata's metatable, without actually using an existing variable.

Answer 3

I'm going to be sloppy here: In Lua a table "inherits" from another table by using it as metatable. So if you export C++ Foo to Lua and want a Lua "class" derived from Foo, you can create a table and set its metatable to Foo. Then when you access the new table with a field that does not exist in table, it will look in metatable to see if it is there. Example pseudocode:

baseTable = {a=123}
assert( getmetatable(baseTable) == nil )
derived={b=456}
assert(derived.a == nil)
setmetatable(derived, baseTable )
assert(derived.a == 123)

The baseTable is the C++ class exported to Lua via SWIG, you can't change its metatable but you can change that of tables you create in Lua. So you shouldn't have to do any mods to SWIG code or use SWIG directives, you can just operate in Lua. Example:

-- define Foo class:
Foo = {}
Foo.__index = Foo -- make Foo a Lua "class"
setmettable(Foo, YourCppBaseClass) -- derived from your C++ class
-- instantiate it:
foo = {}
setmetatable(foo, Foo) -- foo is of "class" Foo