How to pass `keys_and_args` to redis-py' `eval` function, and on to lua script

Question

I wish to pass a number of keys and values from python to a lua script, via redis's eval function which is documented as:

eval(script, numkeys, *keys_and_args)

Execute the Lua script, specifying the numkeys the script will touch and the key names and argument values in keys_and_args. Returns the result of the script.

In practice, use the object returned by register_script. This function exists purely for Redis API completion.

I am following this answer as a starting point. That script increment the scores of all values in the sorted set specified by 1. As I wish to specify the values to update (key names) and the increment count for each (argument values) my script looks like this:

-- some logging
local loglist = "lualog"
redis.pcall("DEL", loglist)

local function logit(msg)
  redis.pcall("RPUSH", loglist, msg)
end

logit("started")

-- count & log the keys provided
local countofkeys = table.getn(KEYS)
logit(countofkeys)

-- loop through each key and increment
for n = 1, countofkeys do
  redis.call("zincrby", "test_set", ARGV[n], KEYS[n])
end

I can run this from the command line with:

$ redis-cli --eval script.lua apple orange , 1 1

Then in Python confirm that the values have incremented:

>>> r.zrange('test_set', start = 0, end = -1, withscores=True)
[(b'apple', 1.0), (b'orange', 1.0)]

However I don't know how to run this using eval:

>>> c.eval(script,1,{'orange':1,'apple':1})
redis.exceptions.DataError: Invalid input of type: 'dict'. Convert to a byte, string or number first.

>>> c.eval(script,2,'apple orange , 1 1')
redis.exceptions.ResponseError: Number of keys can't be greater than number of args

>>> c.eval(script,1,'apple orange , 1 1')
redis.exceptions.ResponseError: Error running script (call to f_aaecafd58b474f08bafa5d4fefe9db98a58b4084): @user_script:21:
 @user_script: 21: Lua redis() command arguments must be strings or integers 

The documentation isn't too clear on what keys_and_args should look like. Also at the comand line numkeys isn't actually required by the looks of things. Does anyone know what this should look like?

Bonus question: How to avoid hard coding "test_set" into the lua script.

Answer 1

eval() receives 3 parameters or to make it easy to understand "4" parameters

1. script: string

2. number_of_keys: integer

3. key_list: unpacked iterable key objects, for example, *{1, 2, 3} => 1, 2, 3 We have 3 keys, so number_of_keys should be 3

4. argument_list: unpacked iterable argument objects, for example, *{'one', 'two', 'three'} => 'one', 'two', 'three'

If we want to access 2nd element of the key list, use KEYS[2] in the LUA script.

If we want to access the 1st element of the argument list, user ARGV[1].

To return a list of KEYS[2] and ARGV[1]:

cache.eval('return {KEYS[2], ARGV[1]}', 3, 1, 2, 3, 'one', 'two', 'three')

So, now if we are back with 3 parameters, the last one should be *keys_and_arguments: unpacked iterable keys and arguments.

Answer 2

The first argument in the documentation is numkeys and the rest of the arguments are termed as *keys_and_args. The way to provide arguments is similar to argc and argv. So you would do something like this:

redis.eval(lua_script, 1, "BUCKET_SIZE", total_bucket_size, refill_size)

The numkeys will specify that the first argument should be considered as a key and the following as args. The 1 specifies the number of keys present in your *keys_and_args array.

Answer 3

*keys_and_args should be an iterable (e.g. a list) - the use of an asterisk as a prefix to the argument's name is the Pythonic way of saying that.

Bonus tip: look into redis-py Script helper.

Bonus answer: Any key names touched by the script need to be provided via the KEYS table. Your script is doing it all wrong - read the documentation about EVAL.

Also at the comand line numkeys isn't actually required by the looks of things

This is only with the cli when used in that fashion - the comma (',') delimits between key names and arguments.