CODEWITHSUNDEEP
pythonkdb+
Will
Will

Reputation: 942

how to pass a dictionary holding a list of minutes through qpython to kdb

I'm trying to query a specific function through qpython. the function expects a dictionary with several arguments, the first one being a date (type -14), the second one being a list of minutes (type 17).

I wrote this little example which hopefully illustrates faithfully the issue:

\d .test

testfun:{[args]
    
    one:args[`one];
    two:args[`two];
    
    ' string type args[`two];
    :42;
    };

now when I query through qpython:

from qpython import qconnection
from qpython.qcollection import QDictionary, qlist
import numpy as np
from qpython.qtype import QLONG_LIST, QSYMBOL_LIST, QMINUTE_LIST


query='{[arg_dict] :.test.testfun[arg_dict] }'

kdbconfig = {
        'q_host': 'q_host',
        'q_port': 42,
        'q_usr': 'q_usr',
        'q_pwd': 'q_pwd' 
        }

params = {
    "one" : np.datetime64('2021-01-06', 'D'),
    "two" : qlist([ np.timedelta64(10*60, 'm'), np.timedelta64(10*60+30, 'm')] , qtype = QMINUTE_LIST)
    }

qparams = QDictionary(list(params.keys()), list(params.values()))


with qconnection.QConnection(host=kdbconfig['q_host'],
                                     port=kdbconfig['q_port'],
                                     username=kdbconfig['q_usr'],
                                     password=kdbconfig['q_pwd'] ) as q:
        data = q.sendSync(query, qparams, pandas = True)
        if data is None:
            raise ValueError("didnt return any data")

but I get QException: b'-14' while I would expect type 17
qparams.values is worth: [numpy.datetime64('2021-01-06'), QList([420, 450], dtype='timedelta64[m]')] so it looks reasonable to me.

Anyone would have an idea how to make this work please?

Upvotes: 1

Views: 544

Answers (1)

Matt Moore
Matt Moore

Reputation: 2800

The keys are being sent as strings not symbols:

q).debug
"one"| 2021.01.06
"two"| 10:00 10:30

q).debug[`two]
0Nd
q)type .debug[`two]
-14h

`two is returning a null according to the type of the first item in the dictionary. I'm not actually sure how qpython handles strings vs symbols but you could update the q function to this:

.test.testfun:{[args]
  one:args["one"];
  two:args["two"];
  string type args["two"]
  };

Edit: How to debug

If the query is sending to the q process ok, it is best to debug on the kdb side if it is not acting as expected. I defined the test q function as:

.test.testfun:{[args].debug:args;};

This allowed me to view what kdb was receiving. Nice one on the np.string_, this defines it as symbols:

qparams = QDictionary(np.string_(list(params.keys())), list(params.values()))

q).debug
one| 2021.01.06
two| 10:00 10:30

You could also define the q function as follows so it returns to python:

// .Q.s1 returns the string representation of an object
q).test.testfun:{[args].debug:args;.Q.s1 .debug};

python test.py
b'`one`two!(2021.01.06;10:00 10:30)'

My Script:

from qpython import qconnection
from qpython.qcollection import QDictionary, qlist
import numpy as np
from qpython.qtype import QLONG_LIST, QSYMBOL_LIST, QMINUTE_LIST


query='{[arg_dict] :.test.testfun[arg_dict] }'

params = {
    "one" : np.datetime64('2021-01-06', 'D'),
    "two" : qlist([ np.timedelta64(10*60, 'm'), np.timedelta64(10*60+30, 'm')] , qtype = QMINUTE_LIST)
    }

qparams = QDictionary(np.string_(list(params.keys())), list(params.values()))

with qconnection.QConnection(host="localhost",
                                     port=12345,
                                     username="",
                                     password="" ) as q:
        data = q.sendSync(query, qparams)
        if data is None:
            raise ValueError("didnt return any data")
print(data)

Upvotes: 3

Related Questions