CODEWITHSUNDEEP
kdb+
Jason_L
Jason_L

Reputation: 589

kdb - persisting functions on kdb server & Context management

I see a lot of info regarding serializing tables on kdb but is there a suggested best practice on getting functions to persist on a kdb server? At present, I and loading a number of .q files in my startup q.q on my local and have duplicated those .q files on the server for when it reboots.

As I edit, add and change functions, I am doing so on my local dev machine in a number of .q files all referencing the same context. I then push them one-by-one sending them to the server using code similar to below which works great for now but I am pushing the functions to the server and then manually copying each .q file and then manually editing the q.q file on the server.

\p YYYY;
h:hopen `:XXX.XXX.XX.XX:YYYY;
funcs: raze read0[`$./funcs/funcsAAA.q"]; 
funcs: raze read0[`$./funcs/funcsBBB.q"]; 
funcs: raze read0[`$./funcs/funcsCCC.q"]; 
h funcs;

I'd like to serialize them on the server (and conversely get them when the system reboots. I've dabbled with on my local and seems to work when I put these in my startup q.q

`.AAA set get `:/q/AAAfuncs
`.BBB set get `:/q/BBBfuncs
`.CCC set get `:/q/CCCfuncs

My questions are:

  1. Is there a more elegant solution to serialize and call the functions on the server?
  2. Clever way to edit the q.q on the server to add the .AAA set get :/q/AAAfuncs
  3. Am I thinking about this correctly? I recognize this could be dangerous in a prod enviroment

ReferencesKDB Workspace Organization

Upvotes: 0

Views: 416

Answers (1)

terrylynch
terrylynch

Reputation: 13572

In my opinion (and experience) all q functions should be in scripts that the (production) kdb instance can load directly using either \l /path/to/script.q or system"l /path/to/script.q", either from local disk or from some shared mount. All scripts/functions should ideally be loaded on startup of that instance. Functions should never have to be defined on the fly, or defined over IPC, or written serialised and loaded back in, in a production instance.

Who runs this kdb instance you're interacting with? Who is the admin? You should reach out to the admins of the instance to have them set up a mechanism for having your scripts loaded into the instance on startup.

An alternative, if you really can't have your function defined server side, is to define your functions in your local instance on startup and then you send the function calls over IPC, e.g.

system"l /path/to/myscript.q"; /make this load every time on startup

/to have your function executed on the server without it being defined on the server
h:hopen `:XXX.XXX.XX.XX:YYYY;
res:h(myfunc1;`abc);

This loads the functions in your local instance but sends the function to the remote server for evaluation, along with the input parameter `abc

Edit: Some common methods for "loading every time on startup" include:

  1. Loading a script from the startup command line, aka
q myscript.q -p 1234 -w 10000

You could have a master script which loads subscripts.

  1. Load a database or script directory contains scripts from the startup command line, aka
q /path/to/db -p 1234 -w 10000

Jeff Borror mentions this here: https://code.kx.com/q4m3/14_Introduction_to_Kdb%2B/#14623-scripts and here: https://code.kx.com/q4m3/14_Introduction_to_Kdb%2B/#14636-scripts

  1. Like you say, you can have a q.q script in your QHOME

Upvotes: 3

Related Questions