CODEWITHSUNDEEP
kdb+
user983716
user983716

Reputation: 2082

Convert a q table into a pandas dataframe using embedpy

I look for a way to save a q/kdb table into a parquet file. The most straightforward way that I figured out is to convert a q table into a pandas dataframe using embedPy. Does someone has achieve this?

cheers,

didier

Upvotes: 1

Views: 1694

Answers (1)

Fionnuala Carr
Fionnuala Carr

Reputation: 102

To convert a q table to a pandas dataframe,you can use this function:

tab2df:{
r:.p.import[`pandas;`:DataFrame;x][@;cols x];
$[count k:keys x;r[`:set_index]k;r]}

To convert the pandas dataframe to a q table, you can use this function:

df2tab:{n:$[.p.isinstance[x`:index;.p.import[`pandas]`:RangeIndex]`;0;x[`:index.nlevels]`];n!flip $[n;x[`:reset_index][];x][`:to_dict;`list]`}

The df2tab requires the pandas dataframe to an embedPy object. You can use .p.wrap .
See an example below

q)\l p.q  
q)tab:([]a:10?10.;b:10?10;c:10?`aaa`bbb`ccc)
q)tab
   a         b c  
  ---------------
  1.086824  2 ccc
  9.598964  7 aaa
  0.3668341 8 aaa
  6.430982  5 ccc
  6.708738  6 aaa
  6.789082  4 bbb
  4.12317   1 aaa
  9.877844  3 aaa
  3.867353  3 aaa
  7.26781   7 ccc
q)tab2df[tab]
  {[f;x]embedPy[f;x]}[foreign]enlist
q)print tab2df[tab]
          a  b    c
0  1.086824  2  ccc
1  9.598964  7  aaa
2  0.366834  8  aaa
3  6.430982  5  ccc
4  6.708738  6  aaa
5  6.789082  4  bbb
6  4.123170  1  aaa
7  9.877844  3  aaa
8  3.867353  3  aaa
9  7.267810  7  ccc
q)pdtab:tab2df[tab]
q)df2tab[pdtab] 
  a         b c    
 -----------------
 1.086824  2 "ccc"
 9.598964  7 "aaa"
 0.3668341 8 "aaa"
 6.430982  5 "ccc"
 6.708738  6 "aaa"
 6.789082  4 "bbb"
 4.12317   1 "aaa"
 9.877844  3 "aaa"
 3.867353  3 "aaa"
 7.26781   7 "ccc"

Hope this helps!!

Upvotes: 5

Related Questions