CODEWITHSUNDEEP
kdb+
chrise
chrise

Reputation: 4253

kdb type error when inserting I as F

I have a certain file type that contains a column with floats which I read in using insert

`table insert ("TISISIFIIIFFIbIFFFFFFIIIFIIFFFFIIIIIIIIIIIIFFFFFFIIFFIIIIFFIIIIIIIIIIIIIIIII"; enlist "\t" ) 0:`:my_file.txt

unfortunately, sometimes the values in column happen to be all integers and in the txt file the are saved as ints, not floats, so as 1 instead of 1.0 and it seems kdb is throwing a type error. Is there a way make kdb accept ints saved in that format as floats?

I do have a lot of columns with floats and theoretically, the problem can appear in any of them. Is there some way to tell kdb on insert to treat any int as float if the column type is float?

Upvotes: 1

Views: 940

Answers (1)

Sean O'Hagan
Sean O'Hagan

Reputation: 1697

The 'type error is actually happening from your insert. You are trying to insert some parsed data, but the types of each column are not conforming to the types of each column in 'table'. You are basically saying that your raw data can contain floats, therefore you are going to have to read them in as floats. What you do with that column after the parse is up to you though.

1) keep as floats, read in as floats, insert as floats, column should be a float in 'table' pre-read (I presume this is what you want going by your question):

update "f"$COLUMN from `table
`table insert (1#"F";1#"\t") 0:`myfile.txt

2) update to an integer and then insert into 'table' - you are going to have update the schema of table first, read in as floats, and then run an update after every read:

update "i"$COLUMN from `table
`table insert update "i"COLUMN from (1#"F";1#"\t") 0:`myfile.txt

Another option you may want to consider, but please test first as it may replace too much, is to replace the trailing ".0" from your floats, and then just read in as integers:

q)\cd /var/tmp
q)`:myfile.txt 0:("x\tx1";"1.0\t2.0";"3.0\t1.0")
q)\sed -i -e 's/.0//g' myfile.txt
q)("II";1#"\t")0:`myfile.txt

Upvotes: 2

Related Questions