CODEWITHSUNDEEP
apache-kafkaksqldb
Mikhail
Mikhail

Reputation: 2935

cannot get data from table KSQL

I create a rekeyed stream

CREATE STREAM details_stream_rekeyed2 as \
  select CONCAT(IdSeq,IdTime,'') as root,ServerId,Server,\
         IdTime ,IdSeq \
  from voip_details_stream  \
  partition by root;

select from this stream and i get > 100 items

then i try to create a table

create table voip_details_table3 \
  (ROOT varchar,ServerId long , Server varchar ,IdTime varchar,IdSeq long ) \
  with ( kafka_topic = 'DETAILS_STREAM_REKEYED2', \
         value_format = 'json',\
         key='ROOT');

when i run 

SELECT ROWKEY,ROOT  FROM VOIP_DETAILS_TABLE3;

i get the only less than 10 items likes ;

12018-04-04T18:56:35.080-04:00 | 12018-04-04T18:56:35.080-04:00

i run the command:

kafkacat -C -K: -b "$BROKER_LIST" -f 'Key:    %k\nKey Bytes: %K\nValue:  %s\nValue Bytes: %S\n\n' -t DETAILS_STREAM_REKEYED2

everething is ok. and i get data likes

 Key:    12018-02-05T15:16:07.113-05:00
 Key Bytes: 30
 Value:  {"SERVER":null,"IDSEQ":1,"ROOT":"12018-02-05T15:16:07.113-05:00","SERVERID":null,"SESSIONIDTIME":"2018-02-05T15:16:07.113-05:00"}

Value Bytes: 158

where the problem ?

Upvotes: 1

Views: 678

Answers (1)

Robin Moffatt
Robin Moffatt

Reputation: 32130

A KSQL table differs from a KSQL Stream, in that it gives you the latest value for a given key. So if you are expecting to see the same number of messages in your table as your source stream, you should have the same number of unique keys.

If you're seeing fewer messages then it suggests that ROOT is not unique.

Depending on the problem that you're modelling, you should either :

  • (a) be using a Stream not a Table, or
  • (b) change the key that you are using

Ref:

Upvotes: 3

Related Questions