AK GamePlay
AK GamePlay

Reputation: 1

Flink table api leftouter join is fetching records with null for matching records on right side

Table result = s.leftouterjoin(t,$("key").isequal("t_key")).where( $("t_key").isnull()). select(key,value, date);

Table s = tableenv.from("source"); Table t = tableenv.from("target").as("t_key","t_value","t_date");

Both table are different and has same schema with primary set for key field and not null constraint. I changed field names for target as both have same fields. This table are connected through mongo connector. While joining both table getting null value for the matching records and the matched records is repeated from s, also sometimes it is repeating the records from with records from t on the right side for matched records. My goal is to fetch records from source which are not present in target with the same key. How to solve this. Table api, flink sql, flink

It is just basic query to fetch new records for the unmatched key but it is fetching matched records also with some anamoly. Flink is configured in streaming mode. I printed the change logs of the resul query. It is printing insert and delete logs for matched records and insert only logs for the unmatched records. Thus deleting the records which are already present in the target table when sinked.

I had even tried not exists query also. It is also not giving the desired result. I tried sql query like Tableenv . ExecuteSqL (" select source.* From source left join target on source.key = target.key where target.key = null ") I had tried different options. Still not getting the desired result.

Upvotes: 0

Views: 8

Answers (0)

Related Questions