CODEWITHSUNDEEP
apache-sparkdatastaxdatastax-enterprisedatastax-startup
peter
peter

Reputation: 684

Spark Error message: Please consider a new data model based on the query pattern instead of using ALLOW FILTERING

My DSE Opscenter sends me this message:

Please consider a new data model based on the query pattern instead of using ALLOW FILTERING.

And after changing my spark code I already removed the below column value from my query. But the below error message still keeps popping up. I don't know why? Also the error message only occurs in my OPScenter on in the actual table. Thanks for your help.

Query:

select * from dse_perf.node_slow_log

Column value/ error mesage

SELECT "XXX", "XXX", "XXX", "likes", "XXX" FROM "XXX"."axes" WHERE token("article") > ? AND token("article") <= ?   ALLOW FILTERING

Please consider a new data model based on the query pattern instead of using ALLOW FILTERING.

Upvotes: 1

Views: 67

Answers (1)

Nom de plume
Nom de plume

Reputation: 461

Opscenter is warning you that your request can be pretty expensive and suggesting your review the use case.

"Allow Filtering" can be pretty expensive as described here:

http://www.datastax.com/dev/blog/allow-filtering-explained-2

It maybe your use falls into the OK category - in which case you can ignore the warning. If not - it may be worth looking at other ways of modelling your data that allow you to sort it in a more efficient manner.

Upvotes: 2

Related Questions