CODEWITHSUNDEEP
apache-sparkapache-spark-sqlignite
srujana
srujana

Reputation: 182

Trying to retrieve data from sharedRDD of apache ignite context

I am trying to integrate apache ignite with spark and I am new to apache ignite. I want to save data in distributed cache and retrieve it.

I have created a dataframe by loading a file in spark and trying to save in cache with sharedRDD.savePairs(key,value) of Apache Ignite. Key is of type string and value is of type spark dataframe. Now I want to retrieve the stored data and print it. I am not even sure if it is actually saved with type dataframe.

Upvotes: 3

Views: 821

Answers (1)

dmagda
dmagda

Reputation: 1785

To retrieve data from RDD you can leverage at least one of the following ways:

1) sharedRDD.filter(...).collect() approach. As an example the code below gets all the values that contain the word "river" from the cache named "testCache" 

val cache = igniteContext.fromCache("testCache")
val result = cache.filter(_._2.contains("river")).collect()

Reading values using 'filter' method

2) sharedRDD.sql(...) method. 

val cacheRdd = igniteContext.fromCache("personsCache")
val result = cacheRdd.sql(
  "select name from Person where id > ? and id < ?", 10, 100)

Getting values using SQL

Upvotes: 5

Related Questions