CODEWITHSUNDEEP
scalanetwork-programmingapache-sparkspark-streaming
user3823859
user3823859

Reputation: 469

Spark receiver for capturing network traffic

I wanted to write a spark streaming application . I have written a scala code that captured packets using java library . Now, I wanted to get these packets to a spark streaming receiver program (0.9). Should I write a new spark receiver code or is it possible to create receiver in the scala code already written to capture the packets? How can it be done? Could anyone please help me.

Upvotes: 1

Views: 634

Answers (1)

Tathagata Das
Tathagata Das

Reputation: 1808

Have you seen the instructions for creating a custom receiver in spark.

http://spark.apache.org/docs/latest/streaming-custom-receivers.html

Its pretty easy in general. However since I am not sure how your existing packet capturing code work (does it capture in real time and dump into file, can that be changed to store in an ArrayBuffer, etc.), its hard to say how much of the code you can reuse.

Upvotes: 1

Related Questions