CODEWITHSUNDEEP
marklogicmarklogic-dhf
coen1821
coen1821

Reputation: 21

Load Social Media Data like Facebook, Instagram , Twitter into Data HUB marklogic

Looking for an easy way to load Social Media Data from Facebook, Instagram and Twitter into Marklogic's data-hub.

I could imagine that storing Social Media Data in to HDFS and then into MarkLogic would be a way to go for Huge amount of data. Using MLCP. I did check in Apache NIFI if there is anyway to source Social Media Data and create a flow into MarkLogic. But nothing which connection direct into Facebook or Insta only twitter was available.

Main Goal: Find all Food, Sport , Racing information from the socials order by topics , Ratings etc an other search criteria .. Delivering Source , Documents, Images and text information

Upvotes: 2

Views: 170

Answers (1)

Dave Cassel
Dave Cassel

Reputation: 8422

The challenge here is getting it from the social media platforms more than actually getting it into MarkLogic. I suggest you continue googling for how to use NiFi to get that data -- it's very likely been done. Once you've got that, just send to a PutMarkLogic processor and you're in good shape.

For example, I found this pretty quickly: https://github.com/iheb-boughzala/AllFbPostsUsingNifi. I haven't looked into this one in detail, but it looks like you'd just need to replace the PutHDFS processor with PutMarkLogic.

Upvotes: 2

Related Questions