CODEWITHSUNDEEP
azure-cosmosdbapache-nifi
Valeria
Valeria

Reputation: 21

Post to Azure Cosmos Db from NiFi

I created Azure CosmosDb database and container for my documents. I use NiFi as a main data ingestion tool and want to feed my container with documents from NiFi flow files. Can anybody please share a way to post flowfile content to Azure Cosmos Db from NiFi?

Thanks in advance

UPDATE(2019.05.26): In the end I used Python script and called it from NiFi to post messages. I passed a message as a parameter. The reason I chose python is because it has some examples on official Microsoft site with all the required connection settings and libraries, so it was easy to connect to Cosmos. I tried Mongo component, but couldn't connect to Azure (security config didn't work), didn't really go too far with it as Python script worked just fine.

Upvotes: 1

Views: 1064

Answers (2)

Sivaprasanna Sethuraman
Sivaprasanna Sethuraman

Reputation: 4132

Azure CosmosDB exposes MongoDB API so you can use the following MongoDB processors which are available in NiFi to read/query/write to & from Azure CosmosDB using Apache NiFi.

  • DeleteMongo
  • GetMongo
  • PutMongo
  • PutMongoRecord
  • RunMonogAggregation

Useful Links

Upvotes: 3

Jay Gong
Jay Gong

Reputation: 23782

Valeria. According to the components list supported by Apache Nifi related to Azure, you could only get Azure Blob Storage, Queue Storage, Event Hub etc,not including Cosmos DB.

So,I suggest you using PutAzureBlobStorage to feed azure blob container with documents from NiFi flow files. Then please create a copy activity pipeline in Azure Data Factory to transfer data from Azure Blob Storage into Azure Cosmos DB.

Upvotes: 0

Related Questions