CODEWITHSUNDEEP
csvautomationetlazure-data-explorerkql
an1que
an1que

Reputation: 423

Automation: Push data from local CSVs to Kusto db tables

I have different data downloaded from a web page and saved as .csv files on my local directory. I would like to push that data to the tables that I've created on Kusto Explorer and ideally I want this process to be scheduled to take place everyday automatically because that data from the website is updated daily.

For example:

Here I have 3 .csv files:

1.csv
2.csv
3.csv

and here I have 3 tables created on Kusto to store each csv data respectively:

Table_1
Table_2
Table_3

The question is: Is it possible to automate and schedule the data push/load from local CSVs to Kusto? The data push/load type is Replace for now but might need to be Append in the future.

I was recommended Flow (https://preview.flow.microsoft.com/en-us/) to check if any similar templates were set up already. However, I was not able to find any and am new to these tools so I need help from someone who knows about this. Thanks so much in advance. Please let me know if you need any further information.

Upvotes: 4

Views: 2997

Answers (1)

Yoni L.
Yoni L.

Reputation: 25995

if you want to control the schedule:

an alternative approach would be:

  1. have the CSV files you generate uploaded to cloud storage once they're created.

  2. setup a data connection for the storage container they're uploaded to, so that any file that gets uploaded to it is automatically ingested into your cluster: https://learn.microsoft.com/en-us/azure/data-explorer/ingest-data-event-grid

Upvotes: 5

Related Questions