CODEWITHSUNDEEP
azuretime-seriesazure-data-lakeazure-data-explorer
Christian Pfisterer
Christian Pfisterer

Reputation: 23

Connecting Azure-Data-Explorer with Azure-Data-Lake-Gen2

I try to use Azure-Data-Explorer to create time-series of IoT data that is stored in Azure-Data-Lake-Gen2. The folder structure in Azure-Data-Lake is "archive/yyyy/MM/dd/HH/*.json".

The idea is:

  1. Create an external table and json mapping (https://learn.microsoft.com/en-us/azure/data-explorer/data-lake-query-data)
  2. Make time-series (https://learn.microsoft.com/en-us/azure/data-explorer/time-series-analysis)
  3. Use analytics to get cool insights ;-)

I've tried for step one (in the Azure-Data-Explorer query):

.create external table ExternalAdlGen2 (timestamp:datetime, timestamp2:datetime, deviceId:string, telemetry:dynamic)
kind=adl
partition format_datetime="'year='yyyy/'month='MM/'day='dd/'hour='HH" by bin(timestamp, 1h)
dataformat=json
( 
   @h'abfss://filesystem@<storage acount name>.dfs.core.windows.net/archive/path;sharedkey=<access key>'
)

and also:

.create external table ExternalAdlGen2 (timestamp:datetime, timestamp2:datetime, deviceId:string, telemetry:dynamic)
kind=adl
partition format_datetime="'year='yyyy/'month='MM/'day='dd/'hour='HH" by bin(timestamp, 1h)
dataformat=json
( 
   @h'abfss://filesystem@<storage acount name>.dfs.core.windows.net/archive/path;sharedkey=<access key>'
)
with 
(
   docstring = "Docs",
   folder = "ExternalTables",
   namePrefix="Prefix"
)

The result is the following error message: "Error I could not parse that, sorry. clientRequestId: KustoWebV2; some_guid"

Do you have any suggestion how to do that properly?

Thanks in advance.

Upvotes: 2

Views: 693

Answers (1)

Yoni L.
Yoni L.

Reputation: 25895

there was a bug in the documentation which will be fixed shortly.

in the URL specification - please replace @h'abfss://... with h@'abfss://...

Upvotes: 1

Related Questions