CODEWITHSUNDEEP
azure-data-lake
HaveSpacesuit
HaveSpacesuit

Reputation: 3994

Is There a Local Emulator for the Azure Data Lake Store

When developing for Azure storage accounts, I can run the Microsoft Storage Emulator to locally keep Blobs, Queues, and Tables without having to connect to Azure online.

Is there something equivalent for the Azure Data Lake Store? It would be nice to develop locally for a while without having to connect to Azure online.

Upvotes: 8

Views: 3150

Answers (3)

BerndK
BerndK

Reputation: 1080

Same problem here.

AFAIK the Storage Emulator is not yet able to really handle Data Lake (ADSL Gen2) Requests.
This Uri works (but looks for a file, not a dir):

http://127.0.0.1:10000/devstoreaccount1/packages-container/Dir/SubDir?sv=2020-04-08&se=2022-10-13T14%3A43%3A39Z&sr=b&sp=rcwl&sig=d2SxwYCkJGyx%2BHac9vntYQZOTt5QVs1bKgKb4%2FgcQ9k%3D

This one doesn't:
Error: Status: 403 (Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.) ErrorCode: AuthorizationFailure

http://127.0.0.1:10000/devstoreaccount1/packages-container/Dir/SubDir?sv=2020-04-08&se=2022-10-13T14%3A43%3A39Z&sr=d&sp=rcwl&sdd=2&sig=KU%2Fcu6W0Nsv8CucMgusubo8RbXWabFO8nDMkFxU1tTw%3D

The difference is that the second one uses the resource 'sr=d' (directory) while the first uses 'sr=b' (blob). Both items are working on real Azure Storage (with ADSL Gen2).

The request is already tracked here: https://github.com/Azure/Azurite/issues/553
Tested on VS 2022 17.3.6 using Server: Azurite-Blob/3.18.0

Upvotes: 0

Sachin Sheth
Sachin Sheth

Reputation: 309

As pointed out by David, you can develop Azure Data Lake Analytics (ADLA) projects locally without needing connectivity to Azure for the ADLA account or the associated Azure Data Lake Store (ADLS) account. Is there some other application you would like to use with ADLS?

Thanks, Sachin Sheth Azure Data Lake team

Upvotes: 1

David Paul Giroux
David Paul Giroux

Reputation: 657

Have you tried Visual Studio with the Azure Data Lake Tools plug-in?

Upvotes: 1

Related Questions