CODEWITHSUNDEEP
c#azure.net-coreazure-storageazure-blob-storage
RaZiQ PS
RaZiQ PS

Reputation: 157

How read all files from azure blob storage in C# Core

I want to read files from an azure blob storage (the files inside the folder), the blob storage contains many folders. I want to read my folder 'blobstorage' ,it contains many JSON files performing .read to each file and some manipulations. I tried many code that did not work:

 CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference($"blobstorage");

The above code uses 'Microsoft.WindowsAzure.Storage' nuget package. This code is not working as expected. In many questions and answers found in stack overflow I found that most of them are outdated and does not work. Note: if any nuget mention that also bcs they are many packages

Upvotes: 14

Views: 58353

Answers (5)

bhumika sharma
bhumika sharma

Reputation: 1

Try Below Code Out :

var connectionString = "your connection string";

        CloudStorageAccount storageacc = CloudStorageAccount.Parse(connectionString);
        //Create Reference to Azure Blob
        CloudBlobClient blobClient = storageacc.CreateCloudBlobClient();
        CloudBlobContainer container = blobClient.GetContainerReference("containerName");
        var blobs = container.GetDirectoryReference("FolderName").GetDirectoryReference("FolderName").ListBlobs().OfType<CloudBlockBlob>().ToList();
        Console.WriteLine("Total files found in directory: " + blobs.Count.ToString());
        var tempBlobList = blobs.Where(b => b.Name.Contains("fileName")).ToList();
        var response = await tempBlobList[0].DownloadTextAsync();

Upvotes: 0

Mana
Mana

Reputation: 1929

If you have mass data to download and are looking for efficiency, you probably don't want to download them 1 by 1 on a single thread. Use multiple threads and async.

Here's a good read on the topic:

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-scalable-app-download-files?tabs=dotnet

downloading 1000 files:

  • with single-thread : 30seconds download time
  • with multi-thread : 4seconds download time

Upvotes: 1

nmishr
nmishr

Reputation: 414

You can find example code in the SDK github repo here for c#: https://github.com/Azure/azure-sdk-for-net/tree/Azure.Storage.Blobs_12.8.0/sdk/storage/Azure.Storage.Blobs/

You can use the following command to add the package to your dotNet Core project.

dotnet add package Azure.Storage.Blobs

Based on the examples there, you can enumerate the blobs and then read the one you're looking for.

Upvotes: 0

manuelrb98
manuelrb98

Reputation: 374

I found the solution in this post and worked perfectly for me. You just have to read it as a normal stream after the download.

BlobServiceClient blobServiceClient = new BlobServiceClient("connectionString");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("containerName");
BlobClient blobClient = containerClient.GetBlobClient("blobName.csv");
if (await blobClient.ExistsAsync())
{
  var response = await blobClient.DownloadAsync();
  using (var streamReader= new StreamReader(response.Value.Content))
  {
    while (!streamReader.EndOfStream)
    {
      var line = await streamReader.ReadLineAsync();
      Console.WriteLine(line);
    }
  }
}

Upvotes: 23

Pankaj Rawat
Pankaj Rawat

Reputation: 4573

I don't see any option to list all blob using Microsoft.WindowsAzure.Storage package. If you can use Azure.Storage.Blobs package then try below code.

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;

namespace ConsoleApp2
{
    class Program
    {
        static string connectionString = "DefaultEndpointsProtocol=https;AccountName=storage******c9709;AccountKey=v**************************************;EndpointSuffix=core.windows.net";
        static string container = "azure-webjobs-hosts";
        static void Main(string[] args)
        {
            // Get a reference to a container named "sample-container" and then create it
            BlobContainerClient blobContainerClient = new BlobContainerClient(connectionString, container);
            blobContainerClient.CreateIfNotExists();
            Console.WriteLine("Listing blobs...");
            // List all blobs in the container
            var blobs = blobContainerClient.GetBlobs();
            foreach (BlobItem blobItem in blobs)
            {
                Console.WriteLine("\t" + blobItem.Name);
            }            
            Console.Read();
        }
    }
}

Output

enter image description here

You can also download the content of blob, Check this link

Upvotes: 6

Related Questions