CODEWITHSUNDEEP
azurepowershellblobazure-data-factory
OrganicMustard
OrganicMustard

Reputation: 1426

Create blob storage container via Azure data factory

I am working on a project which uses Azure data factory and Blob Storage. I have a requirement but not sure how to implement it.

Requirement:

#1 Check if blob storage container exists. #2 Create of new container in Data Factory via a power shell script. Basicaly, the container will only be created the first time that the pipeline is run. #3 Call API.

#1==> i found way to check if blob exists, but did not find container.

#2==> What is the best way to create a container from Azure Data Factory via a power shell script : via a batch process ? Or is it better to do it via a function?

Thank you very much for your help in advance

Upvotes: 0

Views: 1046

Answers (1)

Joseph Xu
Joseph Xu

Reputation: 6083

As the post said, we can use GetMetadata activity and If Condition activity to check if blob exists. If blob does not exist we can use Azure function to create the container and specify the name of the container through http get request. I think using Azure function is more convenient.

For example, here I'm using nodejs in Azure function. According to Azure function javascript and Manage blobs with JavaScript v12 SDK in Node.js. We can start a project.

Add dependencies to the nodejs demo. enter image description here

Then modify some code index.js so we can manage the blob container(Create a container).

const { BlobServiceClient } = require('@azure/storage-blob');


module.exports = async function (context, req) {

    const AZURE_STORAGE_CONNECTION_STRING = "<your_connect_string>";
    const blobServiceClient = BlobServiceClient.fromConnectionString(AZURE_STORAGE_CONNECTION_STRING);

    const name = (req.query.name || (req.body && req.body.name));

    const containerClient = blobServiceClient.getContainerClient(name);

    // Create the container
    const createContainerResponse = await containerClient.create();

    const responseMessage = name
        ? "Container:" + name + " has been created!"
        : "...";

    context.res = {
        // status: 200, /* Defaults to 200 */
        body: responseMessage
    };
}

After deployed to Azure function, I tested the code, it works well. enter image description here The container was created. enter image description here

Upvotes: 3

Related Questions