Please suggest a way to store a temp file in Windows Azure

Question

Here I have a simple feature on ASP.NET MVC3 which host on Azure.

• 1st step: user upload a picture
• 2nd step: user crop the uploaded picture
• 3rd: system save the cropped picture, delete the temp file which is the uploaded original picture

Here is the problem I am facing now: where to store the temp file?

I tried on windows system somewhere, or on LocalResources: the problem is these resources are per Instance, so here is no guarantee the code on an instance shows the picture to crop will be the same code on the same instance that saved the temp file.

Do you have any idea on this temp file issue?

1. normally the file exist just for a while before delete it
2. the temp file needs to be Instance independent
3. Better the file can have some expire setting (for example, 1H) to delete itself, in case code crashed somewhere.

Answer 1

Under sample will be help you. https://code.msdn.microsoft.com/How-to-store-temp-files-in-d33bbb10

you have two way of temp file in Azure. 1, you can use Path.GetTempPath and Path.GetTempFilename() functions for the temp file name 2, you can use Azure blob to simulate it.

    private long TotalLimitSizeOfTempFiles = 100 * 1024 * 1024;
    private async Task SaveTempFile(string fileName, long contentLenght, Stream inputStream)
    {
        try
        {
            //firstly, we need check the container if exists or not. And if not, we need to create one.
            await container.CreateIfNotExistsAsync();

            //init a blobReference
            CloudBlockBlob tempFileBlob = container.GetBlockBlobReference(fileName);

            //if the blobReference is exists, delete the old blob
            tempFileBlob.DeleteIfExists();

            //check the count of blob if over limit or not, if yes, clear them.
            await CleanStorageIfReachLimit(contentLenght);

            //and upload the new file in this
            tempFileBlob.UploadFromStream(inputStream);
        }
        catch (Exception ex)
        {
            if (ex.InnerException != null)
            {
                throw ex.InnerException;
            }
            else
            {
                throw ex;
            }
        }
    }

    //check the count of blob if over limit or not, if yes, clear them. 
    private async Task CleanStorageIfReachLimit(long newFileLength)
    {
        List<CloudBlob> blobs = container.ListBlobs()
            .OfType<CloudBlob>()
            .OrderBy(m => m.Properties.LastModified)
            .ToList();

        //get total size of all blobs.
        long totalSize = blobs.Sum(m => m.Properties.Length);

        //calculate out the real limit size of before upload
        long realLimetSize = TotalLimitSizeOfTempFiles - newFileLength;

        //delete all,when the free size is enough, break this loop,and stop delete blob anymore
        foreach (CloudBlob item in blobs)
        {
            if (totalSize <= realLimetSize)
            {
                break;
            }

            await item.DeleteIfExistsAsync();
            totalSize -= item.Properties.Length;
        }
    }

Answer 2

OK. So what you're after is basically somthing that is shared storage but expires. Amazon have just announced a rather nice setting called object expiration (https://forums.aws.amazon.com/ann.jspa?annID=1303). Nothing like this for Windows Azure storage yet unfortunately, but, doesnt mean we can't come up with some other approach; indeed even come up with a better (more cost effective) approach.

You say that it needs to be instance independant which means using a local temp drive is out of the picture. As others have said my initial leaning would be towards Blob storage but you will have cleanup effort there. If you are working with large images (>1MB) or low throughput (<100rps) then I think Blob storage is the only option. If you are working with smaller images AND high throughput then the transaction costs for blob storage will start to really add up (I have a white paper coming out soon which shows some modelling of this but some quick thoughts are below).

For a scenario with small images and high throughput a better option might be to use the Windows Azure Cache as your temporary storaage area. At first glance it will be eye wateringly expensive; on a per GB basis (110GB/month for Cache, 12c/GB for Storage). But, with storage your transactions are paid for whereas with Cache they are 'free'. (Quotas are here: http://msdn.microsoft.com/en-us/library/hh697522.aspx#C_BKMK_FAQ8) This can really add up; e.g. using 100kb temp files held for 20 minutes with a system throughput of 1500rps using Cache is about $1000 per month vs $15000 per month for storage transactions.

The Azure Cache approach is well worth considering, but, to be sure it is the 'best' approach I'd really want to know;

1. Size of images
2. Throughput per hour
3. A bit more detail on the actual client interaction with the server during the crop process? Is it an interactive process where the user will pull the iamge into their browser and crop visually? Or is it just a simple crop?

Answer 3

Here is what I see as a possible approach:

1. user upload the picture
2. your code saves it to a blob and have some data backend to know the relation between user session and uploaded image (mark it as temp image)
3. display the image in the cropping user interface interface

4. when user is done cropping on the client:

   4.1. retrieve the original from the blob

   4.2. crop it according the data sent from the user

   4.3. delete the original from the blob and the record in the data backend used in step 2

   4.4. save the final to another blob (final blob).

And have one background process checking for "expired" temp images in the data backend (used in step 2) to delete the images and the records in the data backend.

Please note that even in WebRole, you still have the RoleEntryPoint descendant, and you still can override the Run method. Impleneting the infinite loop in the Run() (that method shall never exit!) method, you can check if there is anything for deleting every N seconds (depending on your Thread.Sleep() in the Run().

Answer 4

You can use the Azure blob storage. Have look at this tutorial.