ImageProcessor / Windows Azure Storage issue, returns 403 Forbidden

Question

These images can be loaded in local ENV, and also in production ENV. However, for no reason at all, the staging environment cannot load these.

<package id="ImageProcessor" version="2.2.0.0" targetFramework="net45" />
  <package id="ImageProcessor.Web" version="4.2.1.0" targetFramework="net45" />
  <package id="ImageProcessor.Web.Config" version="2.2.0.0" targetFramework="net45" />
  <package id="ImageProcessor.Web.Plugins.AzureBlobCache" version="1.0.0.0" targetFramework="net45" />
  <package id="ImageProcessor.Web.PostProcessor" version="1.0.2.0" targetFramework="net45" />
  <package id="UmbracoAzureBlobStorageProvider" version="1.0.10.5" targetFramework="net45" />
  <package id="WindowsAzure.Storage" version="4.3.0" targetFramework="net45" />

I'm using ImageProcessor and domains are whitelisted as required:

<whitelist>
        <add url="http://conceptjp.blob.core.windows.net/"/>
        <add url="https://az739977.vo.msecnd.net/"/>
</whitelist>

https://staging.conceptjp.com/remote.axd?https://az739977.vo.msecnd.net/media/6883/logo-sparitual.png?quality=70 (do not works)

https://conceptjp.com/remote.axd?https://az739977.vo.msecnd.net/media/6883/logo-sparitual.png?quality=70 (works)

https://cjp.local/remote.axd?https://az739977.vo.msecnd.net/media/6883/logo-sparitual.png?quality=70 (works, local environment)

2016-10-04 13:31:11.2393 Logging.TheLogger The remote server returned an error: (403) Forbidden. The remote server returned an error: (403) Forbidden.    at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndExecuteAsync[T](IAsyncResult result)
   at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions.<>c__DisplayClass1`1.<CreateCallback>b__0(IAsyncResult ar)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at ImageProcessor.Web.Plugins.AzureBlobCache.AzureBlobCache.<IsNewOrUpdatedAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at ImageProcessor.Web.HttpModules.ImageProcessingModule.<ProcessImageAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.TaskAsyncHelper.EndTask(IAsyncResult ar)
   at System.Web.HttpApplication.AsyncEventExecutionStep.OnAsyncEventCompletion(IAsyncResult ar)

Answer 1

I'm not sure what to say here. Almost everything is being used in a manner I wouldn't expect nor recommend.

If you are using Umbraco on Azure you should be using the following plugin for your media.

https://github.com/JimBobSquarePants/UmbracoFileSystemProviders.Azure

The FileSystemProvider you are using has been obsolete for about a year and a half. It actually recommends using the plugin mention above in it's homepage.

NEW package UmbracoFileSystemProviders.Azure is online! see: https://our.umbraco.org/projects/collaboration/umbracofilesystemprovidersazure/

I recommend it instead of this, especially if you are on Umbraco 7.3 or higher. It solves a lot of issues you would otherwise have in the back office.

https://our.umbraco.org/projects/backoffice-extensions/azure-blob-storage-provider

The reason for this is that the original provider had flaws which could not be fixed without a complete rewrite.

1. Media was stored in the database using the absolute url. This meant that the same media could not be used across multiple environments.
2. Media was downloaded and displayed unprocessed in the backoffice. This led to 100s MB of data being downloaded unnecessarily which destroyed performance in larger sites.

You will have to replace your media references before getting started in your database to strip out the domain from the stored url. I personally would recommend rebuilding your media section from scratch.

There are comprehensive instructions on the GitHub page but I will list them here also below.

First. Uninstall the old plugin, update all ImageProcessor libraries to their latest versions and install the recommended FileSystemProvider plugin

Then update ~/Config/FileSystemProviders.config replacing the default provider with the following:

<?xml version="1.0"?>
<FileSystemProviders>
  <Provider alias="media" type="Our.Umbraco.FileSystemProviders.Azure.AzureBlobFileSystem, Our.Umbraco.FileSystemProviders.Azure">
    <Parameters>
      <add key="containerName" value="media" />
      <add key="rootUrl" value="http://[myAccountName].blob.core.windows.net/" />
      <add key="connectionString" value="DefaultEndpointsProtocol=https;AccountName=[myAccountName];AccountKey=[myAccountKey]"/>
      <!--
        Optional configuration value determining the maximum number of days to cache items in the browser.
        Defaults to 365 days.
      -->
      <add key="maxDays" value="365" />
    </Parameters>
  </Provider>
</FileSystemProviders>

You now need to configure the CloudImageService to capture all requests beginning with /media/

<?xml version="1.0"?>
<security>
  <services>
    <service name="LocalFileImageService" type="ImageProcessor.Web.Services.LocalFileImageService, ImageProcessor.Web"/>
    <service prefix="media/" name="CloudImageService" type="ImageProcessor.Web.Services.CloudImageService, ImageProcessor.Web">
      <settings>
        <setting key="Container" value="media"/>
        <setting key="MaxBytes" value="8194304"/>
        <setting key="Timeout" value="30000"/>
        <setting key="Host" value="http://[myAccountName].blob.core.windows.net/media"/>
      </settings>
    </service>
  </services>  

Ensure your Cache config is set up properly.

<caching currentCache="AzureBlobCache">
  <caches>
    <!-- Disk cache configuration removed for brevity -->
    <cache name="AzureBlobCache" type="ImageProcessor.Web.Plugins.AzureBlobCache.AzureBlobCache, ImageProcessor.Web.Plugins.AzureBlobCache" maxDays="365">
      <settings>
        <!-- The Account, Container and CDN details -->
        <setting key="CachedStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=[CacheAccountName];AccountKey=[CacheAccountKey]"/>
        <setting key="CachedBlobContainer" value="cache"/>
        <!-- Whether to add the container name to the CDN url. Newer Azure formats require false. -->
        <setting key="UseCachedContainerInUrl" value="true"/>
        <!-- Full CDN root url e.g http://123456.vo.msecnd.net/ -->
        <setting key="CachedCDNRoot" value="[CdnRootUrl]"/>
        <!-- Optional setting for a timeout limit in milliseconds when attempting to communicate with the CDN url. -->
        <setting key="CachedCDNTimeout" value="1000"/>
        <!-- 
            Optional settings for better identifcation of source images if stored in 
            Azure blob storage.
         -->
        <setting key="SourceStorageAccount" value=""/>
        <setting key="SourceBlobContainer" value=""/>
        <!-- 
            Optional settings facilitate streaming of the blob resource directly instead of a redirect. This is beneficial
            for CDN purposes but caution should be taken if not used with a CDN as it will add quite a bit of overhead 
            to the site. 
         -->
        <setting key="StreamCachedImage" value="false"/>
      </settings>
    </cache>
  </caches>
</caching>

Image requests should now be created using the root relative /media/ path only which the built in Virtual Path Provider will intercept and process accordingly.

e.g /media/1046/car-small.jpg?width=500&height=500&mode=boxpad&bgcolor=hotpink