Parsing config to IOptions<T> fails in Azure

Question

I'm using the IOptions pattern to parse my config file into a strongly typed class.

I'm doing this in ConfigureServices:

services.AddOptions();
services.Configure<StorageSettings>(Configuration.GetSection("StorageSettings"));

The StorageSettings class looks like this:

public class StorageSettings
{
    public ConnectionStrings ConnectionStrings { get; set; }
}

public class ConnectionStrings
{
    public TableStorageConnectionStrings TableStorage { get; set; }

    public EventHubConnectionStrings EventHubs { get; set; }

    public ServiceBusConnectionStrings ServiceBus { get; set; }
}

public class TableStorageConnectionStrings
{
    public string LogEntriesByAccount { get; set; }
}

public class EventHubConnectionStrings
{
    public string EventHubName { get; set; }

    public string IngestionQueue { get; set; }

    public string AccessKeyName { get; set; }

    public string AccessKeyToken { get; set; }

    public string ServiceNameSpace { get; set; }
}

public class ServiceBusConnectionStrings
{
    public string GlobalQueue { get; set; }
}

And my JSON configuration file looks like this:

{
    "SecuritySettings": {
        "SystemToken": "123"
    },
    "StorageSettings": {
        "ConnectionStrings": {
            "TableStorage": {
                "LogEntriesByAccount": "DefaultEndpointsProtocol=https;AccountName=account;AccountKey=key=="
            },
            "EventHubs": {
                "EventHubName": "myhub",
                "AccessKeyName": "key",
                "AccessKeyToken": "token",
                "ServiceNameSpace": "ns",
                "IngestionQueue": ""
            },
            "ServiceBus": {
                "GlobalQueue": "DISABLED"
            }
        }
    },
    "Logging": {
        "IncludeScopes": false,
        "LogLevel": {
            "Default": "Debug",
            "System": "Warning",
            "Microsoft": "Warning"
        }
    }
}

It works when running locally, and the config sections and properties are parsed just fine. But when running i Azure I get this error:

[13:54:40 INF] Starting up API in Production
Application startup exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: Failed to convert '@{LogEntriesByAccount=DefaultEndpointsProtocol=https;AccountName=mystore001;AccountKey=KEY====000}' to type 'MyApp.Storage.Configuration.TableStorageConnectionStrings'. ---> System.NotSupportedException: TypeConverter cannot convert from System.String.
   at System.ComponentModel.TypeConverter.GetConvertFromException(Object value)
   at System.ComponentModel.TypeConverter.ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, Object value)
   at System.ComponentModel.TypeConverter.ConvertFromInvariantString(String text)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.ConvertValue(Type type, String value)
   --- End of inner exception stack trace ---
   at Microsoft.Extensions.Configuration.ConfigurationBinder.ConvertValue(Type type, String value)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.BindInstance(Type type, Object instance, IConfiguration config)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.BindProperty(PropertyInfo property, Object instance, IConfiguration config)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.BindNonScalar(IConfiguration configuration, Object instance)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.BindInstance(Type type, Object instance, IConfiguration config)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.BindProperty(PropertyInfo property, Object instance, IConfiguration config)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.BindNonScalar(IConfiguration configuration, Object instance)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.BindInstance(Type type, Object instance, IConfiguration config)
   at Microsoft.Extensions.Configuration.ConfigurationBinder.Bind(IConfiguration configuration, Object instance)
   at Microsoft.Extensions.Options.ConfigureFromConfigurationOptions`1.<>c__DisplayClass0_0.<.ctor>b__0(TOptions options)
   at Microsoft.Extensions.Options.ConfigureOptions`1.Configure(TOptions options)
   at Microsoft.Extensions.Options.OptionsCache`1.CreateOptions()
   at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
   at System.Threading.LazyInitializer.EnsureInitialized[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
   at Microsoft.Extensions.Options.OptionsCache`1.get_Value()
   at Microsoft.Extensions.Options.OptionsManager`1.get_Value()
   at MyApp.Storage.Azure.EventHubAMQPIngestionQueue..ctor(IOptions`1 options, ILoggerFactory loggerFactory) in C:\Workspace\Dev\MyApp\src\MyApp.Storage\Azure\EventHubAMQPIngestionQueue.cs:line 29
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.SingletonCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass12_0.<RealizeService>b__0(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at MyApp.API.Startup.ConfigureServices(IServiceCollection services) in C:\Workspace\Dev\MyApp\src\MyApp.API\Startup.cs:line 124
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at Microsoft.AspNetCore.Hosting.Internal.ConfigureServicesBuilder.Invoke(Object instance, IServiceCollection exportServices)
   at Microsoft.AspNetCore.Hosting.Internal.ConfigureServicesBuilder.<>c__DisplayClass4_0.<Build>b__0(IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.ConfigureServices(IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
Hosting environment: Production
Content root path: D:\home\site\wwwroot
Now listening on: http://localhost:23758
Application started. Press Ctrl+C to shut down.

Answer 1

I decided to create a test project to see if I could re-produce the issue in a small and more isolated project. I could not.

It turns out that my Azure App Service (Web App) instance that I'd used for a long time were the culprit - in desperation I deployed the project that failed in Azure to a new App Service instance and it started working.

I tried to delete a lot of folders using Kudu but it didn't work.

I also compared all Application Settings and verified they were the same.

I ended up deleting the App Service instance I was using in favor of a newly created one.

The test project I used is this one: https://github.com/martinnormark/ConfigOptionsTest