Totally unexplainable NullReferenceException

Question

Ok, so I very sporadically get a NullReferenceException on this line of code:

if (!_oracleTenantSettings.OraclePlanSettings.ContainsKey(_key) || _oracleTenantSettings.OraclePlanSettings[_key] == null)

and/or this line:

_oraclePlanSettings = _oracleTenantSettings.OraclePlanSettings[_key];

where OraclePlanSettings is a SortedList, and it can't be null, because the code in question is surrounded by:

 if (_oracleTenantSettings.OraclePlanSettings != null && _oracleTenantSettings.OraclePlanSettings.Count > 0)

So I'm getting a NRE, but there is not a single part of the entire line of code that could ever possibly be null, ever. Period. (sense the frustration?) And that includes the key, but that wouldn't throw a NRE anyway. I do not understand. Is it possible that VS is just misplacing the CLR exception? If so, where would be a good place to start looking?

The stack trace is just a one-liner:

 at company.product.Mvc.OracleSettingsStoreCache.VerifyValueInCacheOrInsert[T](T& returnVal, SettingsType settingType, String tenantId, String planId, String pageMnemonic, String processId, String transcationType, String language, String country, String wapTransactionType, String wapCodeGroup, String wapLoanReasons, String palleteType, Boolean isInsert, Object _cacheValue) in blahblahblah.OracleSettingsStoreCache.cs:line 290

Here is the entire block of code:

if (!string.IsNullOrEmpty(tenantId) && (!IsWacMode() || (IsWacMode() && settingType == OracleSettingsType.SettingsType.FetchWAPInvestmentTransfer)) && _useCache != "false")
                {
                    tenantId = tenantId.ToUpper().Trim();

                    _oracleTenantSettings = null;

                    if (_oracleCacheManager.Contains(_cacheKey))
                        _oracleTenantSettings = _oracleCacheManager.Get<OracleTenantSetting>(_cacheKey);

                    if (_oracleTenantSettings != null)
                    {
                        if (_oracleTenantSettings.OraclePlanSettings != null && _oracleTenantSettings.OraclePlanSettings.Count > 0)
                        {
                            _key = language + "_" + country + "_" + tenantId;
           ***LINE 290***   if (!_oracleTenantSettings.OraclePlanSettings.ContainsKey(_key) || _oracleTenantSettings.OraclePlanSettings[_key] == null)
                            {
                                _objectMissing = TypeOfObjectMissing.TenantObjectDoesNotExist;
                            }
                        }

Answer 1

Update

I am proposing that somewhere in the code the Equality operator or the == has been overloaded on one of the related objects and the failure occurs when either null is not be checked for properly (or fails) or something is returned as equal when it is not.

Check for all operator overloads on == for any class object being used in this situation and rectify..

Original

Change the logic to this, for you want to first check for no key...THEN..do the check when there is a valid key but (and) its value is null:

 if ((_oracleTenantSettings.OraclePlanSettings.ContainsKey(_key) == false) || 
     ((_oracleTenantSettings.OraclePlanSettings.ContainsKey(_key)) && 
       _oracleTenantSettings.OraclePlanSettings[_key] == null)))

Its actually expected if you think through the logic flow of the original statements why it intermittently fails. :-)

EDIT: Let me explain, Follow this logic by steps

1. In the original if clause when it evaluates the (!ContainsKey(_key)) means that when the key is NOT there (true) it gets changed to FALSE.
2. Then the Or kicks in because of False in #1. It evaluates the OraclePlanSettings[_key] BUT the key is not there right?
3. So it executes code to check on null for an invalid key and throws an exception.

Only by breaking out the logic as I have shown, will the excpetion not be thrown.

Answer 2

I agree with the previous answers, It's probably a threading issue, but I have someting to add.

Here is a quick and dirty test to determine if it's threading or not.

Set up a scenerio that reproduces the error (say running it in several (10) threads in a constant loop overnight)

Apply this attribute to your class

[Synchronization]

Your Class must inherit from ContextBoundObject.

That forces all the instances of the class to run on a single thread (way slower).

Re run your test.

If your problem goes away, you have a threading issue. If speed is a concern, you need to go back and do all the locking around the code that touches that object. If you convert everything to use properties for the objects in question you can just lock the getter and setter.

If the quick and dirty test fails to fix the issue, it's probabaly something else. For instance if you're using unsafe code or unsafe dlls (ie stuff written in non .Net c++), it might be a memory corruption problem.

Hope this helps.

Here is more detail on the attribute, including inheriting from ContextBoundObject.

Ms Docs

Code sample:

// Context-bound type with the Synchronization context attribute.
[Synchronization()]
public class SampleSynchronized : ContextBoundObject {

    // A method that does some work, and returns the square of the given number.
    public int Square(int i)  {

        Console.Write("The hash of the thread executing ");
        Console.WriteLine("SampleSynchronized.Square is: {0}", 
                             Thread.CurrentThread.GetHashCode());
        return i*i;
    }
}

Answer 3

Without seeing the context that code lives within, it's hard to be sure. But based on the symptoms you describe, i.e. very sporadic...inexplicable...something's null that can't be... I would strongly suspect a threading issue. For example, if the collection is static and potentially accessed by multiple threads, it can happen (although it's a rare occurrence of chance timing) that a second thread modifies the collection contents between when the first thread tests if something is there and when it accesses that something.

If that's the case, you must make your code more threadsafe. You can use lock or concurrent collections to avoid this problem. To use lock, you need to use a synchronization object (not a new object created on the fly). You would also want to hunt up ALL places where that collection is accessed, and surround every one with a lock...code that just looks at the collection must use lock as well as code that modifies the collection. This is a big topic for a SO answer, so I would recommend you use this really great resource:

http://www.albahari.com/threading/

Here is how you could get NREs in this case:

thread 1 checks if entry exists in SortedList myList for _key="hello" 
gets true
thread 1 checks if entry for _key="hello" is non-null
gets true
thread 2 sets myList["hello"] = null
thread 1 executes myList["hello"].Something() and gets NRE.

Based on the edits to your post, it seems that in these lines

if (_oracleTenantSettings != null) {
    if (_oracleTenantSettings.OraclePlanSettings != null && _oracleTenantSettings.OraclePlanSettings.Count > 0) {
        _key = language + "_" + country + "_" + tenantId;
         if (!_oracleTenantSettings.OraclePlanSettings.ContainsKey(_key) || _oracleTenantSettings.OraclePlanSettings[_key] == null)

if the NRE occurs on the last line, then right after executing the first line or second line, another thread can either set _oracleTenantSettings or _oracleTenantSettings.OraclePlanSettings to null. Either of those things happening would cause the final line to throw an NRE.

The following code is not the proper way to make your code thread safe, but might serve as a quick way to see if this is indeed the case since it would make this situation (the null reference exception) less likely:

var oracleTS = _oracleTenantSettings;
if (oracleTS != null) {
    var planSettings = oracleTS.OraclePlanSettings;
    if ((planSettings != null) && (planSettings.Count > 0)) {
        _key = language + "_" + country + "_" + tenantId;
        if (!planSettings.ContainsKey(_key) || planSettings[_key] == null)

Note that the final line could still have other issues related to threading like the key being removed by another thread between the first part of the conditional and the second part, or planSettings Count changing after being tested. But if this code drastically reduces the NREs, then you have a pretty good clue what was happening, and that you should go through and properly make your code thread safe with locks where needed. To say further, a person would need to know more about what other code is doing, especially code that modifies _oracleTenantSettings.

Answer 4

My guess is that there is another thread accessing the property.

A quick way to fix it would be locking on it every time you access it like this:

var oraclePlanSettings = _oracleTenantSettings.OraclePlanSettings;
lock (oraclePlanSettings)
{
    // from now on you can safely access your cached reference "oraclePlanSettings"
    if (oraclePlanSettings != null && oraclePlanSettings.Count > 0)
        _oraclePlanSettings = oraclePlanSettings[_key]; // ... blabla
}

Beware of deadlocks tho.