CODEWITHSUNDEEP
pythondjangomod-wsgimongoengine
Alanyst
Alanyst

Reputation: 1410

Weirdness with mongoengine ReferenceField

This is a puzzling problem that's difficult even to name, let alone describe. I'll start with the essential facts and then give what background information might be relevant.

Consider two mongoengine document models:

class Bar(Document):
    # ...
    # field definitions
    # ...
    def bar_func(self):
        pass  # ...or some arbitrary code


class Foo(Document):
    bar = ReferenceField(Bar)

The following is inconsistently producing an AttributeError on our production server:

# Assume foo_id references a valid Foo document in Mongo
# and that its 'bar' reference is to a valid Bar document.
foo = Foo.objects.with_id(foo_id)
foo.bar.bar_func()  # <-- AttributeError on 'bar_func'

If I place debugging code right before the location of the error, evaluating type(foo.bar) as a string produces <class 'bson.dbref.DBRef'>. Obviously a DBRef doesn't have a bar_func attribute, but why is a DBRef being returned instead of an instance of Bar?

Further debugging code shows that the following condition is failing in the ReferenceField.__get__ function in mongoengine/fields.py:

if isinstance(value, (pymongo.dbref.DBRef)):
        value = _get_db().dereference(value)

But (pymongo.dbref.DBRef) is actually bson.dbref.DBRef, which seems to be the same as type(foo.bar)! Why does isinstance fail?

This is where things get really weird:

id(type(foo.bar)) == id(bson.dbref.DBRef)  # <-- Evaluates to False!

In other words, type(foo.bar) is a different bson.dbref.DBRef than the one obtained by referencing bson.dbref.DBRef directly. In fact, inspecting the __dict__ of these two types shows different memory locations for their functions and properties.

Note: I'll call the type returned by type(foo.bar) fooDBRef for convenience below, to distinguish it from the type referenced by bson.dbref.DBRef.

To debug further, I modified the DBRef code to add a metaclass that inspects the system modules at the time the DBRef type is created, and stores a list of the IDs of those modules in an extra class attribute of DBRef. The results show that the set of modules in existence when fooDBRef is created is entirely distinct from the set of modules in existence when the bare bson.dbref.DBRef type is created. All module IDs for one are different from all module IDs for the other.

Some possibly relevant factors:

Can anyone help me figure out what is causing fooDBRef to be different from bson.dbref.DBRef?

Upvotes: 3

Views: 1969

Answers (1)

Graham Dumpleton
Graham Dumpleton

Reputation: 58523

Are you using embedded mode or daemon mode of mod_wsgi? If using daemon mode of mod_wsgi are you delegating each site to a different daemon process group and then in turn forcing the application to run in main Python interpreter?

It may be the case that mongodb Python client module may not work properly in Python sub interpreters, especially when the module is used at the same time in a different sub interpreter of the same process.

So, you may have to run each site in separate daemon process group using WSGIDaemonProcess/WSGIProcessGroup and then force the user of the Python main interpreter using WSGIApplicationGroup with argument of '%{GLOBAL}'.

Note that when forcing use of main interpreter for both sites, you can no longer use embedded mode, or have them both run in same daemon process group. Thus why you need to force each to run in separate daemon process group.

Upvotes: 4

Related Questions