Motor: RuntimeError: maximum recursion depth exceeded while encoding an object to BSON

Question

I have an API, built on asynchronous Tornado and mongoDB. It works fine, except one handler:

@gen.coroutine
def get(self, *args, **kwargs):
    """
    Gets tracking lib
    """
    data = self._get_request_data()
    self._serialize_request_data(AuthValidator, data)

    tags = yield self.motor.tags.find_one({"client_id": data["client_id"]})
    raise Return(self.write(tags))

When request comes, tornado returns HTTP 500 with following stack trace:

response: Traceback (most recent call last):
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/tornado/web.py", line 1334, in _execute
result = yield result
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/tornado/gen.py", line 617, in run
value = future.result()
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/tornado/concurrent.py", line 109, in result
raise_exc_info(self._exc_info)
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/tornado/gen.py", line 620, in run
yielded = self.gen.throw(*sys.exc_info())
  File "/Users/artemkorhov/Projects/cartreminder/cartreminder_app/tracking_api/api_handlers/endpoints.py", line 35, in get
tags = yield self.motor.tags.find_one({"client_id": data["client_id"]})
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/tornado/gen.py", line 617, in run
value = future.result()
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/tornado/concurrent.py", line 109, in result
raise_exc_info(self._exc_info)
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/motor/__init__.py", line 676, in call_method
result = sync_method(self.delegate, *args, **kwargs)
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/pymongo/collection.py", line 721, in find_one
for result in cursor.limit(-1):
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/pymongo/cursor.py", line 1038, in next
if len(self.__data) or self._refresh():
  File "/Users/artemkorhov/Projects/cartreminder/env/lib/python2.7/site-packages/pymongo/cursor.py", line 982, in _refresh
self.__uuid_subtype))
RuntimeError: maximum recursion depth exceeded while encoding an object to BSON 

In mongoDB "tags" collection i have (for example):

{
"_id" : ObjectId("540eec8227c565f77d4dcd23"),
"client_id" : "1111",
"tags" : {
    "cart_add" : [
        {
            "action_element" : "#addbutton1",
            "info_element" : "#product_element1"
        }
    ],
    "cart_delete" : [
        {
            "action_element" : "#deleteButton1",
            "info_element" : "#product_element1"
        }
    ],
    "email_known" : {
        "info_element" : ".tag1"
    },
    "order_complete" : {
        "action_element" : "#order_button1",
        "info_element" : {
            "product_wrap" : ".product_wrap",
            "product_id" : ".product_id_element",
            "quantity" : ".product_quantity_element",
            "price" : ".product_price_element"
        }
    }
}
}

The interesting part is that same 'find' method works perfect in other handlers, which built almost the same

Answer 1

Your "data" dictionary has a circular reference, so when Motor passes "data" to PyMongo to be encoded as BSON and sent to the server, the BSON encoder recurses more than 1000 times. I can reproduce this error message like so:

>>> import bson
>>> d = {}
>>> d['key'] = d  # Circular reference!
>>> bson.BSON.encode(d)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/emptysquare/.virtualenvs/motor/lib/python2.7/site-packages/bson/__init__.py", line 590, in encode
    return cls(_dict_to_bson(document, check_keys, uuid_subtype))
RuntimeError: maximum recursion depth exceeded while encoding an object to BSON

Try doing "pprint.pprint" on "data" to see where the self-reference occurs:

>>> import pprint
>>> pprint.pprint(d)
{'key': <Recursion on dict with id=140199700593680>}