CODEWITHSUNDEEP
mongodbdockerflaskmongoengineflask-mongoengine
imhans4305
imhans4305

Reputation: 697

Mongo Auto Reconnect Error Continues even after resetting mongod.lock file (MongoEngine Connect)

I am using flask celery and mongo in seperate containers. WHen I start the containers, I am hit with the following error

worker           | pymongo.errors.AutoReconnect: connection pool paused
worker           | 
worker           | The above exception was the direct cause of the following exception:
worker           | 
worker           | Traceback (most recent call last):
worker           |   File "/opt/venv/lib/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
worker           |     R = retval = fun(*args, **kwargs)
worker           |   File "/opt/venv/lib/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
worker           |     return self.run(*args, **kwargs)
worker           |   File "/app/app/routes/celery_tasks.py", line 76, in nlp_ml
worker           |     data = CrawlData.objects(ml_proc='null')
worker           |   File "/opt/venv/lib/python3.8/site-packages/mongoengine/queryset/manager.py", line 38, in __get__
worker           |     queryset = queryset_class(owner, owner._get_collection())
worker           |   File "/opt/venv/lib/python3.8/site-packages/mongoengine/document.py", line 233, in _get_collection
worker           |     cls.ensure_indexes()
worker           |   File "/opt/venv/lib/python3.8/site-packages/mongoengine/document.py", line 913, in ensure_indexes
worker           |     collection.create_index(fields, background=background, **opts)
worker           |   File "/opt/venv/lib/python3.8/site-packages/pymongo/collection.py", line 1687, in create_index
worker           |     return self.__create_indexes([index], session, **cmd_options)[0]
worker           |   File "/opt/venv/lib/python3.8/site-packages/pymongo/collection.py", line 1557, in __create_indexes
worker           |     with self._socket_for_writes(session) as sock_info:
worker           |   File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
worker           |     return next(self.gen)
worker           |   File "/opt/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1099, in _get_socket
worker           |     with server.get_socket(
worker           |   File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
worker           |     return next(self.gen)
worker           |   File "/opt/venv/lib/python3.8/site-packages/pymongo/pool.py", line 1371, in get_socket
worker           |     sock_info = self._get_socket(all_credentials)
worker           |   File "/opt/venv/lib/python3.8/site-packages/pymongo/pool.py", line 1436, in _get_socket
worker           |     self._raise_if_not_ready(emit_event=True)
worker           |   File "/opt/venv/lib/python3.8/site-packages/pymongo/pool.py", line 1407, in _raise_if_not_ready
worker           |     _raise_connection_failure(
worker           |   File "/opt/venv/lib/python3.8/site-packages/pymongo/pool.py", line 250, in _raise_connection_failure
worker           |     raise AutoReconnect(msg) from error
worker           | pymongo.errors.AutoReconnect: mongo:27017: connection pool paused

When I delete /data/db/mongod.lock once and restarting container, the same error appears. But if deleted again, and restarted then this error wont appear till I make a change in flask code. If flask is reloaded with some code changes the same thing has to repeat again. How is it possible to remove this error permanently.

Upvotes: 0

Views: 571

Answers (1)

imhans4305
imhans4305

Reputation: 697

import logging

from mongoengine import *

class SafeDocumentMixin:

    def save_safe(self, *args, **kwargs):
        for attempt in range(5):
            try:
                return self.save(*args, **kwargs)
            except pymongo.errors.AutoReconnect as e:
                wait_t = 0.5 * pow(2, attempt) # exponential back off
                l.warning("PyMongo auto-reconnecting... %s. Waiting %.1f seconds.", str(e), wait_t)
                time.sleep(wait_t)

    @classmethod
    def objects_safe(cls, *args, **kwargs):
        for attempt in range(5):
            try:
                return cls.objects(*args, **kwargs)
            except pymongo.errors.AutoReconnect as e:
                wait_t = 0.5 * pow(2, attempt) # exponential back off
                logging.warning("PyMongo auto-reconnecting... %s. Waiting %.1f seconds.", str(e), wait_t)
                time.sleep(wait_t)

class Person(Document, SafeDocumentMixin):
    name = StringField()
    age = IntField()

SOlved using this explained in How to handle PyMongo/MongoEngine AutoReconnect?

Upvotes: 0

Related Questions