CODEWITHSUNDEEP
djangopostgresqlgunicorn
Michele Matera
Michele Matera

Reputation: 141

django.db.utils.InterfaceError: connection already closed

Stack: Ubuntu (20.04 LTS) Nginx Postgresql (v13.3) An AWS load balancer sends traffic to the Ubuntu instance(k8s cluster), which is handled by Nginx, which forwards on to Django (4.0.3) running in gunicorn (19.9.0). Django connects to the database using psycopg2 (2.8.6). The issue I have is that the database connection seems to shut down randomly.

Django reports errors like this:

`InterfaceError: connection already closed
  File "django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "django/views/generic/base.py", line 84, in view
    return self.dispatch(request, *args, **kwargs)
  File "django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "django/views/decorators/cache.py", line 62, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "django/views/generic/base.py", line 119, in dispatch
    return handler(request, *args, **kwargs)
  File "django/views/generic/detail.py", line 108, in get
    self.object = self.get_object()
  File "django/views/generic/detail.py", line 53, in get_object
    obj = queryset.get()
  File "django/db/models/query.py", line 492, in get
    num = len(clone)
  File "django/db/models/query.py", line 302, in __len__
    self._fetch_all()
  File "django/db/models/query.py", line 1507, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "django/db/models/query.py", line 57, in __iter__
    results = compiler.execute_sql(
  File "django/db/models/sql/compiler.py", line 1359, in execute_sql
    cursor = self.connection.cursor()
  File "django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "django/db/backends/base/base.py", line 284, in cursor
    return self._cursor()
  File "django/db/backends/base/base.py", line 262, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "django/db/backends/base/base.py", line 262, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "django/db/backends/postgresql/base.py", line 256, in create_cursor
    cursor = self.connection.cursor()`

Postgresql does not report any major errors: I can only assume that the connection was closed correctly and Django was not expecting it. This happens quite rarely, but enough to be a concern: once every 3 days. Any suggestions of how to investigate this further would be gratefully accepted. Thanks in advance!

Upvotes: 14

Views: 6342

Answers (2)

HasanYıldız
HasanYıldız

Reputation: 11

For Django daemons, calling db.close_old_connections() once in a while to refresh your db connection fixes a similar problem.

A better solution might be adding a retry functionality to db backend directly like this library: django-postgreconnect

Upvotes: 0

Jon Hebert
Jon Hebert

Reputation: 13

I'm using django-service-objects and Django-Q for a distributed data mining code. I ran into a error "django.db.utils.InterfaceError: connection already closed" when I was running Django-Q cluster async_task(..., sync=True) to debug and attempted to query the Django PostgreSQL 13 DB using Model.object.get(pk=...).

I resolved the issue by setting db_transaction = False within the Service class.

class dataMining(Service):
    db_transaction = False

https://django-service-objects.readthedocs.io/en/stable/pages/usage.html#database-transactions

The effect of db_transaction=True (enabled by default) is to "...process method on services ... inside a transaction". Meaning changes to the Django PostgreSQL DB are not actually saved until the def process(self) method of the Service class completes successfully. https://django-service-objects.readthedocs.io/en/stable/pages/usage.html#service

I suspect the self.object = self.get_object() call in your code sample is experiencing the same problem of being 'locked out' during a process method within a DB transaction.

Upvotes: 0

Related Questions