Flask, Pymongo and Mongoengine - ImportError: cannot import name 'app'

Question

I'm running a flask application using mongoengine as a mongodb orm, on python 3.

I'm configuring mongo connection in the general config file as follows:

MONGODB_SETTINGS = {
    'db': "dbname",
    'host': "myhost",
    'port': "27017",
    # 'username': 'xxxxxx',
    # 'password': 'xxxxxx'
}

I'm creating my mongoengine instance: db = MongoEngine(app)

However in my model.py if I don't include an implicit connect command, it doesn't get the params, and it tries to connect to localhost insted of the host I configured.

So I add the connect trying to use the config params as follows:

from app.engine import app
connect(db=app.config['MONGODB_SETTINGS']['db'], host=app.config['MONGODB_SETTINGS']['host'])

class Module(Document):
    identifier = StringField()
    path = StringField()
    description = StringField()
    name = StringField()
    method = StringField()
    meta = {'collection': 'modules'}
    ...

And I get this exception:

[2019-01-08 14:56:51 +0000] [8] [INFO] Starting gunicorn 19.9.0
[2019-01-08 14:56:51 +0000] [8] [INFO] Listening at: https://0.0.0.0:5000 (8)
[2019-01-08 14:56:51 +0000] [8] [INFO] Using worker: eventlet
[2019-01-08 14:56:51 +0000] [10] [INFO] Booting worker with pid: 10
[2019-01-08 14:56:51 +0000] [10] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/app/env/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/app/env/lib/python3.6/site-packages/gunicorn/workers/geventlet.py", line 102, in init_process
    super(EventletWorker, self).init_process()
  File "/app/env/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/app/env/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/app/env/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/app/env/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/app/env/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/app/env/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/app/engine.py", line 10, in <module>
    from app.controllers.modules import modules
  File "/app/controllers/modules.py", line 8, in <module>
    from app.domain.model import DownloadPending
  File "/app/domain/model.py", line 1, in <module>
    from app.engine import app
ImportError: cannot import name 'app'
[2019-01-08 14:56:51 +0000] [10] [INFO] Worker exiting (pid: 10)
[2019-01-08 14:56:51 +0000] [8] [INFO] Shutting down: Master
[2019-01-08 14:56:51 +0000] [8] [INFO] Reason: Worker failed to boot.

Besides of that I also have a service which uses pymongo aggregations directly to store data in collections that are dynamically named, so I have to include -again- the mongo params and a connect command in that service.

I would like to know if there is any way to reuse the mongo params as they are in the config file for all the cases.

UPDATE This is the app.engine code (main py):

import logging
import os
from datetime import date, timedelta

from celery.schedules import crontab
from flask import Flask
from flask import request

from app.controllers.auth import auth
from app.controllers.modules import modules
from app.controllers.companies import companies
from app.service.services import ModuleService
from flask_jwt_extended import (
    JWTManager, jwt_required
)
from flask_cors import CORS
from flask_socketio import SocketIO

from celery import Celery

import eventlet

from app.utils.JSONEncoder import JSONEncoder
from flask_mongoengine import MongoEngine


eventlet.monkey_patch()

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


def create_app(config_name):
    _app = Flask(__name__)
    _app.config.from_pyfile('config.py', silent=True)
    return _app


def make_celery(_app):
    celery = Celery(
        _app.import_name,
        backend=_app.config['CELERY_RESULT_BACKEND'],
        broker=_app.config['CELERY_BROKER_URL']
    )
    celery.conf.update(_app.config)

    class ContextTask(celery.Task):
        def __call__(self, *args, **kwargs):
            with _app.app_context():
                return self.run(*args, **kwargs)

    celery.Task = ContextTask
    return celery


app = create_app('profile')
# app = create_app('config')

db = MongoEngine(app, config={
    'db': "dbname",
    'host': "myhost",
    'port': 27017,
})

CORS(app)

UPLOAD_FOLDER = '/app/import'

app.config['SECRET_KEY'] = 'xxxxxxxxxxxxxxxxxxxxx'
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER

app.config.update(
    CELERY_BROKER_URL='redis://redis:6379/0',
    CELERY_RESULT_BACKEND='redis://redis:6379/1',
)

celery = make_celery(app)
socketio = SocketIO(app, message_queue='redis://redis:6379/2')

"""
Services initialization
"""
modules_service = ModuleService()

jwt = JWTManager(app)


@celery.task()
def do(module_name, json_input):
    (..........)


app.register_blueprint(companies)
app.register_blueprint(modules)
app.register_blueprint(auth)

if __name__ == "__main__":
    socketio.run(host='0.0.0.0', threaded=True)

What's the best practise here?

I'd appreciate any help

Thank you.

Answer 1

Thanks @Danila Ganchar

I had to refactor the whole code in such a way that avoids circular dependencies conflicts and issues with imports order. Also externalized the database connection to another class, using Mongoengine's connect() command instead of using the mongoengine flask extension.

Everything works fine now. Thanks!