PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: './instance/test_db.sqlite'

Question

I'm having a trouble with tear down of my pytest context. I have a Flask app which creates sqlite3 database in my instance directory, looks like this:

# ./src/app_factory/__init__.py

def create_app():
    app = Flask(__name__, instance_path=f'{os.path.abspath("instance")}', instance_relative_config=True, template_folder='templates')

# some code here

    with app.app_context():
        from database.database_factory import init_db
        init_db()

# some other code here

    return app

init_db() function is defined like this:

# ./src/database/database_factory.py

engine = create_engine(current_app.config['SQLALCHEMY_DATABASE_URI'])
db_session = scoped_session(sessionmaker(autocommit=False,
                                         autoflush=False,
                                         bind=engine))


def init_db():
    from database.models import Base, Table1, Table2, Table3, Table4
    Base.metadata.create_all(bind=engine,
                             tables=[Table1.__table__, Table2.__table__, Table3.__table__, Table4.__table__])


@current_app.teardown_appcontext
def shutdown_session(exception=None):
    db_session.remove()

I'm writing some tests for this app and I have a problem with removing ./tests/instance/test_db.sqlite file after testing. I want to tear it down after I yield my app in a fixture, which looks like this:

# ./tests/unit/conftest.py

@pytest.fixture()
def app():
    app = create_app()
    yield app
    os.remove('./instance/test_db.sqlite')

The test fails with error "PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: './instance/test_db.sqlite'"

From what I understand, after yield app in a fixture, app context should be not available anymore, so as scoped_session should be closed. So I don't understand what is causing this problem and I'm not even sure if active app context is the reason of this error.

I'm not sure if it's relevant but layout of my app looks like this:

.
├── src/
│   ├── app_factory/
│   │   └── __init__.py
│   ├── database/
│   │   └── database_factory.py
│   └── instance/
│       └── test_db.sqlite
└── tests/
    ├── unit/
    │   ├── conftest.py (this has fixtures)
    │   └── test_db.py
    └── instance/
        └── test_db.sqlite (I want to delete this one as part of teardown)

sqlite3 db is created in instance folder which is relative to absolute path of where the create_app() was invoked (either in src or tests directory).

I found this SO post Python PermissionError: [WinError 32] The process cannot access the file..... but my file is closed and I tried to explicitly close the session within teardown part of fixture like this:

@pytest.fixture()
def app():
    app = create_app()
    yield app
    from database.database_factory import db_session
    db_session.close()
    db_session.remove()
    os.remove('./instance/test_db.sqlite')

but this does not help either. What is also strange is that in the last example I can import db_session after yield app, because in any other test case, test fails if I try to do this import outside of app context.

Kindly please for any advice.

Answer 1

Ok, I searched a bit more and I found this post: https://stackoverflow.com/a/21742461/9280629

Looks like the problem was not with closing session but with engine connection active. Once I disposed my engine with dispose() function, the sqlite3 db file can be successfully removed.