CODEWITHSUNDEEP
pythonmultithreadingflaskpython-multithreadingbackground-processing
Bilal Zafar
Bilal Zafar

Reputation: 11

Background thread stops unexpectedly and silently when cleaning up batches in Flask app

Question:

I am working on a Flask application where I need to periodically clean up stale batches in the database. To avoid unnecessary processing, I set up a background thread that checks for inactive batches and deletes them if they haven’t received any data. The background thread is started when the website receives a request and stops when there is no activity for 10 minutes.

However, the background thread closes unexpectedly and silently without any error, and I’m unable to figure out why. It seems like the thread is terminating, but there’s no exception or log to indicate what’s going wrong.

Here’s a simplified version of my code:

Code:


import threading
from time import sleep
from datetime import datetime, timedelta
from flask import Flask, request
from sqlalchemy import text

app = Flask(__name__)

last_request_time = None
background_task_running = False
task_lock = threading.Lock()

# Function to update last request time on each user request
def update_last_request_time():
    global last_request_time
    last_request_time = datetime.utcnow()

# Function to start the background task, ensuring it only runs when necessary
def start_background_task():
    global background_task_running
    with task_lock:
        if not background_task_running:
            try:
                print("Starting background task")
                background_thread = threading.Thread(target=check_for_inactive_batches)
                background_thread.daemon = True  # Daemon thread will stop with the app
                background_thread.start()
                background_task_running = True
            except Exception as e:
                print(f"Error Starting background task: {e}")

# Function to stop the background task
def stop_background_task():
    global background_task_running
    with task_lock:
        if background_task_running:
            print("Stopping background task")
            background_task_running = False

# Background task to check for inactive batches and delete them if necessary
def check_for_inactive_batches():
    global background_task_running
    engine = db_connection()  # Database connection setup
    while background_task_running:
        if last_request_time and (datetime.utcnow() - last_request_time).total_seconds() > 600:
            stop_background_task()
            print("No requests for 10 minutes, stopping background task.")
            break

        current_time = datetime.utcnow()
        with engine.connect() as conn:
            try:
                delete_query = text("""
                    DELETE FROM TB_BATCHES
                    WHERE LastHeartbeat < :threshold_time
                    AND Status = 'scanning' OR Status = 'uploading'
                """)
                threshold_time = current_time - timedelta(seconds=70)
                conn.execute(delete_query, {'threshold_time': threshold_time})
                conn.commit()
                print(f"Checked and deleted inactive batches at {current_time}")
            except Exception as e:
                print(f"Error deleting inactive batches: {e}")

        sleep(60)  # Check every 60 seconds

# Flask @before_request hook to update user activity and start the background task if necessary
@app.before_request
def track_user_activity():
    if request.endpoint == 'get_batches':
        return  # Skip cleanup logic for this route as receiving periodic calls
    
    update_last_request_time()
    start_background_task()

Environment:

WSGI:

What I've Tried:

  1. Logging: I added print statements, but they don’t show any errors or exceptions before the thread stops. The log just stops without further output at "Starting background task".

  2. Thread Daemon: I made the thread a daemon so that it stops when the app stops, but that doesn’t seem related to the issue.

  3. SQLAlchemy Errors: I included try/except blocks around my database queries, but no errors are caught.

  4. Checked Resource Usage: There doesn't seem to be any significant memory or CPU usage spikes when this issue occurs.

Expected Behavior:

The background thread should keep running until the app receives no requests for 10 minutes and start again after new request, and it should not terminate silently without completing the cleanup process.

Actual Behavior:

The background thread stops without any error or log output Last output I get is "Starting background task", and I’m unable to determine why the actual query doesn't run.

Upvotes: 0

Views: 49

Answers (0)

Related Questions