Flask Session Data Lost on Server (Gunicorn + Nginx) but Works Locally

Question

I am experiencing an issue with session persistence in a Flask application. The application works fine on my local machine, but on the server, the session data (user_id) is lost between requests. Logs on Local (Works as Expected):

2025-01-05 01:26:38 - INFO - Session before token handling: {'_permanent': True}
2025-01-05 01:26:39 - INFO - Dataset saved for user 5.
2025-01-05 01:26:39 - INFO - Token processed successfully for user 5.
2025-01-05 01:26:39 - INFO - Session at the start of chat: {'_permanent': True, 'user_id': '5'}

Logs on Server (Session Data Missing):

2025-01-05 01:28:52 - INFO - Session before token handling: {'_permanent': True}
2025-01-05 01:28:53 - INFO - Dataset saved for user 5.
2025-01-05 01:28:53 - INFO - Token processed successfully for user 5.
2025-01-05 01:28:53 - INFO - Session at the start of chat: {'_permanent': True}
2025-01-05 01:28:53 - WARNING - User not logged in. Using default dataset.

As seen in the logs, the user_id is set in the session during the token handling route but is missing in the subsequent request to the chat route.

Here is the relevant Flask setup and session configuration:

from flask import Flask, session
from flask_session import Session
import os
from datetime import timedelta

app = Flask(__name__)
app.secret_key = os.urandom(24)

# Session configuration
app.config["SESSION_TYPE"] = "filesystem"
SESSION_FILE_DIR = "./flask_session"
if not os.path.exists(SESSION_FILE_DIR):
    os.makedirs(SESSION_FILE_DIR)
app.config["SESSION_FILE_DIR"] = SESSION_FILE_DIR
app.config["PERMANENT_SESSION_LIFETIME"] = timedelta(minutes=30)
app.config["SESSION_USE_SIGNER"] = True
app.config["SESSION_COOKIE_HTTPONLY"] = True
app.config["SESSION_COOKIE_SAMESITE"] = "Lax"
app.config["SESSION_COOKIE_SECURE"] = False  # Using HTTP for now
Session(app)

Session data is being set in the following route:

@app.route("/<path:token>", methods=["GET"])
def handle_token(token):
    session["user_id"] = "5"  # Example user_id
    return "Token processed successfully"

And read in the chat route:

@app.route("/", methods=["GET"])
def chat():
    user_id = session.get("user_id")
    if not user_id:
        return "User not logged in. Using default dataset.", 400
    return f"User logged in with ID: {user_id}"

Environment Details

• Local Environment:

• OS: macOS/Linux

• Flask Version: 2.x

• Python Version: 3.9.x

• Server Environment:

• OS: Ubuntu 20.04

• Deployed via Gunicorn behind Nginx

• Flask Version: 2.x

• Python Version: 3.9.x

What I’ve Tried

1. Session Directory Verification:

• Confirmed that SESSION_FILE_DIR exists and is writable on the server.
• Observed that session files are created, but they do not persist data.

2. Cookie Configuration:

• Verified that the session cookie is sent to the browser. It appears to be correctly set.

3. Logging:

• Added extensive logging to track session data at various points in the application.

4. Secure Cookies:

• Set SESSION_COOKIE_SECURE = False for HTTP in the server environment.

5. Browser Dev Tools:

• Checked that the session cookie is present and has attributes:
• Secure: False (since HTTPS is not used).
• SameSite: Lax.
• Domain: localhost.

**Potential Hypotheses

1. File-Based Session Storage Issues:

• The session file might not be read consistently between requests.

2. Cookie Misconfiguration:

• Although the cookie is present, it might not be properly linked to the session file.

3. Load Balancer Configuration:

• If the app is behind a load balancer, requests might be routed to different server instances, causing session inconsistency.

Questions

1. Could the issue be related to the file-based session mechanism on the server? Would switching to Redis or another backend help?
2. How can I ensure session persistence in a production environment using Flask's filesystem-based session storage?
3. Is there anything specific about running Flask behind Gunicorn and Nginx that could cause session data to be lost?

Additional Notes

• I’ve tested with simplified routes for session set and get operations and observed the same behavior.
• The session works perfectly in my local environment, and the issue only occurs on the server.