Error deploying a python flask app with Blender script

Question

I have a python flask app, with some APIs that take image links, process those images using a blender script using pip bpy and return the images.

The blender script is working when I run it locally using python script.py, however if I try to call it using the api either locally or deployed on a Linux server, it gives error

ImportError: libX11.so.6: cannot open shared object file: No such file or directory

I have blender GUI on my laptop locally so I think it provides all needed rendering libraries for the script, so I tried installing blender, bpy package and some other required libraries to my server to get it working but still the same error.

I'm deploying to Railway.app, which uses Nixpacks to automate installion, build and deployment, and here is my nixpacks.toml:

[phases.setup]
nixPkgs = [
  "blender",
  "libX11",
  "libXrender",
  "libXext",
  "mesa",
  "glu",
  "libxi",
  "libxxf86vm",
  "libsm",
  "libopenal",
  "ffmpeg"
]

and my requirements.txt:

asttokens==2.4.1
attrs==23.2.0
backcall==0.2.0
beautifulsoup4==4.12.3
bleach==6.1.0
blinker==1.8.2
bpy==4.1.0
certifi==2024.7.4
charset-normalizer==3.3.2
click==8.1.7
colorama==0.4.6
decorator==5.1.1
defusedxml==0.7.1
docopt==0.6.2
executing==2.0.1
fastjsonschema==2.20.0
Flask==3.0.3
idna==3.7
ipython==8.12.3
itsdangerous==2.2.0
jedi==0.19.1
Jinja2==3.1.4
jsonschema==4.23.0
jsonschema-specifications==2023.12.1
jupyter_client==8.6.2
jupyter_core==5.7.2
jupyterlab_pygments==0.3.0
MarkupSafe==2.1.5
matplotlib-inline==0.1.7
mistune==3.0.2
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
numpy==1.26.4
opencv-python-headless==4.9.0.80
packaging==24.1
pandocfilters==1.5.1
parso==0.8.4
pickleshare==0.7.5
pillow==10.3.0
pipreqs==0.5.0
platformdirs==4.2.2
prompt_toolkit==3.0.47
pure-eval==0.2.2
Pygments==2.18.0
python-dotenv==1.0.1
# commented as they threw errors during deployment, package not found
# python-dateutil==2.9.0.post0
# pywin32==306
pyzmq==26.0.3
referencing==0.35.1
requests==2.32.3
rpds-py==0.19.0
six==1.16.0
soupsieve==2.5
stack-data==0.6.3
tinycss2==1.3.0
tornado==6.4.1
traitlets==5.14.3
urllib3==2.2.2
waitress==3.0.0
wcwidth==0.2.13
webencodings==0.5.1
Werkzeug==3.0.3
yarg==0.1.9

and my main.py flask server:

from flask import Flask, request, send_file, jsonify, abort
import os
from waitress import serve

app = Flask(__name__)

@app.route('/process-image', methods=['POST'])
def process_image():
    data = request.get_json()
    design_path = data.get('designPath')
    
    
    if not design_path:
        return "Invalid design path", 400

    output_path = 'processed.png'
    crop_output_path = 'processed_cropped.png'

    process_image(design_path, output_path, crop_output_path)
    return send_file(output_path, mimetype='image/png')

def handle_bad_request(e):
    return 'bad request!', 400

app.register_error_handler(400, handle_bad_request)

if __name__ == '__main__':
    port = int(os.getenv('FLASK_PORT', 3000))
    serve(app, host='0.0.0.0', port=port)

Any help deploying blender and fixing this error will be much appreciated. Thanks