CODEWITHSUNDEEP
pythonopencvvideo-streamingstreamingfastapi
Swaraj
Swaraj

Reputation: 109

Video Streaming App using FastAPI and OpenCV

I am trying to render an HTML page that shows video streaming from a webcam. However, I am facing the following error:

500 Server Error TypeError: TemplateResponse() missing 1 required positional argument: 'context'

My FastAPI app:

from fastapi import FastAPI
import uvicorn
from fastapi import Depends, FastAPI
from fastapi import FastAPI, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
import cv2

app = FastAPI(debug=True)
templates = Jinja2Templates(directory="templates")

@app.get("/")
async def index():
    return templates.TemplateResponse("index.html")


async def gen_frames(camera_id):
    cap=  cv2.VideoCapture(0)

    while True:
        # for cap in caps:
        # # Capture frame-by-frame
        success, frame = cap.read()  # read the camera frame
        if not success:
            break
        else:
            ret, buffer = cv2.imencode('.jpg', frame)
            frame = buffer.tobytes()
            yield (b'--frame\r\n'b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')  
    
   if __name__ == '__main__':
    uvicorn.run(app,  host="127.0.0.1",port=8000)

My HTML Page (index.html):

<!doctype html>
<html lang="en">
<head>
    <!-- Required meta tags -->
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

    <!-- Bootstrap CSS -->
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css"
          integrity="sha384-MCw98/SFnGE8fJT3GXwEOngsV7Zt27NXFoaoApmYm81iuXoPkFOJwJ8ERdknLPMO" crossorigin="anonymous">

    <title>Multiple Live Streaming</title>
</head>
<body>
<div class="container">
    <div class="row">
        <div class="col-lg-7">
            <h3 class="mt-5">Multiple Live Streaming</h3>
            <img src="{{ url_for('video_feed', id='0') }}" width="100%">
        </div>
    </div>
</div>
</body>
</html>

Traceback:

Error capture

Upvotes: 7

Views: 11255

Answers (2)

Chris
Chris

Reputation: 34045

With regard to the error presented in your question, that is:

 TemplateResponse() missing 1 required positional argument: 'context'

when using Templates, you need to pass the Request object in the contextdictionary of TemplateResponse using request as the key, as shown below (Update: As described in the documentation link above, FastAPI/Starlette has recently made some changes, including the request object not being passed as part of the context anymore, as well as the name of the template not being the first argument in TemplateResponse (with request taking its place). However, as shown in the relevant source code, Starlette seems that it still supports—not sure for how long though—both ways of creating a TemplateResponse, even though the initial one is now deprecated, as can be seen from the source code. Please refer to the links above for more details):

from fastapi import Request

@app.get('/')
async def index(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})

Below are given two options (with complete code samples) on how to stream (live) video using FastAPI and OpenCV. Option 1 demonstrates an approach based on your question using the HTTP protocol and FastAPI/Starlette's StreamingResponse. Option 2 uses the WebSocket protocol, which can easily handle HD video streaming and is supported by FastAPI/Starlette (documentation can be found here and here)

I would also suggest having a look at this answer to better understand async/await in Python, and FastAPI in particular, as well as when to define an endpoint in FastAPI with async def or normal def. For instance, the functions of cv2 library, such as camera.read() and cv2.imencode(), are synchronous blocking (either IO-bound or CPU-bound) functions, which means that, when called, they would block the event loop, until they complete. That is the reason that both the StreamingResponse generator (i.e., the gen_frames() function) and the /video_feed endpoint in Option 1 are defined with normal def, as this would cause any requests to that endpoint to be run in a separate thread from an external threadpool that would be then awaited (hence, FastAPI will still work asynchronously). It should be noted though that, in the case of StreamingResponse, even if one defined the /video_feed endpoint with async def and the StreamingResponse generator with normal def (as it contains blocking operations), FastAPI/Starlette, in order to prevent the event loop from getting blocked, would use iterate_in_threadpool() to run the StreamingResponse generator in a separate thread that would be then awaited (similar to what FastAPI would do with def endpoints)—see this answer for more details and the relevant source code. Hence, any requests to that async def endpoint would not block the event loop because of some blocking StreamingResponse generator. Note that you should avoid defining the generator with async def, if it contains blocking operations, as FastAPI/Starlette would then run it directly in the event loop, instead of inside a separate thread. Only define it with async def, when you are certain that there is no synchronous operation taking place inside that would block the event loop, and/or when need to await for coroutines/async functions.

In Option 2, the /ws endpoint had to be defined with async def, as FastAPI/Starlette's websockets functions are async functions and need to be awaited. In this case, the cv2 blocking functions will block the event loop every time they get called, until they complete (which may or may not be trivial, given the time taken to complete those operations, as well as the requirements of one's project; for instance, whether the application is expected to serve multiple requests/users at the same time). However, one could still run blocking operations, such as cv2's functions, inside async def endpoints without blocking the event loop, as well as having multiple instances (workers/processes) of the application running at the same time. Please take a look at the linked answer above for more details and solutions on this subject.

Option 1 - Using HTTP Protocol

You can access the live streaming at http://127.0.0.1:8000/.

app.py

import cv2
import time
import uvicorn
from fastapi import FastAPI, Request
from fastapi.templating import Jinja2Templates
from fastapi.responses import StreamingResponse

app = FastAPI()
camera = cv2.VideoCapture(0, cv2.CAP_DSHOW)
templates = Jinja2Templates(directory="templates")


def gen_frames():
    while True:
        success, frame = camera.read()
        if not success:
            break
        else:
            ret, buffer = cv2.imencode('.jpg', frame)
            frame = buffer.tobytes()
            yield (b'--frame\r\n'
                   b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')
        time.sleep(0.03)


@app.get('/')
async def index(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})


@app.get('/video_feed')
def video_feed():
    return StreamingResponse(gen_frames(), media_type='multipart/x-mixed-replace; boundary=frame')


if __name__ == '__main__':
    uvicorn.run(app, host='127.0.0.1', port=8000, debug=True)

templates/index.html

<!DOCTYPE html>
<html>
    <body>
        <div class="container">
            <h3> Live Streaming </h3>
            <img src="{{ url_for('video_feed') }}" width="50%">
        </div>
    </body>
</html>

Option 2 - Using WebSocket Protocol

You can access the live streaming at http://127.0.0.1:8000/. Related answers using the WebSocket protocol can be found here, as well as here and here.

app.py

from fastapi import FastAPI, Request, WebSocket, WebSocketDisconnect
from websockets.exceptions import ConnectionClosed
from fastapi.templating import Jinja2Templates
import uvicorn
import asyncio
import cv2

app = FastAPI()
camera = cv2.VideoCapture(0,cv2.CAP_DSHOW)
templates = Jinja2Templates(directory="templates")


@app.get('/')
async def index(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})


@app.websocket("/ws")
async def get_stream(websocket: WebSocket):
    await websocket.accept()
    try:
        while True:
            success, frame = camera.read()
            if not success:
                break
            else:
                ret, buffer = cv2.imencode('.jpg', frame)
                await websocket.send_bytes(buffer.tobytes()) 
            await asyncio.sleep(0.03)
    except (WebSocketDisconnect, ConnectionClosed):
        print("Client disconnected")   

 
if __name__ == '__main__':
    uvicorn.run(app, host='127.0.0.1', port=8000)

Below is the HTML template for establishing the WebSocket connection, receiving the image bytes and creating a Blob URL (which is released after the image is loaded, so that the object will subsequently be garbage collected, rather being kept in memory, unnecessarily), as shown here, to display the video frame in the browser.

templates/index.html

<!DOCTYPE html>
<html>
    <head>
        <title>Live Streaming</title>
    </head>
    <body>
        <img id="frame" src="">
        <script>
            let ws = new WebSocket("ws://localhost:8000/ws");
            let image = document.getElementById("frame");
            image.onload = function(){
                URL.revokeObjectURL(this.src); // release the blob URL once the image is loaded
            } 
            ws.onmessage = function(event) {
                image.src = URL.createObjectURL(event.data);
            };
        </script>
    </body>
</html>

Below is also a Python client based on the websockets library and OpenCV, which you may use to connect to the server, in order to receive and display the video frames in a Python app.

client.py

from websockets.exceptions import ConnectionClosed
import websockets
import numpy as np
import asyncio
import cv2


async def main():
    url = 'ws://127.0.0.1:8000/ws'
    
    async for websocket in websockets.connect(url):
        try:
             #count = 1
             while True:
                contents = await websocket.recv()
                arr = np.frombuffer(contents, np.uint8)
                frame = cv2.imdecode(arr, cv2.IMREAD_UNCHANGED)
                cv2.imshow('frame', frame)
                cv2.waitKey(1)
                
                #cv2.imwrite("frame%d.jpg" % count, frame)
                #count += 1
        except ConnectionClosed:
            continue  # attempt reconnecting to the server (otherwise, call `break` instead)


asyncio.run(main())

Upvotes: 6

Yagiz Degirmenci
Yagiz Degirmenci

Reputation: 20598

You need to pass a Request when you are working with templates.

from fastapi import Request

@app.get("/")
async def index(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})

So you can also serve your video as another path, with StreamingResponse

from fastapi.responses import StreamingResponse


@app.get("/serve/{camera_id}", include_in_schema=False)
async def serve_video(camera_id: int):
    return StreamingResponse(gen_frames(camera_id))

Then fetch that response with Ajax or Axios etc.

Upvotes: 8

Related Questions