CODEWITHSUNDEEP
pythonfastapistarlette
Alichszn
Alichszn

Reputation: 251

RuntimeError: No response returned in FastAPI when refresh request

I got this error in my application and i didn't know why. After many search and debugging just figured out that it happens when i refresh my request before getting response(cancel request and send another request while processing previous request). Because of that my application need more than 2 seconds to respond, i get too many of this type of error.

So far i know its from my middleware but i don't know why it happens and what should i do.

Any idea how to fix this issue ?

This is the error i get:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 81, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 76, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 35, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 101, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 367, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 55, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/gateway/./app/core/middlewares.py", line 26, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 37, in call_next
    raise RuntimeError("No response returned.")
RuntimeError: No response returned.

and this is my middleware:

class LoggerMiddleWare(BaseHTTPMiddleware):

    def __init__(self, app: ASGIApp):
        super().__init__(app)
        self.logger = logging.getLogger(self.__class__.__name__)
        self.logger.setLevel(logging.INFO)
        file_handler = logging.FileHandler('api.log')
        file_handler.setFormatter(JSONFormatter())
        self.logger.addHandler(file_handler)
        self.logger.addFilter(APIFilter())

    async def dispatch(self, request: Request, call_next):
        request.state.services = {}
        response = await call_next(request)
        self.logger.info(None, extra={'request': request, 'response': response})

        return response

I'm using fastapi 0.73 and starlette 0.17.1.

To reproduce this issue, we need to add two middlewares.

A minimal reproducible example can be found here: https://github.com/encode/starlette/issues/1634#issuecomment-1124806406

Upvotes: 25

Views: 31266

Answers (2)

aaron
aaron

Reputation: 43128

Update: As of 14 Nov 2022, this has been fixed in starlette==0.21.0 and fastapi==0.87.0.

This is due to how starlette uses anyio memory object streams with StreamingResponse in BaseHTTPMiddleware.

  1. When you cancel a request, the ASGI app receives the "http.disconnect" message.
  2. After your route function returns, your last middleware will await response(...).
  3. StreamingResponse's async def __call__ will call self.listen_for_disconnect and then task_group.cancel_scope.cancel() since the request is already disconnected. The stream is closed by the cancellation check in await checkpoint() of MemoryObjectSendStream.send before it has a chance to send the "http.response.start" message.
  4. Your second-to-last and earlier middlewares will encounter anyio.EndOfStream while await recv_stream.receive() in this part of BaseHTTPMiddleware's __call__ method: 
    try:
    message = await recv_stream.receive()
except anyio.EndOfStream:
    if app_exc is not None:
        raise app_exc
    raise RuntimeError("No response returned.")

assert message["type"] == "http.response.start"

#4 is why, to reproduce this issue, we need two middlewares that inherit BaseHTTPMiddleware.

Workaround 1

You can subclass BaseHTTPMiddleware to ignore that exception if the request is disconnected:

class MyBaseHTTPMiddleware(BaseHTTPMiddleware):

    async def __call__(self, scope, receive, send):
        try:
            await super().__call__(scope, receive, send)
        except RuntimeError as exc:
            if str(exc) == 'No response returned.':
                request = Request(scope, receive=receive)
                if await request.is_disconnected():
                    return
            raise

    async def dispatch(self, request, call_next):
        raise NotImplementedError()

Usage:

# class LoggerMiddleWare(BaseHTTPMiddleware):
class LoggerMiddleWare(MyBaseHTTPMiddleware):

Workaround 2

Actually, only the outermost BaseHTTPMiddleware needs to handle the exception, so you can just implement a SuppressNoResponseReturnedMiddleware and put it as your first middleware:

class SuppressNoResponseReturnedMiddleware(BaseHTTPMiddleware):

    async def dispatch(self, request, call_next):
        try:
            return await call_next(request)
        except RuntimeError as exc:
            if str(exc) == 'No response returned.' and await request.is_disconnected():
                return Response(status_code=HTTP_204_NO_CONTENT)
            raise

Reference: https://github.com/encode/starlette/discussions/1527#discussioncomment-2234702

Upvotes: 19

anjaneyulubatta505
anjaneyulubatta505

Reputation: 11705

The problem is that client is disconnecting the connection. After checking the code I found that starlette BaseHTTPMiddleware.__call__ method is throwing this error when the client disconnects the request.

so, the fix is to override __call__ method like below

    async def __call__(self, scope, receive, send) -> None:
        request = Request(scope, receive, send)
        try:
            await super().__call__(scope, receive, send)
        except RuntimeError as e:
            if not await request.is_disconnected():
                raise e
            print("remote disconnected")

I was able to reproduce the issue with https://github.com/encode/starlette/issues/1634#issuecomment-1124806406

You can find the code below to test it

playground.py

import asyncio
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import JSONResponse
from starlette.routing import Route


async def homepage(request):
    await asyncio.sleep(5)
    return JSONResponse({"hello": "world"})


routes = [
    Route("/", homepage, name="homepage"),
]


class CustomHeaderMiddleware1(BaseHTTPMiddleware):
    async def __call__(self, scope, receive, send) -> None:
        request = Request(scope, receive, send)
        try:
            await super().__call__(scope, receive, send)
        except RuntimeError as e:
            if not await request.is_disconnected():
                raise e
            print("remote disconnected")

    async def dispatch(self, request, call_next):
        return await call_next(request)


class CustomHeaderMiddleware2(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        return await call_next(request)


all_middleware = [
    Middleware(CustomHeaderMiddleware1),
    Middleware(CustomHeaderMiddleware2),
]


app = Starlette(debug=True, routes=routes, middleware=all_middleware)

Now, run the app. uvicorn playground:app

If we try curl http://127.0.0.1:8000 -m 3 it will give below output.

curl http://127.0.0.1:8000 -m 3
curl http://127.0.0.1:8000 -m 4
curl http://127.0.0.1:8000 -m 6

enter image description here

Because we are sleeping for 5 sec before the response. with the above cURL calls we are timing out the request for 3, 4 sec. so it's disconnected. for timeout 6sec. It returned the response.

so, we need to override the BaseHTTPMiddleware.__call__ method.

For your case, you update your code like below.

from starlette.requests import Request


class HTTPRemoteDisconnectedMixin:
    async def __call__(self, scope, receive, send) -> None:
        request = Request(scope, receive, send)
        try:
            await super().__call__(scope, receive, send)
        except RuntimeError as e:
            if not await request.is_disconnected():
                raise e
            print("remote disconnected")

class LoggerMiddleWare(HTTPRemoteDisconnectedMixin, BaseHTTPMiddleware):

    def __init__(self, app: ASGIApp):
        super().__init__(app)
        self.logger = logging.getLogger(self.__class__.__name__)
        self.logger.setLevel(logging.INFO)
        file_handler = logging.FileHandler('api.log')
        file_handler.setFormatter(JSONFormatter())
        self.logger.addHandler(file_handler)
        self.logger.addFilter(APIFilter())

    async def dispatch(self, request: Request, call_next):
        request.state.services = {}
        response = await call_next(request)
        self.logger.info(None, extra={'request': request, 'response': response})

        return response

Upvotes: 1

Related Questions