semenbari
semenbari

Reputation: 805

What makes triton return 503 error sometime?

I had deployed 14 models on triton server and called them with 100 http rest api request at once after finishing them, calling over again and over again. Firsttime deploying, it looks fine. But after 24h later, it began to show 503 errors about 20 ~ 30 in 100 requests. There were no high cpu usage, memory, GPU usage shown, I only checked this triton web page. Since I got this development evironment in my company and related guy said there are no console, so I couldn't get more detail information such as log. After rollback, I tested varying the count of request and 10 request at once looks fine and no error and deploy again. After 30 hours later now, it is fine. But how could I know what the real problem was? I guess there are some configuration on triton. What makes triton return 503 error without hardware performance issue? It has some leak issue? Thanks

Upvotes: 0

Views: 16

Answers (0)

Related Questions