Graph Mail API: 504 Gateway Timeout Errors

Question

I have a script which downloads messages from an Office 365 mailbox. In the logs for the script I am noticing a lot of responses with this error Response: 504, Gateway Timeout. The error occurs a few times a week at somewhat sporadic times. Here are the dateimes for the last few occurances:

• 12/05/2018 07:51:44 PM
• 12/05/2018 05:50:58 PM
• 12/05/2018 03:50:54 PM
• 12/05/2018 03:51:01 AM
• 12/04/2018 05:53:13 PM
• 11/28/2018 03:50:53 AM
• 11/22/2018 11:50:53 PM

The rest of the time the script works perfectly without error. The documentation state's that this error code indicates that:

"The server, while acting as a proxy, did not receive a timely response from the upstream server it needed to access in attempting to complete the request. May occur together with 503."

I have seen a few other posts about this but none of them really offered an explanation as to why this error occurs, or what can be done to avoid it. If the server response is timing out does this mean that I need to change my requests? Right now I am requesting the messages in chunks of 1000 using the @odata.nextLink response parameter to request the next chunk.

Answer 1

The 504 is most likely being caused by your page size. 1000 is a massive page size and the request is most likely timing out attempting to fetch that many emails at once (particularly if the emails themselves are large).

Microsoft Graph is an API aggrigator and under the covers, it is relaying each incoming request to one or more underlying APIs. In this case, it is proxying the request to the Exchange/Outlook APIs.

With a page size of 1000, Graph has to wait for Exchange to compose a response with 1,000 emails in it and pass it back into Graph for final processing (Graph rewrites, filters and merges API responses so you get consistent OData back, regardless of the endpoint(s) you hit). A 504 - Gateway timeout exception means the Graph timed out before the underlying workload could return the result.

Using a smaller page size (100-300 is pretty typical) should ensure the underlying workload can respond before timing out. Some folks have tried to trim back 100 at a time until they're eliminated but I'd not recommend this methodology as it inevitably means you haven't solved the problem, only introduced a rarer race condition.