How to create a file caching layer in microservices/serverless architecture

Question

Here is the essence of the what I want:

• There is a service X which other services can use to stream files that service X stores. E.g. GET /files/8c267d1c-2b6d-4fe3-969f-4820fe8b3a9c which returns foo.txt's content might be requested by services A, B, C, multiple times by each.
• Service X be implemented using some serverless technology such as Azure Functions
• Obviously, I want some type of caching system so that I don't have to stream the file from it's stored location every time. Instead of a 2-part stream like

A <------ Instance of X Application <----- X's storage (e.g. Azure Files)

it would be great to have it like

A <------ [TBD]

• Obviously, I want this to scale
• Ideally, I want to use out-of-box solutions
• Ideally, I don't want to expose to the client service A, B, C the storage mechanism for them to directly access.

Are these goals incompatible?

Answer 1

Not sure if you have already considered CDN. There are quite a few providers in market and most of them are globally distributed. A lot depends on how frequent you update these contents. If they are not really static and updated frequently then CDN may not fit the purpose.

It does take care of your first two goals, but CND has a predefined and a well published architecture on how it operates, so not really sure of your point regarding not exposing the storage mechanism. You can control the access policies on these contents though.