CODEWITHSUNDEEP
cachingmemcachedreverse-proxyvarnishesi
Pascal
Pascal

Reputation: 85

Varnish & ESIs : Fetching in parallel and possible workarounds

I'm investigating using Varnish with ESIs to cache page content for a high traffic forum-like website.

Context : I want to cache content for visitors only (connected users will have a really different display and need absolute fresh content). Still, for visitors, some parts of a page need to be dynamic : - not cachable, for example for a visitor-dependant module (think a 'suggestion' widget that is fed with a real-time analysis of the pages viewed, thanks to a beacon) - cachable with a small TTL of 15mn, for example for a 'latest posts' widget or for very changing ads campaigns.

Currently we are using Apache/PHP/symfony/memcache to cache pages and have in-house ESI-like mecanism : a page from cache is parsed and some specific tags are interpreted (including calls to web services and/or databases). This is not performant enough since server time is then around 700ms.

In remplacement of this solution, we can have Varnish+ESIs. The total nb of ESIs included in a page can reach 15. The real number of ESIs to fetch will be less than that but not so much given the ESI's TTLs. The critical problem is that Varnish fetches the ESIs sequencially instead of parallel and this is not acceptable. This feature is somewhere late in Varnish's roadmap.

So,

Thanks, P.

Upvotes: 4

Views: 893

Answers (1)

Arjan
Arjan

Reputation: 416

Currently i work for a high traffic site, and performance is everything for us. On several pages we use a lot (20+) of ESI's, for example on our search resultlist. The resultlist is JSON response, and every resultblock in it is a seperate ESI. Okay, we do cache warming. But we didn't run in any performance issues on this. The number of ESI's will be a problem if the backend requests are real slow.

Parallel ESI fetching is on the feature request list of Varnish, but i think it didn't make it in version 4.1.

Upvotes: 1

Related Questions