CODEWITHSUNDEEP
cachinghardware
Nathan Long
Nathan Long

Reputation: 126102

Does calling `writev` repeatedly with the same memory address allow hardware caching?

I've read some performance claims about how Elixir and Erlang use hardware, and I'm trying to see if I understand their basis. Some background:

First, Erlang supports writing nested lists of immutable strings (iolists) to IO (files, sockets, etc) and uses writev and the strings' memory addresses to do so (see Evan Miller's blog post on this).

Second, the docs for an Erlang web framework called Chicago Boss say:

Erlang Respects Your RAM!

Erlang is different from other platforms because when rendering a server-side template, it doesn't create a separate copy of a web page in memory for each connected client. Instead, it constructs pointers to the same pieces of immutable memory across multiple requests.

So if two people request two different profile pages at the same time, they're actually sent the same chunks of memory for the header, footer, and other shared template snippets. The result is a server that can construct complex, uncached web pages for hundreds of users per second without breaking a sweat.

Third, a book about an Elixir (Erlang VM) web framework called Phoenix says:

Templates are precompiled. Phoenix doesn’t need to copy strings for each rendered template. At the hardware level, you’ll see caching come into play for these strings where it never did before.

From looking at the source, I know that this framework uses iolists to represent a completed response template.

Putting all this together, I think what's being implied is that if a web framework uses writev to tell the OS to send the same header and footer strings from the same memory locations, one web request after another, the hardware will be able to say "oh, I know that value, it's already in CPU cache so I don't have to look in RAM for it."

Is that right? (I have very little understanding of system calls and hardware.) If not, any ideas on how hardware caching is involved?

(Bonus if you can tell me how to see or infer what's happening.)

Upvotes: 1

Views: 121

Answers (1)

Mircea
Mircea

Reputation: 10566

Yes, it's mostly the processor caches that help you. The time needed to retrieve the data is smaller as it's in a faster memory (ie the CPU caches).

Some pointers for understanding what the caches are and how they work:

To see this, measure how much a request takes (client side) in the normal server operation. After that have a separate process within the same vm that constantly creates and writes to disk a very large string (it probably has to be megabytes in size - whatever the size of the L2/L3 caches on your process are). Remeasure how much the request takes - if done correctly this should be at least 1 order of magnitude slower.

Upvotes: 1

Related Questions