Does heap allocation always mean immediate RAM access?

Question

Assuming Java JIT-compiles code such as building a linked list, is it possible for new Link (and its constructor) to return without accessing RAM at all? In other words, can VM, at least in theory, perform allocation only in the processor cache and only do actual memory allocation when flushing (for example, allocating the whole list or a list segment in one go)?

Aside from being interesting on its own, this came up while I was considering the actual relative penalty of using the volatile keyword when paired with new object creation. Assume a mutable list defined like this:

class Link<E> {
    final E elem;
    Link<E> next = null
    Link(E e, Link<E> tail) {
        elem = e;
        next = tail;
    }

    public void append(E e) {
        next = new Link(e, null);
    }
}

Can adding a volatile keyword to field next impact the performance of repeated calls to append (possibly shortly followed by dereferencing the whole collection, freeing the memory for garbage collection), limiting optimisations that JVM is able to do with the code in a significant way (at least in theory)?

Answer 1

In other words, can VM, at least in theory, perform allocation only in the processor cache and only do actual memory allocation when flushing...

Real machines don't generally have this ability. A memory word has "an address" (let's ignore any MMU for this discussion), and this address identifies the word whether it's in CPU cache or the real memory. That the OS knows an address means that it has in some sense "allocated memory". The CPU writes to memory. The hardware might store that in cache and/or memory, now or later, depending on implementation.

It is not usual to have "do not write back from cache" instructions (outside of processor initialization). Thus the CPU does not have a means to prevent the change from showing up in memory. It usually does have a means to force the change to memory; 'volatile' tells the compiler this is wanted.

The JVM, of course, is a virtual machine, so it can do what it wants. But I'd be quite surprised if there was a 'virtual cache' distinct from the real CPU cache.

One further note - operating systems usually have a mechanism for allocating demand-zero pages, so although the virtual memory exists, it is not necessarily backed by page frames until it is touched, i.e. "RAM" has not been "accessed". A large heap object could conceivably be created partially in this state. However, this is nothing to do with CPU caches.

As a long-practicing programmer, I find it generally not useful to think about "RAM", but instead think about "address space" and (virtual) "memory".

Answer 2

Can adding a volatile keyword to field next impact the performance of repeated calls to append (possibly shortly followed by dereferencing the whole collection, freeing the memory for garbage collection), limiting optimisations that JVM is able to do with the code in a significant way (at least in theory)?

The answer is, of course, yes. The JVM is an abstracted concept. Generalized statements about the performance of some java code only fall in these buckets:

• The Java Memory Model guarantees that this works fine (but that's always about whether something is buggy (sometimes buggy in a way that no existing VM can reproduce, but buggy nonetheless, in that some future version may break your code and it'd be your fault)) - this isn't about performance characteristics, though.

• A statement like: For all major platforms and all major VM releases, at their current versions, this code will perform well. This does not include guarantees about what will happen with the next release or if java runs on a new platform. Also, actually putting this together is incredibly difficult: You either need to be a walking dictionary that knows all the details about all the major VM implementations, or you need to do the work and have about 15 JVMs installed on each of about 3 OSes, on 30 different kinds of hardware platforms, for a total of over a 1000 combinations, that you'd need to performance test before making such statements.

So, given that the above is borderline insanity, the answer to your question is 'Yes'.

Mostly because that's the answer to any question of the form:

Can (anything) impact the performance of (anything) by limiting optimizations that a JVM is able to do in a significant way at least in theory?

Is 'Yes'.

Some specific knowledge that could be useful:

• A JVM is usually capable of (nearly) eliminating the performance impact of a volatile keyword if the keyword isn't doing anything useful (e.g. that field is only being read/written to be a single thread or the multi-thread accesses are spaced far apart in time).
• Generally you don't add volatile for funsies. You have a need for the threading behaviour, so you can't just compare performance of having the keyword vs. not having the keyword - not having the keyword means the code is broken, so that's obviously a useless and unfair comparison. Instead you need to compare against the next best alternative, such as using an AtomicX or rewriting the code to move the sync-up between threads up.