For which sizes are plain loads and store to global memory in CUDA atomic?

Question

Are general reads and writes to global memory atomic in CUDA if:

• It is a 4 byte instruction? (I assume yes)
• It is a 8 byte or 16 byte instruction? (I assume yes)

Are at least on Kepler and Fermi general 4 byte reads and writes to global memory atomic on Warp level or 8/16 Byte instructions atomic on half/quater Warp level if:

• All warp threads access the same 32-byte L2 transaction block? (I assume yes)
• Warp threads access different 32-byte L2 transaction blocks but all warp threads access the same 128 byte L2 cache line? (I assume no)
• All warp threads accesss different L2 cache lines? (I assume no)

If any of those assumptions about the atomicness on warp level is correct, is there any method of harnessing this knowledge without risking the compability to future Compute Capabilites?

Answer 1

Reads and writes generally take place with respect to the caches. By the time the transactions are issued to global memory, there is no guarantee of atomicity in the CUDA programming or memory model, unless atomic instructions are used.

For example, suppose a thread in a threadblock updates a 4-byte quantity in L2 on Kepler. Now, another thread, in another warp, threadblock, or kernel could update just one of those 4 bytes, in the L2, before that cacheline gets evicted to global memory. By the time the cacheline gets evicted to global memory, it may not represent what was written either by the original thread or even the second thread (for example if a third write came along...).

Keep in mind the L2 is a write-back cache, cannot be disabled, and is not bypassed by global reads and writes, except in the case of atomic instructions.