CUDA warp scheduling: another warp from the current block or a warp from another block?

Question

When a running warp on a SM is stalled, another warp must be scheduled on the SM to exchange the stalled one to cover latency. But is the substitute from the same block as the stalled warp or it must belong to another block, or both cases are OK?

Answer 1

The topic is an open research in GPU architecture community. People have evaluated several warp scheduling policies at here. Look at the following recent papers for pros and cons of different scheduling policies:

• Some people suggest to use two-level warp scheduling for energy efficiency: Energy-efficient Mechanisms for Managing Thread Context in Throughput Processors
• Some people suggest to use two-level warp scheduling for improving latency hiding: Improving GPU Performance via Large Warps and Two-Level Warp Scheduling
• Some other people suggest to schedule the warp in the order reducing contention on L1 cache of SM: Cache-Conscious Wavefront Scheduling
• And finally, here is a patent from NVIDIA researchers proposing an scheduler for improving latency hiding and energy efficiency.

In answer to your question, both policies are OK and can be employed in any architecture. But the real underling has not been disclosed to public yet.