CODEWITHSUNDEEP
vulkan
Jingbee
Jingbee

Reputation: 23

Why `VkPipelineStageFlagBits` does not only define pipeline stages?

VkPipelineStageFlagBits defines flags corresponding to stages that we would expect in a Graphics Pipeline such as VK_PIPELINE_STAGE_VERTEX_SHADER_BIT or VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT, etc. We also have Compute Pipeline stages such as VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT.

But there are also some flags defined here that does not seem to correspond to any Graphics or Compute Pipeline stage at all, such as VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT and VK_PIPELINE_STAGE_ALL_COMMANDS_BIT (from which VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT and VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT are defined). Instead, these are defined in the specification by referring to commands. For instance:

VK_PIPELINE_STAGE_TRANSFER_BIT specifies the following commands:

  • All copy commands, including vkCmdCopyQueryPoolResults
  • vkCmdBlitImage2 and vkCmdBlitImage
  • vkCmdResolveImage2 and vkCmdResolveImage
  • All clear commands, with the exception of vkCmdClearAttachments

So if it is not a pipeline stage, why is it listed in Vk**PipelineStage**FlagBits? Why is it named this way? What does it mean if I use VK_PIPELINE_STAGE_TRANSFER_BIT as srcStageMask or dstStageMask in VkSubpassDependency for instance?

Upvotes: 1

Views: 920

Answers (2)

krOoze
krOoze

Reputation: 13246

VK_PIPELINE_STAGE_TRANSFER_BIT is from transfer pipeline, which has one stage.

VK_PIPELINE_STAGE_HOST_BIT pseudostage communicates there will be memory domain transfers between host and device.

VK_PIPELINE_STAGE_ALL_COMMANDS_BIT is a shortcut for |ing together all the bits applicable in a given context.

VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT and VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT are pipeline terminators. They do not mean any stage, but they are necessary on the API level to express "before first stage", and "after last stage".

What does it mean if I use VK_PIPELINE_STAGE_TRANSFER_BIT as srcStageMask or dstStageMask in VkSubpassDependency for instance?

You are not allowed to, unless that dependency is VK_SUBPASS_EXTERNAL.

In ordinary pipeline barrier or VK_SUBPASS_EXTERNAL dependency it means either some work is dependent on a copy being finished, or some copy depending on some writes to be made first.

How host memory sync works is that Vulkan divides it into domains. There's host memory domain, and there's device memory domain. If changes to a memory are not transitioned from one domain to the other before reading, it might be so the writes will not be visible on the other domain.

That's where VK_PIPELINE_STAGE_HOST_BIT (plus related access mask) comes in. It instructs the driver to do this potentialy expensive and disruptive domain transfer.

Now there's a rare implicit Vulkan behavior. The host write ordering guarantee. If you write from host to memory, and then vkQueueSubmit something, you don't need to do anything. Operations in that submission automatically see the host writes, and you won't use VK_PIPELINE_STAGE_HOST_BIT.

This doesn't work other way around though. If you want to read something on host that was written on device, you always need to use dstStage = VK_PIPELINE_STAGE_HOST_BIT (followed by a fence or such), otherwisely the writes might not be visible on the host domain when you try to read them.

Third option is when you submit first, and then you write something on the host. Previously this could happen with Events (but that was subsequently banned), and now it can happen when using extended semaphores. In that case you need to use srcStage = VK_PIPELINE_STAGE_HOST_BIT in addition to that host-device semaphore, otherwise your host writes might not be visible to the reads on that device.

Upvotes: 2

Simon Richter
Simon Richter

Reputation: 29586

Transfers can be inserted pretty much anywhere into a pipeline to move data between buffers. These are normal commands with dependencies that can be queued, and typically have dedicated hardware so other shaders can run in parallel. The "clear" commands use a constant value as the source for the copy operation, but still use the DMA engine, so they are in the same group.

These bits largely correspond to the resources that will be used. In the case of VkSubpassDependency, they define which resources need to be synchronized for the pipeline to deliver correct results, so if you use a Copy or Clear operation in the first subpass, you need to specify VK_PIPELINE_STAGE_TRANSFER_BIT in the srcStageMask to denote that the first subpass will use the DMA engine, so handover to the second subpass requires the write caches of the DMA engine to be flushed. Omitting this bit would allow an invalid optimization, so it is diallowed to use a Copy or Clear command then.

The "All Commands", "Top of Pipe" and "Bottom of Pipe" are just shorthands to save on negotiation during pipeline setup.

Upvotes: 0

Related Questions