Vulkan image layout not transitioning in pipeline barrier

Question

I ported most of Introduction to Vulkan - Tutorial 02 from C++ into a single Rust function to keep it simple. The function calls Vulkan through Rust FFI provided by ash.

I'm having an issue getting the pipeline barriers and semaphores to function correctly. As far as I can tell, this code seems to create the same validation debug log info as the C++ code.

When I run the C++ code with validation layers enabled, vkQueueSubmit is successful. When I run the Rust function below (with validation layers enabled), queue_submit fails and I receive

Cannot submit cmd buffer using image (0x6) [sub-resource: aspectMask 0x1 array layer 0, mip level 0], with layout VK_IMAGE_LAYOUT_UNDEFINED when first use is VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL.

...which implies to me that there the image layout transition isn't taking place, so there may be something wrong with how I configure/submit my pipeline barriers.

#[macro_use]
extern crate ash;
extern crate kernel32;
extern crate winit;
use std::default::Default;
use std::ffi::{CStr, CString};
use std::ptr;
use ash::vk;
use ash::Entry;
use ash::extensions::{DebugReport, Surface, Swapchain, Win32Surface};
use ash::version::{DeviceV1_0, EntryV1_0, InstanceV1_0};
use winit::os::windows::WindowExt;

unsafe extern "system" fn vulkan_debug_callback(
    _: vk::DebugReportFlagsEXT,
    _: vk::DebugReportObjectTypeEXT,
    _: vk::uint64_t,
    _: vk::size_t,
    _: vk::int32_t,
    _: *const vk::c_char,
    p_message: *const vk::c_char,
    _: *mut vk::c_void,
) -> u32 {
    println!("{:?}", CStr::from_ptr(p_message));
    1
}

fn main() {
    let mut events_loop = winit::EventsLoop::new();
    let window_width = 1024;
    let window_height = 768;
    let window = winit::WindowBuilder::new()
        .with_title("Example")
        .with_dimensions(window_width, window_height)
        .build(&events_loop)
        .unwrap();

    unsafe {
        let entry = Entry::new().unwrap();
        let name = CString::new("Example").unwrap();
        let name_raw = name.as_ptr();

        let app_info = [
            vk::ApplicationInfo {
                s_type: vk::StructureType::ApplicationInfo,
                p_next: ptr::null(),
                p_application_name: name_raw,
                application_version: 0,
                p_engine_name: name_raw,
                engine_version: 0,
                api_version: vk_make_version!(1, 0, 36),
            },
        ];

        let layer_names = [CString::new("VK_LAYER_LUNARG_standard_validation").unwrap()];
        let layer_names_raw: Vec<*const i8> = layer_names.iter().map(|x| x.as_ptr()).collect();

        let extension_names_raw = vec![
            Surface::name().as_ptr(),
            Win32Surface::name().as_ptr(),
            DebugReport::name().as_ptr(),
        ];

        let create_info = vk::InstanceCreateInfo {
            s_type: vk::StructureType::InstanceCreateInfo,
            p_next: ptr::null(),
            flags: Default::default(),
            p_application_info: app_info.as_ptr(),
            pp_enabled_layer_names: layer_names_raw.as_ptr(),
            enabled_layer_count: layer_names_raw.len() as u32,
            pp_enabled_extension_names: extension_names_raw.as_ptr(),
            enabled_extension_count: extension_names_raw.len() as u32,
        };

        let instance = entry.create_instance(&create_info, None).unwrap();

        let debug_info = vk::DebugReportCallbackCreateInfoEXT {
            s_type: vk::StructureType::DebugReportCallbackCreateInfoExt,
            p_next: ptr::null(),
            flags: vk::DEBUG_REPORT_ERROR_BIT_EXT | vk::DEBUG_REPORT_WARNING_BIT_EXT |
                vk::DEBUG_REPORT_PERFORMANCE_WARNING_BIT_EXT |
                vk::DEBUG_REPORT_INFORMATION_BIT_EXT |
                vk::DEBUG_REPORT_DEBUG_BIT_EXT,
            pfn_callback: vulkan_debug_callback,
            p_user_data: ptr::null_mut(),
        };

        let debug_report_loader = DebugReport::new(&entry, &instance).unwrap();

        let debug_callback = debug_report_loader
            .create_debug_report_callback_ext(&debug_info, None)
            .unwrap();

        let surface_loader = Surface::new(&entry, &instance).unwrap();
        let win32_create_info = vk::Win32SurfaceCreateInfoKHR {
            s_type: vk::StructureType::Win32SurfaceCreateInfoKhr,
            p_next: ptr::null(),
            flags: Default::default(),
            hinstance: kernel32::GetModuleHandleW(ptr::null()) as *mut _,
            hwnd: window.get_hwnd() as *mut _,
        };

        let surface = Win32Surface::new(&entry, &instance)
            .unwrap()
            .create_win32_surface_khr(&win32_create_info, None)
            .unwrap();

        let (physical_device, graphics_queue_family_index, present_queue_family_index) = instance
            .enumerate_physical_devices()
            .unwrap()
            .iter()
            .filter_map(|&p| {
                let candidates =
                    instance
                        .get_physical_device_queue_family_properties(p)
                        .iter()
                        .enumerate()
                        .filter_map(|(index, info)| {
                            let has_graphics = info.queue_flags.subset(vk::QUEUE_GRAPHICS_BIT);
                            let has_present = surface_loader
                                .get_physical_device_surface_support_khr(p, index as u32, surface);
                            if has_graphics || has_present {
                                Some((index as u32, has_graphics, has_present))
                            } else {
                                None
                            }
                        })
                        .collect::<Vec<(u32, bool, bool)>>();
                match candidates.iter().find(|&x| x.1 && x.2) {
                    Some(ref both) => Some((p, both.0, both.0)),
                    None => match candidates.iter().find(|&x| x.1) {
                        Some(ref graphics) => match candidates.iter().find(|&x| x.2) {
                            Some(ref present) => Some((p, graphics.0, present.0)),
                            None => None,
                        },
                        None => None,
                    },
                }
            })
            .nth(0)
            .unwrap();

        let device_extension_names_raw = [Swapchain::name().as_ptr()];
        let queue_priorities = [1.0];
        let queue_create_infos = [
            vk::DeviceQueueCreateInfo {
                s_type: vk::StructureType::DeviceQueueCreateInfo,
                p_next: ptr::null(),
                flags: Default::default(),
                queue_family_index: graphics_queue_family_index,
                p_queue_priorities: queue_priorities.as_ptr(),
                queue_count: queue_priorities.len() as u32,
            },
        ];

        let device_create_info = vk::DeviceCreateInfo {
            s_type: vk::StructureType::DeviceCreateInfo,
            p_next: ptr::null(),
            flags: Default::default(),
            queue_create_info_count: 1 as u32,
            p_queue_create_infos: queue_create_infos.as_ptr(),
            enabled_layer_count: 0,
            pp_enabled_layer_names: ptr::null(),
            enabled_extension_count: device_extension_names_raw.len() as u32,
            pp_enabled_extension_names: device_extension_names_raw.as_ptr(),
            p_enabled_features: ptr::null(),
        };
        let device = instance
            .create_device(physical_device, &device_create_info, None)
            .unwrap();

        let present_queue = device.get_device_queue(present_queue_family_index, 0);

        let image_available_semaphore_create_info = vk::SemaphoreCreateInfo {
            s_type: vk::StructureType::SemaphoreCreateInfo,
            p_next: ptr::null(),
            flags: Default::default(),
        };
        let image_available_semaphore = device
            .create_semaphore(&image_available_semaphore_create_info, None)
            .unwrap();

        let render_complete_semaphore_create_info = vk::SemaphoreCreateInfo {
            s_type: vk::StructureType::SemaphoreCreateInfo,
            p_next: ptr::null(),
            flags: Default::default(),
        };
        let render_complete_semaphore = device
            .create_semaphore(&render_complete_semaphore_create_info, None)
            .unwrap();
        let swapchain_loader = Swapchain::new(&instance, &device).unwrap();

        let pool_create_info = vk::CommandPoolCreateInfo {
            s_type: vk::StructureType::CommandPoolCreateInfo,
            p_next: ptr::null(),
            flags: vk::CommandPoolCreateFlags::empty(),
            queue_family_index: present_queue_family_index,
        };

        device.device_wait_idle().unwrap();
        let surface_capabilities = surface_loader
            .get_physical_device_surface_capabilities_khr(physical_device, surface)
            .unwrap();

        let surface_formats = surface_loader
            .get_physical_device_surface_formats_khr(physical_device, surface)
            .unwrap();

        let surface_format =
            if surface_formats.len() == 1 && surface_formats[0].format == vk::Format::Undefined {
                vk::SurfaceFormatKHR {
                    format: vk::Format::B8g8r8Unorm,
                    color_space: surface_formats[0].color_space,
                }
            } else {
                match (surface_formats)
                    .iter()
                    .find(|&sf| sf.format == vk::Format::B8g8r8Unorm)
                {
                    Some(sf) => sf.clone(),
                    None => surface_formats[0].clone(),
                }
            };

        let image_extent = match surface_capabilities.current_extent.width {
            std::u32::MAX => vk::Extent2D {
                width: window_width,
                height: window_height,
            },
            _ => surface_capabilities.current_extent.clone(),
        };

        let mut image_count = surface_capabilities.min_image_count + 1;
        if surface_capabilities.max_image_count > 0 &&
            image_count > surface_capabilities.max_image_count
        {
            image_count = surface_capabilities.max_image_count;
        }

        image_count = image_count;

        let pre_transform = if surface_capabilities
            .supported_transforms
            .subset(vk::SURFACE_TRANSFORM_IDENTITY_BIT_KHR)
        {
            vk::SURFACE_TRANSFORM_IDENTITY_BIT_KHR
        } else {
            surface_capabilities.current_transform
        };

        let present_modes = surface_loader
            .get_physical_device_surface_present_modes_khr(physical_device, surface)
            .unwrap();

        let present_mode = present_modes
            .iter()
            .cloned()
            .find(|&mode| mode == vk::PresentModeKHR::Fifo)
            .unwrap_or(vk::PresentModeKHR::Fifo);

        let swapchain_create_info = vk::SwapchainCreateInfoKHR {
            s_type: vk::StructureType::SwapchainCreateInfoKhr,
            p_next: ptr::null(),
            flags: Default::default(),
            surface: surface,
            min_image_count: image_count,
            image_color_space: surface_format.color_space,
            image_format: surface_format.format,
            image_extent: image_extent,
            image_usage: vk::IMAGE_USAGE_COLOR_ATTACHMENT_BIT | vk::IMAGE_USAGE_TRANSFER_DST_BIT,
            image_sharing_mode: vk::SharingMode::Exclusive,
            pre_transform,
            composite_alpha: vk::COMPOSITE_ALPHA_OPAQUE_BIT_KHR,
            present_mode,
            clipped: 1,
            old_swapchain: vk::SwapchainKHR::null(),
            image_array_layers: 1,
            p_queue_family_indices: ptr::null(),
            queue_family_index_count: 0,
        };

        let swapchain = swapchain_loader
            .create_swapchain_khr(&swapchain_create_info, None)
            .unwrap();

        let command_pool = device.create_command_pool(&pool_create_info, None).unwrap();

        let command_buffer_allocate_info = vk::CommandBufferAllocateInfo {
            s_type: vk::StructureType::CommandBufferAllocateInfo,
            p_next: ptr::null(),
            command_buffer_count: image_count,
            command_pool: command_pool,
            level: vk::CommandBufferLevel::Primary,
        };

        let command_buffers = device
            .allocate_command_buffers(&command_buffer_allocate_info)
            .unwrap();

        let swapchain_images = swapchain_loader
            .get_swapchain_images_khr(swapchain)
            .unwrap();

        let command_buffer_begin_info = vk::CommandBufferBeginInfo {
            s_type: vk::StructureType::CommandBufferBeginInfo,
            p_next: ptr::null(),
            p_inheritance_info: ptr::null(),
            flags: vk::COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT,
        };

        let subresource_range = vk::ImageSubresourceRange {
            aspect_mask: vk::IMAGE_ASPECT_COLOR_BIT,
            base_mip_level: 0,
            level_count: 1,
            base_array_layer: 0,
            layer_count: 1,
        };

        for (index, swapchain_image) in swapchain_images.iter().enumerate() {
            let barrier_from_present_to_clear = vk::ImageMemoryBarrier {
                s_type: vk::StructureType::ImageMemoryBarrier,
                p_next: ptr::null(),
                src_access_mask: vk::ACCESS_MEMORY_READ_BIT,
                dst_access_mask: vk::ACCESS_TRANSFER_WRITE_BIT,
                old_layout: vk::ImageLayout::Undefined,
                new_layout: vk::ImageLayout::TransferDstOptimal,
                src_queue_family_index: vk::VK_QUEUE_FAMILY_IGNORED,
                dst_queue_family_index: vk::VK_QUEUE_FAMILY_IGNORED,
                image: *swapchain_image,
                subresource_range: subresource_range.clone(),
            };

            let barrier_from_clear_to_present = vk::ImageMemoryBarrier {
                s_type: vk::StructureType::ImageMemoryBarrier,
                p_next: ptr::null(),
                src_access_mask: vk::ACCESS_TRANSFER_WRITE_BIT,
                dst_access_mask: vk::ACCESS_MEMORY_READ_BIT,
                old_layout: vk::ImageLayout::TransferDstOptimal,
                new_layout: vk::ImageLayout::PresentSrcKhr,
                src_queue_family_index: vk::VK_QUEUE_FAMILY_IGNORED,
                dst_queue_family_index: vk::VK_QUEUE_FAMILY_IGNORED,
                image: *swapchain_image,
                subresource_range: subresource_range.clone(),
            };

            let command_buffer = command_buffers[index];

            device
                .begin_command_buffer(command_buffer, &command_buffer_begin_info)
                .unwrap();

            device.cmd_pipeline_barrier(
                command_buffer,
                vk::PIPELINE_STAGE_TRANSFER_BIT,
                vk::PIPELINE_STAGE_TRANSFER_BIT,
                vk::DependencyFlags::empty(),
                &[],
                &[],
                &[barrier_from_present_to_clear],
            );

            let clear_color = vk::ClearColorValue::new_float32([1.0, index as f32 / 4.0, 0.2, 0.0]);

            device.cmd_clear_color_image(
                command_buffer,
                *swapchain_image,
                vk::ImageLayout::TransferDstOptimal,
                &clear_color,
                &[subresource_range.clone()],
            );

            device.cmd_pipeline_barrier(
                command_buffer,
                vk::PIPELINE_STAGE_TRANSFER_BIT,
                vk::PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,
                vk::DependencyFlags::empty(),
                &[],
                &[],
                &[barrier_from_clear_to_present],
            );

            device.end_command_buffer(command_buffer).unwrap();
        }

        events_loop.run_forever(|event| match event {
            winit::Event::WindowEvent {
                event: winit::WindowEvent::Closed,
                ..
            } => {
                device.device_wait_idle().unwrap();
                device.destroy_semaphore(image_available_semaphore, None);
                device.destroy_semaphore(render_complete_semaphore, None);

                device.destroy_command_pool(command_pool, None);
                swapchain_loader.destroy_swapchain_khr(swapchain, None);
                device.destroy_device(None);
                surface_loader.destroy_surface_khr(surface, None);
                debug_report_loader.destroy_debug_report_callback_ext(debug_callback, None);
                instance.destroy_instance(None);
                winit::ControlFlow::Break
            }
            _ => {
                let image_index = swapchain_loader
                    .acquire_next_image_khr(
                        swapchain,
                        std::u64::MAX,
                        image_available_semaphore,
                        vk::Fence::null(),
                    )
                    .unwrap();

                let present_semaphores = [image_available_semaphore];
                let render_semaphores = [render_complete_semaphore];
                let command_buffer = [command_buffers[image_index as usize]];

                let submit_info = [
                    vk::SubmitInfo {
                        s_type: vk::StructureType::SubmitInfo,
                        p_next: ptr::null(),
                        wait_semaphore_count: present_semaphores.len() as u32,
                        p_wait_semaphores: present_semaphores.as_ptr(),
                        p_wait_dst_stage_mask: &vk::PIPELINE_STAGE_TRANSFER_BIT,
                        command_buffer_count: command_buffer.len() as u32,
                        p_command_buffers: command_buffer.as_ptr(),
                        signal_semaphore_count: render_semaphores.len() as u32,
                        p_signal_semaphores: render_semaphores.as_ptr(),
                    },
                ];

                device
                    .queue_submit(present_queue, &submit_info, vk::Fence::null())
                    .unwrap();

                let present_info = vk::PresentInfoKHR {
                    s_type: vk::StructureType::PresentInfoKhr,
                    p_next: ptr::null(),
                    wait_semaphore_count: render_semaphores.len() as u32,
                    p_wait_semaphores: render_semaphores.as_ptr(),
                    swapchain_count: 1,
                    p_swapchains: &swapchain,
                    p_image_indices: &image_index,
                    p_results: ptr::null_mut(),
                };

                swapchain_loader
                    .queue_present_khr(present_queue, &present_info)
                    .unwrap();

                winit::ControlFlow::Continue
            }
        });
    }
}

It's possible that some of the values could be deallocated too early, or that I'm passing pointers incorrectly, but I can't see where (I'm relatively new to Rust).

Sorry for the code length, I'm not sure how to simplify this example further. The interesting parts are at the bottom where I submit the pipeline barriers and semaphores. The Vulkan SDK I'm using is 1.0.57.0, ash is 0.18.4, winit is 0.7, kernel32-sys is 0.2.2.

I would appreciate any suggestions to go about debugging this as well. I tried (and will keep trying) stepping into the validation layers to check the individual API calls and compare side-by-side, but it wasn't obvious where the difference is introduced.

Answer 1

VK_ERROR_VALIDATION_FAILED_EXT is often returned in the case when you return VK_TRUE from your debug callback. The specification advises against doing that:

The callback returns a VkBool32 that indicates to the calling layer the application’s desire to abort the call. A value of VK_TRUE indicates that the application wants to abort this call. If the application returns VK_FALSE, the command must not be aborted. Applications should always return VK_FALSE so that they see the same behavior with and without validation layers enabled.

The value VK_TRUE is intended only for the purposes of layer development. Its current use is for unit testing of the layers, which requires the command to be aborted before it reaches the GPU driver (to prevent crashes the test is not interested in). It is common mistake to use it in applications. As the quote says applications are supposed to always use VK_FALSE.

In your case the layer is behaving bit oddly, but what probably happened is:

1. Both your app and the tutorial produce a benign warning on vkCmdPipelineBarrier.
2. Because you return VK_TRUE the barrier is aborted and does not count.
3. vkCmdPipelineBarrier returns void (i.e. cannot return VK_ERROR_VALIDATION_FAILED_EXT) so you never learned it was actually aborted.
4. Because the layout transition was aborted you have image in wrong layout on vkQueueSubmit and so you get an appropriate error.