How do I visualize custom classes in the debugger in a visual studio cross-platform CMake project?

Question

I'm currently getting into cross-platform development using visual studio and CMake. Before, I would only create native Visual Studio applications and I could easily create debug visualizations of custom classes using natvis files. However, every way I've tried to add a natvis file to my CMake projects has failed. For instance I've tried to adapt this answer by user12322620, but to no avail.

This is my directory structure:

dbe_prj
├── CMakeLists.txt
├── CMakePresets.json
├── _out
│   └── build ...
├── _dbe_test
│   ├── CMakeLists.txt
│   └── source_files
└── _dbe_lib
    ├── CMakeLists.txt
    └── source_files

My top level CMakeLists.txt file:

cmake_minimum_required (VERSION 3.8)

project ("dbe_prj")

add_subdirectory ("dbe_lib")
add_subdirectory ("dbe_test")

My dbe_lib CMakeLists.txt file:

cmake_minimum_required (VERSION 3.8)

add_library(dbe_lib STATIC "dbe_lib.cpp" "dbe_lib.h")

target_include_directories(dbe_lib PUBLIC "<include directories>")

My dbe_test CMakeLists.txt file:

cmake_minimum_required (VERSION 3.8)

add_executable(dbe_test "dbe_lib.cpp")

target_include_directories(dbe_test PRIVATE "../dbe_lib")
target_link_libraries(dbe_test PRIVATE dbe_lib)

I'm currently targeting both windows as well as WSL2 Ubuntu and RPi through ssh. My visual studio version is 2019, but I've also tried it on Visual Studio 2022 RC3.

Is it possible to add a nice debug visualizer file like a natvis file to a CMake project like this? Is there a cross-platform solution for this? I would guess it's possible since STL classes like std::vector have nice debug visualizers regardless of the target platform.