CODEWITHSUNDEEP
iosscenekitarkitmetal
Matt Bierner
Matt Bierner

Reputation: 65303

Integrating a metal depth buffer with scenekit rendering

I'm using Metal to render a scene with a z buffer and now need to integrate this z-buffer into SceneKit's rendering. However I can't figure out how to get SceneKit to use this depth better correctly and am not even 100% sure what format SceneKit expects it's z-buffers to be in

Base on this question, my understanding was that SceneKit uses a reverse logarithmic z-buffer in the range of 1 (near) to 0 (far). However I can't get this working and objects I draw with SceneKit don't properly respect the depth buffer: they are either always showing or always hidden

First, here's how the generate a z-buffer texture in a Metal render pass:

struct FragmentOut {
    float4 color [[color(0)]];
    float depth [[depth(any)]];
};

fragment FragmentOut metalRenderFragment(const InOut in [[ stage_in ]]) {
    FragmentOut out;
    out.depth = 0; // 0 is far with reverse z buffer

    ...
    
    float cameraSpaceZ = ...; // Computed in shader

    // There constants are taken from SceneKit's camera and inlined here
    const float zNear = 0.0010000000474974513; 
    const float zFar = 1000.0;

    float logDepth = log(z / zNear) / log(zFar / zNear);

    out.depth = 1.0 - logDepth; // Reverse the depth for scenekit

    return out;
}

Then to integrate the depth buffer into SceneKit, I render a full screen quad in scenekit with a SCNProgram that uses the depth texture generated in the previous step:

fragment FragmentOut sceneKitFullScreenQuadFragment(const InOut in [[ stage_in ]],
                                depth2d<float, access::sample> depthTexture [[texture(1)]])
{
    constexpr sampler sampler(filter::linear);
    
    const float depth = depthTexture.sample(sampler, in.uv);
    return {
        .color = float4(0),
        .depth = depth,
    };
}

So two questions:

  1. What format does SceneKit use for its z-buffer? Is it a reversed logarithmic z-buffer?

  2. What am I doing wrong in generating the z-buffer values for SceneKit?

Upvotes: 2

Views: 397

Answers (2)

Matt Bierner
Matt Bierner

Reputation: 65303

Andy Jazz's answer helped but I still found the links confusing. Here's what ultimately worked for me (although there are possibly other ways to do this):

When generating the depth map (this would be inside the the metal shader in my original example) pass in SceneKit's projection transform matrix and use this to transform the depth value:

// In a metal shader generating the depth map

// The z distance from the camera, e.g. if the object
// at the current position is 5 units away, this would be 5.
const float z = ...;

// The camera points along the -z axis, so transform the -z position
// with SceneKit's projection matrix (you can get this from SCNCamera)
const float4 depthPos = (sceneKitState.projectionTransform * float4(0, 0, -z, 1));

// Then do perspective division to get the final depth value
out.depth = depthPos.z / depthPos.w;

Then inside of the SceneKit shader, simply write out the depth, taking into account usesReverseZ:

// In a scenekit, full screen quad shader

const float depth = depthTexture.sample(sampler, in.uv);
return {
    .color = float4(0),
    .depth = 1.0 - depth,
};

❗️ The above assumes you are using sceneView.usesReverseZ = true (the default). If you are using usesReverseZ = false, simply do .depth = depth instead

Upvotes: 0

Andy Jazz
Andy Jazz

Reputation: 58113

SceneKit uses a reverse logarithmic Z-Buffer. This post and this post show you how to get a normalized linear mapping space [0...1]. You need the opposite formula.

Also, you can toggle the value from reverseZ to directZ this way:

let sceneView = self.view as! SCNView
sceneView.usesReverseZ = true             // default

Upvotes: 2

Related Questions