Depth testing

Since Coordinate Systems a depth buffer has been quietly keeping the far side of every cube behind its near side. We created it, cleared it every frame and otherwise ignored it. This chapter looks at what it actually stores, how the test that uses it works, and why the numbers in it are less evenly spread than you might expect.

The depth buffer (or z-buffer) is an image the same size as the render target, holding one value per pixel instead of a colour: how far away the nearest surface drawn there so far is. Every 3D lesson calls window_create_depth right after window_create, and it lives in window.asm:

; A depth buffer the size of the window: 24 bits of depth, 8 of stencil.
align 16
window_create_depth proc
    local desc:D3D11_TEXTURE2D_DESC
    invoke mem_zero, ADDR desc, sizeof D3D11_TEXTURE2D_DESC
    mov eax, scr_width
    mov desc._Width, eax
    mov eax, scr_height
    mov desc.Height, eax
    mov desc.MipLevels, 1
    mov desc.ArraySize, 1
    mov desc.Format, DXGI_FORMAT_D24_UNORM_S8_UINT
    mov desc.SampleDesc.Count, 1
    mov desc.BindFlags, D3D11_BIND_DEPTH_STENCIL
    vcall device, ID3D11Device_CreateTexture2D, ADDR desc, 0, ADDR depth_texture
    vcall device, ID3D11Device_CreateDepthStencilView, depth_texture, 0, ADDR depth_dsv
    ret
window_create_depth endp

In OpenGL the windowing library makes this buffer for you; in Direct3D it is an ordinary texture that we create ourselves. DXGI_FORMAT_D24_UNORM_S8_UINT packs 24 bits of depth, stored as a fixed-point fraction from 0 to 1, with 8 bits of stencil that the next chapter uses. 24 bits is the common choice; DXGI_FORMAT_D32_FLOAT gives a 32-bit float instead. To be drawn into, the texture gets a depth-stencil view, which window_begin binds beside the colour target at the start of each frame and clears to 1.0, the farthest possible depth:

window_begin proc color:qword
    vcall context, ID3D11DeviceContext_OMSetRenderTargets, 1, ADDR backbuffer_rtv, depth_dsv
    vcall context, ID3D11DeviceContext_RSSetViewports, 1, ADDR viewport
    vcall context, ID3D11DeviceContext_ClearRenderTargetView, backbuffer_rtv, color
    cmp depth_dsv, 0
    je no_depth
        ; The depth is a float: the fourth argument, counting `this`, so it travels in xmm3.
        movf xmm3, 1.0
        vcall context, ID3D11DeviceContext_ClearDepthStencilView, depth_dsv, D3D11_CLEAR_DEPTH or D3D11_CLEAR_STENCIL, 0, 0
    no_depth:
    ret
window_begin endp

Forgetting the clear has the effect you would guess: last frame's depths stay, and this frame's pixels are tested against surfaces that are no longer there.

With a depth buffer bound, the depth test runs for every pixel a triangle covers. The pixel's own depth is compared against the value stored there; if the test passes, the pixel's colour is written and its depth replaces the stored one, and if it fails, the pixel is thrown away. The test happens in the output merger, the last stage of the pipeline, so conceptually after the pixel shader has run.

Unlike OpenGL, where depth testing is off until glEnable(GL_DEPTH_TEST), Direct3D has it on by default. What controls it is a depth-stencil state, an immutable object like the rasterizer state, set with OMSetDepthStencilState. Our depth testing lesson never creates one: binding none (or null) gives the defaults, which are depth testing on, depth writes on, and a LESS comparison. What it does need is a depth-stencil view bound with OMSetRenderTargets; with null there, nothing is tested at all.

A depth-stencil state is made from a D3D11_DEPTH_STENCIL_DESC. Its depth part has three fields, here as the stencil lesson sets them:

mov eax, dword ptr depth
mov desc.DepthEnable, eax
mov desc.DepthWriteMask, D3D11_DEPTH_WRITE_MASK_ALL
mov desc.DepthFunc, D3D11_COMPARISON_LESS

DepthEnable turns the test on or off. DepthWriteMask says whether passing pixels update the buffer: D3D11_DEPTH_WRITE_MASK_ZERO keeps testing but stops writing, a read-only depth buffer, which is what glDepthMask(GL_FALSE) does. That is useful for things like particles, which should be hidden by walls without hiding each other. DepthFunc is the comparison.

Depth test function

The comparison decides which pixels survive. It is written "new depth function stored depth", and Direct3D offers the same eight as OpenGL's glDepthFunc:

LESS is "keep whatever is nearer", which is what we want nearly all the time. To see what the test buys us, imagine a state with ALWAYS:

mov desc.DepthEnable, 1
mov desc.DepthWriteMask, D3D11_DEPTH_WRITE_MASK_ALL
mov desc.DepthFunc, D3D11_COMPARISON_ALWAYS     ; every pixel passes

Every pixel passes, so whatever is drawn last ends up on top, which is the same as having no depth test. Our scene draws the floor first and then the two cubes, so with ALWAYS the cubes would still cover the floor. Within each cube, though, the faces would cover each other in the order they appear in the vertex array rather than by distance, and some far faces would show through near ones. Try it in the exercises.

The drawing itself is short. From this chapter on, draw_vertices in transforms.asm draws any buffer of position-normal-uv vertices with a model matrix (it is draw_cube with the buffer and the vertex count as arguments), and bind_texture in gpu.asm puts one texture in one register:

; the floor, then two cubes on it
invoke bind_texture, 0, floor_texture
invoke m4_identity, ADDR model
invoke draw_vertices, transforms_buffer, ADDR t, plane_vb, 6, ADDR model
invoke bind_texture, 0, cube_texture
invoke m4_translate, ADDR model, ADDR cube_a
invoke draw_vertices, transforms_buffer, ADDR t, cube_vb, 36, ADDR model
invoke m4_translate, ADDR model, ADDR cube_b
invoke draw_vertices, transforms_buffer, ADDR t, cube_vb, 36, ADDR model

Depth value precision

The depth buffer holds values from 0 to 1, but the distances in our scene run from the near plane to the far plane, here 0.1 to 100. Something maps one onto the other. The obvious mapping would be linear:

where \(z\) is the distance from the camera. Equal steps in distance would then be equal steps in depth, as in LearnOpenGL's graph (drawn for near = 1 and far = 50):

Depth buffers practically never work that way. A perspective projection divides by distance (that is what makes far things small), and the depth it produces is divided too. What ends up in the buffer is proportional to \(1/z\):

This is exactly the depth Direct3D stores. We can check it against our projection matrix, m4_perspective in math.asm. A matrix there is sixteen floats row by row, so byte offset 40 is element 10 (row 2, column 2), 44 is element 11 and 56 is element 14 (row 3, column 2):

movss real4 ptr [rcx + 40], xmm0        ; far / (near - far)
movss xmm0, real4 ptr z_near
mulss xmm0, real4 ptr z_far
divss xmm0, xmm2
movss real4 ptr [rcx + 44], xmm0        ; near * far / (near - far)
mov dword ptr [rcx + 56], 0BF800000h    ; -1

(0BF800000h is the bit pattern of the float -1.0: storing it as an integer saves a constant.) For a point at distance \(z\) in front of the camera (view-space z equal to \(-z\), since the camera looks down -z), row 2 gives clip-space z and row 3 gives w:

which is 0 at the near plane, 1 at the far plane, and, with a little algebra, the same as the formula above. OpenGL's projection produces the same curve over -1 to 1 and then maps it to 0 to 1 when writing the buffer; Direct3D's lands in 0 to 1 directly, which is one reason the D3D version of the formulas below is slightly simpler.

LearnOpenGL's graph shows how lopsided this is. With our near of 0.1 and far of 100 it is more extreme still. Half of the whole range, 0 to 0.5, is used up by the first tenth of a unit past the near plane (depth reaches 0.5 at a distance of about 0.2). One unit away the depth is already 0.90, two units away 0.95, ten units away 0.99. Everything from ten units to a hundred shares the last one percent.

That sounds wasteful, but it is roughly what we want: things close to the camera are big on screen and need fine depth distinctions, while far things are small and need fewer. The catch is that the precision far away can run out, which we come back to with z-fighting.

Visualizing the depth buffer

The lesson shows the depth directly. In a pixel shader, the SV_Position input is not a clip-space position any more; it is the pixel itself: x and y in pixels from the top-left corner (OpenGL's gl_FragCoord counts from the bottom-left), and z the depth about to be tested. So the shader can return that z as a grey:

float4 ps_main(vs_out p) : SV_Target
{
    // In a pixel shader SV_Position is the pixel: x and y in pixels, z the
    // depth about to be tested. GL calls it gl_FragCoord.
    float depth = p.position.z;
    if (mode == 1) return float4(depth.xxx, 1.0);
    if (mode == 2) return float4((linear_depth(depth) / 10.0).xxx, 1.0);
    return albedo.Sample(linear_wrap, p.uv);
}

The mode comes from a small constant buffer in b1, together with the near and far planes:

cbuffer view_mode : register(b1)
{
    int mode;
    float near_plane;
    float far_plane;
};

; Mirrors `cbuffer view_mode` in depth_testing.hlsl.
view_constants struct
    mode dword ?            ; 0 textured, 1 raw depth, 2 linear depth
    near_plane real4 ?
    far_plane real4 ?
    pad real4 ?
view_constants ends

v view_constants <0, 0.1, 100.0, 0.0>

The padding makes the struct 16 bytes, one HLSL register, as constant buffers require. Here the floats can be written straight into the initializer, since a variable in .data is data and MASM converts the literals itself. Tab steps through the three modes, with window_mode from window.asm (the number keys pick one directly); the program starts in mode 2, the linear view, and Tab goes on to the textured scene and then the raw depth:

invoke window_mode_default, 2

invoke window_mode, 3
mov v.mode, eax

Here is the raw depth, mode 1, beside the ordinary scene:

Nearly everything is white. Given the numbers in the previous section, that is no surprise: the camera starts about five units from the cubes, where the depth is around 0.98. Only the bit of floor right in front of the camera is a shade darker. Fly very close to a cube and it will turn grey and then black, within a fraction of a unit. (The background is the clear colour, since nothing was drawn there.)

To see real distances we undo the projection. Solving the depth formula for \(z\):

which in HLSL is

// The eye-space distance a 0..1 depth came from: the projection, undone.
float linear_depth(float depth)
{
    return near_plane * far_plane / (far_plane - depth * (far_plane - near_plane));
}

LearnOpenGL's version first turns the depth back into a -1 to 1 value, z * 2.0 - 1.0, because that is the range OpenGL's projection produced. Ours skips that step, since Direct3D's projection already produced 0 to 1. Copying the GLSL formula as it is would give wrong distances, a typical small trap when porting.

The result is a distance in world units, from 0.1 to 100. The shader divides it by 10 to fit a range that suits this small scene: black at the camera, white from ten units on. LearnOpenGL divides by the far plane, which leaves its picture nearly black.

Now the floor fades evenly from dark near the camera to white in the distance, and the cubes, about five units away, are a middle grey. Move around and the greys change in proportion to how far you move. The full program is here, and its shader here.

Z-fighting

When two surfaces lie in the same plane, or so close that the depth buffer cannot tell them apart, neither one consistently wins the depth test. Which one shows flips from pixel to pixel and from frame to frame as the camera moves, in jagged stripes. This is called z-fighting.

Our scene has a case of it built in. The cubes are 1 unit tall and centred at y = 0, so their bottoms are at y = -0.5, exactly the height of the floor. From outside you never see a cube's bottom face, but you can from inside. The Face culling lesson starts out removing the cubes' front faces, which opens them up, and its first frame shows the fight on their floors:

The stripes at the bottom of each cube are the metal floor and the cube's own bottom face taking turns. Z-fighting gets worse with distance, because, as we saw, precision drops off the farther a surface is.

Prevent z-fighting

It cannot be removed entirely, but a few habits keep it away:

• Do not put surfaces on top of each other. Lifting our cubes by a hundredth of a unit would end the fight here and nobody would see the gap. This is the most effective fix, but it has to be done object by object.
• Push the near plane out. Precision is concentrated just past the near plane, so moving it from 0.1 to, say, 0.5 spreads much more of it over the rest of the scene. Too far and nearby objects get clipped, so it is a matter of tuning per scene.
• Use more bits. DXGI_FORMAT_D32_FLOAT (or D32_FLOAT_S8X24_UINT with stencil) has more precision than a 24-bit buffer, at some cost in memory and bandwidth.

Additional resources

• Configuring depth-stencil functionality: Microsoft's guide to depth-stencil textures, views and states.
• Depth Precision Visualized: Nathan Reed's illustrated explanation of depth precision and reversed-Z.
• OpenGL Projection Matrix: Song Ho Ahn's derivation of the projection, including where the \(1/z\) comes from (for OpenGL's -1 to 1 range).

Exercises

1. Create a depth-stencil state with DepthFunc set to D3D11_COMPARISON_ALWAYS, bind it with OMSetDepthStencilState before drawing, and look at the cubes from different sides. Which faces show through, and why those?
2. Change the near plane in camera_projection (the movf xmm3, 0.1 in camera.asm) and in near_plane from 0.1 to 1.0, then look at the raw depth view again. How close can you get to a cube before it is clipped?
3. Get rid of the z-fighting in the face culling lesson by lifting the cubes slightly. How small a lift still works when you fly far away?