Shaders

In Hello Triangle we met shaders briefly: small programs that run on the GPU, one stage of the pipeline each, once per vertex or once per pixel. A shader is a function from inputs to outputs and nothing more. It can't call another shader or keep anything between one invocation and the next; everything it knows arrives through its inputs, and everything it does leaves through its outputs. This chapter looks at the language we write them in, HLSL, and at the two ways data gets into them from our program.

HLSL

The High-Level Shading Language is Direct3D's shading language, as GLSL is OpenGL's. Both are C-like, and both are built around vectors and matrices, which is most of what graphics code computes with. If you have read GLSL, HLSL will look familiar within minutes; the differences are mostly in how a shader is put together, not in the arithmetic.

A GLSL shader is a file that starts with a #version line, declares global in, out and uniform variables, and has a main function. An HLSL file is organised differently:

• There is no version line. The shader model is chosen when the shader is compiled, by the target we pass: vs_5_0 for a vertex shader, ps_5_0 for a pixel shader.
• There is no main. Each shader is an ordinary function with a name we choose, its entry point, and one file can hold several of them. Ours always hold a vertex shader called vs_main and a pixel shader called ps_main.
• Inputs and outputs are not global variables. They are the function's parameters and return value, and each is tagged with a semantic that says what it is.
• Values that are the same for a whole draw, GLSL's uniforms, are grouped into constant buffers, declared with cbuffer.

In outline, an HLSL file looks like this:

cbuffer name : register(b0)
{
    type constant_name;
};

struct vs_out
{
    float4 position : SV_Position;
    type out_name : SOME_SEMANTIC;
};

vs_out vs_main(type in_name : INPUT_SEMANTIC)
{
    vs_out o;
    // compute o.position and the other outputs from the inputs
    return o;
}

float4 ps_main(vs_out p) : SV_Target
{
    // compute the pixel's colour
}

Inputs to a vertex shader are also called vertex attributes. Direct3D 11 lets an input layout describe up to 32 of them, far more than any chapter here needs.

Types

HLSL has C's basic scalar types and a few of its own: bool, int, uint, float, double, and half, nominally a lower-precision float, which shader model 5 simply treats as a float. On top of those it has vectors and matrices, which we use all the time. Matrices wait for the Transformations chapter.

Vectors

A vector type is a scalar type followed by its number of components, from 1 to 4:

• float2, float3, float4: vectors of floats, the counterpart of GLSL's vec2 to vec4, and what we use almost always.
• int3, uint4, bool2 and so on for the other scalar types, GLSL's ivec, uvec and bvec.

The components are named .x, .y, .z and .w, or, when the vector holds a colour, .r, .g, .b and .a. (GLSL's third set, stpq, does not exist in HLSL.) Both sets support swizzling: naming up to four components in any order, repeated if we like, makes a new vector out of them.

float2 a = float2(0.5, 0.7);
float4 b = a.xyxx;            // (0.5, 0.7, 0.5, 0.5)
float3 c = b.zyw;             // (0.5, 0.7, 0.5)
float4 d = a.xxxx + c.yxzy;
float4 e = float4(a, 0.0, 1.0);
float4 f = float4(e.rgb, 1.0);

The two sets can't be mixed in one swizzle, and we can't name a component the vector doesn't have, such as .z of a float2. Constructors take any mixture of scalars and vectors as long as the component count comes out right, as the last two lines show. That is exactly how our vertex shaders extend a position to four components: float4(position, 1.0).

Ins and outs

Shaders are only useful as part of the pipeline, so each needs inputs and outputs that connect to the stages around it. GLSL connects them by name: an out in the vertex shader feeds the in of the same name and type in the fragment shader. HLSL connects them by semantic.

At the start, the vertex shader's inputs are filled from vertex buffers, and the semantics are the link: the input layout we create on the CPU names a semantic for each attribute, and the attribute goes to the vertex shader input with that semantic. We saw that in Hello Triangle, where POSITION joined the two. Semantics like POSITION and COLOR are names we pick; only the ones starting with SV_, the system values, mean something to the pipeline.

At the other end, the pixel shader must produce a colour. It returns a float4 marked SV_Target, which goes to the bound render target.

In between, the vertex shader's outputs travel through the rasterizer to the pixel shader's inputs, matched by semantic. A vertex shader always outputs SV_Position, the clip-space position the rasterizer needs, and can output more alongside it. The tidiest way to write this is one struct for the vertex shader's output, used again as the pixel shader's input, so the two sides cannot disagree. Here is this lesson's shader file, shaders.hlsl, apart from its constant buffer:

struct vs_in
{
    float3 position : POSITION;
    float3 color : COLOR;
};

// What the vertex shader hands on. Anything beyond SV_Position is
// interpolated across the triangle on its way to the pixel shader.
struct vs_out
{
    float4 position : SV_Position;
    float3 color : COLOR;
};

vs_out vs_main(vs_in v)
{
    vs_out o;
    o.position = float4(v.position, 1.0);
    o.color = v.color;
    return o;
}

The inputs are a struct too this time, vs_in, with two attributes: a position and a colour. The vertex shader copies the colour through unchanged into its output, under the semantic COLOR, and the pixel shader receives it:

float4 ps_main(vs_out p) : SV_Target
{
    return float4(p.color * tint.rgb, 1.0);
}

Ignore tint for a moment; it is the constant buffer we come to next. What matters here is p.color: a value computed per vertex, handed to the pixel shader. The pixel shader also receives p.position. By the time it gets there it is no longer in clip space: SV_Position in a pixel shader holds the pixel's own coordinates in the render target, which can be handy and which we use in later chapters.

Constant buffers

Vertex attributes give each vertex its own values. Often we want the opposite: one value that every vertex and pixel in a draw sees, such as the time, a colour for the whole object, or later a transformation matrix. GLSL calls these uniforms. HLSL puts them in constant buffers: a block of named constants, declared with cbuffer, whose values come from a buffer object we fill on the CPU. This lesson's has a single member, a colour to tint everything with:

cbuffer frame : register(b0)
{
    float4 tint;
};

register(b0) places it in constant buffer slot 0. That number is how our program and the shader find each other. OpenGL looks uniforms up by name with glGetUniformLocation; Direct3D does no lookup at all. We bind a buffer to slot 0, the shader reads slot 0, and inside it the members sit at fixed byte offsets. So the CPU side declares a struct with the same layout:

// Mirrors `cbuffer frame` in the shader, register for register.
frame_constants: struct = {
    tint: vec4;
}

vec4 is four floats, from includes/math.rin. With one member the layouts trivially agree, but they won't always be so simple. HLSL packs a constant buffer into 16-byte registers, and a member never straddles two of them. A float3 followed by a float share one register, but a float3 followed by another float3 leaves a gap of 4 bytes between them, and our rin struct has to leave the same gap. Later lessons add explicit padding fields for this.

Creating and filling the buffer

The buffer itself is created like the vertex buffers, with different settings:

// A dynamic constant buffer: the CPU may rewrite it. Its size must be a
// multiple of 16 bytes, the size of one HLSL register.
constants_desc: D3D11_BUFFER_DESC = {
    .ByteWidth = sizeof(frame_constants),
    .Usage = D3D11_USAGE.D3D11_USAGE_DYNAMIC,
    .BindFlags = D3D11_BIND_CONSTANT_BUFFER,
    .CPUAccessFlags = D3D11_CPU_ACCESS_WRITE,
};
constants: *ID3D11Buffer = null;
ID3D11Device_CreateBuffer(device, constants_desc&, null, constants&);

D3D11_BIND_CONSTANT_BUFFER says what it is for. Our vertex buffers were IMMUTABLE, fixed at creation; this one is DYNAMIC, with CPU write access, because we will rewrite it every frame. There is no initial data (the null), since we fill it before every draw.

Each frame, the lesson computes a tint that pulses between black and green over time, the same sin trick LearnOpenGL uses, and copies it into the buffer:

c: frame_constants = { .tint = v4(0.0, sinf(current_frame) * 0.5 + 0.5, 0.0, 1.0) };
mapped: D3D11_MAPPED_SUBRESOURCE = {};
ID3D11DeviceContext_Map(context, cast(constants, *ID3D11Resource), 0, D3D11_MAP.D3D11_MAP_WRITE_DISCARD, 0, mapped&);
memcpy(mapped.pData, c&, sizeof(frame_constants));
ID3D11DeviceContext_Unmap(context, cast(constants, *ID3D11Resource), 0);

\(\sin\) swings between -1 and 1, so halving it and adding a half gives a green channel that moves smoothly between 0 and 1. current_frame is the time in seconds since the program started, from a small get_time procedure that plays the part of glfwGetTime:

// Seconds since the first call: what glfwGetTime is to LearnOpenGL.
get_time: proc() -> f32 = {
    now: LARGE_INTEGER = {};
    if (perf_frequency == 0) {
        QueryPerformanceFrequency(now&);
        perf_frequency = now.QuadPart;
        QueryPerformanceCounter(now&);
        perf_start = now.QuadPart;
    }
    QueryPerformanceCounter(now&);
    return cast(now.QuadPart - perf_start, f32) / cast(perf_frequency, f32);
}

QueryPerformanceCounter is Windows' high-resolution clock; dividing its count by its frequency turns counts into seconds.

Back to filling the buffer. Map gives us a pointer to memory we can write the buffer's new contents into, memcpy copies our struct there whole, and Unmap hands it back. In place of OpenGL's family of glUniform setters, one per type, there is just this: write the bytes of the struct.

The D3D11_MAP_WRITE_DISCARD flag matters. The GPU runs behind the CPU, often by a frame or more, so when we map the buffer the GPU may not yet have run the draw that read its previous contents. Overwriting that memory in place would mean either waiting for the GPU or changing values under a draw that has not happened yet. "Discard" says we don't need the old contents at all, so the driver hands us a fresh piece of memory, and the earlier draw keeps the values it was given. This is how a buffer that changes every frame should be updated.

Binding it

The last step is to bind the buffer to slot 0, and here Direct3D differs from OpenGL in a way worth noticing. An OpenGL uniform belongs to a program object. A Direct3D constant buffer slot belongs to a stage: the vertex shader has its own slots, the pixel shader its own. Only our pixel shader reads tint, so we bind it there only:

ID3D11DeviceContext_PSSetConstantBuffers(context, 0, 1, constants&);

If the vertex shader needed it too, VSSetConstantBuffers would bind the same buffer to the vertex stage's slot 0. Like every other piece of context state, a binding stays until it is replaced, so this happens once per frame however many draws follow.

The lesson draws its left triangle with the pulsing green tint, then maps the buffer again, writes plain white, and draws the right triangle:

ID3D11DeviceContext_IASetVertexBuffers(context, 0, 1, left_vb&, stride&, offset&);
ID3D11DeviceContext_Draw(context, 3, 0);

c.tint = v4(1.0, 1.0, 1.0, 1.0);
ID3D11DeviceContext_Map(context, cast(constants, *ID3D11Resource), 0, D3D11_MAP.D3D11_MAP_WRITE_DISCARD, 0, mapped&);
memcpy(mapped.pData, c&, sizeof(frame_constants));
ID3D11DeviceContext_Unmap(context, cast(constants, *ID3D11Resource), 0);
ID3D11DeviceContext_IASetVertexBuffers(context, 0, 1, right_vb&, stride&, offset&);
ID3D11DeviceContext_Draw(context, 3, 0);

Mapping with discard between the two draws is safe for the same reason as before: the first draw keeps the green it was given, and the second sees white. The pixel shader multiplies each vertex colour by the tint. The left triangle's vertices are all white, so it shows the tint alone and fades between black and green; the right one is multiplied by white, so it shows its vertex colours unchanged.

More attributes!

The right triangle is the interesting one. Each of its vertices carries a colour of its own, as three more floats after the position:

// x, y, z, then r, g, b
...
right_triangle: [18]f32 = {
     0.1, -0.5, 0.0,   1.0, 0.0, 0.0,
     0.9, -0.5, 0.0,   0.0, 1.0, 0.0,
     0.5,  0.5, 0.0,   0.0, 0.0, 1.0,
};

Red at the bottom left, green at the bottom right, blue at the top. (The left triangle has the same layout, with white for every colour.) Each vertex is now six floats, 24 bytes, laid out as in LearnOpenGL's diagram:

The input layout gets a second element to match:

layout: [2]D3D11_INPUT_ELEMENT_DESC = {
    { .SemanticName = "POSITION", .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32_FLOAT, .AlignedByteOffset = 0 },
    { .SemanticName = "COLOR", .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32_FLOAT, .AlignedByteOffset = 12 },
};

Both attributes are three floats. AlignedByteOffset says where each starts within a vertex: the position at byte 0, the colour 12 bytes in, right after the position's three floats. The semantic names match the ones in vs_in. The distance from one vertex to the next, the stride, is not part of the layout; it is given when the vertex buffer is bound, and is now 24:

stride: UINT = 24;
offset: UINT = 0;

With the layout, the buffers and the shaders in place, this is what the program shows:

The left triangle's green rises and falls over time. The right one deserves a closer look: we gave it three colours, one per corner, and it shows a smooth blend of them across its whole surface. That is interpolation. The rasterizer works out which pixels a triangle covers, and for each of them it blends the values the three vertices output, weighted by how close the pixel is to each vertex. A pixel right at the red corner gets almost pure red; a pixel halfway along the bottom edge gets half red and half green; one in the middle gets a mix of all three. Every output of the vertex shader other than SV_Position is interpolated this way before the pixel shader sees it, which is why the comment on vs_out says so.

The triangle has three vertices, but tens of thousands of pixels, and every one of them got its own colour without us doing anything per pixel. This is the basic economy of the GPU: compute what you can per vertex and let interpolation spread it.

Our own shader helper

Hello Triangle compiled its two shaders and created its input layout by hand: read the file, call D3DCompile twice, check for errors, create the two shader objects, create the layout. Every lesson needs the same, so from here on it is done by a helper, includes/learn/shader.rin, just as LearnOpenGL writes its Shader class at this point. It holds what one .hlsl file turns into:

shader: struct = {
    vs: *ID3D11VertexShader;
    gs: *ID3D11GeometryShader;   // null unless the file has a gs_main
    ps: *ID3D11PixelShader;
    layout: *ID3D11InputLayout;  // null for shaders that make their own vertices
}

A vertex shader, a pixel shader, an optional geometry shader for a later chapter, and the input layout. Since Direct3D has no program object, this struct is the nearest thing: four objects that belong together.

Reading and compiling

shader_create reads the whole file with filesystem_read from learn/filesystem.rin, then compiles each entry point and creates its object:

shader_create: proc(device: *ID3D11Device, path: [*]const char, layout: []const D3D11_INPUT_ELEMENT_DESC) -> shader = {
    size: u64 = 0;
    source: [*]u8 = filesystem_read(path, size&);
    if (source == null) {
        printf("shader: cannot read %s\n", path);
        exit(1);
    }
    result: shader = {};

    vs: *ID3DBlob = shader_compile(path, source, size, "vs_main", "vs_5_0");
    ID3D11Device_CreateVertexShader(device, ID3D10Blob_GetBufferPointer(vs), ID3D10Blob_GetBufferSize(vs), null, result.vs&);
    if (layout.length > 0) {
        if (ID3D11Device_CreateInputLayout(device, layout.data, cast(layout.length, UINT),
                                           ID3D10Blob_GetBufferPointer(vs), ID3D10Blob_GetBufferSize(vs), result.layout&) < 0) {
            printf("shader: the input layout does not match vs_main in %s\n", path);
            exit(1);
        }
    }
    ID3D10Blob_Release(vs);
    ...
    ps: *ID3DBlob = shader_compile(path, source, size, "ps_main", "ps_5_0");
    ID3D11Device_CreatePixelShader(device, ID3D10Blob_GetBufferPointer(ps), ID3D10Blob_GetBufferSize(ps), null, result.ps&);
    ID3D10Blob_Release(ps);
    return result;
}

The layout is passed as a slice, []const D3D11_INPUT_ELEMENT_DESC: a pointer and a length together, which rin makes from an array automatically. An empty slice means the shader reads no vertex buffers (some later shaders make their vertices up from SV_VertexID), and then no layout is created. Creating the layout is where Direct3D checks it against the vertex shader's inputs, as Hello Triangle explained, so a mismatch stops the program here with a message naming the file. The bytecode blobs are released once their shader objects exist, since nothing needs them after that. The part cut out in the middle does the same for gs_main if the file contains one.

The compile step itself is Hello Triangle's, with three additions:

shader_compile: proc(path: [*]const char, source: [*]const u8, size: u64, entry: [*]const char, target: [*]const char) -> *ID3DBlob = {
    code: *ID3DBlob = null;
    errors: *ID3DBlob = null;
    hr: HRESULT = D3DCompile(source, size, path, null, D3D_COMPILE_STANDARD_FILE_INCLUDE, entry, target,
                             D3DCOMPILE_ENABLE_STRICTNESS | D3DCOMPILE_DEBUG, 0, code&, errors&);
    if (hr < 0) {
        printf("shader: %s\n", errors != null ? cast(ID3D10Blob_GetBufferPointer(errors), [*]const char) : path);
        exit(1);
    }
    if (errors != null) { ID3D10Blob_Release(errors); }
    return code;
}

• D3D_COMPILE_STANDARD_FILE_INCLUDE lets a shader use #include, with paths relative to the file being compiled.
• D3DCOMPILE_ENABLE_STRICTNESS rejects old, deprecated syntax, and D3DCOMPILE_DEBUG keeps debug information in the bytecode so that graphics debuggers such as PIX or RenderDoc can show the HLSL source.
• The errors blob can be filled even when compilation succeeds, with warnings, so it is released either way.

On failure, the compiler's message already starts with the file name and line, so it is printed as it is and the program exits.

Using it

Binding a shader is four calls, wrapped in one:

shader_use: proc(context: *ID3D11DeviceContext, s: *const shader) -> void = {
    ID3D11DeviceContext_IASetInputLayout(context, s[0].layout);
    ID3D11DeviceContext_VSSetShader(context, s[0].vs, null, 0);
    ID3D11DeviceContext_GSSetShader(context, s[0].gs, null, 0);
    ID3D11DeviceContext_PSSetShader(context, s[0].ps, null, 0);
}

Binding a null geometry shader is not a mistake: it switches that stage off, so a shader without gs_main also clears out one that an earlier draw left bound. (In rin, s[0] reads the struct a pointer points at, where C would write s->.) Unlike LearnOpenGL's class, ours has no setters for uniforms: constant buffers are written as whole structs and bound by slot, as we just did, which needs no help from the shader.

The lesson then needs only two lines, one at startup and one each frame:

s: shader = shader_create(device, filesystem_path("src/1.getting_started/3.1.shaders/shaders.hlsl"), layout);

shader_use(context, s&);

The full program is here, and its shaders here.

Additional resources

• HLSL reference: Microsoft's language reference, including every intrinsic function.
• Semantics: the system values and how inputs and outputs are matched.
• Packing rules for constant variables: the 16-byte register rules, with examples.
• How to use dynamic resources: Map with WRITE_DISCARD and its relatives.
• LearnOpenGL's Shaders: the same chapter in GLSL.

Exercises

1. Change the vertex shader so that both triangles are drawn upside down.
2. Add a float offset to the frame constant buffer and use it in the vertex shader to move the left triangle sideways. You will need to grow frame_constants to match, padding it so its size stays a multiple of 16 bytes, and bind the buffer to the vertex shader stage as well.
3. Output the vertex position from the vertex shader under its own semantic and use it as the pixel colour. Both triangles turn into gradients. Why is the left triangle's bottom-left corner black?