Textures

In the last chapter we coloured a triangle by giving each vertex a colour and letting the rasterizer blend them. That only goes so far. To draw something with real detail, a wooden crate or a brick wall, we would need a vertex wherever the colour changes, which means a huge number of vertices, each carrying its own colour.

The usual answer is a texture: an image that we stretch over the surface of our geometry, like a printed sticker on a box. A single image can hold as much detail as we like, and the geometry under it can stay simple. Textures can hold other kinds of data too, as later chapters will show, but here they are just pictures.

To put an image on a shape, each vertex has to say which point of the image it corresponds to. That point is the vertex's texture coordinate. The rasterizer interpolates texture coordinates across the triangle like any other vertex output, so every pixel ends up with its own coordinate, and the pixel shader looks up the image's colour there. Looking up a texture's colour at a coordinate is called sampling.

Texture coordinates

Texture coordinates run from 0 to 1 along each axis of the image, whatever its size in pixels. They are usually called \(u\) (across) and \(v\) (down), and in Direct3D they start at the image's top-left corner: \((0, 0)\) is the top-left, \((1, 0)\) the top-right, \((0, 1)\) the bottom-left and \((1, 1)\) the bottom-right.

This is the opposite of OpenGL, where \((0, 0)\) is the bottom-left of a texture, and it is the one convention in this chapter that differs. Image files store their top row first; Direct3D puts the first row it is given at \(v = 0\), and so the two agree, and an image loaded straight from a file is the right way up. OpenGL puts the first row at the bottom, which is why LearnOpenGL has to flip images while loading them. We don't, ever.

This lesson textures the rectangle from Hello Triangle. Each vertex now has a position followed by its texture coordinate, five floats in all:

// x, y, z, then u, v. D3D's texture coordinate (0, 0) is the image's
// top-left, where GL's is its bottom-left, so nothing is flipped on load.
rectangle_vertices: [20]f32 = {
    -0.5, -0.5, 0.0,   0.0, 1.0,
     0.5, -0.5, 0.0,   1.0, 1.0,
     0.5,  0.5, 0.0,   1.0, 0.0,
    -0.5,  0.5, 0.0,   0.0, 0.0,
};

The top corners of the rectangle (positive \(y\)) take \(v = 0\), the top of the image, and the bottom corners take \(v = 1\). The four corners of the image land on the four corners of the rectangle, and everything between is filled in by interpolation.

That describes which point of the image each pixel wants. How the image is read there is a separate question, and it has several possible answers. In Direct3D they are collected into a separate object, the sampler, whose settings are the subject of the next three sections.

Texture wrapping

Nothing stops a texture coordinate from going below 0 or above 1. What the sampler does then is its address mode. Direct3D's main modes, with OpenGL's names for comparison:

• D3D11_TEXTURE_ADDRESS_WRAP (GL_REPEAT): only the fractional part of the coordinate counts, so the image repeats. 1.25 reads the same as 0.25.
• D3D11_TEXTURE_ADDRESS_MIRROR (GL_MIRRORED_REPEAT): the image repeats, flipped on every other repetition, so adjacent copies meet edge to matching edge.
• D3D11_TEXTURE_ADDRESS_CLAMP (GL_CLAMP_TO_EDGE): the coordinate is clamped to 0..1, so everything outside repeats the colours of the image's outermost pixels, smeared outwards.
• D3D11_TEXTURE_ADDRESS_BORDER (GL_CLAMP_TO_BORDER): everything outside gets one fixed colour, the sampler's BorderColor.

Here they are along one axis, with the image in the middle (from 0 to 1, dashed) and a copy's width either side:

The mode is set separately for each axis, in the sampler description's AddressU, AddressV and AddressW (the last is for 3D textures). This lesson wraps on all three; none of its coordinates leave 0..1 yet, but the exercises change that.

Texture filtering

A texture coordinate is a pair of real numbers, but an image is a grid of pixels. To avoid confusion with the pixels on screen, a texture's pixels are called texels. A coordinate almost never lands exactly on the centre of a texel, so the sampler has to decide what colour to return for the point in between. That is texture filtering, and there are two basic choices.

Point filtering (OpenGL's GL_NEAREST) returns the texel whose centre is nearest to the coordinate. In LearnOpenGL's diagram the cross marks the coordinate, and the top-left texel wins:

Linear filtering (GL_LINEAR) blends the four texels around the coordinate, each weighted by how close the coordinate is to its centre. The result is a colour in between them:

The difference shows when a small texture is stretched over a large area, so that each texel covers many pixels on screen. Point filtering then shows the texels as hard-edged blocks, the look of old pixel-art games. Linear filtering gives a smooth, slightly blurry image without visible blocks, which is what we want most of the time.

The filter can be chosen separately for magnification, when the texture is drawn larger than its real size, and minification, when it is drawn smaller. OpenGL sets each with its own parameter. Direct3D has one Filter field, whose values name every combination: D3D11_FILTER_MIN_MAG_MIP_POINT uses point filtering throughout, D3D11_FILTER_MIN_MAG_MIP_LINEAR linear throughout, and in between are values such as D3D11_FILTER_MIN_LINEAR_MAG_MIP_POINT. The third word in those names, MIP, needs one more idea.

Mipmaps

Picture a long floor covered in one tiled texture. Near the camera, a texel covers a few screen pixels. Far away, a single screen pixel covers dozens of texels. Sampling just one or four of those texels picks an essentially random colour out of the many the pixel covers, and as the camera moves, which texels get picked changes from frame to frame. Distant surfaces shimmer and sparkle. Reading a large texture to draw a few pixels is also slow, because it uses the GPU's memory caches badly.

The fix is to prepare smaller copies of the texture in advance, each half the width and height of the one before, down to a single texel. These are mipmaps, and together they form the texture's mip chain:

When sampling, the GPU estimates how many texels one screen pixel covers and reads from the level where that is about one. Distant surfaces read small, pre-averaged levels, which is both stable and fast. The whole chain costs only a third more memory than the full-size image. The level with the full image is level 0; a 512 by 512 texture, like the crate in this lesson, has ten levels, ending at 1 by 1.

The sampler then has one more choice to make: whether to use the single nearest level (point) or to blend between the two nearest levels (linear), which hides the visible seam where one level gives way to the next. That is the MIP part of the filter names. D3D11_FILTER_MIN_MAG_MIP_LINEAR, linear within each level and between levels, is often called trilinear filtering, and corresponds to OpenGL's GL_LINEAR_MIPMAP_LINEAR with a GL_LINEAR magnification filter.

Our sampler

The lesson creates one sampler at startup:

// A sampler: how textures are filtered and what happens past their edges.
sampler_desc: D3D11_SAMPLER_DESC = {
    .Filter = D3D11_FILTER.D3D11_FILTER_ANISOTROPIC,
    .AddressU = D3D11_TEXTURE_ADDRESS_MODE.D3D11_TEXTURE_ADDRESS_WRAP,
    .AddressV = D3D11_TEXTURE_ADDRESS_MODE.D3D11_TEXTURE_ADDRESS_WRAP,
    .AddressW = D3D11_TEXTURE_ADDRESS_MODE.D3D11_TEXTURE_ADDRESS_WRAP,
    .MaxAnisotropy = 8,
    .MaxLOD = D3D11_FLOAT32_MAX,
};
sampler: *ID3D11SamplerState = null;
ID3D11Device_CreateSamplerState(device, sampler_desc&, sampler&);
ID3D11DeviceContext_PSSetSamplers(context, 0, 1, sampler&);

The filter is a step beyond trilinear. Anisotropic filtering is for surfaces seen at a slant, like that long floor, where a screen pixel covers a long thin strip of texels rather than a square patch. Trilinear filtering has to pick a mip level small enough for the strip's length, which blurs it across its width. Anisotropic filtering instead takes several samples along the strip, here up to MaxAnisotropy = 8 of them, and keeps the surface sharp. On a rectangle facing the camera squarely, as ours does, it gives the same result as trilinear.

MaxLOD is the highest mip level the sampler may use (LOD, level of detail, is another name for the mip level). Setting it to the largest float means "no limit".

Samplers are bound to sampler slots of a shader stage, as constant buffers are bound to theirs, here slot 0 of the pixel shader. The binding stays until it is replaced, so this lesson binds its only sampler once, before the render loop, and never again. OpenGL stores filtering and wrapping on each texture object; Direct3D keeps them apart, so any texture can be read with any sampler, and one sampler serves every texture that wants the same treatment.

Loading and creating textures

Before we can use an image we have to get its pixels out of a file. Image formats are complicated, each in its own way: PNG compresses losslessly, JPEG approximates the image with frequency components, and both have many variants. Writing a loader for even one of them is a project. LearnOpenGL uses stb_image.h, a popular single-file C library. We don't need a library at all: Windows comes with the Windows Imaging Component, WIC, which decodes PNG, JPEG, BMP, GIF, TIFF and more. Our includes/image.rin wraps it in one procedure.

image.rin

WIC is a COM API like Direct3D. Its starting point is an imaging factory, created once:

image_init: proc() -> void = {
    if (image_wic != null) { return; }
    memops_arena_initialize(image_arena&);
    CoInitializeEx(null, COINIT_MULTITHREADED);
    CoCreateInstance(CLSID_WICImagingFactory&, null, CLSCTX_INPROC_SERVER, IID_IWICImagingFactory&, cast(image_wic&, **void));
}

CoInitializeEx starts COM for the program, and CoCreateInstance creates an object by its class ID, here the factory. (Direct3D didn't need this step because D3D11CreateDeviceAndSwapChain creates its objects itself.) image_load then asks the factory for a decoder for the file, takes its first frame (a GIF can have several; our images have one), and converts it:

IWICImagingFactory_CreateFormatConverter(image_wic, converter&);
IWICFormatConverter_Initialize(converter, cast(frame, *IWICBitmapSource), GUID_WICPixelFormat32bppRGBA&,
                               WICBitmapDitherType.WICBitmapDitherTypeNone, null, 0.0,
                               WICBitmapPaletteType.WICBitmapPaletteTypeCustom);
w: UINT = 0;
h: UINT = 0;
IWICFormatConverter_GetSize(converter, w&, h&);
size: u64 = cast(w, u64) * cast(h, u64) * 4;
result.pixels = cast(memops_arena_push(image_arena&, size, 16), [*]u8);
IWICFormatConverter_CopyPixels(converter, null, w * 4, cast(size, UINT), result.pixels);

The format converter turns whatever the file contains (three channels for a JPEG, possibly a palette or 16-bit channels for a PNG) into one fixed layout: 32 bits per pixel, red, green, blue and alpha, one byte each. CopyPixels writes them, top row first, into memory from the image arena, a region that lives as long as the program (so there is no counterpart of stbi_image_free to call). The result is a small struct:

image: struct = {
    pixels: [*]u8;       // RGBA8, width * height * 4 bytes
    width: i32;
    height: i32;
}

Always converting to four channels spares us a detail LearnOpenGL has to handle: telling GL_RGB images from GL_RGBA ones. Direct3D has no 8-bit three-channel texture format anyway, so even a JPEG with no alpha becomes RGBA, with alpha 255.

Generating a texture

With the pixels in memory, the lesson's load_texture creates a texture from them. It plays the part of LearnOpenGL's loadTexture function, and later lessons have the same procedure.

load_texture: proc(device: *ID3D11Device, context: *ID3D11DeviceContext, path: [*]const char, srgb: b32) -> *ID3D11ShaderResourceView = {
    img: image = image_load(path);
    if (img.pixels == null) {
        printf("Texture failed to load at path: %s\n", path);
        exit(1);
    }
    desc: D3D11_TEXTURE2D_DESC = {
        .Width = cast(img.width, UINT),
        .Height = cast(img.height, UINT),
        .MipLevels = 0,   // the whole chain
        .ArraySize = 1,
        .Format = srgb ? DXGI_FORMAT.DXGI_FORMAT_R8G8B8A8_UNORM_SRGB : DXGI_FORMAT.DXGI_FORMAT_R8G8B8A8_UNORM,
        .SampleDesc = { .Count = 1 },
        // Generating mips draws into the texture, so it is a render target too.
        .BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET,
        .MiscFlags = D3D11_RESOURCE_MISC_GENERATE_MIPS,
    };
    texture: *ID3D11Texture2D = null;
    srv: *ID3D11ShaderResourceView = null;
    ID3D11Device_CreateTexture2D(device, desc&, null, texture&);
    ID3D11Device_CreateShaderResourceView(device, cast(texture, *ID3D11Resource), null, srv&);
    ID3D11DeviceContext_UpdateSubresource(context, cast(texture, *ID3D11Resource), 0, null, img.pixels, cast(img.width * 4, UINT), 0);
    ID3D11DeviceContext_GenerateMips(context, srv);
    return srv;
}

This is the longest description so far, so let's take it field by field.

• Width and Height are the image's size in texels.
• MipLevels is the number of mip levels. Zero is special: it means the whole chain, down to 1 by 1, however many levels that takes for this size.
• ArraySize is 1, a single image. Textures can also be arrays of images, which is how cubemaps are stored in a later chapter.
• Format is R8G8B8A8_UNORM, matching the bytes WIC gave us. The srgb parameter would choose an sRGB format instead; it matters for colour accuracy, and Gamma Correction explains it. This lesson passes 0 for both its textures.
• SampleDesc.Count is 1: an ordinary texture, not multisampled.
• BindFlags say how the texture will be used. It will be read by shaders, so D3D11_BIND_SHADER_RESOURCE. It is also marked as a render target, because of the next field.
• MiscFlags asks for D3D11_RESOURCE_MISC_GENERATE_MIPS, permission to have the GPU compute the mipmaps for us. The GPU does that by drawing each level from the one above it, so the texture must be usable as a render target too.

The Usage field is left at zero, which is D3D11_USAGE_DEFAULT: memory the GPU reads and writes, which the CPU can only update through the context. With the description complete, four calls finish the job:

1. CreateTexture2D creates the texture. We pass no initial data, the null: initial data has to cover every mip level, and we only have level 0.
2. CreateShaderResourceView creates the view a shader reads the texture through. With no view description it covers the whole texture, all its levels.
3. UpdateSubresource copies our pixels into subresource 0, which for this texture is mip level 0. The null box means the whole level, and the next argument is the row pitch, the number of bytes from the start of one row to the next: four bytes per texel, times the width.
4. GenerateMips fills in every other level from level 0, the counterpart of glGenerateMipmap. It takes the shader resource view, and generates all the levels the view covers.

The procedure returns the view. That is all the rest of the program needs: shaders read textures through views, never through the texture object itself.

The lesson loads two textures this way, a wooden crate and a smiling face, both from resources/textures:

textures: [2]*ID3D11ShaderResourceView = {
    load_texture(device, context, filesystem_path("resources/textures/container.jpg"), 0),
    load_texture(device, context, filesystem_path("resources/textures/awesomeface.png"), 0),
};

Applying textures

The vertices have a texture coordinate now, so the input layout describes it, right after the position:

layout: [2]D3D11_INPUT_ELEMENT_DESC = {
    { .SemanticName = "POSITION", .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32B32_FLOAT, .AlignedByteOffset = 0 },
    { .SemanticName = "TEXCOORD", .Format = DXGI_FORMAT.DXGI_FORMAT_R32G32_FLOAT, .AlignedByteOffset = 12 },
};

DXGI_FORMAT_R32G32_FLOAT is two floats, \(u\) and \(v\), starting 12 bytes into the vertex. A vertex is five floats, so the stride we bind the vertex buffer with is 20 bytes. The rectangle is drawn with the same index buffer as in Hello Triangle, two triangles from four vertices.

The vertex shader passes the coordinate through to the pixel shader, so that it is interpolated across the rectangle:

struct vs_out
{
    float4 position : SV_Position;
    float2 uv : TEXCOORD;
};

vs_out vs_main(float3 position : POSITION, float2 uv : TEXCOORD)
{
    vs_out o;
    o.position = float4(position, 1.0);
    o.uv = uv;
    return o;
}

The inputs are plain parameters this time, each with its semantic, rather than a struct: HLSL accepts either. The pixel shader needs the textures and the sampler, and they are declared at the top of the file:

Texture2D crate : register(t0);
Texture2D face : register(t1);
SamplerState linear_wrap : register(s0);

A Texture2D is a 2D texture the shader reads, through a view bound to a texture slot: register(t0) is slot 0, t1 slot 1. A SamplerState is a sampler, in sampler slot s0. GLSL combines both into one sampler2D variable; HLSL keeps them apart, matching Direct3D's separate objects. Sampling then takes both, in method-call form:

float4 ps_main(vs_out p) : SV_Target
{
    return lerp(crate.Sample(linear_wrap, p.uv), face.Sample(linear_wrap, p.uv), 0.2);
}

crate.Sample(linear_wrap, p.uv) reads the crate texture at the pixel's interpolated coordinate, filtered and addressed as the sampler says, and returns a float4 colour. GLSL writes texture(sampler, uv). The face is sampled the same way, with the same sampler, and the two are blended with lerp, HLSL's name for GLSL's mix: lerp(a, b, t) is \(a + (b - a)\,t\), so with \(t = 0.2\) we get 80% crate and 20% face.

Texture units

OpenGL has texture units: we activate a unit, bind a texture to it, and tell each sampler2D uniform which unit to read with glUniform1i. Direct3D's texture slots play the same role, and the shader states which slot each texture lives in, as we saw with register(t0) and register(t1), so there is nothing to tell it from the program. All that remains is to bind the two views to those slots, which one call does, since slots are consecutive:

// bind textures on corresponding texture registers, t0 and t1
ID3D11DeviceContext_PSSetShaderResources(context, 0, 2, textures[0]&);

Starting at slot 0, bind two views from the array: the crate to t0 and the face to t1. Each stage has 128 texture slots and 16 sampler slots, which is more than enough. Then the usual drawing calls follow, and this is the result:

One happy crate, the right way up, without flipping anything. The image looks stretched sideways because the rectangle spans half the window in each direction, and the window is wider than it is tall. That gets fixed when we add a projection matrix in Coordinate Systems.

The full program is here, its shaders here, and the image loader here.

Additional resources

• D3D11_SAMPLER_DESC: every sampler setting, with its default.
• D3D11_FILTER: the full list of filter combinations.
• Subresources: how mip levels and array slices are numbered.
• About the Windows Imaging Component: the decoders behind image.rin.
• LearnOpenGL's Textures: the same chapter in OpenGL.

Exercises

1. Change only the pixel shader so that the face looks the other way, while the crate stays as it is.
2. Change the texture coordinates to run from 0 to 2 instead of 0 to 1, so the face appears four times. Then try each address mode on the sampler. Finally, make the crate clamp while the face still repeats: you'll need a second sampler in slot s1, used for one of the two textures.
3. Choose texture coordinates that show only a small area in the middle of the textures, so that each texel covers many pixels. Set the filter to D3D11_FILTER_MIN_MAG_MIP_POINT and compare it with linear filtering.
4. Put the 0.2 blend amount in a constant buffer, as in the Shaders chapter, and change it with the up and down arrow keys (VK_UP and VK_DOWN).