Theory

Physically based rendering, or PBR, is not one algorithm but a family of techniques that share a common idea: model the way light really behaves when it meets a surface, closely enough that the results hold up. The Phong and Blinn-Phong lighting of earlier chapters were built by eye. They have a diffuse colour, a specular colour and a shininess exponent, and we nudged those until a scene looked right. That works for one scene under one set of lights, and then breaks when the lighting changes. A PBR material is described instead by physical properties (how rough it is, whether it is a metal), and because the lighting model respects physics, the same material looks right in a dim room and in sunlight without retuning.

It is still an approximation. We are not simulating photons; we are using equations that are based on physics, which is why the name is physically based rather than physical. In practice a lighting model earns the label when it meets three conditions, each of which gets its own section below:

1. it is built on the microfacet surface model,
2. it conserves energy,
3. it uses a physically based BRDF.

The flavour of PBR we follow is the one Disney researched and Epic Games adapted for real-time use in Unreal Engine 4. It is built on the metallic workflow, it is well documented, nearly every modern engine uses some variant of it, and it looks very good. By the end of these four chapters our spheres will look like this:

These chapters lean on much of what came before: render targets, cubemaps, gamma correction, HDR and normal mapping. There is also more mathematics than anywhere else in the book. None of it is hard once it is unpacked, and we will unpack it one term at a time. This chapter is all theory; the next one turns it into a shader.

The microfacet model

Look at any surface under a strong enough microscope and it stops being flat. It becomes a landscape of tiny facets, each one small and flat enough to act as a perfect mirror. These are microfacets, and the whole of PBR rests on describing them. On a polished surface they all point roughly the same way. On a rough one they point every which way:

That alignment decides what reflections look like. Light arriving at a smooth surface bounces off facets that agree with each other, so the reflected rays leave together and we see a small, sharp highlight. On a rough surface each ray meets a facet tilted its own way, the reflected rays fan out, and the highlight spreads into a wide, dim glow:

Microfacets are far smaller than a pixel, so we never model them one by one. We describe them statistically, with a single roughness number from 0 to 1, and ask a statistical question: what share of the microfacets is oriented so that it reflects light from the light direction \(l\) into the view direction \(v\)? A mirror does that only if its normal sits exactly halfway between the two, which is the halfway vector we met in Advanced Lighting:

The more microfacets face along \(h\), the stronger and tighter the reflection. A low roughness packs many facets close to the surface normal, so when \(h\) lines up with the normal the reflection is intense, and when it doesn't there is almost nothing. A high roughness spreads them out, so some reflection happens over a wide range of angles but none of it is intense. We will see exactly how that looks when we reach the normal distribution function.

Energy conservation

The second rule is simple to state: a surface may not send out more light than it receives (light sources aside). It already explains something visible in the picture above. As roughness grows, the highlight gets bigger and dimmer. If it kept the same brightness while growing, the rough surface would be reflecting more energy than arrived, which is impossible. The same light is spread over a larger area, so each part of it is fainter.

To keep the books balanced we have to separate two things that happen when light hits a surface. Part of it is reflected straight off the surface without entering; that is the specular light. The rest is refracted: it goes into the material. What happens then is the diffuse light.

Refracted light doesn't simply vanish. Inside the material it bumps into particles, losing some energy at each collision (turned into heat) and changing direction each time. Some of it is absorbed completely. The rest eventually finds its way back out, in a more or less random direction, and that re-emerging light is the colour we see as the surface's diffuse colour. PBR makes a simplification here: it assumes the light comes back out very close to where it went in. Materials where light travels a noticeable distance under the surface, such as skin, marble, milk or wax, need subsurface scattering techniques, which look much better on them and cost a good deal more.

Metals are a special case. Light refracted into a metal is absorbed straight away, with no scattering, so a metal has no diffuse light at all: everything we see of it is reflection. Non-metals, called dielectrics, have both. That difference matters enough that the PBR model handles the two kinds of material differently, as we will see with Fresnel.

Reflection and refraction also compete for the same light. Whatever fraction is reflected is not available to be refracted, and the other way round. So once we know the reflected fraction, which we will call \(k_s\), the refracted fraction follows directly:

ks = (fraction of light reflected, from Fresnel)
kd = 1.0 - ks        // whatever is left enters the surface

With the two fractions defined like this they can never add up to more than 1, which is exactly the guarantee we wanted. Our Phong shaders never had it: their diffuse and specular terms were added with no relation between them, so a bright specular colour on top of a bright diffuse colour simply created light.

The reflectance equation

The serious theory of light transport is the rendering equation, which describes how light bounces around an entire scene. PBR uses a narrower version of it that only deals with one surface point at a time, the reflectance equation. It is the centre of everything in this chapter and the next three, so here it is in full:

It reads, roughly: the light leaving point \(p\) towards the viewer is the sum, over every direction light can arrive from, of the incoming light, weighted by the angle it arrives at and by how much of it this material sends in the viewer's direction. To make that precise we need a few terms from radiometry, the measurement of electromagnetic radiation (visible light included). There are many radiometric quantities; we only need the ones that build up to radiance, written \(L\) in the equation.

Radiant flux

Radiant flux, \(\Phi\), is the total energy a light source emits per second, measured in watts. Real light is a mixture of wavelengths, and a light source's flux is really a function over the spectrum: so much energy at 450nm, so much at 600nm, and so on across the visible range of about 390nm to 700nm. Carrying a whole spectrum through a shader is impractical, so we sample it at three points and store the result as an RGB triple. That loses information, but for what the eye sees the loss is rarely noticeable. In practice the "light colour" we have always passed to our shaders is a radiant flux in RGB.

Solid angle

The solid angle, \(\omega\), is to directions what area is to surfaces. Picture yourself at the centre of a unit sphere looking at some shape. Project its outline onto the sphere; the area of that patch is the shape's solid angle. You can think of it as a direction with some thickness: not a single ray, but a bundle of them.

Radiant intensity

Radiant intensity is flux per solid angle: how much energy a source sends into a given bundle of directions. A point light that shines equally in every direction has the same intensity whichever way you look at it.

Radiance

Radiance combines all of these. It is the flux passing through a small area \(A\) within a small solid angle \(\omega\), corrected for the angle \(\theta\) between the light and the area's normal:

The \(\cos\theta\) is the same factor as in our very first diffuse lighting: light that strikes a surface at a glancing angle spreads its energy over more of the surface, so each point receives less. In a shader it is the dot product we have written many times:

cos_theta = dot(light_dir, n)

Radiance is the quantity we want because it contains everything: an amount of energy, a direction, and a point. If we shrink the area and the solid angle to nothing, radiance describes a single ray of light arriving at a single point. The solid angle becomes a direction vector \(\omega\) and the area becomes a point \(p\), which is exactly what a pixel shader has to work with: one point on a surface and the directions light comes from.

Back to the equation

So \(L_i(p, \omega_i)\) is the radiance arriving at \(p\) from direction \(\omega_i\), and \(n \cdot \omega_i\) is the \(\cos\theta\) that weakens light arriving at an angle. \(L_o(p, \omega_o)\) is the radiance leaving \(p\) in the direction of the viewer, \(\omega_o\): the colour of our pixel. Summing radiance over all incoming directions gives the total light falling on the point, its irradiance, and the reflectance equation is a weighted version of that sum.

The directions to sum over are all those on the side of the surface facing outwards: a hemisphere \(\Omega\) centred on \(p\) and aligned with the normal \(n\). Light from below the surface can't reach the front of it.

The integral sign \(\int\) is how mathematics writes "add up continuously over every direction \(d\omega_i\) in \(\Omega\)". There is no closed-form solution for it in general, so we compute it numerically: pick many directions, evaluate the expression for each, and add them up, each weighted by the small piece of the hemisphere it stands for. That is a Riemann sum, and in pseudo-code it looks like this:

steps = 100
d_omega = 1.0 / steps            // the share of the hemisphere each step stands for
sum = 0
for i in 0 .. steps-1:
    wi = next_incoming_direction(i)
    sum += brdf(p, wi, wo) * radiance(p, wi) * dot(n, wi) * d_omega

The more steps, the closer the sum gets to the true integral. Keep this shape in mind: it is literally what the convolution shaders of the IBL chapters do. For a handful of point lights it is much simpler, because only a few directions carry any light at all, and the integral becomes a loop over the lights.

Where does the incoming radiance come from? It can be ordinary light sources, as in the next chapter, or it can be an entire photographed environment, in which every direction has some radiance; that is image based lighting. Either way, the one remaining unknown is \(f_r\).

BRDF

\(f_r\) is the BRDF, the bidirectional reflectance distribution function. Given an incoming direction \(\omega_i\), an outgoing direction \(\omega_o\), the normal \(n\) and the material's properties (roughness above all), it says how much of the light arriving from \(\omega_i\) leaves towards \(\omega_o\). A perfect mirror would have a BRDF of zero for every pair of directions except the one where \(\omega_o\) is the exact mirror image of \(\omega_i\). A real material spreads its response out according to its microfacets.

A BRDF counts as physically plausible if it conserves energy: over all outgoing directions, it never returns more than came in. Blinn-Phong is technically a BRDF, since it also takes a light and a view direction, but it doesn't conserve energy, so it isn't physically based. There are several physically based BRDFs; real-time renderers almost universally use the Cook-Torrance BRDF, which has a diffuse part and a specular part:

\(k_d\) and \(k_s\) are the refracted and reflected fractions from the energy conservation section. The diffuse part is Lambertian diffuse, which is just a constant:

\(c\) is the albedo, the surface's base colour. The division by \(\pi\) is a normalisation: integrating \(\cos\theta\) over the hemisphere gives exactly \(\pi\), so without it a white Lambertian surface would reflect \(\pi\) times the light it received. (We will meet this \(\pi\) again in Diffuse irradiance.)

This is the same diffuse lighting we have been doing since Basic Lighting, the colour times the cosine. The cosine hasn't gone anywhere: it is the \(n \cdot \omega_i\) at the end of the reflectance equation, outside the BRDF. More elaborate diffuse models exist and look a little more realistic, but Epic found Lambert good enough for real-time work, and so do we.

The specular part is where the microfacets come in:

Three functions multiplied together, over a normalisation factor. Each answers one question about the microfacets:

• D, the normal distribution function: what share of the microfacets faces along the halfway vector, given the roughness? This is the heart of the microfacet model.
• G, the geometry function: what share of those microfacets is actually visible to both the light and the viewer, rather than hidden behind other microfacets?
• F, the Fresnel equation: of the light hitting those microfacets, what fraction is reflected rather than refracted, at this angle?

Each of the three is itself an approximation, and the literature offers several versions of each, some more accurate and some cheaper. Brian Karis of Epic Games compared many of them; we use the ones Unreal Engine 4 settled on: Trowbridge-Reitz GGX for \(D\), Schlick's approximation for \(F\), and Smith's method with Schlick-GGX for \(G\). They are the functions at the top of every chapter 6 shader, under this comment:

// The Cook-Torrance BRDF used throughout chapter 6.
//
//   f = kd * albedo / pi + D * F * G / (4 (n.v) (n.l))

Normal distribution function

The normal distribution function \(D\) gives the relative concentration of microfacets whose normal is exactly \(h\). Trowbridge-Reitz GGX writes it as

where \(\alpha\) is the roughness parameter. Shading a sphere with \(D\) alone, lit from the viewer's side, shows what it does across roughness values from smooth to rough:

At low roughness the facets are packed tightly around the surface normal. Where \(h\) matches it the function is very large, and a little way off it falls to nearly nothing: a small, intensely bright spot. At high roughness the facets are spread over many orientations, so many more halfway vectors find some facets aligned with them, but never very many: a wide, grey area. The total stays the same; this is energy conservation built into \(D\).

In HLSL it is a direct transcription:

float distribution_ggx(float3 n, float3 h, float roughness)
{
    float a = roughness * roughness;
    float a2 = a * a;
    float n_dot_h = max(dot(n, h), 0.0);
    float d = n_dot_h * n_dot_h * (a2 - 1.0) + 1.0;
    return a2 / (PI * d * d);
}

Notice the first line: the function takes the roughness an artist would set and squares it to get \(\alpha\). Disney observed, and Epic adopted, that roughness feels much more even across its range when it is squared before use. With \(\alpha\) taken directly from the slider, most of the visible change crowds into the low end. The next chapter comes back to this.

Geometry function

The geometry function \(G\) estimates how much the microfacets get in each other's way. On a rough surface, a facet may be correctly aligned to reflect the light towards us and still contribute nothing, because a neighbouring bump blocks the light before it arrives (shadowing) or blocks the reflected ray before it leaves (masking).

Like \(D\), \(G\) depends on roughness: the rougher the surface, the more likely a facet is to be hidden. The function we use combines GGX with the Schlick-Beckmann approximation and is called Schlick-GGX:

\(k\) is a remapping of roughness, and the right remapping depends on what kind of light we are computing. Writing \(r\) for the roughness an artist sets (before any squaring):

The first is for light from point, directional and spot lights; the second for image based lighting. Both come from Epic's notes, and we will use each where it belongs.

Blocking can happen on either leg of the journey, so we need \(G\) for both the view direction and the light direction. Smith's method combines them by multiplying the two:

On spheres over the same range of roughness it looks like this; white means no microfacet shadowing at all and black means total shadowing:

Smooth surfaces barely shadow themselves; rough ones darken, most of all near the silhouette where both light and view arrive at grazing angles. In our shaders the two functions are:

float geometry_schlick_ggx(float n_dot_x, float k)
{
    return n_dot_x / (n_dot_x * (1.0 - k) + k);
}

// k depends on the use: direct light remaps roughness one way, image based
// lighting (the `ibl` path) another.
float geometry_smith(float n_dot_v, float n_dot_l, float roughness, bool ibl)
{
    float r = roughness + 1.0;
    float k = ibl ? roughness * roughness / 2.0 : r * r / 8.0;
    return geometry_schlick_ggx(n_dot_v, k) * geometry_schlick_ggx(n_dot_l, k);
}

Our version takes the two dot products directly, since the caller has already computed them, and an ibl flag that picks between the two remappings. One function serves the direct lights of this chapter and the BRDF table of the specular IBL chapter.

Fresnel equation

The Fresnel equation (the s is silent: "freh-nel") gives the fraction of light that is reflected rather than refracted, and that fraction depends strongly on the angle we see the surface at. Every material reflects a certain amount when looked at straight on, its base reflectivity. Tilt it towards a glancing angle and reflections get much stronger. You can see it on a table top: looked at from above, it shows only a faint reflection; lower your eye almost to the level of the table and the far end turns into a mirror. In the limit of a perfectly grazing angle, every surface reflects all the light. The full Fresnel equations are complicated; Schlick's approximation is cheap and close:

\(F_0\) is the base reflectivity, the reflectance at normal incidence. Here is Fresnel on a sphere: dark in the middle, where we look straight at the surface, brightening steeply towards the rim:

For dielectrics, \(F_0\) can be computed from the material's index of refraction. For metals that computation doesn't work, and a correct treatment needs a different Fresnel equation altogether. The practical way around this is to skip indices of refraction entirely and store \(F_0\) itself, measured or looked up, then let Schlick's formula interpolate from it towards white at grazing angles. The same formula then covers metals and dielectrics alike. Measured values are collected in tables such as the one in Naty Hoffman's course notes; a few of them:

Two patterns stand out. Dielectrics reflect very little head on: diamond's 0.17 is unusually high, and most sit between 0.02 and 0.05. Metals start far higher, mostly between 0.5 and 1.0. And only metals are tinted. A dielectric reflects all wavelengths about equally, so its reflections are white; a metal reflects some wavelengths more than others, which is why gold reflections are gold. That is why \(F_0\) is an RGB triple rather than one number.

This split is what the metallic workflow is built on. A material gets one extra parameter, metalness (we call it metallic), saying whether it is a metal. Physically the answer is yes or no. In practice renderers accept anything from 0 to 1, because a texel often covers a mix: a metal sheet with a little dust or paint on it is neither wholly one nor the other.

With metalness in hand, \(F_0\) needs no separate parameter. For dielectrics we use 0.04, an average that looks right for most of them. For metals, \(F_0\) is the coloured reflectance, and since a metal has no diffuse colour, we can store it in the albedo. Metalness blends between the two:

// Dielectrics reflect about 4% head on; metals reflect their own colour.
float3 f0 = lerp(0.04.xxx, albedo, metallic);

lerp is HLSL's name for GLSL's mix, and 0.04.xxx swizzles a scalar into a float3. The Fresnel function itself is one line:

float3 fresnel_schlick(float cos_theta, float3 f0)
{
    return f0 + (1.0 - f0) * pow(saturate(1.0 - cos_theta), 5.0);
}

cos_theta is the dot product of the halfway vector and the view direction (or, as we will see in the IBL chapters, of the normal and the view direction). saturate clamps to 0..1; floating-point error can push 1.0 - cos_theta a hair below zero, and pow of a negative number is undefined, which shows up as black speckles.

Cook-Torrance reflectance equation

Put the Cook-Torrance BRDF into the reflectance equation and we have the complete model:

There is one redundancy left. \(k_s\), the reflected fraction, is exactly what Fresnel computes, and \(F\) already appears in the specular term. Multiplying by \(k_s\) as well would count Fresnel twice. So \(k_s\) is dropped, and \(F\) inside the specular term does its job:

\(k_d\) is still derived from \(F\), as \(1 - F\), and one more factor goes in for metals: they refract nothing back out, so their diffuse share is zero however much light Fresnel leaves over. In our shaders that is a single line:

float3 kd = (1.0 - f) * (1.0 - metallic);

This equation is what people generally mean when they say PBR. If it isn't yet clear how it becomes code, that is what the next three chapters are for: Lighting evaluates it for four point lights, and the two IBL chapters evaluate it for an entire environment.

Authoring PBR materials

The model takes a small set of physical inputs per surface point, and an artist provides them as textures, one per property, so each texel can be its own material. A typical PBR material is five maps:

• Albedo gives the surface colour of each texel, or, where the texel is metallic, its \(F_0\). It resembles the diffuse textures we used in Lighting maps, with one important difference: it holds no lighting at all. Diffuse textures often have shading painted in (darker cracks, soft shadows), which an albedo map must not: the renderer computes all of that.
• Normal is the same normal map as in Normal Mapping, giving each texel its own normal so a flat surface looks bumpy.
• Metallic says, per texel, whether it is metal. Depending on the engine artists paint it as greyscale or as pure black and white.
• Roughness gives the roughness per texel, which sets the spread of the microfacets: sharp reflections where it is low, broad blurry ones where it is high. Some engines take a smoothness map instead, which is just one minus roughness.
• AO, ambient occlusion, is an extra darkening for places that ambient light has trouble reaching, such as the mortar between bricks. It is how crevice shading gets back in without being baked into the albedo. It is usually generated by a modelling tool rather than painted by hand. We computed it per frame in SSAO; a baked map does the same job for detail too fine for that.

Our lessons read these five files from a folder per material (resources/textures/pbr/rusted_iron/ and so on), named albedo.png, normal.png, metallic.png, roughness.png and ao.png. Only the albedo holds colours; the other four hold data. That makes a difference on the GPU, as the next chapter shows: the albedo is decoded from sRGB to linear as it is sampled, and the rest are read as they are stored.

This is the real payoff of PBR for the people making content. The values describe what a material physically is, which doesn't change with the lighting, so a material authored once looks right in every scene and in every PBR renderer that uses the same conventions. Artists can base values on real measurements, and nobody has to retune specular exponents because a level got darker.

Additional resources

• Background: Physics and Math of Shading, Naty Hoffman: the physics behind all of this, far deeper than one chapter can go, and the source of the \(F_0\) table.
• Real Shading in Unreal Engine 4, Brian Karis: the model these chapters implement, including the choice of D, F and G and the roughness remappings.
• Physically Based Shading, Krzysztof Narkowicz: an interactive ShaderToy that shows each term of the BRDF on its own.
• PBR Theory, Marmoset: an introduction written for artists, and a good read for programmers too.
• Physically based rendering and Cook-Torrance at Coding Labs: the rendering equation and the Cook-Torrance BRDF, derived step by step.
• LearnOpenGL's PBR Theory: the OpenGL original of this chapter.

Exercises

1. In pbr.hlsl of the next chapter's lesson, make ps_main return only the normal distribution term, distribution_ggx(n, h, roughness) for one light, as a grey value. Compare the spheres across the grid with the picture in the normal distribution section.
2. Do the same for geometry_smith and for fresnel_schlick, so you have seen each of D, G and F on its own. Which of the three changes with metallic, and which with roughness?
3. Replace the constant 0.04 in the f0 line with water's 0.02 and with diamond's 0.17. Which rows of spheres change, and why do the top rows not change at all?