Sixth and Eighth: The Graphics of Halo 3

April 21, 2016

2-Lighting

What constitutes Lighting?

What happens when light strikes a point on an opaque surface?

Some of it might penetrate the surface, get rattled around by particles within the surface, and exit scattered in all directions. This sort of thing is called diffuse reflections.

DiffuseScatter
Extremely high-fidelity depiction of a diffuse reflection of a ray of light

 

DiffuseDirectionalDiffuse reflection from a directional light striking a smooth sphere

On the other hand, some of the light might glance off the surface, shooting off more pointedly. This is specular reflections. A very smooth surface with strong specular reflectance can look like a mirror, where if light rays are arriving from the same direction, they glance off into the same direction as each other.

SpecSurfaceSmoothDepiction of specular reflections of rays off of a smooth surface

Specular reflections from a rough surface will be less sharp, since rays coming from the same direction will bounce off in different directions, depending on the orientation of the surface where they strike.

SpecSurfaceRoughDepiction of specular reflections of rays off of a rough surface

CTSpecWithRoughness
Specular reflection from a directional light striking spheres of varying roughness

Video games tend to break their lighting into these two components: diffuse and specular. When the hardware is trying to decide how a pixel on surface looks when being struck by a directional light, it might take into account a variety of values: the direction the surface is facing, where the camera is viewing the surface from, surface properties such as color and roughness, and properties of the light such as brightness and color and the direction it’s coming from.
Plug those values into a formula for diffuse lighting, and you get the diffuse reflection. Plug them into a formula for specular lighting, and you get the specular reflection. Combine the two in a shading formula, and you get the color of the lit pixel.

RedBallShading
How diffuse and specular reflections from a directional light striking a red sphere might be combined

Expressive Specular

Halo 3 was going to focus on having lots of nice shiny surfaces. To be shiny and have cutting-edge lighting, an obvious focus is the quality of the specular reflections.

There are a huge number of formulas out there for modeling specular reflections.

Historically, in real-time applications, the model called Phong (and its modifications like Blinn-Phong) have been popular.
What is Phong? In a nutshell: if you think about how specular reflections behave, and try to come up with the simplest possible formula that sort of models them, Phong is what you get.

PhongSpecWithRoughnessPhong specular from a directional light striking spheres of varying roughness

Cook-Torrance is another specular model. It is a “microfacet” model. That is, it assumes that surfaces are made up of tiny microscopic “facets” which face in different directions; a rougher surface is one that has greater variation in the directions the facets face. The facets can do things like shadow and occlude each other. The overall behavior is described in a formula with 3 terms:
1-A distribution term (D) which describes how the incoming light is reflecting off the microfacets and getting to the camera, without considering factors like shadowing between facets.
2-A geometry term (G) which describes how the microfacets shadow themselves and each other, from the point of view of the camera.
3-A Fresnel term (F). Consider the appearance of a calm lake. If you look toward the far side, you can see distant objects reflecting off the water as if it were a mirror. But if you look straight down into the water, you can make out things below the surface. Light striking a surface at glancing angles tends to be more likely to glance off the surface than pierce through/into it, resulting in stronger specular. The Fresnel term accounts for this effect.

CookTorranceTerms
The various terms of Cook-Torrance and the result, from a directional light striking a sphere

CTSpecWithRoughness
Cook-Torrance specular from a directional light striking spheres of varying roughness

As a reflection model, Cook-Torrance is more accurate to the behavior of real-world light than Blinn-Phong, and produces results with a more complex appearance.

testPhong1Phong reflection of directional light rotating around a sphere

testPhong2Blinn-Phong reflection of directional light rotating around a sphere

testCT
Cook-Torrance reflection of directional light rotating around a sphere

Classically, Phong was popular for real-time applications, while Cook-Torrance and similar models were rarely considered. The reasoning is simple: processing (and sometimes memory) requirements are typically much higher for Cook-Torrance. Today, many games use microfacet models, including a variation on Cook-Torrance called GGX, but this change didn’t begin to gain traction until the launch of the eighth-gen consoles neared.

At any rate, Bungie elected to use Cook-Torrance for Halo 3.

Complex Light Sourcing: Area Lighting

A charged shot from a plasma pistol has light emanating in all directions from a point; a warthog headlight sends light emanating in a cone from a point. Most of the light sources in game are point sources like these.

However, an object sitting in a real environment is receiving light from all directions at once. If you’re standing outside on a clear day, you might be receiving blue light scattered everywhere from a wide stretch of sky, green-tinted sunlight bounced up from all around on a grassy field, red light from a nearby parked car, and direct light from the Sun itself. Capturing all this at once can be considered area lighting.

AreaLighting

The classical solution to this problem is a cubemap, a 360-degree render of the scene from a point. If an object is standing near where a cubemap was rendered from, it can be used to give an “accurate” reflection of the environment.

CubemapCenter
Here, the cubemap reflection on the visor aligns well with the game world

However, the farther something is from where a cubemap is rendered, the less accurate it is.

CubemapEdge
Here, the cubemap reflection in the visor is obviously offset from where it was rendered

Furthermore, decently high-resolution cubemaps tend to consume a lot of memory, and the Xbox 360 has only 512MB available, so placing tons of them everywhere was out of the question; while Halo 3 has single cubemaps assigned to large areas (as shown in the images above), they’re not used as an alternative to smaller-scale lighting data.

Halo 3 uses a pre-computed (baked) lightmap for environmental lighting. What if every point in that lightmap was like a very low resolution cubemap, so that at every point you knew the full 360-degree distribution of incoming light at that point?
This is essentially what Bungie chose to do. However, instead of cubemaps, they used Spherical Harmonics (SH), an alternative means of encoding a light distribution. This allowed them to store a smooth (albeit very blurry) light distribution in just four numbers per color channel. That is, at every point in the lightmap, there are four numbers describing the distribution of incoming red light, four numbers describing the distribution of green light, and four describing the distribution of blue light.

Using SH distributions in diffuse lighting calculations is practical, and quite common in games. One exceptional aspect of Halo 3’s environment lighting is that it computes the Cook-Torrance specular response to the SH lighting data, a very costly computation to provide subtle benefits.

GreenHallwayDirectional effect of lightmap near green hallway.

How does this work with Halo 3’s cubemaps? To address the issue of cubemaps being inaccurate away from their render locations, Halo 3 adjusts the strength of the cubemap according to the SH specular. For instance, if there’s a glowing green wall to your right, the cubemap will become greenish in that direction. If there’s a sunlit red turbine to your left, the cubemap will take on some amount of red tone from your left.

CubemapColorSame cubemap on the assault rifle, different color due to different area in lightmap

This way, a cubemap assigned to each area provides fine details that convey the gist of that area, and the SH specular adjusts the cubemap’s strength and color so that surfaces are overall lit plausibly for any point. It’s not as accurate as having sharp lighting information at each point, but objects can “sit better” in their environments than if the cubemaps were being used by themselves, and the cubemaps still provide overall area-correct flavor.

GI
The Master Chief stands in a spot of shadow. The windows above create a wide white reflection from above. A ray of sunlight striking the top of the turbine provides a dim red glow from below.

Additional Environment Lighting

The SH lightmap is far too blurry to capture sharp directional lights like the Sun. In addition to the SH data, Halo 3’s lightmap contains an extra value to describe the strength of the dominant directional light at every point. Reflections from this dominant light source get calculated separately from the SH.

DominantSunlight
Here, the Sun is the dominant light

This is useful for the large number of environmental point light sources baked into Halo 3’s light maps. Even though the main lightmap data is blurry, you can get sharp light sources, albeit only one for any given point in the lightmap.

DominantInteriorLight
Here, an interior light is the dominant light. Note same shape of reflection on assault rifle as before.

Although the intensity of the dominant light is specified in its own number in the lightmap, the direction of the dominant light is actually derived from the SH. This direction is also seemingly used to decide which direction a dynamic object’s shadow will be cast.

Shadows

Because the lightmap is used for all environmental light – not just bounced lighting, but also direct light – Halo 3 does not need to render shadows for static objects. Shadows cast by static objects are just dark areas in the lightmap. This reduces shadow rendering costs, and it avoids the shimmering and crawling that plagues real-time shadows.

EnvShadows
Some environment shadows in Halo 3

However, there are two drawbacks to baking shadows into the lightmap.

First, shadows for dynamic objects need to be rendered separately, and they tend to combine poorly with the environmental shadows.

ShadowInteractionBad
The shadow from the building and the shadow from the character should combine into the same shadow, but instead stack on each other

Second, the textures that make up Halo 3’s light map are not very high-resolution. The baked environment shadows can sometimes have a chunky appearance when they’re representing complex shadows.

DamTurbineChunky
The fine shadows from the window gratings aren’t captured entirely smoothly in the lightmap

Results

Well, to state the obvious: Halo 3’s lighting choices were very computationally expensive, each pixel far from cheap. Given the decisions that were made, it’s not shocking that the game renders at 1152×640, lower than most 360 games.

Environmental Lighting

To consider the overall environmental lighting model, look at a warthog exiting a cave on Tsavo Highway.
First, it’s dark. Then you get directional yellow lighting coming from the cave entrance, sunlight bounced off the dirt outside. Then, outside but in shadow, the warthog takes on a soft blue reflection of the wide sky overhead. Then the warthog is hit by strong directional sunlight.

Dynamic Lights

Halo 3 does not like to handle huge numbers of dynamic lights, in the sense that many small sources (like projectiles) do not cast light. It also usually avoids using lights quite as large as those in Halo 1, although this is probably a good idea for any game intending to avoid serious framerate spikes.
However, Halo 3 is pretty good about keeping dynamic lights consistent in quality. You don’t run into big lights that are missing specular reflections, for instance.

There is a caveat to the above, however: on some surfaces, relative to the environmental lighting, dynamic lights seem to cast softer specular or even no specular at all.

Inconsistent
On the left, just environment light. On the right, flashlight is on; the yellow railing doesn’t have much of a specular highlight, despite being shiny in the left image

Materials

Halo 3’s materials do not tend to look very accurate to real life. However, materials which are not very translucent – such as metals, opaque plastics, rocks, etc – can look interesting, complex, and quite snazzy.

Cook-Torrance describes how light bounces off of a surface, but does not describe how it scatters while travelling through a translucent material. There’s some special-purpose functionality for water (more on that later!), but otherwise Halo 3 does not have a general way of handling translucent objects. For example, backlit foliage doesn’t glow.

FoliageBack
Leaves reflect the flashlight, but no light is shining through the vertical leaves

Of particular note is skin. Halo 3 has a very complex lighting model for its day, but the complexities of human skin – peach fuzz, the glossy oily surface layer, subsurface scattering – are not modeled well, or at all. Getting skin right in a high-fidelity game world was a frequent problem of the era. The results can look simultaneously flat and waxy.

Keyes

Advertisements

Pages: 1 2 3 4 5

4 Responses to “Sixth and Eighth: The Graphics of Halo 3”

  1. Halo 3 is great. I wonder if modern games match the HDR. They don’t look like it usually… UE4’s lighting seems very high range. maybe its just the tonemapping.

    • Bungie’s Lighting and Material of Halo 3 presentation indicates that if the 360’s ROPs had better support for FP16, they would have used it and had massively higher dynamic range for the same memory footprint. Modern GPUs have good support for FP16 output (compute shaders have no trouble with direct FP16 access, and the ROPs have robust single-cycle FP16 capabilities), and the ideas behind PBR encourage mimicking realistic range… I’d be surprised if modern games don’t often beat Halo 3 by a large margin.

  2. Wow, this was really great! I learned a lot about from this. You did a great job explaining the different types of lighting, HDR, and the implementation of water and physics. Looking forward to more articles!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: