Sixth and Eighth: The Graphics of Halo 3

April 21, 2016

3-HDR (High Dynamic Range)

Just what is HDR?

HDR stands for high dynamic range. It is a somewhat vague term which refers to the ability to handle a wide variation in magnitude.

BrightnessVariationVibrant sunlight and dim caverns, all on-screen at once!

Imagine that you want to measure weight, and you grab a scale, only to discover that it has a single-digit readout in pounds. Place a brick on the scale, it might read 5lb. Replace the brick with a feather, and the scale doesn’t register anything, reading 0lb. Replace the feather with a big old CRT television, and the scale is over-range (assuming it didn’t break).
The largest value this scale can measure is only one order of magnitude greater than the smallest non-zero value. A scale salesperson would have a hard time arguing that it’s a very “HDR” scale.

For a computer trying to render a frame, storing the color of a pixel is a similar situation.

Pixels and Color

Once it makes it to the television screen, the color of a pixel is determined by a combination of the intensities of 3 different color channels: red, green, and blue. On the original Xbox, each color would typically be stored as an 8-bit integer. With 8 bits, you have 256 possible combinations, so you can specify each color as a value from 0 to 255.

Untitled
Some example RGB colors with an 8-bit integer for each color channel

The greatest brightness a color channel can have on-screen is its “white level”, which in this case corresponds to 255.

Color and Brightness in Rendering

During the process of rendering, original Xbox games usually devote 32 bits to the color of each pixel: 8 bits for red, 8 bits for green, 8 bits for blue, and 8 bits for an “alpha” channel that can be used as additional information for various graphical techniques.

RGBA8
To an extent, this works fine. However, we’re talking about television screens that do not have very high contrast. In order to have a useful and pleasant image, it’s often the case that a lightbulb and the Sun are both white on-screen, even though the Sun is vastly brighter in real life.

This becomes an issue with post-processing. For example, when bloom effects are applied to an image, the Sun should flare up much more than the light bulb. But if both the Sun and the light bulb are (255,255,255), there’s no way for the post-processing algorithms to make the distinction.

LDRScenePreBloom
Suppose that the square on the right is white, and the square on the left is supposed to be bluish and much brighter, but has been stored as white because the blue channel can’t go higher than 255. Then, when we calculate bloom, we
want to get…

Bloom
…this, but since the bloom calculation thinks that they’re both white squares, we just get…

BloomDesaturated
…this, which has the blue square appearing desaturated, and no brighter than the other square.

The solution is to render into a format that can keep track of colors brighter than white level. An HDR format.

HDR Formats

An easy approach: what if we just redefined our format so that 63 was white level? 255 would then be about four times brighter. The image would look dark if viewed normally (since everything that we would normally render as 100 would now be 25, etc), but we could just quadruple the brightness of the image after post-processing to produce our finished frame.

DimLDRScenePreBloom
The previous scene rendered much dimmer. Now our data shows that the left square is brighter and bluer than the right square…

DimBloom
…so after post-processing, the bloom is brighter and bluer on the left square. At this point we’re done with the scene, so it’s time to scale it back to its intended brightness…

DimBloomScaled
…giving us something that looks similar to the desired result.

However, we just sacrificed color precision within the normal range. If white level is only 63, then we only have 64 steps from black to white for each color channel. This means that our final images will be prone to color banding.

Bloom
Here’s the original desired result. Note the much smoother gradients as the bloom falls away to black.

gradCompareA greyscale comparison of 256 brightness steps on the top half, to just 64 brightness steps on the bottom half.

Using more than 8 bits can help solve the problem. For example, if each color channel has a 16-bit integer, it can store a value from 0 to 65535. White level could remain 255, and you could keep track of things up to 256 times brighter, a very healthy HDR range.

The Xbox 360 and Halo 3

Aiming to use lots of brightly shining objects in high-contrast scenes, Bungie wanted a fairly wide HDR range for Halo 3. The Xbox 360’s GPU can render to a variety of formats, not just the old 8-bit-per-channel RGBA.

A common HDR format on 360 is “7e3”. Each color channel uses a 10-bit floating point number, with two additional bits used for alpha. In total, the same number of bits are used for each pixel as the classic RGBA8 format, 32. Bungie would go on to use 7e3 for Halo Reach. Here, they were able to track colors up to 8 times brighter than white level without problematic banding; good, but not the range they wanted for Halo 3.

ReachSword
Reach has vibrant elements, but an overall more subdued pallet than Halo 3

Instead, Halo 3 actually leverages the classic RGBA8 integer format in an unusual way. When Halo 3 finishes calculating a pixel’s color, it spits the result into two buffers: a “normal” RGBA8 render, and a second RGBA8 render that is much dimmer. Because it is so dark, the second render can distinguish between brightnesses up to 128 times brighter than the white level of the “normal” render.

Halo3FormatHalo 3’s format, approximately. On the left, a standard RGBA8 render of a sphere reflecting a very bright light, with an area that’s blown out to white. On the right, a dimmer render that lets us distinguish brightnesses in the bright area.

Because the GPU has to write each pixel to both buffers, the output fillrate is essentially halved. Since Halo 3’s expensive lighting is likely generating pixels slowly anyway, it’s possible that this doesn’t actually bottleneck things that badly.

Two RGBA8 buffers makes for a whopping 64 bits per pixel. The 360’s GPU renders into a 10MB memory pool, and it’s convenient to be able to fit the entire framebuffer (plus some other stuff) into that pool simultaneously. At high resolutions, 64 bits per pixel would make this problematic. Halo 3’s low resolution helps keep this practical.

Results

Halo 3’s post-processing, especially the bloom, can smoothly handle huge brightness variations without clamping or otherwise having strange color issues.

Although the slowness of writing into two buffers potentially doesn’t slow down expensive lighting passes very badly, it’s probably more problematic for effects. Even if a transparent layer is set up to use very cheap lighting, its pixels will output slowly to the dual buffers.
Furthermore, Halo 3 doesn’t use any methods for rendering transparent effects at lower resolution than the rest of the imagery. Everything is full-quality.
So, if you want to render multiple huge transparent layers to the screen in Halo 3, there isn’t a way to do this that won’t be massively detrimental to performance.

Fortunately, the deep HDR can help effects look lively with fewer layers.

For example, the fragmentation grenade uses one transparent layer for its main blast. Normally this would look “flat,” but the layer has an ultra-bright yellowish region in its center, which the bloom turns into a nice fireball/flash effect that smoothly contours against objects in the scene.

GrenadeFlash
Flashy fireball that smoothly wraps around objects in the scene, accomplished with one textured layer thanks to Halo 3’s post-processing.

The HDR can also help with things like the low number of dynamic lights. While not as convincing as “real” light sources, glow from a bright texture can look adequate for some purposes and is “free.”

TextureGlow
Blue glows on the elite, and green glows on the assault rifle, looking reasonably good just from rendering bright spots on the objects

Advertisements

Pages: 1 2 3 4 5

4 Responses to “Sixth and Eighth: The Graphics of Halo 3”

  1. Halo 3 is great. I wonder if modern games match the HDR. They don’t look like it usually… UE4’s lighting seems very high range. maybe its just the tonemapping.

    • Bungie’s Lighting and Material of Halo 3 presentation indicates that if the 360’s ROPs had better support for FP16, they would have used it and had massively higher dynamic range for the same memory footprint. Modern GPUs have good support for FP16 output (compute shaders have no trouble with direct FP16 access, and the ROPs have robust single-cycle FP16 capabilities), and the ideas behind PBR encourage mimicking realistic range… I’d be surprised if modern games don’t often beat Halo 3 by a large margin.

  2. Wow, this was really great! I learned a lot about from this. You did a great job explaining the different types of lighting, HDR, and the implementation of water and physics. Looking forward to more articles!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: