Postprocessing Effects

I've been experimenting with full-screen postprocessing effects in 3DWorld using the fragment shader on the GPU. The shader has access to the color and depth buffers from the current frame, and overwrites the color buffer before displaying it. Some of the effects worked out well, but others did not. Here is a list of what effects have been added (in no particular order). This is a long post, but it's mostly images and videos and not too much text.

Fog
This is a very simple and common shader effect. It's also very cheap, so it doesn't affect the framerate much. However, I'm not really using it as a full-screen postprocessing effect. Instead, it's added as a postprocessing effect at the end of the regular draw shaders for all objects.

I'm doing more than the standard fog here, where the standard fog technique uses the depth (distance from the camera) of each pixel to select the fog color from no fog near the camera to full gray fog in the far distance. The 3DWorld fog is actually computed by ray marching a few steps through the scene and using the precomputed indirect lighting at each step to determine the fog color. Brightly lit areas have light gray fog, and dark shadowed areas have black fog. That way, the basement has black fog rather than the normal white-ish outdoor fog. For performance reasons, I'm doing a few ray steps, and only for some object types. It looks much better (with smoother color transitions) with more steps, but it's slower. Here are two screenshots for comparison.

Without Fog

With fog

God Rays
God rays, or crepuscular rays, are shafts of sunlight that hit small particles in the air, producing a glowing effect. Many games use God rays to increase the realism of bright direct lighting. 3DWorld uses a common approach of sampling the color buffer, ray marching from the sun position outward in screen space, and using the depth buffer to occlude the smear of color. This gives bright lines that radiate outward from the sun between other objects that block it. I have it working for scene objects, trees, and other solid objects, but not clouds (which don't contribute to the depth buffer). Here is another pair of screenshots showing the effect.

Without God rays

With God rays

Here is an image of God rays and fog in tiled terrain mode, looking through some trees. Note how the sunlight shines though the backs of the leaves as well. Real leaves are partially transparent to light.

A foggy morning, with God rays showing through the tree leaves. Also note the sunlight illuminating the backs of the leaves.

Another view. This time I remembered to turn off the onscreen display text.

Bloom
Bloom is a type of HDR (High Dynamic Range) postprocessing effect that makes bright objects appear even brighter than maximum monitor white by bleeding their color into surrounding pixels. If a light source is only a few pixels wide, the addition of a halo around it makes it seem brighter. This works well for small light sources that are only a few pixels wide, where the max white RGB value of {255, 255, 255} is not enough.

I'm using a 9x9 X/Y separable filter kernel (also used by the DOF and underwater blur effects). The color blur is done independently in X and Y using a 9x1 filter and a 1x9 filter. This requires 2*9 = 18 color texture lookups rather than 9x9 = 81. This is much faster and looks almost as good.

I think the bloom effect works well for small local light sources, but I don't like the effect on the sun and clouds. It's not clear how the fragment/pixel shader knows which pixels are sun vs. other light sources. Right now it's only testing for color intensity greater than 75%. Here are two comparison screenshots.

Without bloom

With bloom. The sun and sky are very bright! The courtyard lights are also bright. (but why are they on in the daytime?)

Heat Waves
I added screen space vertical heat waves as a postprocessing effect when the player is near a heat source such as fire. This is just a simple vertical wavy line effect on the entire screen, with decreasing wave amplitude from bottom to top. The shader doesn't actually know where the fire is on the screen, or if it's even visible in the view frustum. All it can do is add the effect or not. Here you can also see the billboard smoke particles effect. Smoke is colored based on lighting and shadows, so the smoke near the fire appears more reddish.

Heat waves rising from the fires. Don't get too close or you'll be burned!

Depth of Field
Depth of Field (DOF) is a common effect used in many games. The general idea is that the player is focusing on some particular object at a particular distance from the eye/camera. Objects both nearer and further than the object are out of focus and appear blurry. This simulates focusing of the human eye and of some types of camera lenses. Personally, I don't like this effect in games. I don't like anything that makes the scene look blurry - better to have sharp, crisp images. I added this effect to 3DWorld because it looked neat, but it's not used in-game. Here is what it looks like - click on the image for a larger view.

Depth of Field test image. The camera is focusing on the lamp in the center of the screen (in the white circle).

Here is a DOF video. I hope the blurriness of the video due to image compression artifacts doesn't mask the effect.

Underwater Blur and Distortion
Underwater blur is used to increase the realism when the player is underwater, and make it more obvious when the water boundary is crossed by the eye/camera. A wavy distortion is also added when the player is out of oxygen and drowning to simulate dizziness. When you see this effect, you know to get out of the water fast! I originally came across this effect through random experimentation, and I wanted to use it for something. It would make a great drunk effect if I ever add alcoholic drinks to 3DWorld. Here is an example of this rather strange effect.

Underwater drowning effect. Or maybe drunken effect? Watching this sure makes me dizzy.

Screen Space Ambient Occlusion
Not all of my postprocessing effects experiments were successful. SSAO was actually my first attempt at a postprocessing shader, but I should have started with something simpler. It works by calculating light occlusion in screen space using the depth and normal information, and then darkening screen pixels based on the amount of occlusion. That way, small folds and crevices appear darker than open areas. It doesn't work perfectly: for example, occluders that are off-screen or blocked by other objects can't be used.

This effect never really worked correctly in 3DWorld, for a number of reasons. First, 3DWorld uses a forward rendering pipeline, not a deferred rendering pipeline, so there is no normal buffer. Without object normals, it's impossible to tell front faces from back faces and which faces point toward each other. This results in incorrect darkening of areas that aren't actually occluded from the light source.

Second, SSAO doesn't really work with transparent objects. If the transparent object such as a glass window writes to the depth buffer, the front of it is darkened as if it was an opaque wall, which looks wrong. If it doesn't write to the depth buffer, the ambient occlusion isn't blended correctly and the dark area appears in front of the window, which also looks wrong. Here is a pair of screenshots with and without SSAO. The effect is subtle, so you probably need to click on the bottom image to get a higher resolution view.

Without SSAO

With SSAO. Not the intended effect. The color banding, black outlines from the transparent windows, false occlusion, etc. ruin the realism of the scene. Oh well, it was worth a try.

I'll see if I can think up any additional full-screen postprocessing effects.