Cube Map Reflections

As promised in my previous post, I managed to get cube map environment reflections working in 3DWorld. It was more difficult than I expected, but in the end actually looks better than I expected. Part of the problem was that I kept running into graphics problems that were difficult to debug, such as cryptic OpenGL errors, shaders that didn't work, upside down images, incorrect lighting, half black images, etc. It's very difficult to debug graphics code, especially when you're trying to implement an entirely new system from scratch. There are some helpful tutorials out there such as this, but many of the online info is incorrect, outdated, or doesn't work on my OS/graphics card.

How do cube map reflections work? The scene is rendered for each of the six sides of a cube surrounding the object. Each cube side has a 90 degree field of view looking out from the cube center. This is similar to a 360 degree panoramic photograph created from 4 individual photographs, but in addition has the top and bottom faces that you would get by taking a picture of the ground and the sky. Together, the six faces generate an image that covers all possible directions from the center of the cube. Each image is created by rendering the scene once and caching the resulting image as a texture on the GPU. This is a slow process, but only needs to be done when the reflective object moves or something in the scene changes. The texture images are then used to look up the reflected color for each pixel of the object in the fragment shader. The reflected view vector is used to index into one of the pixels in one of the faces based on the ray direction.

If the background of the scene, the "environment", is sufficiently far away from the reflecting object(s), the same cube map can be used for the entire scene. This is called environment mapping, and the cube map textures form a skybox. It works well for the sky, clouds, and distant mountains, but not as well for building interiors. The problem is that the parallax looks incorrect when the camera moves around the object. Also, the reflections of nearby objects appear more distorted. Therefore, for the scenes I'm using in 3DWorld, I had to create a separate cube map for each reflective object.

I'm attaching a lot of screenshots to this post. In fact, I have so many screenshots that I'll need to split the post into two parts. Keep in mind that everything is rendered in realtime even when the player/camera are moving. The scene and reflective objects can move as well, though the frame rate drops to 40-60 FPS with one reflective object as the scene is rendered up to 8 times (normal scene, 6 cube faces, and reflective floor). Actually, I cheated a bit and skipped drawing of the cube faces that are oriented away from the camera, which isn't always legal (but mostly works for convex reflectors).

I initially put all reflective objects in the same place in the scene so that the screenshots can be compared with different materials. Later screenshots will show more variety. I really just want to get the materials and reflection vectors correct to start with. The office building scene is one I've shown many times before. I put the reflective object above the desk in the lobby. The floor is partially reflective tile, the ceiling is a grid of white and brown, and the walls are white with some tan. The windows and doors are mostly transparent so that the player can see outside.

My first test used a traditional perfectly reflective metal sphere, like a large ball bearing. Here is how it looks when viewed from up close:

Metal sphere with a perfect mirror reflective surface (pure specular reflection).

Later, I realized that the reflection vector was incorrect. I updated the metal sphere image since it was the most incorrect, but didn't update the others because the images changed very little. Here is the new screenshot, with a sphere that extends slightly below the desk so that you can see the reflection of the inside of the desk. I was moving it around and didn't get it back into the original position.

Reflective metal sphere with corrected reflection vector resting on (and partially inside of) a desk.

Note that the tile floor reflections are shown in the sphere reflections, though I don't think it's quite right. It looks like the floor is reflecting the sky instead of the building interior. I'll have to work on that later. [Update: I figured it out, it's a transform matrix problem, but it's not so easy to fix.] Also, the player has no model and therefore creates no reflection.

Next, I changed the metal sphere to a dielectric material. Here is a giant glass marble, using real material parameters (index of refraction, etc.) and a Fresnel reflection equation. Only the edges of the sphere that are viewed at a glancing angle are highly reflective. This is physically correct - try it yourself at home.

White reflective dielectric sphere (smooth marble).

Here is the same sphere, but this time it's black and viewed from the other side. The effect is similar, and there is an additional white specular reflection from the ceiling light near the top of the sphere.

Black reflective dielectric sphere (smooth marble).

I then added some dynamic light sources to test out reflections from dynamic objects. The addition of dynamic objects reduces the frame rate, but it's still realtime.

Black reflective marble sphere with dynamic light sources creating specular highlights.

There are 100 floating, moving, colored lights in this scene. Several of the nearby lights create specular highlights on the sphere, and some of the lights behind the player are visible as reflections. You can also see the reflection of the chair that's directly behind and below the camera, and the table with bottles on the left side of the sphere.

Okay, it's nice to have reflective spheres. But what about real objects? The objects I want to use in 3DWorld are more complex than simple spheres. One likely candidate for a reflective object is the gold Chinese dragon that I placed on the desk in the lobby. I've had that statue sitting there on the desk for months and every time I look at it, I think, "That should be reflective." Well, it finally is!

Reflective gold dragon statue (871K triangles).

The dragon model is 871K triangles, which shows that reflections can be applied to an object of high complexity. I have no idea what the reflections should look like here. Fortunately, I spent all that time making reflections correct for simple spheres and cubes, so I'm pretty confident that the dragon reflections are at least somewhat correct.

Wait, I didn't add any reflective cube images? Well, they're not very interesting, they're basically just mirrors. I did add a reflective cube that the player can push around the scene. I'll add a video of that to a future post.

Note that the reflective object is concave, but is not self-reflective. One part of the object is not reflected in another part of the object. I ran into too many problems with the object intersecting the near clipping plan and had to move the near plane out beyond the furthest vertex of the object. Maybe some sort of back face culling would have helped, if all of my models actually had the correct face orientations. That's too much to expect from free models that I found online.

Here is another view of the gold dragon, during night time with some dynamic floating lights. This is still rendered in realtime.

Reflective gold dragon with dynamic floating lights.

For some reason, only the green lights like to float near the dragon. Here is a closer view with at least one green light reflecting off the gold. Someone must have put a lot of effort into keeping this dragon shiny by cleaning it every day.

Closeup of gold dragon showing specular reflections from a floating green dynamic light.

The dragon statue material can also be changed. Here it is in white marble.

Ceramic dragon with Fresnel surface reflections from the environment.

 And here it is in black marble.

Black glossy ceramic dragon lit only by reflected specular environment lighting.

The white lines near the silhouette edges of the dragon are actually a form of specular aliasing. The view vector is nearly tangent to the surface at those locations, which makes the Fresnel reflectivity approach 1.0 (mirror reflection). In addition, the triangles in those locations come together at sharp edges with high frequency noise, so their normals change very quickly. This makes the reflected rays bounce around randomly. Many of the rays hit the bright white walls and ceiling, creating bright white spots (pixels) in those locations. I'm not sure how to remove those artifacts, or if they're real physical effects. I don't happen to have a shiny black marble dragon around for reference. I could clamp the max specular reflection value, but that would break the properties of my metal mirror surfaces.

I think the small white spots on the dragon are triangles that have incorrect orientation so that their normals have the wrong sign. I've seen those on other images but they really stand out with the shiny black material. I'm not sure how much effort I want to put into fixing this, considering it only occurs in this combination of material and model. I guess I better make the dragon white or gold in the final scene.

I think 10 screenshots is enough for this post. I want to make sure the page doesn't take too long to load for those of you on mobile devices or slow internet connections. I'll make a follow-up post with more screenshots of multiple inter-reflecting objects, reflections in other scenes, and possibly some videos. I'll also try to include a section on "future work".