Building Interior Cube Map Reflections

This post will be a bit different. I haven't added any new building, room, or object types. Well, technically I did add few new object types since the last post, but that's not what I'm going to talk about here. I've finally gotten around to adding cube map reflections for metal objects in 3DWorld's procedural buildings. I've been wanting to do this for a while now.

Reflective metal was one of those tasks that was easy to get the basics working but very difficult to make good looking and fast. I started from the existing cube map reflection support I had in my shaders and C++ code from all the way back in 2016: Part I, Part II, and Part III. This works by drawing the scene six times, once for each face of the cube. Each face uses a different view direction, and the total of all faces covers all 360 degrees of view. I limited drawing to the interior of the current building and the exterior of nearby buildings in the same terrain tile to save runtime. The initial reflection image is colored a light blue to match the day time sky, and will transition to a dark gray at night time. The reflection image does include dynamic objects such as people and animals in the current building.

One of the most difficult steps was determining where to put the origin of the reflection cube. It's not as simple as rendering an environment map (or skybox) where the background can be considered as infinitely far away. This had to work for building interiors, including high aspect ratio rooms and complex non-convex room shapes. I also had to use the same cube map for multiple reflecting objects at different angles and distances from the camera. I originally wanted to sample the surroundings from the center of each room and only update the cube map sparsely when the player moved to a new room to improve performance. However, this didn't work for big open areas such as malls where the center of the room was very far away from the camera and reflective objects. And in some cases the center of the room was inside an object such as an elevator.

I eventually gave up and used the camera position as the cube map origin. Since this is inside the player's head, I wasn't able to draw the reflection of the player model because it would have been inside-out. This also forced me to update all of the cube faces any time the player moved. One additional advantage of using the player position is that it should have never been inside a building object. Except, of course, for elevators and stairwells that the player is also inside and which have proper interiors to draw.

The fact that the reflection follows the camera makes it look more dynamic, but it's not physically correct. This is more obvious when the player is standing very close to an object such as a wall and that object takes up most of the reflected image. I did apply parallax correction to my cube maps in the fragment shader, but that's not perfect because it assumes the cube map extends equally in all directions. This simply isn't the case when the player is standing next to a wall and looking at a reflective surface on the other side of the room.

The next step was to determine what objects to make reflective. I started with shiny materials that I was already flagging as metal. This was then extended to include textured metal with paint, scratches, and rust. I later added other specular surfaces including dielectrics (insulators/non-metals) and panes of glass such as windows. The basic theory and equations work for all of these object types, which means I can use the same cube map and shader code for everything. I'll go over some of these object types below and show a number of screenshots.

First we have the objects that are already loaded as 3D models and have metal properties in their material files. This includes objects such as door handles. I made the brass and steel handles reflective and left the black painted handles with the original white specular light reflections only. The complex curved surface helps to hide the inaccuracy of the reflections.

Reflective brass door handle. Yes, it's a mixture of a mismatched decal on the door texture and a 3D handle model.

Railings along the sides of stairs are also reflective. These use the same material as door handles. They're composed of cylinders in various orientations, which also hide the imperfections of the reflections in their curved surfaces. I really like the way railings turned out.

Reflective brass railings to match the door handles. The metal colors don't always match, though they happen to match in this house.

Moving on to office buildings, we have reflective elevator doors. Actually, there are quite a few smaller reflective objects in office buildings, but the elevator doors stand out because they're almost like perfect mirrors. The reflection isn't as crisp because I'm only using a 512x512 texture for each cube face as another method of improving framerate. This lower resolution leads to a somewhat rougher look compared to a mirror's reflection rendered at full screen resolution.

Elevator doors with mirror-like surface. There are lockers visible in the reflection, so this must be a school rather than an office building.

Malls have the most reflective surfaces. (Factories are probably second.) This includes escalators, railings, floor trim, security gates, round trashcans, ducts, and vents. The sides of escalators are another example of near perfect mirrors. I didn't like the look of mirror reflections in the ducts and trim though, so I added support for roughness/shininess using my existing code. Rough material surfaces use a combination of texture mipmaps and 2D blur to average together adjacent pixels in the reflected image. This is another way to hide the visual artifacts in the captured reflection images.

Mall interior with reflective metal objects such as escalators, trash cans, and ducts along the ceiling.

It took quite a bit of effort to create these cube maps efficiently in large open spaces such as malls and factories. The brute force approach of drawing the building interior six times (once per cube face) cut the framerate to less than half. I was able to optimize this through a combination of reducing the draw distance of 3D models in the reflection pass and skipping materials associated with small objects that don't contribute much to reflections. These small objects are mostly from the shelf racks in retail stores and include bottles, cans, vases, balls, flashlights, candles, shoes, and various similar items. The small objects usually aren't very visible in reflections, especially with the blurring of rough materials. In addition, retail stores don't have any large reflective objects. It was time consuming to add no_reflect support to the geometry drawing system and flag dozens of individual items as non-reflecting.

While I was at this, I also marked many of these items as not shadow casters. It doesn't look quite as nice visually, but this does appear to completely fix the problem with lag when the player turns quickly. In this case the shadow manager needs to generate shadows for multiple light sources that come into view each frame. Malls are full of both lights and shadow casting objects, so this is doubly expensive. Since there are more lights than there are shadow map slots, my system that caches and reuses shadow maps across frames doesn't really help here.

After a lot of work, I was finally able to reduce the cube map generation overhead to something reasonable. My main test case was standing at the end of a very long mall. The original framerate was around 260 FPS. The initial implementation of reflection capture dropped this down to about 50 FPS, but I was able to get it all the way up to 180 FPS. Note that this only applies when the player is moving or a person is walking within the reflection distance. The worst case I was able to find was a looking out the window of the retail room of a large office building with an attached mall. This was around 110 FPS without reflections and a bit under 100 FPS with reflections. My performance goal is 150 FPS minimum on the new computer, which translates to roughly 60 FPS on my old computer that I haven't used in over a year. That old computer is what I would consider the minimum supported hardware for procedural city mode.

At this point I only had support for reflective colored metals with variable roughness. This worked for many objects, but not for something like water heaters. They're far too shiny. Making them rough doesn't look much better.

Shiny metal, mirror-like water heaters in an appliance/plumbing store.

 

The fix was to allow metalness values in the full range between 0.0 and 1.0. This allowed me to support materials such as painted metal that have a reflective layer on top of a diffuse/matte layer. My solution isn't a physically correct multi-layer material model, but I think it looks good enough. For example, a 40% metal material applied to a gray albedo (base color) on the water heaters leads to this much more reasonable result. Note that the pipes are still very reflective.

Water heaters with a more realistic surface that's darker and less reflective.

This extends to non-metal (dielectric) materials as well. These are flagged as specular with an index of refraction greater than one but metalness of 0. These objects have a white Fresnel reflection that varies based on the view angle. We can see this in the toilets that are on the other side of the same appliance/plumbing store as the water heaters shown above.

Rows of shiny toilets in the same appliance/plumbing store. Even dielectric materials can be reflective.

Some of my 3D models have the materials setup correctly, while others don't. I have to put some effort into fixing this at some point. I already had to fix some of the clothing models that had index of refraction set to 1.45 for some reason, which made them look like plastic. It would have worked if I had a raincoat model!

The same effect can be applied to glass surfaces such as TV and computer monitor screens. Here you can see that each of the TVs on the shelf of this retail area reflect the environment. The strength of the reflection is a function of the view angle. The reflection disappears when the TV is turned on as the emissive color drowns out the faint reflection.

TVs on shelves have reflective glass screens.

Here is an example of a whiteboard in an office conference room. The conference table and TV on the back wall can be seen in the whiteboard reflection. One limitation is that this system can't draw reflections inside reflections. [I do have recursive rendering set up for my older scene (in the blog posts from 2016), but that only works because each object has its own cube map. Here I only have one cube map that doesn't support reading from itself while writing to itself.] In most cases, reflective objects aren't really positioned in a way where this is obviously wrong, so I don't feel the need to fix it.

Reflective surface on an office building conference room whiteboard. The TV on the opposite wall is also reflective. So is the window on the side of the room.

Finally, we have glass found in windows, floors, and tables. Glass is technically both reflective and refractive, but the refraction term is only approximated with alpha blending. However, the reflected rays are in fact bent based on the index of refraction of the glass. This is basically the cube map equivalent of my planar reflections for retail glass floors. Sorry, I don't have a blog post to reference for that feature. The screenshot below shows a faint reflection of the mall lights and support pillars in the glass of this pet store. Some of the metal objects in the pet store are reflective as well.

This pet store glass window has a faint reflection. All glass surfaces have an index of refraction and proper Fresnel reflections. I turned the lights off in the pet store because it was easier to see the reflection in a dark window.

Here is a short YouTube video that shows off many of these reflective surfaces. Unfortunately, the YouTube compression makes it difficult to see many of the fine details. This is why I included so many high resolution screenshots above. I do need to add a video to show the way the reflections change as the player moves.

The logical next step is to make this same system work for exteriors as well as building interiors. In particular, I would like to make the exterior walls of metal and glass city office buildings reflective. That should be possible, since exteriors use the same base shader.

As a final note, I passed the 200K lines of C++ code milestone for the 3DWorld project!