Friday, October 2, 2020

Building Mirror Reflections

This time my post is not about objects I've added to building interiors. Well, technically that's not true. I have added mirrors to office and house bathrooms, but most of the content isn't about mirror placement because that was a trivial task. This post is about how I implemented mirror reflections, which also required me to finally add a player model.

I started by copying parts of my existing water plane reflection code. This was all oriented in the Z (up) direction, while I needed mirrors oriented in X or Y to place on walls. That required changing all of my code for reflection matrices and the camera frustum to work with different reflection directions, but fortunately I only had to handle the axis aligned cases.

Then I had to deal with the mirror changing the winding direction of all polygons in the scene, which affects the front vs. back face culling tests. I had to hunt down various face direction tests in the building drawing code and invert them in the mirror reflection case. I also had to fix problems with polygon bias direction in some of the passes. At first I didn't quite understand what was going on and fixed it using trial-and-error to flip signs.

The next task was to avoid drawing the other side of the wall behind the mirror. Mirror reflections work by moving the camera to the other side of the mirror and negating the view direction. This effectively puts the reflection camera outside the bathroom looking in through the back side of the mirror. But there's a wall behind the mirror, so the camera will be looking at the other side of the wall! I had to add a custom clip plane at the mirror surface to remove everything behind it, which would have been in front of the back of the mirror for the reflected camera.

Finally, I had to handle the case where multiple mirrors were visible in the same office building bathroom. This produced a reflection of the opposite mirror, which didn't work correctly because the opposite mirror wasn't drawn as it was outside of the player's view frustum. After various failed attempts, I decided to simply limit bathrooms to having a single mirror when reflections were enabled. That's good enough for now.

After all these steps (and more), I finally had it working:

Realtime reflection in a mirror in the men's room.

Unfortunately, it was pretty slow. My first implementation nearly doubled the frame time when a mirror was visible. Fixing this required some combination of selective drawing, object occlusion culling, screen space stencil testing, and visible surface determination. At first I drew the reflection at half screen resolution, which helped. That's why you can see a bit of blurriness if you fullscreen these images.

One of the easiest optimizations was to check if the mirror was in a windowless room, and if so, only draw the building the player is in. None of the other buildings can be seen (unless a window is visible across the hallway, in the reflection of the doorway). Some of the other outdoor scene content can similarly be ignored. This was easy to do, but didn't help very much because I was limited by the fragment shader lighting calculations rather than the vertex shader or OpenGL driver overhead.

Next, I had to determine when a mirror was visible to the player. If it's not visible then there's no reason to create the reflection image. First, it must be within the camera's view frustum. Then I added a check if the player was actually inside the room containing the mirror. That almost worked, except for the case where the player was in the adjacent hallway looking in through an open door. So I included the connected hallway in the set of rooms for which the mirror needs to be drawn. But the hallway could run the entire length of the building, while the mirror is only visible for a small subset of the hall. I added a check for distance to the room/doorway based on the room size and hallway width. That mostly worked, but there were still a few situations when it either drew the mirror unnecessarily, or failed to draw it when some part was visible. My final solution was to trace a number of horizontal rays from the camera to points on the mirror's surface and skip drawing it only if all rays hit a building interior wall. That solution worked pretty well, so I left it at that.

The last optimization was to render only the interior part of the mirror rather than the entire screen. When the player is relatively far from the mirror, this makes a considerable difference. The mirror only occupies a small fraction of the pixels in the images above and below. I used a stencil test for this. The mirror was drawn in the stencil buffer, and the downstream room draw calls all tested the stencil buffer for each pixel/fragment. That optimization was surprisingly simple and effective compared to the ray intersection optimization. With that change in place, I was able to go back to rendering the reflection at full screen resolution.

I also finally added a model for the player. I used whatever human person model is specified first in the config text file. It happens to be the model of an old man. If you look closely, he (we the player?) even casts a shadow on the wall behind him. The player now always casts a proper shadow, even if not in a room with a mirror. The legs of the shadows are even animated when the player walks.

A reflection of the player model in a bathroom mirror that also casts a shadow. Oops, I think this is the women's room because there are no urinals!

But really the player model can be anything specified in the config file. For example, it can be a car. Here I had to fly up toward the ceiling to show off my new "player" model because it's so low to the ground.

The player as a car, reflected in the bathroom mirror. Who says a car can't use the bathroom?

What other fun model can I use? Let me see. How about this one. Can you guess what it is from only its shadow cast on the bathroom floor?

The player's shadow. What do you suppose the player is?

I'm sure you guessed it was a toilet. What else were you expecting, right? After all, we are in a bathroom.

That's okay, I'm beautiful on the inside ... as long as you flush me first.

Next, I added mirrors to bathrooms in houses. This was more difficult because these bathrooms usually have windows that can be seen in the mirror. That means I have to draw the outdoor part of the scene with terrain, vegetation, exteriors of other buildings, etc. This was challenging because I was creating the reflection texture from within the building drawing code by clearing the depth and color buffers before drawing buildings. Some of the outdoor objects that had previously been drawn into the frame buffer were cleared in the process. It wasn't as easy as drawing the reflection to a different buffer (FBO or PBO) because I needed a color buffer, depth buffer, and stencil buffer. I also couldn't easily move the reflection creation to somewhere else in the drawing control flow because it depended on the room lights setup code at the top of the draw function. So far I haven't come up with a good solution to this problem.

In addition, my logic to swap front vs. back faces interfered with the multiple rendering passes used to cut windows and doors into exterior walls, so I had to go back and rework that. Eventually I was able to get some of this to work. Here is a screenshot of a small mirror above the sink in a house bathroom. This one was drawn at full screen resolution.

Small mirror above the bathroom sink. It's on a medicine cabinet and spaced somewhat away from the wall, though it's hard to tell from this angle.

I have 3DWorld drawing window cutouts of the current building so that the exteriors and windows of other buildings can be seen through them in mirror reflections. I don't have the interiors of other buildings or terrain, vegetation, or roads working yet. I'm not sure how to do that in a clean and efficient way. Moving the reflection texture creation out of the building drawing loop and higher in the control flow would partially fix this, but it may be more complexity than I'm willing to add at this time. For now we'll have to avoid looking into the mirror in such a way that an exterior window can be seen.

Also note that bathroom mirrors are only added to interior walls to avoid having them overlap windows cut into exterior walls. I could check for window intersections like I do with some other room objects, but I believe most of them would intersect. This probably wouldn't be worth the trouble. Maybe I should also avoid putting mirrors on walls opposite exterior walls to work around the other problems.


  1. Oooh, very neat. Yeah, you're really getting into the weeds here with the render pipeline.
    One approach you could consider for rendering exterior views, no idea if this would work, but it just occurred to me. Could you cache the rendered view out a window/mirror to a texture? Then you could just use that texture in the next frame for further windows/reflections. There would be a slight lag in update, like with video feedback, but it should allow infinite reflections and looking through buildings and out the far window, as well as allowing you to defer the refraction map for bathroom windows to a later time slot when that portion of the pipeline isn't busy drawing the next frame.

    1. Mirrors are already rendered to textures, but they're done before the main draw pass to avoid lag. I don't think it would help to defer them to the next frame. Draw time is mostly limited by fragment shader processing of all the light sources. The current flow clips the drawn area to the screen space of the mirror so that it can be done at full resolution but only for visible reflected pixels. I later added occlusion culling of models, and that helped a lot, so mirrors are almost free in terms of frame time now.

      I you mean caching across multiple frames, that doesn't work. I already tried it. The parallax effect is wrong and the player model won't show up properly. Or are you talking about caching the view out a window so that it can be drawn as a texture in the reflection? That would work, but some bathrooms have multiple windows, and that could get expensive very quickly. There's also a parallax effect involved that would look wrong because the window isn't a pinhole camera. As you move around within a room, you see a different view outside the window, and this also applies when looking through a mirror. Caching would require render to texture with a very large FOV and calculating the proper reprojected texture coordinates in the shader, which sounds complex and expensive. I doubt it would be overall better than what I have now.

      Note that the portal system I have for teleporters does allow for realtime infinite reflections if you place two mirror teleporters next to each other. The rendering pipeline is much simpler though, and a portal is generally small in screen space.

      I haven't figured out what to do with bathroom window refractions yet. Maybe that can use the same system, or maybe it needs something different. I can't even get alpha blending to work with bathroom windows.

    2. Huh. Seems like unifying your interior and exterior rendering code is the next big refactor. Then you could make every mirror a portal into the mirror world!

    3. It's not interior vs. exterior. It's game mode vs. tiled terrain mode. Cities/buildings are all placed in my infinite tiled terrain system. These are static objects with no physics/interactivity where the data is all sent to the GPU once and stored there. This is very scalable.

      Gameplay mode is for smaller limited bounds scenes. Since everything is generally physics enabled and destructible, it's mostly stored on the CPU side and the geometry is streamed to the GPU each frame. All of the objects are also kept in bounding volume hierarchies and sorted for back to front drawing to handle transparency.

      They're completely different systems. They use the same vertex/shader/framebuffer rendering system, but everything above that level is different. I'm not sure how it would be possible to unify them as their goals are very different. Right now the only code shared between teleporters/portals and building mirrors is the render-to-texture, matrix math, view frustum, screen space clipping, shaders, etc. The rest is different, including the higher level scene management, material system, and object batching.

    4. I guess it's similar to how you wouldn't necessarily use the same engine for an arena-like indoor FPS vs. a large open world exploration game.

      The indoor building lighting does work the same though. I simply create a finite bounds some distance around the camera and "clip that out" to put into the game mode flow that expects a small limited area.

    5. When I wrote "The rendering pipeline is much simpler [for portals] though," I didn't really mean the rendering pipeline. There are two things that are simpler for game mode and portals. #1, all of the objects and geometry is generated at scene load time rather than dynamically, so you don't need to worry about on-the-fly procedural content generation in the middle of the rendering.

      #2, the number of total objects in game mode is small enough that you can simply iterate over a big flat list to determine what to draw. In tiled terrain mode there are far too many objects to do this. Instead there's some nesting such as for each city => for each tile in that city => for each building => for each floor => for each room. So the scene graph traversal is much more complex, and expensive to perform multiple times per frame.