Cube Map Reflections

As promised in my previous post, I managed to get cube map environment reflections working in 3DWorld. It was more difficult than I expected, but in the end actually looks better than I expected. Part of the problem was that I kept running into graphics problems that were difficult to debug, such as cryptic OpenGL errors, shaders that didn't work, upside down images, incorrect lighting, half black images, etc. It's very difficult to debug graphics code, especially when you're trying to implement an entirely new system from scratch. There are some helpful tutorials out there such as this, but many of the online info is incorrect, outdated, or doesn't work on my OS/graphics card.

How do cube map reflections work? The scene is rendered for each of the six sides of a cube surrounding the object. Each cube side has a 90 degree field of view looking out from the cube center. This is similar to a 360 degree panoramic photograph created from 4 individual photographs, but in addition has the top and bottom faces that you would get by taking a picture of the ground and the sky. Together, the six faces generate an image that covers all possible directions from the center of the cube. Each image is created by rendering the scene once and caching the resulting image as a texture on the GPU. This is a slow process, but only needs to be done when the reflective object moves or something in the scene changes. The texture images are then used to look up the reflected color for each pixel of the object in the fragment shader. The reflected view vector is used to index into one of the pixels in one of the faces based on the ray direction.

If the background of the scene, the "environment", is sufficiently far away from the reflecting object(s), the same cube map can be used for the entire scene. This is called environment mapping, and the cube map textures form a skybox. It works well for the sky, clouds, and distant mountains, but not as well for building interiors. The problem is that the parallax looks incorrect when the camera moves around the object. Also, the reflections of nearby objects appear more distorted. Therefore, for the scenes I'm using in 3DWorld, I had to create a separate cube map for each reflective object.

I'm attaching a lot of screenshots to this post. In fact, I have so many screenshots that I'll need to split the post into two parts. Keep in mind that everything is rendered in realtime even when the player/camera are moving. The scene and reflective objects can move as well, though the frame rate drops to 40-60 FPS with one reflective object as the scene is rendered up to 8 times (normal scene, 6 cube faces, and reflective floor). Actually, I cheated a bit and skipped drawing of the cube faces that are oriented away from the camera, which isn't always legal (but mostly works for convex reflectors).

I initially put all reflective objects in the same place in the scene so that the screenshots can be compared with different materials. Later screenshots will show more variety. I really just want to get the materials and reflection vectors correct to start with. The office building scene is one I've shown many times before. I put the reflective object above the desk in the lobby. The floor is partially reflective tile, the ceiling is a grid of white and brown, and the walls are white with some tan. The windows and doors are mostly transparent so that the player can see outside.

My first test used a traditional perfectly reflective metal sphere, like a large ball bearing. Here is how it looks when viewed from up close:

Metal sphere with a perfect mirror reflective surface (pure specular reflection).

Later, I realized that the reflection vector was incorrect. I updated the metal sphere image since it was the most incorrect, but didn't update the others because the images changed very little. Here is the new screenshot, with a sphere that extends slightly below the desk so that you can see the reflection of the inside of the desk. I was moving it around and didn't get it back into the original position.

Reflective metal sphere with corrected reflection vector resting on (and partially inside of) a desk.

Note that the tile floor reflections are shown in the sphere reflections, though I don't think it's quite right. It looks like the floor is reflecting the sky instead of the building interior. I'll have to work on that later. [Update: I figured it out, it's a transform matrix problem, but it's not so easy to fix.] Also, the player has no model and therefore creates no reflection.

Next, I changed the metal sphere to a dielectric material. Here is a giant glass marble, using real material parameters (index of refraction, etc.) and a Fresnel reflection equation. Only the edges of the sphere that are viewed at a glancing angle are highly reflective. This is physically correct - try it yourself at home.

White reflective dielectric sphere (smooth marble).

Here is the same sphere, but this time it's black and viewed from the other side. The effect is similar, and there is an additional white specular reflection from the ceiling light near the top of the sphere.

Black reflective dielectric sphere (smooth marble).

I then added some dynamic light sources to test out reflections from dynamic objects. The addition of dynamic objects reduces the frame rate, but it's still realtime.

Black reflective marble sphere with dynamic light sources creating specular highlights.

There are 100 floating, moving, colored lights in this scene. Several of the nearby lights create specular highlights on the sphere, and some of the lights behind the player are visible as reflections. You can also see the reflection of the chair that's directly behind and below the camera, and the table with bottles on the left side of the sphere.

Okay, it's nice to have reflective spheres. But what about real objects? The objects I want to use in 3DWorld are more complex than simple spheres. One likely candidate for a reflective object is the gold Chinese dragon that I placed on the desk in the lobby. I've had that statue sitting there on the desk for months and every time I look at it, I think, "That should be reflective." Well, it finally is!

Reflective gold dragon statue (871K triangles).

The dragon model is 871K triangles, which shows that reflections can be applied to an object of high complexity. I have no idea what the reflections should look like here. Fortunately, I spent all that time making reflections correct for simple spheres and cubes, so I'm pretty confident that the dragon reflections are at least somewhat correct.

Wait, I didn't add any reflective cube images? Well, they're not very interesting, they're basically just mirrors. I did add a reflective cube that the player can push around the scene. I'll add a video of that to a future post.

Note that the reflective object is concave, but is not self-reflective. One part of the object is not reflected in another part of the object. I ran into too many problems with the object intersecting the near clipping plan and had to move the near plane out beyond the furthest vertex of the object. Maybe some sort of back face culling would have helped, if all of my models actually had the correct face orientations. That's too much to expect from free models that I found online.

Here is another view of the gold dragon, during night time with some dynamic floating lights. This is still rendered in realtime.

Reflective gold dragon with dynamic floating lights.

For some reason, only the green lights like to float near the dragon. Here is a closer view with at least one green light reflecting off the gold. Someone must have put a lot of effort into keeping this dragon shiny by cleaning it every day.

Closeup of gold dragon showing specular reflections from a floating green dynamic light.

The dragon statue material can also be changed. Here it is in white marble.

Ceramic dragon with Fresnel surface reflections from the environment.

 And here it is in black marble.

Black glossy ceramic dragon lit only by reflected specular environment lighting.

The white lines near the silhouette edges of the dragon are actually a form of specular aliasing. The view vector is nearly tangent to the surface at those locations, which makes the Fresnel reflectivity approach 1.0 (mirror reflection). In addition, the triangles in those locations come together at sharp edges with high frequency noise, so their normals change very quickly. This makes the reflected rays bounce around randomly. Many of the rays hit the bright white walls and ceiling, creating bright white spots (pixels) in those locations. I'm not sure how to remove those artifacts, or if they're real physical effects. I don't happen to have a shiny black marble dragon around for reference. I could clamp the max specular reflection value, but that would break the properties of my metal mirror surfaces.

I think the small white spots on the dragon are triangles that have incorrect orientation so that their normals have the wrong sign. I've seen those on other images but they really stand out with the shiny black material. I'm not sure how much effort I want to put into fixing this, considering it only occurs in this combination of material and model. I guess I better make the dragon white or gold in the final scene.

I think 10 screenshots is enough for this post. I want to make sure the page doesn't take too long to load for those of you on mobile devices or slow internet connections. I'll make a follow-up post with more screenshots of multiple inter-reflecting objects, reflections in other scenes, and possibly some videos. I'll also try to include a section on "future work".

More Reflections

I can't get enough reflections in 3DWorld. They really bring life to the scenes. I recently added reflective surface support for horizontal "floor" polygons of imported models. The fragment shader enables reading from the reflection texture for fragments with z-values within the z-range of the reflection plane. Other than that change, the rest of the reflection texture creation works like I explained in my previous post.


Here is a screenshot showing the reflective marble floor from the Crytek Sponza atrium scene. You can see the reflected colored curtains and fire in the floor.

Realtime reflections on the shiny marble floor of the Crytek Sponza atrium.

I also added support for multiple reflective z-planes so that more than one floor can be reflective. The active reflective surface is selected as the one with z-value closest to the camera position from the visible surfaces below the camera. Surfaces above the camera are ignored because the system assumes only the top of the objects are reflective. Here is a screenshot showing reflections on the upper floor of the Sponza atrium with some moving dynamic light sources.

Reflections of the scene and some colored lights on the upper level of the Sponza scene.

I made the marble floor of the museum scene reflective as well. It makes the scene look much more realistic. Take a look at the way the windows look reflected from the floor.

Reflections on the marble floor of the museum scene.

Here is the museum scene shown from a different viewpoint where the dinosaur skeletons are visible.

Another view of the museum scene showing the reflective floor.

I'm currently working on cube map reflections for small placed reflective objects such as the golden dragon in the office building scene. Cube map reflections are more difficult than planar reflections because they need to be rendered from all six sides of a cube rather than a single reflection plane. It's probably too slow to generate all six cube face images each frame for large scenes, so other tricks need to be played. For example, if the scene is static (no moving objects), the reflection images can be rendered once as preprocessing rather than per-frame. I'll make another post if/when I get this working correctly.

Overhead Map View

This past week I have been making improvements to 3DWorld's overhead map view. This was originally a low resolution, cartoon-ish overhead view of the entire level that showed the level bounds, player position and direction, enemy positions, and item locations. The player can zoom in and pan around in the image interactively using the arrow keys.

I decided to add some improvements to the original system that I wrote many years ago. First, I added mouse wheel zoom and mouse click-and-drag for a "slippy map" interface that works like Google Maps. The update rate of map view is somewhere between 10 and 30 FPS, which seems to be okay for a drag-able map.

Each pixel of the map is computed independently. The colors are determined from a height-based table lookup and match the normal colors you tend to see in maps: blue for water, brown for dirt, green for vegetation, gray for exposed mountain rock, and white for snowy peaks. I added sun lighting by computing the normal at each pixel to make the peaks and valleys stand out more. Here is what this view looks like for a procedurally generated island scene. The player is located at the red dot in the center facing in the direction of the black dot.

Map view of an island with correct lighting applied, which produces mountains that look 3D.

This approach works for heightmap textures as well. The user can interactively pan and zoom within a high resolution heightmap and view a pseudo-colored image of the terrain. It works just like an image viewing tool. In the following screenshot, I used the 16K by 16K pixel Puget Sound heightmap dataset available here.

Overhead map view of Mount Rainier from the 16K x 16K pixel Puget Sound texture with real lighting.

Note that interactive heightmap editing done in 3D mode will automatically update the overhead map view. Some of my custom height editing can be seen in the area around the player on the left. Those small lakes and green peaks aren't in the original heightmap dataset.

Here is a video where I pan around the Puget Sound heightmap and change the sun position/direction to prove that map view really does compute per-pixel lighting. The sun moves slowly because I use a keyboard key to move it a few degrees per key press.

Next, I added ray tracing of each pixel to pick up all of the other scene objects, and another ray toward the light source to generate shadows. I'm getting around 20M primary rays per second throughput on a 4 core CPU using OpenMP, which makes it interactive. I was getting over 30M rays per second during snow coverage map creation from the previous post, so this isn't quite as fast. I suspect the slowdown is due to cache misses looking up the texture data for each hit point on the scene geometry. I didn't need this extra step when creating the snow coverage map. When the view is zoomed out this far, texture samples are spaced pretty far apart, making the texture access pattern pretty random. Texture aliasing artifacts can be seen in the screenshot below. The small red/yellow and blue/yellow circles are red and blue team AI players.

Ray-traced overhead view of building scene with shadows enabled in gameplay mode.

You can see some shadows cast by the buildings and trees here. I'm simply darkening the pixels in the overhead image when the sun is not visible from the primary (vertical) ray hit location. Shadows are enabled by default, but they add additional runtime. Shadow rays are faster to compute than primary rays because the search can terminate as soon as an intersection is found. However, they're less coherent, especially near complex surfaces, and that adds to the query time.

The results remind me of old sprite-based top down PC games and some of the simple phone/tablet games I've played. Here is another screenshot, this time of the house scene. The trees cast shadows on the ground and the rest of the scene. This scene uses a custom heightmap which is not drawn in map mode, so all terrain is green.

Ray-traced overhead view of house scene with shadows enabled in gameplay mode.

Overall, this was a fun side project. It doesn't look as good as what I could have drawn on the GPU, and it's much slower to compute on the CPU, but it does show off 3DWorld's ray-tracing support. It's just a different way to approach the problem. There are other benefits, such as improved ability to debug the collision volumes, ray tracing code, object placement, etc.

Rain and Snow Progress

The past few weeks of 3DWorld development have mostly been spent improving rain and snow effects. I have added several new features, including dynamic wetness/drying effects with puddles, realtime snow accumulation and coverage, ice, and snow footprints. I'll briefly cover each of these below.

Puddles

I continued implementing features from this blog post, and the next item up was puddles. I wanted to have puddles generated procedurally rather than having to place/paint them all by hand. 3DWorld is all about procedural generation and automating as much of the level design as possible. It's too much work to add every puddle by hand to every horizontal surface of the scene, for all of my test scenes. I don't own any 3D modeling tools and I only use free image editing tools such as Gimp.

Normally, puddles are formed in low areas of the ground, but most of the man-made (non-natural) surfaces in 3DWorld such as concrete floors are modeled as flat planes. The normal maps are too regular and finely detailed to use for puddle depth, so some other tiny invisible adjustment to surface height was required. It seemed like this was another place where I could apply Perlin noise, and as expected, it worked pretty well.

The wet/dry/rain cycle works like this:

1. The ground is dry
2. Rain starts, making mostly horizontal surfaces that are open to the sky uniformly wet over time
3. Rain stops, and the ground dries non-uniformly, leaving puddles
4. After some time has passed, the ground becomes dry again (same as 1.)

There are three states of rain surfaces: wet, dry, and full of puddles. These are implemented with shader flags that control how the surface is rendered, affecting diffuse albedo, specular intensity, specular gloss, and reflectivity (Fresnel term). Here is a screenshot of some procedural puddles from state 3 generated on the ground after the rain has stopped.

Procedural puddles on the ground as the concrete is drying out after a heavy rainstorm. Wet surfaces are reflective.

And here is a YouTube video showing the transition from dry ground, to wet reflective ground, to puddles, and back to dry ground. Oh, and yes, I finally added procedurally generated palm trees.

Realtime Snow Accumulation

My previous post was on volumetric/3D snow accumulation. That system produced good results, but required dropping a very large number (1 billion) of snowflakes from the sky and recording where each of them landed. This works well for a static scene where a one-time preprocessing step is acceptable. But I wanted another solution for faster snow accumulation that could be used with 3DWorld's realtime, dynamic weather system.

I decided to add support for a thin dusting of snow that "paints" upward oriented surfaces a white color. This isn't true volumetric snow with depth, but it still produces a nice snowy scene effect. The most difficult problem was determining which parts of each surface receive snow. Outdoor areas only please - we don't want snow accumulating in the lobby, offices, or basement!

My first attempt was to cast rays through scene vertices and tag a surface as "outdoors" if any rays reach the sky, and "indoors" otherwise. This worked somewhat, and was good enough to use for the rain wet mask, but wasn't quite right for snow. It left areas under objects such as benches snowy because there was a single large polygon covering a large surface area. It looked correct to have wet areas under cover during rain because rainwater will flow across flat surfaces over time, but snow doesn't do this.

The next thing I tried was closer to how shadow mapping works. From what I understand, this is how other games mask snowy and wet areas. The scene is drawn in a "shadow" pass where the objects are projected down, or in the direction of the falling snow if there is wind. This is normally done on the GPU, but that would have produced aliasing and other artifacts. Since shadow mapping tends to sample a pixel once, this means that a small/thin object such as a flagpole or lamppost that happened to fall in a bad spot near the pixel center would leave a hole in the snow mask. Have you ever seen a flagpole that blocks the snow from accumulating around it's base? It looks very odd and unnatural.

Instead of rendering the scene to a snow shadow map, I compute the coverage map on the CPU. A group of rays are traced through a pixel in a 3x3 grid pattern, and all ray intersections with the scene are determined. The furthest/deepest distance is used as the snow depth for that grid position (pixel). This way, a thin object such as a flagpole may block some of the rays, but won't block all 9 of them. It took a bit of further experimenting, but in the end I was able to remove all of the incorrect holes in the snow mask. I also tried adding random noise to the samples, which helped remove some of the artifacts from the snow mask, but also added noise to the smooth edges of snow around building walls and overhangs. So in the end I used a simple fixed grid.

The snow coverage mask is sent to the GPU and used in the fragment shader for per-pixel snow masking. The (x,y) position of the surface to be drawn is used to look up the depth (z) value in the snow mask, which tells the GPU how deep the snow penetrates the scene in z at that position. If the surface has a z-value below this point, it's underneath some other surface that blocks the snow. If its z-value is less than or equal to the snow map depth, it's covered with snow. I addition, there is a soft transition from no snow to full snow based on the difference in depth/z between the snow mask and the surface. Finally, I used the slope of the surface (technically the z-component of the normal vector) to blend snow density from 1.0 for horizontal surfaces to 0.0 for nearly vertical surfaces.

The advantage of this approach is that I was able to produce a 256x256 pixel 2D snow coverage mask using only 256*256*3*3 = ~600K samples. Compare this to 1 billion samples required for smooth snow in the 3D volumetric case. Using spatial coherency of the rays to trace the 3x3 rays as a group, the 3DWorld ray tracing system can achieve around 35M rays per second on 4 CPU cores, which translates into a mere 17ms of runtime. That's quite a bit faster than the ~20 min. required for volumetric snow accumulation! This algorithm allows the snow coverage mask to be recomputed in realtime as the scene is modified and the weather changes. As a bonus, no precomputed snow coverage files need to be stored on disk and loaded.

Below is a screenshot showing the results of realtime dynamic show accumulation. Note that only areas of the scene that have an unobstructed view of the sky accumulate snow (except for the grass, which doesn't use the snow mask).

Realtime procedural snow accumulation in the office building scene.

Here is a close-up view of the snow on objects of various shapes. Note that there is no snow under the trees and benches.

Close-up view of snow accumulating on various objects based on a generated sky coverage mask.

I also recorded a video showing how snow accumulates during a heavy snowfall.

Ice

I decided to implement the mixing of rain and snow to produce ice. The ice effect is somewhat different depending on which comes first. Rain on top of snow melts the snow, but snow on top of rain forms a layer of reflective ice on horizontal surfaces. Angled surfaces accumulate less water and eventually turn to pure snow. It looks like this:

Rain mixed with snow produces a layer of reflective ice on horizontal surfaces. Angled surfaces are more snowy.

Note that the water in the fountain has been frozen and covered with a layer of snow.


Snow Footprints

The day after I published the previous blog post, someone asked me if players left footprints in the snow. What a great idea! It wasn't even that hard to add, though I went back and tweaked the parameters several times. I had to define the player stride, foot width, foot spacing, foot height/crush depth, etc. to get realistically shaped and placed footprints. The numbers don't quite work out for my spherical "smiley face" character model, but it's close enough. Both the user/player and AI controlled players leave footprints in deep snow. Here is a screenshot showing footprints in the snow after I walked around for a while. The gaps in the footsteps where the places where I jumped (which doesn't leave footprints). If you look closely, you can see that the footprints are actually changes applied to the snow mesh rather than decals because that footprint on the very edge on the top of the wall leaves a small gap in the shadow on the snow below.

Footprints created using mesh deformation when the player and AI characters walk in the snow.

I made a Fraps video of a walk in the snow, complete with footstep sounds. The footsteps are a bit too fast, so they sound more like I'm running. Note that snow and concrete have different footstep sounds. I'll probably add a larger variety of footsteps and other sounds in the future.

That's all for now. I wonder what other interesting weather effects I can add?

Simulation and Rendering of Snow

As promised, here is my post on snow simulation and rendering in 3DWorld. Last month, before the holidays, I was working on rain. In fact, I'm still working on rain. I traveled back to Pittsburgh, PA to visit my parents for two weeks and expected to see snow, but there was only rain (except for the last day). It's also raining here in California. The weather really isn't motivating me to work on snow. I'll just discuss the original snow system for various modes, since I think snow works pretty well already.

There are several aspects to snow in a 3D game engine. First is drawing all of the snowflakes as they fall through the air. Second, the engine needs to deal with adding snow to the ground/terrain texture so that snow covered peaks are white. Third, snow should be drawn as actual 3D geometry when it's deep and near the player. The challenging part is figuring out where snow will actually accumulate, given the terrain, scene geometry, weather factors (temperature, wind, etc.), and snow depth. This is what I call snow "simulation".


Snow Ground Coverage

First, I will compare 3DWorld's automatic snow map generation based on altitude and terrain slope to a reference photograph. Higher altitudes are cooler and therefore have more precipitation in the form of snow (as opposed to rain). Also, snow does not remain on steep slopes; it falls down the slope, leaving exposed rock behind. Here is a photograph of Mount Rainier covered in a layer of snow.

Photograph of Mount Rainier viewed from the southwest, from Wikipedia.

Here is 3DWorld's realtime rendering of Mount Rainier from a high resolution heightmap, where snow cover has been calculated from the terrain.

Mount Rainier drawn by 3DWorld using only a heightmap. Snow coverage was simulated using terrain slope and altitude.

Note that I did this image comparison after setting the snow parameters in 3DWorld and didn't go back to tweak anything. It happened to turn out very close to the actual snow coverage, validating the height and slope model. I cheated a bit with the clouds though, and set the average cloud height experimentally based on the reference image. The trees aren't really to scale (I wasn't too interested in fixing them here). Also, 3DWorld's mountain appears to be taller because there was no reference to convert the height map values from the image I found online to real height values, so I took a guess at it and overestimated. Oh, and someone pointed out that I'm missing the glacier on the top of Mount Rainier - I haven't gotten to simulating glaciers yet.


Falling Snowflakes

3DWorld has two forms of physical falling snowflakes. The original snowflakes are all dynamic physics objects that use the same game object class as balls, rockets, shell casings, etc. They have very accurate physical models that include collisions with the exact scene meshes, gravity, friction, wind resistance, and thermal effects (melting in high temperature areas). Unfortunately, it takes a large amount of CPU work to simulate tens of thousands of snowflakes this way. I had to find a faster alternative.

The second and newer snowflake model is more GPU based. Each snowflake is a single {x,y,z} point rendered as a point sprite on the GPU. The physics and collision detection is still performed on the CPU, but it's simpler and more efficient. For example, the velocity of each snowflake is not even tracked - all snowflakes have the same velocity based on gravity and wind. Collision detection is performed coarsely against a 3D voxel (volume grid) model of the scene, and the time consuming detailed collision detection is only performed when the voxel test returns a possible collision. Snowflakes are destroyed and respawned high in the sky when they collide, rather than bouncing or sticking as in the slower model. This approach scales up to 100K snowflakes and looks almost indistinguishable from the more accurate model. You can really only tell the difference when running physics in slow motion or freezing the frame. The lighting computation in the shaders is all the same as well.

In the end, I used a mixture of around 1:5 of old model vs. new model snowflakes. That way, most of the snowflakes are fast, but there are also some that actually stick to the scene objects. Here is what a scene looks like with 40K snowflakes. Can you tell which flakes use which model? I think the only way to tell would be to visually track an individual snowflake's path though the scene.

Snow falling in the office building scene and beginning to accumulate in the grass.

The white spots on the grassy ground mesh are the beginning of snow accumulation. When an old model snowflake collides with the terrain, it colors the terrain texture white in that area. After several minutes of snowfall the ground will be mostly white, except for the grass itself, which remains green. The office building surfaces are unaffected. There is no realtime snow accumulation - that is more expensive (in terms of CPU time) and done as a preprocessing step (see below).

One of the more challenging aspects of snow is drawing individual close-up snowflakes. Sure, snow looks fine in the distance when drawn with white points. But when you actually look closely at a snowflake, it's not a white dot. It has real structure. In fact, every snowflake has a unique fractal crystal pattern. I decided to use a single snowflake texture for nearby snowflakes. It looks nice, but it's not very realistic - more of a cartoon effect. Here is a screenshot of a snowflake on the player's eye/"camera lens".

A snowflake falls in front of the camera. There are 40K total snowflakes falling from the sky.

Snow Accumulation

Next, I will explain how snow accumulation works in 3DWorld. To get an immersive snowy scene, snow needs to have real depth, and needs to cover every object. Just painting it onto the terrain mesh texture doesn't work. But adding a constant layer of snow on top of every horizontal surface doesn't look right either - snow depth changes with the slope and geometry of the objects nearby.

Rather than trying to come up with a physical model for snow accumulation, I decide to approach it using a simulation of random falling snowflakes. One billion snowflakes are dropped from the sky with uniform distribution over the scene. The path of each snowflake is traced, taking into account wind and other factors, until a collision with the scene is found. Then, depending on the collision normal/direction, slope of the surface, friction, and wind, the snowflake either sticks or bounces off and continues falling. Snowflakes that stick are added to a sparse 3D accumulation volume implemented with a hash map. For the house scene below, I used a 1000x1000x500 volume. The actual number of nonempty cells (with at least one sticking snowflake) is around 4M, which can easily be stored in memory. The entire process takes around 20 min. on a 4 core CPU.

This first step produces a volume map of snow accumulation. The next step is to convert this volume to an actual surface. First we need to determine how much volume each of the 1B snowflakes contributes. Given a snowflake volume V, and scene size W by H, and voxel size X by Y by Z (=1000x1000x500 in this case):
volume = width*height*depth
V = voxel_width*voxel_height*delta_z
delta_z = V/(voxel_width*voxel_height) = V/[(W/X)*(H/Y)]

As you can see, the final snow depth is a linear function of the volume of a snowflake. Changing the value of V will vary the snow coverage from a light dusting to heavy accumulation to snow that covers the entire scene. The conversion from snow volume to a surface only takes about two seconds. Once the expensive (20 min.) simulation of 1B falling snowflakes is performed we can vary the snow depth interactively, or save the snow map and use it for different scenes with the same geometry but different snow coverage. 3DWorld saves the snow accumulation in a very compact, sparse structure that is only 13MB for the house scene.

The process of converting snow volume to a 3D surface works by extracting contiguous rows of voxels into triangle strips, then collecting adjacent triangle strips into mesh sections. The height (z-value) of each vertex in the triangle strip is determined by the amount of snow accumulated in that voxel in the snow map. If all of the nearby voxels in an area have similar volumes, this produces a flat snow cover (for example roads). If the volumes vary significantly between adjacent voxels, this produces rough snow (for example on trees). Any vertical (Z) stack of nonzero volume voxels will merge into a deep snowdrift, where the Z height of the mesh is taken from the sum of the volumes. This models snow stacked against the side of a house or tree. Areas that have stair-step patterns of filled voxels where adjacent nonempty voxels are one Z column above or below each other will produce steep triangle strips (for example on sloped roofs). This approach is able to extract multiple overlapping surfaces at different heights, such as the snow on top of the fence and the snow below the fence at the same {x,y} position. Here is what this algorithm produces for moderate snow depth on the house scene.

Snowy house scene with falling snow and heavy snow accumulation on the ground and scene collision objects.

This approach works very well for smooth surfaces, which account for most of the scene. The areas that need improvement are near scene objects that are smaller than a few voxels in width or size. In this case, the voxel snow volumes don't have enough resolution to accurately map a triangle mesh surface onto the small objects. This is an example of the Nyquist Samping Theorem. The result is single triangle patches of snow that in some cases are larger than the objects they are resting on. Using a higher resolution volume would help with this issue, though it would consume more runtime, memory, and disk space.


Snow Surface Rendering

Finally, I will discuss how the snow surface is rendered in 3DWorld. It's not all that difficult. The meshes that were formed from triangle strips are grouped together into vertex buffer objects (VBOs) and stored on the GPU for improved performance. They are static for the scene. [Well, there is dynamic snow deformation due to collisions that can be enabled, but that's a topic for another post.] Snow is then drawn using the normal rendering pipeline for 3DWorld with shadow mapping, normal mapping, and sharp specular lighting to get those nice bright highlights. The strength of the normal mapping is reduced for distant snow to avoid specular aliasing and texture tiling artifacts. Here is what the surface of the snow looks like up close, when looking toward the sun.

Snow drawn as a 3D geometry layer on the ground. Snow depth is based on simulating 1 billion falling snowflakes with collisions and tracking volume accumulation.

The snow surface itself casts shadows on the rest of the scene, which requires it to be drawn without texturing or normal mapping a second time to create the shadow map. When there are no dynamic objects, the snow shadows are cached between frames to improve frame rate. Snow rendering is surprisingly fast, considering the snow mesh contains nearly 4M vertices. On my new Nvidia card the addition of snow decreases the framerate from 500FPS to 250FPS. This is less frame time than grass (170FPS), so if the snow covers the grass the framerate actually increases!

Update: Here is a screenshot of player footsteps in the snow, simulated while the player is walking based on stride, foot length, foot spacing, player speed, and player direction.

Player footsteps in the snow.

Update2 (10/16/20): Here is snow accumulation in the house scene with a wind effect added. Snow has been pushed against the near wall of the house, and there's a gap in the snow on the left side of the house.

Snow accumulation with a strong wind coming from the top right direction.

Reflective Surfaces

This past week I decided to go back and work on rain effects again. The ground doesn't look "wet enough" - it needs to be more reflective. So I reused the plane reflection technique from tiled terrain water to get a reflective wet surface effect in ground mode.

I specify a reflection cube in the config file to enable reflective surfaces in the scene. Any objects with a flat top surface (currently cubes and vertical cylinders) that intersects this cube are treated as reflective. The cube is very narrow in Z height so that all of the objects have their top surfaces close to a single Z-value and can use the same reflection plane. For the office building scene, this includes the first floor walkable area inside and outside of the building. This covers most of the surfaces that get wet in the rain.

The general idea is to draw the scene twice in different modes. In the first pass, the reflection pass, the scene is drawn from the eye position mirrored about the reflection plane, which puts the eye/camera below the reflective surface. This image is stored in a texture at half screen resolution to reduce runtime and memory usage and improve framerate. Then the scene is rendered as usual. The reflective surfaces are drawn using a special shader that looks up the reflection texture and computes a Fresnel reflection term for the material.

I'm using the normal maps of the surfaces for the reflection normal, which effects the reflectivity through the Fresnel equations. Wet surfaces with accumulated water have a smoother, planar water surface (ignoring ripples), so their normal is close to +Z. I blend between the normal map normal and +Z depending on the water depth, which produces an effect of water filling the cracks between the concrete blocks and also a thin layer of water on top of the concrete. The idea was taken from this blog post. Take a look at these screenshots:

Reflections on the wet concrete block floor. Water has accumulated in the cracks between the blocks.

Reflections on wet concrete from another angle.

Reflections also work on smaller objects, even dynamic objects that can be moved by the player, as long as their top surfaces are near the Z reflection plane. Here I have pushed a glass block down the stairs so that it's top surface is at the right height for reflections. The glowing, light emitting spheres are there to test dynamic object reflections and dynamic lighting on reflected objects.

Reflections of the scene with glowing spheres on the top of a glass block.

Reflections are not only for wet surfaces. Shiny indoor surfaces such as tile and marble can also reflect the scene. This greatly improves the realism of these materials, at the cost of increased GPU time and lower frame rates. This tile floor is reflective with a normal map that results in the Fresnel reflection term varying over different parts of the tile, changing the reflectivity per-pixel. The raised marble platform is a smooth reflective surface (like a mirror).

Reflections of the scene with glowing spheres on shiny tile and marble floor.

I ran into a few problems using this approach. First of all, there are several objects at slightly different heights that all use the same reflection texture with the same mirrored Z-value. In the office building scene, there is a raised marble platform slightly above the tile floor, which itself is slightly above the outside concrete. All three surfaces are at different Z values and reflective/visible at the same time when rain is enabled. I use the average Z value from the top surface heights of all visible reflective surfaces. When there is only a single reflective surface it works great, but it's not always so simple. This can produce a small amount of light leaking from under an object that is occasionally visible, but difficult to remove. Here is an example image where the very bottom of the windows are visible in the reflection on the marble, just under the desk. It makes the desk look like it's floating a few inches in the air, because the reflection plane is off by a few inches from the surface. The only real fix for this is to render the scene at multiple Z values in multiple passes, which is slow.

Light leaking under the desk due to top marble surface slightly above the reflection Z plane.

Another bug I ran into is related to objects below the reflection plane that are incorrectly rendered into the reflection image. Think of it like this: The view vector and camera frustum are reflected about the top surface of the floor. This is effectively like moving the camera down by twice the distance it is above the floor, so that the reflection camera is under the ground. What do you see when under the ground? Tree roots! Take a look at this screenshot:

Incorrect reflection of roots under the ground resulting from missing Z clip of geometry.

The problem here is that the tree roots aren't actually visible because they're occluded by the reflective top surface itself, which isn't drawn in the reflection pass. This fix is to clip/discard all object pixels below the reflection Z value in the fragment shader when rendering the reflected scene. This contributes a bit more light leaking to reflective surfaces that are misaligned slightly with the reflection plane, but otherwise fixes the problem.

I found a third bug, where the bottom of the columns that support the stairs appeared to be misaligned with the top of the floor. Here is what it looks like. Notice the apparent gap between the columns and there floor where you can see the reflection of the bottom of the columns.

Small gap between the bottom of vertical columns and the floor exposed by reflections.

At first, I thought this was a problem with the reflection plane similar to the screenshot of the desk above. But the tile floor is the only visible reflective surface in this screenshot (since rain is disabled), so the reflection plane Z should be exactly at the floor height. I spent quite a while trying to debug this. It turns out that it was really an error in the scene geometry - there actually was a small gap between the bottom of the columns and the floor! I could only see it by disabling collision detection so that I could move the camera to near floor height. Once I figured it out, the problem was trivial to fix by editing the scene geometry.

Reflections can be enabled in all scenes that contain flat horizontal surfaces. Here is a view of the wet road and driveway surfaces in the house scene. The reflections are perturbed by the road's normal map to create a rough/bumpy wet surface effect.

Reflections of the house, fence, and trees on the wet road and driveway surfaces.

I suppose the next step is to implement reflections for more than one plane. The goal is to find a way to do this without needing to render the scene reflected about more than one Z value, and to allow reflections in other directions. A further improvement would be to support reflections on curved surfaces. This likely requires rendering to the six faces of an environment cube map. Environment mapping techniques work well for static scenes, but are more challenging when dynamic objects are involved. It's probably too slow to render the scene multiple times (3?) each frame for the visible faces of the cube map. I'll have to put more thought into it.

Shadows Update

I worked a bit more on tiled terrain shadows and fixed a few problems from the previous post, so I'll post a few updated images here.

Scenery Shadows
I forgot to add shadows for the other ground scenery, which includes rocks, stumps, and logs. This has been fixed - they now use the same shader as the tree branches. Everything in the scene now receives shadows.

Leaf Back Lighting
I took a screenshot of the leaf back lighting from the sun during sunrise or sunset (not that there's a difference between them in 3DWorld). The unshadowed leaves on the lit side of the tree transmit some light to their back sides, but the shadowed leaves on the unlit side remain dark. This creates a halo of light around the trees similar to "rim lighting" sometimes used in graphics and photography.

Back-lit trees at sunset.

Here are some examples of real photos of back-lit trees, though they were taken with brighter lighting conditions:
Example 1
Example 2
Example 3


Water Shadows
I finally fixed the mesh shadows on the water surface so that they work even when the water is deep/high above the mesh. I folded the terrain mesh into shadow map creation by determining which tiles fall within the frustum (bounding area) between the light source (sun) and the tile to be shadowed. Any tiles whose bounding volume (bounding cube in this case) intersects the light frustum may contribute shadows and need to be drawn into the shadow map. When the sun is high in the sky (large +z component), only the current tile and it's 8 neighbors in {N, NE, E, SE, S, SW, W, NW} are included. The neighbors may contain trees and other objects near/crossing the tile boundary that cast shadows on the adjacent tile. When the sun is low on the horizon, any additional strip of tiles stretching from the sun to the receiver tile are included. I was worried that this would be too many tiles, but it turns out that only 3 or 4 additional tiles are needed. The increased shadow map creation time is barely noticeable. Here is what it looks like in a screenshot where a distant peak casts a long shadow across the surface of the water.

Shadows of a distant peak on the water surface.

Mesh Shadows
3DWorld supports importing of 3D models in the Wavefront object file format. These models can be instanced in the scene with various transforms and colors. I have added three instances of the Chinese dragon statue and one instance of the Sibenik Cathedral to my scene. Each object model casts shadows on itself and on the rest of the scene, including the terrain mesh, trees, grass, and plants. Their shadows can extend far across the mesh. Here are some examples of what it looks like.

Shadows from Chinese Dragon models on the mesh, trees, and grass.

Shadows from Chinese Dragon models cast on the terrain from several tiles away.

That's it for now. I'm moving on to implementing reflections for planar surfaces that should significantly improve the wet surface rain effect. This will be the topic of a future post (assuming I get it working).