Physically Based Materials

I'm continuing to work on improving object reflections in 3DWorld. The past few weeks I've been trying to integrate reflective objects into the engine as physically-based materials. As a first step, I provided a way to create and throw spheres and cubes with user-defined material properties as dynamic objects in the scene. The properties are specified in a config text file in keyword/value format. There is also a simple UI for realtime editing of material parameters and creating new materials. The UI is an in-game text overlay with arrow key input similar to the onscreen display you would find in a computer monitor. It's very simple but usable. I would like to use the mouse to select menu items, but I think it would interfere with the user's ability to play the game and interact with the world while the menu system was active.

The material parameters supported are:

• Material Name - User-defined text string identifier
• Texture - Name/filename of texture to use; "none" to disable texturing
• Normal Map - Name/filename of normal map texture to use; "none" to disable normal mapping
• Shadows - Flag to enable cube map shadows for point light spheres
• Emissive - Flag to mark as having an emissive color (no lighting)
• Reflective - Flag to mark surface as reflective (using an environment cube map)
• Destroyability - Tag to mark as destroyable, shatterable, exploding, static, etc.
• Metalness - Value in [0,1] to represent dielectric vs. metal
• Hardness - Value in [0,1] to set hardness for elastic collision physics
• Density - Value of material density, used to compute mass and buoyancy in water
• Specular Magnitude - Magnitude of specular light reflection in [0,1]
• Specular Shininess - Shininess of specular light reflection, converted to surface roughness
• Alpha - Value in [0,1] to specify alpha value of partially transparent objects such as glass
• Light Attenuation - Factor for computing transparency and scattering within the material
• Index of Refraction - Value for controlling reflection and refraction in glass, plastic, etc.
• Light Radius - Radius of light emission for light source spheres
• Diffuse Color - {R,G,B} diffuse, emissive, or light source color value
• Specular Color - {R,G,B} specular color value ((1,1,1)=white for non-metals)

For example, the material "Gold" is specified as:
hardness 0.8
density 19.29
alpha 1.0
reflective 1
metalness 1.0
specular_mag 1.0
specular_exp 128.0
diffuse_color 0.0 0.0 0.0
specular_color 0.9 0.6 0.1
add_material Gold

I recorded several videos showing how 3DWorld's dynamic, throw-able spheres and cubes work, including realtime editing of material parameters. I feel that videos are required to show these features. It's just too hard to tell what's going on in static images. I can only cover a small fraction of the materials, parameters, and features available in these short videos.

Sorry, none of these videos were recorded with sound. The only sounds I have enabled in these tests are for throwing and bouncing anyway. These videos are too long to record with the free version of Fraps (which has sound). The FFmpeg video recording wrapper support in 3DWorld can record unlimited length videos and compress them in realtime, but I haven't figured out how to record audio yet in Windows.

Here is a video of me throwing spheres of various materials around in the scene and editing the material parameters in realtime. Everything in the scene is reflected in mirror surfaces, including the placeholder smiley player model.

This is a video showing dynamic sphere point lights and cube mapped shadows in a dark room. Lighting, shadows, reflections, and various other parameters can be assigned to materials and edited in-game.

I later decided to add support for dynamic material cubes as well as spheres. Here is a video of me throwing some cubes around and changing their textures and normal maps. Cubes and spheres use partially elastic collision models and will propagate collision forces around when piled up on top of or against each other. They can be stacked, pushed around the scene, and the player can stand on them, though there are some issues with simulation/physics stability.

Density is one of the material parameters that can be modified in realtime through the material editor. The material's density affects the amount of resistance to pushing and its buoyancy in water. In this video, I edit the density of the brick cubes, which affects how high they float in the water or how quickly they sink. The player can stand on and stack objects on the cubes as well, and everything works correctly. Spheres can also be used.

This is a video of my incomplete puzzle/platformer scene. It uses a variety of different effects and materials. The marble floor and some of the glass surfaces are plane reflectors. I haven't finished all of the traps and obstacles, and the various sections aren't even fully connected. I had to use the "flight mode" cheat to get to the second section. I'll post more screenshots and videos of this map later when it nears completion.

 I'm continuing to work on dynamic objects and materials. I would like to add support for the other shape types supported by 3DWorld: polygon, extruded polygon, cylinder, cone, capsule, and torus. I'm also considering adding more physics properties to the editable materials list, for example parameters for friction, air resistance, deformation, elasticity, player damage, etc. Regular dynamic 3DWorld objects such as weapon projectiles and pickup items use fixed materials, which already have all of these properties. Finally, I would like to add a way to make these objects into efficient static scene objects so that this mode acts like an in-game scene/map editor. I'm curious to see what the performance is when there are thousands of placed objects of dozens of different materials in the scene.

Reflections and Roughness

This post continues my work on cube map reflections from where I left off in an earlier post on this topic. I had it working pretty well at the time. However, I was never able to get 100 reflective objects in the scene in realtime because I didn't have enough GPU memory on my 2GB card. I now have a GeForce GTX 1070 with 8GB of video memory, which should allow me to add as many as 300 reflective objects.

Another problem that I had with the earlier reflection framework was the lack of surface roughness support. Every object was a perfect mirror reflector. I did some experiments with mipmap biasing to try and get a proper rough surface (such as brushed metal), but I never got it working at a reasonable performance and quality point. I think I've finally solved this one, as I'll explain below.

3DWorld uses a Phong specular exponent (shininess factor) lighting model because of its simplicity. Physically based rendering incorporates more complex and accurate lighting models, which often include a factor for surface roughness. I'm converting shininess to surface roughness by mapping the specular exponent to a texture filter/mipmap level, which determines which power-of-two sampling window to use to compute each blurred output texel. I use an equation I found online for the conversion:
filter_level = log2(texture_size*sqrt(3)) - 0.5*log2(shininess + 1.0)

The problem with using lower mipmap levels to perform the down-sampling/blurring of the reflection texture is the poor quality of the filtering. Mipmaps use a recursive 2x2 pixel box filter, which produces blocky artifacts in the reflection as seen in the following screenshot. Here the filter_level is equal to 5, which means that each pixel is an average of 2^5 x 2^5 = 32*32 source texels. Click on the image to zoom in, and look closely at the reflection of the smiley in the closest sphere.

Rough reflection using mipmap level 5 (32x32 pixel box filter) with blocky artifacts.

The reflection would look much better with a higher order filter, such as a bi-cubic filter. Unfortunately, there is no GPU texture hardware support for higher order filtering. Only linear filtering is available. Adding bi-cubic texture filtering is possible through shaders, but is complex and would make the rendering time increase significantly.

An alternative approach is to do the filtering directly in the fragment shader when rendering the reflective surface, by performing many texture samples within a window. This is more of a brute force approach. Each sample is offset to access a square area around the target pixel. I use an NxN tap Gaussian weighted blur filter, where:
N = 2^(filter_level+1) - 1
A non-blurred perfect mirror reflection with filter_level=0 has a single sample computed as N = 2^(0+1)-1 = 1. [Technically, a single filter sample still linearly interpolates between 4 adjacent texels using the hardware interpolation unit.] A filter_level=5 Gaussian kernel has N= 2^(5+1)-1 = 63 samples in each dimension, for 3969 samples total. That's a lot of texture samples! It really kills performance, dropping the framerate from 220 FPS to only 19 FPS as shown in the screenshot below. Note the framerate in the lower left corner of the image. But the results look great!

Rough reflection using a 63x63 hardware texture filter kernel taking 3969 texture samples and running at only 19 FPS.

The takeaway is that mipmaps are fast but produce poor visual results, and shader texture filtering is slow but produces good visual results. So what do we do? I chose to combine the two approaches: select a middle mipmap level, and filter it using a small kernel. This has a fraction of the texture lookups/runtime cost, but produces results that are almost as high quality as the full filtering approach. For a filter_level of 5, I split this into a mipmap_filter_level of 2 and a shader_filter_level of 3. The mipmap filtering is applied first with a 2^2 x 2^2 = 4x4 pixel mipmap. Then the shader filtering is applied with a kernel size N= 2^(3+1)-1 = 15. The total number of texture samples is 15x15 = 225, which is nearly 18x fewer texture accesses. This gets the frame rate back up to around 220 FPS.

I'm not sure exactly why it's as fast as a 1x1 filter. The texture reads from the level 2 mipmap data are likely faster due to better GPU cache coherency between the threads. That would make sense if the filtering was texture memory bandwidth limited. I assume the frame rate is limited by something else for this scene + view, maybe by the CPU or other shader code.

Here is what the final image looks like. It's almost identical in quality to the 63x63 filter kernel image above. The amount of blur is slightly different due to the inexactness of the filter_level math (it's integer, not floating-point, so there are rounding issues). Other than that, the results are perfectly acceptable. Also, this image uses different blur values for the other spheres to the right, so concentrate on the closest sphere on the left for comparison with the previous two images.

Rough reflection using a combination of mipmap level 2 and a 15x15 texture filter kernel taking 225 texture samples.

Here is a view of 8 metal spheres of varying roughness, from matte (fully diffuse lighting) on the left to mirror reflective (fully specular lighting) on the right. Each sphere is one filter_level different from the one next to it; the specular shininess factor increases by 2x from left to right.

Reflective metal spheres of varying roughness with roughest on the left and mirror smooth on the right.

This screenshot shows a closer view of the rough sphere on the left, with the filter_level/specular exponent biased a bit differently to get a clearer reflection. There are no significant filtering artifacts even at this extreme blurring level.

Smiley reflection in rough metal sphere showing high quality blur.

I'm pretty happy with these results, and the solution is relatively simple. The next step is to make the materials editable by the user and to make the reflective shapes dynamic so that they can be moved around the level. In fact, I've already done this, but I'll have to show it in a later post.