Reflective and Emissive Spheres Test

I came across this Unity GPU path tracing article by David Kuri of Three Eyed Games and decided that I wanted to produce a similar scene in 3DWorld. However, I'm taking a very different approach. Rather than using a ray/path tracing based solution, I'm using precomputed indirect lighting, shadow maps, and cube map reflections. The image quality isn't as good, but it runs in realtime (60+ FPS) and can be viewed from any angle. In fact, the user can even interactively change the sun position and move spheres around in the scene! Ray/path tracing approaches take seconds to minutes per frame, which is thousands of times longer.

I added config file options to place random spheres in 3DWorld. The user specifies the number, size range, spatial distribution, and probabilities for various sphere materials in a text file. These can be added to any scene. In this case, I used a simple white cube for a ground plane. The spheres are placed randomly within a circular region on the ground. I used between 100 and 200 spheres for these tests because that gave reasonable runtime numbers.

3DWorld implements reflective materials using one cube map per object (sphere in this case), as shown in this blog post from last year. The basic description of how cube map reflections work in 3DWorld is shown here. All six cube faces of each object are rendered to textures during update frames. Several update frames are needed to get the nested reflections between multiple nearby reflective objects. For example, two nearby metal spheres will reflect each other recursively. Each update frame takes around 250ms to compute almost 100 cube maps, or around 500 cube face texture images.

Moving individual non-emissive spheres takes 50-100ms per frame to update the needed cube map faces. Moving the sun requires the full 250ms update due to the global affect of sun lighting in the cube maps of all the spheres. Frames where nothing moves but the camera take only around 14ms for these scenes, or about 70 FPS.

Indirect lighting is computed in about two minutes for 14 light sources in 3DWorld. I used the same method as in other scenes, as described in this post. The lighting data can be saved and reused at a later time as long as none of the emissive light spheres move. The lighting grid is stored in a 3D texture on the GPU and used to lookup indirect lighting data in the fragment shader.

For reference, here is the Three Eyed Games image:

GPU path traced image on Three Eyed Games blog that took several minutes to generate.

Here is as close as I was able to get using 3DWorld:

Realtime 3DWorld rendering of similar scene using precomputed indirect lighting, shadow maps, and cube map reflections.

I'm not trying to exactly duplicate the reference image, I just want to make something in a similar style. The goal is to compare the two different rendering approaches.

There are a variety of differences. The most obvious one is that the 3DWorld version is darker than the reference image. This is because there are fewer emissive light source spheres. 3DWorld is currently limited to around a dozen dynamic point lights with cube map shadow maps because it has a fixed number of shadow map slots (64). I could probably increase the limit, though it may hurt performance.

The 3DWorld image has hard shadows, whereas the reference image has soft shadows. That's the difference between using shadow maps vs. ray tracing. There's not much I can do about this one. Soft shadows from many point light sources are difficult to do in realtime.

Another difference is the mixture of materials. The reference image contains a combination of emissive, metal, dielectric, and diffuse spheres. The reflective spheres have variable roughness. 3DWorld has the same types of materials, but seems to use a higher percentage of shiny metals. There are some rough metals and some dielectrics, but the lower lighting levels makes them difficult to see. I've shown rough materials previously. I could tune the random number generation parameters to make them more in agreement. I kind of like all of the reflective spheres in my image though.

The two images have different types of noise. The reference image has high frequency pixel noise that can only really be seen when viewing the image fullscreen. 3DWorld has lower frequency noise due to the sampling of the precomputed indirect lighting grid, currently 192x192x16 texels in size. I spent a while adjusting various parameters in the code and config file to try and minimize noise, and it definitely improved things. I could further reduce the noise with a higher resolution grid. However, this would take more precomputation time, more disk space to write, more GPU memory, and more work in the fragment shader. This leads to increased render time/lower frame rate.

This screenshot shows the same scene during the day under bright sunlight.

Same 3DWorld scene as above, but this time during the day. The bright sun produces sharp shadows on the ground plane.

Here is another view of the 3DWorld scene at night. Now that I look at it, this seems closer to the reference image than my first screenshot.

Another view of night time 3DWorld scene.

Here is a similar scene with 100 spheres in day time. The bright sun makes it easier to see the different materials. This time I used transparent spheres and refractive glass spheres in addition to the other types. The glass spheres with higher index of refraction produce inverted refraction images. I removed the tiny spheres that were difficult to see.

Day time rendering of a similar spheres scene with strong sun lighting. This one has transparent/glass objects as well.

I also found a scene that had two large light emitters near a small glass sphere. This produces caustic lighting where the glass focused the light sources to two bright white spots of light on the ground plane.

Metal, glass, ceramic, and emissive spheres during the day. A caustic refraction can be seen near the center.

Here is the same scene and view, but at night. The indirect sphere lighting is easier to see.

Metal, glass, ceramic, and emissive spheres at night. The two white lights are focused through the glass sphere in the center to produce caustic lighting patterns on the ground.

Here is the same scene, but a different view. We're now looking at the other side of the glass sphere.

Same scene as above, but viewed from the front. The noise in the lighting is due to grid sampling artifacts.

That's it for now. I'll consider making a more complex and interesting scene in the future. I would also like to reduce the indirect lighting noise, though I don't know how to do it efficiently.

City Trees and Other Improvements

I haven't done too much work on cities since the last post. I've been working on a variety of other things in 3DWorld, including portals and a weapon that fires teleporters that bounce around the scene, teleporting anything they touch into the clouds above the scene. However, I'm not here to discuss these changes. You can read about these here.

Previously I had added windows and night time window lights to city buildings. The results looked pretty good, but I did go back and make some improvements to window lights. A Google image search for city lights gave me some ideas for how to make lighting look more realistic. I added a brightness gradient to make the bottom of the windows a bit darker, as if furniture and other objects were blocking some of the light. I made the dark windows have a small amount of light, as if there was some indirect lighting coming from adjacent lit rooms. I also added some random color variation to window lights. Colors are washed out by the bloom effect in bright windows but are more apparent in the darker windows.

The latest version of city window lights at night. Lights have variable brightness and slight color variation.

Night time city streetlights, car headlights, and building window lights.

I would like to add more detail to window lights to make them look more varied, such as dark shadows due to furniture and other objects blocking the light. Shamus Young did a good job with this in PixelCity. However, he took a pre-generated texture approach. That doesn't really work in my flow where the windows themselves are drawn in the fragment shader. I would need to somehow create the darker shapes and color variation within the shader itself. That should be possible, but so far I haven't figured out quite how to do it in a clean and efficient way.


The next type of city object I added was trees. I already had a system for generating and placing procedural trees in the scene, but it was quite a bit more work to add trees to cities. First, I had to generate tree placements that didn't overlap buildings, streetlights, traffic lights, roads, etc. Here are some screenshots showing trees placed in a city.

Procedurally generated and placed trees inside a city, with shadows.

Procedural city trees viewed from above.

The most difficult part was getting dynamic lighting (including shadows) from streetlights and car headlights to work with trees. Dynamic lighting and shadows works with trees in other scene/viewing modes, but I had to make it work with the city system and tiled terrain mode. It's more challenging to get the dynamic lighting solution to work when the player can move over a very large area. This required piecing together various bits of C++ and shader code, and a lot of trial-and-error. The process was different for tree branches vs. leaves, each of which uses a different rendering system. Here are the results. Note that tree leaves use a double sided lighting model, the light is facing downward, the camera is looking mostly at the unlit sides of leaves, which is why they appear so dark.

Night time city trees receiving light from cars and streetlights and casting shadows on the scene.

More lit and shadowing trees at night.

Another view of city trees at night.

I also made some improvements that I can't show in screenshots. For example, I implemented player collision detection with streetlights and traffic lights. [I already had collision detection with buildings.] I tried to add car collision detection, but I found that it didn't work very well. There were various problems. For one, it's too easy for the player to get stuck between parked cars and cars stopped at traffic lights. Maybe I need to make the player model smaller, or less round? It's not clear exactly how cars interact with the player anyway. I haven't figured out how to turn this into a game yet.

I'm still working on adding benches and other city details. I already have the framework to in place. Benches can be placed and integrated into the collision system. The problem is that it's not very fun and interesting. If I draw benches using downloaded 3D models like the cars, it's just more work finding models and fiddling with all the parameters to make them look good and draw fast at the same time. On the other hand, creating them myself is also not really something I want to do. I don't have any good 3D modeling software and I'm not sure how far I can get by hard-coding the shapes into the source code. I know I did that with streetlights and traffic lights, and I could do that again with benches. But it doesn't really seem like productive work. I just have to copy the code for those other objects and change it all to work with benches. Repeat for any other object type. I guess it would really help if I had someone with more artistic skills working on this project.

Update:
Someone suggested adding rows of lit window lights for buildings. I made the change, and here's how it looks:

Buildings with groups of lit windows on each floor.

You can definitely see the groups of lit windows now. Not all buildings should have long rows of lit vs. dark windows. I changed the shader code so that building window group sizes varies per-building and per-floor. I also made the ratio of lit to dark windows vary per-floor. Here is what that looks like:

Rows of lit windows that vary in group size per building and per floor.

Are these results better (as in more realistic)? I don't know, maybe, it's hard to tell.