Lighting in video games has always been held back due to limited processing power. Even now with ray-tracing (RTX), the number of lights is limited to just a dozen or two in most games. This is primarily due to the fact that even with Turing, the number the rays cast per scene are mostly less than one ray per pixel. To simulate realistic lighting with thousands of light sources if not millions, you’d need to cast at least a couple of rays for every source. With today’s technology that’s simply not possible.
However, NVIDIA and the VC Lab at Dartmouth College have somehow made it a reality, and that too with present-gen hardware. The method still uses ray-tracing instead of rasterization and doesn’t require any complex changes to the scene’s lighting. All lights cast shadows, everything can move arbitrarily, and new emitters can be added dynamically.
The paper, “Spatiotemporal Reservoir Resampling for Real-Time Ray Tracing with Dynamic Direct Lighting,” provides theoretical and implementation details for the reservoir-based spatiotemporal importance resampling (ReSTIR) technique.
Our key insight was to reuse information from spatially and temporally adjacent pixels to inform which rays to trace” says Benedikt Bitterli, one of the paper’s authors and a Dartmouth Ph.D. student recently awarded a 2020 NVIDIA graduate fellowship. “This idea resembles modern post-process denoising and antialiasing, but while denoising filters the pixel colors directly, we filter probabilities to help guide rays to reduce future noise.”
“This will bring about a sea change for content creation,” says Kevin Margo, CGI Creative Director at NVIDIA. “This research lets us light scenes intuitively, placing arbitrary numbers of emissive objects that cast dynamic shadows, and the lighting ‘just works’ in the renderer.”
This sounds a lot like another form of temporal rendering, just what DLSS 2.0 as well as present and upcoming game consoles leverage. As I already highlighted in my recent post, temporal techniques are going to be a lot more complex and fine-tuned in the coming years. It lets you render more complex scenes with the data you already have and without requiring too much processing power.
The PS5 Demo (Running @ 1440p) Proves Next-Gen Consoles’ll be All About Render Optimizations: VRS, Temporal & AI Upscaling