Anti-Aliasing Differences: FXAA vs TAA vs SMAA? Bilinear vs Trilinear vs Anisotropy Filtering

SMAA, FXAA, TAA, or DLAA, which one should you choose? Four options and this is just anti-aliasing we’re talking about here. Modern games include a slew of graphics settings to choose from, to get the best performance from your hardware. Other than Anti-Aliasing (AA), you’ve got Ambient Occlusion, Screen Space Reflections, Shadows, Texture Filtering, Post-Processing, and much more.

Anti-Aliasing: MSAA vs FXAA vs SMAA vs TAA

Anti-aliasing is one of the most common graphics options found in video games. You’ve got the traditional MSAA and SSAA, and the newer shader-based FXAA, SMAA, and TAA that have become the norm. So, what does anti-aliasing do? In short, it gives the image a cleaner look by removing the rough or jagged edges of complex objects.

Above is how FXAA (fast approximate anti-aliasing) improves the image quality by reducing the jaggies. Here’s a comparison of how AA impacts your game. Below you can see SMAA in action:

The differences are subtle but exist across the entire image. For example, the electric pole and wiring. They lose the “teeth” along their edges when SMAA is turned on. The buntings and the vegetation also get the same treatment. However, unlike FXAA, SMAA isn’t too strong. It gets rid of the aliasing, without blurring the texture detail. This is because it uses a more precise edge-detected technique while FXAA relies on simple luma-based edge detection.

Types of Anti-Aliasing

Traditional AA: These include MSAA (Multi-sampling AA) and SSAA (Super Sampling AA) which were popular last-gen and for good reason. They produce the best image quality but severely diminish the performance. They work by rendering the image at a higher resolution and then scaling it down to fit the native resolution. This essentially makes the entire image sharper and more detailed, scaling down the rough edges in the process but not removing them entirely.

Supersampling or SSAA renders the entire frame at a higher resolution and then scales it down to fit the target resolution. You are sampling the color and depth samples two or four times per pixel instead of one. The final pixel color is determined by taking the average of the four samples. This significantly improves image quality, eliminating the jaggies. Unfortunately, it’s extremely expensive as you’re essentially rendering the image at 2x or 4x the native resolution.

Multi-sampling or MSAA works similarly to SSAA, but it focuses only on the edges rather than the entire frame. Like SSAA, the amount of additional sampling varies from 2x to 8x. The rasterization stage of both SSAA and MSAA are identical. The depth (buffer) sample of each pixel is rendered at Nx where N is the sampling rate. This includes coverage and occlusion tests (checking which triangles are visible).

The difference between MSAA and SSAA boils down to the latter half of the pipeline. While SSAA applies the pixel shader to each pixel multiple times, MSAA executes it once. It only samples the depth buffer, smoothening out the edges in the process.

One drawback of MSAA is that it can’t be used with deferred rendering.

Shader-based AA is more efficient and has replaced MSAA/SSAA in modern games. They work by blending neighboring pixels, making the image smoother but also reducing the sharpness. FXAA uses luma or contrast-based edge detection, which isn’t very accurate and tends to blur objects with complex multi-colored patterns.

SMAA was introduced to rectify the shortcomings of FXAA and MLAA, and it works well in most scenarios. It leverages edge detection based on color-specific luma.

SMAA uses multiple (1-pixel long for sharp geometry and multiple pixels long for diagonal patterns) crossing lines to enhance edge detection, reduce blurring, and improve coverage. With thin lines prone to blurring, a rounding factor scales the coverage areas obtained by the pixel-long crossing edges.

Temporal Anti-aliasing or TAA is the latest and most popular AA technique. TAA compares neighboring frames (temporally) and blends them to create a cleaner image in motion. The following is a rough description of a simple TAA filter.

• The present frame (n) is rendered along with the geometry and shading.
• Frame n is reprojected on the previous image using the jitter offsets and motion vectors.
• A rectify filter is used to compare the frames and check for any ghosting.
• Finally, the post-processing effects are applied, thereby completing the frame.
• This frame is used for reconstructing (by reprojection) the next consecutive frame, and the process continues.

TAA is an approximation. It uses two images to extrapolate the final image, so it also causes a good deal of blurring, losing some texture detail in the process. The image is rendered across different frames at a particular resolution, and with each additional image, a jitter offset (camera shift of a few pixels) is applied. This produces multiple images from a single frame which are then used for upscaling.

Temporal upscaling uses a similar method to upscale lower-resolution images. The core difference is that, unlike TAA, alternating pixels are rendered in consecutive frames, and the gaps are filled using interpolation (samples from neighboring pixels). Here’s a comparison of FXAA vs TAA.

NVIDIA’s DLAA is a temporal anti-aliasing solution. It works like DLSS but uses native-resolution images instead of low-res inputs. These images and the motion vectors are fed to the neural network, producing a high-quality result generally better than native. The resultant frame is then fed to the network as the temporal feedback to accumulate and blend future frames.

Ambient Occlusion

On the next page…

Ambient Occlusion

Ambient occlusion refers to the ambient shadows that exist in crevices, edges, and surfaces hidden from the sun. It’s a form of global illumination or indirect lighting. The casting object and the shadow often overlap here. The most popular ambient occlusion technique is Screen Space Ambient Occlusion and its improved variant Horizontal Ambient Occlusion.

Newer GI techniques such as RTGI are gaining prominence, but they are quite performance-intensive. Voxels (3D triangles) form the basis of VXAO/SVOGI.

Ray Traced Global Illumination (RTGI)

Global Illumination is the process of lighting or illuminating a scene by calculating the light emitted by luminant sources on and off the screen (by approximations or ray tracing). The rays are cast and illuminate various objects in the scene as they hit them. The rays behave differently depending on the nature of the objects. For example, glossy objects result in reflections, while less glossy or opaque ones redirect it somewhere else. This redirection of light rays is known as indirect or diffuse lighting.

The diffuse rays act as newly cast rays, further crashing into other objects and illuminating more of the scene. The object that redirects them acts as an independent light source. When the ray finally reaches the camera, the information gathered by it is used to determine the lighting of the scene.

Continue reading about RTGI here.

Texture Filtering

Texture filtering is one of those settings that makes your game look sharper. Well, then how is it different from the regular sharpening filter? It makes sure that the mipmaps of textures are properly visible from the in-game camera. Traditionally, mipmaps are smaller than the original texture by a factor of 2 and there are points (texels) where multiple mipmaps may converge. These must be filtered to avoid blurring and other artifacts.

Bilinear filtering, the simplest form of texture filtering uses the following approach to calculate the color of a texel: It takes four texel samples from the approximate position of the texel (a texel is to texture what a pixel is to resolution) as indicated by the game engine and calculates its average which is then used as the final value. However, bilinear filtering only uses samples or texels from mipmaps identified by the game engine and if at any point with perspective-distorted textures, two different mipmaps are used, there are shifts in texture clarity.

Trilinear filtering improves on bilinear filtering by continuously sampling and interpolating (averaging) texels from the two closest mipmap sizes for the target texel, but like BF, this technique assumes that the texture is displayed as a square from the player’s perspective, and suffers a loss in quality when viewed from an oblique angle (especially when perpendicular to the screen).

This is due to the texel covering a depth (area along the axis perpendicular to the screen) longer than and a width narrower than the samples extracted from the mipmaps, resulting in blurring due to under and over-sampling, respectively.

To solve this, Anisotropic Filtering is used which scales either the height or width of a mipmap by a ratio relative to the angle of the texture against the screen. The ratio is dependent on the maximum sampling value specified, followed by taking the appropriate samples. AF can function with anisotropy levels between 1 (no scaling) and 16, defining the maximum degree by which a mipmap can be scaled but AF is commonly offered to the user in powers of two: 2x, 4x, 8x, and 16x.

The difference between these settings is the maximum angle that AF will filter the texture by. For example, 4x will filter textures at angles twice as steep as 2x, but will still apply standard 2x filtering to textures within the 2x range to optimize performance.