⎪⎪
⎨
⎪⎪
⎪⎩
1 0≤e≤ fl
1−Pp
fl−pl(e−fl) +1 fl<e≤ pl
Pp pl<e
, (4.1)
where flis the fovea limit,pl is the periphery limit, andPpis the periphery probabil-ity of path staying alive. In a typical implementation, probabilprobabil-ity 1 means one spp.
However, nothing prevents the use of more than one spp. The sample distribution of this kind of probabilities is visualized in Figure 4.2a.
One drawback of linear fall-off is that some post-processing techniques, like de-noising, require a full resolution G-buffer. In this case, one can first rasterize the G-buffer and use the probabilities for deciding if the path should continue recur-sively or not. It would also be possible to build hardware level support for this by decoupling shading and visibility[Vai+14]. The probabilities can also be modified based on the G-buffer data. For instance, better quality foveation can be achieved if more complicated shading is used in pixels which represent edges in the 3D scene [Ste+16].
4.2 Other Coordinate Spaces
In the recent years rendering in other screen coordinate spaces than Cartesian space have gained more interest [FRS19; Men+18]. One reason for the interest is that general purpose computing is currently fast enough for real-time applications, which allows other rendering methods than just hardware accelerated rasterization. For
(a)Linear Fall-Off (b)Log-Polar [Men+18]
(c)Visual Acuity
Figure 4.2 Sample distributions of different foveated path tracing methods. Higher and at the same time brighter vertexes mean more samples. The fovea is always at the center of the highest sampling.
instance, in ray tracing the screen space sample locations can be decided freely. In contrast, using other coordinate spaces with rasterization requires, for example, first rasterizing the G-buffer in Cartesian space[Men+18]or tessellation. Then, the G-buffer data is mapped to the other coordinate space for deferred shading and finally the shaded results are mapped back to the Cartesian screen space pixels. Even with all this mapping, the other coordinate space can be beneficial since shading is typically the most expensive operation and the coordinate spaces being used are designed so that they reduce work significantly.
4.2.1 Polar-Space
One simple way to render more samples at the gaze point is to shade a uniform grid of samples in polar coordinates[Men+18]. Specifically, to render a 2D frame, where one coordinate is the distanceρfrom the gaze point and the other coordinate is the angleφaround the gaze point.
The problem is that the distribution is not close to the visual acuity function.
Specifically, there are as many samples as the maximum of pixels in the angle coor-dinate which has a property ofρ= 0. With a typical resolution, this means that
there are approximately a thousand samples at the gaze point. A huge sample count in just one pixel is wasteful because the fovea is bigger and, therefore, the user is also seeing worse quality rendering. In addition, there is always some error in eye-tracking systems. For instance, the error can be approximately one eccentricity de-gree[Wei+18b].
When moving away from the gaze point, the count of samples radically decreases.
In fact, the sampling density on the eccentricity angle follows the function S(e) = max(φ)
ρ+1 , (4.2)
which is visualized with its solid of revolution in Figure 4.2b. Note that here the maximum spp is max(φ), which is measured in pixels in the Polar-Space frame. In other words the height of the Polar-Space frame if angle coordinateφis on vertical axis.
The distribution can be improved by using a nonlinearρaxis. One way is to take the natural logarithm of theρcoordinate[Men+18]. Another approach is to use a more complicated function to map the distribution to the visual acuity function [P5].
4.2.2 Visual Acuity Function
In some foveated ray tracing work the sample distribution follows the visual acuity function[Sie+19]. For instance the following function can be used:
V(e) =
⎧⎨
⎩
60.0 0≤e≤5.79
449.4
(0.3e+1)2 e>5.79 , (4.3)
where e is the eccentricity angle andV(e) is the visual acuity in cycles per degree [Red97]. In other words, the maximum number of times the image can change from completely white to black. Using Equation 4.3 for path tracing sampling with a contemporary display device requires taking the minimum of the equation and the highest number cycles per degree the used device is capable of displaying. By using Equation 4.3 for guiding the sampling, the samples are used exactly where they are needed and the gain from the foveation can be greater, assuming that the eye tracking system is good enough. In case of poor eye tracking accuracy, the fovea area can be
made bigger to allow room for some error.
This kind of more complicated sample distribution like the visual acuity func-tion can be achieved, for example, with precomputed sample locafunc-tions[Sie+19]or by computing pseudorandom sample locations with the wanted distribution[P3;
P4]. Both techniques have drawbacks. It is hard to move the precomputed sam-ple locations so that they would contribute well to such temporal effects as TAA or progressive path tracing. On the other hand, completely random locations break the coherence of the ray traversal. Both problems are avoided if Polar-Space sample distribution is modified to follow visual acuity function like in Visual-Polar space [P5].
4.2.3 Combining Visual Acuity Function with Content Features
Even more sampling can be reduced if the sample location decision takes the content into account. For instance, on top of foveation M. Stengel et al.[Ste+16]determine which samples to shade based on silhouettes, object saliency and specular highlights.
However, the decision process requires full resolution G-buffer. Then pull-push al-gorithm is used to fill data into in-between fragments. Similar sampling decision strategies can also be used with light field data[Sun+17]. Another approach makes the shading decisions with bigger granularity by using a low-resolution estimation of the frame[Tur+19]. The problem is that all the rendering passes must be doubled so that the low-resolution estimation can be initially generated.
Even though these methods can save 50-75% of sampling, all the extra steps com-bined with the sparse shading or compaction are not optimal. Therefore, the actual savings in the total timings are more modest or even negligible, at least in the case of simple fragment shaders.
4.3 Efficient Sample Distribution Implementations
For performance reasons, it is important to preserve the coherence of the primary rays and avoid sparse shading, in other words, fill all SIMT/SIMD lanes with mean-ingful work without the extra step of packing. This section describes two different ways that fulfill these requirements while still achieving the visual acuity function distribution of the samples.
One way to achieve visual acuity distributions is to stretch the grid of uniform sampling so that the area around the gaze point is magnified[FRS19]. The area near the gaze point gets more samples and peripheral areas get fewer samples compared to the original uniform sampling. A magnification function is needed for the sample distribution and its inverse is needed for mapping the samples back to the screen space.
Magnification can also be done with rasterization in a vertex shader. The new lo-cation of each vertex is decided with the magnifilo-cation function based on the vertex’s screen space location. However, this produces bent edges in the final screen space frame. Therefore, the primitive size should be kept small, which hides the bending effect. The size can be modified at run time either by selecting smaller LOD levels [Lue+03]or by splitting the primitive with a tessellation shader.
The benefits of magnification include preserving the coherence of the primary rays and filling all the SIMT/SIMD lines. The main drawback of magnification is that, like with polar spaces, when used with a rectangular grid of samples, some of them go outside of the screen area.