# Ray Tracer Notes
## Rasterization vs. Ray Tracing
* The main difference is the order we process samples.
* Rasterizer processes one primitive at a time $\Rightarrow$ hard to take global illumination into account.
* Usually rasterizer use several "tricks" to make the picture realistic.
* On the other hand, ray tracer process one ray at a time. Ray know everything it intersects, so it can take global illumination into account.
## 3D Transformations
### Homogeneous Coordinates
* A *point* has a **nonzero** homogeneous coordinate.
* A *vector* has a **zero** homogeneous coordinate.
### Translate
$$\begin{bmatrix}
1 & 0 & 0 & \Delta x\\
0 & 1 & 0 & \Delta y\\
0 & 0 & 1 & \Delta z\\
0 & 0 & 0 & 1
\end{bmatrix}$$
### Scale
$$\begin{bmatrix}
a & 0 & 0 & 0\\
0 & b & 0 & 0\\
0 & 0 & c & 0\\
0 & 0 & 0 & 1
\end{bmatrix}$$
### Rotate
#### Around x axis
$$\begin{bmatrix}
1 & 0 & 0 & 0\\
0 & \cos\theta & -\sin\theta & 0\\
0 & \sin\theta & \cos\theta & 0\\
0 & 0 & 0 & 1
\end{bmatrix}$$
#### Around y axis
$$\begin{bmatrix}
\cos\theta & 0 & \sin\theta & 0\\
0 & 1 & 0 & 0\\
-\sin\theta & 0 & \cos\theta & 0\\
0 & 0 & 0 & 1
\end{bmatrix}$$
#### Around z axis
$$\begin{bmatrix}
\cos\theta & -\sin\theta & 0 & 0\\
\sin\theta & \cos\theta & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & 1
\end{bmatrix}$$
#### Around arbitary axis
* Rotate around axis $\mathbf{u}$ by $\theta$,
$$\begin{bmatrix}\cos \theta +u_{x}^{2}\left(1-\cos \theta \right)&u_{x}u_{y}\left(1-\cos \theta \right)-u_{z}\sin \theta &u_{x}u_{z}\left(1-\cos \theta \right)+u_{y}\sin \theta & 0 \\u_{y}u_{x}\left(1-\cos \theta \right)+u_{z}\sin \theta &\cos \theta +u_{y}^{2}\left(1-\cos \theta \right)&u_{y}u_{z}\left(1-\cos \theta \right)-u_{x}\sin \theta & 0 \\u_{z}u_{x}\left(1-\cos \theta \right)-u_{y}\sin \theta &u_{z}u_{y}\left(1-\cos \theta \right)+u_{x}\sin \theta &\cos \theta +u_{z}^{2}\left(1-\cos \theta \right) & 0\\0&0&0&1\end{bmatrix}
$$
## Camera
### Depth of field (defocus blur)
* Real camera need a big hole (rather than just a pinhole) to gather light. That hole is called the "aperture".
* The bigger the aperture, the more defocus blur we will get.
* We can randomly sample points on the lens, and shoot rays from the sampled points to the focal plane to produce defocus blur.
### Positionable camera
#### Setting 1
```cpp
Vec3 w = (lookfrom-lookat).unit_vector();
v = (vup-dot(vup, w)*w).unit_vector();
u = cross(v, w);
origin = lookfrom;
horizontal = focus_dist * viewpoint_width * u;
vertical = focus_dist * viewpoint_height * v;
lower_left_corner = origin - horizontal/2 - vertical/2 - focus_dist*w;
```
#### Setting 2
:::spoiler
```cpp
vec3 w;
w = (lookfrom - lookat).unit_vector();
u = -(vup - dot(vup, w) * w).unit_vector();
v = cross(w, u);
origin = lookfrom;
vertical = focus_dist * viewpoint_height * u;
horizontal = focus_dist * viewpoint_width * v;
upper_left_corner = origin - vertical / 2 - horizontal / 2 - focus_dist * w;
lens_radius = aperture / 2;
```
```cpp
ray get_ray(double s, double t) {
vec3 rd = lens_radius * vec3::random_in_unit_disk();
vec3 offset = u * rd.x() + v * rd.y(); // depth of field
return ray(
origin + offset,
upper_left_corner + s * vertical + t * horizontal - (origin + offset)
);
}
```
:::
## Color
### Gamma correction
* Gamma correction is a nonlinear operation used to encode and decode luminance values in digital systems.
* Gamma encoding of images is used to optimize the usage of bits when encoding an image, or bandwidth used to transport an image, by taking advantage of the non-linear manner in which humans perceive light and color.
* If images are not gamma-encoded, they allocate too many bits or too much bandwidth to highlights that humans cannot differentiate, and too few bits or too little bandwidth to shadow values that humans are sensitive to and would require more bits/bandwidth to maintain the same visual quality.
* In most computer display systems, images are encoded with a gamma of about 0.45 and decoded with the reciprocal gamma of 2.2.
* Usually, 8-bit PPM format stores colors in a nonlinear format, conventionally CIE Rec. 709 for red, green, and blue, adjusted by the CIE Rec. 709 gamma transfer function.
Rec. 709 OETF is as follows:
$$V=\begin{cases}4.500L&L<0.018\\1.099L^{0.45}-0.099&L\geq 0.018\end{cases}$$
* which can be approximated by a pure power function with a gamma 0.50-0.53 (about 1/1.9 - 1/2.0).
$$V_{\text{out}}=V_{\text{in}}^{\gamma}$$
## Radiometry
### Measures
#### System of units and measures for measuring EM radiation
#### Irradiance falloff with distance
#### Radiance
* Spectral radiance: radiance per unit wavelength
* Rendering is all about computing radiance!
#### Compute irradiance from radiance
### Lambert's law
Irradiance at surface is propotional to cosine of angle between light direction and surface normal.
### Ambient occlusion
* Assume there is a spherical light source at infinity.
* Irradiance is now rotation, translation invariant.
* Precompute and append to texture to enhance shading.
## Rendering Theory
### The rendering equation
* The rendering equation is recursive $\Rightarrow$ difficult to evaluate.
### Reflection functions
#### The BRDF
* Bidirectional reflectance distribution function: $f_r(\omega_i\rightarrow\omega_o)$
* Given incoming direction $\omega_i$, how much light gets scattered in outgoing direction $\omega_o$?
* Positivity: $f_r(\omega_i\rightarrow\omega_o)\geq0$
* Helmholtz reciprocity: $f_r(\omega_i\rightarrow\omega_o)=f_r(\omega_o\rightarrow\omega_i)$
* Conservation of energy: $\int_{H^2} f_r(\omega_i\rightarrow\omega_o)\cos{\theta_o}\, d\omega_o \le 1$
##### Radiometric description of BRDF
$$f_r(\omega_i\rightarrow\omega_o)={\frac {\operatorname {d} L_o(\omega _o)}{\operatorname {d} E_i(\omega _i)}}={\frac {\operatorname {d} L_o(\omega _o)}{L_i(\omega _i)\cos \theta _i\,\operatorname {d} \omega _i}}$$
* "For a given change in the incident irradiance, how much does the exitant radiant change?"
#### The BSSRDF
* The BRDF cannot model *subsurface light transport*.
* The BSSRDF generalizes the BRDF and describes the more general setting of light reflection from translucent materials.
#### The refection equation
$$L_o(p,\omega_o)=\int_{H^2}f_r(p,\omega_i \rightarrow \omega_o)L_i(p,\omega_i)\cos\theta_i\,\mathrm{d}\omega_i$$
* Approximately,
$$L_o(p,\omega_o)\approx\frac{1}{N}\sum_{i=1}^N\frac{f_r(p,\omega_i \rightarrow \omega_o)L_i(p,\omega_i)\cos\theta_i}{p(\omega_i)}$$
* To reduce variance, $p(\omega)$ should match BRDF or incident radiance function.
#### Example: Lambertian reflection
### Models of scattering
#### Refraction
* Suppose incoming ray is $\mathbf{u}$, outgoing ray is $\mathbf{v}$, and normal vector of the plane is $\mathbf{n}$ (all are unit vectors). Then,
$$\mathbf{v}_{\parallel}=\frac{\eta}{\eta'}(\mathbf{u}+\cos\theta\mathbf{n})\\
\mathbf{v}_{\perp}=-\sqrt{1-\|\mathbf{v}_{\parallel}\|^2}\,\mathbf{n}\\
\mathbf{v}=\mathbf{v}_{\parallel}+\mathbf{v}_{\perp}
$$
##### Total internel reflectance
* If $\frac{\eta}{\eta'}\sin\theta>1$, there is no solution for $\sin\theta'$. The material must reflect the ray.
#### Fresnel reflectance for dielectrics
* The Fresnel equations describe the amount of light reflected from a surface; they are the solution to Maxwell’s equations at smooth surfaces.
* The Fresnel reflectance formulae for dielectrics are
$$r_\parallel=\frac{\eta'\cos\theta-\eta\cos\theta'}{\eta'\cos\theta+\eta\cos\theta'}\\
r_\perp=\frac{\eta\cos\theta-\eta'\cos\theta'}{\eta\cos\theta+\eta'\cos\theta'}$$ where $r_\parallel$ is the Fresnel reflectance for parallel polarized light and $r_\perp$ is the reflectance for perpendicular polarized light.
* For unpolarized light, the Fresnel reflectance is
$$F_r=\frac{1}{2}\left(r_\parallel^2+r_\perp^2\right)$$
##### Schlick's approximation
* Schlick’s approximation, named after Christophe Schlick, is a formula for approximating the contribution of the Fresnel factor in the specular reflection of light from a non-conducting interface (surface) between two media.
$$R(\theta )=R_{0}+(1-R_{0})(1-\cos \theta )^{5}$$ where$$R_{0}=\left({\frac {\eta-\eta'}{\eta+\eta'}}\right)^{2}$$
#### Fresnel reflectance for conductors
#### Specular reflection
* Suppose incoming ray is $\mathbf{u}$, outgoing ray is $\mathbf{v}$, and normal vector of the plane is $\mathbf{n}$ (all are unit vectors). Then,
$$\mathbf{v}=\mathbf{u}-2(\mathbf{u}\cdot\mathbf{n})\mathbf{n}$$
## Monte Carlo Rendering
* The rendering equation is difficult to solve $\Rightarrow$ use numerical methods to render image instead.
### Monte carlo integration
$$\lim_{N\rightarrow\infty}\frac{|\Omega|}{N}\sum_{i=1}^Nf(X_i)=\int_\Omega f(x)dx$$
* Error of estimate is independent of the dimensionality of the integrand.
* Depends on the number of random samples used: $O(N^{1/2})$.
### Importance sampling
* We want to use monte carlo integration to calculate the area under the curve.
* If we uniformly pick samples in the function domain, we will get many small values (for example, when the sampled point is on the right hand side of the function), which doesn't help a lot in evaluating the integral.
* Therefore, we use importance sampling so that we have higher change in sampling points that contributes more to the integral (as the right picture shows).
$$\int_\Omega f(x)dx\approx\frac{1}{N}\sum_{i=1}^N\frac{f(X_i)}{p(X_i)}$$
### Cosine-weighted sampling
* This is achieved by picking random points on the surface of the unit sphere, offset along the surface normal. Picking random points on the unit sphere can be achieved by picking random points in the unit sphere, and then normalizing those.
### Sampling Lights Directly
### Russian roulette
* Randomly discard low-contribution samples in the way that leaves estimator unbiased.
### Path Tracing
* We would like to estimate the value of the exitant radiance from the camera ray's intersection point $p_1$,
$$L(p_1\rightarrow p_0)=\sum_{i=1}^{\infty}P(\bar p_i)$$
* The Monte Carlo estimate of $P(\bar p_i)$ is
$$\frac{L_e(p_i\to p_{i-1})f(p_i\to p_{i-1}\to p_{i-2})G(p_i\leftrightarrow p_{i-1})}{p_A(p_i)}\times\prod_{j=1}^{i-2}\frac{f(p_{j+1}\to p_j\to p_{j-1})|\cos\theta_j|}{p_\omega(p_{j+1}-p_j)}$$where$$G(p_i\leftrightarrow p_{i-1})=V(p_i\leftrightarrow p_{i-1})\frac{|\cos\theta_i||\cos\theta_{i-1}|}{\|p_i-p_{i-1}\|^2}$$$$V(p_i\leftrightarrow p_{i-1})=\begin{cases}0 & p_i\text{ and } p_{i-1} \text{ is occluded}\\ 1 & p_i\text{ and } p_{i-1} \text{ is unoccluded}\end{cases}$$
* In practice, we use multiple importance sampling-based direct lighting instead of using $p_A(p_i)$ to sample the light and estimate its radiance.
## Material
### Matte
* [Lambertian reflection](#Example-Lambertian-reflection)
* Use [Cosine weighted sampling](#Cosine-weighted-sampling) to render the scene.
### Mirror
* [Specular reflection](#Specular-reflection)
### Metal
* Approximate BRDF by perturbing the endpoint of the reflected ray.
* Another physically based approach to simulate the roughness of metal is by introducing the microfacet model.
### Glass
* When a ray hits it, the ray splits into a reflected ray and a refracted ray.
* We can randomly choose between reflection or refraction, and only generate one scattered ray per interaction.
* [Refraction](#Refraction)
* [Fresnel reflectance](#Fresnel-reflectance)
### Plastic
under construction...
## Texture
### Texture coordinates
#### Sphere
## Sampling
### Sampling a triangle
* To sample a barycentric coordinate $(u,v)$ of a triangle uniformly with respect to area,
$$u=1-\sqrt{\xi_1}\\v=\xi_2\sqrt{\xi_1}$$where $\xi_1$ and $\xi_2$ are two uniform random variables on $[0,1)$.
### Sampling a disk
$$r=\sqrt{\xi_1}\\\theta=2\pi\xi_2$$
### Sampling a sphere
$$x=\sqrt{1-z^2}\cos(2\pi\xi_2)\\
y=\sqrt{1-z^2}\sin(2\pi\xi_2)\\z=1-2\xi_1$$
### Cosine-weighted sampling a hemisphere
* *Malley's Method*: Uniformly sample points on the unit disk and project them up to the unit sphere.
* Another approach is by sampling a sphere whose center is $p$ offsected by unit normal. This approach is adopted by [Ray Tracing in One Weekend](https://raytracing.github.io/books/RayTracingInOneWeekend.html#diffusematerials/truelambertianreflection) and can be proven to be correct ([reference](https://github.com/RayTracing/raytracing.github.io/issues/145)).
### Multiple Importance Sampling
* If two sampling distributions $p_f$ and $p_g$ are used to estimate the value of $\int f(x)g(x)\mathrm{d}x$, the new Monte Carlo estimator given by MIS is
$$\frac{1}{n_f}\sum_{i=1}^{n_f}\frac{f(X_i)g(X_i)w_f(X_i)}{p_f(X_i)}+\frac{1}{n_g}\sum_{j=1}^{n_g}\frac{f(Y_j)g(Y_j)w_g(Y_j)}{p_g(Y_j)}$$where $n_f$ is the number of samples taken from the $p_f$ distribution method, $n_g$ is the number of samples taken from $p_g$, and and $w_f$ and $w_g$ are special weighting functions chosen such that the expected value of this estimator is the value of the integral of $f(x)g(x)$. A particular set of weighting functions is referred to as a *combination strategy*.
* All *combination strategies* yield unbiased estimators, but they can differ in their variance. The two most commonly used combination strategies are the *balance* and *power heuristics*, sharing the common form
$$\frac{(n_sp_s(x))^\beta}{\sum_i(n_ip_i(x))^\beta}$$
* For the *balance heuristic*, we have $\beta=1$. Veach and Guibas showed that no other combination strategy can have significantly lower variance than the balance heuristic.
* The *power heuristic*, for $\beta>1$, is a strategy better suited for low-variance problems, i.e., those where one $p_i$ closely matches the integrand. Usually we set $\beta=2$, the choice that Veach and Guibas considered the best.
## Lights
### Conversion between area and solid angle
* Differential area is related to differential solid angle by
$$\mathrm{d}\omega=\frac{\mathrm{d}A\cos\theta}{r^2}$$
where $\theta$ is the angle between the surface normal of $\mathrm{d}A$ and the vector to $p$, and $r$ is the distance from $p$ to $\mathrm{d}A$. ([reference](https://pbr-book.org/3ed-2018/Color_and_Radiometry/Working_with_Radiometric_Integrals#IntegralsoverArea))
* Conversion between a density with respect to area to a density with respect to solid angle can be done by multipling by the factor $\frac{r^2}{\cos\theta}$, since
$$\mathrm{d}A=\frac{r^2\mathrm{d}\omega}{\cos\theta}$$
* For example, suppose we are uniformly sampling a small area $\mathrm{d}A$ on the surface area of a shape whose total area is $A$, the probability density of sampling any small area $\mathrm{d}A$ on the surface is
$$\frac{\mathrm{d}A}{A}=\frac{\frac{r^2\mathrm{d}\omega}{\cos\theta}}{A}=\frac{r^2\mathrm{d}\omega}{A\cos\theta}$$
## Spatial data structures
### BVH
[BVH ideas & algorithms - HackMD](https://hackmd.io/@zOZhMrk6TWqOaocQT3Oa0A/HJUqrveG5)
### K-D tree
### Uniform Grid
* Best suited for scenes whose primitives are uniformly distributed.
### Comparison
## Antialiasing
* Aliasing occurs when there is an abrupt difference between pixels.
* This picture shows when we undersample high-frequency signals, aliasing will occur:
* An antialiasing method is *supersampling*:
## Miscellaneous
### Shadow acne problem
* Floating point precision issue
* [A Fast and Robust Method for Avoiding Self-Intersection](https://link.springer.com/content/pdf/10.1007%2F978-1-4842-4427-2_6.pdf)
### Phong shading
#### Visual illustration
#### Calculation
* The hats indicate that the vectors are normalized.
* $k_a$ is "ambient reflection constant". The ratio of reflection of the ambient term present in all points in the scene rendered.
* $k_d$ is "diffuse reflection constant". The ratio of reflection of the diffuse term of incoming light.
* $k_s$ is "specular reflection constant". The ratio of reflection of the specular term of incoming light.
* $\alpha$ is "shininess constant for this material", which is larger for surfaces that are smoother and more mirror-like. When this constant is large the specular highlight is small.
* $i_a$ controls the ambient lighting; it is sometimes computed as a sum of contributions from all light sources.
* $i_d$ and $i_s$ are defined as the intensities of the diffuse and specular components of the light sources.
* $\text{lights}$, which is the set of all light sources.
* ${\hat {L}}_{m}$, which is the direction vector from the point on the surface toward each light source ($m$ specifies the light source).
* $\hat {N}$, which is the normal at this point on the surface.
* $\hat {R}_{m}$, which is the direction that a perfectly reflected ray of light would take from this point on the surface.
* $\hat {V}$, which is the direction pointing towards the viewer (such as a virtual camera).
* Each term should only be included if the term's dot product is positive. (Additionally, the specular term should only be included if the dot product of the diffuse term is positive.)
* [Optimal multiple importance sampling | ACM Transactions on Graphics](https://dl.acm.org/doi/10.1145/3306346.3323009)
## References
[15-462/662 Fall 2020](http://15462.courses.cs.cmu.edu/fall2020/)
[Radiometric Concepts | Radiometry and Reflectance - YouTube](https://www.youtube.com/watch?v=tflz0loWhIY)
[Phong shading - Wikipedia](https://en.wikipedia.org/wiki/Phong_shading)
[Schlick's approximation - Wikipedia](https://en.wikipedia.org/wiki/Schlick's_approximation)
[Importance Sampling - YouTube](https://www.youtube.com/watch?v=C3p2wI4RAi8)
[Memo on Fresnel equations | Sébastien Lagarde](https://seblagarde.wordpress.com/2013/04/29/memo-on-fresnel-equations/)
[Microfacet BRDF: Theory and Implementation of Basic PBR Materials [Shaders Monthly #9] - YouTube](https://www.youtube.com/watch?v=gya7x9H3mV0)
[Optimal multiple importance sampling | ACM Transactions on Graphics](https://dl.acm.org/doi/10.1145/3306346.3323009)
[Physically Based Rendering: From Theory to Implementation](https://pbr-book.org/)
#### Ray Tracing in One Weekend Series
[Ray Tracing in One Weekend](https://raytracing.github.io/books/RayTracingInOneWeekend.html)
[Ray Tracing: The Next Week](https://raytracing.github.io/books/RayTracingTheNextWeek.html)
[Ray Tracing: The Rest of Your Life](https://raytracing.github.io/books/RayTracingTheRestOfYourLife.html)
Sign in with Wallet
Connect another wallet