---
# System prepended metadata

title: Computer Graphics
tags: [note, computer graphic]

---

# Computer Graphics
## author: Panda、Buffett
## hardware
The Rendering Pipeline
Host interface $\rightarrow$Vertex processing $\rightarrow$Triangle setup $\rightarrow$Pixel processing $\rightarrow$Memory interface

Host interface: 
- move data from CPU to GPU

Vertex Processing: 
- obj space $\rightarrow$screen space
- Normals, texture coordinates etc. also transformed

Triangle setup:
- screen $\rightarrow$ pixel(fragment)

Pixel processing:
- **performance bottleneck**
- color/texture

Memory interface: 
- some fragments are rejected by the Tests
- produce final image

## IntroductionGLSL
rendering
 - Converting geometric object description into frame buffer values 
 - convert model to image
croodinate
(x,y,z,w) => (x/w,y/w,z/w)
![image](https://hackmd.io/_uploads/BylkWfrQC.png)
**rasterization**:determine which pixels should be processed

screen space geometry is the input, pixel are the output

### GLSL program
#### keyword of shader
**uniform**: 
- pass data from js to GLSL can be used in both vertex and fragment shader

**attribute**:
- only for vertex shader
- same attribute variable of all thread suppose have deffrent values

**varying**: 
- pass data from vertex and fragment shader
- interpolation
    - use on a surface

**precision** highp/mediump/lowp float
#### Vertex Shader(per vertex operation)
gl_Position:a vector with 4 elements(vec4) the position of pixel (want clip space position)
gl_PointSize: size of a point(must be float)
#### Fragment Shader(per pixel operation)
calculate the color(texture) of each pixel
gl:FragColor:a vector with 4 elements
#### compile shader
1. creater vertex and fragment shader
2. source code to shader
3. compile the shader
4. create program
5. attach shader to program
6. link program
### initialize
1. get canvas from html:document.getElementByID('webgl')
2. get context of webgl canvas.getContext(webgl2)
3. compile shader to program
4. use the program on gl
5. set background (RGBA):gl.clearColor(R,G,B,A)
6. clear screen by background color:gl.clear(gl.COLOR_BUFFER_BIT)
## VBO
![image](https://hackmd.io/_uploads/ByEvtbHQ0.png)
1. create vertex buffer: gl.createBuffer()
2. bind the buffer:gl.bindBuffer()
3. write vertice to buffer:gl.bufferData()
4. get attribute location:gl.getAttribLocation(program, attribute name)
5. assign buffer to variable in vertex shader:
    - gl.vertexAttribPointer(att, size, data type, normal,stride, offset)
    - att:attribute you want to pass
    - size:the number of data
    - normalize the data to a range or not
    - stride:the offset in bytes of every data(how many bytes you want to jump when go through another vertex)
    - offset:the offset of first attribute 
## Transformation
**transformation the object space(move the axis of object)**
### homogeneous
with n-D, we need a vector with n+1 element(x,y,z,...,w)
w is 1
and coordiniate is $(\frac{x}{w},\frac{y}{w},\frac{z}{w},...)$

### translation
move x,y a offset $t_x$, $t_y$ to x+$t_x$,y+$t_y$
$\begin{bmatrix}
1 & 0 & t_x\\
0 & 1 & t_y\\
0 & 0 & 1 
\end{bmatrix}\begin{bmatrix}
x\\
y\\
w
\end{bmatrix}$
### scaling
scale x,y to $s_x\times x$,$s_y\times y$
$\begin{bmatrix}
s_x & 0 & 0\\
0 & s_y & 0\\
0 & 0 & 1 
\end{bmatrix}\begin{bmatrix}
x\\
y\\
w
\end{bmatrix}$
### rotation
rotate x,y $\theta$degrees
$\begin{bmatrix}
\cos\theta & -\sin\theta & 0\\
\sin\theta & \cos\theta & 0\\
0 & 0 & 1 
\end{bmatrix}\begin{bmatrix}
x\\
y\\
w
\end{bmatrix}$
#### 3D rotation
![image](https://hackmd.io/_uploads/H1ETtiHmC.png)
given axis of rotation $u=(u_x,u_y,u_x)$(u is unit vector)
and rotate theta
rotate x,y $\theta$degrees
$\begin{bmatrix}
\cos\theta+u_x^2(1-\cos\theta) & u_xu_y(1-\cos\theta)-u_z\sin\theta & u_xu_z(1-\cos\theta)+u_y\sin\theta & 0\\
u_yu_z(1-\cos\theta) & \cos\theta+u_y^2(1-\cos\theta) & u_yu_z(1-\cos\theta)-u_x\sin\theta & 0\\
u_zu_x(1-\cos\theta)-u_y\sin\theta & u_zu_y(1-\cos\theta)-u_x\sin\theta & \cos\theta+u_z^2(1-\cos\theta) & 0 \\
0 & 0& 0&1
\end{bmatrix}\begin{bmatrix}
x\\
y\\
z\\
w
\end{bmatrix}$
### HierachicalTransformation
use a stack to save all object matrix make them connect together
## SpaceTransformationPipeline
$local\ space\xrightarrow{model\ matrix}world\ space\xrightarrow{view\ matrix}view\ space\xrightarrow{projection\ matrix}clip\ space\xrightarrow{view port\ transorm}screen\ space$
![image](https://hackmd.io/_uploads/rkb-NpS70.png)

### view matrix
(world to view Space)
![image](https://hackmd.io/_uploads/Sk5VOnrmA.png)
$N=eye-col$
$n=\frac{N}{|N|}$ like z axis
$U=v_{up}\times n$
$u = \frac{U}{|U|}$
$v=n\times u$
t is translation
$view\ matrix=\begin{bmatrix}
u_0 & v_0 & n_0 & 0\\
u_1 & v_1 & n_1 & 0\\
u_2 & v_2 & n_2 & 0\\
0 & 0 & 0 & 1\\
\end{bmatrix}\times\begin{bmatrix}
1 & 0 & 0 & t_0\\
0 & 1 & 0 & t_1\\
0 & 0 & 1 & t_2\\
0 & 0 & 0 & 1\\
\end{bmatrix}$
$View matrix (𝑀)=rotation\ \times\ translation$

### Projection Matrix 
(View to Clip Space)
![image](https://hackmd.io/_uploads/ryRCnnB7C.png)
#### Orthographic

$\begin{bmatrix}
\frac{1}{width} & 0 & 0 & 0\\
0 & \frac{1}{height} & 0 & 0\\
0 & 0 & -\frac{2}{Z_{far}-Z_{near}} & -\frac{Z_{far}+Z_{near}}{Z_{far}-Z_{near}}\\
0 & 0 & 0 & 1\\
\end{bmatrix}$
#### Perspective
$\begin{bmatrix}
\tan^{-1}(\frac{FOV_x}{2}) & 0 & 0 & 0\\
0 & \tan^{-1}(\frac{FOV_y}{2}) & 0 & 0\\
0 & 0 & -\frac{Z_{far}+Z_{near}}{Z_{far}-Z_{near}} & -\frac{2Z_{far}Z_{near}}{Z_{far}-Z_{near}}\\
0 & 0 & -1 & 0\\
\end{bmatrix}$
![image](https://hackmd.io/_uploads/SJ7l1aSm0.png =50%x)
### transformation
![image](https://hackmd.io/_uploads/SJywl6r7R.png)

$S_x=X\frac{V_r-V_l}{1-(-1)}+\frac{V_r+V_l}{1-(-1)}$
$S_y=Y\frac{V_t-V_b}{1-(-1)}+\frac{V_t+V_b}{1-(-1)}$
## Illumination
K is Illumination color
I is object color
Ambient Light
- light of envirment
- $K_a* I_a$
$K_a$ is Ambient Light color, $I_a$ is object color

Diffuse Light
![image](https://hackmd.io/_uploads/ByNEvarmA.png)
- $cos\theta=N*L$
- $K_d*I_d*cos\theta$

Speclar Light
![image](https://hackmd.io/_uploads/S1XinarmA.png)
- $K_s*I_s*cos^n(\phi)$
- n:surface normal
- $\phi$:angle between R,V
- $cos^n(\phi)$ n is larger, smaller value of this term
illumination
ambient+diffuse+speclar
### flat shader
- for each surface 1 color
-  fast
-  use at
    -  polygon small
    -  light is far
    -  eye is far
-  mach band effect

### gouraud shader
- for each vertex 1 color
### phong shader
- Interpolate normal with normal matrix
- $n_{real} = {M^{-1}}^T \times n$

## ElementArray
use a array to record each surface's vertices index in vertices
## LoadExternalModel
### obj file
`# comment`
`mtllib file`
`o name`
`v x y x` vertex position
`vt x y z` vertex textue coordinate
`vn x y z` normal vertex vector
`usemtl mtlName`
`f v1 v2 v3...` face with n vertices
v1,v2,v3... ->vi=v/vt/vn index()  can multiple  properties
**no color**

write parse function to parse obj file to webgl
## texture
### mapping texture
each vertex maping a location in picture
calculate surface by interpolation texture coordinates
![image](https://hackmd.io/_uploads/B1z1QbFmR.png)

![image](https://hackmd.io/_uploads/SyLDNZYmR.png)

![image](https://hackmd.io/_uploads/SJHMpzBQA.png =60%x)
### mtl file
`newmtl name` create new mtl
`Ns` 
shininess of the material
`Ka r g b`ambient light
`Kd r g b` diffuse light
`Ks r g b`specular light
`illum mode`illumination model
`d/Tr alpha` transparency
`map_Ka file` picture this texture use
## FrameBuffer
put what I want to show to a buffer for later use(e.g use like texture, reflection or shadow)
### Shadow
depth=Calculate the nearest obstacle to the light source on the line from each point to the light source
![image](https://hackmd.io/_uploads/SkKGKGK70.png)

we take picture from light(camera location=light location) and calculate depth
if shadowCoord > depth+deMachThreshold =>have shadow->make color deeper
### CubeMap
map some pictures  as a box
![image](https://hackmd.io/_uploads/BkAr3Gtm0.png)
can use to draw refraction or skybox
![image](https://hackmd.io/_uploads/Hy5UpMK7A.png)
->$\frac{position}{2}+0.5$ reset range to (0,1)
#### sky box
dont create cube, rander 6 quad which covers the whole camera view
![image](https://hackmd.io/_uploads/BkE-bQYXR.png)

#### Reflection
##### envirment
![image](https://hackmd.io/_uploads/rkr0QXtmC.png)
##### dynamic
use renfering of cubemap 
put camera at center of reflective obj, create cubemap texture on the fly
and look up this cubemap texture to the reflective object
#### Refraction
![image](https://hackmd.io/_uploads/BkBtH7YXC.png =60%x)
$\frac{sin\theta_1}{sin\theta_2} = \frac{n_1}{n_2}$
where n is refraction index

#### Rendering
take picture with 6 dictionary to record the background looklike and render on frame buffer
## Fog
$fogFactor(f)=\frac{end-distance(camera, objPoint)}{end-start}$
$color=(f\times objColor)+((1-f)\times fogColor)$
## Bump mapping
control the normal vector to let people think it high poly
![image](https://hackmd.io/_uploads/HySso7YQR.png)
normal.xyz set to normal map.rgb
(if we look object at z=0)
(so most surface is [0,0,1], and mapping to rgb is [0.5,0.5,1])
### normal map with tangent space
the vector is only correct for z-plane
->rotate the normal vector
-> transform normalmap from tangent space to object space
![image](https://hackmd.io/_uploads/BkA21VYQC.png)
$\begin{bmatrix}
 x_o \\
 y_o \\
 z_o \\
\end{bmatrix}=TBN\times\begin{bmatrix}
 x_n \\
 y_n \\
 z_n \\
\end{bmatrix}$
n is normal space
o is object space
![image](https://hackmd.io/_uploads/rkXgb4tmR.png)
$TBN\ matrix = [T,B,N]$
![image](https://hackmd.io/_uploads/r1USWVY7A.png)
if we use $T \times B$ and when we map triangle to texture space is clock-wise point order=>it will get wrong normal vector
=>check it's clock-wise or not
## WebGL Programming
### webgl datatype
mat4 (a,b,c,d...)
is meaning (colume major)
$\begin{bmatrix}
a & e & \dots \\
 b & f & \dots\\
 c & g & \dots\\
 d & h & \dots\\
\end{bmatrix}$
**get matrix element can't use nonconstant index**
vector*matrix :let vector be a $1\times n$ matrix
### draw
gl.drawArrays(mode, first, count)
run Vertex Shader, Fragment Shader
TRI: 012 345
STRIP:012 123
FAN:012 023
mode:
![image](https://hackmd.io/_uploads/r1j8TMS7R.png)

first:
- the first vertex for drawing in vertex array
count:
- number of vertices
### pass data
1. get reference of uniform variable
    gl.getUniformLocation(program, 'varible name')
2. pass data
gl.uniform$n$f(location, v0,...,v$n$), $n$ is 1-4
`e.g. gl.uniform4f(program.u_pos, r,g,b,a)`