# Computer Graphics
## author: Panda、Buffett
## hardware
The Rendering Pipeline
Host interface $\rightarrow$Vertex processing $\rightarrow$Triangle setup $\rightarrow$Pixel processing $\rightarrow$Memory interface

Host interface:
- move data from CPU to GPU

Vertex Processing:
- obj space $\rightarrow$screen space
- Normals, texture coordinates etc. also transformed

Triangle setup:
- screen $\rightarrow$ pixel(fragment)

Pixel processing:
- **performance bottleneck**
- color/texture

Memory interface:
- some fragments are rejected by the Tests
- produce final image

## IntroductionGLSL
rendering
- Converting geometric object description into frame buffer values
- convert model to image
croodinate
(x,y,z,w) => (x/w,y/w,z/w)
**rasterization**:determine which pixels should be processed

screen space geometry is the input, pixel are the output

### GLSL program
#### keyword of shader
**uniform**:
- pass data from js to GLSL can be used in both vertex and fragment shader

**attribute**:
- only for vertex shader
- same attribute variable of all thread suppose have deffrent values

**varying**:
- pass data from vertex and fragment shader
- interpolation
- use on a surface

**precision** highp/mediump/lowp float
#### Vertex Shader(per vertex operation)
gl_Position:a vector with 4 elements(vec4) the position of pixel (want clip space position)
gl_PointSize: size of a point(must be float)
#### Fragment Shader(per pixel operation)
calculate the color(texture) of each pixel
gl:FragColor:a vector with 4 elements
#### compile shader
1. creater vertex and fragment shader
2. source code to shader
3. compile the shader
4. create program
5. attach shader to program
6. link program
### initialize
1. get canvas from html:document.getElementByID('webgl')
2. get context of webgl canvas.getContext(webgl2)
3. compile shader to program
4. use the program on gl
5. set background (RGBA):gl.clearColor(R,G,B,A)
6. clear screen by background color:gl.clear(gl.COLOR_BUFFER_BIT)
## VBO
1. create vertex buffer: gl.createBuffer()
2. bind the buffer:gl.bindBuffer()
3. write vertice to buffer:gl.bufferData()
4. get attribute location:gl.getAttribLocation(program, attribute name)
5. assign buffer to variable in vertex shader:
- gl.vertexAttribPointer(att, size, data type, normal,stride, offset)
- att:attribute you want to pass
- size:the number of data
- normalize the data to a range or not
- stride:the offset in bytes of every data(how many bytes you want to jump when go through another vertex)
- offset:the offset of first attribute
## Transformation
**transformation the object space(move the axis of object)**
### homogeneous
with n-D, we need a vector with n+1 element(x,y,z,...,w)
w is 1
and coordiniate is $(\frac{x}{w},\frac{y}{w},\frac{z}{w},...)$

### translation
move x,y a offset $t_x$, $t_y$ to x+$t_x$,y+$t_y$
$\begin{bmatrix}
1 & 0 & t_x\\
0 & 1 & t_y\\
0 & 0 & 1
\end{bmatrix}\begin{bmatrix}
x\\
y\\
w
\end{bmatrix}$
### scaling
scale x,y to $s_x\times x$,$s_y\times y$
$\begin{bmatrix}
s_x & 0 & 0\\
0 & s_y & 0\\
0 & 0 & 1
\end{bmatrix}\begin{bmatrix}
x\\
y\\
w
\end{bmatrix}$
### rotation
rotate x,y $\theta$degrees
$\begin{bmatrix}
\cos\theta & -\sin\theta & 0\\
\sin\theta & \cos\theta & 0\\
0 & 0 & 1
\end{bmatrix}\begin{bmatrix}
x\\
y\\
w
\end{bmatrix}$
#### 3D rotation
given axis of rotation $u=(u_x,u_y,u_x)$(u is unit vector)
and rotate theta
rotate x,y $\theta$degrees
$\begin{bmatrix}
\cos\theta+u_x^2(1-\cos\theta) & u_xu_y(1-\cos\theta)-u_z\sin\theta & u_xu_z(1-\cos\theta)+u_y\sin\theta & 0\\
u_yu_z(1-\cos\theta) & \cos\theta+u_y^2(1-\cos\theta) & u_yu_z(1-\cos\theta)-u_x\sin\theta & 0\\
u_zu_x(1-\cos\theta)-u_y\sin\theta & u_zu_y(1-\cos\theta)-u_x\sin\theta & \cos\theta+u_z^2(1-\cos\theta) & 0 \\
0 & 0& 0&1
\end{bmatrix}\begin{bmatrix}
x\\
y\\
z\\
w
\end{bmatrix}$
### HierachicalTransformation
use a stack to save all object matrix make them connect together
## SpaceTransformationPipeline
$local\ space\xrightarrow{model\ matrix}world\ space\xrightarrow{view\ matrix}view\ space\xrightarrow{projection\ matrix}clip\ space\xrightarrow{view port\ transorm}screen\ space$

### view matrix
(world to view Space)
$N=eye-col$
$n=\frac{N}{|N|}$ like z axis
$U=v_{up}\times n$
$u = \frac{U}{|U|}$
$v=n\times u$
t is translation
$view\ matrix=\begin{bmatrix}
u_0 & v_0 & n_0 & 0\\
u_1 & v_1 & n_1 & 0\\
u_2 & v_2 & n_2 & 0\\
0 & 0 & 0 & 1\\
\end{bmatrix}\times\begin{bmatrix}
1 & 0 & 0 & t_0\\
0 & 1 & 0 & t_1\\
0 & 0 & 1 & t_2\\
0 & 0 & 0 & 1\\
\end{bmatrix}$
$View matrix (𝑀)=rotation\ \times\ translation$

### Projection Matrix
(View to Clip Space)
#### Orthographic

$\begin{bmatrix}
\frac{1}{width} & 0 & 0 & 0\\
0 & \frac{1}{height} & 0 & 0\\
0 & 0 & -\frac{2}{Z_{far}-Z_{near}} & -\frac{Z_{far}+Z_{near}}{Z_{far}-Z_{near}}\\
0 & 0 & 0 & 1\\
\end{bmatrix}$
#### Perspective
$\begin{bmatrix}
\tan^{-1}(\frac{FOV_x}{2}) & 0 & 0 & 0\\
0 & \tan^{-1}(\frac{FOV_y}{2}) & 0 & 0\\
0 & 0 & -\frac{Z_{far}+Z_{near}}{Z_{far}-Z_{near}} & -\frac{2Z_{far}Z_{near}}{Z_{far}-Z_{near}}\\
0 & 0 & -1 & 0\\
\end{bmatrix}$
### transformation

$S_x=X\frac{V_r-V_l}{1-(-1)}+\frac{V_r+V_l}{1-(-1)}$
$S_y=Y\frac{V_t-V_b}{1-(-1)}+\frac{V_t+V_b}{1-(-1)}$
## Illumination
K is Illumination color
I is object color
Ambient Light
- light of envirment
- $K_a* I_a$
$K_a$ is Ambient Light color, $I_a$ is object color

Diffuse Light
- $cos\theta=N*L$
- $K_d*I_d*cos\theta$

Speclar Light
- $K_s*I_s*cos^n(\phi)$
- n:surface normal
- $\phi$:angle between R,V
- $cos^n(\phi)$ n is larger, smaller value of this term
illumination
ambient+diffuse+speclar
### flat shader
- for each surface 1 color
- fast
- use at
- polygon small
- light is far
- eye is far
- mach band effect

### gouraud shader
- for each vertex 1 color
### phong shader
- Interpolate normal with normal matrix
- $n_{real} = {M^{-1}}^T \times n$

## ElementArray
use a array to record each surface's vertices index in vertices
## LoadExternalModel
### obj file
`# comment`
`mtllib file`
`o name`
`v x y x` vertex position
`vt x y z` vertex textue coordinate
`vn x y z` normal vertex vector
`usemtl mtlName`
`f v1 v2 v3...` face with n vertices
v1,v2,v3... ->vi=v/vt/vn index() can multiple properties
**no color**

write parse function to parse obj file to webgl
## texture
### mapping texture
each vertex maping a location in picture
calculate surface by interpolation texture coordinates


### mtl file
`newmtl name` create new mtl
`Ns`
shininess of the material
`Ka r g b`ambient light
`Kd r g b` diffuse light
`Ks r g b`specular light
`illum mode`illumination model
`d/Tr alpha` transparency
`map_Ka file` picture this texture use
## FrameBuffer
put what I want to show to a buffer for later use(e.g use like texture, reflection or shadow)
### Shadow
depth=Calculate the nearest obstacle to the light source on the line from each point to the light source

we take picture from light(camera location=light location) and calculate depth
if shadowCoord > depth+deMachThreshold =>have shadow->make color deeper
### CubeMap
map some pictures as a box
can use to draw refraction or skybox
->$\frac{position}{2}+0.5$ reset range to (0,1)
#### sky box
dont create cube, rander 6 quad which covers the whole camera view

#### Reflection
##### envirment
##### dynamic
use renfering of cubemap
put camera at center of reflective obj, create cubemap texture on the fly
and look up this cubemap texture to the reflective object
#### Refraction
$\frac{sin\theta_1}{sin\theta_2} = \frac{n_1}{n_2}$
where n is refraction index

#### Rendering
take picture with 6 dictionary to record the background looklike and render on frame buffer
## Fog
$fogFactor(f)=\frac{end-distance(camera, objPoint)}{end-start}$
$color=(f\times objColor)+((1-f)\times fogColor)$
## Bump mapping
control the normal vector to let people think it high poly
normal.xyz set to normal map.rgb
(if we look object at z=0)
(so most surface is [0,0,1], and mapping to rgb is [0.5,0.5,1])
### normal map with tangent space
the vector is only correct for z-plane
->rotate the normal vector
-> transform normalmap from tangent space to object space
$\begin{bmatrix}
x_o \\
y_o \\
z_o \\
\end{bmatrix}=TBN\times\begin{bmatrix}
x_n \\
y_n \\
z_n \\
\end{bmatrix}$
n is normal space
o is object space
$TBN\ matrix = [T,B,N]$
if we use $T \times B$ and when we map triangle to texture space is clock-wise point order=>it will get wrong normal vector
=>check it's clock-wise or not
## WebGL Programming
### webgl datatype
mat4 (a,b,c,d...)
is meaning (colume major)
$\begin{bmatrix}
a & e & \dots \\
b & f & \dots\\
c & g & \dots\\
d & h & \dots\\
\end{bmatrix}$
**get matrix element can't use nonconstant index**
vector*matrix :let vector be a $1\times n$ matrix
### draw
gl.drawArrays(mode, first, count)
run Vertex Shader, Fragment Shader
TRI: 012 345
STRIP:012 123
FAN:012 023
mode:

first:
- the first vertex for drawing in vertex array
count:
- number of vertices
### pass data
1. get reference of uniform variable
gl.getUniformLocation(program, 'varible name')
2. pass data
gl.uniform$n$f(location, v0,...,v$n$), $n$ is 1-4
`e.g. gl.uniform4f(program.u_pos, r,g,b,a)`