###### tags: `Linear Algebra` `LA01` # L06 Angle --- ## From Last Week - Introduction of Maching Learning - Implemenet Knn Algorithm in fully details - Evaluate Model performance with Accuracy --- ## This week - More on Inner Product - Definition of Angle in a Mathematic way - Cosine Similarity and its application --- ### 1. Inner Product Recall that **Inner Product** is defined as: $$a^Tb = a_1b_1 + a_2b_2 + \dots + a_nb_n$$ The definition above is pretty "algebra". Is there a visual intuision of Inner Product ? ---- ### 1.1 Inner Product as multiplication of Magnitute  ##### The sum of products of two Magnitutes on "ordinary coordinates" ---- ### 1.2 What if we use one of vector as coordinate?  The product of magnitutes with b, and a's projection onto b. ---- ### 1.3 Cauchy-Schwarz inequality $$|a^Tb| \le ||a|| \cdot ||b||$$ so, if a and b are nonzero vectors, $$\frac{|a^Tb|}{||a|| \cdot ||b||} \le 1$$ (With visual explaination) --- ### 2. Definition of Angle between two vectors Now we can formally define "Angle" of two vectors: $$\theta = arccos(\frac{a^Tb}{||a||\cdot||b||})$$ whereas *arccos* denote the inverse cosine, normalized to the interval [0, $\pi$] ---- ### 2.1 Acute and obtuse angles - If the angle is $\frac{\pi}{2} = 90^o$, i.e., $a^Tb=0$, the vectors are said to be orthogonal. We write $a\perp b$ if a and b are orthogonal. - If the angle is zero, which means $a^Tb = ||a||\cdot ||b||$, the vectors are aligned. - If the angle is $\pi = 180^o$ , which means $a^Tb = -||a||\cdot ||b||$, the vectors are anti- aligned. ---- ### 2.2 Acute and obtuse angles - If the angle $< \frac{\pi}{2} = 90^o$, the vectors are said to make an acute angle. This is the same as $a^Tb>0$ - If the angle $> \frac{\pi}{2} = 90^o$, the vectors are said to make an obtuse angle. This is the same as $a^Tb<0$ ---- ### 2.2 Acute and obtuse angles  --- ### 3. Cosine Similarity Now we can use the angle between two vectors as a second metric to evaluate how "different" these two vectors are. which is simply: $$Cosine\ Similarity := \frac{a^Tb}{||a||\cdot||b||}$$ for two vectors a and b ---- ### 3.1 Cosine Similarity Cosine Similarity is a metric with range [-1, 1] - The closer the value to 1, means more similar; - 0 means not similar at all; - where as -1 means two vectors are opposite We usually use this metric in a range [0, 1] --- ### Applications - Document similarity; - Time-series related task; Let's do some Python things!!