1. What is Vector?

Created: August 3, 2021 10:56 AM
Reviewed: No

Hadamard product (= elementwise product = 성분곱)

$$\bm x = [x_1, x_2, ... , x_d]$$

$$\bm y = [y_1, y_2, ... y_d]$$

$$\bm x \odot \bm y = [ x_1y_1, x_2y_2, ... x_dy_d ] $$

In Numpy, elementwise product equals *****

norm

norm is equal to the distance from the origin (0, 0).

1) $L_1-norm$

$$L_1-norm = ||\bm x||_1 = \sum_{\substack{i=1}}^{d}|\bm x_i|$$

= sum(변화량의 절댓값)

ex. Robust Training, Lasso Regression

Regularization: This speeds up the dimensionality reduction and removes unimportant parts. Unlike $L_2$-norm, the influence of an unnecessary variable can be made 0.

$${ \bm x : ||\bm x ||_1 = 1 }$$

2) $L_2-norm$

$$L_2-norm = ||\bm x||_2 = \sqrt {\sum_{\substack{i=1}}^{d} |\bm x_i|^2 }$$

= Euclidean distance ( 유클리드 거리 )

In Numpy, It is equal to numpy.linalg.norm

ex. Laplace approximation, Ridge Regression

Regularization: $L_2$-norm makes the influence of insignificant variables close to zero.

$${ \bm x : ||\bm x ||_2 = 1 }$$

The distance between vectors

The distance between vectors = The distance between two points of vectors

벡터 사이의 거리 = 두 점 사이의 거리

$$d = || \bm y - \bm x || = || \bm x - \bm y ||$$

The angle between two vectors

Calculating the angle is only possible with $L_2$-norm.
The dot product is the value adjusted to fit the length of the orthographic projection to $||\bm y||$.
The dot product can be used to measure the similarity of two vectors.

각도를 계산하는 것은 $L_2$-norm에서만 가능하다.
내적은 정사영의 길이를 $||\bm y||$에 맞게 조정한 값이다.
내적은 두 벡터의 similarity를 측정할 때 사용 가능하다.

$$cos \theta = {|| \bm x ||_2^2 + || \bm y ||_2^2 - || \bm x - \bm y ||_2^2 \over 2|| \bm x ||_2|| \bm y ||_2} = { 2<\bm x_1, \bm y_1 > \over 2|| \bm x ||_2|| \bm y ||_2} = { <\bm x_1, \bm y_1 > \over || \bm x ||_2|| \bm y ||_2}$$

Dot product = inner product = 내적

$$ <\bm x_1, \bm y_1>=\sum_{\substack{i=1}}^{d}\bm x_i\bm y_i$$

Numpy: np.inner