---

Definition of a Vector

A vector is an ordered collection of numbers, which are called the components or entries of the vector. A vector can be represented as:

$\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}_{n\times 1}$

where each $v_i$ is a scalar, and $n$ is the dimension of the vector. A vector in $n$-dimensional space is denoted as a column vector.

Example:

For a vector in 3-dimensional space:

$\mathbf{v} = \begin{pmatrix} 2 \\ -3 \\ 4 \end{pmatrix}$

The vector $\mathbf{v}$ has 3 components, so it is a 3-dimensional vector.

---

Size of a Vector

The size of a vector refers to the number of components or entries in the vector. A vector with $n$ components is said to have size $n$ or be a $n$-dimensional vector.

• If $\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$, then the size of $\mathbf{v}$ is $n$.

---

Operations on Vectors

1. Addition: Given two vectors $\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$ and $\mathbf{w} = \begin{pmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{pmatrix}$ of the same size, their sum $\mathbf{v} + \mathbf{w}$ is:
   $\mathbf{v} + \mathbf{w} = \begin{pmatrix} v_1 + w_1 \\ v_2 + w_2 \\ \vdots \\ v_n + w_n \end{pmatrix}$
2. Scalar Multiplication: If $\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$ is a vector and $c$ is a scalar, then the scalar multiplication $c \cdot \mathbf{v}$ is:
   $c \cdot \mathbf{v} = \begin{pmatrix} c \cdot v_1 \\ c \cdot v_2 \\ \vdots \\ c \cdot v_n \end{pmatrix}$

---

Norm of a Vector

The norm of a vector $\mathbf{v}$, denoted as $||\mathbf{v}||$, is a measure of the vector’s length or magnitude. There are different types of norms:

1. L1 Norm (Manhattan Norm): The L1 norm of a vector $\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$ is defined as: $$
   ||\mathbf{v}||_1 = |v_1| + |v_2| + \dots + |v_n|$$
   Example: $If \mathbf{v} = \begin{pmatrix} 2 \\ -3 \\ 4 \end{pmatrix}$, then: $$
   ||\mathbf{v}||_1 = |2| + |-3| + |4| = 2 + 3 + 4 = 9$$
2. L2 Norm (Euclidean Norm): The L2 norm of a vector$\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$ is defined as: $$
   ||\mathbf{v}||_2 = \sqrt{v_1^2 + v_2^2 + \dots + v_n^2}
   $$ Example: If $\mathbf{v} = \begin{pmatrix} 2 \\ -3 \\ 4 \end{pmatrix}$, then: $$
   ||\mathbf{v}||_2 = \sqrt{2^2 + (-3)^2 + 4^2} = \sqrt{4 + 9 + 16} = \sqrt{29} \approx 5.385
   $$

---

Unit Vector

A unit vector is a vector with a magnitude of 1. To convert any given vector to a unit vector, we divide the vector by its L2 norm.

The L2 norm (Euclidean norm) of a vector $\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$ is defined as:

$$
||\mathbf{v}||_2 = \sqrt{v_1^2 + v_2^2 + \dots + v_n^2}
$$

To convert a vector $\mathbf{v}$ to a unit vector $\hat{\mathbf{v}}$, we divide each component of $\mathbf{v}$ by its L2 norm:

$$
\hat{\mathbf{v}} = \frac{\mathbf{v}}{||\mathbf{v}||_2}
$$

---

Example: Converting a Vector to a Unit Vector

Consider the vector:

$\mathbf{v} = \begin{pmatrix} 3 \\ 4 \end{pmatrix}$

1. Compute the L2 norm of $\mathbf{v}$:

$$
||\mathbf{v}||_2 = \sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5
$$

2. Divide the vector by its L2 norm to obtain the unit vector:

$\hat{\mathbf{v}} = \frac{1}{5} \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} \frac{3}{5} \\ \frac{4}{5} \end{pmatrix}$

Thus, the unit vector $\hat{\mathbf{v}}$ is:

$\hat{\mathbf{v}} = \begin{pmatrix} 0.6 \\ 0.8 \end{pmatrix}$

Now, the vector $\hat{\mathbf{v}}$ has a magnitude of 1, making it a unit vector.

---

Orthogonal Vectors

Two vectors $\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$ and $\mathbf{w} = \begin{pmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{pmatrix}$ are said to be orthogonal if their dot product is zero:

$$
\mathbf{v} \cdot \mathbf{w} = v_1 w_1 + v_2 w_2 + \dots + v_n w_n = 0
$$

In other words, if $\mathbf{v}$ and $\mathbf{w}$ are orthogonal, they are at a right angle to each other.

---

Orthonormal Vectors

Two vectors $\mathbf{v}$ and $\mathbf{w}$ are said to be orthonormal if they are both orthogonal and normalized (i.e., their L2 norm is 1):

$$
\mathbf{v} \cdot \mathbf{w} = 0 \quad \text{and} \quad ||\mathbf{v}||_2 = ||\mathbf{w}||_2 = 1
$$

Orthonormal vectors are often used in orthogonal bases where each vector is both orthogonal to the others and has a unit length.

---

Orthonormal Matrix

An orthonormal matrix is a square matrix $Q$ whose columns (or rows) are orthonormal vectors. This means that:

$$
Q^T \cdot Q = I_n
$$

where $Q^T$ is the transpose of the matrix $Q$ and $I_n$ is the identity matrix of size $n \times n$. This property implies that the matrix $Q$ is invertible, and its inverse is its transpose:

$$
Q^{-1} = Q^T
$$

---

Linearly Independent Vectors

A set of vectors $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k$ are said to be linearly independent if the only solution to the equation:

$$
c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \dots + c_k \mathbf{v}_k = 0
$$

is $c_1 = c_2 = \dots = c_k = 0$. In other words, no vector in the set can be written as a linear combination of the others.

If there exists a non-trivial solution (where not all $c_i$ are zero), the vectors are linearly dependent.

---

Example of Linearly Independent Vectors (LI)

Consider the following vectors in $\mathbb{R}^2$:

$\mathbf{v}_1 = \begin{pmatrix} 1 \\ 2 \end{pmatrix}, \quad \mathbf{v}_2 = \begin{pmatrix} 3 \\ 4 \end{pmatrix}$

To check if these vectors are linearly independent, we set up the equation:

$$
c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 = 0
$$

Substituting the values of the vectors:

$c_1 \begin{pmatrix} 1 \\ 2 \end{pmatrix} + c_2 \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$

This leads to the system of linear equations:

$$
c_1 + 3c_2 = 0
$$

$$
2c_1 + 4c_2 = 0
$$

From the first equation, we have $c_1 = -3c_2$. Substituting into the second equation:

$$
2(-3c_2) + 4c_2 = 0
$$

$$
-6c_2 + 4c_2 = 0
$$

$$
-2c_2 = 0 \quad \Rightarrow \quad c_2 = 0
$$

Since $c_2 = 0$, we substitute into $c_1 = -3c_2$ to get $c_1 = 0$. Therefore, the only solution is $c_1 = c_2 = 0$, which means that the vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ are linearly independent.

---

Example of Linearly Dependent Vectors (LD)

Consider the following vectors in $\mathbb{R}^2$:

$\mathbf{v}_1 = \begin{pmatrix} 1 \\ 2 \end{pmatrix}, \quad \mathbf{v}_2 = \begin{pmatrix} 2 \\ 4 \end{pmatrix}$

These vectors are linearly dependent because $\mathbf{v}_1$ is a scalar multiple of $\mathbf{v}_2$. Specifically:

$$
\mathbf{v}_1 = \frac{1}{2} \mathbf{v}_2
$$

Thus, the vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ are linearly dependent.

---