# Matrix & Vector Algebra

Matrix notation was invented1 primarily to express linear algebra relations in compact form. Compactness enhances visualization and understanding of essentials.Note that all vectors and matrices are 1-indexed as standard

## Indexes

• $n$ - the number of distinct data points, or observations, in a sample.
• $p$ - denote the number of variables that are available for use in making predictions
• $X_{ij}$ - is an index reference which refers to the element in the ith row and $jth$ column of matrix X. where $i = 1,2,\cdots,n$ and $j = 1,2,\cdots,p$
• $v_{i}$ is an index reference which refers to the element in the ith row of the vector $v$.
• $i$ is be used to index the samples or observations (from 1 to n)
• $j$ is be used to index the variables (from 1 to p)

### Key Terms

Standard notations used across the board (unless specified elsewhere)

• Matrices are usually denoted by uppercase names i.e. $X$
• Vectors are lowercase bold i.e.$x$
• 'Scalar' means that an object is a single value, and is not a vector or matrix.
• Determinant is a useful value that can be computed from the elements of a square matrix.

## Object Types

$a \in \mathbb{R}$ indicates that the object $a$ is scalar.

$a \in \mathbb{R}^n$ indicates that the object $a$ is a vector of length $n$.

$A \in \mathbb{R}^{n \cdot p}$ indicates that the object $a$ is a vector of columns $n$ and rows $p$.

## Referencing

Matrices which have a single row are called row vectors, and those which have a single column are called column vectors

Let $X$ denote a $n×p$ matrix.

It can be indexed by its (i,j)th element as $x_{ij}$. Lower-case letters, with two subscript indices (for example, $x_{11}$, or $x_{1,1}$), represent the entries.

The entry in the i-th row and j-th column of a matrix A is sometimes referred to as the $i,j$, $(i,j)$, or $(i,j)$th entry of the matrix

An asterisk is occasionally used to refer to whole rows or columns in a matrix. For example, $x_{i,∗}$ refers to the $i$th row of X, and $a_{∗,j}$ refers to the $j$th column of A.

$$X = \begin{pmatrix} x_{11} & x_{12} & \cdots & x_{1p} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n1} & x_{n2} & \cdots & x_{np}\end {pmatrix}$$

A single row of $X$ can be referred to by $x_{1}, x_{2}, \cdots, x_{n}$ where $x_{1}$ can be written as this vector:

$$x_{1} = \begin{bmatrix} x_{i1} \\ x_{i2} \\ \vdots \\ x_{ip} \end{bmatrix}$$

A single column of $X$ can be referred to by $\boldsymbol{x_{1}}, \boldsymbol{x_{2}}, \cdots, \boldsymbol{x_{n}}$ where $\boldsymbol{x_{1}}$ can be written as this vector:

$$\boldsymbol{x_{1}} = \begin{bmatrix} x_{1i} \\ x_{2i} \\ \vdots \\ x_{ni} \end{bmatrix}$$

### Invertible Matrix

$A$ is invertible (also nonsingular or nondegenerate) if there exists an $n-by-n$ square matrix B such that.

$$AB = BA = {I}_{n}$$

Where $I_n$ denotes the $n-by-n$ identity matrix and the multiplication used is ordinary matrix multiplication. $B$ would be called $A$'s inverse. The inverse of a matrix $A$ is denoted $A^{−1}$. Multiplying by the inverse results in the identity matrix.

$$\begin{bmatrix} a & b \\ c & d \\ \end{bmatrix} +\begin{bmatrix} w & x \\ y & z \\ \end{bmatrix} =\begin{bmatrix} a+w & b+x \\ c+y & d+z \\ \end{bmatrix}$$

## Matrix-Scalar Multiplication

$$\begin{bmatrix} a & b \\ c & d \\ \end{bmatrix} \times x = \begin{bmatrix} a \times x & b \times x \\ c \times x & d \times x \\ \end{bmatrix}$$

## Matrix-Matrix Multiplication

Let $A \in \mathbb{R}^{n \cdot p}$ where $A = \begin{pmatrix} 1 & 3 & 5 \\ 2 & 4 & 6 \end{pmatrix}$

Let $B \in \mathbb{R}^{d \cdot s}$ where $B = \begin{pmatrix} 7 & 8 \\ 9 & 10 \\ 11 & 12 \end{pmatrix}$

Then matrix multiplication would produce the vector $AB$ of size $AB^{n \cdot s}$. Note matrix multiplication can only occur with two matrices where where the number of columns in A is the same as the number of rows B. Matrix mulitplication of $AB$ would work like this:

$$AB = \begin{pmatrix} 1 & 3 & 5 \\ 2 & 4 & 6 \end{pmatrix} \begin{pmatrix} 7 & 8 \\ 9 & 10 \\ 11 & 12 \end{pmatrix}$$

$$AB = \begin{pmatrix} (1 \times 7) + (3 \times 9) + (5 \times 11) = 89 \\ (2 \times 7) + (4 \times 9) + (6 \times 11) = 116 \end{pmatrix}$$ $$append$$ $$\begin{pmatrix} (1 \times 8) + (3 \times 10) + (5 \times 12) = 98 \\ (2 \times 8) + (4 \times 10) + (6 \times 12) = 128 \end{pmatrix}$$

$$AB = \begin{pmatrix} 89 & 98 \\ 116 & 128 \end{pmatrix}$$