latex.matrix_delimiters(left='[', right=']')
latex.matrix_column_alignment('c')

$\newcommand{\la}{\langle}$ $\newcommand{\ra}{\rangle}$ $\newcommand{\vu}{\mathbf{u}}$ $\newcommand{\vv}{\mathbf{v}}$ $\newcommand{\vw}{\mathbf{w}}$ $\newcommand{\vzz}{\mathbf{z}}$ $\newcommand{\nc}{\newcommand}$ $\nc{\Cc}{\mathbb{C}}$ $\nc{\Rr}{\mathbb{R}}$ $\nc{\Qq}{\mathbb{Q}}$ $\nc{\Nn}{\mathbb{N}}$ $\nc{\cB}{\mathcal{B}}$ $\nc{\cE}{\mathcal{E}}$ $\nc{\cC}{\mathcal{C}}$ $\nc{\cD}{\mathcal{D}}$ $\nc{\mi}{\mathbf{i}}$ $\nc{\span}[1]{\langle #1 \rangle}$ $\nc{\ol}[1]{\overline{#1} }$ $\nc{\norm}[1]{\left\| #1 \right\|}$ $\nc{\abs}[1]{\left| #1 \right|}$ $\nc{\vz}{\mathbf{0}}$ $\nc{\vo}{\mathbf{1}}$ $\nc{\DMO}{\DeclareMathOperator}$ $\DMO{\tr}{tr}$ $\DMO{\nullsp}{nullsp}$ $\nc{\va}{\mathbf{a}}$ $\nc{\vb}{\mathbf{b}}$ $\nc{\vx}{\mathbf{x}}$ $\nc{\ve}{\mathbf{e}}$ $\nc{\vd}{\mathbf{d}}$ $\nc{\ds}{\displaystyle}$ $\nc{\bm}[1]{\begin{bmatrix} #1 \end{bmatrix}}$ $\nc{\gm}[2]{\bm{\mid & \cdots & \mid \\ #1 & \cdots & #2 \\ \mid & \cdots & \mid}}$ $\nc{\MN}{M_{m \times n}(K)}$ $\nc{\NM}{M_{n \times m}(K)}$ $\nc{\NP}{M_{n \times p}(K)}$ $\nc{\MP}{M_{m \times p}(K)}$ $\nc{\PN}{M_{p \times n}(K)}$ $\nc{\im}{\mathrm{Im\ }}$ $\nc{\ev}{\mathrm{ev}}$ $\nc{\Hom}{\mathrm{Hom}}$ $\nc{\com}[1]{[\phantom{a}]^{#1}}$ $\nc{\rBD}[1]{ [#1]_{\cB}^{\cD}}$ $\DMO{\id}{id}$ $\DMO{\rk}{rk}$ $\DMO{\nullity}{nullity}$ $\DMO{\End}{End}$ $\DMO{\proj}{proj}$ $\nc{\GL}{\mathrm{GL}}$

Matrix Operations I

Addition and Scalar Multiplication

The addition and multiplication of $K$, generalize to addition and scalar multiplication on $\MN$, respectively.

Given $A, B \in \MN$ and $c \in K$, $A+B$ is the matrix with $(A+B)_{ij} = (A)_{ij} + (B)_{ij}$ and $cA$ is the matrix with $(cA)_{ij} = c(A)_{ij}$.

Checkpoint Let $A = \left[\begin{array}{rrrr} 1 & 0 & 3\\ -1 & 1 & -1 \\ -1 & 1 & 2 \end{array}\right]$ and

$B = \left[\begin{array}{rrrr} -3 & -1 & 0 \\ 1 & 0 & 0 \\ -1 & 1 & -2 \end{array}\right]$

Find $A-2B$.

%display latex
rows_of_A = [ (1,0,3), (-1,1,-1), (-1, 1, 2)]
rows_of_B = [(-3, -1, 0), (1, 0, 0), (-1, 1, -2)]
A = matrix(rows_of_A); B = matrix(rows_of_B); (A,B)

$$\newcommand{\Bold}[1]{\mathbf{#1}}\left(\left[\begin{array}{ccc} 1 & 0 & 3 \\ -1 & 1 & -1 \\ -1 & 1 & 2 \end{array}\right], \left[\begin{array}{ccc} -3 & -1 & 0 \\ 1 & 0 & 0 \\ -1 & 1 & -2 \end{array}\right]\right)$$

A-2*B #finding A - 2B

$$\newcommand{\Bold}[1]{\mathbf{#1}}\left[\begin{array}{ccc} 7 & 2 & 3 \\ -3 & 1 & -1 \\ 1 & -1 & 6 \end{array}\right]$$

The $m \times n$ zero matrix over $K$ is the $m \times n$ matrix whose entries are the $0$ of $K$. If $K$ is understood and we do not need to specify the dimensions, then we simply call it the zero matrix and denote it by $\vz$.

Clearly, $A + \vz = A = \vz +A$ for all $A \in \MN$. In other words, $\vz \in \MN$ is the identity element of the addition of $\MN$.

Also, $1 A = A$ for all $A \in \MN$ where $1 \in K$ is the multiplicative identity of $K$.

Matrix addition and scalar multiplication have many properties (induced by the operations on $K$), e.g.

• $A+B = B+A$ (commutative)
• $c(A+B) = cA +cB$ (scalar multiplication distribute over addition)
• Every matrix has a unique additive inverse denoted by $-A$ and that $-A = (-1)A$.

We summaries the properties of matrix addition and scalar multiplication by saying that $\MN$ forms a vector space over $K$ (or a $K$ vector space, in short).

Matrix Multiplication

Matrix multiplication is trickier: for $A \in \MP$ and $B \in \PN$, the product $AB$ is defined to be the $m \times n$ matrix whose $(i,j)$-entry is

$$ (AB)_{ij} = \sum_{k=1}^p a_{ik}b_{kj}. $$

Compare this with the dot product (from vector calculus): $(AB)^{i}_j =A^i_* \cdot B^*_j$, i.e. the dot product of $i$-th row of $A$ with the $j$-th column of $B$.

At the first sight, the definition of matrix multiplication seems unmotivated. However, we will see that it becomes completely natural when we study representation of linear maps (corresponding to composition of linear maps).

One checks readily that matrix multiplication is associative, i.e. $A(BC) = (AB)C$ whenever the products are defined. Consequently, it is unambiguous to write a product of matrices without any parentheses.

Example Let $A = \left[\begin{array}{cccc} 2 & 1 & -1 & 2 \\ 1 & -1 & -1 & -1 \end{array}\right]$, $B = \left[\begin{array}{cc} 3 & -1 \\ -2 & -2 \\ 0 & -1 \\ 3 & 1 \end{array}\right]$ and $C = \left[\begin{array}{cc}1 & 1 \\ 0 & -1\end{array}\right]$

Then $AB = \left[\begin{array}{cc} 10 & -1 \\ 2 & 1 \end{array}\right]$ and $BA = \left[\begin{array}{cccc} 5 & 4 & -2 & 7 \\ -6 & 0 & 4 & -2 \\ -1 & 1 & 1 & 1 \\ 7 & 2 & -4 & 5 \end{array}\right]$.

Checkpoint Check that $(AB)C = A(BC)$. Check also that $(AB)^T = B^TA^T$. Give matrices $A$ and $B$ so that $AB$ is defined but not $BA$.

Einstein summation convention: write the row index as superscript and column index as subscript. Whenever an index appear both as superscript and subscript is autonmatically sum over the range of that index (with the summation sign suppressed). For example, the above sum can be written succinctly written as

$$ (AB)^i_j = (A)^i_k (B)^k_j $$

A quick computation should convince the reader the usefulness of this convention.

$((AB)^T)^i_j = (AB)^j_i = (A)^j_k(B)^k_i = (A^T)^k_j(B^T)^i_k = (B^T)^i_k(A^T)^k_j = (B^TA^T)^i_j$.

Therefore, $(AB)^T = B^TA^T$.

Instead of multiplying matrices entry-by-entry, it is often useful to multiply matrices column-by-column.

First note that $A\vb$, where $\vb$ is the column vector $\bm{b_1 \\ \vdots \\ b_p}$, is

$$ b_1 \va_1 + \cdots + b_p \va_p $$

And $AB = \bm{ \vert & \vert & \vert \\ A\vb_1 & \cdots & A\vb_n \\ \vert & \vert & \vert }$.

Also by taking transpose, that means one can multiply matrix row-by-row as well.

$$ AB = \bm{ -- & \va^1B & -- \\ & \vdots & \\ -- & \va^mB & --} $$

Checkpoint Compute the multiplications in the previous example column-by-column and row-by-row.