Computing with matrices

At a reductive level, a matrix is a table of numbers that obeys certain algebraic laws. But matrices are pervasive in scientific computation, mainly because they represent linear operations on vectors. Moreover, vectors go far beyond the three-dimensional representations of physical quantities you learned about in calculus.

2.2.1Notation¶

We use capital letters in bold to refer to matrices, and lowercase bold letters for vectors. All named vectors in this book are column vectors. The bold symbol $\boldsymbol{0}$ may refer to a vector of all zeros or to a zero matrix, depending on context; we use 0 as the scalar zero only.

To refer to a specific element of a matrix, we use the uppercase name of the matrix without boldface, as in $A_{24}$ to mean the $(2,4)$ element of $\mathbf{A}$.^[1] To refer to an element of a vector, we use just one subscript, as in $x_3$. If you see a boldface character with one or more subscripts, then you know that it is a matrix or vector that belongs to a sequence or indexed collection.

We will have frequent need to refer to the individual columns of a matrix as vectors. Our convention is to use a lowercase bold version of the matrix name with a subscript to represent the column number. Thus, $\mathbf{a}_1,\mathbf{a}_2,\ldots,\mathbf{a}_n$ are the columns of the $m\times n$ matrix $\mathbf{A}$. Conversely, whenever we define a sequence of vectors $\mathbf{v}_1,\ldots,\mathbf{v}_p$, we can implicitly consider them to be columns of a matrix $\mathbf{V}$. Sometimes we might write $\mathbf{V}=\bigl[ \mathbf{v}_j \bigr]$ to emphasize the connection.

The notation $\mathbf{A}^T$ is used for the transpose of a matrix, in which the rows and columns switch places. In the case of complex matrices, the complex conjugate^[2] becomes involved with this operation.

If $\mathbf{A}$ is real, then $\mathbf{A}^*=\mathbf{A}^T$.

2.2.2Block matrix expressions¶

We will often find it useful to break a matrix into separately named pieces. For example, we might write

It’s understood that blocks that are on top of one another have the same number of columns, and blocks that are side by side have the same number of rows. Typically, if the blocks all have compatible dimensions, then they can be multiplied as though the blocks were scalars. For instance, continuing with the definitions above, we say that $\mathbf{A}$ is block-$2\times 3$ and $\mathbf{B}$ is block-$3\times 1$, so we can write

provided that the individual block products are well-defined. For transposes, we have, for example,

2.2.3Vector and matrix basics¶

Vectors and matrices are integral to scientific computing. All modern languages provide ways to work with them beyond manipulation of individual elements.

2.2.4Row and column operations¶

A critical identity in matrix multiplication is

Furthermore, the expression

reproduces three columns. An equivalent expression in Julia would be A[:,1:2:5].

We can extend the same idea to rows by using the general identity $(\mathbf{R}\mathbf{S})^T=\mathbf{S}^T\mathbf{R}^T$. Let $\mathbf{B}=\mathbf{A}^T$ have columns $\bigl[ \mathbf{b}_j \bigr]$, and note

But $\mathbf{e}_j^T$ is the $j$th row of $\mathbf{I}$, and $\mathbf{b}_j^T$ is the transpose of the $j$th column of $\mathbf{B}$, which is the $j$th row of $\mathbf{A}$ by $\mathbf{B}=\mathbf{A}^T$. Thus, multiplication on the left by row $j$ of the identity extracts the $j$th row. Extracting the single element $(i,j)$ from the matrix is, therefore, $\mathbf{e}_i^T \mathbf{A} \mathbf{e}_j$.

Being able to extract specific rows and columns of a matrix via algebra makes it straightforward to do row- and column-oriented operations, such as linear combinations.

2.2.5Exercises¶