Module 11 Range & Nullspace of a Linear Transformation
In this module you will learn
The definition of the range and null space of a linear transformation.
How to precisely notate the matrix for a linear transformation.
The fundamental subspaces corresponding to a matrix (row space, column space, null space) and how they relate to the range and null space of a linear transformation.
How to find a basis for the fundamental subspaces of a matrix.
The definition of rank and the rank-nullity theorem.
Associated with every linear transformation are two specially named subspaces: the range and the null space.
Section 11.1 Range
Definition 11.1.1. Range.
The range (or image) of a linear transformation \(T:V\to W\) is the set of vectors that \(T\) can output. That is,
\begin{equation*}
\Range(T)=\Set{\vec y\in W \given \vec y=T\vec x\text{ for some }\vec x\in V}.
\end{equation*}
The range of a linear transformation has the exact same definition as the range of a function—it’s the set of all outputs. In other words, the range of a linear transformation is the image of the entire domain with respect to that linear transformation. However, unlike the range of an arbitrary function, the range of a linear transformation is always a subspace.
Theorem 11.1.2.
Let \(T:\R^{n}\to\R^{m}\) be a linear transformation. Then \(\Range(T)\subseteq \R^{m}\) is a subspace.
Proof.
Since \(\Range(T)=T(\R^{n})\) and \(\R^{n}\) is non-empty, we know that \(\Range(T)\) is non-empty. Therefore, to show that \(\Range(T)\) is a subspace, what remains to be shown is (i) that it’s closed under vector addition, and (ii) that it is closed under scalar multiplication.
-
Let \(\vec x,\vec y\in\Range(T)\text{.}\) By definition, there exist \(\vec u,\vec v\in\R^{n}\) such that \(\vec x=T(\vec u)\) and \(\vec y=T(\vec v)\text{.}\) Since \(T\) is linear,
\begin{equation*}
\vec x+\vec y=T(\vec u)+T(\vec v)=T(\vec u+\vec v),
\end{equation*}
and so \(\vec x+\vec y\in\Range(T)\text{.}\)
-
Let \(\vec x\in\Range(T)\) and let \(\alpha\) be a scalar. By definition, there exists \(\vec u\in\R^{n}\) such that \(\vec x=T(\vec u)\text{,}\) and so by the linearity of \(T\text{,}\)
\begin{equation*}
\alpha\vec x=\alpha T(\vec u)=T(\alpha\vec u).
\end{equation*}
Therefore \(\alpha\vec x\in\Range(T)\text{.}\)
When analyzing subspaces, we are often interested in how big they are. That information is captured by a number—the dimension of the subspace. For transformations, we also have a notion of how “big” they are, which is captured in a number called the rank.
Definition 11.1.3. Rank of a Linear Transformation.
For a linear transformation \(T:\R^{n}\to \R^{m}\text{,}\) the rank of \(T\text{,}\) denoted \(\Rank(T)\), is the dimension of the range of \(T\text{.}\)
The rank of a linear transformation can be used to measure its complexity or compressibility. A rank \(0\) transformation must send all vectors to \(\vec 0\text{.}\) A rank \(1\) transformation must send all vectors to a line, etc.. So, by knowing just a single number—the rank—you can judge how complicated the set of outputs of a linear transformation will be.
Example 11.1.4.
Let \(\mathcal{P}\) be the plane given by \(x+y+z=0\text{,}\) and let \(T:\R^{3}\to\R^{3}\) be projection onto \(\mathcal{P}\text{.}\) Find \(\Range(T)\) and \(\Rank(T)\text{.}\)
Solution.
First we will find \(\Range(T)\text{.}\) Since \(T\) is a projection onto \(\mathcal{P}\text{,}\) we know \(\Range(T)\subseteq \mathcal{P}\text{.}\) Because \(T(\vec p)=\vec p\) for all \(\vec p\in \mathcal{P}\text{,}\) we know \(\mathcal{P}\subseteq \Range(T)\text{,}\) and so
\begin{equation*}
\Range(T)=\mathcal{P}.
\end{equation*}
Since \(\mathcal{P}\) is a plane, we know \(\Dim(\mathcal{P})=2=\Dim(\Range(T))=\Rank(T)\text{.}\)
Section 11.2 Null Space
The second special subspace is called the null space.
Definition 11.2.1. Null Space.
The null space (or kernel) of a linear transformation \(T:V\to W\) is the set of vectors that get mapped to the zero vector under \(T\text{.}\) That is,
\begin{equation*}
\Null(T)=\Set{\vec x\in V \given T\vec x=\vec 0}.
\end{equation*}
We’ve seen null spaces before. In the context of matrices when we asked questions like, “Are these column vectors linearly independent?” Now that we understand linear transformation and subspaces, we can consider this question anew.
Just like the range of a linear transformation, the null space of a linear transformation is always a subspace.
Theorem 11.2.2.
Let \(T:\R^{n}\to\R^{m}\) be a linear transformation. Then \(\Null(T)\subseteq \R^{n}\) is a subspace.
Proof.
Since \(T\) is linear, \(T(\vec 0)=\vec 0\) and so \(\vec 0\in\Null(T)\) which shows that \(\Null(T)\) is non-empty. Therefore, to show that \(\Null(T)\) is a subspace, we only need to show (i) that it’s closed under vector addition, and (ii) that it is closed under scalar multiplication.
-
Let \(\vec x,\vec y\in\Null(T)\text{.}\) By definition, \(T(\vec x)=T(\vec y)=\vec 0\text{.}\) By linearity we see
\begin{equation*}
T(\vec x+\vec y)=T(\vec x)+T(\vec y)=\vec 0+\vec 0=\vec 0,
\end{equation*}
and so \(\vec x+\vec y\in\Null(T)\text{.}\)
-
Let \(\vec x\in\Null(T)\) and let \(\alpha\) be a scalar. By definition, \(T(\vec x)=\vec 0\text{,}\) and so by the linearity of \(T\text{,}\)
\begin{equation*}
T(\alpha\vec x)=\alpha T(\vec x)=\alpha\vec 0=\vec 0.
\end{equation*}
Therefore \(\alpha\vec x\in\Null(T)\text{.}\)
Akin to the rank–range connection, there is a special number called the nullity which specifies the dimension of the null space.
Definition 11.2.3. Nullity.
For a linear transformation \(T:\R^{n}\to \R^{m}\text{,}\) the nullity of \(T\text{,}\) denoted \(\Nullity(T)\text{,}\) is the dimension of the null space of \(T\text{.}\)
Example 11.2.4.
Let \(\mathcal{P}\) be the plane given by \(x+y+z=0\text{,}\) and let \(T:\R^{3}\to\R^{3}\) be projection onto \(\mathcal{P}\text{.}\) Find \(\Null(T)\) and \(\Nullity(T)\text{.}\)
Solution.
First we will find \(\Null(T)\text{.}\) Since \(T\) is a projection onto \(\mathcal{P}\) (and because \(\mathcal{P}\) passes through \(\vec 0\)), we know every normal vector for \(\mathcal{P}\) will get sent to \(\vec 0\) when \(T\) is applied. And, besides \(\vec 0\) itself, these are the only vectors that get sent to \(\vec 0\text{.}\) Therefore
\begin{equation*}
\Null(T)=\Set{\text{normal vectors}}\cup\Set{\vec 0}=\Span\Set{\mat{1\\1\\1}}.
\end{equation*}
Since \(\Null(T)\) is a line, we know \(\Nullity(T)=1.\)
Section 11.3 Fundamental Subspaces of a Matrix
Every linear transformation has a range and a null space. Analogously, every matrix is associated with three fundamental subspaces.
Definition 11.3.1. Fundamental Subspaces.
Associated with any matrix \(M\) are three fundamental subspaces: the row space of \(M\text{,}\) denoted \(\Row(M)\), is the span of the rows of \(M\text{;}\) the column space of \(M\text{,}\) denoted \(\Col(M)\), is the span of the columns of \(M\text{;}\) and the null space of \(M\text{,}\) denoted \(\Null(M)\), is the set of solutions to \(M\vec x=\vec 0\text{.}\)
Computationally, it’s much easier to find the row space/column space/null space of a matrix than it is to find the range/null space of a linear transformation because we can turn matrix questions into systems of linear equations.
Example 11.3.2.
Find the null space of \(M=\mat{1&2&5\\2&-2&-2}\text{.}\)
Solution.
To find the null space of \(M\text{,}\) we need to solve the homogeneous matrix equation \(M\vec x=\vec 0\text{.}\) Row reducing, we see
\begin{equation*}
\Rref(M)=\mat{1&0&1\\0&1&2},
\end{equation*}
and so the \(z\) column is a free variable column. Therefore, the complete solution can be expressed in vector form as
\begin{equation*}
\mat{x\\y\\z}=t\mat{-1\\-2\\1},
\end{equation*}
and so
\begin{equation*}
\Null(M)=\Span\Set{\mat{-1\\-2\\1}}.
\end{equation*}
The column space and row space are just as easy to compute, since it just involves picking a basis from the existing row or column vectors.
Example 11.3.3.
Let \(M=\mat{1&2&5\\2&-2&-2}\text{.}\) Find a basis for the row space and the column space of \(M\text{.}\)
Solution.
First the column space. We need to pick a basis for \(\Span\Set{\mat{1\\2},\mat{2\\-2},\mat{5\\-2}}\text{,}\) which is the same thing as picking a maximal linearly independent subset of \(\Set{\mat{1\\2},\mat{2\\-2},\mat{5\\-2}}\text{.}\)
Putting these vectors as columns in a matrix and row reducing, we see
\begin{equation*}
\Rref\left(\mat{1&2&5\\2&-2&-2}\right) = \mat{1&0&1\\0&1&2}.
\end{equation*}
The first and second columns are the only pivot columns and so the first and second original vectors form a maximal linearly independent subset. Thus,
\begin{equation*}
\Col(M) = \Span\Set{\mat{1\\2},\mat{2\\-2}}= \R^{2}\qquad\text{and a basis is}\qquad \Set{\mat{1\\2},\mat{2\\-2}}.
\end{equation*}
To find the row space, we need to pick a basis for \(\Span\Set{\mat{1\\2\\5},\mat{2\\-2\\-2}}\text{.}\) Repeating a similar procedure, we see
\begin{equation*}
\Rref\left(\mat{1&2\\2&-2\\5&-2}\right) = \mat{1&0\\0&1\\0&0},
\end{equation*}
and so \(\Set{\mat{1\\2\\5},\mat{2\\-2\\-2}}\) is linearly independent. Therefore
\begin{equation*}
\Row(M)=\Span\Set{\mat{1\\2\\5},\mat{2\\-2\\-2}}\qquad\text{and a basis is}\qquad \Set{\mat{1\\2\\5},\mat{2\\-2\\-2}}.
\end{equation*}
When talking about fundamental subspaces, we often switch between talking about column vectors and row vectors belonging to a matrix. The operation of swapping rows for columns is called the transpose.
Definition 11.3.4. Transpose.
Let \(M\) be an \(n\times m\) matrix defined by
\begin{equation*}
M=\matc{ a_{11}&a_{12}&a_{13}&\cdots & a_{1m}\\ a_{21}&a_{22}&a_{23}&\cdots&a_{2m}\\ \vdots &\vdots&\vdots &\ddots&\vdots\\ a_{n1}&a_{n2}&a_{n3}&\cdots&a_{nm}}.
\end{equation*}
The transpose of \(M\text{,}\) notated \(M^{T}\), is the \(m\times n\) matrix produced by swapping the rows and columns of \(M\text{.}\) That is
\begin{equation*}
M^{T}=\matc{ a_{11}&a_{21}&\cdots&a_{n1}\\ a_{12}&a_{22}&\cdots&a_{n2}\\ a_{13}&a_{23}&\cdots&a_{n3}\\ \vdots&\vdots&\ddots&\vdots\\ a_{1m}&a_{2m}&\cdots&a_{nm}}.
\end{equation*}
Using the transpose, we can make statements like
\begin{equation*}
\Col(M)=\Row(M^{T})\qquad\text{and}\qquad \Row(M)=\Col(M^{T}).
\end{equation*}
In addition, it helps us state the following theorem.
Theorem 11.3.5. Row-Col Dimension.
For a matrix \(A\text{,}\) the dimension of the row space equals the dimension of the column space.
Proof.
For this proof, we will rely on what we know about the row reduction algorithm and what the reduced row echelon form of a matrix tells us.
Claim 1: \(\Row(\Rref(A))\subseteq \Row(A)\text{.}\) To see this, observe that to get \(\Rref(A)\text{,}\) we take linear combinations of the rows of \(A\text{.}\) Therefore, it must be that the span of the rows of \(\Rref(A)\) is contained in the span of the rows of \(A\text{.}\)
Claim 2: \(\Row(\Rref(A))=\Row(A)\text{.}\) To see this, observe that every elementary row operation is reversible. Therefore every row in \(A\) can be obtained as a linear combination of rows in \(\Rref(A)\) (by just reversing the steps). Thus the row vectors of \(\Rref(A)\) and the row vectors of \(A\) must have the same span.
Claim 3: The non-zero rows of \(\Rref(A)\) form a basis for \(\Row(A)\text{.}\) We already know that the non-zero rows of \(\Rref(A)\) span \(\Row(A)\text{,}\) so we only need to argue that they are linearly independent. However, this follows immediately from the fact that \(\Rref(A)\) is in reduced row echelon form. Above and below every pivot in \(\Rref(A)\) are zeros. Therefore, a row in \(\Rref(A)\) with a pivot cannot be written as a linear combination of any other row. Since every non-zero row has a pivot, this proves the claim.
Now, note the following two facts.
The columns of \(A\) corresponding to pivot columns of \(\Rref(A)\) form a basis for \(\Col(A)\text{.}\)
The non-zero rows of \(\Rref(A)\) form a basis for \(\Row(A)\text{.}\)
To complete the proof, note that every pivot of \(\Rref(A)\) lies in exactly one row and one column. Therefore, the number of basis vectors in \(\Row(A)\) is the same as the number of basis vectors in \(\Col(A)\text{.}\)
Section 11.4 Equations, Null Spaces, and Geometry
Let \(M=\mat{1&2&5\\2&-2&-2}\text{.}\) Using the typical row-reduction steps, we know that the complete solution to \(M\vec x=\vec 0\) (i.e., the null space of \(M\)) can be expressed in vector form as
\begin{equation*}
\vec x=t\mat{-1\\-2\\1}.
\end{equation*}
Similarly, the complete solution to \(M\vec x=\vec b\) where \(\vec b=\mat{1\\2}\) can be expressed in vector form as
\begin{equation*}
\vec x=t\mat{-1\\-2\\1}+\mat{1\\0\\0}.
\end{equation*}
The set of solutions to \(M\vec x=\vec 0\) and \(M\vec x=\vec b\) look very similar. In fact,
\begin{equation*}
\Set{\text{solutions to $M\vec x=\vec b$}}= \Set{\text{solutions to $M\vec x=\vec 0$}}+\Set{\vec p}\qquad\text{where}\qquad \vec p=\mat{1\\0\\0}.
\end{equation*}
Or, phrased another way, the solution set to \(M\vec x=\vec b\) is
\begin{equation*}
\Null(M)+\Set{\vec p}.
\end{equation*}
In the context of what we already know about lines and translated spans, this makes perfect sense. We know that the solution set to \(M\vec x=\vec b\) is a line (which doesn’t pass through the origin) and may therefore be written as a translated span \(\Span\Set{\vec d}+\Set{\vec p}\text{.}\) Here \(\vec d\) is a direction vector for the line and \(\vec p\) is a point on the line.
Because \(\vec p\in\Span\Set{\vec d}+\Set{\vec p}\text{,}\) we call \(\vec p\) a particular solution to \(M\vec x=\vec b\text{.}\) Using a similar argument, we can show that for any matrix \(A\text{,}\) and any vector \(\vec b\text{,}\) the set of all solutions to \(A\vec x=\vec b\) (provided there are any) can be expressed as
\begin{equation*}
V+\Set{\vec p}
\end{equation*}
where \(V\) is a subspace and \(\vec p\) is a particular solution. In fact, we can do better. We can say \(V=\Null(A)\text{.}\)
Theorem 11.4.1.
Let \(A\) be a matrix, \(\vec b\) be a vector, and let \(\vec p\) be a particular solution to \(A\vec x=\vec b\text{.}\) Then, the set of all solutions to \(A\vec x=\vec b\) is
\begin{equation*}
\Null(A)+\Set{\vec p}.
\end{equation*}
Proof.
Let \(S=\Set{\text{all solutions to $A\vec x=\vec b$}}\) and assume \(\vec p\in S\text{.}\) We will show \(S=\Null(A)+\Set{\vec p}\text{.}\)
First we will show \(\Null(A)+\Set{\vec p}\subseteq S\text{.}\) Let \(\vec v\in \Null(A)+\Set{\vec p}\text{.}\) By definition, \(\vec v=\vec n+\vec p\) for some \(\vec n\in \Null(A)\text{.}\) Now, by linearity of matrix multiplication and the definition of the null space,
\begin{equation*}
A\vec v=A(\vec n+\vec p)=A\vec n+A\vec p=\vec 0+\vec b=\vec b,
\end{equation*}
and so \(\vec v\in S\text{.}\)
Next we will show \(S\subseteq \Null(A)+\Set{\vec p}\text{.}\) First observe that for any \(\vec u,\vec v\in S\) we have
\begin{equation*}
A(\vec u-\vec v)=A\vec u-A\vec v=\vec b-\vec b=\vec 0,
\end{equation*}
and so \(\vec u-\vec v\in \Null(A)\text{.}\)
Fix \(\vec w\in S\text{.}\) By our previous observation, \(\vec w-\vec p\in \Null(A)\text{.}\) Therefore
\begin{equation*}
\vec w=(\vec w-\vec p)+\vec p\in \Null(A)+\Set{\vec p},
\end{equation*}
which completes the proof.
Null spaces are also closely connected with row spaces. Let \(\mathcal{P}\subseteq \R^{3}\) be the plane with equation \(x+2y+2z=0\text{.}\) We can rewrite this equation as a matrix equation and as the equation of a plane in normal form.
\begin{equation*}
\underbrace{\mat{1&2&2}\mat{x\\y\\z}=\vec 0}_{\text{a matrix equation}}\qquad\qquad \underbrace{\mat{1\\2\\2}\cdot \mat{x\\y\\z}=0}_{\text{normal form}}
\end{equation*}
Now we see that \(\mathcal{P}=\Null\left(\mat{1&2&2}\right)\) and that every non-zero vector in \(\Row\left(\mat{1&2&2}\right)\) is a normal vector for \(\mathcal{P}\text{.}\) In other words, \(\Null\left(\mat{1&2&2}\right)\) is orthogonal to \(\Row\left(\mat{1&2&2}\right)\text{.}\)
This is no coincidence. Let \(M\) be a matrix and let \(\vec r_{1},\ldots,\vec r_{n}\) be the rows of \(M\text{.}\) By definition,
\begin{equation*}
M\vec x=\matc{\vec r_1\cdot \vec x\\\vdots\\\vec r_n\cdot \vec x},
\end{equation*}
and so solutions to \(M\vec x=\vec 0\) are precisely the vectors which are orthogonal to every row of \(M\text{.}\) In other words, \(\Null(M)\) consists of all vectors orthogonal to the rows of \(M\text{.}\) Conversely, \(\Row(M)\) consists of all vectors orthogonal to everything in \(\Null(M)\text{.}\) We can use this fact to approach questions in a new way.
Example 11.4.3.
Let \(\vec a=\mat{1\\2\\5}\) and \(\vec b=\mat{2\\-2\\-2}\text{.}\) Find the set of all vectors orthogonal to both \(\vec a\) and \(\vec b\text{.}\)
Solution.
Let \(M=\mat{1&2&5\\2&-2&-2}\) be the matrix whose rows are \(\vec a\) and \(\vec b\text{.}\) Since \(\Null(M)\) consists of all vectors orthogonal to \(\Row(M)\text{,}\) the set we are looking for is \(\Null(M)\text{.}\) Computing via row reduction, we find
\begin{equation*}
\Null(M)=\Span\Set{\mat{-1\\-2\\1}}.
\end{equation*}
Section 11.5 Transformations and Matrices
Matrices are connected to systems of linear equations via matrix equations (like \(A\vec x=\vec b\)) and to linear transformations through matrix transformations (like \(\mathcal{T}(\vec x)=M\vec x\)). This means that we can think about systems of equations in terms of linear transformations and we can gain insight about linear transformations by looking at systems of equations!
In preparation for this, let’s reconsider matrix transformations and be pedantic about our notation.
Let \(\mathcal{T}:\R^{n}\to\R^{n}\) be a linear transformation and let \(M\) be its corresponding matrix. \(\mathcal{T}\) is a function that inputs and outputs vectors. \(M\) is a box of numbers, which has no meaning by itself, but we know how to multiply \(M\) by lists of numbers (or other boxes of numbers). Therefore, strictly speaking, the expression “\(M\vec x\)” doesn’t make sense. The quantity “\(\vec x\)” is a vector, but we only know how to multiply \(M\) by lists of numbers.
Ah! But we know how to turn \(\vec x\) into a list of numbers. Just pick a basis! The expression
\begin{equation*}
M[\vec x]_{\mathcal{E}}
\end{equation*}
makes perfect sense since \([\vec x]_{\mathcal{E}}\) is a list of numbers. Continuing to be pedantic, we know \(\mathcal{T}(\vec x)\neq M[\vec x]_{\mathcal{E}}\) since the left side is a vector and the right side is a list of numbers. We can fix this by either turning the right side into a vector or the left side into a list of numbers. Doing this, we see the precise relationship between a linear transformation \(\mathcal{T}:\R^{n}\to\R^{n}\) and its matrix \(M\) is
\begin{equation*}
[\mathcal{T}(\vec x)]_{\mathcal{E}}=M[\vec x]_{\mathcal{E}}.
\end{equation*}
If we have a matrix \(M\text{,}\) by picking a basis (usually the standard basis), we can define a linear transformation by first taking the input vector and rewriting it in the basis, next multiplying by the matrix, and finally taking the list of numbers and using them as coefficients for a linear combination involving the basis vectors. This is what we actually mean when we say that a matrix induces a linear transformation.
Definition 11.5.1. Induced Transformation.
Let \(M\) be an \(n\times m\) matrix. We say \(M\) induces a linear transformation \(\mathcal{T}_{M}:\R^{m}\to\R^{n}\) defined by
where \(\mathcal{E}\) is the standard basis for \(\R^{m}\) and \(\mathcal{E}'\) is the standard basis for \(\R^{n}\text{.}\)
Previously, we would write “\(\mathcal{T}(\vec x)=M\vec x\)” which hides the fact that when we relate a matrix and a linear transformation, there is a basis hidden in the background. And, like before, when we’re only considering a single basis, we can be sloppy with our notation and write things like “\(M\vec x\)”, but when there are multiple bases or when we’re trying to be extra precise, we must make sure our boxes/lists of numbers and our transformations/vectors stay separate.
Example 11.5.2.
Let \(\mathcal{T}\) be the transformation induced by the matrix \(M=\mat{1&2&5\\2&-2&-2}\text{,}\) and let \(\vec v=3\xhat-3\zhat\text{.}\) Compute \(\mathcal{T}(\vec v)\text{.}\)
Solution.
Since \(\mathcal{T}\) is induced by \(M=\mat{1&2&5\\2&-2&-2}\text{,}\) by definition,
\begin{equation*}
[\mathcal{T}_{M}\vec v]_{\mathcal{E}'}= M[\vec v]_{\mathcal{E}}=\mat{1&2&5\\2&-2&-2}[\vec v]_{\mathcal{E}}.
\end{equation*}
Further, since \(\vec v=3\xhat-3\zhat\text{,}\) by definition we have \([\vec v]_{\mathcal{E}}=\mat{3\\0\\-3}\text{.}\) Therefore,
\begin{equation*}
[\mathcal{T}_{M}\vec v]_{\mathcal{E}'}=\mat{1&2&5\\2&-2&-2}\mat{3\\0\\-3}=\mat{-12\\12}.
\end{equation*}
In other words, \(\mathcal{T}(\vec v) = \mat{-12\\12}_{\mathcal{E}'}=-12\xhat+12\yhat\text{.}\)
Using induced transformations, we can extend linear-transformation definitions to matrix definitions. In particular, we can define the rank and nullity of a matrix.
Definition 11.5.3. Rank of a Matrix.
Let \(M\) be a matrix. The rank of \(M\text{,}\) denoted \(\Rank(M)\), is the rank of the linear transformation induced by \(M\text{.}\)
Definition 11.5.4. Nullity of a Matrix.
Let \(M\) be a matrix. The nullity of \(M\text{,}\) denoted \(\Nullity(M)\), is the nullity of the linear transformation induced by \(M\text{.}\)
Section 11.6 Range vs. Column Space & Null Space vs. Null Space
Let \(M=\mat{C_1&C_2&\cdots &C_m}\) be an \(m\times m\) matrix with columns \(C_{1}\text{,}\) …, \(C_{m}\text{,}\) and let \(\mathcal{T}\) be the transformation induced by \(M\text{.}\) The column space of \(M\) is the set of all linear combinations of the columns of \(M\text{.}\) But, let’s be precise. The columns of \(M\) are lists of numbers, so to talk about the column space of \(M\text{,}\) we need to turn them into vectors. Fortunately, we have a nice notation for that. Since \(C_{i}\) is a list of numbers, \([C_{i}]_{\mathcal{E}}\) is a (true) vector, and
\begin{equation*}
\Col(M)=\Span\Set{[C_1]_{\mathcal{E}},[C_2]_{\mathcal{E}},\ldots,[C_m]_{\mathcal{E}}}.
\end{equation*}
Can we connect this to the range of \(\mathcal{T}\text{?}\) Well, by the definition of matrix multiplication, we know that
\begin{equation*}
M\matc{1\\0\\\vdots\\0}=M[\vec e_{1}]_{\mathcal{E}}= C_{1}
\end{equation*}
and in general \(M[\vec e_{i}]_{\mathcal{E}}=C_{i}\text{.}\) By the definition of induced transformation, we know
\begin{equation*}
[\mathcal{T}(\vec e_{i})]_{\mathcal{E}}= M[\vec e_{i}]_{\mathcal{E}}= C_{i},
\end{equation*}
and so
\begin{equation*}
\mathcal{T}(\vec e_{i})= [C_{i}]_{\mathcal{E}}.
\end{equation*}
Every input to \(\mathcal{T}\) can be written as a linear combination of \(\vec e_{i}\)’s (because \(\mathcal{E}\) is a basis) and so, because \(\mathcal{T}\) is linear, every output of \(\mathcal{T}\) can be written as a linear combination of \([C_{i}]_{\mathcal{E}}\)’s. In other words,
\begin{equation*}
\Range(\mathcal{T})=\Col(M).
\end{equation*}
This means that when trying to answer a question about the range of a linear transformation, we could think about the column space of its matrix instead (or vice versa).
Example 11.6.1.
Let \(\mathcal{T}:\R^{3}\to\R^{2}\) be defined by
\begin{equation*}
\mathcal{T}\mat{x\\y\\z}= \matc{2x-z\\4x-2z}.
\end{equation*}
Find \(\Range(\mathcal{T})\) and \(\Rank(\mathcal{T})\text{.}\)
Solution.
Let \(M\) be a matrix for \(\mathcal{T}\text{.}\) We know \(\Range(\mathcal{T})=\Col(M)\) and \(\Rank(\mathcal{T})=\Dim(\Range(\mathcal{T})) =\Dim(\Col(M))\text{.}\) By inspection, we see that
\begin{equation*}
M=\mat{2&0&-1\\4&0&-2}.
\end{equation*}
Again, by inspection, we see that \(\Set{\mat{2\\4}}\) is a basis for \(\Col(M)\) and \(\Col(M)\) is one dimensional. Therefore,
\begin{equation*}
\Range(\mathcal{T}) = \Span\Set{\mat{2\\4}}\qquad \text{and}\qquad \Rank(\mathcal{T})=1.
\end{equation*}
There is an alternative definition of the rank of a matrix which commonly appears. We’ll state it as a theorem.
Theorem 11.6.2.
Let \(M\) be a matrix. The rank of \(M\) is equal to the number of pivots in \(\Rref(M)\text{.}\)
Proof.
We know that \(\Rank(M)=\Dim(\Range(\mathcal{T}_{M}))=\Dim(\Col(M))\) where \(\mathcal{T}_{M}\) is the transformation induced by \(M\text{.}\) Further, a basis for \(\Col(M)\) consists of a maximal linearly independent subset of the columns of \(M\text{.}\) To find such a subset, we row reduce \(M\) and look at the columns of \(M\) that correspond to pivot columns of \(\Rref(M)\text{.}\)
When all is said and done, the number of elements in a basis for \(\Col(M)\) will be the number of pivots in \(\Rref(M)\text{,}\) which is the same as \(\Rank(M)\text{.}\)
Just like the range–column-space relationship, we also have a null-space–null-space relationship. More specifically, if \(\mathcal{T}\) is a linear transformation with matrix \(M\text{,}\) then \(\Null(\mathcal{T})=\Null(M)\text{.}\) From this fact, we deduce the following theorem.
Theorem 11.6.4.
Let \(\mathcal{T}\) be a linear transformation and let \(M\) be a matrix for \(\mathcal{T}\text{.}\) Then \(\Nullity(\mathcal{T})\) is equal to the number of free variable columns in \(\Rref(M)\text{.}\)
Proof.
We know \(\Nullity(\mathcal{T})=\Dim(\Null(\mathcal{T}))=\Dim(\Null(M))\text{.}\) Further, we know that the complete solution to \(M\vec x=\vec 0\) will take the form
\begin{equation*}
\vec x=t_{1}\vec d_{1}+\cdots +t_{k}\vec d_{k}
\end{equation*}
where \(k\) is the number of free variable columns in \(\Rref(M)\text{.}\) The algorithm for writing the complete solution to \(M\vec x=\vec 0\) ensures that \(\Set{\vec d_1,\ldots,\vec d_k}\) is a basis for \(\Null(M)\text{,}\) and so \(\Nullity(\mathcal{T})=k\text{,}\) which completes the proof.
Combining these facts, we can reformulate the Row-Col Dimension theorem as a theorem about ranks.
Theorem 11.6.5.
For a matrix \(A\text{,}\) we have \(\Rank(A)=\Rank(A^{T})\text{.}\)
Proof.
By the Row-Col Dimension theorem, we know \(\dim(\Col(A))=\dim(\Row(A))\text{.}\) By the definition of the transpose, we know \(\Row(A)=\Col(A^{T})\text{.}\) Therefore,
\begin{equation*}
\dim(\Col(A))=\dim(\Col(A^{T})),
\end{equation*}
which is another way of saying \(\Rank(A)=\Rank(A^{T})\text{.}\)
Section 11.7 The Rank-Nullity Theorem
The rank and the nullity of a linear transformation/matrix are connected by a powerful theorem.
Theorem 11.7.1. Rank-nullity Theorem for Matrices.
For a matrix \(A\text{,}\)
\begin{equation*}
\Rank(A)+\Nullity(A)=\ #\text{ of columns in $A$}.
\end{equation*}
The rank-nullity theorem’s statement is simple, but it is surprisingly useful. For example, consider the question: how many normal directions does a plane have in \(\R^{3}\) or in \(\R^{4}\) or in \(\R^{5}\text{?}\)
We already know the answer in \(\R^{3}\text{:}\) a plane has a two-dimensional set of direction vectors and a one-dimensional set (a line) of normal vectors. But, we can verify this fact using the rank-nullity theorem.
Let
\begin{equation*}
M=\mat{1&2&2}
\end{equation*}
and let \(\mathcal{P}=\Null(M)\text{.}\) We know \(\mathcal{P}\) is a plane with equation \(x+2y+2z=0\) and therefore is two dimensional. Further, non-zero vectors in the row space of \(M\) are normal vectors for \(\mathcal{P}\text{.}\) Since \(\Null(M)\) is two-dimensional and \(M\) has three columns, the rank-nullity theorem tells us that \(\Rank(M)=1\text{.}\) Therefore \(\Dim(\Col(M))=\Dim(\Row(M))=1\text{.}\) We conclude the set of normal vectors to \(\mathcal{P}=\Null(M)\) is a line (if we include \(\vec 0\)).
By contrast, let \(\mathcal{Q}\subseteq \R^{4}\) be the plane in \(\R^{4}\) given in vector form by
\begin{equation*}
\vec x=t\mat{1\\2\\2\\2}+s\mat{-1\\1\\-1\\1}.
\end{equation*}
How many normal vectors does \(\mathcal{Q}\) have? Well, the matrix \(A=\mat{1&2&2&2\\-1&1&-1&1}\) has rank \(2\) and therefore has nullity \(2\text{.}\) This means there exist two linearly independent normal directions for \(\mathcal{Q}\).
There is an equivalent rank-nullity theorem for linear transformations.
Theorem 11.7.2. Rank-nullity Theorem for Linear Transformations.
Let \(\mathcal{T}\) be a linear transformation. Then
\begin{equation*}
\Rank(\mathcal{T})+\Nullity(\mathcal{T})=\Dim(\text{domain of $\mathcal{T}$}).
\end{equation*}
Just like the rank-nullity theorem for matrices, the rank-nullity theorem for linear transformations can give insights about linear transformations that would be otherwise hard to see.
Exercises 11.8 Exercises
1.
For the following matrices, find their null space, column space, and row space.
\(M_{1}=\mat{1&2&1\\3&1&-2\\8&6&-2}\text{.}\)
\(M_{2}=\mat{0&2&1\\3&2&5}\text{.}\)
\(M_{3}=\mat{1&2\\3&1\\4&0}\text{.}\)
\(M_{4}=\mat{1&-2&0&-1\\3&5&-1&0\\2&3&-2&0\\0&0&0&1}\text{.}\)
Solution.
-
\(M_{1}= \mat{1&2&1\\3&1&-2\\8&6&-2}\text{;}\) \(\rref(M_{1})=\mat{1&0&-1\\0&1&1\\0&0&0}\text{.}\)
\(\Null(M_{1})=\Set{\vec{x} \in \R^3: M_1 \vec{x}=\vec 0}\text{;}\) therefore, we need to solve \(\mat{M_1|\vec0}\text{.}\)
\begin{align*}
\Null(M_{1})&= \Set{\vec x \in \R^3:\vec x = s\mat{1\\-1\\1} \; \text{for some } s\in\R}\\
&= \Span\Set{\mat{1\\-1\\1}}
\end{align*}
\begin{align*}
\Col(M_{1})&= \Span\Set{\mat{1\\3\\8},\mat{2\\1\\6},\mat{1\\-2\\-2}}\\
&= \Span\Set{\mat{1\\3\\8},\mat{2\\1\\6}}
\end{align*}
\begin{align*}
\Row(M_{1})&= \Span\Set{\mat{1\\2\\1},\mat{3\\1\\-2},\mat{8\\6\\-2}}\\
&= \Span\Set{\mat{1\\2\\1},\mat{3\\1\\-2}}\\
&= \Span\Set{\mat{1\\0\\-1},\mat{0\\1\\1}}
\end{align*}
-
\(\rref(M_{2})=\mat{1&0&4/3\\0&1&1/2}\text{.}\)
\begin{align*}
\Null(M_{2})&=\Span\Set{\mat{4/3\\1/2\\-1}}\\
\Col(M_{2})&=\Span\Set{\mat{0\\3},\mat{2\\2}}=\R^{2}\\
\Row(M_{2})&=\Span\Set{\mat{0\\2\\1},\mat{3\\2\\5}}
\end{align*}
-
\(\rref(M_{3})=\mat{1&0\\0&1\\0&0}\text{.}\)
\begin{align*}
\Null(M_{3})&=\Set{\mat{0\\0}}\\
\Col(M_{3})&=\Span\Set{\mat{1\\3\\4},\mat{2\\1\\0}}\\
\Row(M_{3})&=\Span\Set{\mat{1\\2},\mat{3\\1}}=\R^{2}
\end{align*}
-
\(\rref(M_{4})=I_{4\times4}\text{.}\)
\begin{align*}
\Null(M_{4})&=\Set{\mat{0\\0\\0\\0}}\\
\Col(M_{4})&=\R^{4}\\
\Row(M_{4})&=\R^{4}
\end{align*}
2.
Let \(\mathcal{P}\) be the plane given by \(3x+4y+5z=0\text{,}\) and let \(T:\R^{3}\to\R^{3}\) be projection onto \(\mathcal{P}\text{.}\)
Find \(\Range(T)\) and \(\Rank(T)\text{.}\)
Find \(\Null(T)\) and \(\Nullity(T)\text{.}\)
Solution.
-
Since \(T\) is the projection onto \(\mathcal{P}\text{,}\) \(T(\vec{x})=\vec{x}\) for any \(\vec{x}\in\mathcal{P}\) and thus \(\mathcal{P}\subseteq\Range(T)\text{.}\)
Let \(\vec{y}\in\Range(T)\text{.}\) This means that \(\vec{y}=T(\vec{x})\) for some \(\vec{x}\in\R^{3}\text{.}\) By definition of a projection, \(\vec{y}\) is the closest point in \(\mathcal{P}\) to \(\vec{x}\text{,}\) so \(\vec{y}\in\mathcal{P}\) and thus \(\Range(T)\subseteq\mathcal{P}\text{.}\)
This shows that \(\Range(T)=\mathcal{P}\text{.}\)
Since \(\Range(T)=\mathcal{P}\) is a plane, we have \(\Rank(T)=\Dim (\Range(T))=\Dim(\mathcal{P})=2\text{.}\)
-
By the rank-nullity theorem,
\begin{equation*}
\Rank(T)+\Nullity(T)=\Dim(\text{domain of $T$}).
\end{equation*}
Thus,
\begin{equation*}
\Nullity(T)=\Dim(\text{domain of $T$})-\Rank(T)=3-2=1.
\end{equation*}
Since \(\vec{n}=\mat{3\\4\\5}\) is normal to \(\mathcal{P}\text{,}\) we have \(T(\vec{n})=\vec{0}\) and thus \(\vec{n}\in\Null(T)\text{.}\) Since \(\Nullity(T)=1\) and \(\vec{n}\) is a non-zero vector in \(\Null( T)\text{,}\) we have
\begin{equation*}
\Null(T)=\Span\Set{\mat{3\\4\\5}}.
\end{equation*}
3.
Find the range and null space of the following linear transformations.
\(\mathcal{P}:\R^{2}\to\R^{2}\text{,}\) where \(\mathcal{P}\) is projection on to the line \(y=x\text{.}\)
Let \(\theta\in \R\) and let \(\mathcal{R}:\R^{2}\to\R^{2}\) to be the transformation which rotates all vectors by counter-clockwise by \(\theta\) radians.
\(\mathcal{F}:\R^{2}\to\R^{2}\text{,}\) where \(\mathcal{F}\) reflects over the \(x\)-axis.
\(\mathcal{M}:\R^{3}\to\R^{3}\) where \(\mathcal{M}\) is the matrix transformation given by \(\mat{1&2&3\\4&5&6\\7&8&9}\text{.}\)
\(\mathcal{Q}:\R^{3}\to\R^{1}\) defined by \(\mathcal{Q}\mat{x\\y\\z}=x+z\text{.}\)
Solution.
-
Similar to Problem 2.(a),
\begin{align*}
\Range(P)&=\Set{\text{line given by the equation } y=x}\\
&=\Span\Set{\mat{1\\1}}\\
\Null(P)&=\Set{\text{line given by the equation } y=-x}\\
&=\Span\Set{\mat{1\\-1}}
\end{align*}
Notice that if we have a non-zero vector, after rotation the vector still remains non-zero as rotation does not change magnitude of vectors. Thus, \(\Null(R)=\Set{\mat{0\\0}}\text{.}\) From the rank-nullity theorem, \(\Rank(R) = 2\) implies that \(\Range(R) = \R^{2}\text{.}\)
Since \(F\mat{x\\y}= \mat{x\\-y}\text{,}\) if \(F\mat{x\\y}= \mat{0\\0}\text{,}\) then \(x=y=0\text{.}\) Thus, \(\Null(F)=\Set *{\mat{0\\0}}\text{.}\) From the rank-nullity theorem, \(\Rank(F)=2\) implies that \(\Range(F)=\R^{2}\text{.}\)
Let M be the matrix of the transformation \(\mathcal{M}\text{.}\) \(\Rref(M)= \mat{1&0&-1\\0&1&2\\0&0&0}\) So, \(\Range(M)=\Col(M) =\Span\Set{\mat{1\\4\\7},\mat{2\\5\\8}}\) and \(\Null(M)=\Span \Set{\mat{1\\-2\\1}}\text{.}\)
Notice that \(\Range(Q)\subseteq\R\text{.}\) For any \(t\in\R^{1}\text{,}\) we have \(Q\mat{t\\0\\0}=t+0=t\) and thus \(t\in\Range(Q)\text{.}\) Hence, \(\Range(Q) = \R^{1}\text{.}\) For \(\vec x \in \R^{3}\text{,}\) \(Q(\vec x)=Q\mat{x\\y\\z}=x+z=0\text{.}\) Thus, \(\Null(Q)\) is the plane given by the equation \(x+z=0\) Alternatively, \(\Null(Q) = \Span\Set{\mat{1\\0\\-1},\mat{0\\1\\0}}\)
4.
Let \(\mathcal{T}\) be the transformation induced by the matrix \(\mat{7&5\\-2&-2}\text{,}\) and \(\vec v=3\xhat-3\yhat\text{.}\) Compute \(\mathcal{T}\vec v\) and \([\mathcal{T}\vec v]_{\mathcal{E}}\text{.}\)
Let \(\mathcal{T}\) be the transformation induced by the matrix \(\mat{3&7&5\\1&-2&-2}\text{,}\) and \(\vec v=2\xhat+0\yhat+4\zhat\text{.}\) Compute \(\mathcal{T}\vec v\) and \([\mathcal{T}\vec v]_{\mathcal{E}}\text{.}\)
Solution.
-
\([\vec v]_{\varepsilon}= \mat{3\\-3}\;\) So, \([T\vec v]_{\varepsilon}= \mat{7&5\\-2&-2}\mat{3\\-3}=\mat{6\\0}\)
Thus \(T\vec v =6\vec e_{1}.\)
-
\([\vec v]_{\varepsilon}= \mat{2\\0\\4}\;\) So, \([T\vec v]_{\varepsilon}= \mat{3&7&5\\1&-2&-2}\mat{2\\0\\4}=\mat{26\\-6}\)
Thus \(T\vec v =26\vec e_{1}-6\vec e_{2}\;\) [Here \(\vec e_{1}, \vec e_{2}\in \R^{2}\)]
5.
For each statement below, determine whether it is true or false. Justify your answer.
Let \(A\) be an arbitrary matrix. Then \(\Col(A)=\Col(A^{T})\text{.}\)
Let \(T:\R^{m}\to\R^{n}\) be a transformation (not necessarily linear). If \(\Null(T)=\Set{\vec x\in\R^m \given T(\vec x)=\vec 0}\) is a subspace, then \(T\) is linear.
Let \(T:\R^{m}\to\R^{n}\) be a linear transformation. Then \(\Nullity(T) \geq n\text{.}\)
Let \(T:\R^{m}\to\R^{n}\) be a linear transformation induced by a matrix \(M\text{.}\) If \(\Rank(T) = n\text{,}\) then \(\Nullity(M) = 0\text{.}\)
Solution.
-
False. \(\Range(A)\) and \(\Range(A^{T})\) need not even be in the same space.
For example, take \(A=\mat{1&0\\0&1\\0&0}, A^{T}=\mat{1&0&0\\0&1&0}\) Then \(\Range(A)=\Set{xy-\text{plane in }\R^3}\text{,}\) whereas \(\Range(A^{T})=\R^{2}\)
-
False. Consider, \(T:\R^{2}\to\R^{2}\) given by
\begin{equation*}
T\mat{x\\y}=\mat{x^2\\y}
\end{equation*}
Then, \(\Null(T)= \Set{\mat{0\\0}}\) which is a subspace. But \(T\) is not linear.
-
False. From rank-nullity theorem,
\begin{align*}
\Nullity(T)&\leq \Dim(\text{ domain of }T)&=m \quad \text{ (in this case) }
\end{align*}
So, for any \(n>m\) this is false. Consider for example
\begin{align*}
&T:\R^{1}\to\R^{2}\;\text{given by}&T(x)=\mat{x\\0};\; \Nullity(T)=0
\end{align*}
False. This is false whenever \(m>n\) (follows from rank-nullity theorem) Take for example: \(T:\R^{3}\to\R^{2}\) induced by the matrix \(A=\mat{1&0&0\\0&1&0}\text{;}\) we have that \(\Rank(T)=\Dim( \Col A)=2\text{,}\) and \(\Nullity(T) = \Nullity(A)=1\text{.}\)
6.
Give an example of a \(3\times4\) matrix \(M\) with the specified rank, or explain why one cannot exist.
\(\displaystyle \Rank(M) = 0\)
\(\displaystyle \Rank(M) = 1\)
\(\displaystyle \Rank(M) = 3\)
\(\displaystyle \Rank(M) = 4\)
Solution.
\(\displaystyle \mat{0&0&0&0\\0&0&0&0\\0&0&0&0}\)
\(\displaystyle \mat{1&0&0&0\\0&0&0&0\\0&0&0&0}\)
\(\displaystyle \mat{1&0&0&0\\0&1&0&0\\0&0&1&0}\)
A \(3\times4\) matrix with rank 4 cannot exist. Examples of justifications are: 1) The column space of a \(3\times 4\) matrix is a subspace of \(\R^{3}\) and so cannot be four dimensional; 2) A matrix with rank 4 has 4 pivots, but a \(3\times 4\) matrix can have at most 3 pivots, which is less than 4.
7.
Let \(\mathcal{P}:\R^{3}\to\R^{3}\) be the projection onto the \(xy\)-plane.
Find a matrix \(M_{\mathcal{P}}\) for the transformation.
Find the range of \(\mathcal{P}\text{.}\)
Find the column space of \(M_{\mathcal{P}}\text{.}\) Are there any similarities to your answer in the previous part?
Find the null space of \(\mathcal{P}\) and \(M_{\mathcal{P}}\text{.}\) Are there similarities between the null space of a linear transformation and its associated matrix?
Solution.
\(\displaystyle \mat{1&0&0\\0&1&0\\0&0&0}\)
The range of \(\mathcal{P}\) is the \(xy\)-plane.
The column space of \(M_{\mathcal{P}}\) is \(\Span\Set{\mat{1\\0\\0},\mat{0\\1\\0}}\text{,}\) which is the \(xy\)-plane. The column space of a matrix is the same as the range of its induced transformation.
The null space of \(\mathcal{P}\) and \(M_{\mathcal{P}}\) is \(\Span \Set{\mat{0\\0\\1}}\text{,}\) which is the \(z\)-axis. The null space of a matrix is the same as the null space of its induced transformation.
