When analyzing linear transformations or matrices, we’re often interested in studying the subspaces where vectors are stretched by only one eigenvalue. These are called the eigenspaces.
Now is the time when linear algebra and regular algebra (the solving of non-linear equations) combine. We know, every root of the characteristic polynomial of a matrix gives an eigenvalue for that matrix. Since the degree of the characteristic polynomial of an \(n\times n\) matrix is always \(n\text{,}\) the fundamental theorem of algebra tells us exactly how many roots to expect.
Consider the following two theorems.
We can now deduce the following.
Proof.
Let \(A\) be an \(n\times n\) matrix with eigenvalues \(\lambda_{1},\ldots,\lambda_{k}\text{.}\) Let \(E_{1},\ldots,E_{k}\) be bases for the eigenspaces corresponding to \(\lambda_{1},\ldots,\lambda_{k}\text{.}\) We will start by showing \(E=E_{1}\cup\cdots\cup E_{k}\) is a linearly independent set using the following two lemmas.
No New Eigenvalue Lemma. Suppose that \(\vec v_{1},\ldots,\vec v_{k}\) are linearly independent eigenvectors of a matrix \(A\text{,}\) and let \(\lambda_{1},\ldots,\lambda_{k}\) be the corresponding eigenvalues. Then, any eigenvector for \(A\) contained in \(\Span\Set{\vec v_1,\ldots,\vec v_k}\) must have one of \(\lambda_{1},\ldots,\lambda_{k}\) as its eigenvalue.
The proof goes as follows. Suppose \(\vec v=\sum_{i\leq k}\alpha_{i}\vec v_{i}\) is an eigenvector for \(A\) with eigenvalue \(\lambda\text{.}\) We now compute \(A\vec v\) in two different ways: once by using the fact that \(\vec v\) is an eigenvector, and again by using the fact that \(\vec v\) is a linear combination of other eigenvectors. Observe
\begin{equation*}
A\vec v=\lambda \vec v=\lambda\left(\sum_{i\leq k}\alpha_{i}\vec v_{i}\right) =\sum_{i\leq k}\alpha_{i}\lambda\vec v_{i}
\end{equation*}
and
\begin{equation*}
A\vec v=A\left(\sum_{i\leq k}\alpha_{i}\vec v_{i}\right) =\sum_{i\leq k}\alpha_{i} A\vec v_{i} =\sum_{i\leq k}\alpha_{i}\lambda_{i}\vec v_{i}.
\end{equation*}
We now have
\begin{equation*}
\vec 0=A\vec v-A\vec v = \sum_{i\leq k}\alpha_{i}\lambda\vec v_{i} -\sum_{i\leq k}\alpha_{i}\lambda_{i}\vec v_{i} =\sum_{i\leq k}\alpha_{i}(\lambda-\lambda_{i})\vec v_{i}.
\end{equation*}
Because \(\vec v_{1},\ldots,\vec v_{k}\) are linearly independent, we know \(\alpha_{i}(\lambda-\lambda_{i})=0\) for all \(i\leq k\text{.}\) Further, because \(\vec v\) is non-zero (it’s an eigenvector), we know at least one \(\alpha_{i}\) is non-zero. Therefore \(\lambda-\lambda_{i}=0\) for at least one \(i\text{.}\) In other words, \(\lambda=\lambda_{i}\) for at least one \(i\text{,}\) which is what we set out to show.
Basis Extension Lemma. Let \(P=\Set{\vec p_1,\ldots,\vec p_a}\) and \(Q=\Set{\vec q_1,\ldots,\vec q_b}\) be linearly independent sets, and suppose \(P\cup\Set{\vec q}\) is linearly independent for all non-zero \(\vec q\in\Span Q\text{.}\) Then \(P\cup Q\) is linearly independent.
To show this, suppose \(\vec 0=\alpha_{1}\vec p_{1}+\cdots+\alpha_{a}\vec p_{a}+\beta_{1}\vec q_{1}+\cdots+\beta_{b}\vec q_{b}\) is a linear combination of vectors in \(P\cup Q\text{.}\) Let \(\vec q=\beta_{1}\vec q_{1}+\cdots+\beta_{b}\vec q_{b}\text{.}\) First, note that \(\vec q\) must be the zero vector. If not, \(\vec 0=\alpha_{1}\vec p_{1}+\cdots+\alpha_{a}\vec p_{a}+\vec q\) is a non-trivial linear combination of vectors in \(P\cup\Set{\vec q}\text{,}\) which contradicts the assumption that \(P\cup\Set{\vec q}\) is linearly independent. Since we’ve established \(\vec 0=\vec q=\beta_{1}\vec q_{1}+\cdots+\beta_{b}\vec q_{b}\text{,}\) we conclude \(\beta_{1}=\cdots=\beta_{b}=0\) because \(Q\) is linearly independent. It follows that since \(\vec 0=\alpha_{1}\vec p_{1}+\cdots+\alpha_{a}\vec p_{a}+\vec q =\alpha_{1}\vec p_{1}+\cdots+\alpha_{a}\vec p_{a}+\vec 0\text{,}\) we must have that \(\alpha_{1}=\cdots=\alpha_{a}=0\) because \(P\) is linearly independent. This shows that the only way to express \(\vec 0\) as a linear combination of vectors in \(P\cup Q\) is as the trivial linear combination, and so \(P\cup Q\) is linearly independent.
Now we can put our lemmas to good use. We will use induction to show that \(E=E_{1}\cup\cdots\cup E_{k}\) is linearly independent. By assumption \(E_{1}\) is linearly independent. Now, suppose \(U=E_{1}\cup\cdots\cup E_{j}\) is linearly independent. By construction, every non-zero vector \(\vec v\in\Span E_{j+1}\) is an eigenvector for \(A\) with eigenvalue \(\lambda_{j+1}\text{.}\) Therefore, since \(\lambda_{j+1}\neq \lambda_{i}\) for \(1\leq i\leq j\text{,}\) we may apply the No New Eigenvalue Lemma to see that \(\vec v\notin\Span U\text{.}\) It follows that \(U\cup\Set{\vec v}\) is linearly independent. Since \(E_{j+1}\) is itself linearly independent, we may now apply the Basis Extension Lemma to deduce that \(U\cup E_{j+1}\) is linearly independent. This shows that \(E=E_{1}\cup\cdots\cup E_{k}\) is linearly independent.
To conclude notice that by construction, \(\text{geometric mult}(\lambda_{i})=\abs{E_i}\text{.}\) Since \(E=E_{1}\cup\cdots\cup E_{k}\) is linearly independent, the \(E_{i}\)’s must be disjoint and so \(\sum\text{geometric mult}(\lambda_{i})=\sum \abs{E_i}=\abs{E}\text{.}\) If \(\sum\text{geometric mult}(\lambda_{i})=n\text{,}\) then \(E\subseteq\R^{n}\) is a linearly independent set of \(n\) vectors and so is a basis for \(\R^{n}\text{.}\) Finally, because we have a basis for \(\R^{n}\) consisting of eigenvectors for \(A\text{,}\) we know \(A\) is diagonalizable.
Conversely, if there is a basis \(E\) for \(\R^{n}\) consisting of eigenvectors, we must have a linearly independent set of \(n\) eigenvectors. Grouping these eigenvectors by eigenvalue, an application of the No New Eigenvalue Lemma shows that each group must actually be a basis for its eigenspace. Thus, the sum of the geometric multiplicities must be \(n\text{.}\)
Finally, if complex eigenvalues are allowed, the algebraic multiplicities sum to \(n\text{.}\) Since the algebraic multiplicities bound the geometric multiplicities, the only way for the geometric multiplicities to sum to \(n\) is if corresponding geometric and algebraic multiplicities are equal.
