Linear Algebra

If spans remind you of vector forms of lines and planes, your intuition is keen. Consider the line \(\ell\) given in vector form by

The line \(\ell\) passes through the origin, and if we unravel its definition, we see

Similarly, if \(\mathcal{P}\) is a plane given in vector form by

then

If the “\(\vec p\)” in our vector form is \(\vec 0\text{,}\) then that vector form actually defines a span. This means (if you accept that every line/plane through the origin has a vector form) that every line/plane through the origin can be written as a span. Conversely, if \(X=\Span\Set{\vec v_1,\ldots,\vec v_n}\) is a span, we know \(\vec 0=0\vec v_{1}+\cdots+0\vec v_{n}\in X\text{,}\) and so every span passes through the origin.

As it turns out, spans exactly describe points, lines, planes, and volumes

¹

We use the word volume to indicate the higher-dimensional analogue of a plane.

through the origin.

We’re going to work around the fact that only objects which pass through the origin can be written as spans, but first let’s take a detour and learn about set addition.

Set sums are very different than regular sums despite using the same symbol, “\(+\)”

¹

For example, \(A+\Set{}=\Set{}\text{,}\) which might seem counterintuitive for an “addition” operation.

. However, they are very useful. Let \(C=\Set{\vec x\in\R^2\given \norm{\vec x}=1}\) be the unit circle centered at the origin, and consider the sets

Rewriting, we see \(X=\Set{\vec x+\yhat\given \norm{\vec x}=1}\) is just \(C\) translated by \(\yhat\text{.}\) Similarly, \(Y=\Set{\vec x+\vec v\given \norm{\vec x}=1\text{ and }\vec v=3\xhat\text{ or }\vec v=\yhat}=(C+\Set{3\xhat})\cup (C+\Set{\yhat})\text{,}\) and so \(Y\) is the union of two translated copies of \(C\)

²

If you want to stretch your mind, consider what \(C+C\) is as a set.

. Finally, \(Z\) is the union of three translated copies of \(C\text{.}\)

Set addition allows us to easily create parallel lines and planes by translation. For example, consider the lines \(\ell_{1}\) and \(\ell_{2}\) given in vector form as \(\vec x=t\vec d\) and \(\vec x=t\vec d+\vec p\text{,}\) respectively, where \(\vec d=\mat{2\\1}\) and \(\vec p=\mat{-1\\1}\text{.}\) These lines differ from each other by a translation. That is, every point in \(\ell_{2}\) can be obtained by adding \(\vec p\) to a corresponding point in \(\ell_{1}\text{.}\) Using the idea of set addition, we can express this relationship by the equation

Note: it would be incorrect to write “\(\ell_{2}=\ell_{1}+\vec p\)”. Because \(\ell_{1}\) is a set and \(\vec p\) is not a set, “\(\ell_{1}+\vec p\)” does not make mathematical sense.

We can now see translated spans provide an alternative notation to vector form for specifying lines and planes. If \(Q\) is described in vector form by

then

Let

Since \(\vec w=\vec u+\vec v\text{,}\) we know that \(\vec w\in\Span\Set{\vec u,\vec v}\text{.}\) Geometrically, this is also clear because \(\Span\Set{\vec u,\vec v}\) is the \(xy\)-plane in \(\R^{3}\) and \(\vec w\) lies on that plane.

What about \(\Span\Set{\vec u,\vec v,\vec w}\text{?}\) Intuitively, since \(\vec w\) is already a linear combination of \(\vec u\) and \(\vec v\text{,}\) we can’t get anywhere new by taking linear combinations of \(\vec u\text{,}\) \(\vec v\text{,}\) and \(\vec w\) compared to linear combinations of just \(\vec u\) and \(\vec v\text{.}\) So \(\Span\Set{\vec u,\vec v}=\Span\Set{\vec u,\vec v,\vec w}\text{.}\)

Can we prove this from the definitions? Yes! Suppose \(\vec r\in \Span\Set{\vec u,\vec v,\vec w}\text{.}\) By definition,

for some \(\alpha,\beta,\gamma\in\R\text{.}\) Since \(\vec w=\vec u+\vec v\text{,}\) we see

Thus, \(\Span\Set{\vec u,\vec v,\vec w}\subseteq \Span\Set{\vec u,\vec v}\text{.}\) Conversely, if \(\vec s\in\Span\Set{\vec u,\vec v}\text{,}\) by definition,

for some \(a,b\in \R\text{,}\) and so \(\vec s\in\Span\Set{\vec u,\vec v,\vec w}\text{.}\) Thus \(\Span\Set{\vec u,\vec v}\subseteq\Span\Set{\vec u,\vec v,\vec w}\text{.}\) We conclude \(\Span\Set{\vec u,\vec v}=\Span\Set{\vec u,\vec v,\vec w}\text{.}\)

In this case, \(\vec w\) was a redundant vector—it wasn’t needed for the span. When a set contains a redundant vector, we call the set linearly dependent.

We will also refer to sets of vectors (for example \(\Set{\vec v_1,\ldots,\vec v_n}\)) as being linearly independent or linearly dependent. For technical reasons, we didn’t state the definition in terms of sets

¹

The issue is, every element of a set is unique. Clearly, the vectors \(\vec v\) and \(\vec v\) are linearly dependent, but \(\Set{\vec v,\vec v}=\Set{\vec v}\text{,}\) and so \(\Set{\vec v,\vec v}\) is technically a linearly independent set. This issue would be resolved by talking about multisets instead of sets, but it isn’t worth the hassle.

.

The geometric definition of linear dependence says that the vectors \(\vec v_{1},\ldots,\vec v_{n}\) are linearly dependent if you can remove at least one vector without changing the span. In other words, \(\vec v_{1},\ldots,\vec v_{n}\) are linearly dependent if there is a redundant vector.

We can also think of linear independence/dependence from an algebraic perspective. Suppose the vectors \(\vec u\text{,}\) \(\vec v\text{,}\) and \(\vec w\) satisfy

The set \(\Set{\vec u,\vec v,\vec w}\) is linearly dependent since \(\vec w\in\Span\Set{\vec u,\vec v}\text{,}\) but equation (3.4.1) can be rearranged to get

Here we have expressed \(\vec 0\) as a linear combination of \(\vec u\text{,}\) \(\vec v\text{,}\) and \(\vec w\text{.}\) By itself, this is nothing special. After all, we know \(\vec 0=0\vec u+0\vec v+0\vec w\) is a linear combination of \(\vec u\text{,}\) \(\vec v\text{,}\) and \(\vec w\text{.}\) However, the right side of equation (3.4.2) has non-zero coefficients, which makes the linear combination non-trivial.

We can always write \(\vec 0\) as a linear combination of vectors if we let all the coefficients be zero, but it turns out we can only write \(\vec 0\) as a non-trivial linear combination of vectors if those vectors are linearly dependent. This is the inspiration for another definition of linear independence/dependence.

The idea of a “redundant vector” coming from the geometric definition of linear dependence is easy to visualize, but it can be hard to prove things with—checking for linear independence with the geometric definition involves verifying for every vector that it is not in the span of the others. The algebraic definition on the other hand is less obvious, but the reasoning is easier. You only need to analyze solutions to one equation!

The algebraic definition of linear independence can teach us something about solutions to systems of equations.

Recall the linearly dependent vectors

which satisfy the non-trivial relationship \(\vec u+\vec v-\vec w=\vec 0\text{.}\) Since \(\vec u+\vec v-\vec w=\vec 0\) is a non-trivial relationship giving \(\vec 0\text{,}\) we can use it to generate others. For example,

are all different non-trivial linear combinations that give \(\vec 0\text{.}\) In other words, if the equation \(\alpha\vec u+\beta \vec v+\gamma \vec w=\vec 0\) has a non-trivial solution, it has infinitely many non-trivial solutions. Conversely, if the equation \(\alpha\vec u+\beta \vec v+\gamma \vec w=\vec 0\) has infinitely many solutions, one of them has to be non-trivial!

Equations where one side is \(\vec 0\) show up often and are called homogeneous equations.

This insight links linear independence and homogeneous systems together, and is encapsulated in the following theorem.

This theorem has a practical application: suppose you wanted to decide if the vectors \(\vec a\text{,}\) \(\vec b\text{,}\) and \(\vec c\) were linearly dependent. You could (i) find a non-trivial solution to \(x\vec a+y\vec b+z\vec c=\vec 0\text{,}\) or (ii) merely show that \(x\vec a+y\vec b+z\vec c=\vec 0\) has more than one solution. Sometimes one is easier than the other.

The equation

represents a plane in vector form whenever \(\vec d_{1}\) and \(\vec d_{2}\) are non-zero, non-parallel vectors. In other words, \(\vec x=t_{1}\vec d_{1}+t_{2}\vec d_{2}\) represents a plane whenever \(\Set{\vec d_1,\vec d_2}\) is linearly independent.

Does this reasoning work for lines too? The equation

represents a line in vector form precisely when \(\vec d\neq \vec 0\text{.}\) And \(\Set{\vec d}\) is linearly independent exactly when \(\vec d\neq 0\text{.}\)

This reasoning generalizes to volumes. The equation

represents a volume in vector form exactly when \(\Set{\vec d_1,\vec d_2,\vec d_3}\) is linearly independent. To see this, suppose \(\Set{\vec d_1,\vec d_2,\vec d_3}\) were linearly dependent. That means one or more vectors could be removed from \(\Set{\vec d_1,\vec d_2,\vec d_3}\) without changing its span. Therefore, if \(\Set{\vec d_1,\vec d_2,\vec d_3}\) is linearly dependent \(\vec x=t_{1}\vec d_{1}+t_{2}\vec d_{2}+t_{3}\vec d_{3}\) at best represents a plane (though it could be a line or a point).

We now have a way of testing the validity of a vector-form representation of a line/plane/volume. Just check whether the chosen direction vectors are linearly independent!