Linear Maps - II

We will now discuss certain additional topics related to linear maps that are useful in the study of continuum mechanics.

Important invariants of linear maps

The fact that the set of all volume forms on a finite dimensional inner product space is itself a vector space of dimension $1$ permits a basis-independent means to define a variety of useful functions on linear maps. Two such functions, the determinant and trace of a linear map are discussed now. These functions are called invariants of linear maps since their definitions are independent of the choice of a basis.

Determinant

Given a linear map $\mathsf{T} \in L(V,V)$, where $V$ is a finite dimensional inner product space of dimension $n$, the determinant of $\mathsf{T}$, written $\text{det}(\mathsf{T})$ is defined as follows: for any $\mathsf{u}_1, \ldots, \mathsf{u}_n \in V$, $$\text{det}(\mathsf{T}) = \frac{\mathsf{\epsilon}(\mathsf{T}\mathsf{u}_1, \ldots, \mathsf{T}\mathsf{u}_n)}{\mathsf{\epsilon}(\mathsf{u}_1, \ldots, \mathsf{u}_n)}.$$ Here, $\mathsf{\epsilon} \in \Omega^n(V)$ is a chosen volume form on $V$. To see that this definition of the determinant is well-defined, note that it is possible to define a volume form $\mathsf{\epsilon}_{\mathsf{T}} \in \Omega^n(V)$ on $V$, given $\mathsf{\epsilon} \in \Omega^n(V)$ and $\mathsf{T} \in L(V,V)$, as follows: for any $\mathsf{u}_1, \ldots, \mathsf{u}_n \in V$, $$\mathsf{\epsilon}_{\mathsf{T}}(\mathsf{u}_1, \ldots, \mathsf{u}_n) = \mathsf{\epsilon}(\mathsf{T}\mathsf{u}_1, \ldots, \mathsf{T}\mathsf{u}_n).$$ The fact that the set of all volume forms is a vector space of dimension $1$ implies that $\mathsf{\epsilon}_{\mathsf{T}}$ is a scalar multiple of $\mathsf{\epsilon}$. This scalar multiple is, in fact, defined as the determinant of $\mathsf{T}$.

Example

To see how this definition is related to the determinant of a matrix encountered in elementary linear algebra, it is helpful to work in $\mathbb{R}^3$. The determinant of a linear map $\mathsf{T}:\mathbb{R}^3 \to \mathbb{R}^3$ is computed as follows: choosing $\mathsf{u}_1, \mathsf{u}_2, \mathsf{u}_3 \in \mathbb{R}^3$ to be the standard basis $(\mathsf{e}_i)$ of $\mathbb{R}^3$, and $\mathsf{\epsilon}$ to be the standard volume form on $\mathbb{R}^3$, $$\begin{split} \mathsf{\epsilon}(\mathsf{e}_i, \mathsf{e}_j, \mathsf{e}_k) \text{det}(\mathsf{T}) &= \mathsf{\epsilon}(\mathsf{T}\mathsf{e}_i, \mathsf{T}\mathsf{e}_j, \mathsf{T}\mathsf{e}_k)\\ \epsilon_{ijk} \text{det}(\mathsf{T}) &= \mathsf{\epsilon}\left(\sum T_{ai}\mathsf{e}_a, \sum T_{bj} \mathsf{e}_b, T_{ck} \mathsf{e}_c\right)\\ &= \sum T_{ai}T_{bj}T_{ck}\mathsf{\epsilon}(\mathsf{e}_a, \mathsf{e}_b, \mathsf{e}_c)\\ &= \sum \epsilon_{abc} T_{ai} T_{bj} T_{ck}. \end{split}$$ Note that the final expression is the familiar expression for the determinant of the matrix $[\mathsf{T}]$. It is a good exercise to expand this and check that it indeed reduces to the familiar expression for the determinant.

Example

To see the advantage in the abstract and basis-independent definition of the determinant provided here, consider the determinant of the product of two linear maps $\mathsf{S}, \mathsf{T} \in L(V,V)$. Given any $\mathsf{u}_1, \ldots, \mathsf{u}_n \in V$, $$\begin{split} \text{det}(\mathsf{S}\mathsf{T}) &= \frac{\mathsf{\epsilon}(\mathsf{S}\mathsf{T}\mathsf{u}_1, \ldots, \mathsf{S}\mathsf{T}\mathsf{u}_n)}{\mathsf{\epsilon}(\mathsf{u}_1, \ldots, \mathsf{u}_n)}\\ &= \frac{\mathsf{\epsilon}(\mathsf{S}(\mathsf{T}\mathsf{u}_1), \ldots, \mathsf{S}(\mathsf{T}\mathsf{u}_n))}{\mathsf{\epsilon}(\mathsf{T}\mathsf{u}_1, \ldots, \mathsf{T}\mathsf{u}_n)} \frac{\mathsf{\epsilon}(\mathsf{T}\mathsf{u}_1, \ldots, \mathsf{T}\mathsf{u}_n)}{\mathsf{\epsilon}(\mathsf{u}_1, \ldots, \mathsf{u}_n)}\\ &= \text{det}(\mathsf{S})\text{det}(\mathsf{T}). \end{split}$$ Notice how this proof of the fact that $\text{det}(\mathsf{S}\mathsf{T}) = \text{det}(\mathsf{S})\text{det}(\mathsf{T})$ is significantly simpler than the proof in terms of the elementary definition of the determinant in terms of the components of a linear map with respect to suitable choice of bases.

Remark

It is left as a simple exercise to verify, along the same lines as the previous example, that for any $a,b \in \mathbb{R}$, $$\text{det}(a\mathsf{S} + b\mathsf{T}) \neq a\,\text{det}(\mathsf{S}) + b\,\text{det}(\mathsf{T}).$$ The determinant is thus not a linear map.

Example

Let $\mathsf{u},\mathsf{v} \in V$ be any two vectors in an inner product space $V$ of dimension $n$. Then the determinant of the linear map $\mathsf{u} \otimes \mathsf{v} \in L(V,V)$ is always $0$. To see this, note that for any $\mathsf{w}_1, \ldots, \mathsf{w}_n \in V$, $$\begin{split} \text{det}(\mathsf{u} \otimes \mathsf{v}) &= \frac{\epsilon(\mathsf{u} \otimes \mathsf{v}(\mathsf{w}_1), \ldots, \mathsf{u}\otimes\mathsf{v}(\mathsf{w}_n))}{\epsilon(\mathsf{w}_1, \ldots, \mathsf{w}_n)}\\ &= \prod_{i=1}^n (\mathsf{v}\cdot\mathsf{w}_i) \; \frac{\epsilon(\mathsf{u}, \ldots, \mathsf{u})}{\epsilon(\mathsf{w}_1, \ldots, \mathsf{w}_n)}\\ &= 0. \end{split}$$ The last step follows from the skew-symmetry of the volume form $\epsilon \in \Omega^n(V)$. We have thus demonstrated that $\text{det}(\mathsf{u} \otimes \mathsf{v}) = 0$.

Example

Suppose that $\mathsf{A} \in L(V,V)$ is a linear map on an $n$-dimensional inner product space $V$. Then, for any real number $a\in\mathbb{R}$, $$\text{det}(a\mathsf{A}) = a^n \text{det}(\mathsf{A}).$$ This is easily proved as follows: for any $\mathsf{w}_1, \ldots, \mathsf{w}_n \in V$, $$\begin{split} \text{det}(a\mathsf{A}) &= \frac{\epsilon(a\mathsf{A}\mathsf{w}_1, \ldots, a\mathsf{A}\mathsf{w}_n)}{\epsilon(\mathsf{w}_1, \ldots, \mathsf{w}_n)}\\ &= a^n \frac{\epsilon(\mathsf{A}\mathsf{w}_1, \ldots, \mathsf{A}\mathsf{w}_n)}{\epsilon(\mathsf{w}_1, \ldots, \mathsf{w}_n)}\\ &= a^n \text{det}(\mathsf{A}). \end{split}$$

Trace

The trace of the linear map $\mathsf{T} \in L(V,V)$, written $\text{tr}(\mathsf{T})$, is defined as follows: for any $\mathsf{u}_1, \ldots, \mathsf{u}_n \in V$, $$\text{tr}(\mathsf{T}) = \sum_i \frac{\mathsf{\epsilon}(\mathsf{u}_1, \ldots, \mathsf{T}\mathsf{e}_i, \ldots, \mathsf{u}_n)}{\mathsf{\epsilon}(\mathsf{u}_1, \ldots, \mathsf{u}_n)}.$$ Despite the cumbersome form of the definition of the trace of a linear map adopted here, it will prove to be very convenient later on.

Example

To understand this definition better, consider the special case of a linear map $\mathsf{T} \in L(\mathbb{R}^3,\mathbb{R}^3)$. In this case, it easily follows from the definition that $$\begin{split} \epsilon_{ijk}\text{tr}(\mathsf{T}) &= \mathsf{\epsilon}(\mathsf{T}\mathsf{e}_i, \mathsf{e}_j, \mathsf{e}_k) + \mathsf{\epsilon}(\mathsf{e}_i, \mathsf{T}\mathsf{e}_j, \mathsf{e}_k) + \mathsf{\epsilon}(\mathsf{e}_i, \mathsf{e}_j, \mathsf{T}\mathsf{e}_k)\\ &= \sum (\epsilon_{ajk} T_{ai} + \epsilon_{iak} T_{aj} + \epsilon_{ija} T_{ak})\\ &= \epsilon_{ijk}(T_{ii} + T_{jj} + T_{kk}). \end{split}$$ The last expression follows from the fact that $\epsilon_{ijk}$ is non-zero only when $i,j,k$ are distinct. Note also that there is no summation over the repeated indices in the final expression. Choosing $(i,j,k) = (1,2,3)$ the familiar expression for the trace of $[\mathsf{T}]$ is recovered: $$\text{tr}(\mathsf{T}) = \sum T_{ii}.$$

Example

Unlike the determinant, the trace operation is a linear map. To see this, note that given any two linear maps $\mathsf{S},\mathsf{T} \in L(V,V)$ on an $n$-dimensional inner product space $V$ and real numbers $a,b \in \mathbb{R}$, $$\begin{split} \text{tr}(a\mathsf{S} + b\mathsf{T}) &= \sum_i \frac{\epsilon(\mathsf{u}_1, \ldots, (a\mathsf{S} + b\mathsf{T})\mathsf{u}_i, \ldots, \mathsf{u}_n)}{\epsilon(\mathsf{u}_1, \ldots, \mathsf{u}_n)}\\ &= \sum_i a\frac{\epsilon(\mathsf{u}_1, \ldots, \mathsf{S}\mathsf{u}_i, \ldots, \mathsf{u}_n)}{\epsilon(\mathsf{u}_1, \ldots, \mathsf{u}_n)} + b\frac{\epsilon(\mathsf{u}_1, \ldots, \mathsf{T}\mathsf{u}_i, \ldots, \mathsf{u}_n)}{\epsilon(\mathsf{u}_1, \ldots, \mathsf{u}_n)}\\ &= a \text{tr}(\mathsf{S}) + b \text{tr}(\mathsf{T}). \end{split}$$ where $\mathsf{u}_1, \ldots, \mathsf{u}_n \in V$ are any set of $n$ vectors in $V$.

Example

Many identities involving the trace operation are conveniently proved by working with representations with respect to an orthonormal basis of the vector space under consideration. For instance, suppose that $V$ is an $n$-dimensional inner product space, and let $\mathsf{u},\mathsf{v} \in V$ be any two vectors. Let us prove that $$\text{tr}(\mathsf{u} \otimes \mathsf{v}) = \mathsf{u} \cdot \mathsf{v}.$$ Let $(\mathsf{e}_i)$ is an orthonormal basis of $V$. Then, with respect to this basis, the trace of the linear map $\mathsf{u} \otimes \mathsf{v} \in L(V,V)$ can be written as $$\text{tr}(\mathsf{u} \otimes \mathsf{v}) = \sum [\mathsf{u} \otimes \mathsf{v}]_{ii} = \sum u_i v_i = \mathsf{u} \cdot \mathsf{v}.$$ Note that despite the fact that we have used a special basis to prove this fact, the equation remains true in general. This is because the final equation $\text{tr}(\mathsf{u} \otimes \mathsf{v}) = \mathsf{u} \cdot \mathsf{v}$ has no explicit dependence on the basis vectors. This is a useful trick in proving many identities that we will often exploit later.

Example

As another example of the foregoing trick in proving identities involving traces, let us now show that given any two linear maps $\mathsf{S},\mathsf{T} \in L(V,V)$ on a finite dimensional inner product space $V$, $$\text{tr}(\mathsf{S}\mathsf{T}) = \text{tr}(\mathsf{T}\mathsf{S}).$$ If $(\mathsf{e})_i$ is an orthonormal basis of $V$, then $$\begin{split} \text{tr}(\mathsf{ST}) &= \sum S_{ij}T_{ji},\\ \text{tr}(\mathsf{TS}) &= \sum T_{ij}S_{ji} = \sum S_{ij}T_{ji}. \end{split}$$ The identity is thus proved.

It is possible to introduce an inner product in the space $L(V,W)$ of all linear maps from $V$ into $W$ as follows. Given $\mathsf{S},\mathsf{T} \in L(V,W)$, the inner product $\cdot:L(V,W) \times L(V,W) \to \mathbb{R}$ is defined as follows: for any $\mathsf{S},\mathsf{T} \in L(V,W)$, $$\mathsf{S} \cdot \mathsf{T} = \text{tr}(\mathsf{S}\mathsf{T}^T).$$ It is left as an easy exercise to verify that this is indeed an inner product on $L(V,W)$. Note also that $\text{tr}(\mathsf{S}\mathsf{T}^T) = \text{tr}(\mathsf{S}^T\mathsf{T})$.

Example

As an illustration of the inner product just introduced, let us prove a very useful way of rewriting the expression for the trace of a linear map $\mathsf{T} \in L(V,V)$ on a finite dimensional inner product space $V$. If $\mathsf{I} \in L(V,V)$ denotes the identity map, then $$\text{tr}(\mathsf{T}) = \mathsf{I} \cdot \mathsf{T}.$$ To see this, choose an orthonormal basis $(\mathsf{e})_i$ of $V$. With respect to this basis, $$\mathsf{I} \cdot \mathsf{T} = \sum \delta_{ij} T_{ij} = \sum T_{ii} = \text{tr}(\mathsf{T}).$$ It is a good exercise to prove this using a general basis $(\mathsf{g})_i$ of $V$.

Example

Given linear maps $\mathsf{R},\mathsf{S},\mathsf{T} \in L(V,V)$, we have the following identities: $$\mathsf{R} \cdot \mathsf{ST} = \mathsf{S}^T\mathsf{R} \cdot \mathsf{T} = \mathsf{R}\mathsf{T}^T\cdot\mathsf{S}.$$ To see why, note that $$\begin{split} \mathsf{R} \cdot \mathsf{ST} &= \text{tr}(\mathsf{R}^T\mathsf{ST})\\ &= \text{tr}(\mathsf{T}^T\mathsf{S}^T\mathsf{R})\\ &= \mathsf{T} \cdot \mathsf{S}^T\mathsf{R}\\ &= \mathsf{S}^T\mathsf{R} \cdot \mathsf{T}. \end{split}$$ In proving this, we have used the fact that trace of the transpose of a linear map is just the trace of that linear map - a fact that is easily checked. The other identity is similarly proved by noting that $$\begin{split} \mathsf{R} \cdot \mathsf{ST} &= \text{tr}(\mathsf{R}(\mathsf{ST})^T)\\ &= \text{tr}(\mathsf{R}\mathsf{T}^T\mathsf{S}^T)\\ &= \mathsf{R}\mathsf{T}^T \cdot \mathsf{S}. \end{split}$$

Special groups of linear maps

To conclude the current introductory discussion on tensor algebra, a few important class of linear maps are considered now. Throughout this discussion, $V$ stands for an inner product space of dimension $n$, and attention is focused on certain special subsets of $L(V,V)$.

Groups

It is useful at this juncture to introduce the notion of a group. A set $G$ is said to be a group if there exists a map $\odot:G \times G \to G$ that satisfies the following properties:

1. Associativity of group operation: for any $f, g, h \in G$, $f \odot (g \odot h) = (f \odot g) \odot h$,

2. Existence of group identity: there exists $e \in G$, called the group identity, such that for a $g \odot e = e \odot g = g$,

3. Existence of inverse: for every $g \in G$, there exists $g^{-1} \in G$, called the inverse of $g$, such that $g \odot g^{-1} = g^{-1} \odot g = e$.

If it is further the case that $f \odot g = g \odot f$ for every $f,g \in G$, then $G$ is said to be a commutative group, or an Abelian group. A subset $H \subseteq G$ of $G$ is said to be a sub-group of $G$ if $H$ is a group by itself, with respect to the same group operation.

As a simple example, note that given any vector space $V$, $V$ itself forms a commutative group with $+:V \times V \to V$ as the group operation: for any $\mathsf{u}, \mathsf{v} \in V$, $\mathsf{u} \odot \mathsf{v} = \mathsf{u} + \mathsf{v}$. In this case $\mathsf{0} \in V$ is the group identity, and for any $\mathsf{v} \in V$, $-\mathsf{v}$ is the inverse of $\mathsf{v}$.

As another example more relevant to the current discussion, consider the set $M_n$ consisting of all invertible $n \times n$ matrices matrices, and define the binary map $\odot:M_n \times M_n \to M_n$ as follows: given $A,B \in M_n$, $A \odot B = AB$. Here, $AB$ denotes the familiar matrix multiplication of the matrices $A$ and $B$. It is easy to check that $M_n$ is a group with respect to this binary operation. Note that $M_n$ is not a commutative group.

Additive groups of linear maps

A linear map $\mathsf{T} \in L(V,V)$ is said to be symmetric if $\mathsf{T}^T = \mathsf{T}$, and is said to be skew-symmetric if $\mathsf{T}^T = -\mathsf{T}$. To see the connection with the elementary definitions of symmetry and skew-symmetry, let $(\mathsf{g}_i)$ be an orthonormal basis of $V$. If $\mathsf{T}$ is symmetric, it follows that $$T^T_{ij} = \mathsf{g}_i \cdot \mathsf{T}^T\mathsf{g}_j = \mathsf{T}\mathsf{g}_i \cdot \mathsf{g}_j = T_{ji}.$$ It follows from a similar argument that if $\mathsf{T}$ is skew-symmetric, then $T^{T}_{ij} = -T_{ji}$.

The set of all symmetric linear maps on $V$ is called the symmetric linear group on $V$, and written $\text{sym}(V)$: $$\text{sym}(V) = \{ \mathsf{T} \in L(V,V) \,|\, \mathsf{T}^T = \mathsf{T} \}.$$ The skew-symmetric linear group on $V$, written $\text{skw}(V)$, is similarly defined as $$\text{skw}(V) = \{ \mathsf{T} \in L(V,V) \,|\, \mathsf{T}^T = -\mathsf{T} \}.$$ The group operation for both $\text{sym}(V)$ and $\text{skw}(V)$ is the addition of linear maps. Note that both $\text{sym}(V)$ and $\text{skw}(V)$ are sub-groups of $L(V,V)$ with respect to the group operation being the vector addition in $L(V,V)$.

Remark

There is an important relationship between skew-symmetric linear maps and the cross product, in the context of the three dimensional Euclidean space $\mathbb{R}^3$. Given any $\mathsf{v} \in \mathbb{R}^3$, associate with it the skew-symmetric linear map $\check{\mathsf{v}} \in \text{skw}(\mathbb{R}^3)$, defined as follows: for any $\mathsf{u} \in \mathbb{R}^3$, $$\check{\mathsf{v}}\mathsf{u} = \mathsf{v} \times \mathsf{u}.$$ It is straightforward to check that $\check{\mathsf{v}}$ is indeed a linear map. To verify that it is skew symmetric, note that for any $\mathsf{u}, \mathsf{w} \in \mathbb{R}^3$, $$\begin{split} \check{\mathsf{v}}^T\mathsf{u} \cdot \mathsf{w} &= \mathsf{u} \cdot \check{\mathsf{v}}\mathsf{w}\\ &= \mathsf{u} \cdot (\mathsf{v} \times \mathsf{w})\\ &= \mathsf{\epsilon}(\mathsf{u}, \mathsf{v}, \mathsf{w})\\ &= -\mathsf{\epsilon}(\mathsf{w}, \mathsf{v}, \mathsf{u})\\ &= -\mathsf{w} \cdot (\mathsf{v} \times \mathsf{u})\\ &= -\check{\mathsf{v}}\mathsf{u} \cdot \mathsf{w}. \end{split}$$ Since this is true for any $\mathsf{u}, \mathsf{w} \in \mathbb{R}^3$, it follows that $\check{\mathsf{v}}^T = -\check{\mathsf{v}}$. The vector $\mathsf{v}$ is called the axial vector of the skew-symmetric linear map $\check{\mathsf{v}}$. In terms of the standard basis of $\mathbb{R}^3$, it can be verified using a simple calculation that, for any $\mathsf{v} \in \mathbb{R}^3$, $$\check{v}_{ij} = \epsilon_{ijk}v_k, \qquad v_i = -\frac{1}{2}\epsilon_{ijk}\check{v}_{jk}.$$ Using matrix notation, the foregoing equations can be written as follows: $$= \begin{bmatrix}v_1\\ v_2\\ v_3\end{bmatrix}, \qquad [\check{\mathsf{v}}] = \begin{bmatrix} 0 & -v_3 & v_2\\ v_3 & 0 & -v_1\\ -v_2 & v_1 & 0\end{bmatrix}.$$ Note that the matrix representation of the skew-symmetric linear map $\check{\mathsf{v}}$ is also skew-symmetric, as expected.

A linear map $\mathsf{T} \in L(V,V)$ is said to be positive definite if it is true that for any $\mathsf{v} \in V$, $\mathsf{v} \cdot \mathsf{T}\mathsf{v} \ge 0$, and $\mathsf{v} \cdot \mathsf{T}\mathsf{v} = \mathsf{0}$ iff $\mathsf{v} = 0$. An especially important sub-group of $L(V,V)$ is the group $\text{sym}_+(V)$, defined as $$\text{sym}_+(V) = \{ \mathsf{T} \in L(V,V) \,|\, \mathsf{T} \text{ is symmetric and positive-definite}\},$$ of all symmetric and positive definite linear maps on $V$. Note that the group operation here is again addition in $L(V,V)$. This sub-group will turn out to be very useful in the study of continuum mechanics.

Mutliplicative groups of linear maps

The general linear group on $V$, written $\text{GL}(V)$, is the set of all linear maps on $V$ with non-zero determinant: $$\text{GL}(V) = \{\mathsf{T} \in L(V,V) \,|\, \text{det}(\mathsf{T}) \neq 0\}.$$ The group operation, in this case, is the product of linear maps: given $\mathsf{S},\mathsf{T} \in \text{GL}(V)$, $\mathsf{S} \odot \mathsf{T} = \mathsf{S}\mathsf{T}$. It is easy to check that $\text{GL}(V)$ is indeed a group: indeed, given any $\mathsf{S},\mathsf{T} \in \text{GL}(V)$, note that $\text{det}(\mathsf{S}\mathsf{T}) = \text{det}(\mathsf{S})\text(\mathsf{T}) \neq 0$. Note that if $\mathsf{T} \in \text{GL}(V)$, then $\mathsf{T}$ is invertible: this means that there exists a linear map $\mathsf{T}^{-1} \in \text{GL}(V)$, called the inverse of $\mathsf{T}$, such that $$\mathsf{T} \mathsf{T}^{-1} = \mathsf{T}^{-1}\mathsf{T} = \mathsf{I},$$ where $\mathsf{I} \in L(V,V)$ is the identity map in $V$.

A few important sub-groups of $\text{GL}(V)$ are discussed next. The set of all linear maps with determinant $1$ is called the special linear group on $V$, and written $\text{SL}(V)$: $$\text{SL}(V) = \{ \mathsf{T} \in \text{GL}(V) \,|\, \text{det}(\mathsf{T}) = 1\}.$$ A linear map $\mathsf{T} \in \text{GL}(V)$ is said to be orthogonal if the inverse of $\mathsf{T}$ is the transpose of $\mathsf{T}$. The set of all orthogonal linear maps, written $\text{O}(V)$ is called the orthogonal group of $V$: $$\text{O}(V) = \{ \mathsf{T} \in \text{GL}(V) \,|\, \mathsf{T}^{-1} = \mathsf{T}^T \}.$$ The determinant of an orthogonal linear map $\mathsf{T} \in \text{O}(V)$ is $\pm 1$. To see this, note that $1 = \text{det}(I) = \text{det}(\mathsf{T}^{-1}\mathsf{T}) = \text{det}(\mathsf{T}^T\mathsf{T}) = (\text{det}(\mathsf{T}))^2$. In deriving this result, use has been made of the fact that $\text{det}(\mathsf{T}^T) = \text{det}(\mathsf{T})$ for any $\mathsf{T} \in L(V,V)$. The set of all orthogonal maps with determinant equal to $1$ is called the special orthogonal group on $V$, written $\text{SO}(V)$: $$\text{SO}(V) = \{\mathsf{T} \in \text{GL}(V) \,|\, \mathsf{T}^{-1} = \mathsf{T}^T, \; \text{det}(\mathsf{T}) = 1\}.$$ It is straightforward to check that $\text{SO}(V) \subset \text{O}(V) \subset \text{GL}(V)$.

Remark

In the special case when $V = \mathbb{R}^n$, it is customary to denote the various groups discussed above as $\text{GL}(n)$, $\text{SL}(n)$, $\text{O}(n)$, and $\text{SO}(n)$.

Eigenvalues and Eigenvectors of linear maps

To conclude the discussion of the algebraic preliminaries, a simplified introduction to the important ideas of eigenvalues and eigenvectors of a linear map are now discussed. Throughout this section, it is assumed that $V$ is a finite dimensional inner product space of dimension $n$.

Definition

Given a linear map $\mathsf{T}:V \to V$, a vector $\mathsf{v} \in V$ is called an eigenvector of $\mathsf{T}$ with respect to the eigenvalue $\mathsf{a} \in \mathbb{R}$ if $$\mathsf{A}\mathsf{v} = a\mathsf{v}.$$ Note that the zero vector $\mathsf{0} \in V$ trivially satisfies this equation. It is implicitly assumed that this trivial eigenvector is excluded from the discussion.

Remark

It is noted that the eigenvalues can, in general be complex, and the vector space $V$ is typically taken to be a complex vector space. Since only a special class of linear maps which admit only real eigenvalues are considered in this section, the more general theory is not developed here.

The eigenvectors and eigenvalues of $\mathsf{T} \in L(V,V)$ are readily computing by noting that the equation $\mathsf{T}\mathsf{v} = a\mathsf{v}$ can be written as $(\mathsf{T} - a\mathsf{I})\mathsf{v} = \mathsf{0}$, where $\mathsf{I} \in L(V,V)$ is the identity map on $V$. The condition that this equation admits non-trivial solutions immediately yields the following condition: $$\text{det}(\mathsf{T} - a\mathsf{I}) = 0.$$ This is a polynomial equation of order $n$ in $a$ that can be solved to obtain the $n$ eigenvalues $(a_i)$ of $\mathsf{T}$. Using these eigenvalues in the equations $\mathsf{A}\mathsf{v}_i = a_i\mathsf{v}_i$, where $i = 1,\ldots,n$, and solving them results in the $n$ eigenvectors of $\mathsf{T}$.

Symmetric and positive definite linear maps

Of special interest here are linear maps that are both symmetric and positive definite; thus, the discussion to follows focuses on linear maps $\mathsf{T} \in \text{sym}_+(V)$. Given any $\mathsf{T} \in \text{sym}_+(V)$, note that if $\mathsf{u}, \mathsf{v} \in V$ are eigenvectors of $\mathsf{T}$ corresponding to the eigenvalues $a,b \in \mathbb{R}$, respectively, it follows that $$b \mathsf{u} \cdot \mathsf{v} = \mathsf{u} \cdot \mathsf{T}\mathsf{v} = \mathsf{T}^T\mathsf{u} \cdot \mathsf{v} = \mathsf{T}\mathsf{u} \cdot \mathsf{v} = a \mathsf{u} \cdot \mathsf{v}.$$ Thus, $(b - a)\mathsf{u} \cdot \mathsf{v} = 0$. This immediately shows that $a \neq b$ then $\mathsf{u} \cdot \mathsf{v} = 0$. In words, this expresses the fact that the eigenvectors of a symmetric and positive definite map corresponding to distinct eigenvalues are mutually orthogonal. Further, it follows from the positive definiteness of $\mathsf{T}$ that $$b \lVert \mathsf{v} \rVert^2 = \mathsf{v} \cdot \mathsf{T}\mathsf{v} \ge 0 \quad\Rightarrow\quad b \ge 0.$$ Thus, the eigenvectors of a symmetric and positive definite linear map are real and positive.

If $(\mathsf{v}_i)_{i=1}^n$ are the eigenvectors of $\mathsf{T}$ corresponding to the eigenvalues $(a_i)_{i=1}^n$, it can be shown that the eigenvectors $(\mathsf{v}_i)$ of $T$ constitute a basis of $V$. Further, it can be checked using direct substitution that $\mathsf{T}$ admits the representation $$\mathsf{T} = \sum a_i \mathsf{v}_i \otimes \mathsf{v}_i.$$ This equation, which expresses the linear map $\mathsf{T}$ in terms of the basis of $V$ formed by the eigenvectors of $\mathsf{T}$ is called the spectral representation of $\mathsf{T}$.

In the special case of symmetric and positive definite linear maps, the fact that its eigenvectors are positive can be used to defined various functions of linear maps. Specifically, if $f:\mathbb{R}_+ \to \mathbb{R}$ is a function that takes a positive real number and returns a real number, it can be extended to the linear space $\text{sym}_+(V)$ as follows: for any $\mathsf{T} \in \text{sym}_+(V)$, define $$f(\mathsf{T}) = \sum f(a_i) \mathsf{v}_i \otimes \mathsf{v}_i.$$ As important examples of such functions, the logarithm of a symmetric and positive definite linear map $\mathsf{T}$ is defined as $$\log \mathsf{T} = \sum \log a_i \mathsf{v}_i \otimes \mathsf{v}_i.$$ Similarly, the square root of $\mathsf{T}$ is defined as $$\sqrt{\mathsf{T}} = \sum \sqrt{a_i} \mathsf{v}_i \otimes \mathsf{v}_i.$$ It can be checked with a simple calculation that $\sqrt{\mathsf{T}}\sqrt{\mathsf{T}} = \mathsf{T}$, as expected.

Invariants of a linear map

Given a linear map $\mathsf{T} \in L(V,V)$, the equation $\text{det}(\mathsf{T} - a\mathsf{I}) = \mathsf{0}$ is called the characteristic equation of $\mathsf{T}$. When expanded, the characteristic equation takes the form $$(-a)^n + I_1(-a)^{n-1} + \ldots + I_n = 0,$$ where the set of constants $\{I_i\}_{i=1}^n$ are called the invariants of the linear map $\mathsf{T}$. The reason for calling them invariants is that their values do not depend on the choice of any basis for $V$, and are hence invariant with respect to the choice of basis.

The Cayley-Hamilton theorem states that the linear map $\mathsf{T}$ also satisfies the characteristic equation: $$(-\mathsf{T})^n + I_1(-\mathsf{T})^{n-1} + \ldots + I_n\mathsf{I} = \mathsf{0}.$$ Here $\mathsf{T}^k$ is to be understood $\mathsf{T}\mathsf{T}\ldots\mathsf{T}$ ($k$ factors).

It is of interest to consider the case when $V$ is a three dimensional vector space. In this case, the invariants of an invertible linear map $\mathsf{T} \in \text{GL}(V)$ can be shown to be as follows: for any $\mathsf{u},\mathsf{v},\mathsf{w} \in V$, $$\begin{split} I_1 &= \text{tr}(\mathsf{T}) = \frac{\mathsf\epsilon(\mathsf{T}\mathsf{u},\mathsf{v},\mathsf{w})}{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w})} + \frac{\mathsf\epsilon(\mathsf{u},\mathsf{T}\mathsf{v},\mathsf{w})}{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w})} + \frac{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{T}\mathsf{w})}{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w})},\\ I_2 &= \text{tr}(\mathsf{T}^{-1})\text{det}(\mathsf{T}) = \frac{\mathsf\epsilon(\mathsf{T}\mathsf{u},\mathsf{T}\mathsf{v},\mathsf{w})}{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w})} + \frac{\mathsf\epsilon(\mathsf{u},\mathsf{T}\mathsf{v},\mathsf{T}\mathsf{w})}{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w})} + \frac{\mathsf\epsilon(\mathsf{T}\mathsf{u},\mathsf{v},\mathsf{T}\mathsf{w})}{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w})},\\ I_3 &= \text{det}(\mathsf{T}) = \frac{\mathsf\epsilon(\mathsf{T}\mathsf{u},\mathsf{T}\mathsf{v},\mathsf{T}\mathsf{w})}{\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w})}. \end{split}$$ An elegant means to prove this is by applying the definition of the determinant, $\text{det}(\mathsf{T})\mathsf\epsilon(\mathsf{u},\mathsf{v},\mathsf{w}) = \mathsf\epsilon(\mathsf{T}\mathsf{u}, \mathsf{T}\mathsf{v},\mathsf{T}\mathsf{w})$ to evaluate the determinant in the characteristic equation of $\mathsf{T}$: $\text{det}(\mathsf{T} - a\mathsf{I})$.