MATLAB TUTORIAL, part 2.1: Diagonalization

Diagonalization

We show how to define a function of a square matrix using a diagonalization procedure. This method is applicable only for diagonalizable square matrices, and is not suitable for defective matrices. Recall that a matrix A is called diagonalizable if there exists a nonsingular matrix S such that $${\bf S}^{-1} {\bf A} {\bf S} = {\bf \Lambda} ,$$ a diagonal matrix. In other words, the matrix A is similar to a diagonal matrix. An $$n \times n$$ square matrix is diagonalizable if and only if there exist n linearly independent eigenvectors, so geometrical multiplicity of each eigenvalue is the same as its algebraic multiplicity. Then the matrix S can be built from eigenvectors of A, column by column.

Let A be a square $$n \times n$$ diagonalizable matrix, and let $${\bf \Lambda}$$ be the corresponding diagonal matrix of its eigenvalues:

${\bf \Lambda} = \begin{bmatrix} \lambda_1 & 0 & 0 & \cdots & 0 \\ 0&\lambda_2 & 0& \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0&0&0& \cdots & \lambda_n \end{bmatrix} ,$

where $$\lambda_1 , \lambda_2 , \ldots , \lambda_n$$ are eigenvalues (that may be equal) of the matrix A.

Let $${\bf x}_1 , {\bf x}_2 , \ldots , {\bf x}_n$$ be linearly independent eigenvectors, corresponding to the eigenvalues $$\lambda_1 , \lambda_2 , \ldots , \lambda_n .$$ We build the nonsingular matrix S from these eigenvectors (every column is an eigenvector):

${\bf S} = \begin{bmatrix} {\bf x}_1 & {\bf x}_2 & {\bf x}_3 & \cdots & {\bf x}_n \end{bmatrix} .$
For any reasonable (we do not specify this word, it is sufficient to be smooth) function defined on the spectrum (set of all eigenvalues) of the diagonalizable matrix A, we define the function of this matrix by the formula:
$f \left( {\bf A} \right) = {\bf S} f\left( {\bf \Lambda} \right) {\bf S}^{-1} , \qquad \mbox{where } \quad f\left( {\bf \Lambda} \right) = \begin{bmatrix} f(\lambda_1 ) & 0 & 0 & \cdots & 0 \\ 0 & f(\lambda_2 ) & 0 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0&0&0& \cdots & f(\lambda_n ) \end{bmatrix} .$

Example: Consider the $$3 \times 3$$ matrix $${\bf A} = \begin{bmatrix} 1&4&16 \\ 18&20&4 \\ -12&-14&-7 \end{bmatrix}$$ that has three distinct eigenvalues that we identify with Mathematica
A = {{1,4,16},{18,20,4},{-12,-14,-7}}
Eigenvalues[A]
Out[2]= 9, 4, 1
Eigenvectors[A]
Out[3]= {{1, -2, 1}, {4, -5, 2}, {4, -4, 1}}
Using eigenvectors, we build the transition matrix S of its eigenvectors:
${\bf S} = \begin{bmatrix} 1&4&4 \\ -2&-5&-4 \\ 1&2&1 \end{bmatrix} , \quad\mbox{with} \quad {\bf S}^{-1} = \begin{bmatrix} -3&-4&-4 \\ 2&3&4 \\ -1&-2&-3 \end{bmatrix} .$

Then we are ready to construct eight (it is 23 roots because each square root of an eigenvalue has two values; for instance, $$\sqrt{9} = \pm 3$$ ) square roots of this positive definite matrix:

$\sqrt{\bf A} = {\bf S} \sqrt{\Lambda} {\bf S}^{-1} = \begin{bmatrix} 1&4&4 \\ -2&-5&-4 \\ 1&2&1 \end{bmatrix} \begin{bmatrix} \pm 3&0&0 \\ 0&\pm 2&0 \\ 0&0&\pm 1 \end{bmatrix} \begin{bmatrix} -3&-4&-4 \\ 2&3&4 \\ -1&-2&-3 \end{bmatrix} ,$
with appropriate choice of roots on the diagonal. In particular,
$\sqrt{\bf A} = \begin{bmatrix} 3&4&8 \\ 2&2&-4 \\ -2&-2&1 \end{bmatrix} , \quad \begin{bmatrix} 21&28&32 \\ -34&-46&-52 \\ 16&22&25 \end{bmatrix} , \quad \begin{bmatrix} -11&-20&-32 \\ 6&14&28 \\ 0&-2&-7 \end{bmatrix} , \quad \begin{bmatrix} 29&44&56 \\ -42&-62&-76 \\ 18&26&31 \end{bmatrix} .$
We check with Mathematica for specific roots of eigenvalues: 3, 2, and 1. However, we can take any combination of these roots using $$\pm 3, \pm 2, \pm 1$$ next time.
S = Transpose[Eigenvectors[A]]
square = {{3, 0, 0}, {0, 2, 0}, {0, 0, 1}}
S.square.Inverse[S]
Out[7]= {{3, 4, 8}, {2, 2, -4}, {-2, -2, 1}}
Now we build other matrix functions.
$e^{{\bf A}\,t} = {\bf S} \begin{bmatrix} e^{9t}&0&0 \\ 0&e^{4t}&0 \\ 0&0&e^t \end{bmatrix} {\bf S}^{-1} = e^{9t} \begin{bmatrix} -3&-4&-4 \\ 6&8&8 \\ -3&-4&-4 \end{bmatrix} + e^{4t} \begin{bmatrix} 8&12&16 \\ -10&-15&-20 \\ 4&6&8 \end{bmatrix} + e^t \begin{bmatrix} -4&-8&-12 \\ 4&8&12 \\ -1&-2& -3 \end{bmatrix} ,$
which we check with Mathematica standard command:
A = {{1, 4, 16}, {18, 20, 4}, {-12, -14, -7}}
MatrixExp[A*t]
Recall that the exponential matrix function is the unique solution to the following initial value problem:
$\frac{\text d}{{\text d}t}\, {\bf \Phi} = {\bf A}\,{\bf \Phi}(t) = {\bf \Phi}(t)\, {\bf A} , \quad {\bf \Phi}(0) = {\bf I},$
where I is the identity square matrix. Two other important matrix functions depending on time variable t:
\begin{eqnarray*} \frac{\sin \left( \sqrt{\bf A}\,t \right)}{\sqrt{\bf A}} &=& {\bf S} \begin{bmatrix} \frac{\sin 3t}{3}&0&0 \\ 0&\frac{\sin 2t}{2}&0 \\ 0&0&\sin t \end{bmatrix} {\bf S}^{-1} = \sin 3t \begin{bmatrix} -1&-\frac{4}{3} &-\frac{4}{3} \\ 2&\frac{8}{3}&\frac{8}{3} \\ -1&-\frac{4}{3}&-\frac{4}{3} \end{bmatrix} + \sin 2t \begin{bmatrix} 4&6&8 \\ -5&-\frac{15}{2}&-10 \\ 2&3&4 \end{bmatrix} + \sin t \begin{bmatrix} -4&-8&-12 \\ 4&8&12 \\ -1&-2&-3 \end{bmatrix} , \\ \cos \left( \sqrt{\bf A}\,t \right) &=& {\bf S} \begin{bmatrix} \cos 3t&0&0 \\ 0&\cos 2t&0 \\ 0&0&\cos t \end{bmatrix} {\bf S}^{-1} = \cos 3t \begin{bmatrix} -3&-4&-4 \\ 6&8&8 \\ -3&-4&-4 \end{bmatrix} + \cos 2t \begin{bmatrix} 8&12&-12 \\ -10&-15&-20 \\ 4&6&8 \end{bmatrix} + \cos t \begin{bmatrix} -4&-8&-12 \\ 4&8&12 \\ -1&-2&-3 \end{bmatrix} . \end{eqnarray*}
S = Transpose[Eigenvectors[A]]
S.{{Sin[3*t]/3, 0, 0}, {0, Sin[2*t]/2, 0}, {0, 0, Sin[t]}}.Inverse[S]
S.{{Cos[3*t], 0, 0}, {0, Cos[2*t], 0}, {0, 0, Cos[t]}}.Inverse[S]
These two matrix functions are unique solutions of the following initial value problems:
$\ddot{\bf \Phi} + {\bf A}\, {\bf \Phi} = {\bf 0} , \quad {\bf \Phi}(0) = {\bf 0} , \quad \dot{\bf \Phi}(0) = {\bf I} \qquad \mbox{for} \quad {\bf \Phi}(t) = \frac{\sin \left( \sqrt{\bf A}\,t \right)}{\sqrt{\bf A}} ,$
and
$\ddot{\bf \Psi} + {\bf A}\, {\bf \Psi} = {\bf 0} , \quad {\bf \Psi}(0) = {\bf I} , \quad \dot{\bf \Psi}(0) = {\bf 0} \qquad \mbox{for} \quad {\bf \Psi}(t) = \cos \left( \sqrt{\bf A}\,t \right) .$
Example: Consider the $$3 \times 3$$ matrix $${\bf A} = \begin{bmatrix} -20&-42&-21 \\ 6&13&6 \\ 12&24&13 \end{bmatrix}$$ that has two distinct eigenvalues

A = {{-20, -42, -21}, {6, 13, 6}, {12, 24, 13}}
Eigenvalues[A]
Out[2]= 4, 1, 1
Eigenvectors[A]
Out[3]= {{ -7, 2, 4 }, {-1, 0, 1 }, {-2, 1, 0 }}
Since the double eigenvalue $$\lambda =1$$ has two linearly independent eigenvectors, the given matrix is diagonalizable, and we are able to build the transition matrix of its eigenvectors:
${\bf S} = \begin{bmatrix} -7&-1&-2 \\ 2&0&1 \\ 4&1&0 \end{bmatrix} , \quad\mbox{with} \quad {\bf S}^{-1} = \begin{bmatrix} 1&2&1 \\ -4&-8&-3 \\ -2&-3&-2 \end{bmatrix} .$
For three functions, $$f(\lambda ) = e^{\lambda \,t} , \quad \Phi (\lambda ) = \frac{\sin \left( \sqrt{\lambda} \,t \right)}{\sqrt{\lambda}} , \quad \Psi (\lambda ) = \cos \left( \sqrt{\lambda} \,t \right)$$ we construct the corresponding matrix-functions:

\begin{align*} f({\bf A}) &= {\bf S} e^{{\bf \Lambda}\,t} {\bf S}^{-1} = e^{2t} \begin{bmatrix} -7 & -14 & -7 \\ 2&4&2 \\ 4&8&4 \end{bmatrix} + e^t \begin{bmatrix} 8&14&7 \\ -2&-3&-2 \\ -4&-8&-3 \end{bmatrix} , \\ {\bf \Phi} ({\bf A}) &= {\bf S} \frac{\sin \left( \sqrt{\bf \Lambda} \,t \right)}{\sqrt{\bf \Lambda}} {\bf S}^{-1} = \sin 2t \begin{bmatrix} -7/2 & -7 & -7/2 \\ 1&2&1 \\ 2&4&2 \end{bmatrix} + \sin t \begin{bmatrix} 8&14&7 \\ -2&-3&-2 \\ -4&-8&-3 \end{bmatrix} , \\ {\bf \Psi} ({\bf A}) &= {\bf S} \cos \left( {\bf \Lambda}\,t \right) {\bf S}^{-1} = \cos 2t \begin{bmatrix} -7 & -14 & -7 \\ 2&4&2 \\ 4&8&4 \end{bmatrix} + \cos t \begin{bmatrix} 8&14&7 \\ -2&-3&-2 \\ -4&-8&-3 \end{bmatrix} . \end{align*}

These matrix functions are unique solutions of the following initial value problems:
$\frac{\text d}{{\text d}t}\,e^{{\bf A}\,t} = {\bf A}\,e^{{\bf A}\,t} , \qquad \lim_{t\to 0} \,e^{{\bf A}\,t} = {\bf I} , \quad \mbox{where } {\bf I} \mbox{ is the identity matrix};$
$\frac{{\text d}^2}{{\text d}t^2}\,{\bf \Phi} ({\bf A}) + {\bf A}\,{\bf \Phi} ({\bf A}) = {\bf 0} , \qquad \lim_{t\to 0} \,{\bf \Phi} ({\bf A}) = {\bf 0} , \quad \quad \lim_{t\to 0} \,\dot{\bf \Phi} ({\bf A}) = {\bf I} , \quad \mbox{where } {\bf I} \mbox{ is the identity matrix};$
$\frac{{\text d}^2}{{\text d}t^2}\,{\bf \Psi} ({\bf A}) + {\bf A}\,{\bf \Psi} ({\bf A}) = {\bf 0} , \qquad \lim_{t\to 0} \,{\bf \Psi} ({\bf A}) = {\bf I} , \quad \quad \lim_{t\to 0} \,\dot{\bf \Psi} ({\bf A}) = {\bf 0} .$
Example: Consider the $$3 \times 3$$ matrix $${\bf A} = \begin{bmatrix} 1 &2&3 \\ 2 &3&4 \\ 2&-6&-4 \end{bmatrix}$$ that has two complex conjugate eigenvalues $$\lambda = 1 \pm 2{\bf j}$$ and one real eigenvalue $$\lambda = -2 .$$ Mathematica confirms:
A = {{1, 2, 3}, {2, 3, 4}, {2, -6, -4}}
Eigenvalues[A]
Out[2]= {1 + 2 I, 1 - 2 I, -2}
Eigenvectors[A]
Out[3]= {{-1 - I, -2 - I, 2}, {-1 + I, -2 + I, 2}, {-7, -6, 11}}
We build the transition matrix of its eigenvectors:
${\bf S} = \begin{bmatrix} -1-{\bf j} & -1+{\bf j} &-7 \\ -2-{\bf j} & -2+{\bf j} &-6 \\ 2&2&1 \end{bmatrix} , \quad \mbox{with} \quad {\bf S}^{-1} = \frac{1}{6} \begin{bmatrix} 1 - 10{\bf j} & -1 + 13{\bf j} & 1 + 8{\bf j} \\ 1 +10 {\bf j} & -1 -13{\bf j} & 1 -8{\bf j} \\ -4 & 4 & 2 \end{bmatrix} .$
Now we are ready to define a function of the given square matrix. For example, if $$f(\lambda ) = e^{\lambda \, t} ,$$ we obtain the corresponding exponential matrix:
\begin{align*} e^{{\bf A}\,t} &= {\bf S} \begin{bmatrix} e^{(1+2{\bf j})\,t} & 0&0 \\ 0& e^{(1-2{\bf j})\,t} & 0 \\ 0&0&e^{-2t} \end{bmatrix} {\bf S}^{-1} \\ &= \begin{bmatrix} -1-{\bf j} & -1+{\bf j} &-7 \\ -2-{\bf j} & -2+{\bf j} &-6 \\ 2&2&1 \end{bmatrix} \, \begin{bmatrix} e^{t} \left( \cos 2t + {\bf j}\,\sin 2t \right) & 0&0 \\ 0& e^{t} \left( \cos 2t - {\bf j}\,\sin 2t \right) & 0 \\ 0&0&e^{-2t} \end{bmatrix} \, \frac{1}{6} \begin{bmatrix} 1 - 10{\bf j} & -1 + 13{\bf j} & 1 + 8{\bf j} \\ 1 +10 {\bf j} & -1 -13{\bf j} & 1 -8{\bf j} \\ -4 & 4 & 2 \end{bmatrix} \\ &= \frac{1}{3} \, e^{-2t} \begin{bmatrix} 14 & -14& -7 \\ 12&-12& -6 \\ -2&2&1 \end{bmatrix} + \frac{1}{3} \, e^{t} \,\cos 2t \begin{bmatrix} -11&14&7 \\ -12&15&6 \\ 2&-2&2 \end{bmatrix} + \frac{1}{3} \, e^{t} \,\sin 2t \begin{bmatrix} -9&12&9 \\ -19&25&17 \\ 20&-26&-16 \end{bmatrix} . \end{align*}
Here we use Euler's formula: $$e^{a+b{\bf j}} = e^a \left( \cos b + {\bf j} \sin b \right) .$$ Mathematica confirms
S = {{-1-I, -1+I, -7}, {-2-I, -2+I, -6}, {2, 2, 1}}
diag = {{Exp[t]*(Cos[2*t] + I*Sin[2*t]), 0, 0} , {0, Exp[t]*(Cos[2*t] - I*Sin[2*t]), 0}, {0, 0, Exp[-2*t]}}
FullSimplify[S.diag.Inverse[S]*3]
The matrix function $$e^{{\bf A}\,t}$$ is the unique solution of the following matrix initial value problem:
$\frac{\text d}{{\text d}t}\,e^{{\bf A}\,t} = {\bf A}\,e^{{\bf A}\,t} , \qquad \lim_{t\to 0} \,e^{{\bf A}\,t} = {\bf I} ,$
where I is the identity matrix.    ■


Complete