Skip to content

Commit ca47ffd

Browse files
mhjensenmhjensen
committed
Update review.do.txt
1 parent 5f57d27 commit ca47ffd

File tree

1 file changed

+251
-0
lines changed

1 file changed

+251
-0
lines changed

‎doc/src/week1/review.do.txt‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -716,6 +716,256 @@ which is a $2\times 2 $ matrix while
716716
is a $3\times 3 $ matrix. The last row and column of this last matrix
717717
contain only zeros.
718718

719+
!split
720+
===== Eigenvalue problems, basic definitions =====
721+
!bblock
722+
Let us consider the matrix $\mathbf{A}$ of dimension $n$. The eigenvalues of
723+
$\mathbf{A}$ are defined through the matrix equation
724+
!bt
725+
\[
726+
\mathbf{A}\mathbf{x}^{(\nu)} = \lambda^{(\nu)}\mathbf{x}^{(\nu)},
727+
\]
728+
!et
729+
where $\lambda^{(\nu)}$ are the eigenvalues and $\mathbf{x}^{(\nu)}$ the
730+
corresponding eigenvectors.
731+
Unless otherwise stated, when we use the wording eigenvector we mean the
732+
right eigenvector. The left eigenvalue problem is defined as
733+
!bt
734+
\[
735+
\mathbf{x}^{(\nu)}_L\mathbf{A} = \lambda^{(\nu)}\mathbf{x}^{(\nu)}_L
736+
\]
737+
!et
738+
The above right eigenvector problem is equivalent to a set of $n$ equations with $n$ unknowns
739+
$x_i$.
740+
!eblock
741+
742+
743+
!split
744+
===== Eigenvalue problems, basic definitions =====
745+
!bblock
746+
The eigenvalue problem can be rewritten as
747+
!bt
748+
\[
749+
\left( \mathbf{A}-\lambda^{(\nu)} \mathbf{I} \right) \mathbf{x}^{(\nu)} = 0,
750+
\]
751+
!et
752+
with $\mathbf{I}$ being the unity matrix. This equation provides
753+
a solution to the problem if and only if the determinant
754+
is zero, namely
755+
!bt
756+
\[
757+
\left| \mathbf{A}-\lambda^{(\nu)}\mathbf{I}\right| = 0,
758+
\]
759+
!et
760+
which in turn means that the determinant is a polynomial
761+
of degree $n$ in $\lambda$ and in general we will have
762+
$n$ distinct zeros.
763+
!eblock
764+
765+
766+
!split
767+
===== Eigenvalue problems, basic definitions =====
768+
!bblock
769+
The eigenvalues of a matrix
770+
$\mathbf{A}\in {\mathbb{C}}^{n\times n}$
771+
are thus the $n$ roots of its characteristic polynomial
772+
!bt
773+
\[
774+
P(\lambda) = det(\lambda\mathbf{I}-\mathbf{A}),
775+
\]
776+
!et
777+
or
778+
!bt
779+
\[
780+
P(\lambda)= \prod_{i=1}^{n}\left(\lambda_i-\lambda\right).
781+
\]
782+
!et
783+
The set of these roots is called the spectrum and is denoted as
784+
$\lambda(\mathbf{A})$.
785+
If $\lambda(\mathbf{A})=\left\{\lambda_1,\lambda_2,\dots ,\lambda_n\right\}$ then we have
786+
!bt
787+
\[
788+
det(\mathbf{A})= \lambda_1\lambda_2\dots\lambda_n,
789+
\]
790+
!et
791+
and if we define the trace of $\mathbf{A}$ as
792+
!bt
793+
\[
794+
Tr(\mathbf{A})=\sum_{i=1}^n a_{ii}\]
795+
!et
796+
then
797+
!bt
798+
\[
799+
Tr(\mathbf{A})=\lambda_1+\lambda_2+\dots+\lambda_n.
800+
\]
801+
!et
802+
!eblock
803+
804+
805+
!split
806+
===== Abel-Ruffini Impossibility Theorem =====
807+
!bblock
808+
The *Abel-Ruffini* theorem (also known as Abel's impossibility theorem)
809+
states that there is no general solution in radicals to polynomial equations of degree five or higher.
810+
811+
The content of this theorem is frequently misunderstood. It does not assert that higher-degree polynomial equations are unsolvable.
812+
In fact, if the polynomial has real or complex coefficients, and we allow complex solutions, then every polynomial equation has solutions; this is the fundamental theorem of algebra. Although these solutions cannot always be computed exactly with radicals, they can be computed to any desired degree of accuracy using numerical methods such as the Newton-Raphson method or Laguerre method, and in this way they are no different from solutions to polynomial equations of the second, third, or fourth degrees.
813+
814+
The theorem only concerns the form that such a solution must take. The content of the theorem is
815+
that the solution of a higher-degree equation cannot in all cases be expressed in terms of the polynomial coefficients with a finite number of operations of addition, subtraction, multiplication, division and root extraction. Some polynomials of arbitrary degree, of which the simplest nontrivial example is the monomial equation $ax^n = b$, are always solvable with a radical.
816+
!eblock
817+
818+
819+
!split
820+
===== Abel-Ruffini Impossibility Theorem =====
821+
!bblock
822+
823+
The *Abel-Ruffini* theorem says that there are some fifth-degree equations whose solution cannot be so expressed.
824+
The equation $x^5 - x + 1 = 0$ is an example. Some other fifth degree equations can be solved by radicals,
825+
for example $x^5 - x^4 - x + 1 = 0$. The precise criterion that distinguishes between those equations that can be solved
826+
by radicals and those that cannot was given by Galois and is now part of Galois theory:
827+
a polynomial equation can be solved by radicals if and only if its Galois group is a solvable group.
828+
829+
Today, in the modern algebraic context, we say that second, third and fourth degree polynomial
830+
equations can always be solved by radicals because the symmetric groups $S_2, S_3$ and $S_4$ are solvable groups,
831+
whereas $S_n$ is not solvable for $n \ge 5$.
832+
!eblock
833+
834+
835+
!split
836+
===== Eigenvalue problems, basic definitions =====
837+
!bblock
838+
In the present discussion we assume that our matrix is real and symmetric, that is
839+
$\mathbf{A}\in {\mathbb{R}}^{n\times n}$.
840+
The matrix $\mathbf{A}$ has $n$ eigenvalues
841+
$\lambda_1\dots \lambda_n$ (distinct or not). Let $\mathbf{D}$ be the
842+
diagonal matrix with the eigenvalues on the diagonal
843+
!bt
844+
\[
845+
\mathbf{D}= \left( \begin{array}{ccccccc} \lambda_1 & 0 & 0 & 0 & \dots &0 & 0 \\
846+
0 & \lambda_2 & 0 & 0 & \dots &0 &0 \\
847+
0 & 0 & \lambda_3 & 0 &0 &\dots & 0\\
848+
\dots & \dots & \dots & \dots &\dots &\dots & \dots\\
849+
0 & \dots & \dots & \dots &\dots &\lambda_{n-1} & \\
850+
0 & \dots & \dots & \dots &\dots &0 & \lambda_n
851+
852+
\end{array} \right).
853+
\]
854+
!et
855+
If $\mathbf{A}$ is real and symmetric then there exists a real orthogonal matrix $\mathbf{S}$ such that
856+
!bt
857+
\[
858+
\mathbf{S}^T \mathbf{A}\mathbf{S}= \mathrm{diag}(\lambda_1,\lambda_2,\dots ,\lambda_n),
859+
\]
860+
!et
861+
and for $j=1:n$ we have $\mathbf{A}\mathbf{S}(:,j) = \lambda_j \mathbf{S}(:,j)$.
862+
!eblock
863+
864+
865+
!split
866+
===== Eigenvalue problems, basic definitions =====
867+
!bblock
868+
To obtain the eigenvalues of $\mathbf{A}\in {\mathbb{R}}^{n\times n}$,
869+
the strategy is to
870+
perform a series of similarity transformations on the original
871+
matrix $\mathbf{A}$, in order to reduce it either into a diagonal form as above
872+
or into a tridiagonal form.
873+
874+
We say that a matrix $\mathbf{B}$ is a similarity
875+
transform of $\mathbf{A}$ if
876+
!bt
877+
\[
878+
\mathbf{B}= \mathbf{S}^T \mathbf{A}\mathbf{S}, \hspace{1cm} \mathrm{where} \hspace{1cm} \mathbf{S}^T\mathbf{S}=\mathbf{S}^{-1}\mathbf{S} =\mathbf{I}.
879+
\]
880+
!et
881+
The importance of a similarity transformation lies in the fact that
882+
the resulting matrix has the same
883+
eigenvalues, but the eigenvectors are in general different.
884+
!eblock
885+
886+
887+
!split
888+
===== Eigenvalue problems, basic definitions =====
889+
!bblock
890+
To prove this we
891+
start with the eigenvalue problem and a similarity transformed matrix $\mathbf{B}$.
892+
!bt
893+
\[
894+
\mathbf{A}\mathbf{x}=\lambda\mathbf{x} \hspace{1cm} \mathrm{and}\hspace{1cm}
895+
\mathbf{B}= \mathbf{S}^T \mathbf{A}\mathbf{S}.
896+
\]
897+
!et
898+
We multiply the first equation on the left by $\mathbf{S}^T$ and insert
899+
$\mathbf{S}^{T}\mathbf{S} = \mathbf{I}$ between $\mathbf{A}$ and $\mathbf{x}$. Then we get
900+
!bt
901+
\begin{equation}
902+
(\mathbf{S}^T\mathbf{A}\mathbf{S})(\mathbf{S}^T\mathbf{x})=\lambda\mathbf{S}^T\mathbf{x} ,
903+
\end{equation}
904+
!et
905+
which is the same as
906+
!bt
907+
\[
908+
\mathbf{B} \left ( \mathbf{S}^T\mathbf{x} \right ) = \lambda \left (\mathbf{S}^T\mathbf{x}\right ).
909+
\]
910+
!et
911+
The variable $\lambda$ is an eigenvalue of $\mathbf{B}$ as well, but with
912+
eigenvector $\mathbf{S}^T\mathbf{x}$.
913+
!eblock
914+
915+
916+
!split
917+
===== Eigenvalue problems, basic definitions =====
918+
!bblock
919+
The basic philosophy is to
920+
* Either apply subsequent similarity transformations (direct method) so that
921+
!bt
922+
\begin{equation}
923+
\mathbf{S}_N^T\dots \mathbf{S}_1^T\mathbf{A}\mathbf{S}_1\dots \mathbf{S}_N=\mathbf{D} ,
924+
\end{equation}
925+
!et
926+
* Or apply subsequent similarity transformations so that $\mathbf{A}$ becomes tridiagonal (Householder) or upper/lower triangular (the *QR* method to be discussed later).
927+
* Thereafter, techniques for obtaining eigenvalues from tridiagonal matrices can be used.
928+
* Or use so-called power methods
929+
* Or use iterative methods (Krylov, Lanczos, Arnoldi). These methods are popular for huge matrix problems.
930+
!eblock
931+
932+
933+
934+
!split
935+
===== Discussion of methods for eigenvalues =====
936+
!bblock The general overview
937+
938+
One speaks normally of two main approaches to solving the eigenvalue problem.
939+
* The first is the formal method, involving determinants and the characteristic polynomial. This proves how many eigenvalues there are, and is the way most of you learned about how to solve the eigenvalue problem, but for matrices of dimensions greater than 2 or 3, it is rather impractical.
940+
941+
* The other general approach is to use similarity or unitary tranformations to reduce a matrix to diagonal form. This is normally done in two steps: first reduce to for example a *tridiagonal* form, and then to diagonal form. The main algorithms we will discuss in detail, Jacobi's and Householder's (so-called direct method) and Lanczos algorithms (an iterative method), follow this methodology.
942+
!eblock
943+
944+
945+
!split
946+
===== Methods =====
947+
!bblock
948+
Direct or non-iterative methods require for matrices of dimensionality $n\times n$ typically $O(n^3)$ operations. These methods are normally called standard methods and are used for dimensionalities
949+
$n \sim 10^5$ or smaller. A brief historical overview
950+
951+
|----------------------------------------|
952+
| Year | $n$ | |
953+
|----------------------------------------|
954+
| 1950 | $n=20$ |(Wilkinson) |
955+
| 1965 | $n=200$ |(Forsythe et al.)|
956+
| 1980 | $n=2000$ |Linpack |
957+
| 1995 | $n=20000$ |Lapack |
958+
| 2017 | $n\sim 10^5$ |Lapack |
959+
|----------------------------------------|
960+
961+
shows that in the course of 60 years the dimension that direct diagonalization methods can handle has increased by almost a factor of
962+
$10^4$. However, it pales beside the progress achieved by computer hardware, from flops to petaflops, a factor of almost $10^{15}$. We see clearly played out in history the $O(n^3)$ bottleneck of direct matrix algorithms.
963+
964+
Sloppily speaking, when $n\sim 10^4$ is cubed we have $O(10^{12})$ operations, which is smaller than the $10^{15}$ increase in flops.
965+
966+
!eblock
967+
968+
719969

720970

721971
!split
@@ -1482,3 +1732,4 @@ To be added
14821732

14831733

14841734

1735+

0 commit comments

Comments
 (0)