The course number is MAT8500.

The course on numerical linear algebra mainly focuses on efficient numerical methods for solving large scale linear systems, for computing eigenvalues and eigenvectors of large matrices, and for solving linear least squares problems.

First, we discuss the following direct and iterative methods for linear systems: Gaussian Elimination with partial and complete pivoting, LU and Cheloski decomposition, Gauss-Jacobi iteration, Gauss-Seidel iteration, SOR iteration, the Greedy iteration, the Steepest Descent method, Conjugate Gradient Method, GMRES method. We give proofs on the convergence of all the iterative methods above.

Secondly, we discuss numerical methods for eigenvalues of matrices: We start with Householder transform, Given's rotation, Gran-Schmidt's orthonormalization, We present Power methods and QR methods with and without shift for nonsymmetric and symmetric matrices. Also, Jacobi's rotation and bisection method for tridiagonal matrices are introduced. We give proofs on the convergence of these methods.

Thirdly, we discuss numerical methods for linar least squares problems Mainly singular value decomposition and pseudo-inverse will be introduced. Also, QR method with column pivoting and Modified Gram-Schmidt orthonormalization will be used for solving least squares problems.

Finally, we discuss numerical methods for solving systems of nonlinear equations. We start with the convergence of Newton's method for univariate case. Then we introduce multivariate Newton's method and Quasi-Newton's methods. Especially, Broydens method and its convergence will be discussed in details. We give homotopy ideas to find a good initial point.

The reference textbooks are

[1] Kincaid and Cheney, "Numerical Analysis"

[2] Trefethen and Bau, "Numerical Linear Algebra"

[3] Golub and Van Loan, "Matrix Computations",

[4] Kelley, "Iterative Methods for Linear and Nonlinear Equations"

To see the syllabuses of this course, click here. To see some sample of computer projects, click here