Numerical Computation

Numerical Computation Lecture 9: Vector Norms and Matrix Condition Numbers United International College

Review • During our Last Class we covered: • Operation count for Gaussian Elimination, LU Factorization • Accuracy of Matrix Methods • Readings: • Pav, section 3.4.1 • Moler, section 2.8

Today • We will cover: • Vector and Matrix Norms • Matrix Condition Numbers • Readings: • Pav, section 1.3.2, 1.3.3, 1.4.1 • Moler, section 2.9

Vector Norms A vector norm is a quantity that measures how large a vector is (the magnitude of the vector). For a number x, we have |x| as a measurement of the magnitude of x. For a vector x, it is not clear what the “best” measurement of size should be. Note: we will use bold-face type to denote a vector. ( x )

Vector Norms Example: x = ( 4 -1 ) is the standard Pythagorean length of x. This is one possible measurement of the size of x. x

Vector Norms Example: x = ( 4 -1 ) |4| + |-1| is the “Taxicab” length of x. This is another possible measurement of the size of x. x

Vector Norms Example: x = ( 4 -1 ) max(|4|,|-1|) is yet another possible measurement of the size of x. x

Vector Norms A vector norm is a quantity that measures how large a vector is (the magnitude of the vector). Definition: A vector norm is a function that takes a vector and returns a non-zero number. We denote the norm of a vector x by The norm must satisfy: Triangle Inequality: Scalar: Positive: ,and = 0 only when x is the zero vector.

Vector Norms • Our previous examples for vectors in Rn : • Manhattan • Euclidean • Chebyshev • All of these satisfy the three properties for a norm.

Vector Norms Example

Vector Norms • Definition: The Lp norm generalizes these three norms. For p > 0, it is defined on Rn by: • p=1 L1 norm • p=2 L2 norm • p= ∞ L∞ norm

Distance

Distance • Class Practice: • Find the L2 distance between the vectors x = (1, 2, 3) and y = (4, 0, 1). • Find the L ∞ distance between the vectors x = (1, 2, 3) and y = (4, 0, 1).

Which norm is best? • The answer depends on the application. • The 1-norm and ∞-norm are good whenever one is analyzing sensitivity of solutions. • The 2-norm is good for comparing distances of vectors. • There is no one best vector norm!

Matlab Vector Norms • In Matlab, the norm function computes the Lp norms of vectors. Syntax: norm(x, p) >> x = [ 3 4 -1 ]; >> n = norm(x,2) n = 5.0990 >> n = norm(x,1) n = 8 >> n = norm(x, inf) n = 4

Matrix Norms • Definition: Given a vector norm the matrix norm defined by the vector norm is given by: • Example:

Matrix Norms • Example: • What does a matrix norm represent? • It represents the maximum “stretching” that A does to a vector x -> (Ax).

Matrix Norm Properties • || A || > 0 if A ≠ O • || c A || = | c| *||A || if A ≠ O • || A + B || ≤ || A || + || B || • || A B || ≤ || A || *||B || • || A x || ≤ || A || *||x ||

Matrix • Multiplication of a vector x by a matrix A results in a new vector Ax that can have a very different norm from x. • The range of the possible change can be expressed by two numbers, • =||A|| • Here the max, min are over all non-zero vectors x.

Matrix Condition Number • Definition: The condition number of a nonsingular matrix A is given by: κ(A) = M/m by convention if A is singular (m=0) then κ(A) = ∞. • Note: If we let Ax = y, then x = A-1y and

Matrix Condition Number • Theorem: The condition number of a nonsingular matrix A can also be given as: κ(A) = || A || * || A-1|| • Proof: κ(A) = M/m. Also, M = ||A|| and by the previous slide m = 1 / (||A-1 ||). QED

Properties of the Matrix Condition Number • For any matrix A, κ(A) ≥ 1. • For the identity matrix, κ(I) = 1. • For any permutation matrix P, κ(P) =1. • For any matrix A and nonzero scalar c , κ(c A) = κ(A). • For any diagonal matrix D = diag(di), κ(D) = (max|di|)/( min | di| )

What does the condition number tell us? • The condition number is a good indicator of how close is a matrix to be singular. The larger the condition number the closer we are to singularity. • It is also very useful in assessing the accuracy of solutions to linear systems. • In practice we don’t really calculate the condition number, it is merely estimated , to perhaps within an order of magnitude.

Condition Number And Accuracy • Consider the problem of solving Ax = b. Suppose b has some error, say b + δb. Then, when we solve the equation, we will not get x but instead some value near x, say x + δx. A(x + δx) = b + δb • Then,A(x + δx) = b + δb

Condition Number And Accuracy • Class Practice: Show:

Condition Number And Accuracy • The quantity ||δb||/||b|| is the relative change in the right-hand side, and the quantity ||δx||/||x|| is the relative error caused by this change. • This shows that the condition number is a relative error magnification factor. That is, changes in the right-hand side of Ax=b can cause changes κ(A) times as large in the solution.

Numerical Computation