Matrix Norms

⫴ ⫼

The set ℳ_m,n of all m × n matrices under the field of either real or complex numbers is a vector space of dimension m  · n. In order to determine how close two matrices are, and in order to define the convergence of sequences of matrices, a special concept of matrix norm is employed, with notation \( \| {\bf A} \| . \) A norm is a function from a real or complex vector space to the nonnegative real numbers that satisfies the following conditions:

• Positivity:     ‖A‖ ≥ 0     ‖A‖ = 0 iff A = 0.
• Homogeneity:     ‖kA‖ = |k| ‖A‖ for arbitrary scalar k.
• Triangle inequality:     ‖A + B‖ ≤ ‖A‖ + ‖B‖.

Since the set of all matrices admits the operation of multiplication in addition to the basic operation of addition (which is included in the definition of vector spaces), it is natural to require that matrix norm satisfies the special property:

• ‖A · B‖ ≤ ‖A‖ · ‖B‖.

Once a norm is defined, it is the most natural way of measure distance between two matrices A and B as d(A, B) = ‖A − B‖ = ‖B − A‖. However, not all distance functions have a corresponding norm. For example, a trivial distance that has no equivalent norm is d(A, A) = 0 and d(A, B) = 1 if A ≠ B. The norm of a matrix may be thought of as its magnitude or length because it is a nonnegative number. Their definitions are summarized below for an \( m \times n \) matrix A, to which corresponds a self-adjoint (m+n)×(m+n) matrix B:

Here A* denotes the adjoint matrix: \( {\bf A}^{\ast} = \overline{{\bf A}^{\mathrm T}} = \overline{\bf A}^{\mathrm T} . \) The term "induced" refers to the fact that the definition of a norm for vectors such as A x and x is what enables the definition above of a matrix norm. This definition of matrix norm is not computationally friendly, so we use other options. The most important norms are as follow

✼ The operator norm corresponding to the p-norm for vectors, p ≥ 1, is:

where \( \| {\bf x} \|_p = \left( x_1^p + x_2^p + \cdots + x_n^p \right)^{1/p} .\)

✼ 1-norm (is commonly known as the maximum column sum norm) of a matrix A may be computed as

✻ The infinity norm, \( \infty - \) norm of matrix A may be computed as

which is simply the maximum absolute row sum of the matrix.

✺ In the special case of p = 2 we get the Euclidean norm (which is equal to the largest singular value of a matrix)

where σ_max(A) represents the largest singular value of matrix A.

✳ The Frobenius norm (non-induced norm):

The Euclidean norm and the Frobenius norm are related via the inequality:

There is also another function that that provides infinum of all norms of a square matrix: \( \rho ({\bf A}) \le \|{\bf A}\| . \)

For any positive integer k, we have

Some properties of the matrix norms are presented in the following

All these norms are equivalent and we present some inequalities:

where A* is the adjoint matrix to A (transposed and complex conjugate).

Mathematica has a special command for evaluating norms:
Norm[A] = Norm[A,2] for evaluating the Euclidean norm of the matrix A;
Norm[A,1] for evaluating the 1-norm;
Norm[A, Infinity] for evaluating the ∞-norm;
Norm[A, "Frobenius"] for evaluating the Frobenius norm.

 

Example 3: Norms of a 2×3 matrix Example 4: Norms of a 3×3 matrix

 

Subsection: Exercises