Vector Norms
In order to define how close two vectors are, and in order to define the convergence of sequences of vectors, we can use the notion of a
norm. We will heavily use the following notation for nonnegative real numbers:
\[
\mathbb{R}_{+} = \left\{ x \in \mathbb{R} \, : \, x\ge 0 \right\} .
\]
Besides real numbers, we will also use the field of
complex numbers that consists of all ordered pairs (𝑎,
b) = 𝑎 +
jb with appropriate addition and multiplication operations. The unit vector in positive vertical direction is denoted by
j, so that
j² = −1. Also recall that if
z = 𝑎 +
jb ∈ ℂ is a complex number, with real numbers 𝑎,
b ∈ ℝ, then its complex conjugate is
\( \overline{z} = a - {\bf j}b , \) and
\( |z| = |\overline{z}| = \sqrt{a^2 + b^2} \) is the modulus of
z.
Let
V be a vector space over either the field of real numbers ℝ or complex numbers ℂ. A
norm on
V
is a function from a real or complex vector space
V to the nonnegative real numbers ℝ
+ that satisfies the following conditions:
-
Positivity: ‖v‖ ≥ 0,
‖v‖ = 0 iff v = 0.
-
Homogeneity: ‖kv‖ = |k| ‖v‖ for arbitrary scalar k.
-
Triangle inequality: ‖v + u‖ ≤ ‖u‖ + ‖v‖.
A vector space together with a norm ‖·‖ is called a
normed vector space.
We mention four of many norms:
-
For every x = [x1, x2, … , xn] ∈ V, we have the 1-norm:
\[
\| {\bf x}\|_1 = \sum_{k=1}^n | x_k | = |x_1 | + |x_2 | + \cdots + |x_n |.
\]
It is also called
the Taxicab norm or Manhattan norm.
-
The Euclidean norm or ℓ²-norm is
\[
\| {\bf x}\|_2 = \left( \sum_{k=1}^n x_k^2 \right)^{1/2} = \left( x_1^2 + x_2^2 + \cdots + x_n^2 \right)^{1/2} .
\]
-
The Chebyshev norm or sup-norm ‖v‖∞, is defined such that
\[
\| {\bf x}\|_{\infty} = \max_{1 \le k \le n} \left\{ | x_k | \right\} .
\]
-
The ℓp-norm (for p≥1)
\[
\| {\bf x}\|_p = \left( \sum_{k=1}^n x_k^p \right)^{1/p} = \left( x_1^p + x_2^p + \cdots + x_n^p \right)^{1/p} .
\]
Theorem 3:
The following inequalities hold for all
x ∈ ℂ
n or
x ∈ ℝ
n:
-
\( \displaystyle \| {\bf x} \|_{\infty} \le \| {\bf x} \|_{1} \le n\,\| {\bf x} \|_{\infty} , \)
-
\( \displaystyle \| {\bf x} \|_{\infty} \le \| {\bf x} \|_{2} \le \sqrt{n}\,\| {\bf x} \|_{\infty} , \)
-
\( \displaystyle \| {\bf x} \|_{2} \le \| {\bf x} \|_{1} \le \sqrt{n}\,\| {\bf x} \|_{2} .\)
====================================
including a special class of vecors, known as
Euclidean space. Their properties were collected by the ancient Greek mathematician
Euclid in his
Elements.
In given any (real or complex) vector space
V, two norms
‖ ‖
a and ‖ ‖
b are
equivalent if and only if (iff) there exists some positive constants
c1 and
c2 such that
\[
\| {\bf x}\|_a \le c_1 \| {\bf x}\|_b \qquad \mbox{and} \qquad \| {\bf x}\|_b \le c_2 \| {\bf x}\|_a \qquad \mbox{for all } \quad {\bf x} \in V.
\]
Theorem 4:
If V is any real or complex vector space of finite dimension, then any two norms on V are equivalent.
With dot product, we can assign a length of a vector, which is also called the Euclidean norm or 2-norm:
\[
\| {\bf x} \|_2 = \sqrt{ {\bf x}\cdot {\bf x}} = \sqrt{x_1^2 + x_2^2 + \cdots + x_n^2} .
\]
An
inner product space is a vector space with an additional structure called an inner product. So every inner product space inherits the Euclidean norm and becomes a metric space.
In linear algebra, functional analysis, and related areas of mathematics, a norm is a function that assigns a strictly positive length or size to each vector in a vector space—save for the zero vector, which is assigned a length of zero.
On an n-dimensional complex space
\( \mathbb{C}^n ,\) the most common norm is
\[
\| {\bf z} \| = \sqrt{ {\bf z}\cdot {\bf z}} = \sqrt{\overline{z_1} \,z_1 + \overline{z_2}\,z_2 + \cdots + \overline{z_n}\,z_n} = \sqrt{|z_1|^2 + |z_2 |^2 + \cdots + |z_n |^2} .
\]
A unit vector
\begin{equation} \label{EqVector.1}
| {\bf x} \cdot {\bf y} | \le \| {\bf x} \| \, \| {\bf y} \| .
\end{equation}
For
\[
\frac{1}{p} + \frac{1}{q} = 1 .
\]
Then the CBS inequality can be generalized as
\begin{equation} \label{EqVector.2}
| {\bf x} \cdot {\bf y} | \le \| {\bf x} \|_p \, \| {\bf y} \|_q .
\end{equation}
Eq.\eqref{EqVector.2} is known as