Inner Product
In this section, we consider scalars from either the field of real numbers ℝ or complex numbers ℂ by putting rational numbers ℚ into back burner. Roughly speaking, the inner product is a positive-definite bilinear mapping into the field of real or complex numbers.
An inner product of two vectors of the same size, usually denoted by \( \left\langle {\bf x} , {\bf y} \right\rangle ,\) is a generalization of the dot product if it satisfies the following postulates:
- \( \left\langle {\bf v}+{\bf u} , {\bf w} \right\rangle = \left\langle {\bf v} , {\bf w} \right\rangle + \left\langle {\bf u} , {\bf w} \right\rangle . \)
- \( \left\langle {\bf v} , \alpha {\bf u} \right\rangle = \alpha \left\langle {\bf v} , {\bf u} \right\rangle \) for any scalar α.
- \( \left\langle {\bf v} , {\bf u} \right\rangle = \overline{\left\langle {\bf u} , {\bf v} \right\rangle} = \left\langle {\bf u} , {\bf v} \right\rangle^{\ast} , \) where overline or asterisk means complex conjugate.
- \( \left\langle {\bf v} , {\bf v} \right\rangle \ge 0 , \) and equal if and only if \( {\bf v} = {\bf 0} . \)
The fourth condition in the list above is known as the positive-definite condition.
Example 1:
We consider ℂ² with the inner product
\[
\langle {\bf u}, {\bf v} \rangle = \langle (u_1 , u_2 ), (v_1 , v_2 ) \rangle = \overline{u_1} v_1 + 4 \overline{u_2} v_2 .
\]
Then
\[
\langle {\bf u}, {\bf u} \rangle = \left\vert u_1 \right\vert^2 + 4 \left\vert u_2 \right\vert^2 .
\]
For example
\[
\langle (1, 2{\bf j}), ( 1- {\bf j} , 2 -3{\bf j}) \rangle = 1- {\bf j} + 4 \left( -2{\bf j} \right) \left( 2 -3{\bf j} \right) = -23 - 18{\bf j} .
\]
1 -2*I + 4*(-2*I)*(2 - 3*I)
-23 - 18 I
Also
\[
\| (1, 2{\bf j}) \|^2 = \langle (1, 2{\bf j}), (1, 2{\bf j}) \rangle = 1^2 + 4 \left( -2{\bf j} \right) \left( 2{\bf j} \right) = 9 ,
\]
1 + 4 *(-2*I)*(1*I)
9
and
\[
\| ( 1- {\bf j} , 2 -3{\bf j}) \|^2 = \langle ( 1- {\bf j} , 2 -3{\bf j}), (1- {\bf j} , 2 - 3{\bf j}) \rangle = 1^2 + 1^2 + 4 \left( 4 + 9 \right) = 54 .
\]
(1 + I)*(1 - I) + 4*(2 + 3*I)*(2 - 3*I)
54
A vector space together with the inner product is called an inner product space. Every inner product space is a metric space. The metric or norm is given by
\[
\| {\bf u} \| = \sqrt{\left\langle {\bf u} , {\bf u} \right\rangle} .
\]
The nonzero vectors u and v of the same size are orthogonal (or perpendicular) when their inner product is zero:
\( \left\langle {\bf u} , {\bf v} \right\rangle = 0 . \) We abbreviate it as \( {\bf u} \perp {\bf v} . \)
A generalized length function on a vector space can be imposed in many different ways, not necessarily through the inner product. What is important that this generalized length, called in mathematics a norm, should satisfy the following four axioms.
A norm on a vector space V is a nonnegative function
\( \| \, \cdot \, \| \, : \, V \to [0, \infty ) \) that satisfies the following axioms for
any vectors \( {\bf u}, {\bf v} \in V \) and arbitrary scalar k.
If A is an n × n positive definite matrix and u and v are n-vectors, then we can define the weighted Euclidean inner product
- \( \| {\bf u} \| \) is real and nonnegative;
- \( \| {\bf u} \| =0 \) if and only if u = 0;
- \( \| k\,{\bf u} \| = |k| \, \| {\bf u} \| ;\)
- \( \| {\bf u} + {\bf v} \| \le \| {\bf u} \| + \| {\bf v} \| . \)
\[
\left\langle {\bf u} , {\bf v} \right\rangle = {\bf A} {\bf u} \cdot {\bf v} = {\bf u} \cdot {\bf A}^{\ast} {\bf v} \qquad\mbox{and} \qquad {\bf u} \cdot {\bf A} {\bf v} = {\bf A}^{\ast} {\bf u} \cdot {\bf v} .
\]
In particular, if w1, w2, ... , wn are positive real numbers,
which are called weights, and if u = ( u1, u2, ... , un) and
v = ( v1, v2, ... , vn) are vectors in
\( \mathbb{R}^n , \) then the formula
\[
\left\langle {\bf u} , {\bf v} \right\rangle = w_1 u_1 v_1 + w_2 u_2 v_2 + \cdots + w_n u_n v_n
\]
defines an inner product on \( \mathbb{R}^n , \) that is called the weighted Euclidean inner product with weights
w1, w2, ... , wn.
Example 2:
The Euclidean inner product and the weighted Euclidean inner product (when \( \left\langle {\bf u} , {\bf v} \right\rangle = \sum_{k=1}^n a_k u_k v_k , \)
for some positive numbers \( a_k , \ (k=1,2,\ldots , n \) ) are special cases of a general class
of inner products on \( \mathbb{R}^n \) called matrix inner product. Let A be an
invertible n-by-n matrix. Then the formula
\[
\left\langle {\bf u} , {\bf v} \right\rangle = {\bf A} {\bf u} \cdot {\bf A} {\bf v} = {\bf v}^{\mathrm T} {\bf A}^{\mathrm T} {\bf A} {\bf u}
\]
defines an inner product generated by A.
Example 3:
In the set of integrable functions on an interval [a,b], we can define the inner product of two functions f and g as
\[
\left\langle f , g \right\rangle = \int_a^b \overline{f} (x)\, g(x) \, {\text d}x \qquad\mbox{or} \qquad
\left\langle f , g \right\rangle = \int_a^b f(x)\,\overline{g} (x) \, {\text d}x .
\]
Then the norm \( \| f \| \) (also called the 2-norm) becomes the square root of
\[
\| f \|^2 = \left\langle f , f \right\rangle = \int_a^b \left\vert f(x) \right\vert^2 \, {\text d}x .
\]
In particular, the 2-norm of the function \( f(x) = 5x^2 +2x -1 \) on the interval [0,1] is
\[
\| 2 x^2 +2x -1 \| = \sqrt{\int_0^1 \left( 5x^2 +2x -1 \right)^2 {\text d}x } = \sqrt{7} .
\]
Example 4:
Consider a set of polynomials of degree up to n, denoted by ℘[x]. If
\[
{\bf p} = p(x) = p_0 + p_1 x + p_2 x^2 + \cdots + p_n x^n \quad\mbox{and} \quad {\bf q} = q(x) = q_0 + q_1 x + q_2 x^2 + \cdots + q_n x^n
\]
are two polynomials, and if \( x_0 , x_1 , \ldots , x_n \) are distinct real numbers (called sample points), then the formula
\[
\left\langle {\bf p} , {\bf q} \right\rangle = p(x_0 ) q(x_0 ) + p_1 (x_1 )q(x_1 ) + \cdots + p(x_n ) q(x_n )
\]
defines an inner product, which is called the evaluation inner product at \( x_0 , x_1 , \ldots , x_n . \) ■
With dot product, we can assign a length of a vector, which is also called the Euclidean norm or 2-norm:
\[
\| {\bf x} \|_2 = \| {\bf x} \| = \sqrt{ {\bf x}\cdot {\bf x}} = \sqrt{x_1^2 + x_2^2 + \cdots + x_n^2} .
\]
This norm can be generalized for arbitrary real p:
\[
\| {\bf x} \|_p = \left( x_1^p + x_2^p + \cdots + x_n^p \right)^{1/p} .
\]