es

The Wolfram Mathematica notebook which contains the code that produces all the Mathematica output in this web page may be downloaded at this link.

To enhance pedagogical effectiveness, the treatment of the dot product is presented in several distinct sections

Dot product

Geometrical interpretation

Orthogonality

Projection

Although all applications of dot product were presented within the field of real numbers ℝ so far, this section can be applied to arbitrary scalar field, including complex numbers ℂ.

 

Duality

The dot product of two numerical vectors (i.e., vectors from 𝔽n, where 𝔽 is a field of either real numbers ℝ or complex numbers ℂ) is a single number that provides information about the relationship between two vectors. The linearity of dot product tells us that the scalar product of two numerical vectors can be used for definition of linear forms/functionals. The reverse statement (see section on dual spaces in Part 3) is always true: every linear functional on a finite dimensional vector space is generated by a corresponding dot product.

   
Example 12: In calculus, the gradient of a scalar function is a dual vector. It takes a vector (representing a direction) and returns the rate of change of the scalar function in that direction.

In physics, dual vectors appear in various contexts, such as in describing fields (like electromagnetic fields) and in quantum mechanics (using bra-ket notation).    ■

End of Example 12

At the beginning of twentieth century, it was discovered that the dot product is needed for definition of dual spaces (see section in Part3). Although definition of dot product is symmetrical, it is convenient to separate vectors identifying one of them as a vector, called contravariant vector or ket-vector. When ordered basis in a finite dimensional vector space is chosen, [e₁, e₂, … , en], every vector can be uniquely expressed as linear combination of the basis vectors:

\[ \mathbf{x} = x^1 \mathbf{e}_1 + x^2 \mathbf{e}_2 + \cdots + x^n \mathbf{e}_n = \sum_{i=1}^n x^i \mathbf{e}_i = x^i \mathbf{e}_i , \]
whose components are identified with superscripts. The upper indices are not exponents but are indices of coordinates, identifying the coefficients of basis vectors. It is customary to omit the sum symbol in this expression adopting the Einstein summation convention. This convention is routinely used in Einstein’s general theory of relativity which suffers from a proliferation of indices but it also facilitates a simplified notation and more effective computations in many other contexts.

Using the dual basis [e¹, e², … , en], a vector from the dual space can be written as

\[ \mathbf{y} = y_1 \mathbf{e}^1 + y_2 \mathbf{e}^2 + \cdots + y_n \mathbf{e}^n = \sum_{j=1}^n y_j \mathbf{e}^j = y_j \mathbf{e}^j . \]
This vector y from the dual space is called the covariant vector or covector or dual vector or bra-vector. Then using the duality property \( \displaystyle \quad \mathbf{e}^j \mathbf{e}_i = \delta^i_j , \quad \) where δij is the Kronecker's delta, we can write the dot product of bra-vector y with ket-vector x in concise form:
\[ \mathbf{y} \bullet \mathbf{x} = \sum_{i=1}^n y_i x^i = y_i x^i . \tag{2} \]
The dot product
\begin{equation} \label{EqDot.2} {\bf x} \bullet {\bf y} = x_1 y_1 + x_2 y_2 + \cdots + x_n y_n , \end{equation}
defines a linear functional on any n dimensional vector space independently on what field is used (ℂ or ℝ). Then one of the multipliers, say x in the linear combination \eqref{EqDot.2}, is called a vector, but another counterpart, y is known as a covector. Such treatment of vectors in the dot product breaks their "equal rights." In many practical problems, these vectors, x and y, are indeed taken from different spaces, although sometimes look the same.

In geometry, to distinguish these two partners in Eq.\eqref{EqDot.2}, the vector x is called contravariant vector, and the covector y is referred to as covariant vector. In order to decide between these partners who is who, it is common to use superscript for coordinates of contravariant vector, x = [ x¹, x², x³], and subscript for covariant vectors, y = [y₁, y₂, y₃]. In physics, covariant vectors are also called bra-vectors, while contravariant vectors are known as ket-vectors.

 

Duality in matrix/vector multiplication

Although duality is covered in Part 3, we introduce some of its elements here in order to show a different point of view in the context of matrix/vector multiplication.

So far, we have been using a naïve definition of a vector as an ordered (or indexed) finite sequence of scalars; it is usually represented in parenthesis as an n-tuple (x1, x2, … , xn), an element of Cartesian product 𝔽n = 𝔽 × 𝔽 × ⋯ × 𝔽 of n copies of the field of scalars 𝔽. Later, we will extend this definition to include abstract objects; however, for us at this moment, it is sufficient to work with n-tuples. Recall that for scalars we use either rational numbers ℚ or real numbers ℝ or complex numbers ℂ.

Usually, an n-tuple (x1, x2, … , xn), written in parenthesis and a row vector [x1, x2, … , xn], written in brackets, look as the same object to human eyes. One of the pedagogical virtues of any software package is its requirement to pay close attention to the types of objects used in specific contexts. In particular, the computer algebra system Mathematica treats these two versions of a vector differently because

\[ \left( x_1 , x_2 , \ldots , x_n \right) \in \mathbb{F}^n , \]
\[ \mbox{but} \]
\[ \left[ x_1 , x_2 , \ldots , x_n \right] \in \mathbb{F}^{1\times n} , \]
where 𝔽m×n denotes the space of m × n matrices with coefficients from field 𝔽. Hence, Mathematica considers bracket notation as a matrix with one row.
x = {1, 2}; y = {{1, 2}};
x == y
False
Mathematica considers single curly bracket notation as a list of entries, which is treated as a column vector for convenience. On the other hand, a double curly bracket notation is considered as a list of lists and it is represented as a matrix with one row.
x = {1, 2};
MatrixForm[x]
\( \displaystyle \begin{pmatrix} 1 \\ 2 \end{pmatrix} \)
MatrixForm[{x}]
(1, 2)
Note that dot product of these objects is non-commutative. That is, the order matters:
x . y
Dot::dotsh: Tensors {1,2} and {{1,2}} have incompatible shapes.
y . x
{5}

Matrices, as two dimensional arrays, provide a dual version of a row vector, known as a column vector:

\[ {\bf x} = \left[ \begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right] \in \mathbb{F}^{n\times 1} , \qquad {\bf a} = \left[ a_1 , a_2 , \ldots , a_n \right] \in \mathbb{F}^{1\times n} . \]
As matrices, column vectors (x ∈ 𝔽n×1) can be multiplied by row vectors (a ∈ 𝔽1×n) from left when they both have the same size:
\[ \left[ a_1 , a_2 , \ldots , a_n \right] \,\left[ \begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right] = \left[ a_1 x_1 + \cdots + a_n x_n \right] \in \mathbb{R}^{1\times 1} . \]
It is a matrix with one row and one column. In order to extract this entry from 1×1 matrix, mathematicians consider this expression as a linear combination. It also can be defined by dot product:
\[ \left( a_1 , \ldots , a_n \right) \bullet \left( x_1 , \ldots , x_n \right) = a_1 x_1 + a_2 x_2 + \cdots + a_n x_n \in \mathbb{R} , \]
when all entries are real numbers. You need to wait till Part 5 for dot product extension to complex numbers (it is inner product). Although dot product formula above can be used for complex numbers as well, it does not define a metric (distance) in complex vector spaces. Physicists usually use Dirac's notation (also known as bra-ket notation), which is designated as <a | x> for an alternative to linear combination or dot product.
Every numeric vector a = (𝑎1, 𝑎2, … , 𝑎n) ∈ 𝔽n determines the corresponding dual vector (known as a covector) a* that defines a linear functional a* : 𝔽n ⇾ 𝔽 via the simple rule
\[ {\bf a}^{\ast} ({\bf x}) = \left\langle {\bf a} \mid {\bf x} \right\rangle = a_1 x_1 + \cdots + a_n x_n , \quad {\bf x} \in \mathbb{F}^n . \]
When vector a = (𝑎1, 𝑎2, … , 𝑎n) is considered as a covector, it is usually called a bra-vector and is denoted by <a∣, while x = (x1, x2, … , xn) ∈ ℂn is called a ket-vector or just a vector and denoted by ∣x>. Then the linear combination \( \displaystyle a_1 x_1 + a_2 x_2 + \cdots + a_n x_n \) (which is dot product for real numbers) defines a linear functional (or linear form), which we denote by a*. Hence, every n-tuple a = (𝑎1, 𝑎2, … , 𝑎n) defines a covector (or linear functional). In case of real numbers, action of a covector on a vector is just dot product.

Example 1: The linear functional on ℝ4 given by \[ f\left( x_1 , x_2 , x_3 , x_4 \right) = 2\,x_1 - 3\, x_2 + x_3 - 5\, x_4 \] is dual to the numeric vector a = (2, −3, 1, −5). This vector generates a linear functional on ℝ4 by means of dot product
fx1 = {Subscript[x, 1], Subscript[x, 2], Subscript[x, 3], Subscript[x, 4]};
coefVec1 = {2, -3, 1, -5};
fx1 . coefVec1
\( \displaystyle 2\,x_1 - 3\, x_2 + x_3 - 5\, x_4 \)
\[ \langle {\bf a} \mid {\bf x} \rangle = {\bf a} \bullet {\bf x} = 2\,x_1 - 3\, x_2 + x_3 - 5\, x_4 , \] for arbitrary vector x = (x₁, x₂, x₃, x₄) ∈ ℝ4. This functional, written in bra-ket notation, can be redefined in standard mathematical form as \[ {\bf a}^{\ast} \, : \ \mathbb{R} \mapsto \mathbb{R} , \qquad\mbox{with} \qquad {\bf a}^{\ast} ({\bf x}) = \langle {\bf a} \mid {\bf x} \rangle = {\bf a} \bullet {\bf x} . \]

On the other hand, the numeric vector a = (2, −3, 1) ∈ ℝ³ is dual to the linear functional (known as a covector) given by \[ {\bf a}^{\ast} \left( {\bf x} \right) = \left\langle {\bf a} \mid {\bf x} \right\rangle = {\bf a} \bullet {\bf x} = 2\, x_1 -3\, x_2 + x_3 , \quad {\bf x} \in \mathbb{R}^3 . \]

fx2 = {Subscript[x, 1], Subscript[x, 2], Subscript[x, 3]};
coefVec2 = {2, -3, 1};
fx2 . coefVec2
\( \displaystyle 2\,x_1 - 3\, x_2 + x_3 \)
In particular, actions of the covector ⟨a | on some bra-vectors from ℝ³ are given below: \begin{align*} {\bf a}^{\ast} (1, 2, 3) &= {\bf a} \bullet \left( 1, 2, 3 \right) = 2\cdot 1 -3\cdot 2 + 3 = -1, \\ {\bf a}^{\ast} (-1, 2, -5) &= {\bf a} \bullet \left( -1, 2, -5 \right) = 2\cdot (-1) -3 \cdot 2 -5 = -13 . \end{align*}
End of Example 1
   In Linear Algebra, the word duality traditionally refers to the dual roles of vector and covector associated with a given numeric vector, as in definition above. The row/column duality just described is one of many beautiful dualities we highlight in this tutorial. We will see many other examples of duality, including image/inverse image, geometric/numeric, and projection/span—problems that seem difficult from one viewpoint often turn out to be much easier from the dual viewpoint. Each of the two perspectives illuminates the other in a beautiful and useful way, and we shall make good use of this principle many times.

We can regard an m × n matrix, for instance, as a list of m (row) vectors in ℝ1×n, or as a list of n (column) vectors in ℝm×1, and the difference is not superficial, as we shall come to see. Every m × n matrix

\[ {\bf A} = \begin{bmatrix} a_{1,1} & a_{1,2} & a_{1,3} & \cdots & a_{1,n} \\ a_{2,1} & a_{2,2} & a_{2,3} & \cdots & a_{2,n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{m,1} & a_{m,2} & a_{m,3} & \cdots & a_{m,n} \end{bmatrix} \in \mathbb{F}^{m\times n} \]
defines a linear transformation TA : 𝔽n×1 ⇾ 𝔽m×1 by multiplication from left:
\[ T_A ({\bf x}) = {\bf A}\, {\bf x} \in \mathbb{F}^{m\times 1} , \qquad \mbox{with} \quad {\bf x} \in \mathbb{R}^{n\times 1} . \]
However, the same m × n matrix A can operate on row vectors y ∈ 𝔽1×m from right:
\[ {\bf y}\,{\bf A} \in \mathbb{F}^{1\times n} \qquad \iff \qquad {\bf y}\,{\bf A} = \left( {\bf A}^{\mathrm T} {\bf y}^{\mathrm T} \right)^{\mathrm T} , \]
where "T" stands for transposition---a dual operation that transfers rows into columns and vice versa. Since in the following exposition we utilize matrix multiplication from left, we are forced to consider vectors as columns. Then
\begin{align*} {\bf A}\,{\bf x} &= \begin{bmatrix} a_{1,1} & a_{1,2} & a_{1,3} & \cdots & a_{1,n} \\ a_{2,1} & a_{2,2} & a_{2,3} & \cdots & a_{2,n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{m,1} & a_{m,2} & a_{m,3} & \cdots & a_{m,n} \end{bmatrix} \,\begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix} \\ &= \begin{bmatrix} a_{1,1} x_1 + a_{1,2} x_2 + \cdots + a_{1,n} x_n \\ a_{2,1} x_1 + a_{2,2} x_2 + \cdots + a_{2,n} x_n \\ \vdots \\ a_{m,1} x_1 + a_{m,2} x_2 + \cdots + a_{m,n} x_n \end{bmatrix} \\ &= {\bf c}_1 ({\bf A})\, x_1 + {\bf c}_2 ({\bf A})\, x_2 + \cdots + {\bf c}_n ({\bf A})\, x_n , \end{align*}
where ci(A) , i = 1, 2, … , n, are columns of matrix A:
\[ {\bf c}_1 ({\bf A}) = \left[ \begin{array}{c} a_{1,1} \\ a_{2,1} \\ \vdots \\ a_{m,1} \end{array} \right] , \quad{\bf c}_2 ({\bf A}) = \left[ \begin{array}{c} a_{1,2} \\ a_{2,2} \\ \vdots \\ a_{m,2} \end{array} \right] , \quad \cdots \quad , {\bf c}_n ({\bf A}) = \left[ \begin{array}{c} a_{1,n} \\ a_{2,n} \\ \vdots \\ a_{m,n} \end{array} \right] \in \mathbb{F}^{m\times 1} . \]
In other words, A x is a linear combination of column vectors of matrix A with weights taken from coordinates of vector x. Alternatively, using dual covectors, we can say the same thing like this:
\[ {\bf A}\,{\bf x} = \begin{bmatrix} {\bf r}_1^{\ast} ( {\bf x}) \\ {\bf r}_2^{\ast} ( {\bf x}) \\ \vdots \\ {\bf r}_m^{\ast} ( {\bf x}) \end{bmatrix} \in \mathbb{F}^{m\times 1} \qquad\mbox{or} \qquad {\bf A}\,{\bf x} = \begin{bmatrix} {\bf r}_1 ({\bf A}) \bullet {\bf x} \\ {\bf r}_2 ({\bf A}) \bullet {\bf x} \\ \vdots \\ {\bf r}_m ({\bf A}) \bullet {\bf x} \end{bmatrix} \in \mathbb{R}^{m\times 1} , \]
where rj(A), j = 1, 2, … , m, are row vectors of matrix A:
\[ {\bf r}_1 ({\bf A}) = \left( a_{1,1} , a_{1,2} , \ldots , a_{1,n} \right) , \quad \cdots \quad , {\bf r}_m ({\bf A}) = \left( a_{m,1} , a_{m,2} , \ldots , a_{m,n} \right) . \]
Example 2: We consider a 2 × 3 matrix A and vectors x ∈ ℝ3×1, y ∈ ℝ1×2, where \[ {\bf A} = \begin{bmatrix} 1&2&3 \\ 4&5&6 \end{bmatrix} , \qquad {\bf x} = \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} , \quad {\bf y} = \left[ y_1 , y_2 \right] . \]
Clear[A, x, y];
A = {{1, 2, 3}, {4, 5, 6}};
xVec = {Subscript[x, 1], Subscript[x, 2], Subscript[x, 3]};
yVec = {{Subscript[y, 1], Subscript[y, 2]}};
Grid[{MatrixForm[#] & /@ {A, xVec, yVec}}]
\( \displaystyle \begin{pmatrix} 1&2&3 \\ 4&5&6 \end{pmatrix} \qquad \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} \qquad \begin{pmatrix} y_1 & y_2 \end{pmatrix} \)
Since columns of matrix A are \[ {\bf c}_1 ({\bf A}) = \left[ \begin{array}{c} 1 \\ 4 \end{array} \right] , \quad {\bf c}_2 ({\bf A}) = \left[ \begin{array}{c} 2 \\ 5 \end{array} \right] , \quad {\bf c}_3 ({\bf A}) = \left[ \begin{array}{c} 3 \\ 6 \end{array} \right] , \]
Subscript[col, 1] = A[[All, 1]];
Subscript[col, 2] = A[[All, 2]];
Subscript[col, 3] = A[[All, 3]];
Grid[{MatrixForm[#] & /@ {Subscript[col, 1], Subscript[col, 2], Subscript[col, 3]}}]
\( \displaystyle \begin{pmatrix} 1 \\ 4 \end{pmatrix} \qquad \begin{pmatrix} 2 \\ 5 \end{pmatrix} \qquad \begin{pmatrix} 3 \\ 6 \end{pmatrix} \)
and rows are \[ {\bf r}_1 ({\bf A}) = \left( 1, 2, 3 \right) , \quad {\bf r}_2 ({\bf A}) = \left( 4, 5, 6 \right) , \]
Subscript[row, 1] = A[[1]]; Subscript[row, 2] = A[[2]]; Grid[{# & /@ {Subscript[row, 1], Subscript[row, 2]}}]
{1, 2, 3} {4, 5, 6}
we find its product with vector x as \[ {\bf A}\,{\bf x} = \begin{bmatrix} 1\,x_1 + 2\,x_2 + 3\,x_3 \\ 4\, x_1 + 5\, x_2 + 6\,x_3 \end{bmatrix} \in \mathbb{R}^{2 \times 1} . \] This column vector can be splitted into sum of three vectors: \[ {\bf A}\,{\bf x} = x_1 \left[ \begin{array}{c} 1 \\ 4 \end{array} \right] + x_2\left[ \begin{array}{c} 2 \\ 5 \end{array} \right] + x_3 \left[ \begin{array}{c} 3 \\ 6 \end{array} \right] = \begin{bmatrix} 1\,x_1 \\ 4\,x_1 \end{bmatrix} + \begin{bmatrix} 2\,x_2 \\ 5\,x_2 \end{bmatrix} + \begin{bmatrix} 3\,x_3 \\ 6\,x_3 \end{bmatrix} \in \mathbb{R}^{2\times 1} , \] Each entry in the column vector A x is a dot product of two vectors: \[ {\bf A}\,{\bf x} = \begin{bmatrix} 1\,x_1 + 2\,x_2 + 3\,x_3 \\ 4\, x_1 + 5\, x_2 + 6\,x_3 \end{bmatrix} = \begin{bmatrix} {\bf r}_1 ({\bf A}) \bullet {\bf x} \\ {\bf r}_2 ({\bf A}) \bullet {\bf x} \end{bmatrix} \in \mathbb{R}^{2\times 1} , \] where \begin{align*} {\bf r}_1 ({\bf A}) \bullet {\bf x} &= \left( 1, 2, 3 \right) \bullet \left( x_1 , x_2 , x_3 \right) = x_1 + 2\, x_2 + 3\, x_3 \in \mathbb{R} , \\ {\bf r}_2 ({\bf A}) \bullet {\bf x} &= \left( 4, 5, 6 \right) \bullet \left( x_1 , x_2 , x_3 \right) = 4\,x_1 + 5\, x_2 + 6\, x_3 \in \mathbb{R} . \end{align*}
Transpose[A]*xVec
% // MatrixForm
\( \displaystyle \left\{ \left\{ x_1 \ 4\,x_1 \right\} , \ \left\{ 2\,x_2 , \ 5\, x_2 \right\} , \ \left\{ 3\,x_3 , \ 6\,x_3 \right\} \right\} \)
\( \displaystyle \begin{pmatrix} x_1 & 4\, x_1 \\ 2\,x_2 & 5\, x_2 \\ 3\,x_3 & 6\, x_6 \end{pmatrix} \)
Grid[{{Subscript[row, 1] . xVec}, {Subscript[row, 2] . xVec}}]
\( \displaystyle \begin{array} \ \ x_1 + 2\,x_2 + 3\,x_3 \\ 4\,x_1 + 5\,x_2 + 6\, x_3 \end{array} \)
On the other hand, the dual multiplication from right yields \begin{align*} {\bf y}\,{\bf A} &= \left[ {\bf y} \bullet {\bf c}_1 ({\bf A}) , {\bf y} \bullet {\bf c}_2 ({\bf A}) ,{\bf y} \bullet {\bf c}_3 ({\bf A}) \right] \\ &= \left[ y_1 + 4\,y_2 , \ 2\,y_1 + 5\,y_2 , \ 3\, y_1 + 6\,y_2 \right] \in \mathbb{R}^{1\times 3} . \end{align*}
End of Example 2

 

 

  1. Anton, Howard (2005), Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International
  2. Beezer, R., A First Course in Linear Algebra, 2015.
  3. Beezer, R., A Second Course in Linear Algebra, 2013.