Dot Product and Cross Product

Q: What is the difference between dot product and cross product?

Dot product takes two vectors and returns a scalar. Cross product (defined only in 3D) takes two vectors and returns a vector perpendicular to both. Dot product captures alignment and angle; cross product captures area and perpendicular direction.

Q: How do I compute the dot product?

Sum the products of corresponding components: u · v = u₁v₁ + u₂v₂ + ... + uₙvₙ. Equivalently, it equals |u||v|cos(θ), where θ is the angle between the vectors. The component formula is the standard way to compute it.

Q: How do I compute the cross product?

Use the formula u × v = (u₂v₃ − u₃v₂, u₃v₁ − u₁v₃, u₁v₂ − u₂v₁) or the determinant mnemonic with î, ĵ, k̂ in the first row. The result is a vector perpendicular to both u and v with magnitude |u||v|sin(θ).

Q: What does it mean if the dot product is zero?

The two vectors are perpendicular (orthogonal), assuming neither is zero. The dot product is the standard test for orthogonality, used everywhere from Gram-Schmidt orthogonalization to neural network weight initialization.

Q: What does it mean if the cross product is zero?

The two vectors are parallel (or one is the zero vector). The cross product magnitude equals the area of the parallelogram spanned by the vectors; parallel vectors form a degenerate parallelogram with zero area.

Q: Why does the cross product only exist in 3D?

Because 'perpendicular to two given vectors' uniquely defines a single direction (up to sign) only in 3D. In 2D, perpendicular points out of the plane. In higher dimensions, the perpendicular subspace is multidimensional. The exception is 7D, where a related cross product exists for algebraic reasons.

Q: What is the right-hand rule?

Point your right hand's fingers along the first vector and curl them toward the second; your thumb points in the direction of the cross product. It's the convention for orienting cross products in 3D and ensures u, v, u × v form a right-handed coordinate system.

Q: How is the dot product used in machine learning?

Cosine similarity (dot product divided by product of magnitudes) is the standard metric for comparing embeddings, recommendation matching, document similarity, and many other ML tasks. Neural network weights and inputs are combined via dot products at every layer.

Q: How is the cross product used in physics?

Torque = r × F. Angular momentum = r × p. Magnetic force = qv × B. Anywhere rotational or perpendicular geometry matters, cross products show up. In rigid body dynamics, the inertia tensor and angular velocity combine via cross products to give torque equations.

Q: What is cosine similarity?

It's the cosine of the angle between two vectors: cos_sim(u, v) = (u · v) / (|u| · |v|). It returns a value between −1 (opposite directions) and 1 (same direction), with 0 indicating perpendicular. Used everywhere from search engines to recommendation systems to embedding space analysis.

Dot product and cross product are the two fundamental ways to multiply vectors. They’re not just calculation rules — they’re carriers of geometric meaning. The dot product encodes angle and projection; the cross product encodes perpendicular direction and area. Together they handle nearly every vector calculation in physics, computer graphics, and engineering.

The dot product produces a scalar from two vectors. The cross product (defined only in 3D) produces a vector perpendicular to both inputs. Each captures a different aspect of how vectors relate, and learning to switch between them quickly is one of the practical superpowers of linear algebra.

This study note covers the formulas, geometric interpretations, properties, the right-hand rule for cross products, worked examples, common confusions, and the applications that make these two operations indispensable across science and engineering.

Dot product and cross product formulas right-hand rule modern textbook illustration

The Dot Product Formula

The dot product (inner product, scalar product) of two vectors is defined two equivalent ways:

$$\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + \ldots + u_n v_n = |\mathbf{u}||\mathbf{v}| \cos\theta$$

The first form (component-wise sum) is computational. The second form (magnitudes times cosine of angle) is geometric. Both produce the same scalar.

The component formula generalizes to any number of dimensions. The geometric formula assumes the standard Euclidean structure but extends to inner-product spaces in functional analysis.

Geometric Meaning of the Dot Product

The dot product captures how much one vector lies in the direction of the other. If $\mathbf{u} \cdot \mathbf{v} > 0$, the vectors point in roughly the same direction (angle less than 90°). If $\mathbf{u} \cdot \mathbf{v} < 0$, they point in roughly opposite directions (angle greater than 90°). If $\mathbf{u} \cdot \mathbf{v} = 0$, they're perpendicular.

Geometrically, $\mathbf{u} \cdot \mathbf{v}$ equals $|\mathbf{u}|$ times the length of the projection of $\mathbf{v}$ onto $\mathbf{u}$ (or vice versa). This makes the dot product a measure of “alignment” — how parallel the vectors are.

For unit vectors, the dot product reduces to just $\cos\theta$, making the dot product the standard tool for computing angles between vectors.

Properties of the Dot Product

Commutative: $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$.
Distributive over addition: $\mathbf{u} \cdot (\mathbf{v} + \mathbf{w}) = \mathbf{u} \cdot \mathbf{v} + \mathbf{u} \cdot \mathbf{w}$.
Compatible with scalar multiplication: $(c\mathbf{u}) \cdot \mathbf{v} = c(\mathbf{u} \cdot \mathbf{v}) = \mathbf{u} \cdot (c\mathbf{v})$.
Self-dot equals magnitude squared: $\mathbf{v} \cdot \mathbf{v} = |\mathbf{v}|^2$.
Cauchy-Schwarz inequality: $|\mathbf{u} \cdot \mathbf{v}| \leq |\mathbf{u}||\mathbf{v}|$, with equality if and only if the vectors are parallel.

The Cauchy-Schwarz inequality is one of the most useful inequalities in mathematics; it underpins many proofs in analysis, statistics, and probability theory.

Worked Example: Dot Product

Compute $\mathbf{u} \cdot \mathbf{v}$ and the angle between them, where $\mathbf{u} = (3, 4, 0)$ and $\mathbf{v} = (1, 0, 2)$.

Component-wise:

$$\mathbf{u} \cdot \mathbf{v} = 3 \cdot 1 + 4 \cdot 0 + 0 \cdot 2 = 3$$

Magnitudes: $|\mathbf{u}| = \sqrt{9 + 16} = 5$, $|\mathbf{v}| = \sqrt{1 + 4} = \sqrt{5}$.

Angle: $\cos\theta = 3 / (5 \cdot \sqrt{5}) \approx 0.268$, so $\theta \approx 74.5°$.

The dot product gives both a quick orthogonality check (positive here, so the vectors aren’t perpendicular) and a clean angle calculation when normalized.

The Cross Product Formula

The cross product is defined only in 3D (and 7D for separate algebraic reasons). For $\mathbf{u}, \mathbf{v} \in \mathbb{R}^3$:

$$\mathbf{u} \times \mathbf{v} = (u_2 v_3 – u_3 v_2, u_3 v_1 – u_1 v_3, u_1 v_2 – u_2 v_1)$$

This produces a new vector perpendicular to both inputs. The magnitude is:

$$|\mathbf{u} \times \mathbf{v}| = |\mathbf{u}||\mathbf{v}| \sin\theta$$

which equals the area of the parallelogram spanned by $\mathbf{u}$ and $\mathbf{v}$. Direction follows the right-hand rule.

The Right-Hand Rule

To find the direction of $\mathbf{u} \times \mathbf{v}$, point your right hand’s fingers along $\mathbf{u}$ and curl them toward $\mathbf{v}$. Your thumb points in the direction of the cross product.

Equivalent statement: $\mathbf{u}$, $\mathbf{v}$, and $\mathbf{u} \times \mathbf{v}$ form a right-handed coordinate system, with the cross product playing the role of the third axis.

The right-hand rule is convention. Some physics communities (especially older European ones) use a left-hand rule for certain quantities; the math is the same up to sign. Modern textbooks universally adopt the right-hand convention.

Properties of the Cross Product

Anticommutative: $\mathbf{u} \times \mathbf{v} = -(\mathbf{v} \times \mathbf{u})$. Order matters and reversing it flips the sign.
Distributive: $\mathbf{u} \times (\mathbf{v} + \mathbf{w}) = \mathbf{u} \times \mathbf{v} + \mathbf{u} \times \mathbf{w}$.
Scalar multiplication: $(c\mathbf{u}) \times \mathbf{v} = c(\mathbf{u} \times \mathbf{v})$.
Not associative: $(\mathbf{u} \times \mathbf{v}) \times \mathbf{w} \neq \mathbf{u} \times (\mathbf{v} \times \mathbf{w})$ in general.
Cross product with itself is zero: $\mathbf{v} \times \mathbf{v} = \mathbf{0}$.
Cross product of parallel vectors is zero: if $\mathbf{u}$ and $\mathbf{v}$ are parallel, $\mathbf{u} \times \mathbf{v} = \mathbf{0}$.

The non-associativity is a classic gotcha. It’s why parenthesization matters in expressions involving multiple cross products and why the Jacobi identity (the cyclic relation $\mathbf{u} \times (\mathbf{v} \times \mathbf{w}) + \mathbf{v} \times (\mathbf{w} \times \mathbf{u}) + \mathbf{w} \times (\mathbf{u} \times \mathbf{v}) = \mathbf{0}$) is the formal compensation.

Worked Example: Cross Product

Compute $\mathbf{u} \times \mathbf{v}$ for $\mathbf{u} = (1, 2, 3)$ and $\mathbf{v} = (4, 5, 6)$.

Using the formula:

$$\mathbf{u} \times \mathbf{v} = (2 \cdot 6 – 3 \cdot 5, 3 \cdot 4 – 1 \cdot 6, 1 \cdot 5 – 2 \cdot 4) = (-3, 6, -3)$$

Verify perpendicularity:

$\mathbf{u} \cdot (\mathbf{u} \times \mathbf{v}) = 1 \cdot (-3) + 2 \cdot 6 + 3 \cdot (-3) = -3 + 12 – 9 = 0$. ✓

$\mathbf{v} \cdot (\mathbf{u} \times \mathbf{v}) = 4 \cdot (-3) + 5 \cdot 6 + 6 \cdot (-3) = -12 + 30 – 18 = 0$. ✓

The result is perpendicular to both inputs, as required.

Determinant Form for the Cross Product

The cross product can be written as a determinant:

$$\mathbf{u} \times \mathbf{v} = \det \begin{pmatrix} \hat{i} & \hat{j} & \hat{k} \\ u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \end{pmatrix}$$

This is a mnemonic, not a formal determinant (the first row contains vectors, not scalars). Expanding along the first row reproduces the standard cross product formula.

The determinant form makes some properties easier to see. Two parallel vectors give two identical rows, hence zero determinant — explaining why parallel vectors have zero cross product. Switching $\mathbf{u}$ and $\mathbf{v}$ swaps the second and third rows, flipping the sign — explaining anticommutativity.

The Triple Product

The scalar triple product $\mathbf{u} \cdot (\mathbf{v} \times \mathbf{w})$ gives the signed volume of the parallelepiped spanned by the three vectors. It equals the determinant:

$$\mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) = \det \begin{pmatrix} u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \\ w_1 & w_2 & w_3 \end{pmatrix}$$

The triple product is invariant under cyclic permutation: $\mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) = \mathbf{v} \cdot (\mathbf{w} \times \mathbf{u}) = \mathbf{w} \cdot (\mathbf{u} \times \mathbf{v})$. It changes sign under non-cyclic swaps.

The vector triple product $\mathbf{u} \times (\mathbf{v} \times \mathbf{w})$ satisfies the BAC-CAB rule: $\mathbf{u} \times (\mathbf{v} \times \mathbf{w}) = \mathbf{v}(\mathbf{u} \cdot \mathbf{w}) – \mathbf{w}(\mathbf{u} \cdot \mathbf{v})$. This is one of the most useful identities in vector calculus.

Why the Cross Product Only Exists in 3D

In 2D, “perpendicular to two vectors” doesn’t define a unique direction within the plane — the perpendicular points out of the plane. In 4D and higher, “perpendicular to two given vectors” is a multidimensional subspace, not a single direction. So a single perpendicular vector valued operation doesn’t exist there.

The exception is 7D, where a related cross product exists due to the algebraic structure of octonions. This is largely a curiosity of higher mathematics; for everyday work, the cross product is a 3D-only operation.

In 2D, the analogue is the “perpendicular dot product” (also called the wedge product or 2D cross product), which gives a scalar equal to the signed area of the parallelogram. It’s not a vector, but it captures the same area information that the 3D cross product magnitude captures.

Where Dot and Cross Products Show Up

Physics: work = force · displacement (dot product); torque = r × F (cross product); angular momentum L = r × p (cross product); magnetic force F = qv × B (cross product).
Computer graphics: lighting calculations (Lambertian shading uses light direction · normal); backface culling (sign of normal · view direction); surface normal computation (cross product of two edge vectors).
Machine learning: cosine similarity uses dot products to measure similarity between vectors (recommendation systems, document retrieval, embedding alignment).
Engineering: work-energy calculations; structural analysis; fluid mechanics where vorticity involves cross products.
Geometry: determining whether points are on the same side of a line (sign of cross product); area and volume calculations; orientation tests.
Robotics: joint angle computations and end-effector kinematics rely heavily on dot and cross products.

Cosine Similarity in Machine Learning

Cosine similarity measures how aligned two vectors are, ignoring their magnitudes:

$$\text{cos\_sim}(\mathbf{u}, \mathbf{v}) = \frac{\mathbf{u} \cdot \mathbf{v}}{|\mathbf{u}||\mathbf{v}|} = \cos\theta$$

It returns a value between −1 and 1. Cosine similarity is the standard metric for comparing word embeddings, document vectors, image feature vectors, and recommendation systems where direction matters more than scale.

Why magnitude doesn’t matter: in many embedding systems, vector magnitude reflects something other than semantic content (e.g., document length). Stripping out magnitude lets the comparison focus on what the vectors mean rather than how large they are. Read more in best machine learning courses.

Common Mistakes With Dot and Cross Products

Confusing the two operations. Dot product produces a scalar; cross product produces a vector. They have different geometric meanings.
Forgetting that the cross product is anticommutative. $\mathbf{u} \times \mathbf{v} = -(\mathbf{v} \times \mathbf{u})$. The order matters.
Trying to compute a 2D cross product as a vector. The 2D cross product is a scalar (area), not a vector. The vector cross product needs at least 3 dimensions.
Assuming cross product is associative. It isn’t. Use the BAC-CAB rule for vector triple products.
Misapplying the right-hand rule. The fingers point along the first vector and curl toward the second. Reversing this gives the wrong direction.
Forgetting that orthogonal vectors have zero dot product, not zero cross product. Perpendicular vectors have maximum cross product magnitude (they form a parallelogram with the largest possible area).

A Brief History

The dot product traces back to William Rowan Hamilton’s quaternion algebra (1843) and Hermann Grassmann’s exterior algebra (1844). Josiah Willard Gibbs and Oliver Heaviside extracted the modern dot and cross products from these systems in the 1880s, presenting them in the form we use today.

Gibbs’ Vector Analysis (1881) and Heaviside’s Electromagnetic Theory (1893) made vector notation standard in physics and engineering. The modern format — dot product as scalar, cross product as vector — has been the textbook standard for over a century.

Inner Products in General Vector Spaces

The dot product generalizes to inner products on abstract vector spaces. An inner product is any positive-definite, symmetric, bilinear form. Function spaces have inner products like $\langle f, g \rangle = \int f(x) g(x)\, dx$. Polynomial spaces, matrix spaces, and infinite-dimensional Hilbert spaces all support inner-product geometry.

Inner products give length ($\|f\| = \sqrt{\langle f, f \rangle}$) and angle ($\cos\theta = \langle f, g\rangle / (\|f\| \|g\|)$) in any vector space. This is the foundation of Hilbert space theory, Fourier analysis, and quantum mechanics. The familiar Euclidean dot product is just the simplest example of a much broader concept.

The Wedge Product and Its Generalizations

The wedge product (exterior product) generalizes the cross product to any dimension. $\mathbf{u} \wedge \mathbf{v}$ is a “bivector” — an oriented area element — rather than a vector. In 3D, the wedge product is dual to the cross product (a single perpendicular vector encodes the bivector).

Wedge products extend cleanly to higher dimensions and produce the right geometric objects (areas in 2D, volumes in 3D, hypervolumes in nD). They underlie differential forms, exterior calculus, and the most general formulations of multivariable calculus and physics.

Gram-Schmidt Orthogonalization

The Gram-Schmidt process turns any linearly independent set of vectors into an orthonormal set spanning the same subspace. Each step uses dot products to subtract out the components already covered by earlier vectors, leaving a vector orthogonal to all previous ones, which is then normalized to unit length.

Gram-Schmidt is the foundation of QR decomposition and many least-squares regression algorithms. It’s also a clean illustration of dot products in action — every step of the algorithm computes dot products, projections, and subtractions to enforce orthogonality. Numerical implementations use modified Gram-Schmidt or Householder reflections for stability, but the underlying idea is the same.

Cosine Similarity in Practice

Cosine similarity computes $\text{cos\_sim}(\mathbf{u}, \mathbf{v}) = \mathbf{u} \cdot \mathbf{v} / (|\mathbf{u}||\mathbf{v}|)$. It returns values in $[-1, 1]$: 1 means identical direction, 0 means perpendicular, −1 means opposite.

Cosine similarity is the standard tool for comparing high-dimensional embeddings — word vectors (word2vec, GloVe), sentence embeddings (BERT, GPT), image embeddings (CLIP), and user/item vectors in recommender systems. It’s preferred over Euclidean distance because it ignores magnitude, which often reflects irrelevant factors (document length, signal strength).

Computing cosine similarity at scale is well-supported by vector databases (Pinecone, Weaviate, Milvus, FAISS). Approximate nearest-neighbor algorithms speed up cosine-similarity search across billions of vectors.

Worked Example: Torque Calculation

A wrench applies force $\mathbf{F} = (10, 0, 0)$ Newtons at the end of a lever arm $\mathbf{r} = (0, 0.3, 0)$ meters from a bolt. Compute the torque.

Torque $\boldsymbol{\tau} = \mathbf{r} \times \mathbf{F}$:

$$\boldsymbol{\tau} = (0.3 \cdot 0 – 0 \cdot 0, 0 \cdot 10 – 0 \cdot 0, 0 \cdot 0 – 0.3 \cdot 10) = (0, 0, -3) \text{ N·m}$$

The torque is 3 N·m about the negative z-axis. The magnitude is 3 (consistent with $|\mathbf{r}||\mathbf{F}|\sin 90° = 0.3 \cdot 10$). The direction (negative z) tells you the bolt rotates clockwise as seen from above.

Every torque calculation in mechanical engineering uses the cross product this way. The same math underlies angular momentum, magnetic force, and rotational dynamics.

FAQs

What is the difference between dot product and cross product?

Dot product takes two vectors and returns a scalar. Cross product (defined only in 3D) takes two vectors and returns a vector perpendicular to both. Dot product captures alignment and angle; cross product captures area and perpendicular direction.

How do I compute the dot product?

Sum the products of corresponding components: u · v = u₁v₁ + u₂v₂ + … + uₙvₙ. Equivalently, it equals |u||v|cos(θ), where θ is the angle between the vectors. The component formula is the standard way to compute it.

How do I compute the cross product?

Use the formula u × v = (u₂v₃ − u₃v₂, u₃v₁ − u₁v₃, u₁v₂ − u₂v₁) or the determinant mnemonic with î, ĵ, k̂ in the first row. The result is a vector perpendicular to both u and v with magnitude |u||v|sin(θ).

What does it mean if the dot product is zero?

The two vectors are perpendicular (orthogonal), assuming neither is zero. The dot product is the standard test for orthogonality, used everywhere from Gram-Schmidt orthogonalization to neural network weight initialization.

What does it mean if the cross product is zero?

The two vectors are parallel (or one is the zero vector). The cross product magnitude equals the area of the parallelogram spanned by the vectors; parallel vectors form a degenerate parallelogram with zero area.

Why does the cross product only exist in 3D?

Because ‘perpendicular to two given vectors’ uniquely defines a single direction (up to sign) only in 3D. In 2D, perpendicular points out of the plane. In higher dimensions, the perpendicular subspace is multidimensional. The exception is 7D, where a related cross product exists for algebraic reasons.

What is the right-hand rule?

Point your right hand’s fingers along the first vector and curl them toward the second; your thumb points in the direction of the cross product. It’s the convention for orienting cross products in 3D and ensures u, v, u × v form a right-handed coordinate system.

How is the dot product used in machine learning?

Cosine similarity (dot product divided by product of magnitudes) is the standard metric for comparing embeddings, recommendation matching, document similarity, and many other ML tasks. Neural network weights and inputs are combined via dot products at every layer.

How is the cross product used in physics?

Torque = r × F. Angular momentum = r × p. Magnetic force = qv × B. Anywhere rotational or perpendicular geometry matters, cross products show up. In rigid body dynamics, the inertia tensor and angular velocity combine via cross products to give torque equations.

What is cosine similarity?

It’s the cosine of the angle between two vectors: cos_sim(u, v) = (u · v) / (|u| · |v|). It returns a value between −1 (opposite directions) and 1 (same direction), with 0 indicating perpendicular. Used everywhere from search engines to recommendation systems to embedding space analysis.

Is the dot product commutative?

Yes. u · v = v · u. The dot product is symmetric in its two arguments. The cross product, by contrast, is anticommutative: u × v = −(v × u).

Can the dot product be negative?

Yes. Negative dot product means the vectors form an obtuse angle (greater than 90°). It happens naturally when vectors point in roughly opposite directions. Cosine similarity returns negative values in this case.

What’s the geometric meaning of the dot product?

It equals |u||v|cos(θ). For unit vectors, it’s just the cosine of the angle between them. Geometrically, it captures how aligned the vectors are. Positive means roughly aligned; negative means roughly opposite; zero means perpendicular.

What’s the difference between cross product and outer product?

Cross product takes two vectors and returns a third vector (in 3D). Outer product takes two vectors and returns a matrix: (u v^T)_{ij} = u_i v_j. The outer product is sometimes called the tensor product or Kronecker product in different contexts.