In modern control, vectors are not merely “arrows” in a plane. Typical objects include: (i) state vectors \( x \in \mathbb{R}^n \), (ii) input vectors \( u \in \mathbb{R}^m \), (iii) output vectors \( y \in \mathbb{R}^p \), and (iv) function signals (e.g., trajectories) that live in infinite-dimensional spaces.

The central promise of linear algebra is that “linearity” is a structure, not a coordinate system: we can reason abstractly in a vector space and only later choose a basis (coordinates). This separation is essential in state-space thinking because internal variables may be measured, transformed, or represented in many equivalent coordinate systems.

flowchart TD
  A["Physical/abstract quantities"] --> B["Represent as vectors (states, inputs, outputs, signals)"]
  B --> C["Define a vector space: add + scale"]
  C --> D["Identify subspaces: constraints + structure"]
  D --> E["Choose a basis: coordinates for computation"]
  E --> F["Dimension: number of independent coordinates"]
  F --> G["Algorithms: build basis, test independence, compute coordinates"]
        

Throughout, we work over a field \( \mathbb{F} \) (typically \( \mathbb{R} \) or \( \mathbb{C} \)).

A vector space over a field \( \mathbb{F} \) is a set \( V \) equipped with: vector addition \( +:V\times V \to V \) and scalar multiplication \( \cdot:\mathbb{F}\times V \to V \), satisfying the axioms below for all \( u,v,w \in V \) and all \( a,b \in \mathbb{F} \).

\[ \begin{aligned} &\textbf{(A1) Closure (addition):}\quad u+v \in V,\qquad \textbf{(A2) Commutativity:}\quad u+v = v+u,\\ &\textbf{(A3) Associativity:}\quad (u+v)+w = u+(v+w),\\ &\textbf{(A4) Additive identity:}\quad \exists 0\in V:\; v+0=v,\\ &\textbf{(A5) Additive inverse:}\quad \forall v\;\exists (-v)\in V:\; v+(-v)=0,\\ &\textbf{(A6) Closure (scalar mult.):}\quad a v \in V,\qquad \textbf{(A7) Distributivity:}\quad a(u+v)=au+av,\\ &\textbf{(A8) Distributivity:}\quad (a+b)v=av+bv,\qquad \textbf{(A9) Compatibility:}\quad a(bv)=(ab)v,\\ &\textbf{(A10) Scalar identity:}\quad 1v=v. \end{aligned} \]

Examples used constantly in control engineering:

• Euclidean space \( \mathbb{R}^n \) with usual addition/scalar multiplication. State vectors are typically in this space.
• Complex space \( \mathbb{C}^n \), useful for modal/eigenvalue analysis (later lessons).
• Matrix space \( \mathbb{R}^{p\times q} \), the set of real matrices with entrywise operations.
• Polynomial space \( \mathcal{P}_d \), polynomials of degree at most \( d \).
• Signal spaces (conceptual): sets of functions with pointwise addition and scalar multiplication. (We will formalize particular signal spaces only when needed later.)

Two immediate derived facts that will be used repeatedly:

Fact 1: \( 0 \) is unique. Fact 2: additive inverses are unique.

Proof sketch (uniqueness of zero): Suppose \( 0 \) and \( 0' \) are both additive identities. Then

\[ 0 = 0 + 0' = 0' \]

using identity properties, hence they are equal. The inverse uniqueness is similar: if \( v+w=0 \) and \( v+w'=0 \), then \( w = w+0 = w+(v+w') = (w+v)+w' = 0+w' = w' \).

A subset \( W \subseteq V \) is a subspace of a vector space \( V \) if \( W \) is itself a vector space with the operations inherited from \( V \).

The most efficient criterion is the subspace test.

Theorem (Subspace Test): Let \( V \) be a vector space over \( \mathbb{F} \), and let \( W \subseteq V \). Then \( W \) is a subspace of \( V \) if and only if:

• \( 0 \in W \)
• For all \( u,v \in W \), \( u+v \in W \)
• For all \( a \in \mathbb{F} \) and \( v \in W \), \( av \in W \)

Proof:

(Only if) If \( W \) is a vector space with inherited operations, then it must contain its additive identity, which equals the zero vector of \( V \). Closure under addition and scalar multiplication are axioms, so both hold.

(If) Assume the three bullet conditions. We must show the vector space axioms hold in \( W \). The axioms that mention only equalities (commutativity, associativity, distributivity, etc.) hold because they already hold in \( V \) and the operations are the same. The only axioms that require membership in the set are precisely the closure axioms, guaranteed by the bullets, plus existence of additive inverses: for \( v\in W \), we have \( (-1)v \in W \) by scalar closure, and \( v + (-1)v = 0 \). Hence \( W \) is a vector space, i.e., a subspace.

Canonical subspace constructions:

(i) Intersection: If \( W_1, W_2 \) are subspaces of \( V \), then \( W_1 \cap W_2 \) is a subspace.

Proof: Since \( 0 \in W_1 \) and \( 0 \in W_2 \), we have \( 0 \in W_1 \cap W_2 \). If \( u,v \in W_1 \cap W_2 \), then \( u,v \in W_1 \) and \( u,v \in W_2 \), so \( u+v \in W_1 \) and \( u+v \in W_2 \), hence \( u+v \in W_1 \cap W_2 \). Similarly, for \( a \in \mathbb{F} \), \( au \in W_1 \cap W_2 \). Subspace test applies.

(ii) Sum of subspaces: Define \( W_1 + W_2 := \{w_1+w_2 \mid w_1\in W_1,\; w_2\in W_2\} \). Then \( W_1 + W_2 \) is a subspace. (We will use this construction later to decompose spaces into “parts”.)

Let \( v_1,\dots,v_k \in V \). A linear combination is any vector of the form \( a_1 v_1 + \cdots + a_k v_k \) with scalars \( a_i \in \mathbb{F} \).

The span of a set \( S \subseteq V \) is the set of all finite linear combinations of elements of \( S \), denoted \( \mathrm{span}(S) \).

\[ \mathrm{span}(S) := \left\{\sum_{i=1}^{k} a_i s_i \; \middle|\; k \in \mathbb{N},\; s_i\in S,\; a_i\in\mathbb{F}\right\}. \]

Proposition: \( \mathrm{span}(S) \) is a subspace of \( V \), and it is the smallest subspace containing \( S \).

Proof: First, \( 0 \in \mathrm{span}(S) \) using the “empty”/trivial combination (or take all coefficients zero). If \( x=\sum_{i=1}^{k} a_i s_i \) and \( y=\sum_{j=1}^{\ell} b_j t_j \), then

\[ x+y = \sum_{i=1}^{k} a_i s_i + \sum_{j=1}^{\ell} b_j t_j \in \mathrm{span}(S) \]

because it is still a finite linear combination of elements of \( S \). For scalar \( c \in \mathbb{F} \),

\[ cx = c\sum_{i=1}^{k} a_i s_i = \sum_{i=1}^{k} (ca_i)s_i \in \mathrm{span}(S). \]

So \( \mathrm{span}(S) \) is a subspace by the subspace test. If \( W \) is any subspace containing \( S \), then it contains all linear combinations of elements of \( S \) (by closure), hence \( \mathrm{span}(S)\subseteq W \). Therefore, \( \mathrm{span}(S) \) is the smallest such subspace.

A set \( \{v_1,\dots,v_k\} \) is linearly independent if the only solution to

\[ a_1 v_1 + \cdots + a_k v_k = 0 \]

is \( a_1=\cdots=a_k=0 \). Otherwise, it is linearly dependent.

Equivalent characterization (useful in proofs and computation): \( \{v_1,\dots,v_k\} \) is linearly dependent if and only if one of the vectors can be expressed as a linear combination of the others.

Proof: If dependent, there exist coefficients not all zero with \( \sum_{i=1}^{k} a_i v_i=0 \). Choose an index \( j \) such that \( a_j \neq 0 \). Then

\[ v_j = -\sum_{i\neq j}\frac{a_i}{a_j} v_i, \]

expressing \( v_j \) as a combination of the others. Conversely, if \( v_j=\sum_{i\neq j} c_i v_i \), then \( \sum_{i\neq j} c_i v_i - v_j=0 \) gives a nontrivial dependence relation.

A basis of a vector space \( V \) is a list of vectors \( (b_1,\dots,b_n) \) such that: (i) they span \( V \), and (ii) they are linearly independent.

If \( V \) has a finite basis with \( n \) vectors, we say \( V \) is finite-dimensional and define \( \dim(V)=n \).

Theorem (Uniqueness of coordinates in a basis): If \( (b_1,\dots,b_n) \) is a basis of \( V \), then every \( v\in V \) can be written uniquely as \( v=\sum_{i=1}^{n} \alpha_i b_i \).

Proof: Existence holds because the basis spans \( V \). For uniqueness, suppose \( \sum_{i=1}^{n} \alpha_i b_i = \sum_{i=1}^{n} \beta_i b_i \). Subtract to obtain

\[ \sum_{i=1}^{n} (\alpha_i-\beta_i)b_i = 0. \]

Since the basis vectors are independent, all coefficients must be zero, so \( \alpha_i=\beta_i \) for all \( i \).

Corollary (Coordinate map): Fix a basis \( B=(b_1,\dots,b_n) \). Define \( [v]_B := (\alpha_1,\dots,\alpha_n)^\top \in \mathbb{F}^n \) where \( v=\sum_{i=1}^n \alpha_i b_i \). Then the mapping \( v \mapsto [v]_B \) is one-to-one and onto.

Next we address the key structural invariant: the number of basis vectors cannot depend on which basis you choose.

Lemma (Steinitz Exchange Lemma, finite-dimensional form): Let \( (u_1,\dots,u_m) \) be linearly independent in \( V \), and let \( (w_1,\dots,w_n) \) span \( V \). Then \( m \le n \).

Proof: Since \( (w_1,\dots,w_n) \) spans \( V \), we can write \( u_1=\sum_{i=1}^n a_i w_i \). At least one coefficient is nonzero; pick an index \( j_1 \) with \( a_{j_1}\neq 0 \). Solve for \( w_{j_1} \):

\[ w_{j_1} = \frac{1}{a_{j_1}}u_1 - \sum_{i\neq j_1}\frac{a_i}{a_{j_1}} w_i. \]

Therefore, replacing \( w_{j_1} \) by \( u_1 \) preserves the spanning property. Define a new spanning list \( W^{(1)} \) of length \( n \) that contains \( u_1 \).

Now consider \( u_2 \). Express it in the span of \( W^{(1)} \). If it already lies in the span of the remaining \( n-1 \) vectors excluding \( u_1 \), then \( u_2 \) would be in \( \mathrm{span}(u_1,\ldots) \), contradicting independence of \( (u_1,u_2) \). Hence we can again exchange one vector to include \( u_2 \) while maintaining a spanning list. Repeating this process, after \( m \) steps we obtain a spanning list of length \( n \) that contains \( u_1,\dots,u_m \). This is only possible if \( m \le n \).

Theorem (Dimension is well-defined): If \( V \) has a finite basis, then any two bases of \( V \) have the same number of vectors.

Proof: Let \( B \) be a basis with \( n \) vectors and \( C \) be a basis with \( m \) vectors. Since \( B \) spans and \( C \) is independent, the exchange lemma gives \( m \le n \). Similarly, since \( C \) spans and \( B \) is independent, we get \( n \le m \). Hence \( m=n \).

Interpretation for state vectors: if the internal description uses \( n \) independent state variables, then any coordinate representation that still describes the same internal degrees of freedom must have \( n \) coordinates (dimension is invariant).

In engineering workflows, we often begin with a finite set of vectors \( v_1,\dots,v_k \in \mathbb{R}^n \) (e.g., measured features, candidate states, or model-derived vectors) and want:

• a subset that forms a basis for \( \mathrm{span}(v_1,\dots,v_k) \), and
• the dimension of that span (the number of basis vectors).

A standard approach uses row-reduction (Gaussian elimination) on the matrix \( A=[v_1\;\;v_2\;\;\cdots\;\;v_k] \in \mathbb{R}^{n\times k} \). Pivot columns identify a linearly independent subset of the original columns. The number of pivots equals \( \dim(\mathrm{span}(v_1,\dots,v_k)) \).

flowchart TD
  S["Given vectors v1..vk in R^n"] --> M["Form matrix A = [v1 v2 ... vk]"]
  M --> RR["Row-reduce A (Gaussian elimination)"]
  RR --> P["Identify pivot columns"]
  P --> B["Basis = original vectors in pivot columns"]
  P --> D["Dimension = number of pivots"]
  B --> C["Use basis to compute coordinates by solving B*c = v"]
        

Note on numerical practice: For floating-point data, pivot decisions require a tolerance. In later lessons we will revisit numerical conditioning more systematically; here we keep computations conceptually aligned with the theory.

The goal of the implementations is consistent across languages: (i) perform row-reduction, (ii) extract pivot columns as a basis of the span, and (iii) compute the dimension (pivot count). We keep the mathematics explicit rather than relying entirely on “black-box” rank routines, to match the concepts developed above.

7.1 Python (NumPy + optional control stack)

Libraries commonly used in modern control workflows in Python include numpy, scipy, and control. In this lesson we only need core linear algebra operations (numpy).

import numpy as np

def rref(A, tol=1e-10):
    """
    Reduced Row Echelon Form via Gauss-Jordan elimination.
    Returns (R, pivot_cols).
    A is copied to float.
    """
    R = A.astype(float).copy()
    m, n = R.shape
    pivot_cols = []
    row = 0

    for col in range(n):
        if row >= m:
            break

        # Find pivot row with max abs value in current column (partial pivoting)
        pivot = np.argmax(np.abs(R[row:, col])) + row
        if abs(R[pivot, col]) < tol:
            continue

        # Swap pivot row into position
        if pivot != row:
            R[[row, pivot]] = R[[pivot, row]]

        # Normalize pivot row
        R[row] = R[row] / R[row, col]

        # Eliminate other entries in this column
        for r in range(m):
            if r != row:
                R[r] = R[r] - R[r, col] * R[row]

        pivot_cols.append(col)
        row += 1

    # Clean tiny numerical noise
    R[np.abs(R) < tol] = 0.0
    return R, pivot_cols

def basis_from_columns(V, tol=1e-10):
    """
    V: matrix whose columns are candidate vectors.
    Returns (basis_cols, pivot_cols, R).
    """
    R, piv = rref(V, tol=tol)
    basis_cols = V[:, piv] if len(piv) > 0 else V[:, :0]
    return basis_cols, piv, R

# Example: 4D vectors v1..v4 (columns)
V = np.array([
    [1, 2, 0, 1],
    [0, 1, 1, 1],
    [1, 3, 1, 2],
    [0, 0, 1, 1],
], dtype=float)

B, piv, R = basis_from_columns(V)
print("Pivot columns:", piv)
print("Dimension of span:", len(piv))
print("Basis vectors (as columns):\n", B)
print("RREF:\n", R)

# Coordinate representation in the extracted basis:
# Given a target vector v, solve B*c = v if v is in span(B).
v = np.array([3, 2, 5, 2], dtype=float)
if B.shape[1] > 0:
    # Solve least squares; exact if v in span(B)
    c, residuals, _, _ = np.linalg.lstsq(B, v, rcond=None)
    print("Coordinates c (least-squares):", c)
    print("Reconstruction B*c:", B @ c)
    print("Residual norm:", np.linalg.norm(B @ c - v))
      

7.2 C++ (Eigen) — Basis extraction with Gauss–Jordan elimination

C++ control and estimation stacks frequently use Eigen for linear algebra (and, depending on the course later, libraries such as Control Toolbox or custom state-space code). Here we implement RREF explicitly.

#include <iostream>
#include <vector>
#include <Eigen/Dense>

struct RrefResult {
  Eigen::MatrixXd R;
  std::vector<int> pivots;
};

RrefResult rref(const Eigen::MatrixXd& A, double tol = 1e-10) {
  Eigen::MatrixXd R = A;
  const int m = (int)R.rows();
  const int n = (int)R.cols();

  std::vector<int> pivots;
  int row = 0;

  for (int col = 0; col < n && row < m; ++col) {
    // Partial pivoting: pick row with largest magnitude in column
    int pivot = row;
    double maxAbs = std::abs(R(row, col));
    for (int r = row + 1; r < m; ++r) {
      double val = std::abs(R(r, col));
      if (val > maxAbs) {
        maxAbs = val;
        pivot = r;
      }
    }
    if (maxAbs < tol) continue;

    if (pivot != row) R.row(row).swap(R.row(pivot));

    // Normalize pivot row
    double piv = R(row, col);
    R.row(row) /= piv;

    // Eliminate other rows
    for (int r = 0; r < m; ++r) {
      if (r == row) continue;
      double factor = R(r, col);
      R.row(r) -= factor * R.row(row);
    }

    pivots.push_back(col);
    ++row;
  }

  // Zero out tiny noise
  for (int i = 0; i < m; ++i)
    for (int j = 0; j < n; ++j)
      if (std::abs(R(i, j)) < tol) R(i, j) = 0.0;

  return {R, pivots};
}

int main() {
  Eigen::MatrixXd V(4,4);
  V << 1,2,0,1,
       0,1,1,1,
       1,3,1,2,
       0,0,1,1;

  auto res = rref(V);
  std::cout << "Pivot columns: ";
  for (int c : res.pivots) std::cout << c << " ";
  std::cout << "\nDimension of span: " << res.pivots.size() << "\n";

  std::cout << "RREF:\n" << res.R << "\n";

  // Extract basis from original columns
  Eigen::MatrixXd B(V.rows(), (int)res.pivots.size());
  for (int i = 0; i < (int)res.pivots.size(); ++i) {
    B.col(i) = V.col(res.pivots[i]);
  }
  std::cout << "Basis columns:\n" << B << "\n";
  return 0;
}
      

7.3 Java (EJML) — Row-reduction and pivot extraction

In Java-based engineering stacks, EJML is a practical linear algebra library. The code below implements Gauss–Jordan elimination in terms of EJML’s dense matrices.

import org.ejml.data.DMatrixRMaj;
import java.util.ArrayList;
import java.util.List;

public class RREF {
    public static class Result {
        public DMatrixRMaj R;
        public List<Integer> pivots;
        public Result(DMatrixRMaj R, List<Integer> pivots) {
            this.R = R; this.pivots = pivots;
        }
    }

    public static Result rref(DMatrixRMaj A, double tol) {
        DMatrixRMaj R = A.copy();
        int m = R.numRows;
        int n = R.numCols;

        List<Integer> pivots = new ArrayList<>();
        int row = 0;

        for (int col = 0; col < n && row < m; col++) {
            // Pivot selection
            int pivot = row;
            double maxAbs = Math.abs(R.get(row, col));
            for (int r = row + 1; r < m; r++) {
                double v = Math.abs(R.get(r, col));
                if (v > maxAbs) { maxAbs = v; pivot = r; }
            }
            if (maxAbs < tol) continue;

            // Swap rows
            if (pivot != row) {
                for (int j = 0; j < n; j++) {
                    double tmp = R.get(row, j);
                    R.set(row, j, R.get(pivot, j));
                    R.set(pivot, j, tmp);
                }
            }

            // Normalize pivot row
            double piv = R.get(row, col);
            for (int j = 0; j < n; j++) R.set(row, j, R.get(row, j) / piv);

            // Eliminate other rows
            for (int r = 0; r < m; r++) {
                if (r == row) continue;
                double factor = R.get(r, col);
                for (int j = 0; j < n; j++) {
                    R.set(r, j, R.get(r, j) - factor * R.get(row, j));
                }
            }

            pivots.add(col);
            row++;
        }

        // Clean small values
        for (int i = 0; i < m; i++)
            for (int j = 0; j < n; j++)
                if (Math.abs(R.get(i, j)) < tol) R.set(i, j, 0.0);

        return new Result(R, pivots);
    }

    public static void main(String[] args) {
        DMatrixRMaj V = new DMatrixRMaj(new double[][]{
            {1,2,0,1},
            {0,1,1,1},
            {1,3,1,2},
            {0,0,1,1}
        });

        Result res = rref(V, 1e-10);
        System.out.println("Pivot columns: " + res.pivots);
        System.out.println("Dimension of span: " + res.pivots.size());
        System.out.println("RREF:");
        res.R.print();
    }
}
      

7.4 MATLAB + Simulink

MATLAB natively supports row-reduction via rref. The pivot columns of rref(A) correspond to a linearly independent subset of the original columns (subject to numerical tolerance).

% Candidate vectors as columns
V = [1 2 0 1;
     0 1 1 1;
     1 3 1 2;
     0 0 1 1];

[R, piv] = rref(V);   % piv: pivot column indices
dimSpan = numel(piv);
B = V(:, piv);

disp('Pivot columns:'); disp(piv);
disp('Dimension of span:'); disp(dimSpan);
disp('Basis columns:'); disp(B);
disp('RREF:'); disp(R);

% Coordinate representation (solve B*c = v if v is in span(B))
v = [3;2;5;2];
c = B \ v;            % exact if B is square/invertible; otherwise use least squares below
c_ls = lsqr(B, v);    % least squares coordinates
disp('Least-squares coordinates:'); disp(c_ls);
disp('Residual norm:'); disp(norm(B*c_ls - v));
      

Simulink note (vector construction and linear combinations):

Even before introducing state-space models formally, Simulink can represent vectors and linear combinations: use blocks such as Mux (to form a vector), Gain configured as a matrix (to apply a matrix), and Sum blocks for vector addition. Conceptually, these implement the operations required by a vector space: addition and scalar/matrix scaling. (We will formalize “matrix as a linear mapping” in Lesson 2.)

% Programmatically create a minimal Simulink model that forms a 2-vector and scales it
model = 'vector_space_demo';
new_system(model); open_system(model);

add_block('simulink/Sources/Sine Wave', [model '/s1']);
add_block('simulink/Sources/Sine Wave', [model '/s2']);
add_block('simulink/Signal Routing/Mux', [model '/Mux'], 'Inputs', '2');
add_block('simulink/Math Operations/Gain', [model '/MatrixGain'], 'Gain', '[2 0; 0 0.5]');
add_block('simulink/Sinks/Scope', [model '/Scope']);

set_param([model '/s1'], 'Position', [30 40 60 60]);
set_param([model '/s2'], 'Position', [30 100 60 120]);
set_param([model '/Mux'], 'Position', [120 55 150 115]);
set_param([model '/MatrixGain'], 'Position', [210 70 260 100]);
set_param([model '/Scope'], 'Position', [320 70 350 100]);

add_line(model, 's1/1', 'Mux/1');
add_line(model, 's2/1', 'Mux/2');
add_line(model, 'Mux/1', 'MatrixGain/1');
add_line(model, 'MatrixGain/1', 'Scope/1');

save_system(model);
      

7.5 Wolfram Mathematica

Mathematica provides symbolic and numeric row-reduction via RowReduce. Pivot columns can be inferred from leading ones.

V = { {1, 2, 0, 1},
     {0, 1, 1, 1},
     {1, 3, 1, 2},
     {0, 0, 1, 1} };

R = RowReduce[V];
R

(* Identify pivot columns by scanning for leading 1s *)
pivotCols = Module[{m = Length[R], n = Length[R[[1]]]},
  Module[{p = {}, leadPos},
    Do[
      leadPos = FirstPosition[R[[i]], 1, Missing["NotFound"]];
      If[leadPos =!= Missing["NotFound"], AppendTo[p, leadPos[[1]]]],
      {i, 1, m}
    ];
    DeleteDuplicates[p]
  ]
];

pivotCols
dimSpan = Length[pivotCols]
B = V[[All, pivotCols]]  (* basis columns as a matrix *)

(* Coordinate solve: least squares (if needed) *)
v = {3, 2, 5, 2};
c = LeastSquares[B, v];
{c, B.c, Norm[B.c - v]}
      

Problem 1 (Subspace test in practice): Let \( V=\mathbb{R}^3 \) and define \( W=\{(x_1,x_2,x_3)\in\mathbb{R}^3 \mid x_1 - 2x_2 + x_3 = 0\} \). Show \( W \) is a subspace, find a basis, and compute \( \dim(W) \).

Solution: First, \( (0,0,0)\in W \) since \( 0-0+0=0 \). If \( x=(x_1,x_2,x_3)\in W \) and \( y=(y_1,y_2,y_3)\in W \), then

\[ (x_1+y_1) - 2(x_2+y_2) + (x_3+y_3) = (x_1-2x_2+x_3) + (y_1-2y_2+y_3) = 0, \]

so \( x+y\in W \). For scalar \( a\in\mathbb{R} \),

\[ (ax_1) - 2(ax_2) + (ax_3) = a(x_1-2x_2+x_3)=0, \]

hence \( ax\in W \). By the subspace test, \( W \) is a subspace. To find a basis, parametrize: from \( x_1=2x_2-x_3 \), let \( x_2=s \), \( x_3=t \). Then

\[ (x_1,x_2,x_3) = (2s-t,\, s,\, t) = s(2,1,0) + t(-1,0,1). \]

So \( W=\mathrm{span}\{(2,1,0),(-1,0,1)\} \). These two vectors are independent (a short check shows \( a(2,1,0)+b(-1,0,1)=0 \) implies \( a=b=0 \)). Therefore a basis is \( \{(2,1,0),(-1,0,1)\} \) and \( \dim(W)=2 \).

Problem 2 (Span is the smallest subspace): Prove that for any subset \( S\subseteq V \), \( \mathrm{span}(S) \) is the smallest subspace containing \( S \).

Solution: The proof was established in Section 4. Briefly: (i) subspace: closure follows because sums and scalar multiples of finite linear combinations remain finite linear combinations; (ii) minimality: any subspace containing \( S \) must contain all linear combinations of elements of \( S \), hence must contain \( \mathrm{span}(S) \).

Problem 3 (Linear independence via uniqueness of representation): Let \( B=(b_1,\dots,b_n) \) be a basis of \( V \). Prove that if \( \sum_{i=1}^n \alpha_i b_i = 0 \), then all coefficients are zero.

Solution: Since \( 0=\sum_{i=1}^n 0\cdot b_i \) is a representation of \( 0 \), uniqueness of coordinates in a basis implies \( \alpha_i = 0 \) for all \( i \). Equivalently, the basis vectors are linearly independent by definition, which yields the same conclusion.

Problem 4 (Two bases have equal size): Assume \( V \) is finite-dimensional. Use the Steinitz exchange lemma to prove that any two bases of \( V \) have the same number of vectors.

Solution: Let \( B \) be a basis with \( n \) vectors and \( C \) be a basis with \( m \) vectors. Since \( B \) spans and \( C \) is independent, Steinitz implies \( m \le n \). Since \( C \) spans and \( B \) is independent, Steinitz implies \( n \le m \). Hence \( m=n \).

Problem 5 (Dimension of a sum): Let \( U \) and \( W \) be subspaces of a finite-dimensional vector space \( V \). Prove the dimension identity: \( \dim(U+W) = \dim(U) + \dim(W) - \dim(U\cap W) \).

Solution: Let \( (c_1,\dots,c_r) \) be a basis of \( U\cap W \). Extend it to a basis of \( U \): \( (c_1,\dots,c_r,u_{r+1},\dots,u_p) \), and extend it to a basis of \( W \): \( (c_1,\dots,c_r,w_{r+1},\dots,w_q) \). We claim that the concatenated list \( (c_1,\dots,c_r,u_{r+1},\dots,u_p,w_{r+1},\dots,w_q) \) is a basis of \( U+W \).

Spanning: any \( z\in U+W \) can be written as \( z=u+w \) with \( u\in U \), \( w\in W \). Expand \( u \) in the basis of \( U \) and \( w \) in the basis of \( W \); adding gives an expression in the concatenated list.

Independence: suppose

\[ \sum_{i=1}^{r}\alpha_i c_i + \sum_{i=r+1}^{p}\beta_i u_i + \sum_{j=r+1}^{q}\gamma_j w_j = 0. \]

Rearrange as \( u' + w' = 0 \) where \( u'=\sum_{i=1}^{r}\alpha_i c_i + \sum_{i=r+1}^{p}\beta_i u_i \in U \) and \( w'=\sum_{j=r+1}^{q}\gamma_j w_j \in W \). Then \( u'=-w' \in U\cap W \). Since \( (c_1,\dots,c_r) \) is a basis of \( U\cap W \), the vector \( u' \) can be written using only the \( c_i \). But the representation of \( u' \) in the basis of \( U \) is unique, so all \( \beta_i=0 \). Similarly, uniqueness in the basis of \( W \) forces all \( \gamma_j=0 \). Finally, with \( \beta_i=\gamma_j=0 \), we have \( \sum_{i=1}^r \alpha_i c_i=0 \), implying \( \alpha_i=0 \). Hence the concatenated list is independent and thus a basis.

Therefore, \( \dim(U+W)= r + (p-r) + (q-r) = p+q-r = \dim(U) + \dim(W) - \dim(U\cap W) \).

Problem 6 (Basis in a polynomial space): Let \( \mathcal{P}_2 \) be the vector space of real polynomials of degree at most 2. Show that \( \{1,\, t,\, t^2\} \) is a basis and compute \( \dim(\mathcal{P}_2) \).

Solution: Any polynomial \( p(t)=a_0+a_1 t+a_2 t^2 \) is a linear combination of \( 1,t,t^2 \), so the set spans. If \( \alpha_0\cdot 1 + \alpha_1 t + \alpha_2 t^2 = 0 \) as a polynomial identity for all \( t \), then all coefficients must be zero, i.e., \( \alpha_0=\alpha_1=\alpha_2=0 \), so the set is independent. Hence it is a basis and \( \dim(\mathcal{P}_2)=3 \).

We defined vector spaces and subspaces rigorously, proved the subspace test and key closure constructions, developed span and linear independence, and established the basis concept as “independent spanning.” Using the exchange lemma, we proved that dimension is a well-defined invariant of the space. Finally, we connected the theory to computation by extracting a basis (and the dimension of the span) via row-reduction, with implementations in Python, C++, Java, MATLAB/Simulink, and Wolfram Mathematica.