In Lesson 1, uncertainty was separated into parametric uncertainty, unmodeled dynamics, and disturbances. In this lesson we focus on the parametric part. A model is called linearly parameterized if the unknown constants appear linearly, even when the measured signals enter through nonlinear basis functions.

The canonical scalar regression form is \( y(t)=\theta^T\phi(x(t),u(t)) \), where \( y(t) \) is a measured or constructed output, \( \theta\in\mathbb{R}^p \) is an unknown constant parameter vector, and \( \phi(x,u)\in\mathbb{R}^p \) is a known regressor vector built from measurable signals.

\[ y(t)=\theta^T\phi(x(t),u(t)) = \sum_{i=1}^{p}\theta_i\phi_i(x(t),u(t)) \]

The phrase linear in the parameters does not mean that the plant is linear in states or inputs. For example, \( \phi(x,u)=[x,\;u,\;x^2,\;\sin x,\;xu]^T \) produces a nonlinear signal model, but the unknown constants still multiply known functions linearly.

flowchart TD
  A["Measured signals: x(t), u(t)"] --> B["Known basis functions"]
  B --> C["Regressor vector: phi(x,u)"]
  C --> D["Unknown constants: theta"]
  D --> E["Prediction: y_hat = theta_hat^T phi"]
  C --> E
  E --> F["Prediction error: y - y_hat"]
  F --> G["Parameter error enters linearly"]
        

Let \( z=(x,u) \) denote all measurable signals used to build the model. A scalar mapping \( f(z,\theta) \) is linearly parameterized if it can be written as

\[ f(z,\theta)=\theta^T\phi(z) = \theta_1\phi_1(z)+\theta_2\phi_2(z)+\cdots+\theta_p\phi_p(z) \]

The basis functions \( \phi_i(z) \) may be nonlinear, discontinuous, polynomial, trigonometric, or physically derived, provided they are known functions of measurable quantities. The unknowns are only the coefficients \( \theta_i \).

A model such as \( y=\theta_1x+\theta_2u+\theta_3x^2 \) is linear in \( \theta \) but nonlinear in \( x \). By contrast, \( y=\theta_1\sin(\theta_2x) \) is not linearly parameterized because \( \theta_2 \) is inside a sine function.

\[ \begin{aligned} y &= \theta_1x+\theta_2u+\theta_3x^2 &&\text{linear in } \theta \\[2mm] y &= \theta_1\sin(\theta_2x) &&\text{not linear in } \theta \end{aligned} \]

The distinction is crucial: adaptive laws and parameter estimators can exploit the algebraic relation between prediction error and parameter error only when the unknown coefficients enter linearly.

Suppose the measured scalar signal \( y(t) \) is modeled by

\[ y(t)=\theta^T\phi(t)+\varepsilon(t), \qquad \theta,\phi(t)\in\mathbb{R}^{p}, \quad \varepsilon(t)\in\mathbb{R} \]

Here \( \varepsilon(t) \) collects measurement noise, modeling residuals, and the part of the dynamics not represented by the chosen basis. With sampled data \( \{y_k,\phi_k\}_{k=1}^{N} \), stack the equations:

\[ \mathbf{y} = \Phi\theta+\boldsymbol{\varepsilon}, \qquad \mathbf{y}\in\mathbb{R}^{N}, \quad \Phi\in\mathbb{R}^{N\times p}, \quad \theta\in\mathbb{R}^{p} \]

The data matrix is

\[ \Phi= \begin{bmatrix} \phi_1^T\\ \phi_2^T\\ \vdots\\ \phi_N^T \end{bmatrix} = \begin{bmatrix} \phi_1(x_1,u_1) & \phi_2(x_1,u_1) & \cdots & \phi_p(x_1,u_1)\\ \phi_1(x_2,u_2) & \phi_2(x_2,u_2) & \cdots & \phi_p(x_2,u_2)\\ \vdots & \vdots & \ddots & \vdots\\ \phi_1(x_N,u_N) & \phi_2(x_N,u_N) & \cdots & \phi_p(x_N,u_N) \end{bmatrix} \]

A vector-valued model can be written as \( \mathbf{y}=Y(z)\theta \), where \( \mathbf{y}\in\mathbb{R}^{m} \) and \( Y(z)\in\mathbb{R}^{m\times p} \) is often called a regressor matrix rather than a regressor vector.

\[ \mathbf{y}(t)=Y(x(t),u(t))\theta, \qquad Y(x,u)\in\mathbb{R}^{m\times p} \]

Adaptive control does not usually estimate arbitrary curve-fitting coefficients. The parameterization should come from the physics or the controller structure. Below are common linearly parameterized forms.

4.1 First-Order Plant with Unknown Coefficients

Consider a first-order plant with unknown constants \( a \) and \( b \):

\[ \dot{x}(t)=a x(t)+b u(t) \]

Define \( \theta=[a,\;b]^T \) and \( \phi(x,u)=[x,\;u]^T \). Then

\[ \dot{x}(t)=\theta^T\phi(x(t),u(t)) \]

This is the simplest example. Later, in MRAC, the same structural idea will appear inside tracking-error equations.

4.2 Mass-Damper System

For a translational mass-damper model,

\[ m\ddot{q}+c\dot{q}=u \]

If acceleration and velocity are available or estimated, the dynamics can be rearranged as

\[ u= \begin{bmatrix} \ddot{q} & \dot{q} \end{bmatrix} \begin{bmatrix} m\\c \end{bmatrix} = Y(q,\dot{q},\ddot{q})\theta \]

The unknown physical constants are \( \theta=[m,\;c]^T \). The regressor is known when \( \dot{q} \) and \( \ddot{q} \) are known or reconstructed.

4.3 Polynomial Nonlinearity with Unknown Coefficients

Suppose an unknown scalar nonlinear term is approximated by a polynomial with known basis:

\[ f(x)=\theta_1x+\theta_2x^2+\theta_3x^3 = \theta^T \begin{bmatrix} x\\x^2\\x^3 \end{bmatrix} \]

Again, the model is nonlinear in \( x \), but linear in \( \theta \).

Let \( \hat{\theta} \) be an estimate of the unknown parameter vector. The model prediction is

\[ \hat{y}(t)=\hat{\theta}^T\phi(t) \]

Define the parameter error and prediction error as

\[ \tilde{\theta}=\theta-\hat{\theta}, \qquad e_y(t)=y(t)-\hat{y}(t) \]

For the noise-free model \( y=\theta^T\phi \),

\[ e_y(t)=\theta^T\phi(t)-\hat{\theta}^T\phi(t) = \tilde{\theta}^T\phi(t) \]

This identity is one of the main reasons linearly parameterized models dominate classical adaptive control. The error is linear in the unknown parameter error, which allows Lyapunov-based cancellation terms and gradient-based estimation laws in later chapters.

With residual uncertainty \( \varepsilon(t) \),

\[ y(t)=\theta^T\phi(t)+\varepsilon(t) \quad\Longrightarrow\quad e_y(t)=\tilde{\theta}^T\phi(t)+\varepsilon(t) \]

Therefore, unmodeled dynamics and disturbances appear as an additive residual in the prediction error. This connects directly to Lesson 1: adaptive laws can compensate the linearly parameterized part, but not arbitrary unmodeled dynamics.

Before introducing online adaptation, it is useful to understand the static least-squares estimate. Given \( \mathbf{y}=\Phi\theta+\boldsymbol{\varepsilon} \), define the quadratic prediction-error cost:

\[ J(\vartheta) = \frac{1}{2}\|\mathbf{y}-\Phi\vartheta\|_2^2 \]

The gradient is

\[ \nabla_{\vartheta}J = -\Phi^T(\mathbf{y}-\Phi\vartheta) \]

Setting the gradient equal to zero gives the normal equations:

\[ \Phi^T\Phi\hat{\theta}=\Phi^T\mathbf{y} \]

If \( \Phi^T\Phi \) is nonsingular, then

\[ \hat{\theta} = (\Phi^T\Phi)^{-1}\Phi^T\mathbf{y} \]

This is not yet adaptive control. It is an offline or batch parameter estimate. However, it reveals the central role of regressor richness: if the columns of \( \Phi \) are linearly dependent, different parameter vectors may explain the same data.

Proof of the Normal Equations

Expand the cost:

\[ J(\vartheta) = \frac{1}{2} (\mathbf{y}-\Phi\vartheta)^T (\mathbf{y}-\Phi\vartheta) \]

\[ J(\vartheta) = \frac{1}{2}\mathbf{y}^T\mathbf{y} - \vartheta^T\Phi^T\mathbf{y} + \frac{1}{2}\vartheta^T\Phi^T\Phi\vartheta \]

Differentiating with respect to \( \vartheta \) gives

\[ \nabla_{\vartheta}J = -\Phi^T\mathbf{y}+\Phi^T\Phi\vartheta \]

At a stationary point, \( \nabla_{\vartheta}J=0 \), hence

\[ \Phi^T\Phi\hat{\theta}=\Phi^T\mathbf{y} \]

A linearly parameterized model is structurally useful only if the chosen basis functions can separate the unknown parameters. Suppose two parameter vectors \( \theta_a \) and \( \theta_b \) produce exactly the same output for all admissible signals:

\[ \theta_a^T\phi(z)=\theta_b^T\phi(z) \quad \text{for all admissible } z \]

Then

\[ (\theta_a-\theta_b)^T\phi(z)=0 \quad \text{for all admissible } z \]

If this is possible for a nonzero \( \theta_a-\theta_b \), then the parameters are not uniquely identifiable from that signal set. In finite data, the corresponding algebraic condition is full column rank:

\[ \operatorname{rank}(\Phi)=p \quad \Longleftrightarrow \quad \Phi^T\Phi \text{ is nonsingular} \]

Later in the course, this idea becomes the dynamic concept of persistent excitation. For now, the key point is simple: linearly parameterized models make estimation possible, but they do not guarantee that the available data contains enough independent information.

flowchart TD
  A["Choose physical model"] --> B["Select unknown constants theta"]
  B --> C["Move known signal terms into phi"]
  C --> D["Check: unknowns multiply known signals only"]
  D --> E["Build y = Phi theta"]
  E --> F["Check rank or conditioning of Phi"]
  F --> G["Use prediction error for estimation or adaptation"]
        

The Python script below simulates a nonlinear regressor \( \phi(x,u)=[x,\;u,\;x^2,\;\sin x,\;xu]^T \), creates data from \( y=\theta^T\phi+\varepsilon \), and estimates \( \theta \) using least squares.

"""
Chapter2_Lesson2.py

Linearly parameterized model:
    y(t) = theta^T phi(x(t), u(t)) + noise(t)

This script simulates data from a nonlinear-in-signals but linear-in-parameters
model, estimates theta by batch least squares, and compares prediction errors.

Dependencies:
    numpy
    matplotlib

Run:
    python Chapter2_Lesson2.py
"""

import numpy as np
import matplotlib.pyplot as plt


def build_regressor(x: np.ndarray, u: np.ndarray) -> np.ndarray:
    """
    Build phi(x,u) for each sample.

    phi = [x, u, x^2, sin(x), x*u]^T

    The model is nonlinear in measured signals x and u,
    but linear in the unknown parameter vector theta.
    """
    return np.column_stack([
        x,
        u,
        x**2,
        np.sin(x),
        x * u
    ])


def least_squares_theta(Phi: np.ndarray, y: np.ndarray, ridge: float = 1e-10) -> np.ndarray:
    """
    Solve theta_hat = arg min ||y - Phi theta||_2^2.

    A small ridge term is used only for numerical conditioning.
    """
    n_params = Phi.shape[1]
    normal_matrix = Phi.T @ Phi + ridge * np.eye(n_params)
    right_hand_side = Phi.T @ y
    return np.linalg.solve(normal_matrix, right_hand_side)


def main() -> None:
    rng = np.random.default_rng(7)
    n = 800

    # Measured signals
    t = np.linspace(0.0, 20.0, n)
    x = 1.2 * np.sin(0.7 * t) + 0.3 * np.sin(2.1 * t)
    u = 0.8 * np.cos(0.5 * t) + 0.4 * rng.normal(size=n)

    # True unknown constant parameters
    theta_true = np.array([1.50, -0.80, 0.35, 2.00, -0.45])

    # Linear-in-parameters regression
    Phi = build_regressor(x, u)
    noise = 0.05 * rng.normal(size=n)
    y = Phi @ theta_true + noise

    theta_hat = least_squares_theta(Phi, y)
    y_hat = Phi @ theta_hat

    print("True theta:     ", theta_true)
    print("Estimated theta:", theta_hat)
    print("Parameter error:", theta_hat - theta_true)
    print("RMS prediction error:", np.sqrt(np.mean((y - y_hat) ** 2)))

    # A rank check is an algebraic informativeness check.
    # Dynamic persistent excitation is studied later in the course.
    singular_values = np.linalg.svd(Phi, compute_uv=False)
    print("Singular values of Phi:", singular_values)
    print("rank(Phi):", np.linalg.matrix_rank(Phi))

    plt.figure()
    plt.plot(t, y, label="measured y")
    plt.plot(t, y_hat, "--", label="predicted y_hat")
    plt.xlabel("time")
    plt.ylabel("output")
    plt.title("Linearly Parameterized Model: y = theta^T phi(x,u)")
    plt.legend()
    plt.grid(True)
    plt.show()


if __name__ == "__main__":
    main()

The C++ version implements the same least-squares idea without relying on an external linear algebra library. For production control software, libraries such as Eigen, Armadillo, or Blaze are recommended.

/*
Chapter2_Lesson2.cpp

Linearly parameterized model:
    y = theta^T phi(x,u) + noise

This program estimates theta from simulated data using normal equations.
No external linear algebra library is required.

Compile:
    g++ -std=c++17 -O2 Chapter2_Lesson2.cpp -o Chapter2_Lesson2

Run:
    ./Chapter2_Lesson2
*/

#include <cmath>
#include <iomanip>
#include <iostream>
#include <random>
#include <stdexcept>
#include <vector>

using Matrix = std::vector<std::vector<double>>;
using Vector = std::vector<double>;

Vector regressor(double x, double u) {
    return {x, u, x * x, std::sin(x), x * u};
}

Vector solveLinearSystem(Matrix A, Vector b) {
    const int n = static_cast<int>(b.size());

    for (int k = 0; k < n; ++k) {
        int pivot = k;
        for (int i = k + 1; i < n; ++i) {
            if (std::fabs(A[i][k]) > std::fabs(A[pivot][k])) {
                pivot = i;
            }
        }

        if (std::fabs(A[pivot][k]) < 1e-12) {
            throw std::runtime_error("Normal matrix is singular or ill-conditioned.");
        }

        std::swap(A[k], A[pivot]);
        std::swap(b[k], b[pivot]);

        const double diag = A[k][k];
        for (int j = k; j < n; ++j) {
            A[k][j] /= diag;
        }
        b[k] /= diag;

        for (int i = 0; i < n; ++i) {
            if (i == k) continue;
            const double factor = A[i][k];
            for (int j = k; j < n; ++j) {
                A[i][j] -= factor * A[k][j];
            }
            b[i] -= factor * b[k];
        }
    }

    return b;
}

int main() {
    const int N = 800;
    const int p = 5;
    const double dt = 20.0 / (N - 1.0);

    Vector thetaTrue = {1.50, -0.80, 0.35, 2.00, -0.45};

    Matrix normal(p, Vector(p, 0.0));
    Vector rhs(p, 0.0);

    std::mt19937 gen(7);
    std::normal_distribution<double> unitNormal(0.0, 1.0);

    for (int k = 0; k < N; ++k) {
        const double t = k * dt;
        const double x = 1.2 * std::sin(0.7 * t) + 0.3 * std::sin(2.1 * t);
        const double u = 0.8 * std::cos(0.5 * t) + 0.4 * unitNormal(gen);

        Vector phi = regressor(x, u);

        double y = 0.0;
        for (int i = 0; i < p; ++i) {
            y += thetaTrue[i] * phi[i];
        }
        y += 0.05 * unitNormal(gen);

        for (int i = 0; i < p; ++i) {
            rhs[i] += phi[i] * y;
            for (int j = 0; j < p; ++j) {
                normal[i][j] += phi[i] * phi[j];
            }
        }
    }

    // Small ridge regularization for numerical conditioning.
    for (int i = 0; i < p; ++i) {
        normal[i][i] += 1e-10;
    }

    Vector thetaHat = solveLinearSystem(normal, rhs);

    std::cout << std::fixed << std::setprecision(6);
    std::cout << "True theta      Estimated theta      Error\n";
    for (int i = 0; i < p; ++i) {
        std::cout << std::setw(10) << thetaTrue[i] << "      "
                  << std::setw(14) << thetaHat[i] << "      "
                  << std::setw(10) << thetaHat[i] - thetaTrue[i] << "\n";
    }

    return 0;
}

The Java example uses normalized online gradient prediction-error minimization. This is not yet a full adaptive controller, but it shows how linearly parameterized prediction naturally leads to parameter updates proportional to \( \phi e_y \).

/*
Chapter2_Lesson2.java

Linearly parameterized model:
    y = theta^T phi(x,u) + noise

This program uses online gradient prediction-error minimization:
    theta_hat <- theta_hat + gamma * phi * (y - theta_hat^T phi)

Compile:
    javac Chapter2_Lesson2.java

Run:
    java Chapter2_Lesson2
*/

import java.util.Random;

public class Chapter2_Lesson2 {
    static double[] regressor(double x, double u) {
        return new double[] {
            x,
            u,
            x * x,
            Math.sin(x),
            x * u
        };
    }

    static double dot(double[] a, double[] b) {
        double s = 0.0;
        for (int i = 0; i < a.length; i++) {
            s += a[i] * b[i];
        }
        return s;
    }

    public static void main(String[] args) {
        int n = 4000;
        double gamma = 0.015;

        double[] thetaTrue = {1.50, -0.80, 0.35, 2.00, -0.45};
        double[] thetaHat = {0.0, 0.0, 0.0, 0.0, 0.0};

        Random rng = new Random(7);

        double squaredErrorSum = 0.0;

        for (int k = 0; k < n; k++) {
            double t = 20.0 * k / (n - 1.0);
            double x = 1.2 * Math.sin(0.7 * t) + 0.3 * Math.sin(2.1 * t);
            double u = 0.8 * Math.cos(0.5 * t) + 0.4 * rng.nextGaussian();

            double[] phi = regressor(x, u);
            double y = dot(thetaTrue, phi) + 0.05 * rng.nextGaussian();

            double yHat = dot(thetaHat, phi);
            double error = y - yHat;

            double normPhi2 = 1e-8 + dot(phi, phi);
            for (int i = 0; i < thetaHat.length; i++) {
                // Normalized gradient step improves numerical behavior.
                thetaHat[i] += gamma * phi[i] * error / normPhi2;
            }

            squaredErrorSum += error * error;
        }

        System.out.println("True theta      Estimated theta      Error");
        for (int i = 0; i < thetaTrue.length; i++) {
            System.out.printf("%10.6f      %14.6f      %10.6f%n",
                    thetaTrue[i], thetaHat[i], thetaHat[i] - thetaTrue[i]);
        }
        System.out.printf("RMS online prediction error: %.6f%n",
                Math.sqrt(squaredErrorSum / n));
    }
}

MATLAB is convenient for constructing regressor matrices and validating parameterizations. The optional Simulink portion creates a small model that computes \( \hat{y}=\hat{\theta}^T\phi(x,u) \).

% Chapter2_Lesson2.m
%
% Linearly parameterized model:
%     y(t) = theta' * phi(x(t),u(t)) + noise(t)
%
% This script:
%   1) Simulates data from a nonlinear regressor.
%   2) Estimates theta using batch least squares.
%   3) Optionally builds a small Simulink model that computes y_hat.
%
% Run in MATLAB:
%     Chapter2_Lesson2

clear; clc; close all;

rng(7);
N = 800;
t = linspace(0,20,N)';

x = 1.2*sin(0.7*t) + 0.3*sin(2.1*t);
u = 0.8*cos(0.5*t) + 0.4*randn(N,1);

theta_true = [1.50; -0.80; 0.35; 2.00; -0.45];

Phi = [x, u, x.^2, sin(x), x.*u];
y = Phi*theta_true + 0.05*randn(N,1);

% Batch least squares with a small ridge term for numerical conditioning.
ridge = 1e-10;
theta_hat = (Phi'*Phi + ridge*eye(size(Phi,2))) \ (Phi'*y);
y_hat = Phi*theta_hat;

disp('True theta and estimated theta:');
disp(table(theta_true, theta_hat, theta_hat-theta_true, ...
    'VariableNames', {'True','Estimated','Error'}));

fprintf('RMS prediction error: %.6f\n', sqrt(mean((y-y_hat).^2)));
fprintf('rank(Phi) = %d\n', rank(Phi));

figure;
plot(t,y,'LineWidth',1); hold on;
plot(t,y_hat,'--','LineWidth',1.2);
grid on;
xlabel('time');
ylabel('output');
title('Linearly Parameterized Model: y = theta^T phi(x,u)');
legend('measured y','predicted yhat','Location','best');

%% Optional Simulink block diagram generation
% This creates a model with a MATLAB Function block implementing:
%   phi = [x; u; x^2; sin(x); x*u]
%   yhat = thetaHat' * phi
%
% Uncomment the next line to create the model if Simulink is installed.
% buildChapter2Lesson2Simulink(theta_hat);

function buildChapter2Lesson2Simulink(thetaHat)
    model = 'Chapter2_Lesson2_Simulink_Model';
    if bdIsLoaded(model)
        close_system(model,0);
    end

    new_system(model);
    open_system(model);

    add_block('simulink/Sources/Sine Wave', [model '/x_signal'], ...
        'Position', [40 70 110 100], 'Amplitude', '1.2', 'Frequency', '0.7');

    add_block('simulink/Sources/Sine Wave', [model '/u_signal'], ...
        'Position', [40 160 110 190], 'Amplitude', '0.8', 'Frequency', '0.5');

    add_block('simulink/User-Defined Functions/MATLAB Function', ...
        [model '/LIP_Regressor_Model'], 'Position', [190 85 390 185]);

    add_block('simulink/Sinks/Display', [model '/y_hat_display'], ...
        'Position', [460 120 540 150]);

    add_line(model, 'x_signal/1', 'LIP_Regressor_Model/1');
    add_line(model, 'u_signal/1', 'LIP_Regressor_Model/2');
    add_line(model, 'LIP_Regressor_Model/1', 'y_hat_display/1');

    block = [model '/LIP_Regressor_Model'];
    rt = sfroot;
    chart = rt.find('-isa','Stateflow.EMChart','Path',block);
    chart.Script = sprintf([ ...
        'function yhat = fcn(x,u)\n' ...
        '%%#codegen\n' ...
        'thetaHat = [%g; %g; %g; %g; %g];\n' ...
        'phi = [x; u; x^2; sin(x); x*u];\n' ...
        'yhat = thetaHat'' * phi;\n' ...
        'end\n'], thetaHat(1), thetaHat(2), thetaHat(3), thetaHat(4), thetaHat(5));

    save_system(model);
    fprintf('Created Simulink model: %s.slx\n', model);
end

Mathematica is useful for symbolic verification of regression identities and for compact numerical experiments.

(* Chapter2_Lesson2.nb *)

(* Linearly parameterized model:
      y[k] = theta . phi[x[k],u[k]] + noise[k]
   This Wolfram Language code generates data, estimates theta by least squares,
   and verifies the prediction-error identity symbolically.
*)

ClearAll["Global`*"];

phi[x_, u_] := {x, u, x^2, Sin[x], x u};

thetaTrue = {1.50, -0.80, 0.35, 2.00, -0.45};

SeedRandom[7];
n = 800;
t = Subdivide[0, 20, n - 1];

xData = 1.2 Sin[0.7 #] + 0.3 Sin[2.1 #] & /@ t;
uData = 0.8 Cos[0.5 #] + 0.4 RandomVariate[NormalDistribution[0, 1]] & /@ t;

Phi = MapThread[phi, {xData, uData}];
noise = RandomVariate[NormalDistribution[0, 0.05], n];
yData = Phi.thetaTrue + noise;

thetaHat = LinearSolve[Transpose[Phi].Phi + 10^-10 IdentityMatrix[5],
                       Transpose[Phi].yData];

prediction = Phi.thetaHat;
rmsError = Sqrt[Mean[(yData - prediction)^2]];

Print["thetaTrue = ", thetaTrue];
Print["thetaHat  = ", thetaHat];
Print["error     = ", thetaHat - thetaTrue];
Print["RMS prediction error = ", rmsError];

ListLinePlot[
  {Transpose[{t, yData}], Transpose[{t, prediction}]},
  PlotLegends -> {"measured y", "predicted yhat"},
  AxesLabel -> {"time", "output"},
  PlotLabel -> "Linearly Parameterized Model: y = theta^T phi(x,u)"
]

(* Symbolic prediction-error identity *)
theta = Array[Subscript[\[Theta], #] &, 5];
thetaHatSym = Array[Subscript[Overscript[\[Theta], "^"], #] &, 5];
phiSym = Array[Subscript[\[Phi], #] &, 5];

y = theta.phiSym;
yHat = thetaHatSym.phiSym;

FullSimplify[y - yHat == (theta - thetaHatSym).phiSym]

A useful linearly parameterized model should satisfy the following requirements:

1. Known basis: every entry of \( \phi(x,u) \) must be computable from measured or estimated signals.
2. Unknown constants only: the uncertain quantities must appear as coefficients, not inside nonlinear functions.
3. Correct dimensions: for scalar output, \( \theta,\phi\in\mathbb{R}^{p} \); for vector output, use \( Y(x,u)\theta \).
4. Physical interpretability: whenever possible, entries of \( \theta \) should correspond to mass, damping, stiffness, gain, time constant, or other meaningful constants.
5. Data richness: the regressor matrix should not be rank deficient over the intended operating data.
6. Residual awareness: any part not captured by \( \theta^T\phi \) remains as disturbance, measurement noise, or unmodeled dynamics.

Problem 1 (Recognizing Linear Parameterization): Determine whether each model is linearly parameterized in the unknown parameters:

\[ \begin{aligned} \text{(a)}\quad y &= \theta_1x+\theta_2x^2+\theta_3u\\ \text{(b)}\quad y &= \theta_1\sin(x)+\theta_2xu\\ \text{(c)}\quad y &= \theta_1\sin(\theta_2x)\\ \text{(d)}\quad y &= e^{-\theta_1 t}x+\theta_2u \end{aligned} \]

Solution: Models (a) and (b) are linearly parameterized because the unknown parameters multiply known functions of measured signals. Model (c) is not linearly parameterized because \( \theta_2 \) appears inside \( \sin(\cdot) \). Model (d) is not linearly parameterized because \( \theta_1 \) appears inside an exponential.

Problem 2 (Constructing a Regressor): Given \( y=a x+b u+c x u+d\sin x \), construct \( \theta \) and \( \phi(x,u) \) such that \( y=\theta^T\phi(x,u) \).

Solution: Choose

\[ \theta= \begin{bmatrix} a\\b\\c\\d \end{bmatrix}, \qquad \phi(x,u)= \begin{bmatrix} x\\u\\xu\\\sin x \end{bmatrix} \]

Then

\[ \theta^T\phi(x,u) = ax+bu+cxu+d\sin x = y \]

Problem 3 (Prediction Error Identity): Suppose \( y=\theta^T\phi \) and \( \hat{y}=\hat{\theta}^T\phi \). Show that the prediction error can be written as \( e_y=\tilde{\theta}^T\phi \), where \( \tilde{\theta}=\theta-\hat{\theta} \).

Solution:

\[ e_y = y-\hat{y} = \theta^T\phi-\hat{\theta}^T\phi = (\theta-\hat{\theta})^T\phi = \tilde{\theta}^T\phi \]

This identity is exact in the absence of residual uncertainty. If measurement noise or unmodeled dynamics are present, an additive residual term appears.

Problem 4 (Least-Squares Estimate): For the stacked model \( \mathbf{y}=\Phi\theta+\boldsymbol{\varepsilon} \), derive the least-squares estimate minimizing \( J(\vartheta)=\frac{1}{2}\|\mathbf{y}-\Phi\vartheta\|_2^2 \).

Solution: The gradient is

\[ \nabla_{\vartheta}J = -\Phi^T(\mathbf{y}-\Phi\vartheta) \]

Set it to zero:

\[ -\Phi^T\mathbf{y}+\Phi^T\Phi\hat{\theta}=0 \]

Therefore,

\[ \Phi^T\Phi\hat{\theta} = \Phi^T\mathbf{y} \]

If \( \Phi^T\Phi \) is nonsingular,

\[ \hat{\theta} = (\Phi^T\Phi)^{-1}\Phi^T\mathbf{y} \]

Problem 5 (Rank Deficiency): Suppose \( \phi(x,u)=[x,\;2x]^T \) and \( y=\theta_1x+2\theta_2x \). Can \( \theta_1 \) and \( \theta_2 \) be uniquely identified from data?

Solution: No. The model can be written as

\[ y=(\theta_1+2\theta_2)x \]

Only the combined parameter \( \theta_1+2\theta_2 \) is identifiable. The two columns of \( \Phi \) are linearly dependent because the second column is twice the first. Hence \( \operatorname{rank}(\Phi)=1 \), not \( 2 \).

A linearly parameterized model writes uncertain dynamics or uncertain signal maps as \( \theta^T\phi(x,u) \) or \( Y(x,u)\theta \). The measured signals may enter nonlinearly through the regressor, but the unknown constants must enter linearly. This structure produces the fundamental prediction-error identity \( e_y=\tilde{\theta}^T\phi \), which will be used repeatedly when deriving gradient and Lyapunov-based adaptive laws in later chapters.