Consider a robot with state \( x \in \mathbb{R}^n \) and control input \( u \in \mathbb{R}^m \), modeled as

\[ \dot{x} = f(x) + g(x)u , \]

where \( f \) and \( g \) encode the robot dynamics (e.g., in joint-error coordinates). A nominal, model-based controller \( u_b(x) \) has been designed in previous chapters (e.g., PD, computed-torque, robust/adaptive, or MPC) such that the origin \( x = 0 \) is asymptotically stable:

\[ \dot{x} = f(x) + g(x)u_b(x) \] \[ \exists V:\mathbb{R}^n \to \mathbb{R}_{\ge 0},\ \alpha_1(\lVert x \rVert) \le V(x) \le \alpha_2(\lVert x \rVert), \quad \dot{V}(x) \le -\alpha_3(\lVert x \rVert) \]

for class-\(\mathcal{K}_\infty\) functions \( \alpha_1, \alpha_2, \alpha_3 \). We now introduce a learned term \( u_\ell(x) \) (e.g., a neural-network residual, a Gaussian-process mean prediction, or a regression model) to improve performance:

\[ u_\text{cand}(x) = u_b(x) + u_\ell(x). \]

A stability-preserving learning wrapper is a function \( W(x, u_\text{cand}) \) that produces the final applied torque \( u(x) = W(x, u_\text{cand}(x)) \) so that the closed-loop system retains provable stability or ISS properties, even if \( u_\ell \) is imperfect.

flowchart TD
  X["Robot state x_k"] --> B["Baseline controller u_b(x_k)"]
  X --> L["Learner output u_l(x_k)"]
  B --> SUM["Sum candidate u_cand"]
  L --> SUM
  SUM --> WRAP["Wrapper W(x_k, u_cand)"]
  WRAP --> ACT["Applied torque u_k"]
  ACT --> PLANT["Robot dynamics"]
  PLANT --> X
        

The core idea is to treat the learner as a performance-enhancing suggestion that is always filtered by a mathematically designed wrapper enforcing Lyapunov or ISS constraints derived from the nominal controller.

First, we formalize the nominal controller as stabilizing the origin with respect to additive input disturbances. Introduce an additive disturbance input \( d \in \mathbb{R}^m \):

\[ \dot{x} = f(x) + g(x)\big(u_b(x) + d\big). \]

We interpret \( d \) as the combined effect of model mismatch and learned augmentation. Assume the pair \( (f,g,u_b) \) renders the origin input-to-state stable (ISS) with respect to \( d \):

\[ \exists \beta \in \mathcal{KL},\ \gamma \in \mathcal{K} \text{ such that } \lVert x(t) \rVert \le \beta\big(\lVert x(0) \rVert, t\big) + \gamma\!\left(\sup_{0 \le s \le t} \lVert d(s) \rVert\right). \]

Now the learner produces \( u_\ell(x) \), which we use as \( d(x) \). A simple but powerful assumption is that \( u_\ell \) is Lipschitz with respect to the state:

\[ u_\ell(0) = 0,\quad \lVert u_\ell(x_1) - u_\ell(x_2) \rVert \le L \lVert x_1 - x_2 \rVert, \quad \forall x_1,x_2. \]

If the baseline ISS gain \( \gamma \) and learner Lipschitz constant \( L \) satisfy the small-gain condition \( \gamma(L r) < r \) for some radius \( r > 0 \), then the origin remains locally asymptotically stable. In the special case of a linear gain \( \gamma(s) = k s \), the small-gain condition simplifies to \( kL < 1 \).

\[ \text{If } \gamma(s) = k s,\ kL < 1 \ \Rightarrow\ \lVert x(t) \rVert \le \frac{1}{1 - kL}\beta\big(\lVert x(0) \rVert, t\big) \xrightarrow[t \to \infty]{} 0. \]

A stability-preserving wrapper can therefore enforce that the effective gain of the learner (as seen by the nominal closed loop) is small enough, e.g. by shrinking its output or by scaling it as a function of the current state norm.

A more direct approach uses the Lyapunov function \( V \) of the nominal controller to define a safe action set for each state. For the nominal controller:

\[ \dot{V}(x) = \nabla V(x)^\top \big( f(x) + g(x)u_b(x) \big) \le -\alpha_3(\lVert x \rVert) \triangleq -c V(x), \]

for some \( c > 0 \) (for simplicity). If we allow any control \( u \), then

\[ \dot{V}(x;u) = \nabla V(x)^\top \big( f(x) + g(x)u \big). \]

Define the state-dependent safe set of controls

\[ \mathcal{U}(x) = \left\{ u \in \mathbb{R}^m \ \middle|\ \nabla V(x)^\top \big(f(x) + g(x)u\big) \le -c V(x) \right\}. \]

The nominal control \( u_b(x) \) is in \( \mathcal{U}(x) \) by definition. Given a candidate control \( u_\text{cand} = u_b(x) + u_\ell(x) \) from the learner, the wrapper solves the projection problem

\[ u^\star(x) = \arg\min_{u \in \mathbb{R}^m} \frac{1}{2} \lVert u - u_\text{cand}(x) \rVert^2 \quad \text{s.t.} \quad \nabla V(x)^\top \big(f(x) + g(x)u\big) \le -c V(x). \]

This is a convex quadratic program (QP) with a single affine constraint when \( V \) and the dynamics are fixed and \( u \) enters linearly. The wrapper output is then \( u(x) = u^\star(x) \).

Proposition (Lyapunov projection wrapper). Assume:

• \( V \) is positive-definite and radially unbounded.
• The nominal controller \( u_b \) satisfies \( \nabla V(x)^\top (f(x) + g(x)u_b(x)) \le -c V(x) \) for all \( x \neq 0 \).
• The QP defining \( u^\star(x) \) is feasible (which holds since \( u_b(x) \in \mathcal{U}(x) \)).

Then the closed-loop system \( \dot{x} = f(x) + g(x)u^\star(x) \) is globally asymptotically stable.

Sketch of proof. By construction of \( u^\star(x) \) we have

\[ \dot{V}(x) = \nabla V(x)^\top \big( f(x) + g(x)u^\star(x) \big) \le -c V(x),\quad \forall x. \]

Standard Lyapunov arguments imply global asymptotic stability: \( V \) is decrescent and positive definite, and \( \dot{V} \le -c V \) gives \( V(x(t)) \le V(x(0))e^{-ct} \), hence \( x(t) \to 0 \) as \( t \to \infty \).

Consider a single-joint robot in error coordinates \( x = q - q_d \), with simplified (normalized) dynamics

\[ \dot{x} = -a x + u, \quad a > 0. \]

A nominal linear controller \( u_b(x) = -k x \) with \( k > 0 \) yields

\[ \dot{x} = -(a+k)x. \]

Choose Lyapunov function \( V(x) = \tfrac{1}{2}x^2 \). Then

\[ \dot{V} = x \dot{x} = x(-a x + u) = -a x^2 + x u. \]

Under pure nominal control \( u = -k x \), we obtain

\[ \dot{V}_b = -a x^2 + x(-k x) = -(a+k)x^2. \]

Now add a learned residual \( u_\ell(x) \), so that \( u = u_b(x) + u_\ell(x) = -k x + u_\ell(x) \). Then

\[ \dot{V} = -a x^2 + x(-k x + u_\ell(x)) = -(a+k)x^2 + x u_\ell(x). \]

For some desired decay rate \( \lambda \in (0, a+k) \), we want \( \dot{V} \le -\lambda x^2 \) for all \( x \). This is guaranteed if

\[ -(a+k)x^2 + x u_\ell(x) \le -\lambda x^2 \quad \Rightarrow \quad x u_\ell(x) \le (a+k-\lambda)x^2. \]

Using \( |x u_\ell(x)| \le |x|\,|u_\ell(x)| \), it suffices to enforce

\[ |u_\ell(x)| \le c |x|, \quad c \le a+k-\lambda. \]

A simple wrapper is then a state-dependent saturation:

\[ u(x) = -k x + \operatorname{sat}\big(u_\ell(x);\ -c|x|,\ c|x|\big), \]

where the saturation function \( \operatorname{sat}(v;v_\text{min},v_\text{max}) \) clips \( v \) to the interval \( [v_\text{min}, v_\text{max}] \). For example, set \( \lambda = \tfrac{1}{2}(a+k) \), then \( c = \tfrac{a+k}{2} \) is admissible.

flowchart TD
  S["Start sampling"] --> M["Measure joint error x"]
  M --> UB["Compute nominal u_b = -k x"]
  M --> UL["Predict residual u_l(x) from learner"]
  UB --> CAND["Form u_cand = u_b + u_l"]
  UL --> CAND
  CAND --> WR["Apply Lyapunov-based saturation on u_l(x)"]
  WR --> UAP["Final torque u"]
  UAP --> DYN["Send u to joint actuator"]
        

This scalar example illustrates how analytic bounds derived from a Lyapunov function can yield a very simple and efficient wrapper without solving online optimization problems.

We now implement the scalar stability-preserving wrapper in Python. For multi-DOF manipulators, the same principle extends componentwise or via a QP. We assume the learner is implemented with a regression model (e.g., scikit-learn or a small neural network), and low-level robot dynamics are handled by a robotics library such as pinocchio or roboticstoolbox.

import numpy as np

class ScalarJointWrapper:
    """
    Stability-preserving wrapper for a scalar joint:
        x_dot = -a * x + u
    Nominal controller: u_b = -k * x
    Learned residual: u_l(x) from a regression model
    Wrapper enforces |u_l(x)| <= c * |x|.
    """
    def __init__(self, a: float, k: float, c_scale: float = 0.5, learner=None):
        self.a = a
        self.k = k
        # Choose c = c_scale * (a + k), 0 < c_scale < 1
        self.c = c_scale * (a + k)
        self.learner = learner  # e.g., a NN or GP with .predict(x_array)

    def nominal_control(self, x: float) -> float:
        return -self.k * x

    def learned_residual(self, x: float) -> float:
        if self.learner is None:
            return 0.0
        # learner expects shape (N,1)
        x_arr = np.array([[x]])
        u_l = float(self.learner.predict(x_arr))
        return u_l

    def wrapped_control(self, x: float) -> float:
        u_b = self.nominal_control(x)
        u_l = self.learned_residual(x)
        # Lyapunov-based saturation: |u_l| <= c * |x|
        bound = self.c * abs(x)
        if abs(u_l) > bound:
            u_l = np.sign(u_l) * bound
        return u_b + u_l

# Example usage (with a dummy learner)
class DummyLearner:
    def predict(self, X):
        # Pretend we have learned some residual; here just a cubic nonlinearity
        return 0.8 * X[:, 0]**3

learner = DummyLearner()
wrapper = ScalarJointWrapper(a=1.0, k=4.0, c_scale=0.5, learner=learner)

x = 0.3
u = wrapper.wrapped_control(x)
print("State x =", x, "wrapped control u =", u)
      

In a real robot, wrapped_control would be called inside the real-time joint controller loop, where x is the joint position error measured at each sampling instant.

In C++, the wrapper can be embedded in a ROS2 controller node. Libraries such as Eigen (for linear algebra) and orocos_kdl (for kinematics/dynamics) are commonly used. Below is a simplified scalar implementation of the same wrapper logic.

#include <cmath>
#include <iostream>

class DummyLearner {
public:
    double predict(double x) const {
        // Example: cubic nonlinearity as in Python example
        return 0.8 * std::pow(x, 3.0);
    }
};

class ScalarJointWrapper {
public:
    ScalarJointWrapper(double a, double k, double c_scale,
                       const DummyLearner& learner)
        : a_(a), k_(k), c_(c_scale * (a + k)), learner_(learner) {}

    double nominalControl(double x) const {
        return -k_ * x;
    }

    double learnedResidual(double x) const {
        return learner_.predict(x);
    }

    double wrappedControl(double x) const {
        double u_b = nominalControl(x);
        double u_l = learnedResidual(x);
        double bound = c_ * std::abs(x);
        if (std::abs(u_l) > bound) {
            u_l = (u_l > 0.0) ? bound : -bound;
        }
        return u_b + u_l;
    }

private:
    double a_;
    double k_;
    double c_;
    const DummyLearner& learner_;
};

int main() {
    DummyLearner learner;
    ScalarJointWrapper wrapper(1.0, 4.0, 0.5, learner);

    double x = 0.3;
    double u = wrapper.wrappedControl(x);
    std::cout << "x = " << x
              << ", wrapped u = " << u << std::endl;
    return 0;
}
      

In a ROS2 controller, the method wrappedControl would be called inside the update or controlLoop callback, with x computed from the joint state and desired trajectory provided by a higher-level planner.

Java can be used for robot simulators or teaching tools. Linear algebra libraries such as EJML or Apache Commons Math can be leveraged for multi-dimensional variants. Here we show a scalar wrapper similar to the Python and C++ cases.

public interface Learner {
    double predict(double x);
}

public class DummyLearner implements Learner {
    @Override
    public double predict(double x) {
        // Same cubic residual as in other examples
        return 0.8 * x * x * x;
    }
}

public class ScalarJointWrapper {
    private final double a;
    private final double k;
    private final double c;
    private final Learner learner;

    public ScalarJointWrapper(double a, double k, double cScale, Learner learner) {
        this.a = a;
        this.k = k;
        this.c = cScale * (a + k);
        this.learner = learner;
    }

    public double nominalControl(double x) {
        return -k * x;
    }

    public double learnedResidual(double x) {
        return learner.predict(x);
    }

    public double wrappedControl(double x) {
        double u_b = nominalControl(x);
        double u_l = learnedResidual(x);
        double bound = c * Math.abs(x);
        if (Math.abs(u_l) > bound) {
            u_l = (u_l > 0.0) ? bound : -bound;
        }
        return u_b + u_l;
    }

    public static void main(String[] args) {
        Learner learner = new DummyLearner();
        ScalarJointWrapper wrapper = new ScalarJointWrapper(1.0, 4.0, 0.5, learner);

        double x = 0.3;
        double u = wrapper.wrappedControl(x);
        System.out.println("x = " + x + ", wrapped u = " + u);
    }
}
      

This wrapper can be integrated with a Java-based robot simulator, where x is updated using a numerical integrator for the scalar dynamics and the learner is trained offline from logged data.

In MATLAB, we can implement the wrapper as a function used in a Simulink block. The learner could be a neural network from the Deep Learning Toolbox or a regression model from the Statistics and Machine Learning Toolbox. The Simulink wrapper block takes x and the learner output as inputs and outputs the wrapped torque.

function u = scalar_joint_wrapper(x, a, k, cScale, learnerNet)
% scalar_joint_wrapper: stability-preserving wrapper for a scalar joint
% Inputs:
%   x          - joint position error
%   a, k       - dynamics and nominal gain
%   cScale     - 0 < cScale < 1, scaling for Lyapunov bound
%   learnerNet - (optional) trained network, e.g., SeriesNetwork
%
% Output:
%   u          - wrapped control input

if nargin < 5 || isempty(learnerNet)
    u_l = 0;
else
    % Assume learnerNet takes a 1-by-1 input
    u_l = predict(learnerNet, x);
end

u_b = -k * x;
c   = cScale * (a + k);
bound = c * abs(x);

if abs(u_l) > bound
    u_l = sign(u_l) * bound;
end

u = u_b + u_l;
end
      

In Simulink, one can implement this as a MATLAB Function block, with x, a, k, and the learner output as inputs. The rest of the robot dynamics can be simulated using the Robotics System Toolbox or custom dynamic blocks.

A comparable implementation in Wolfram Mathematica uses a symbolic or numeric learner and a pure function for the wrapper:

(* Parameters *)
a = 1.0;
k = 4.0;
cScale = 0.5;
c = cScale * (a + k);

(* Dummy learner: cubic residual *)
learner[x_] := 0.8 x^3;

nominalControl[x_] := -k x;

wrappedControl[x_] := Module[
  {uB, uL, bound},
  uB = nominalControl[x];
  uL = learner[x];
  bound = c Abs[x];
  If[Abs[uL] > bound,
     uL = Sign[uL] bound
  ];
  uB + uL
];

(* Example evaluation *)
x0 = 0.3;
u0 = wrappedControl[x0]
      

Mathematica is particularly useful for analytically deriving the Lyapunov conditions or solving small QPs symbolically before implementing numerical versions in other environments.

Problem 1 (Lyapunov Projection Feasibility). Let \( V \) and \( u_b \) satisfy \( \nabla V(x)^\top (f(x) + g(x)u_b(x)) \le -c V(x) \) for all \( x \neq 0 \), with \( c > 0 \). Show that the QP defining the projected control \( u^\star(x) \) in Section 3 is feasible for all \( x \).

Solution. For each state \( x \), the constraint set for the QP is

\[ \mathcal{U}(x) = \{ u \mid \nabla V(x)^\top(f(x) + g(x)u) \le -cV(x) \}. \]

By assumption on \( u_b(x) \), we have

\[ \nabla V(x)^\top(f(x) + g(x)u_b(x)) \le -cV(x), \]

so \( u_b(x) \in \mathcal{U}(x) \). Thus the constraint set is non-empty for every \( x \), and the QP is feasible. Because the cost is a strictly convex quadratic in \( u \), there is a unique optimal solution \( u^\star(x) \).

Problem 2 (Small-Gain for Scalar Linear System). Consider the scalar ISS system

\[ \dot{x} = -a x + d,\quad a > 0, \]

with input \( d(t) \). Show that there exists a linear gain function \( \gamma(s) = k s \) with \( k = \tfrac{1}{a} \) such that

\[ \lVert x(t) \rVert \le \beta(\lVert x(0) \rVert, t) + \gamma\!\left(\sup_{0 \le s \le t}\lVert d(s) \rVert\right), \]

for some \( \beta \in \mathcal{KL} \).

Solution. Solve the scalar ODE explicitly:

\[ x(t) = e^{-at}x(0) + \int_0^t e^{-a(t-\tau)}d(\tau)\,d\tau. \]

Taking absolute values and using \( |d(\tau)| \le \sup_{0 \le s \le t}|d(s)| \),

\[ |x(t)| \le e^{-at}|x(0)| + \sup_{0 \le s \le t}|d(s)| \int_0^t e^{-a(t-\tau)} d\tau. \]

The integral is

\[ \int_0^t e^{-a(t-\tau)} d\tau = \int_0^t e^{-a\theta} d\theta = \frac{1 - e^{-at}}{a} \le \frac{1}{a}. \]

Therefore,

\[ |x(t)| \le e^{-at}|x(0)| + \frac{1}{a}\sup_{0 \le s \le t}|d(s)|, \]

which is ISS with \( \beta(r,t) = e^{-at}r \) and \( \gamma(s) = \tfrac{1}{a}s \). Thus \( k = \tfrac{1}{a} \).

Problem 3 (Admissible Learner Lipschitz Constant). Connect Problems 1 and 2: Suppose we use the scalar system of Problem 2 and a static learner \( u_\ell(x) \) with Lipschitz constant \( L \). If we interpret \( d(t) = u_\ell(x(t)) \), use the small-gain condition to derive a bound on \( L \) that guarantees stability.

Solution. From Problem 2, \( \gamma(s) = \tfrac{1}{a}s \). For a static learner \( u_\ell \) with Lipschitz constant \( L \), the induced gain from state to disturbance satisfies \( \sup_t |d(t)| \le L \sup_t |x(t)| \). The small-gain condition requires

\[ \gamma\big(\sup_t |d(t)|\big) = \frac{1}{a}\sup_t |d(t)| \le \frac{L}{a}\sup_t |x(t)| < \sup_t |x(t)|. \]

Thus, we need \( \tfrac{L}{a} < 1 \), i.e. \( L < a \). If the learner is scaled so that its Lipschitz constant satisfies \( L < a \), the closed-loop system remains stable.

Problem 4 (Lyapunov Wrapper for Multi-DOF Manipulator). Let a robot manipulator in joint-error coordinates be

\[ M(q)\ddot{e} + C(q,\dot{q})\dot{e} + K_pe + K_d\dot{e} = \tau_\ell, \]

where \( e = q - q_d \) and \( \tau_\ell \) is a learned torque. Assume \( M(q) \) is positive definite and that the nominal controller corresponds to \( \tau_\ell = 0 \). Show that a Lyapunov function of the form

\[ V(e,\dot{e}) = \frac{1}{2}\dot{e}^\top M(q)\dot{e} + \frac{1}{2}e^\top K_pe \]

leads to a sufficient condition of the form \( \dot{V} \le -\lambda \lVert [e^\top\ \dot{e}^\top]^\top \rVert^2 \) if \( \tau_\ell \) is bounded by a state-dependent inequality. Describe qualitatively how a QP wrapper could enforce this inequality.

Solution. Using standard robot control properties (skew-symmetry of \( \dot{M} - 2C \)), the derivative of \( V \) along trajectories with \( \tau_\ell \) becomes

\[ \dot{V} = -\dot{e}^\top K_d \dot{e} + \dot{e}^\top \tau_\ell. \]

We can bound the cross term as \( \dot{e}^\top \tau_\ell \le \lVert \dot{e} \rVert \lVert \tau_\ell \rVert \). If the learned torque satisfies a constraint of the form

\[ \lVert \tau_\ell \rVert \le \alpha_1 \lVert e \rVert + \alpha_2 \lVert \dot{e} \rVert, \]

with sufficiently small \( \alpha_1, \alpha_2 \), then for appropriate design of \( K_d \) there exists \( \lambda > 0 \) such that

\[ \dot{V} \le -\lambda \left(\lVert e \rVert^2 + \lVert \dot{e} \rVert^2\right). \]

A QP wrapper can enforce the bound by treating \( \tau_\ell \) as an optimization variable and constraining its norm via a second-order cone or via linear bounds on each joint torque, while minimizing deviation from the raw learner output.

In this lesson we rigorously integrated learning modules into robot controllers without sacrificing stability. Starting from an ISS nominal controller, we analyzed the learner as a disturbance with bounded gain and used the small-gain theorem to obtain stability conditions on the learner's Lipschitz constant. We then introduced Lyapunov-based action projection as a powerful wrapper mechanism: by defining a state-dependent safe set of controls and projecting the learner's suggestion onto it, we ensure that a Lyapunov function decays at a prescribed rate.

A scalar joint example illustrated how these ideas reduce to simple state-dependent saturations, which can be implemented efficiently in embedded software. We provided Python, C++, Java, MATLAB/Simulink, and Mathematica implementations of such wrappers, showing that the core mathematics transfers straightforwardly across platforms and robotics libraries. These tools provide a foundation for the next lesson, where learning-augmented controllers are put into practice in a lab setting for improved robot tracking.