Chapter 20: Advanced Topics and Research Frontiers in Modeling
Lesson 4: Differentiable Physics for Robotics
This lesson introduces differentiable physics for robotic systems: we view the robot dynamics and numerical integrators as smooth maps, derive forward and backward (adjoint) sensitivity equations, and show how to compute analytic gradients of trajectory-level losses with respect to model parameters and inputs. We emphasize discrete-time formulations compatible with standard robot dynamics models and provide multi-language implementations.
1. Conceptual Overview of Differentiable Physics
In previous chapters we modeled robot dynamics using ordinary differential equations (ODEs) of the form
\[ \dot{\mathbf{x}}(t) = f\big(\mathbf{x}(t), \mathbf{u}(t), \boldsymbol{\theta}\big), \]
where \( \mathbf{x} \) collects joint positions and velocities, \( \mathbf{u} \) contains input torques/forces, and \( \boldsymbol{\theta} \) are physical parameters (masses, inertias, link lengths, friction coefficients, etc.). Differentiable physics means that both the dynamics \( f \) and the numerical integration scheme are treated as differentiable maps, so that trajectory-level functionals such as
\[ \mathcal{L}(\boldsymbol{\theta}, \mathbf{u}_{0:N-1}) = \sum_{k=0}^{N-1} \ell_k(\mathbf{x}_k, \mathbf{u}_k, \boldsymbol{\theta}) + \ell_N(\mathbf{x}_N, \boldsymbol{\theta}) \]
admit gradients \( \nabla_{\boldsymbol{\theta}} \mathcal{L} \) and \( \nabla_{\mathbf{u}_{0:N-1}} \mathcal{L} \). These gradients can be used for:
- parameter identification and model calibration (refining \( \boldsymbol{\theta} \)),
- co-design (simultaneously tuning mechanical parameters and trajectories),
- as a building block for trajectory optimization and learning-based methods (in later courses).
A typical computational loop for differentiable physics in robotics is:
flowchart TD
P["Model parameters theta"] --> F["Dynamics map f(x,u,theta)"]
F --> INT["Time integrator (discrete simulator)"]
INT --> TRAJ["State trajectory x_0,...,x_N"]
TRAJ --> LOSS["Trajectory loss L(theta,u_0,...,u_{N-1})"]
LOSS --> GRAD["Gradients 'dL/dtheta', 'dL/du_k'"]
GRAD --> OPT["Gradient-based update of theta or u_k"]
The key mathematical question of this lesson is: how do we compute these gradients efficiently and accurately, given a robot dynamics model and an integrator?
2. Discrete-Time Robot Dynamics as a Differentiable Map
We operate in discrete time with sampling period \( \Delta t \). Let \( \mathbf{x}_k \approx \mathbf{x}(k\Delta t) \) and \( \mathbf{u}_k \approx \mathbf{u}(k\Delta t) \). A one-step numerical integrator defines a map
\[ \mathbf{x}_{k+1} = \Phi\big(\mathbf{x}_k, \mathbf{u}_k, \boldsymbol{\theta}\big), \quad k = 0,\dots,N-1, \]
where \( \Phi \) depends on the choice of integration scheme. For example, explicit Euler gives
\[ \mathbf{x}_{k+1} = \mathbf{x}_k + \Delta t \, f\big(\mathbf{x}_k,\mathbf{u}_k,\boldsymbol{\theta}\big). \]
For robot manipulators, a common continuous-time dynamics model in joint coordinates is
\[ \mathbf{M}(\mathbf{q},\boldsymbol{\theta}) \ddot{\mathbf{q}} + \mathbf{C}(\mathbf{q},\dot{\mathbf{q}},\boldsymbol{\theta}) \dot{\mathbf{q}} + \mathbf{g}(\mathbf{q},\boldsymbol{\theta}) + \mathbf{F}_d(\dot{\mathbf{q}},\boldsymbol{\theta}) = \boldsymbol{\tau}(t), \]
where \( \boldsymbol{\tau}(t) \) (written as \tau in LaTeX) collects joint torques. Defining \( \mathbf{x} = [\mathbf{q}^\top, \dot{\mathbf{q}}^\top]^\top \), we can write \( \dot{\mathbf{x}} = f(\mathbf{x},\boldsymbol{\tau},\boldsymbol{\theta}) \), and thus a differentiable \( \Phi \).
If \( f \) is continuously differentiable in its arguments, and the integrator uses a finite composition of additions, multiplications, and smooth functions, then \( \Phi \) is also continuously differentiable. This ensures that all Jacobians needed for gradient computation exist:
\[ \mathbf{A}_k := \frac{\partial \Phi}{\partial \mathbf{x}_k}(\mathbf{x}_k,\mathbf{u}_k,\boldsymbol{\theta}),\quad \mathbf{B}_k := \frac{\partial \Phi}{\partial \boldsymbol{\theta}}(\mathbf{x}_k,\mathbf{u}_k,\boldsymbol{\theta}),\quad \mathbf{G}_k := \frac{\partial \Phi}{\partial \mathbf{u}_k}(\mathbf{x}_k,\mathbf{u}_k,\boldsymbol{\theta}). \]
3. Forward Sensitivity Analysis
Consider a trajectory loss functional
\[ \mathcal{L}(\boldsymbol{\theta}) = \sum_{k=0}^{N-1} \ell_k(\mathbf{x}_k,\boldsymbol{\theta}) + \ell_N(\mathbf{x}_N,\boldsymbol{\theta}), \quad \mathbf{x}_{k+1} = \Phi(\mathbf{x}_k,\mathbf{u}_k,\boldsymbol{\theta}), \]
where we treat inputs \( \mathbf{u}_k \) as fixed. Define the sensitivity matrices with respect to parameters:
\[ \mathbf{S}_k := \frac{\partial \mathbf{x}_k}{\partial \boldsymbol{\theta}} \in \mathbb{R}^{n_x \times n_\theta}. \]
Differentiating the discrete dynamics recursion yields
\[ \mathbf{S}_{k+1} = \mathbf{A}_k \mathbf{S}_k + \mathbf{B}_k, \qquad \mathbf{S}_0 = \frac{\partial \mathbf{x}_0}{\partial \boldsymbol{\theta}}. \]
For explicit Euler,
\[ \mathbf{A}_k = \mathbf{I} + \Delta t \frac{\partial f}{\partial \mathbf{x}_k},\quad \mathbf{B}_k = \Delta t \frac{\partial f}{\partial \boldsymbol{\theta}}. \]
By the chain rule,
\[ \frac{\mathrm{d}\mathcal{L}}{\mathrm{d}\boldsymbol{\theta}} = \sum_{k=0}^{N-1} \left( \frac{\partial \ell_k}{\partial \mathbf{x}_k} \mathbf{S}_k + \frac{\partial \ell_k}{\partial \boldsymbol{\theta}} \right) + \frac{\partial \ell_N}{\partial \mathbf{x}_N} \mathbf{S}_N + \frac{\partial \ell_N}{\partial \boldsymbol{\theta}}. \]
Complexity. Forward sensitivity scales like \( \mathcal{O}(N n_x n_\theta) \), which is acceptable if the number of parameters \( n_\theta \) is small, but expensive when many parameters are calibrated. Differentiable physics often uses adjoint methods to reverse this scaling.
4. Discrete Adjoint (Reverse-Mode) Gradient
We derive the adjoint (discrete-time reverse-mode) method that computes \( \nabla_{\boldsymbol{\theta}} \mathcal{L} \) with cost \( \mathcal{O}(N n_x) \), independent of \( n_\theta \). Introduce Lagrange multipliers \( \boldsymbol{\lambda}_{k+1} \) for the dynamic constraints:
\[ \tilde{\mathcal{L}} = \sum_{k=0}^{N-1} \ell_k(\mathbf{x}_k,\boldsymbol{\theta}) + \ell_N(\mathbf{x}_N,\boldsymbol{\theta}) + \sum_{k=0}^{N-1} \boldsymbol{\lambda}_{k+1}^\top \big( \Phi(\mathbf{x}_k,\mathbf{u}_k,\boldsymbol{\theta}) - \mathbf{x}_{k+1} \big). \]
Taking variations with respect to \( \mathbf{x}_k \) and collecting terms yields the adjoint recursion. For \( k = N \) we have
\[ \frac{\partial \tilde{\mathcal{L}}}{\partial \mathbf{x}_N} = \frac{\partial \ell_N}{\partial \mathbf{x}_N} - \boldsymbol{\lambda}_N^\top = \mathbf{0}^\top \quad\Rightarrow\quad \boldsymbol{\lambda}_N^\top = \frac{\partial \ell_N}{\partial \mathbf{x}_N}. \]
For intermediate steps \( k = N-1,\dots,0 \), using the chain rule gives
\[ \frac{\partial \tilde{\mathcal{L}}}{\partial \mathbf{x}_k} = \frac{\partial \ell_k}{\partial \mathbf{x}_k} + \boldsymbol{\lambda}_{k+1}^\top \mathbf{A}_k - \boldsymbol{\lambda}_k^\top = \mathbf{0}^\top, \]
hence the backward recursion
\[ \boldsymbol{\lambda}_k^\top = \frac{\partial \ell_k}{\partial \mathbf{x}_k} + \boldsymbol{\lambda}_{k+1}^\top \mathbf{A}_k, \quad k = N-1,\dots,0. \]
Finally, differentiating \( \tilde{\mathcal{L}} \) with respect to \( \boldsymbol{\theta} \) and using the dynamics constraints gives
\[ \frac{\mathrm{d}\mathcal{L}}{\mathrm{d}\boldsymbol{\theta}} = \sum_{k=0}^{N-1} \left( \frac{\partial \ell_k}{\partial \boldsymbol{\theta}} + \boldsymbol{\lambda}_{k+1}^\top \mathbf{B}_k \right) + \frac{\partial \ell_N}{\partial \boldsymbol{\theta}}. \]
The computational cost is dominated by the forward rollout and the backward adjoint recursion, both scaling with \( N n_x^2 \) if \( \mathbf{A}_k \) is dense, but independent of \( n_\theta \).
flowchart TD
X0["Forward pass: x_0"] --> FWD["Propagate x_{k+1} = Phi(x_k,u_k,theta)"]
FWD --> TRAJ["Store trajectory x_0,...,x_N"]
TRAJ --> LOSS["Compute loss terms l_k, l_N"]
LOSS --> LAMBDA["Initialize lambda_N = dl_N/dx_N"]
LAMBDA --> BWD["Backward pass: lambda_k = dl_k/dx_k + A_k^T lambda_{k+1}"]
BWD --> GRAD["Accumulate gradient with B_k: dL/dtheta"]
5. Example — 1-DOF Rotary Joint Parameter Gradient
Consider a single rotary joint with inertia parameter \( I \), viscous friction \( b \), stiffness \( k \), and input torque \( \tau(t) \). In continuous time:
\[ I \ddot{q} + b \dot{q} + k q = \tau(t). \]
Let the state be \( \mathbf{x} = [q,\dot{q}]^\top \). Then
\[ \dot{\mathbf{x}} = f(\mathbf{x},\tau,I) = \begin{bmatrix} \dot{q} \\ \dfrac{1}{I}\big(\tau - b \dot{q} - k q\big) \end{bmatrix}. \]
Using explicit Euler with step \( \Delta t \):
\[ \mathbf{x}_{k+1} = \mathbf{x}_k + \Delta t \, f(\mathbf{x}_k,\tau_k,I). \]
Suppose we have desired positions \( q_k^{\mathrm{ref}} \) and define the loss
\[ \mathcal{L}(I) = \frac{1}{2}\sum_{k=0}^N \big(q_k - q_k^{\mathrm{ref}}\big)^2 = \sum_{k=0}^N \ell_k(\mathbf{x}_k),\quad \ell_k(\mathbf{x}_k) = \frac{1}{2}(q_k - q_k^{\mathrm{ref}})^2. \]
We want \( \mathrm{d}\mathcal{L} / \mathrm{d}I \). The forward sensitivity matrix \( \mathbf{S}_k = \partial \mathbf{x}_k / \partial I \) obeys
\[ \mathbf{S}_{k+1} = \mathbf{S}_k + \Delta t \left( \frac{\partial f}{\partial \mathbf{x}_k} \mathbf{S}_k + \frac{\partial f}{\partial I} \right), \quad \mathbf{S}_0 = \mathbf{0}. \]
Here
\[ \frac{\partial f}{\partial I} = \begin{bmatrix} 0 \\ - \dfrac{1}{I^2}\big(\tau_k - b \dot{q}_k - k q_k\big) \end{bmatrix}. \]
The gradient is then
\[ \frac{\mathrm{d}\mathcal{L}}{\mathrm{d}I} = \sum_{k=0}^N \frac{\partial \ell_k}{\partial \mathbf{x}_k} \mathbf{S}_k = \sum_{k=0}^N (q_k - q_k^{\mathrm{ref}})\, [1\;\;0]\mathbf{S}_k. \]
Alternatively, the adjoint method introduces \( \lambda_k = \partial \mathcal{L} / \partial \mathbf{x}_k \), yielding a backward recursion and
\[ \frac{\mathrm{d}\mathcal{L}}{\mathrm{d}I} = \sum_{k=0}^{N-1} \boldsymbol{\lambda}_{k+1}^\top \Delta t \frac{\partial f}{\partial I}. \]
6. Python Implementation — Manual and Autodiff Pipelines
Python offers both manual implementations of the adjoint method and
automatic differentiation via libraries such as jax,
torch, or robotics-oriented tools like
pinocchio (for dynamics and kinematics) that can be
combined with autodiff backends. Below is a minimal manual adjoint
implementation for the 1-DOF example:
import numpy as np
def f(x, tau, I, b=0.1, k=1.0):
q, dq = x
ddq = (tau - b * dq - k * q) / I
return np.array([dq, ddq])
def step_euler(x, tau, I, dt):
return x + dt * f(x, tau, I)
def rollout(x0, taus, I, dt):
xs = [x0]
for tau in taus:
xs.append(step_euler(xs[-1], tau, I, dt))
return np.array(xs) # shape: (N+1, 2)
def loss(xs, q_ref):
# xs[:,0] contains q
return 0.5 * np.sum((xs[:, 0] - q_ref)**2)
def grad_I_adjoint(x0, taus, I, dt, q_ref):
xs = rollout(x0, taus, I, dt)
N = len(taus)
# adjoint lambdas: lambda_k in R^2
lambdas = np.zeros_like(xs)
# terminal condition
lambdas[N, 0] = xs[N, 0] - q_ref[N] # d l_N / d q_N
# precompute A_k and dPhi/dI for explicit Euler
A = np.zeros((N, 2, 2))
dPhi_dI = np.zeros((N, 2))
for k in range(N):
q, dq = xs[k]
tau = taus[k]
# partial derivatives of f wrt x
# df/dq = [0, -(k/I)]
# df/ddq = [1, -(b/I)]
df_dq = np.array([0.0, -1.0 * k / I])
df_ddq = np.array([1.0, -1.0 * 0.1 / I])
Jx = np.column_stack([df_dq, df_ddq]) # 2x2
A[k] = np.eye(2) + dt * Jx
# df/dI
ddq = (tau - 0.1 * dq - k * q) / I
dddq_dI = -(tau - 0.1 * dq - k * q) / (I**2)
df_dI = np.array([0.0, dddq_dI])
dPhi_dI[k] = dt * df_dI
# backward adjoint recursion
for k in range(N - 1, -1, -1):
# gradient of stage loss wrt x_k
dL_dxk = np.zeros(2)
dL_dxk[0] = xs[k, 0] - q_ref[k]
lambdas[k] = dL_dxk + A[k].T @ lambdas[k + 1]
# parameter gradient
dL_dI = 0.0
for k in range(N):
dL_dI += lambdas[k + 1] @ dPhi_dI[k]
return dL_dI
# Example usage
dt = 0.01
N = 100
taus = np.ones(N) * 1.0
t_grid = np.arange(N + 1) * dt
q_ref = 0.5 * np.sin(2.0 * np.pi * t_grid)
x0 = np.array([0.0, 0.0])
I = 0.5
xs = rollout(x0, taus, I, dt)
print("Loss:", loss(xs, q_ref))
print("dL/dI (adjoint):", grad_I_adjoint(x0, taus, I, dt, q_ref))
With jax, we can implement the same dynamics in a
functional style and let autodiff compute gradients through the entire
rollout, effectively performing the adjoint method automatically while
we retain full control over the dynamics model.
7. C++ Implementation — Algorithmic Differentiation and Robotics Libraries
In C++, differentiable physics is typically implemented using:
-
Robotics dynamics libraries (e.g.,
Pinocchio,RBDL) to compute \( f \), and -
algorithmic differentiation (AD) tools (e.g.,
CppAD,adept) to obtain Jacobians and gradients.
The idea is to express one simulation step
Phi(x,u,theta) using AD types, then invoke the AD library
to obtain dPhi/dtheta or reverse-mode gradients. A skeletal
example with CppAD for the 1-DOF system is:
#include <iostream>
#include <vector>
#include <cppad/cppad.hpp>
using CppAD::AD;
template <typename Scalar>
std::vector<Scalar> f(const std::vector<Scalar>& x,
Scalar tau, Scalar I,
Scalar b, Scalar k)
{
Scalar q = x[0];
Scalar dq = x[1];
Scalar ddq = (tau - b * dq - k * q) / I;
std::vector<Scalar> dx(2);
dx[0] = dq;
dx[1] = ddq;
return dx;
}
template <typename Scalar>
std::vector<Scalar> stepEuler(const std::vector<Scalar>& x,
Scalar tau, Scalar I,
Scalar b, Scalar k, Scalar dt)
{
std::vector<Scalar> dx = f(x, tau, I, b, k);
std::vector<Scalar> xnext(2);
xnext[0] = x[0] + dt * dx[0];
xnext[1] = x[1] + dt * dx[1];
return xnext;
}
int main()
{
using ADScalar = AD<double>;
// Independent variable: I (inertia parameter)
std::vector<ADScalar> I_vec(1);
I_vec[0] = 0.5;
CppAD::Independent(I_vec);
ADScalar I = I_vec[0];
ADScalar b = 0.1;
ADScalar k = 1.0;
ADScalar dt = 0.01;
ADScalar tau = 1.0;
std::vector<ADScalar> x(2);
x[0] = 0.0; // q
x[1] = 0.0; // dq
// Simulate N steps and accumulate loss
const int N = 100;
ADScalar L = 0.0;
for (int kstep = 0; kstep <= N; ++kstep) {
ADScalar q_ref = 0.5 * CppAD::sin(2.0 * M_PI * dt * kstep);
ADScalar err = x[0] - q_ref;
L += 0.5 * err * err;
if (kstep < N) {
x = stepEuler(x, tau, I, b, k, dt);
}
}
// Declare dependent variable
std::vector<ADScalar> L_vec(1);
L_vec[0] = L;
CppAD::ADFun<double> tape(I_vec, L_vec);
// Compute dL/dI at I = 0.5
std::vector<double> I_val(1), grad(1);
I_val[0] = 0.5;
grad = tape.Jacobian(I_val); // since output dimension is 1
std::cout << "Loss gradient dL/dI = " << grad[0] << std::endl;
return 0;
}
For multi-DOF robots, one typically uses a library like
Pinocchio to compute the dynamics and wraps the state and
parameters in AD types, letting the AD tool perform reverse-mode
differentiation through the entire simulation.
8. Java Implementation — Differentiable Simulation via Linear Algebra Libraries
Java has mature linear algebra libraries such as EJML and
numerical libraries that can be combined with:
- manual implementation of forward/adjoint recursions, or
-
automatic differentiation frameworks (e.g.,
ND4J, custom reverse-mode).
The following simplified example illustrates manual forward sensitivity for the 1-DOF system using plain Java arrays (for clarity). In practice one would use matrix classes from EJML:
public class DifferentiablePendulum {
static double[] f(double[] x, double tau, double I,
double b, double k) {
double q = x[0];
double dq = x[1];
double ddq = (tau - b * dq - k * q) / I;
return new double[]{dq, ddq};
}
static double[] stepEuler(double[] x, double tau, double I,
double b, double k, double dt) {
double[] dx = f(x, tau, I, b, k);
return new double[]{
x[0] + dt * dx[0],
x[1] + dt * dx[1]
};
}
public static void main(String[] args) {
double dt = 0.01;
int N = 100;
double I = 0.5;
double b = 0.1;
double k = 1.0;
double tau = 1.0;
double[] x = new double[]{0.0, 0.0};
double[][] xTraj = new double[N + 1][2];
xTraj[0] = x.clone();
// Forward rollout
for (int kstep = 0; kstep < N; ++kstep) {
x = stepEuler(x, tau, I, b, k, dt);
xTraj[kstep + 1] = x.clone();
}
// Simple reference trajectory
double[] qRef = new double[N + 1];
for (int kstep = 0; kstep <= N; ++kstep) {
double t = dt * kstep;
qRef[kstep] = 0.5 * Math.sin(2.0 * Math.PI * t);
}
// Forward sensitivity S_k = d x_k / dI (2x1 vector)
double[][] S = new double[N + 1][2];
S[0][0] = 0.0;
S[0][1] = 0.0;
for (int kstep = 0; kstep < N; ++kstep) {
double q = xTraj[kstep][0];
double dq = xTraj[kstep][1];
// Jacobian of f wrt x
double df_dq0 = 0.0;
double df_dq1 = -k / I;
double df_ddq0 = 1.0;
double df_ddq1 = -b / I;
// Jx * S_k
double JxS0 = df_dq0 * S[kstep][0] + df_ddq0 * S[kstep][1];
double JxS1 = df_dq1 * S[kstep][0] + df_ddq1 * S[kstep][1];
// df/dI
double num = tau - b * dq - k * q;
double dddq_dI = -num / (I * I);
double df_dI0 = 0.0;
double df_dI1 = dddq_dI;
S[kstep + 1][0] = S[kstep][0] + dt * (JxS0 + df_dI0);
S[kstep + 1][1] = S[kstep][1] + dt * (JxS1 + df_dI1);
}
// Gradient dL/dI
double dLdI = 0.0;
for (int kstep = 0; kstep <= N; ++kstep) {
double q = xTraj[kstep][0];
double err = q - qRef[kstep];
dLdI += err * S[kstep][0]; // only q-component
}
System.out.println("Gradient dL/dI = " + dLdI);
}
}
This implementation mirrors the continuous-time sensitivity derivation and shows that Java can host differentiable physics computations using only basic language features plus a linear algebra package.
9. MATLAB/Simulink Implementation — Symbolic and Simulation-Based
In MATLAB, differentiable physics can be approached in (at least) two complementary ways:
- Symbolic differentiation of the dynamics using the Symbolic Math Toolbox, then generating efficient numeric functions for simulation and gradients.
- Simulink-based sensitivity analysis, using Simscape Multibody for the physics and Simulink Design Optimization / linearization tools to compute sensitivities of outputs to physical parameters.
A symbolic example for the 1-DOF system:
syms q dq I b k tau real
x = [q; dq];
f = [dq;
(tau - b*dq - k*q)/I];
% Jacobians needed for sensitivities
A = jacobian(f, x); % df/dx
B = jacobian(f, I); % df/dI
disp('A(q,dq,I) = ');
disp(A);
disp('B(q,dq,I) = ');
disp(B);
% Generate numeric functions for use in simulation loops
matlabFunction(f, 'File', 'f_pendulum', 'Vars', {q, dq, tau, I, b, k});
matlabFunction(A, 'File', 'A_pendulum', 'Vars', {q, dq, tau, I, b, k});
matlabFunction(B, 'File', 'B_pendulum', 'Vars', {q, dq, tau, I, b, k});
These generated functions can be called from a numeric script that unrolls the dynamics and the forward or adjoint recursions. For multi-body robots, one can use the Robotics System Toolbox or custom symbolic derivations of \( \mathbf{M}(\mathbf{q}) \), \( \mathbf{C}(\mathbf{q},\dot{\mathbf{q}}) \), and \( \mathbf{g}(\mathbf{q}) \), and differentiate them with respect to inertial parameters.
In Simulink, differentiable physics is realized by constructing a Simscape Multibody model of the robot, marking physical parameters (masses, inertias) as tunable, and using built-in tools to compute gradients of cost functionals (tracking error, energy, etc.) with respect to these parameters. This is conceptually equivalent to the discrete adjoint method derived earlier.
10. Wolfram Mathematica Implementation — Symbolic and Numeric Adjoint
Wolfram Mathematica is well suited for differentiable physics because it
treats symbolic and numeric computations in a unified framework. One can
define the dynamics symbolically, solve or numerically integrate the
system, and then differentiate a loss functional with respect to
parameters using D or Grad.
(* 1-DOF rotary system *)
ClearAll["Global`*"];
q[t_]; dq[t_];
params = {I, b, k};
tau[t_] := 1.0;
eq = {
q'[t] == dq[t],
dq'[t] == (tau[t] - b*dq[t] - k*q[t])/I
};
ic = {q[0] == 0, dq[0] == 0};
tmax = 1.0;
sol = NDSolve[
Join[eq, ic],
{q, dq},
{t, 0, tmax},
Method -> "StiffnessSwitching"
][[1]];
qFun[t_, I_?NumericQ, b_?NumericQ, k_?NumericQ] :=
Evaluate[q[t] /. sol /. {I -> I, b -> b, k -> k}];
(* Reference and loss functional *)
qRef[t_] := 0.5*Sin[2*Pi*t];
L[I_, b_, k_] := NIntegrate[
0.5*(qFun[t, I, b, k] - qRef[t])^2,
{t, 0, tmax}
];
(* Gradient with respect to parameters *)
gradL[I0_, b0_, k0_] :=
Grad[L[I, b, k], {I, b, k}] /. {I -> I0, b -> b0, k -> k0};
(* Example evaluation *)
gradVal = gradL[0.5, 0.1, 1.0];
gradVal
For higher-dimensional robots, Mathematica can symbolically derive the manipulator equations via Lagrange's method, automatically compute Jacobians and sensitivity equations, and export optimized C or C++ code for embedded differentiable physics computations.
11. Problems and Solutions
Problem 1 (Forward Sensitivity Recursion): Consider a discrete-time system \( \mathbf{x}_{k+1} = \Phi(\mathbf{x}_k,\boldsymbol{\theta}) \), \( k = 0,\dots,N-1 \), with parameter vector \( \boldsymbol{\theta} \in \mathbb{R}^{n_\theta} \). Show that the sensitivity matrices \( \mathbf{S}_k = \partial \mathbf{x}_k / \partial \boldsymbol{\theta} \) satisfy
\[ \mathbf{S}_{k+1} = \mathbf{A}_k \mathbf{S}_k + \mathbf{B}_k,\quad \mathbf{A}_k = \frac{\partial \Phi}{\partial \mathbf{x}_k},\quad \mathbf{B}_k = \frac{\partial \Phi}{\partial \boldsymbol{\theta}}. \]
Solution:
By differentiating the recursion with respect to \( \boldsymbol{\theta} \):
\[ \frac{\partial \mathbf{x}_{k+1}}{\partial \boldsymbol{\theta}} = \frac{\partial \Phi}{\partial \mathbf{x}_k} \frac{\partial \mathbf{x}_k}{\partial \boldsymbol{\theta}} + \frac{\partial \Phi}{\partial \boldsymbol{\theta}} = \mathbf{A}_k \mathbf{S}_k + \mathbf{B}_k. \]
This uses the multivariate chain rule with \( \mathbf{x}_{k+1} = \Phi(\mathbf{x}_k(\boldsymbol{\theta}),\boldsymbol{\theta}) \). The initial sensitivity follows from \( \mathbf{x}_0 \)'s dependence on \( \boldsymbol{\theta} \).
Problem 2 (Adjoint Recursion Derivation): For the loss \( \mathcal{L} = \sum_{k=0}^{N-1} \ell_k(\mathbf{x}_k) + \ell_N(\mathbf{x}_N) \) subject to \( \mathbf{x}_{k+1} = \Phi(\mathbf{x}_k,\boldsymbol{\theta}) \), derive the adjoint recursion
\[ \boldsymbol{\lambda}_N^\top = \frac{\partial \ell_N}{\partial \mathbf{x}_N},\quad \boldsymbol{\lambda}_k^\top = \frac{\partial \ell_k}{\partial \mathbf{x}_k} + \boldsymbol{\lambda}_{k+1}^\top \mathbf{A}_k, \quad k = N-1,\dots,0. \]
Solution:
Define the augmented functional
\[ \tilde{\mathcal{L}} = \sum_{k=0}^{N-1} \ell_k(\mathbf{x}_k) + \ell_N(\mathbf{x}_N) + \sum_{k=0}^{N-1} \boldsymbol{\lambda}_{k+1}^\top \big(\Phi(\mathbf{x}_k,\boldsymbol{\theta}) - \mathbf{x}_{k+1}\big). \]
Setting the gradient of \( \tilde{\mathcal{L}} \) with respect to each \( \mathbf{x}_k \) to zero gives necessary conditions. For \( k = N \), only \( \ell_N \) and the constraint term with \( \mathbf{x}_N \) appear, yielding \( \partial \ell_N / \partial \mathbf{x}_N - \boldsymbol{\lambda}_N^\top = \mathbf{0}^\top \). For \( k < N \), differentiating the \( k \)-th stage loss and the two constraint terms involving \( \mathbf{x}_k \) gives
\[ \frac{\partial \tilde{\mathcal{L}}}{\partial \mathbf{x}_k} = \frac{\partial \ell_k}{\partial \mathbf{x}_k} + \boldsymbol{\lambda}_{k+1}^\top \mathbf{A}_k - \boldsymbol{\lambda}_k^\top = \mathbf{0}^\top, \]
from which the adjoint recursion follows.
Problem 3 (Gradient with Respect to Inputs): Extend the adjoint derivation to compute \( \partial \mathcal{L} / \partial \mathbf{u}_k \) for a system \( \mathbf{x}_{k+1} = \Phi(\mathbf{x}_k,\mathbf{u}_k,\boldsymbol{\theta}) \) and loss \( \mathcal{L} = \sum_k \ell_k(\mathbf{x}_k,\mathbf{u}_k) + \ell_N(\mathbf{x}_N) \). Express \( \partial \mathcal{L} / \partial \mathbf{u}_k \) in terms of adjoints and partial derivatives of \( \Phi \).
Solution:
The adjoint recursion is unchanged: \( \boldsymbol{\lambda}_N^\top = \partial \ell_N / \partial \mathbf{x}_N \), \( \boldsymbol{\lambda}_k^\top = \partial \ell_k / \partial \mathbf{x}_k + \boldsymbol{\lambda}_{k+1}^\top \mathbf{A}_k \). Differentiating the augmented functional with respect to \( \mathbf{u}_k \) gives
\[ \frac{\partial \mathcal{L}}{\partial \mathbf{u}_k} = \frac{\partial \ell_k}{\partial \mathbf{u}_k} + \boldsymbol{\lambda}_{k+1}^\top \frac{\partial \Phi}{\partial \mathbf{u}_k} = \frac{\partial \ell_k}{\partial \mathbf{u}_k} + \boldsymbol{\lambda}_{k+1}^\top \mathbf{G}_k. \]
This is the discrete analogue of the continuous-time Pontryagin maximum principle.
Problem 4 (Complexity Comparison): Assume \( \mathbf{x} \in \mathbb{R}^{n_x} \), \( \boldsymbol{\theta} \in \mathbb{R}^{n_\theta} \), and simulation horizon \( N \). Compare the asymptotic computational cost of: (a) forward sensitivity for all parameters, and (b) a single backward adjoint pass. When is adjoint-based differentiable physics advantageous?
Solution:
For forward sensitivities, each step updates \( \mathbf{S}_k \in \mathbb{R}^{n_x \times n_\theta} \) via a matrix-matrix multiplication \( \mathbf{A}_k \mathbf{S}_k \) and an addition with \( \mathbf{B}_k \). This costs \( \mathcal{O}(n_x^2 n_\theta) \) per step or \( \mathcal{O}(N n_x^2 n_\theta) \) overall. The adjoint method stores the trajectory and propagates \( \boldsymbol{\lambda}_k \in \mathbb{R}^{n_x} \) backwards, costing \( \mathcal{O}(n_x^2) \) per step for the matrix-vector product \( \mathbf{A}_k^\top \boldsymbol{\lambda}_{k+1} \), hence \( \mathcal{O}(N n_x^2) \) overall, independent of \( n_\theta \). Thus, when \( n_\theta \gg 1 \) (e.g., many inertial parameters, friction parameters, or neural augmentations), adjoint-based differentiable physics is far more efficient.
Problem 5 (Integrator Consistency and Gradient Accuracy): Explain why using a higher-order integrator (e.g., Runge–Kutta of order 4) for \( \Phi \) generally improves not only the state trajectory accuracy but also the gradient accuracy. What trade-offs arise in differentiable physics?
Solution:
Gradients are computed by differentiating the numerical map \( \Phi \), not the continuous dynamics directly. If the integrator is only first order (e.g., explicit Euler), the discrete map approximates the exact flow with local error \( \mathcal{O}(\Delta t^2) \). The gradient of this approximation generally has additional error of the same order, so even with exact differentiation of \( \Phi \), the resulting gradients are only consistent up to the integrator's truncation error. A higher-order scheme (e.g., RK4) reduces truncation error of both the state and its sensitivity, making gradient-based optimization more reliable. The trade-offs are:
- higher per-step computational cost (more evaluations of \( f \) and Jacobians),
- more storage if intermediate stages are required for adjoint calculations, and
- possible complications when combining with event handling or contact dynamics.
12. Summary
In this lesson we reframed robot dynamics and numerical integrators as differentiable maps, enabling analytic gradient computation of trajectory-level losses with respect to both physical parameters and inputs. We derived forward sensitivity equations and the discrete adjoint (reverse-mode) recursion, highlighting their computational trade-offs. Through a 1-DOF rotary example, we showed how these gradients can be implemented manually and via automatic differentiation in multiple programming languages and environments (Python, C++, Java, MATLAB/Simulink, and Wolfram Mathematica). Differentiable physics thus provides a rigorous bridge between classical robot modeling and modern gradient-based methods for identification and design.
13. References
- Marsden, J.E., & West, M. (2001). Discrete mechanics and variational integrators. Acta Numerica, 10, 357–514.
- Degrave, J., Hermans, M., Dambre, J., & Wyffels, F. (2016). A differentiable physics engine for deep learning in robotics. arXiv preprint arXiv:1611.01652.
- Hu, Y., Anderson, L., Li, T.-M., Sun, Q., Carr, N., Ragan-Kelley, J., & Durand, F. (2019). ChainQueen: A real-time differentiable physical simulator for soft robotics. IEEE International Conference on Robotics and Automation (ICRA), 6265–6271.
- Chen, T.Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018). Neural ordinary differential equations. Advances in Neural Information Processing Systems, 31.
- Ascher, U.M., & Petzold, L.R. (1998). Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations. SIAM.
- Betts, J.T. (1998). Survey of numerical methods for trajectory optimization. Journal of Guidance, Control, and Dynamics, 21(2), 193–207.
- Bryson, A.E., & Ho, Y.-C. (1975). Applied Optimal Control. Hemisphere Publishing.
- Pontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V., & Mishchenko, E.F. (1962). The Mathematical Theory of Optimal Processes. Interscience Publishers.