Let the real robot dynamics be abstracted as \( f_{\text{real}} \) and the simulator dynamics as \( f_{\text{sim}}(\cdot;\boldsymbol{\theta}) \), where \( \boldsymbol{\theta} \) is a vector of simulator parameters (masses, frictions, sensor noise, delays, etc.). Simulation calibration chooses \( \boldsymbol{\theta} \) such that trajectories from the simulator match those from the real system under comparable conditions.

Denote by \( \mathbf{u}_{0:T-1} \) an input or policy sequence applied both in reality and simulation. The corresponding state and observation trajectories are

\[ \mathbf{x}^{\text{real}}_{0:T},\ \mathbf{y}^{\text{real}}_{0:T} \quad\text{and}\quad \mathbf{x}^{\text{sim}}_{0:T}(\boldsymbol{\theta}),\ \mathbf{y}^{\text{sim}}_{0:T}(\boldsymbol{\theta}). \]

A generic calibration problem is to minimize a trajectory-level discrepancy:

\[ J(\boldsymbol{\theta}) = \sum_{k=0}^{T} \big\| \mathbf{\phi}\!\big(\mathbf{y}^{\text{real}}_k\big) - \mathbf{\phi}\!\big(\mathbf{y}^{\text{sim}}_k(\boldsymbol{\theta})\big) \big\|_{\mathbf{W}}^{2}, \]

where \( \mathbf{\phi} \) is a feature map on observations (e.g., joint angles, contact locations, end-effector pose) and \( \mathbf{W} \) is a positive semi-definite weighting matrix encoding which features are most important to match.

Conceptually, systematic calibration answers: Which parameters should we tune, using which data, under which objective, with which optimization strategy? Ad hoc manual tuning is replaced by a repeatable pipeline, summarized as:

flowchart TD
  A["Choose parameter vector theta"] --> B["Design excitation experiments"]
  B --> C["Collect real rollouts (x_real, y_real)"]
  C --> D["Simulate with same inputs in sim (x_sim, y_sim)"]
  D --> E["Compute trajectory mismatch J(theta)"]
  E --> F["Search theta to reduce J(theta)"]
  F --> G["Validate on held-out policies/tasks"]
  G --> H["Deploy calibrated sim for training / testing"]
        

Classical system identification reconstructs an explicit model for \( f_{\text{real}} \) from data. In contrast, our calibration viewpoint treats the simulator as a black-box function \( \boldsymbol{\theta} \mapsto \mathbf{y}^{\text{sim}}_{0:T}(\boldsymbol{\theta}) \) and optimizes mismatch measures over \( \boldsymbol{\theta} \).

Suppose we run \( N \) different experiments (e.g., different policies, reference trajectories, or initial states), indexed by \( n=1,\dots,N \). For experiment \( n \), we record:

• input sequence \( \mathbf{u}^{(n)}_{0:T_n-1} \),
• real outputs \( \mathbf{y}^{\text{real},(n)}_{0:T_n} \),
• simulated outputs \( \mathbf{y}^{\text{sim},(n)}_{0:T_n}(\boldsymbol{\theta}) \).

A general calibration objective aggregates mismatch across experiments:

\[ J(\boldsymbol{\theta}) = \sum_{n=1}^{N} \sum_{k=0}^{T_n} \ell\!\Big( \mathbf{\phi}\!\big(\mathbf{y}^{\text{real},(n)}_k\big), \mathbf{\phi}\!\big(\mathbf{y}^{\text{sim},(n)}_k(\boldsymbol{\theta})\big) \Big), \]

where \( \ell \) is a pointwise loss. Common choices:

• Quadratic loss:

  \[ \ell_{\text{LS}}(\mathbf{a},\mathbf{b}) = (\mathbf{a}-\mathbf{b})^{\top}\mathbf{W}(\mathbf{a}-\mathbf{b}). \]

• Robust (Huber-type) loss:

  \[ \ell_{\text{Huber}}(\mathbf{a},\mathbf{b}) = \sum_{i} \begin{cases} \tfrac{1}{2}(a_i-b_i)^2, & \text{if } |a_i-b_i| \leq \delta,\\[4pt] \delta\,|a_i-b_i| - \tfrac{1}{2}\delta^2, & \text{otherwise}. \end{cases} \]

The calibration problem becomes:

\[ \boldsymbol{\theta}^{\star} \in \arg\min_{\boldsymbol{\theta} \in \Theta} J(\boldsymbol{\theta}), \]

with \( \Theta \) encoding known bounds or constraints (e.g., positive masses, friction coefficients between 0 and 1). We neither derive a closed-form estimator nor assume a particular parametric model for \( f_{\text{real}} \); instead we use optimization over the simulator.

Directly matching raw states can overweight unimportant degrees of freedom or be sensitive to alignment errors in time. Instead, we often calibrate at the level of trajectory features. Let \( \boldsymbol{\phi}_{k}^{(n)} \) denote features from trial \( n \) at time \( k \).

For each experiment \( n \) we can define real and simulated feature distributions. For simplicity, consider the empirical distributions obtained from samples \( \{\boldsymbol{\phi}^{\text{real},(n)}_{k}\} \) and \( \{\boldsymbol{\phi}^{\text{sim},(n)}_{k}(\boldsymbol{\theta})\} \). A distributional calibration objective is:

\[ J_{\text{dist}}(\boldsymbol{\theta}) = \sum_{n=1}^{N} D\!\Big( \hat{p}^{\text{real},(n)}(\boldsymbol{\phi}), \hat{p}^{\text{sim},(n)}(\boldsymbol{\phi};\boldsymbol{\theta}) \Big), \]

where \( D \) is a statistical divergence (e.g., Wasserstein distance or kernel maximum mean discrepancy (MMD)).

For MMD with a positive definite kernel \( k(\mathbf{z},\mathbf{z}') \), the empirical squared MMD between real and simulated features for a fixed experiment is

\[ \operatorname{MMD}^2(\boldsymbol{\theta}) = \frac{1}{n_r^2}\sum_{i=1}^{n_r}\sum_{j=1}^{n_r} k(\mathbf{z}^{\text{real}}_i,\mathbf{z}^{\text{real}}_j) +\frac{1}{n_s^2}\sum_{i=1}^{n_s}\sum_{j=1}^{n_s} k(\mathbf{z}^{\text{sim}}_i(\boldsymbol{\theta}), \mathbf{z}^{\text{sim}}_j(\boldsymbol{\theta})) - \\ \frac{2}{n_r n_s}\sum_{i=1}^{n_r}\sum_{j=1}^{n_s} k(\mathbf{z}^{\text{real}}_i,\mathbf{z}^{\text{sim}}_j(\boldsymbol{\theta})) \]

Minimizing MMD encourages the simulator trajectories to match the distributional statistics of real trajectories, rather than only pointwise alignment. This is especially useful when policies induce stochastic behavior or when exact time alignment is ambiguous.

A calibration objective is meaningful only if the collected data sufficiently excites the system. For instance, inertial parameters cannot be inferred from commands that nearly keep the robot static. Without deriving explicit Fisher information matrices, we can still reason about excitation via the richness of inputs and outputs.

Let \( \mathbf{\psi}_k \) denote a feature vector built from inputs and states, e.g., \( \mathbf{\psi}_k = [\mathbf{u}_k^{\top}, \mathbf{q}_k^{\top}, \dot{\mathbf{q}}_k^{\top}]^{\top} \). Over an experiment, we collect \( M \) such feature samples and stack them into a matrix \( \mathbf{\Psi} \in \mathbb{R}^{M \times d} \).

A simple measure of excitation is the conditioning of the empirical covariance:

\[ \mathbf{\Sigma}_{\psi} = \frac{1}{M}\sum_{k=1}^{M} (\mathbf{\psi}_k - \bar{\mathbf{\psi}}) (\mathbf{\psi}_k - \bar{\mathbf{\psi}})^{\top}, \quad \bar{\mathbf{\psi}} = \frac{1}{M}\sum_{k=1}^{M}\mathbf{\psi}_k. \]

Poorly excited experiments yield nearly singular \( \mathbf{\Sigma}_{\psi} \), whereas well-designed trajectories for calibration seek to make \( \mathbf{\Sigma}_{\psi} \) well-conditioned. In practice, this corresponds to:

• covering a wide range of joint angles and velocities,
• including accelerations and contact events relevant to the target task,
• avoiding repetitive or nearly constant input patterns.

In advanced settings, one can explicitly optimize calibration inputs to maximize a surrogate of information content (e.g., determinant of \( \mathbf{\Sigma}_{\psi} \)), but even simple heuristics like sweeping joints or applying rich noise to reference trajectories can be effective.

Domain randomization (Lesson 2) samples simulator parameters from a broad distribution so that policies become robust to unknown reality. However, if the prior over parameters is too broad or unrealistic, learning becomes inefficient. Calibration can be used to shape the domain-randomization distribution around plausible regions.

Let \( p_0(\boldsymbol{\theta}) \) be an initial prior distribution over parameters (possibly hand-chosen), and let \( J(\boldsymbol{\theta}) \) be the mismatch objective. One conceptual approach is:

1. Perform a calibration phase to obtain a small set of high-quality parameter samples \( \{\boldsymbol{\theta}^{\star}_m\}_{m=1}^{M} \) with low mismatch.
2. Fit an updated distribution \( p_{\text{cal}}(\boldsymbol{\theta}) \) (e.g., a Gaussian mixture) to these samples.
3. Use \( p_{\text{cal}} \) as the domain-randomization distribution for training policies.

If we interpret calibration as Bayesian updating, we can think of \( p_{\text{cal}} \) as proportional to \( p_0(\boldsymbol{\theta}) \exp(-\lambda J(\boldsymbol{\theta})) \), where \( \lambda > 0 \) controls how sharply we focus on low-mismatch regions.

flowchart TD
  P0["Initial parameter prior p0(theta)"] --> CAL["Calibrate: evaluate J(theta) on experiments"]
  CAL --> SEL["Select good candidates with low J"]
  SEL --> FIT["Fit updated distribution p_cal(theta)"]
  FIT --> DR["Sample theta from p_cal(theta) for domain randomization"]
  DR --> POL["Train policy pi in calibrated-randomized sim"]
  POL --> EVAL["Evaluate policy on real robot"]
        

We now illustrate a simple gradient-free calibration loop in Python. The idea is to perform random search over parameters, evaluate the mismatch objective on a fixed set of real trajectories, and retain the best parameters. The code assumes a simulator interface with a function rollout_sim(theta, experiments).

import numpy as np

# Example parameter vector: [link_mass_scale, joint_damping_scale, friction_scale]
PARAM_DIM = 3

def sample_theta():
    # Simple box constraints: theta in [0.5, 1.5]^3
    return 0.5 + np.random.rand(PARAM_DIM)

def trajectory_features(traj):
    # traj: dict with keys "q", "qd", "ee_pos"
    q = traj["q"]      # shape: (T, nq)
    qd = traj["qd"]    # shape: (T, nq)
    ee = traj["ee_pos"]  # shape: (T, 3)
    # Simple feature: stack mean and std of each signal
    feats = []
    for arr in [q, qd, ee]:
        feats.append(arr.mean(axis=0))
        feats.append(arr.std(axis=0))
    return np.concatenate(feats, axis=0)

def feature_mismatch(real_traj, sim_traj, W=None):
    f_real = trajectory_features(real_traj)
    f_sim = trajectory_features(sim_traj)
    diff = f_real - f_sim
    if W is None:
        return float(diff @ diff)
    return float(diff @ (W @ diff))

def calibration_objective(theta, real_experiments, sim_interface, W=None):
    cost = 0.0
    for exp in real_experiments:
        # exp is a dict with "u", "real_traj", "init_state"
        sim_traj = sim_interface.rollout_sim(theta, exp)
        cost += feature_mismatch(exp["real_traj"], sim_traj, W=W)
    return cost

def random_search_calibration(real_experiments, sim_interface,
                              num_samples=100, seed=0):
    rng = np.random.default_rng(seed)
    best_theta = None
    best_cost = np.inf
    for k in range(num_samples):
        theta = sample_theta()
        cost = calibration_objective(theta, real_experiments, sim_interface)
        if cost < best_cost:
            best_cost = cost
            best_theta = theta
        if (k + 1) % 10 == 0:
            print(f"Iter {k+1}: best_cost = {best_cost:.3f}")
    return best_theta, best_cost

# Usage (assuming sim_interface and real_experiments are defined):
# theta_star, J_star = random_search_calibration(real_experiments, sim_interface)
# print("Calibrated theta:", theta_star)
      

While naive, this pattern already enforces a systematic pipeline: clearly defined parameter vector, mismatch objective, and reusable experiment set. More advanced approaches (e.g., CMA-ES or Bayesian optimization) can replace the random search loop without changing the conceptual structure.

In C++, we often interface with simulators such as MuJoCo, Gazebo, or custom dynamics engines. The following code sketch shows the basic structure for evaluating a calibration cost for a given parameter vector theta. Template parameters and interface details will depend on the specific simulator.

#include <vector>
#include <array>
#include <cmath>

struct Trajectory {
    std::vector<std::array<double, 7>> q;      // example: 7-DOF joint angles
    std::vector<std::array<double, 7>> qd;     // joint velocities
    std::vector<std::array<double, 3>> ee_pos; // end-effector position
};

struct Experiment {
    Trajectory real_traj;
    // plus inputs, initial state, etc.
};

class SimulatorInterface {
public:
    Trajectory rolloutSim(const std::vector<double>& theta,
                          const Experiment& exp) const;
};

std::vector<double> trajectoryFeatures(const Trajectory& traj) {
    // Concatenate mean and std of q, qd, ee_pos
    const std::size_t T = traj.q.size();
    const std::size_t nq = traj.q.front().size();
    const std::size_t ne = traj.ee_pos.front().size();

    std::vector<double> feats;
    feats.reserve(2 * (nq + nq + ne)); // mean and std for each coord

    auto accumulate_mean_std = [&](const auto& seq, std::size_t dim) {
        std::vector<double> mean(dim, 0.0);
        std::vector<double> sq(dim, 0.0);
        for (const auto& x : seq) {
            for (std::size_t i = 0; i < dim; ++i) {
                mean[i] += x[i];
                sq[i]   += x[i] * x[i];
            }
        }
        for (std::size_t i = 0; i < dim; ++i) {
            mean[i] /= static_cast<double>(T);
            double var = sq[i] / static_cast<double>(T) - mean[i] * mean[i];
            double stdv = std::sqrt(std::max(var, 0.0));
            feats.push_back(mean[i]);
            feats.push_back(stdv);
        }
    };

    accumulate_mean_std(traj.q, nq);
    accumulate_mean_std(traj.qd, nq);
    accumulate_mean_std(traj.ee_pos, ne);

    return feats;
}

double featureMismatch(const Trajectory& real_traj,
                       const Trajectory& sim_traj) {
    auto fr = trajectoryFeatures(real_traj);
    auto fs = trajectoryFeatures(sim_traj);
    double cost = 0.0;
    const std::size_t n = fr.size();
    for (std::size_t i = 0; i < n; ++i) {
        double d = fr[i] - fs[i];
        cost += d * d;
    }
    return cost;
}

double calibrationObjective(const std::vector<double>& theta,
                            const std::vector<Experiment>& exps,
                            const SimulatorInterface& sim) {
    double total = 0.0;
    for (const auto& e : exps) {
        Trajectory sim_traj = sim.rolloutSim(theta, e);
        total += featureMismatch(e.real_traj, sim_traj);
    }
    return total;
}
      

Optimization over theta can then be performed with any C++ optimization library (e.g., CMA-ES, NLopt), treating calibrationObjective as a black-box function.

In Java-based robotics stacks (e.g., custom simulators or Java bindings), a typical pattern is to encapsulate calibration routines in a dedicated manager class. Below is a simplified example that performs random search over parameter vectors and evaluates the cost.

import java.util.*;
import java.util.stream.*;

class Trajectory {
    public double[][] q;
    public double[][] qd;
    public double[][] eePos;
}

class Experiment {
    public Trajectory realTraj;
    // plus commands, initial state, etc.
}

interface SimulatorInterface {
    Trajectory rolloutSim(double[] theta, Experiment exp);
}

public class CalibrationManager {

    private final SimulatorInterface sim;
    private final List<Experiment> experiments;
    private final Random rng = new Random(0L);

    public CalibrationManager(SimulatorInterface sim,
                              List<Experiment> experiments) {
        this.sim = sim;
        this.experiments = experiments;
    }

    private double[] sampleTheta() {
        double[] theta = new double[3];
        for (int i = 0; i < 3; ++i) {
            theta[i] = 0.5 + rng.nextDouble(); // [0.5, 1.5]
        }
        return theta;
    }

    private double[] trajectoryFeatures(Trajectory traj) {
        List<double[]> all = new ArrayList<>();
        all.addAll(Arrays.asList(traj.q));
        all.addAll(Arrays.asList(traj.qd));
        all.addAll(Arrays.asList(traj.eePos));

        List<Double> feats = new ArrayList<>();
        int dim = all.get(0).length;
        int T = traj.q.length;

        for (int k = 0; k < all.size(); ++k) {
            double[] seq = all.get(k);
        }

        // Means and stds for each coordinate across q, qd, eePos
        for (double[][] seqGroup : new double[][][]{traj.q, traj.qd, traj.eePos}) {
            int dimGroup = seqGroup[0].length;
            double[] mean = new double[dimGroup];
            double[] sq = new double[dimGroup];
            int Tgroup = seqGroup.length;
            for (int t = 0; t < Tgroup; ++t) {
                double[] x = seqGroup[t];
                for (int i = 0; i < dimGroup; ++i) {
                    mean[i] += x[i];
                    sq[i] += x[i] * x[i];
                }
            }
            for (int i = 0; i < dimGroup; ++i) {
                mean[i] /= Tgroup;
                double var = sq[i] / Tgroup - mean[i] * mean[i];
                double stdv = Math.sqrt(Math.max(var, 0.0));
                feats.add(mean[i]);
                feats.add(stdv);
            }
        }

        return feats.stream().mapToDouble(d -> d).toArray();
    }

    private double featureMismatch(Trajectory realTraj,
                                   Trajectory simTraj) {
        double[] fr = trajectoryFeatures(realTraj);
        double[] fs = trajectoryFeatures(simTraj);
        double cost = 0.0;
        for (int i = 0; i < fr.length; ++i) {
            double d = fr[i] - fs[i];
            cost += d * d;
        }
        return cost;
    }

    private double calibrationObjective(double[] theta) {
        double total = 0.0;
        for (Experiment e : experiments) {
            Trajectory simTraj = sim.rolloutSim(theta, e);
            total += featureMismatch(e.realTraj, simTraj);
        }
        return total;
    }

    public double[] randomSearch(int numSamples) {
        double bestCost = Double.POSITIVE_INFINITY;
        double[] bestTheta = null;
        for (int k = 0; k < numSamples; ++k) {
            double[] theta = sampleTheta();
            double cost = calibrationObjective(theta);
            if (cost < bestCost) {
                bestCost = cost;
                bestTheta = theta;
            }
        }
        return bestTheta;
    }
}
      

This pattern is compatible with Java-based reinforcement learning or control stacks, where SimulatorInterface may be backed by JNI bindings to a physics engine.

In MATLAB / Simulink, calibration is naturally formulated as an optimization over a Simulink model. We treat the Simulink model as a black box that maps a parameter vector to output trajectories and employ built-in optimizers such as fmincon or patternsearch.

function [theta_best, J_best] = calibrate_simulink(realData, modelName)
% realData: struct array with fields u, y_real, t
% modelName: name of Simulink model, parameterized by base workspace 'theta'

    theta0 = [1.0; 1.0; 1.0];  % initial guess
    lb = [0.5; 0.5; 0.5];
    ub = [1.5; 1.5; 1.5];

    cost_fun = @(theta) calibrationObjective(theta, realData, modelName);

    opts = optimoptions('patternsearch', ...
        'Display', 'iter', ...
        'MaxIterations', 50);

    [theta_best, J_best] = patternsearch(cost_fun, theta0, ...
        [], [], [], [], lb, ub, [], opts);
end

function J = calibrationObjective(theta, realData, modelName)
    assignin('base', 'theta', theta);  % push to base workspace

    J = 0.0;
    for i = 1:numel(realData)
        exp = realData(i);
        in = Simulink.SimulationInput(modelName);
        in = in.setExternalInput([exp.t, exp.u]);
        out = sim(in, 'ShowProgress', 'off');

        y_sim = out.logsout.getElement('y').Values.Data;
        y_real = exp.y_real;

        % simple quadratic feature mismatch on joint angles
        diff = y_real - y_sim;
        J = J + sum(diff(:).^2);
    end
end
      

In a more detailed implementation, theta would parameterize link inertias, joint damping, and sensor latencies inside the Simulink model via mask parameters or workspace variables. The optimizer requires only a scalar cost from calibrationObjective.

Mathematica is well-suited for defining calibration objectives in a symbolic or functional style, then using NMinimize or FindMinimum. Suppose we have imported real and simulated trajectories as time series; we can define a mismatch objective and search for optimal parameters.

(* realData is a list of associations with keys "u", "yReal", "t" *)
(* simRollout[theta_, exp_] returns a time-aligned simulated trajectory *)

trajectoryFeatures[traj_] := Module[
  {q = traj["q"], qd = traj["qd"], ee = traj["eePos"], feats},
  feats = Join[
    Mean /@ {q, qd, ee},
    StandardDeviation /@ {q, qd, ee}
  ];
  Flatten[feats]
];

featureMismatch[realTraj_, simTraj_] := Module[
  {fr, fs, diff},
  fr = trajectoryFeatures[realTraj];
  fs = trajectoryFeatures[simTraj];
  diff = fr - fs;
  diff.diff
];

calibrationObjective[theta_List, realData_List] := Total[
  featureMismatch[#["realTraj"], simRollout[theta, #]] & /@ realData
];

thetaVars = {theta1, theta2, theta3};

sol = NMinimize[
  calibrationObjective[thetaVars, realData],
  Thread[{theta1, theta2, theta3} \[Element] Interval[{0.5, 1.5}]]
];

thetaStar = thetaVars /. Last[sol];
      

This approach can also be extended to distributional calibration, where trajectoryFeatures returns kernel embeddings or other statistics designed to capture subtle aspects of the trajectories.

Problem 1 (Feature vs State Matching): Consider two calibration objectives for a manipulator:

\[ J_{\text{state}}(\boldsymbol{\theta}) = \sum_{k=0}^{T} \big\| \mathbf{x}^{\text{real}}_k - \mathbf{x}^{\text{sim}}_k(\boldsymbol{\theta}) \big\|^2, \]

\[ J_{\text{task}}(\boldsymbol{\theta}) = \sum_{k=0}^{T} \big\| \mathbf{p}^{\text{real}}_k - \mathbf{p}^{\text{sim}}_k(\boldsymbol{\theta}) \big\|^2, \]

where \( \mathbf{x}_k \) is the full state and \( \mathbf{p}_k \) is the end-effector position. Explain when \( J_{\text{task}} \) may be preferable for sim-to-real transfer of manipulation policies, even if \( J_{\text{state}} \) appears more informative.

Solution: Many manipulation policies depend primarily on task-space behavior (e.g., end-effector pose and contacts) rather than the exact evolution of all joint states. Minimizing \( J_{\text{state}} \) forces the simulator to match unimportant details, such as internal joint configurations that do not affect the task or closed-loop policy. This can lead to overfitting to specific trajectories and misallocation of parameter capacity. In contrast, \( J_{\text{task}} \) focuses calibration on quantities that directly impact task success and policy behavior. For example, matching end-effector paths and contact timings ensures that grasping and placing motions behave similarly in sim and reality, which is precisely what matters for sim-to-real transfer.

Problem 2 (Distributional vs Pointwise Calibration): Let \( p^{\text{real}}(\boldsymbol{\phi}) \) and \( p^{\text{sim}}(\boldsymbol{\phi};\boldsymbol{\theta}) \) be the stationary feature distributions induced by a stochastic policy on the real robot and in simulation. Compare the following two objectives:

\[ J_{\text{traj}}(\boldsymbol{\theta}) = \mathbb{E}\Big[ \big\| \boldsymbol{\phi}^{\text{real}}_k - \boldsymbol{\phi}^{\text{sim}}_k(\boldsymbol{\theta}) \big\|^2 \Big], \]

\[ J_{\text{MMD}}(\boldsymbol{\theta}) = \operatorname{MMD}^2\!\big( p^{\text{real}}(\boldsymbol{\phi}), p^{\text{sim}}(\boldsymbol{\phi};\boldsymbol{\theta}) \big). \]

Explain why \( J_{\text{MMD}} \) may be more robust when the policy is stochastic or when time alignment between real and simulated trajectories is imperfect.

Solution: The objective \( J_{\text{traj}} \) assumes a meaningful pointwise pairing between \( \boldsymbol{\phi}^{\text{real}}_k \) and \( \boldsymbol{\phi}^{\text{sim}}_k(\boldsymbol{\theta}) \), which is violated if the policy is stochastic or if small timing mismatches accumulate (e.g., due to delays or contact events). In such cases, even a well-calibrated simulator can incur large pointwise errors because the systems visit similar regions of state space at slightly different times. In contrast, \( J_{\text{MMD}} \) compares feature distributions, which are invariant to permutations of samples and thus to time misalignment. If the simulator and real robot produce trajectories that explore the same regions of task-relevant state space with similar frequencies, \( J_{\text{MMD}} \) can be small even when pointwise differences are large, correctly indicating good qualitative calibration.

Problem 3 (Role of Experiment Excitation): Suppose that, during calibration, the robot executes only a single low-speed trajectory that keeps joint velocities and accelerations close to zero. Argue qualitatively which types of simulator parameters (e.g., inertias, friction, actuator saturation, sensor latency) can be meaningfully calibrated from such data, and which cannot.

Solution: Low-speed, nearly static motion provides limited information about dynamic effects. In particular:

• Inertias: difficult to calibrate, because inertial effects are proportional to accelerations and are nearly invisible when accelerations are small.
• Friction: certain static friction terms (e.g., Coulomb friction) may be partially identifiable if the motion crosses stick-slip thresholds, but viscous friction that scales with velocity is poorly excited.
• Actuator saturation and torque limits: essentially unobservable, since commands remain in a small range far from saturation.
• Sensor latency: may be weakly observable if the control loop relies on feedback, but small delays can be masked by low dynamics.

In summary, such a trajectory might be adequate to calibrate static offsets (e.g., joint angle biases) but insufficient for meaningful dynamic calibration. Informative calibration must include richer motion with substantial velocities and accelerations.

Problem 4 (Calibration and Domain Randomization): Let \( p_0(\boldsymbol{\theta}) \) be a wide domain-randomization prior, and suppose we perform calibration to obtain a posterior-like distribution \( p_{\text{cal}}(\boldsymbol{\theta}) \) that places most mass near a region of low trajectory mismatch. Explain why training policies under \( p_{\text{cal}} \) can improve both sample efficiency and real-world performance compared to training under \( p_0 \).

Solution: Under the wide prior \( p_0 \), the simulator often uses parameter settings far from those realized in the real robot. Policies must therefore be robust to extreme variations, making learning harder and possibly biasing policies toward overly conservative behaviors that work everywhere but are suboptimal in reality. After calibration, \( p_{\text{cal}} \) concentrates on parameter regions that are consistent with observed trajectories, so simulated experiences resemble the real robot more closely. This tightens the sim-to-real gap and reduces the burden on the policy to handle unrealistic scenarios, improving both learning efficiency (less variance in training outcomes) and real-world performance (the training distribution better matches deployment conditions).

Problem 5 (Overfitting in Calibration): Suppose we calibrate a simulator using a single policy \( \pi_0 \) and obtain parameters \( \boldsymbol{\theta}^{\star} \) that minimize \( J(\boldsymbol{\theta}) \) for trajectories generated by \( \pi_0 \). Later we train a new policy \( \pi_1 \) in this calibrated simulator and observe a large sim-to-real gap. Provide a conceptual explanation for this phenomenon and suggest a mitigation strategy.

Solution: Calibration using only trajectories from \( \pi_0 \) constrains the simulator to be accurate primarily along the subset of state-action space visited by that policy. When a new policy \( \pi_1 \) explores different regions (e.g., higher speeds, different contact patterns), the simulator can be poorly calibrated there, leading to a sim-to-real gap. This is an instance of overfitting the simulator to a narrow behavior distribution. A mitigation strategy is to design calibration experiments that cover a richer set of behaviors, e.g., by using multiple policies, exploratory control signals, or trajectory optimization that excites diverse states. Alternatively, one can iteratively refine calibration as policies evolve, repeatedly collecting new real data under updated policies and updating the calibration objective.

In this lesson, we formalized simulation calibration as an optimization problem over simulator parameters, using trajectory-level mismatch objectives based on task-relevant features and distributional metrics. We discussed the importance of experiment design for informative calibration, the relationship between calibration and domain randomization, and illustrated black-box calibration loops in Python, C++, Java, MATLAB/Simulink, and Mathematica. The key message is that calibration should be systematic, objective-driven, and tightly coupled to the task and policies of interest, rather than ad hoc parameter tweaking.