A post-mortem for a robotics capstone is a structured, technical reflection after your final demo. Instead of simply labeling the robot as “good” or “bad”, we:

• Formally define performance metrics over experimental runs.
• Identify failure modes and their frequencies.
• Quantify trade-offs between tracking quality, control effort, and safety.
• Transform these findings into a mathematically grounded improvement plan.

Let \( k \) index design iterations (e.g., weeks or milestone builds) and \( s = 1,\dots,N_k \) index experimental runs in iteration \( k \). For each run we log:

• Reference trajectory \( r_{k,s}(t) \).
• Measured output (position, angle, etc.) \( y_{k,s}(t) \).
• Control input \( u_{k,s}(t) \) (if available).
• Binary or graded safety indicators (limit hits, collisions, etc.).

A post-mortem is then a mapping from raw logs to decisions about what to change in the next design iteration.

flowchart TD
  A["Define goals and metrics"] --> B["Collect logs from all runs"]
  B --> C["Compute tracking, effort, safety indicators"]
  C --> D["Classify runs: success, degraded, failure"]
  D --> E["Root-cause analysis (hardware, control, software)"]
  E --> F["Prioritize issues by impact and effort"]
  F --> G["Define improvement roadmap for next iteration"]
  G --> B["Re-test with updated design"]
        

From linear control you already know standard time-domain indices. We now apply them to the robot capstone context using discrete log data.

2.1 Tracking error

For run \( (k,s) \), define the tracking error

\[ e_{k,s}(t) = r_{k,s}(t) - y_{k,s}(t). \]

With sampled data at times \( t_0,\dots,t_N \) and step sizes \( \Delta t_i = t_i - t_{i-1} \), we form the discrete root-mean-square (RMS) error:

\[ E^{\text{RMS}}_{k,s} = \sqrt{\frac{1}{N+1}\sum_{i=0}^{N} e_{k,s}(t_i)^2}. \]

Another index is the integral of absolute error (IAE) approximation:

\[ E^{\text{IAE}}_{k,s} = \sum_{i=0}^{N} |e_{k,s}(t_i)|\,\Delta t_i. \]

2.2 Control effort

High control effort may cause actuator saturation or overheating. A discrete approximation of squared control effort is

\[ U^{2}_{k,s} = \sum_{i=0}^{N} u_{k,s}(t_i)^2\,\Delta t_i. \]

Using \( U^{2}_{k,s} \) together with \( E^{\text{RMS}}_{k,s} \) allows you to detect solutions that “cheat” by using very aggressive control action.

2.3 Safety and constraints

Suppose you impose a bound \( |y_{k,s}(t)| \leq y_{\max} \) to avoid collisions or joint limits. Define an indicator for each sample:

\[ X_{k,s}(t_i) = \begin{cases} 1 & \text{if } |y_{k,s}(t_i)| > y_{\max}, \\ 0 & \text{otherwise.} \end{cases} \]

The empirical violation rate in run \( (k,s) \) is

\[ V_{k,s} = \frac{1}{N+1}\sum_{i=0}^{N} X_{k,s}(t_i). \]

For a whole iteration \( k \) with \( N_k \) runs we aggregate:

\[ \bar{E}^{\text{RMS}}_{k} = \frac{1}{N_k}\sum_{s=1}^{N_k} E^{\text{RMS}}_{k,s}, \quad \bar{U}^{2}_{k} = \frac{1}{N_k}\sum_{s=1}^{N_k} U^{2}_{k,s}, \quad \bar{V}_{k} = \frac{1}{N_k}\sum_{s=1}^{N_k} V_{k,s}. \]

These three numbers characterize the typical behavior of the design at iteration \( k \).

For each run we can treat “failure” as a Bernoulli random variable. Let \( Z_{k,s} \in \{0,1\} \) denote:

• \( Z_{k,s} = 1 \) if the run fails (e.g., emergency stop).
• \( Z_{k,s} = 0 \) if the run succeeds.

For iteration \( k \), define the empirical failure probability

\[ \hat{p}_k = \frac{1}{N_k}\sum_{s=1}^{N_k} Z_{k,s}. \]

Under the standard Bernoulli model \( Z_{k,s} \sim \text{Bernoulli}(p_k) \) independently, we have

\[ \operatorname{E}[\hat{p}_k] = p_k, \quad \operatorname{Var}(\hat{p}_k) = \frac{p_k(1-p_k)}{N_k}. \]

A large-sample normal approximation gives a \( 95\% \) confidence interval

\[ \hat{p}_k \pm 1.96 \sqrt{\frac{\hat{p}_k(1-\hat{p}_k)}{N_k}}. \]

A good improvement roadmap should drive \( \hat{p}_k \) and its upper confidence bound downward over iterations.

To compare different iterations \( k \) we collect the aggregate metrics into a vector

\[ m_k = \begin{bmatrix} \bar{E}^{\text{RMS}}_{k} \\ \bar{U}^{2}_{k} \\ \hat{p}_k \end{bmatrix} \in \mathbb{R}^3. \]

Since these metrics have different units and scales, we normalize. For each component \( i \) choose a target value \( m_i^{\text{target}} \) and a scaling constant \( m_i^{\text{scale}} > 0 \), then define

\[ \tilde{m}_{k,i} = \frac{m_{k,i} - m_i^{\text{target}}}{m_i^{\text{scale}}}. \]

A dimensionless scalar score is obtained with a weight vector \( w \in \mathbb{R}^3 \) satisfying \( w_i \geq 0 \) and \( \sum_{i=1}^{3} w_i = 1 \):

\[ J_k = \sum_{i=1}^{3} w_i\,\tilde{m}_{k,i} = w^{\top}\tilde{m}_k. \]

Smaller values of \( J_k \) correspond to better overall performance (since each \( \tilde{m}_{k,i} \) measures deviation from target).

4.1 Monotonicity property

Suppose for two iterations \( k \) and \( \ell \) we have \( \tilde{m}_{k,i} \leq \tilde{m}_{\ell,i} \) for all \( i \) and at least one index has a strict inequality (meaning iteration \( k \) is not worse in any metric and strictly better in at least one).

Then, with all weights nonnegative, we obtain

\[ J_k - J_{\ell} = \sum_{i=1}^{3} w_i \bigl(\tilde{m}_{k,i} - \tilde{m}_{\ell,i}\bigr) \leq 0, \]

and the inequality is strict whenever at least one index with positive weight satisfies the strict improvement. Thus the scalar score preserves the intuitive partial ordering induced by the individual metrics.

4.2 Temporal smoothing

To avoid overreacting to a single bad or good trial, we can use an exponentially weighted moving average of scores:

\[ \bar{J}_K = (1-\lambda)\sum_{k=1}^{K}\lambda^{K-k} J_k, \quad 0 \leq \lambda \leq 1. \]

The parameter \( \lambda \) is often expressed via a time constant \( \tau_{\text{eval}} \) by

\[ \lambda = \exp\!\bigl(-T_{\text{iter}} / (\tau_{\text{eval}})\bigr), \]

where \( T_{\text{iter}} \) is an effective “age” of one iteration. Larger \( \tau_{\text{eval}} \) forgets history more slowly.

flowchart TD
  M["Aggregate metrics m_k for each iteration"] --> N["Normalize to dimensionless ~m_k"]
  N --> W["Choose weights w_i (team priorities)"]
  W --> J["Compute score J_k = w^T ~m_k"]
  J --> R["Sort iterations by J_k"]
  R --> P["Focus roadmap on iterations with poor J_k and dominant failure modes"]
        

Assume you have a CSV log with columns: run_id, t, r, y, u, and constraint_violation (1 if violated, 0 otherwise). The code below aggregates the metrics and computes a simple score \( J_k \).

import numpy as np
import pandas as pd

def compute_run_metrics(df_run):
    t = df_run["t"].to_numpy()
    r = df_run["r"].to_numpy()
    y = df_run["y"].to_numpy()
    u = df_run["u"].to_numpy()
    # Assume t is strictly increasing
    dt = np.diff(t, prepend=t[0])
    e = r - y

    rms = np.sqrt(np.mean(e**2))
    iae = np.sum(np.abs(e) * dt)
    u2 = np.sum(u**2 * dt)
    violation_rate = df_run["constraint_violation"].mean()
    return {
        "rms": rms,
        "iae": iae,
        "u2": u2,
        "violation_rate": violation_rate,
    }

# Load a single iteration's log (all runs for iteration k)
df = pd.read_csv("robot_logs_iteration_k.csv")

grouped = df.groupby("run_id")
rows = []
for run_id, df_run in grouped:
    rows.append({"run_id": run_id, **compute_run_metrics(df_run)})

run_metrics = pd.DataFrame(rows)

# Aggregate across runs
E_rms_bar = run_metrics["rms"].mean()
U2_bar = run_metrics["u2"].mean()
V_bar = run_metrics["violation_rate"].mean()

# Normalization constants and targets
target_rms = 0.01
scale_rms = 0.05
target_u2 = 0.0
scale_u2 = max(U2_bar, 1e-3)
target_v = 0.0
scale_v = 1.0

tilde_rms = (E_rms_bar - target_rms) / scale_rms
tilde_u2 = (U2_bar - target_u2) / scale_u2
tilde_v = (V_bar - target_v) / scale_v

# Weights: emphasize safety, then tracking, then effort
w_rms = 0.3
w_u2 = 0.2
w_v = 0.5

J_k = w_rms * tilde_rms + w_u2 * tilde_u2 + w_v * tilde_v
print("Iteration score J_k =", J_k)
      

You can repeat this for each iteration k and plot J_k versus iteration index to visualize progress in your improvement roadmap.

On the robot, you may not want to store every sample indefinitely. Instead, accumulate enough statistics to compute metrics at the end of a run. The following minimal class accumulates RMS error, IAE, and control effort in a control loop.

#include <cmath>
#include <cstdint>

struct RunMetrics {
    double sum_e2 = 0.0;
    double sum_abs_e_dt = 0.0;
    double sum_u2_dt = 0.0;
    std::uint64_t samples = 0;

    void accumulate(double e, double u, double dt) {
        sum_e2 += e * e;
        sum_abs_e_dt += std::fabs(e) * dt;
        sum_u2_dt += u * u * dt;
        ++samples;
    }

    double rms() const {
        if (samples == 0) return 0.0;
        return std::sqrt(sum_e2 / static_cast<double>(samples));
    }

    double iae() const {
        return sum_abs_e_dt;
    }

    double u2() const {
        return sum_u2_dt;
    }
};

// Example usage in a control loop
void control_loop(RunMetrics& metrics) {
    double t_prev = get_time();
    while (!end_of_run()) {
        double t = get_time();
        double dt = t - t_prev;
        t_prev = t;

        double r = get_reference();
        double y = get_measurement();
        double e = r - y;

        double u = compute_control(e);
        apply_control(u);

        metrics.accumulate(e, u, dt);
    }
}
      

At the end of the run, the controller can log metrics.rms(), metrics.iae(), and metrics.u2() along with safety flags to a file or middleware (e.g., ROS topic) for later analysis.

In Java you might implement a project-level metrics aggregator that processes log rows and computes per-run statistics for visualization in a GUI dashboard.

public final class RunMetrics {
    private double sumE2 = 0.0;
    private double sumAbsE = 0.0;
    private double sumU2 = 0.0;
    private double sumDt = 0.0;
    private long samples = 0L;

    public void accumulate(double e, double u, double dt) {
        sumE2 += e * e;
        sumAbsE += Math.abs(e) * dt;
        sumU2 += u * u * dt;
        sumDt += dt;
        samples++;
    }

    public double getRms() {
        if (samples == 0L) return 0.0;
        return Math.sqrt(sumE2 / (double) samples);
    }

    public double getIae() {
        return sumAbsE;
    }

    public double getU2() {
        return sumU2;
    }
}

// Example skeleton for a log-processing method
// logRows is a list of objects with fields t, r, y, u, runId
public static Map<Integer, RunMetrics> computeMetrics(List<LogRow> logRows) {
    Map<Integer, RunMetrics> map = new HashMap<>();
    Map<Integer, Double> lastTime = new HashMap<>();

    for (LogRow row : logRows) {
        int runId = row.getRunId();
        double t = row.getTime();
        double r = row.getReference();
        double y = row.getOutput();
        double u = row.getControl();

        double tPrev = lastTime.getOrDefault(runId, t);
        double dt = t - tPrev;
        lastTime.put(runId, t);

        RunMetrics m = map.computeIfAbsent(runId, k -> new RunMetrics());
        double e = r - y;
        m.accumulate(e, u, dt);
    }

    return map;
}
      

These per-run metrics can then be mapped to iteration-level scores \( J_k \) using the same formulas as in the Python code, but implemented in Java.

In MATLAB you can post-process data logged from Simulink or from your hardware experiments. Assume vectors t, r, y, and u are already in the workspace for one run.

% Assume t, r, y, u are column vectors of equal length
dt = [t(1); diff(t)];     % time steps
e = r - y;

E_rms = sqrt(mean(e.^2));
E_iae = sum(abs(e) .* dt);
U2 = sum(u.^2 .* dt);

% Basic overshoot and settling time for approximate step response
y_final = mean(y(end-50:end));   % average of final samples
y_max = max(y);
Mp = (y_max - y_final) / max(abs(y_final));   % relative overshoot

% Settling time: find last time index where |y - y_final| exceeds band
band = 0.02 * max(abs(y_final)); % 2 percent
idx = find(abs(y - y_final) > band, 1, "last");
if isempty(idx)
    Ts = 0.0;
else
    Ts = t(idx);
end

fprintf("RMS error: %g\n", E_rms);
fprintf("IAE:       %g\n", E_iae);
fprintf("U^2:       %g\n", U2);
fprintf("Overshoot: %g\n", Mp);
fprintf("Ts:        %g\n", Ts);
      

In Simulink, you typically log the necessary signals using To Workspace blocks or the data logging capabilities of scopes. Re-running this script for each run and each iteration \( k \) gives you the values needed to construct \( m_k \) and \( J_k \).

In this lesson we transformed the informal idea of a capstone “post-mortem” into a quantitative procedure. By defining tracking, effort, and safety metrics over logged trajectories, modeling failures probabilistically, and combining multiple objectives into a scalar score, we obtain a principled basis for ranking design iterations and planning improvements. Code snippets in Python, C++, Java, and MATLAB/Simulink illustrated how to compute these metrics in practice. The same ideas generalize to larger industrial projects where robot systems are continually monitored and refined over their whole lifecycle.