At this stage the capstone project is no longer an abstract method; it is an implemented autonomy stack with well-defined inputs and outputs. We formalize the canonical discrete-time closed-loop model:

\[ x_{t+1} = f(x_t, u_t, w_t), \quad o_t = h(x_t, v_t), \quad u_t = \pi_{\boldsymbol{\theta}}(o_{0:t}, g) \]

where \( x_t \in \mathcal{X} \) is the robot–world state, \( u_t \in \mathcal{U} \) is the control input, \( o_t \in \mathcal{O} \) is the observation, \( g \) encodes the task goal, and \( w_t, v_t \) are process and sensor noise. The policy \( \pi_{\boldsymbol{\theta}} \) may arise from planning (e.g., TAMP, sampling-based motion planning), reinforcement learning, or a hybrid of both (as covered in earlier chapters).

The implementation sprint is about realizing a particular \( \pi_{\boldsymbol{\theta}} \) and its supporting modules (perception, mapping, planning, control) so that it optimizes a task-specific performance functional

\[ J(\boldsymbol{\theta}) = \mathbb{E}\!\left[ \sum_{t=0}^{T-1} \gamma^t c(x_t, u_t) + \gamma^T c_T(x_T) \right], \quad 0 \le \gamma \lt 1 \]

and respects safety or task constraints such as collision avoidance and joint limits encoded as \( g_i(x_t, u_t) \le 0 \), for inequality constraint functions \( g_i \) derived from configuration-space obstacles and dynamics bounds.

We summarize the sprint workflow and the mid-review gate as a pipeline:

flowchart TD
  A["Freeze spec (J, constraints, interfaces)"] --> B["Implement core modules: perception/planning/control"]
  B --> C["Integrate into closed-loop simulation"]
  C --> D["Run evaluation suite (N rollouts)"]
  D --> E["Aggregate metrics (J_hat, p_succ, safety)"]
  E --> F{"Mid-review: \nmeets minimum bar?"}
  F -->|yes| G["Lock interfaces; \nschedule final experiments"]
  F -->|no| H["Diagnose bottlenecks; \nupdate design or method"]
        

The remainder of this lesson is about making each arrow in this diagram mathematically precise and implementable across different programming environments.

For an advanced autonomy stack, each module must have a contract expressed in terms of signals and quantitative error bounds. A typical decomposition is

• Perception: mapping sensor history to a belief.

  \[ b_t = \varphi(s_{0:t}) \in \mathcal{B} \]

• Planning / decision making: mapping belief and goal to a reference trajectory or high-level actions.

  \[ \xi_t = \Pi(b_t, g) = (x_t^{\ast}, x_{t+1}^{\ast}, \dots, x_{t+H}^{\ast}) \]

• Feedback control: tracking the reference.

  \[ u_t = \kappa(x_t^{\ast}, b_t) \]

To reason about implementation quality in the sprint, we postulate contracts like

\[ \mathbb{E}\big[\|b_t - x_t\|\big] \le \varepsilon_{\text{perc}}, \quad \mathrm{dist}\_{\text{C-space}}(\xi_t, \mathcal{X}_{\text{free}}) \ge \delta_{\text{plan}}, \quad \|u_t - u_t^{\ast}\| \le \varepsilon_{\text{ctrl}}, \]

where \( u_t^{\ast} \) is the ideal tracking input that would perfectly follow \( \xi_t \).

Suppose the dynamics are Lipschitz in both state and input,

\[ \|f(x, u, w) - f(x', u', w)\| \le L_x \|x - x'\| + L_u \|u - u'\| \]

for constants \( L_x, L_u \ge 0 \). Define the tracking error \( e_t = x_t - x_t^{\ast} \), and assume the reference \( x_t^{\ast} \) itself satisfies the nominal dynamics with input \( u_t^{\ast} \) (no noise). Then

\[ \begin{aligned} \|e_{t+1}\| &= \|f(x_t, u_t, w_t) - f(x_t^{\ast}, u_t^{\ast}, 0)\| \\ &\le L_x \|x_t - x_t^{\ast}\| + L_u \|u_t - u_t^{\ast}\| \\ &= L_x \|e_t\| + L_u \|u_t - u_t^{\ast}\|. \end{aligned} \]

During the sprint, if the controller implementation can guarantee \( \|u_t - u_t^{\ast}\| \le \varepsilon_{\text{ctrl}} \) for all \( t \) (e.g., via closed-loop gain bounds derived earlier in the course), we obtain the recursive bound

\[ \|e_{t+1}\| \le L_x \|e_t\| + L_u \varepsilon_{\text{ctrl}}. \]

Unrolling this recursion gives

\[ \|e_t\| \le L_x^t \|e_0\| + L_u \varepsilon_{\text{ctrl}} \sum_{k=0}^{t-1} L_x^k = \begin{cases} L_x^t \|e_0\| + L_u \varepsilon_{\text{ctrl}} \dfrac{1 - L_x^t}{1 - L_x}, & \text{if } L_x \ne 1, \\ \|e_0\| + t L_u \varepsilon_{\text{ctrl}}, & \text{if } L_x = 1. \end{cases} \]

This bound turns abstract stability analysis from earlier chapters into a concrete implementation requirement: the realized tracking error during simulation should be below the bound induced by your chosen \( L_x, L_u \) and \( \varepsilon_{\text{ctrl}} \). Violations at mid-review signal either a modeling error or a bug in the control implementation.

The mid-review decision relies on empirical estimates of performance. Let \( \omega \sim \mathbb{P} \) parameterize task instances and randomness (initial states, obstacles, noise, etc.). For each rollout \( i = 1, \dots, N \), draw \( \omega^{(i)} \) and simulate the closed loop to obtain:

• a success indicator \( S^{(i)} \in \{0,1\} \),
• a cumulative cost \( J^{(i)} \),
• a safety violation count \( V^{(i)} \) (e.g., near-collision events).

The empirical estimators for success probability, expected cost, and violation rate are

\[ \hat{p}\_{\text{succ}} = \frac{1}{N} \sum_{i=1}^N S^{(i)}, \quad \hat{J} = \frac{1}{N} \sum_{i=1}^N J^{(i)}, \quad \hat{p}\_{\text{viol}} = \frac{1}{N} \sum_{i=1}^N \mathbf{1}\{V^{(i)} \gt 0\}. \]

Assuming the rollouts are i.i.d., these estimators are unbiased. For example,

\[ \mathbb{E}[\hat{p}\_{\text{succ}}] = \frac{1}{N} \sum_{i=1}^N \mathbb{E}[S^{(i)}] = \mathbb{E}[S^{(1)}] = p\_{\text{succ}}, \]

where \( p\_{\text{succ}} = \mathbb{P}(S = 1) \) under the rollout distribution.

For the Bernoulli estimator, the variance is

\[ \operatorname{Var}(\hat{p}\_{\text{succ}}) = \frac{1}{N^2} \sum_{i=1}^N \operatorname{Var}(S^{(i)}) = \frac{1}{N} p\_{\text{succ}} (1 - p\_{\text{succ}}). \]

By Chebyshev's inequality,

\[ \mathbb{P}\big(|\hat{p}\_{\text{succ}} - p\_{\text{succ}}| \ge \varepsilon\big) \le \frac{\operatorname{Var}(\hat{p}\_{\text{succ}})}{\varepsilon^2} = \frac{p\_{\text{succ}} (1 - p\_{\text{succ}})}{N \varepsilon^2} \le \frac{1}{4 N \varepsilon^2}, \]

since \( p\_{\text{succ}} (1 - p\_{\text{succ}}) \le 1/4 \). Therefore, if we want \( \mathbb{P}(|\hat{p}\_{\text{succ}} - p\_{\text{succ}}| \ge \varepsilon) \le \delta \), it is sufficient to take

\[ N \ge \frac{1}{4 \varepsilon^2 \delta}. \]

This provides a principled way to set the number of simulation rollouts for the mid-review: choose tolerances \( \varepsilon \) (accuracy) and \( \delta \) (risk of mis-estimation), and compute \( N \). For costs \( J^{(i)} \), similar reasoning applies using their variance instead of Bernoulli variance.

For a capstone project, the mid-review should be governed by an explicit decision rule, not by intuition alone. Let the performance metrics vector be

\[ \mathbf{m}(\boldsymbol{\theta}) = \big( p\_{\text{succ}}(\boldsymbol{\theta}), \mathbb{E}[J(\boldsymbol{\theta})], p\_{\text{viol}}(\boldsymbol{\theta}) \big). \]

Suppose your design requirements are

\[ p\_{\text{succ}}(\boldsymbol{\theta}) \ge p\_{\min}, \quad \mathbb{E}[J(\boldsymbol{\theta})] \le J\_{\max}, \quad p\_{\text{viol}}(\boldsymbol{\theta}) \le \alpha. \]

At mid-review you only have empirical estimates. A conservative decision rule is

\[ \begin{aligned} \hat{p}\_{\text{succ}} - \beta\_{\text{succ}} \hat{\sigma}\_{\text{succ}} &\ge p\_{\min}, \\ \hat{J} + \beta_J \hat{\sigma}_J &\le J\_{\max}, \\ \hat{p}\_{\text{viol}} + \beta\_{\text{viol}} \hat{\sigma}\_{\text{viol}} &\le \alpha, \end{aligned} \]

where \( \hat{\sigma}\_{\cdot} \) are the sample standard deviations normalized by \( \sqrt{N} \), and \( \beta\_{\cdot} \) are safety factors (e.g., approximating normal quantiles). If all three inequalities hold, the project is deemed on track.

The corresponding mid-review flow is:

flowchart TD
  A["Run N rollouts with current implementation"] --> B["Compute m_hat = (p_succ_hat, J_hat, p_viol_hat)"]
  B --> C["Compare against thresholds with safety margins"]
  C --> D{"All constraints \nsatisfied?"}
  D -->|yes| E["Proceed to final \nintegration & polishing"]
  D -->|no| F["Identify weakest metric; \nschedule focused sprint"]
        

This creates a mathematically grounded link between your earlier theoretical choices (objective and constraints) and concrete go/no-go decisions in the implementation sprint.

In Python, we can construct a modular evaluation harness around your autonomy stack. You will typically rely on robotics libraries such as numpy, scipy, pybullet, mujoco, or ROS-based Python bindings, but here we keep the abstraction generic.

import numpy as np
from dataclasses import dataclass
from typing import Protocol, Any, Dict

# --- Core protocols for modules ---

class Perception(Protocol):
    def reset(self) -> None: ...
    def update(self, sensor: np.ndarray) -> np.ndarray: ...

class Planner(Protocol):
    def plan(self, belief: np.ndarray, goal: np.ndarray) -> Dict[str, Any]: ...

class Controller(Protocol):
    def reset(self) -> None: ...
    def compute_control(self, x_ref: np.ndarray, belief: np.ndarray) -> np.ndarray: ...

class Simulator(Protocol):
    def reset(self, omega: Dict[str, Any]) -> Dict[str, Any]: ...
    def step(self, u: np.ndarray) -> Dict[str, Any]:
        """
        Returns dict with keys: 'state', 'sensor', 'done', 'collision', 'cost'
        """
        ...

@dataclass
class Metrics:
    J: float
    success: bool
    violation: bool

# --- Single rollout ---

def run_episode(sim: Simulator,
                perc: Perception,
                planner: Planner,
                ctrl: Controller,
                omega: Dict[str, Any],
                horizon: int) -> Metrics:
    sim_state = sim.reset(omega)
    perc.reset()
    ctrl.reset()

    J = 0.0
    violated = False
    success = False

    for t in range(horizon):
        sensor = sim_state["sensor"]
        belief = perc.update(sensor)
        goal = sim_state["goal"]

        plan = planner.plan(belief, goal)
        x_ref = plan["x_ref"]  # e.g., reference state for this step

        u = ctrl.compute_control(x_ref, belief)
        sim_state = sim.step(u)

        J += float(sim_state["cost"])
        if sim_state.get("collision", False):
            violated = True

        if sim_state.get("done", False):
            success = not violated and sim_state.get("task_success", True)
            break

    # If we never reached 'done', treat as failure
    return Metrics(J=J, success=success, violation=violated)

# --- Monte Carlo evaluation ---

def evaluate_stack(sim: Simulator,
                   perc: Perception,
                   planner: Planner,
                   ctrl: Controller,
                   N: int,
                   horizon: int,
                   rng: np.random.Generator) -> Dict[str, float]:
    Js = []
    successes = []
    violations = []

    for i in range(N):
        omega = {
            "init_state": rng.normal(size=sim.state_dim),
            "noise_seed": int(rng.integers(0, 2**31 - 1))
        }
        metrics = run_episode(sim, perc, planner, ctrl, omega, horizon)
        Js.append(metrics.J)
        successes.append(1 if metrics.success else 0)
        violations.append(1 if metrics.violation else 0)

    Js = np.array(Js, dtype=float)
    successes = np.array(successes, dtype=float)
    violations = np.array(violations, dtype=float)

    results = {
        "J_hat": float(Js.mean()),
        "J_std_over_sqrtN": float(Js.std(ddof=1) / np.sqrt(N)),
        "p_succ_hat": float(successes.mean()),
        "p_succ_std_over_sqrtN": float(successes.std(ddof=1) / np.sqrt(N)),
        "p_viol_hat": float(violations.mean()),
        "p_viol_std_over_sqrtN": float(violations.std(ddof=1) / np.sqrt(N)),
    }
    return results


if __name__ == "__main__":
    rng = np.random.default_rng(0)
    # Instantiate your concrete modules here.
    # sim = ...
    # perc = ...
    # planner = ...
    # ctrl = ...
    # results = evaluate_stack(sim, perc, planner, ctrl, N=100, horizon=200, rng=rng)
    # print(results)
      

The output of evaluate_stack can be plugged directly into the mid-review inequalities from Section 4 to automate pass/fail decisions and to generate plots for your progress report.

In C++, the sprint typically integrates with robotics middleware such as ROS 2, as well as libraries like Eigen, OMPL, or custom dynamics engines. Below is a minimal, library-agnostic skeleton capturing the same evaluation logic as the Python code.

#include <iostream>
#include <vector>
#include <random>
#include <cmath>

struct Metrics {
    double J;
    bool success;
    bool violation;
};

struct SimState {
    std::vector<double> state;
    std::vector<double> sensor;
    std::vector<double> goal;
    bool done;
    bool collision;
    bool task_success;
    double cost;
};

class Simulator {
public:
    int state_dim;

    virtual SimState reset(unsigned int noise_seed) = 0;
    virtual SimState step(const std::vector<double>& u) = 0;
    virtual ~Simulator() = default;
};

class Perception {
public:
    virtual void reset() = 0;
    virtual std::vector<double> update(const std::vector<double>& sensor) = 0;
    virtual ~Perception() = default;
};

class Planner {
public:
    virtual std::vector<double> plan(const std::vector<double>& belief,
                                        const std::vector<double>& goal) = 0;
    virtual ~Planner() = default;
};

class Controller {
public:
    virtual void reset() = 0;
    virtual std::vector<double> computeControl(const std::vector<double>& x_ref,
                                                  const std::vector<double>& belief) = 0;
    virtual ~Controller() = default;
};

Metrics runEpisode(Simulator& sim,
                   Perception& perc,
                   Planner& planner,
                   Controller& ctrl,
                   unsigned int noise_seed,
                   int horizon) {
    SimState sim_state = sim.reset(noise_seed);
    perc.reset();
    ctrl.reset();

    double J = 0.0;
    bool violated = false;
    bool success = false;

    for (int t = 0; t < horizon; ++t) {
        std::vector<double> belief = perc.update(sim_state.sensor);
        std::vector<double> x_ref = planner.plan(belief, sim_state.goal);
        std::vector<double> u = ctrl.computeControl(x_ref, belief);
        sim_state = sim.step(u);

        J += sim_state.cost;
        if (sim_state.collision) {
            violated = true;
        }
        if (sim_state.done) {
            success = !violated && sim_state.task_success;
            break;
        }
    }

    Metrics m;
    m.J = J;
    m.success = success;
    m.violation = violated;
    return m;
}

struct EvalResults {
    double J_hat;
    double J_std_over_sqrtN;
    double p_succ_hat;
    double p_succ_std_over_sqrtN;
    double p_viol_hat;
    double p_viol_std_over_sqrtN;
};

EvalResults evaluateStack(Simulator& sim,
                          Perception& perc,
                          Planner& planner,
                          Controller& ctrl,
                          int N,
                          int horizon) {
    std::mt19937 rng(0);
    std::uniform_int_distribution<unsigned int> seed_dist(0, 0xffffffffu);

    std::vector<double> Js;
    std::vector<double> successes;
    std::vector<double> violations;
    Js.reserve(N);
    successes.reserve(N);
    violations.reserve(N);

    for (int i = 0; i < N; ++i) {
        unsigned int seed = seed_dist(rng);
        Metrics m = runEpisode(sim, perc, planner, ctrl, seed, horizon);
        Js.push_back(m.J);
        successes.push_back(m.success ? 1.0 : 0.0);
        violations.push_back(m.violation ? 1.0 : 0.0);
    }

    auto mean_std_over_sqrtN = [N](const std::vector<double>& xs) {
        double mean = 0.0;
        for (double x : xs) mean += x;
        mean /= static_cast<double>(N);

        double var = 0.0;
        for (double x : xs) {
            double d = x - mean;
            var += d * d;
        }
        var /= static_cast<double>(N - 1);
        double std_over_sqrtN = std::sqrt(var / static_cast<double>(N));
        return std::make_pair(mean, std_over_sqrtN);
    };

    auto J_stats = mean_std_over_sqrtN(Js);
    auto s_stats = mean_std_over_sqrtN(successes);
    auto v_stats = mean_std_over_sqrtN(violations);

    EvalResults r;
    r.J_hat = J_stats.first;
    r.J_std_over_sqrtN = J_stats.second;
    r.p_succ_hat = s_stats.first;
    r.p_succ_std_over_sqrtN = s_stats.second;
    r.p_viol_hat = v_stats.first;
    r.p_viol_std_over_sqrtN = v_stats.second;
    return r;
}

int main() {
    // Instantiate concrete Simulator, Perception, Planner, Controller here,
    // then call evaluateStack and compare with mid-review thresholds.
    return 0;
}
      

This structure is directly compatible with ROS nodes: each virtual method can be implemented by classes that wrap ROS topics, services, or actions for your specific robot platform.

While Java is less common for low-level control, it appears in simulation middleware and high-level orchestration. Below, Java interfaces mimic the same autonomy stack and evaluation logic, suitable for integration with Java-based simulators or task planners.

public interface Simulator {
    SimState reset(long noiseSeed);
    SimState step(double[] u);
    int getStateDim();
}

public class SimState {
    public double[] state;
    public double[] sensor;
    public double[] goal;
    public boolean done;
    public boolean collision;
    public boolean taskSuccess;
    public double cost;
}

public interface Perception {
    void reset();
    double[] update(double[] sensor);
}

public interface Planner {
    double[] plan(double[] belief, double[] goal);
}

public interface Controller {
    void reset();
    double[] computeControl(double[] xRef, double[] belief);
}

public class Metrics {
    public double J;
    public boolean success;
    public boolean violation;
}

public class Evaluation {

    public static Metrics runEpisode(
            Simulator sim,
            Perception perc,
            Planner planner,
            Controller ctrl,
            long noiseSeed,
            int horizon) {

        SimState state = sim.reset(noiseSeed);
        perc.reset();
        ctrl.reset();

        double J = 0.0;
        boolean violated = false;
        boolean success = false;

        for (int t = 0; t < horizon; t++) {
            double[] belief = perc.update(state.sensor);
            double[] xRef = planner.plan(belief, state.goal);
            double[] u = ctrl.computeControl(xRef, belief);
            state = sim.step(u);

            J += state.cost;
            if (state.collision) {
                violated = true;
            }
            if (state.done) {
                success = !violated && state.taskSuccess;
                break;
            }
        }

        Metrics m = new Metrics();
        m.J = J;
        m.success = success;
        m.violation = violated;
        return m;
    }

    public static double[] evaluateStack(
            Simulator sim,
            Perception perc,
            Planner planner,
            Controller ctrl,
            int N,
            int horizon) {

        java.util.Random rng = new java.util.Random(0L);

        double[] Js = new double[N];
        double[] succ = new double[N];
        double[] viol = new double[N];

        for (int i = 0; i < N; i++) {
            long seed = rng.nextLong();
            Metrics m = runEpisode(sim, perc, planner, ctrl, seed, horizon);
            Js[i] = m.J;
            succ[i] = m.success ? 1.0 : 0.0;
            viol[i] = m.violation ? 1.0 : 0.0;
        }

        double[] res = new double[6];
        res[0] = mean(Js);
        res[1] = stdOverSqrtN(Js);
        res[2] = mean(succ);
        res[3] = stdOverSqrtN(succ);
        res[4] = mean(viol);
        res[5] = stdOverSqrtN(viol);
        return res;
    }

    private static double mean(double[] xs) {
        double s = 0.0;
        for (double x : xs) s += x;
        return s / xs.length;
    }

    private static double stdOverSqrtN(double[] xs) {
        double m = mean(xs);
        double v = 0.0;
        for (double x : xs) {
            double d = x - m;
            v += d * d;
        }
        v /= (xs.length - 1);
        return Math.sqrt(v / xs.length);
    }
}
      

The evaluateStack method returns a 6-entry array with the same semantics as in the Python and C++ implementations, which can then be visualized or used for automated regression tests within Java-based tooling.

In MATLAB/Simulink, the implementation sprint usually uses a Simulink model for plant dynamics and controllers, with MATLAB scripts handling Monte Carlo experiments and metric aggregation.

function results = evaluate_autonomy_stack(modelName, N, horizon, Ts)
% modelName: name of Simulink model
% N: number of Monte Carlo runs
% horizon: number of steps
% Ts: sampling time

J = zeros(N,1);
succ = zeros(N,1);
viol = zeros(N,1);

for i = 1:N
    rng(i);  % deterministic noise seed

    % Configure initial conditions and parameters for this run
    omega.init_state = randn(6,1);
    omega.goal = [1; 0; 0; 0; 0; 0];

    % For a more sophisticated setup, use Simulink.SimulationInput
    simIn = Simulink.SimulationInput(modelName);
    simIn = simIn.setVariable('omega', omega);
    simIn = simIn.setVariable('horizon', horizon);
    simIn = simIn.setVariable('Ts', Ts);

    simOut = sim(simIn);

    cost_signal = simOut.logsout.get('cost');
    J(i) = sum(cost_signal.Values.Data);

    coll_signal = simOut.logsout.get('collision');
    violated = any(coll_signal.Values.Data > 0.5);
    viol(i) = double(violated);

    success_signal = simOut.logsout.get('task_success');
    done_signal = simOut.logsout.get('done');
    if any(done_signal.Values.Data > 0.5) && ~violated
        succ(i) = double(any(success_signal.Values.Data > 0.5));
    else
        succ(i) = 0;
    end
end

results.J_hat = mean(J);
results.J_std_over_sqrtN = std(J, 1) / sqrt(N);
results.p_succ_hat = mean(succ);
results.p_succ_std_over_sqrtN = std(succ, 1) / sqrt(N);
results.p_viol_hat = mean(viol);
results.p_viol_std_over_sqrtN = std(viol, 1) / sqrt(N);
      

The Simulink model should log the relevant signals (cost, collision, task_success, done) using the Simulation Data Inspector or logsout to make this script reusable throughout the sprint.

Wolfram Mathematica is well suited to complement the sprint with symbolic reasoning and high-level Monte Carlo experiments. The snippet below defines discrete dynamics, simulates trajectories, and estimates success probability and expected cost.

(* Discrete dynamics x_{t+1} = f(x_t, u_t) for a simple planar robot *)
f[x_, u_] := x + u;

stageCost[x_, u_] := Norm[u]^2;
terminalCost[x_] := 10.0*Boole[Norm[x] > 0.1];

policy[x_] := Module[{u},
  (* Replace this with your planner/controller logic *)
  u = -0.5*x;
  u
];

simulateEpisode[x0_, horizon_] := Module[
  {x = x0, J = 0.0, t, u, done = False, success = False, violation = False},
  For[t = 0, t < horizon, t++,
    u = policy[x];
    J += stageCost[x, u];
    x = f[x, u];

    If[Norm[x] < 0.05,
      success = True;
      done = True;
      Break[];
    ];
    If[Norm[x] > 5.0,
      violation = True;
      done = True;
      Break[];
    ];
  ];
  If[!done, J += terminalCost[x]];
  {J, success, violation}
];

evaluateStack[N_, horizon_] := Module[
  {Js, succs, viols, episodes, means, stds},
  episodes = Table[
    With[{x0 = RandomVariate[NormalDistribution[0, 1], 2]},
      simulateEpisode[x0, horizon]
    ],
    {N}
  ];
  Js = episodes[[All, 1]];
  succs = Boole /@ episodes[[All, 2]];
  viols = Boole /@ episodes[[All, 3]];

  means = {Mean[Js], Mean[succs], Mean[viols]};
  stds = {StandardDeviation[Js], StandardDeviation[succs], StandardDeviation[viols]]}/Sqrt[N];
  <|
    "JHat" -> means[[1]],
    "JStdOverSqrtN" -> stds[[1]],
    "PSuccHat" -> means[[2]],
    "PSuccStdOverSqrtN" -> stds[[2]],
    "PViolHat" -> means[[3]],
    "PViolStdOverSqrtN" -> stds[[3]]
  |>
];

(* Example usage *)
(* evaluateStack[200, 100] *)
      

This kind of high-level prototyping can validate your theoretical assumptions and expected trends before porting the logic into the main language used by your project team.

Problem 1 (Sample Complexity for Mid-Review): Suppose the true success probability of your autonomy stack is \( p\_{\text{succ}} \). You want to estimate it with accuracy \( \varepsilon \) and confidence at least \( 1 - \delta \), i.e. \( \mathbb{P}(|\hat{p}\_{\text{succ}} - p\_{\text{succ}}| \le \varepsilon) \ge 1 - \delta \). Using only Chebyshev's inequality, derive a sufficient lower bound on the number of rollouts \( N \).

Solution: We know that for the Bernoulli estimator, \( \operatorname{Var}(\hat{p}\_{\text{succ}}) = p\_{\text{succ}}(1 - p\_{\text{succ}})/N \le 1/(4N) \). Chebyshev implies

\[ \mathbb{P}\big(|\hat{p}\_{\text{succ}} - p\_{\text{succ}}| \ge \varepsilon\big) \le \frac{\operatorname{Var}(\hat{p}\_{\text{succ}})}{\varepsilon^2} \le \frac{1}{4 N \varepsilon^2}. \]

To guarantee this is at most \( \delta \), it suffices that

\[ \frac{1}{4 N \varepsilon^2} \le \delta \quad \Rightarrow \quad N \ge \frac{1}{4 \varepsilon^2 \delta}. \]

Thus a sufficient condition is \( N = \left\lceil \frac{1}{4 \varepsilon^2 \delta} \right\rceil \).

Problem 2 (Error Propagation Bound): Assume the Lipschitz dynamics of Section 2 with constants \( L_x \in [0,1) \) and bound \( \|u_t - u_t^{\ast}\| \le \varepsilon_{\text{ctrl}} \) for all \( t \). Show that \( \|e_t\| \le \frac{L_u}{1 - L_x} \varepsilon_{\text{ctrl}} \) for all \( t \) in the worst case, assuming \( e_0 = 0 \).

Solution: From Section 2, \( \|e_{t+1}\| \le L_x \|e_t\| + L_u \varepsilon_{\text{ctrl}} \). With \( e_0 = 0 \), we obtain by induction

\[ \|e_t\| \le L_u \varepsilon_{\text{ctrl}} \sum_{k=0}^{t-1} L_x^k = L_u \varepsilon_{\text{ctrl}} \frac{1 - L_x^t}{1 - L_x} \le \frac{L_u}{1 - L_x} \varepsilon_{\text{ctrl}}, \]

since \( 0 \le L_x^t \le 1 \). Hence for all horizons \( t \), the worst-case tracking error is uniformly bounded by \( \frac{L_u}{1 - L_x} \varepsilon_{\text{ctrl}} \). This provides a clear numerical target for the mid-review based on the controller's implementation.

Problem 3 (Truncation of Discounted Cost): In practice you simulate only up to a finite horizon \( T \) even if the discount factor \( \gamma \lt 1 \) suggests an infinite horizon. Suppose the stage cost is uniformly bounded: \( |c(x_t, u_t)| \le C_{\max} \). Show that the truncation error

\[ \Delta J = \left| \mathbb{E}\Big[\sum_{t=0}^{\infty} \gamma^t c(x_t, u_t)\Big] - \mathbb{E}\Big[\sum_{t=0}^{T-1} \gamma^t c(x_t, u_t)\Big] \right| \]

is bounded by \( \Delta J \le \frac{\gamma^T}{1 - \gamma} C_{\max} \).

Solution: By the triangle inequality and boundedness of the cost,

\[ \begin{aligned} \Delta J &\le \mathbb{E}\Big[\sum_{t=T}^{\infty} \gamma^t |c(x_t, u_t)|\Big] \le \sum_{t=T}^{\infty} \gamma^t C_{\max} = C_{\max} \sum_{t=T}^{\infty} \gamma^t \\ &= C_{\max} \gamma^T \sum_{k=0}^{\infty} \gamma^k = C_{\max} \gamma^T \frac{1}{1 - \gamma}. \end{aligned} \]

Hence, for a given tolerable truncation error \( \Delta J_{\max} \), you can pick \( T \) such that \( \frac{\gamma^T}{1 - \gamma} C_{\max} \le \Delta J_{\max} \), and this choice should be documented at mid-review.

Problem 4 (Decision Rule Robustness): Consider only the success probability constraint, with threshold \( p\_{\min} \). You use the decision rule \( \hat{p}\_{\text{succ}} - \beta \hat{\sigma}\_{\text{succ}} \ge p\_{\min} \), where \( \hat{\sigma}\_{\text{succ}} \) is the empirical standard deviation divided by \( \sqrt{N} \). Explain qualitatively how increasing \( \beta \) affects false acceptance and false rejection rates at mid-review.

Solution: A larger \( \beta \) inflates the safety margin subtracted from \( \hat{p}\_{\text{succ}} \). This makes the decision rule more conservative: the system must demonstrate a higher empirical success probability to pass. Consequently, the probability of falsely accepting an underperforming system decreases, but the probability of falsely rejecting a system whose true \( p\_{\text{succ}} \) exceeds \( p\_{\min} \) increases. Choosing \( \beta \) therefore reflects a trade-off between these two types of error, analogous to selecting a confidence level in classical hypothesis testing.

Problem 5 (Coupling Metrics Across Modules): Suppose your perception module guarantees \( \mathbb{E}[\|b_t - x_t\|^2] \le \sigma_b^2 \), and your controller's cost function includes a quadratic tracking term \( c(x_t, u_t) = \|x_t - x_t^{\ast}\|^2 + \lambda \|u_t\|^2 \). Explain how an improvement in perception (smaller \( \sigma_b^2 \)) can be expected to impact the mid-review estimate \( \hat{J} \), assuming the planner and controller are unchanged.

Solution: Better perception reduces the expected state-estimation error. For fixed planner and controller, this generally reduces the mismatch between the used belief \( b_t \) and the true state \( x_t \), which in turn improves the tracking of the reference trajectory. That is, the actual tracking error \( \|x_t - x_t^{\ast}\| \) should decrease in expectation. Since \( c(x_t, u_t) \) is quadratic in this error, the expected stage cost \( \mathbb{E}[c(x_t, u_t)] \) decreases, and thus the Monte Carlo estimator \( \hat{J} \) should also decrease, all else equal. This illustrates that improving a single module (perception) can produce measurable gains in mid-review metrics even if the rest of the stack is unchanged.

In this lesson, we transformed the capstone implementation sprint and mid-review into a mathematically grounded process. We:

• Expressed the autonomy stack as a discrete-time closed-loop system with module contracts, and derived bounds on tracking error using Lipschitz dynamics.
• Formulated Monte Carlo estimators for success probability, expected cost, and safety violations, and derived sample-complexity guarantees using Chebyshev's inequality.
• Defined a mid-review decision rule based on empirical metrics and safety margins, making the go/no-go decision explicit and reproducible.
• Implemented language-agnostic evaluation harnesses in Python, C++, Java, MATLAB/Simulink, and Wolfram Mathematica, ensuring that the theoretical constructs are realized in actual code that you can extend within your capstone.

These tools and concepts ensure that your advanced autonomy project is not only implemented, but also evaluated in a principled and scientifically rigorous manner before moving on to full experimental validation in the next lesson.