Consider a robot evolving in discrete time with physical state \( x_t \in \mathbb{R}^n \), control input \( u_t \in \mathbb{R}^m \), and sensor output \( y_t \in \mathbb{R}^p \). A simple state–space model is

\[ x_{t+1} = f(x_t, u_t), \qquad y_t = h(x_t). \]

An autonomy architecture specifies how actions are chosen from the sensor history:

\[ u_t = \pi_t\!\big(y_0, y_1, \dots, y_t\big), \]

where \( \pi_t \) is the control law (policy) implemented by software and hardware modules.

We distinguish three broad families:

• Reactive architectures: \( u_t \) depends only on the current observation \( y_t \), typically through simple rules or low-level feedback.
• Deliberative architectures: maintain an internal world model, reason ahead in time (planning), and then commit to a sequence of actions.
• Hybrid architectures: combine a fast reactive layer for immediate safety and servoing with slower deliberative layers for long-horizon goals.

Formally, we can view an architecture as a composition of modules acting on different time scales and information sets. Even if the implementation is complex (distributed nodes, middleware), the mathematical core is a mapping from sensor histories to actions.

flowchart TD
  Y["Sensors y_t"] --> R["Reactive map u_t = pi_R(y_t)"]
  R --> U["Actuators"]

  Y --> M["World model m_t"]
  M --> P["Planner: sequence u_{t:t+H}"]
  P --> E["Executive / supervisor"]
  E --> U

  classDef group fill:#ffffff,stroke:#000000,stroke-dasharray: 3 3;
  subgraph G1["Reactive"]
    Y --> R --> U
  end
  subgraph G2["Deliberative"]
    Y --> M --> P --> E --> U
  end
        

We formalize architectures by constraining the dependence structure of \( \pi_t \).

Reactive policy. A policy is reactive if

\[ u_t = \pi_R(y_t) \]

for some static function \( \pi_R : \mathbb{R}^p \to \mathbb{R}^m \). The policy ignores past history and uses no additional internal memory. Classical linear state feedback \( u_t = -K x_t \) is reactive with respect to the state (or estimated state).

Deliberative policy. A policy is deliberative if it maintains an internal model \( m_t \) updated by

\[ m_{t+1} = F(m_t, y_t, u_t), \]

and chooses actions using

\[ u_t = \pi_D\big(y_t, m_t\big). \]

Here \( m_t \) may encode a map, task graph, or higher-level symbolic state. Importantly, the pair \( (y_t, m_t) \) summarizes the entire history.

Hybrid policy. A policy is hybrid if

\[ u_t = \pi_R\!\big(y_t, g_t\big), \qquad g_t = G(m_t), \]

where \( g_t \) is a goal or reference generated by a slow deliberative process and \( \pi_R \) is a fast reactive controller that tracks \( g_t \).

2.1 A simple impossibility result for reactive policies

Consider a discrete state space \( X \), observation function \( o : X \to Y \), and action set \( U \). The robot must reach a goal set \( X_{\mathrm{goal}} \subset X \).

Proposition. Suppose there exist two states \( x^1, x^2 \in X \) such that:

• \( o(x^1) = o(x^2) = \bar y \), i.e., the sensor cannot distinguish them.
• Every successful path from \( x^1 \) to \( X_{\mathrm{goal}} \) must start with action \( u^1 \).
• Every successful path from \( x^2 \) to \( X_{\mathrm{goal}} \) must start with action \( u^2 \neq u^1 \).

Then no purely reactive policy \( \pi_R : Y \to U \) can solve the task from both initial states.

Proof. A reactive policy must satisfy

\[ \pi_R\big(o(x^1)\big) = \pi_R\big(o(x^2)\big) = \pi_R(\bar y), \]

because \( o(x^1) = o(x^2) = \bar y \). Thus the first action chosen at \( x^1 \) and \( x^2 \) is identical. If it equals \( u^1 \), then all trajectories from \( x^2 \) fail. If it equals \( u^2 \), then all trajectories from \( x^1 \) fail. Hence no reactive policy succeeds from both initial states. A policy with sufficient internal memory (e.g., a finite-state machine that remembers a prior observation) may succeed. \(\square\)

This illustrates why deliberation or at least internal memory is necessary in partially observable or information-limited environments.

Reactive architectures bind sensors to actuators through simple rules, gains, and saturations. They are close to classical feedback control, extended to more complex sensor spaces (e.g., range finders, cameras).

3.1 Static and piecewise policies

In many mobile robots, behavior is defined by a small set of rules. A generic piecewise reactive policy can be written as

\[ u_t = \begin{cases} g_1(y_t) & \text{if } y_t \in \mathcal{R}_1,\\ g_2(y_t) & \text{if } y_t \in \mathcal{R}_2,\\ \vdots & \vdots \\ g_K(y_t) & \text{if } y_t \in \mathcal{R}_K, \end{cases} \]

where the regions \( \mathcal{R}_k \subset \mathbb{R}^p \) are defined by inequalities (e.g., distance thresholds) and \( g_k \) are simple control laws (e.g., constant velocities or proportional controllers).

3.2 Priority-based behavior arbitration

A classical behavior-based scheme (e.g., subsumption) defines behaviors \( \beta_i : Y \to U \) with binary activation functions \( a_i : Y \to \{0,1\} \) and strict priorities \( 1 \lt 2 \lt \dots \lt N \). Let

\[ \mathcal{I}(y) = \{ i \in \{1,\dots,N\} \mid a_i(y) = 1 \} \]

be the set of active behaviors at observation \( y \), and define

\[ k^*(y) = \max \mathcal{I}(y), \]

i.e., the highest-priority active behavior. The executed control is

\[ u = \beta_{k^*(y)}(y). \]

This structure is still reactive (no internal memory) but can express complex emergent behavior through carefully designed regions and priorities. Its main advantages are:

• Very low computational cost per control cycle.
• Robustness to modeling error (no explicit world model).
• Immediate response to unexpected changes, limited only by sensor and CPU latency.

Limitations stem from the impossibility result above: purely reactive systems cannot correctly solve tasks that fundamentally require memory or prediction.

Deliberative architectures implement the classical Sense–Plan–Act loop. Here the robot builds an explicit representation of the task and reasons over future trajectories before committing to actions.

4.1 World model and estimation

Let \( m_t \) be a world model (map, discrete task graph, symbolic state, etc.). A general update law is

\[ m_{t+1} = F(m_t, y_t, u_t). \]

This encompasses state estimation in control (e.g., observers or filters), map updates, or logical inferences.

4.2 Planning and cost function

At a slower rate than the control loop, a planner selects a sequence of controls over a finite horizon \( H \):

\[ P_t = (u_t^0, u_t^1, \dots, u_t^{H-1}) \in U^H. \]

Given an estimated state \( \hat x_t \) and model \( m_t \), the planner may attempt to minimize a cost functional

\[ J(P;\hat x_t,m_t) = \sum_{k=0}^{H-1} \ell(x_{t+k}, u_t^k) + \Phi(x_{t+H}), \]

subject to the dynamics and task constraints, where \( \ell \) is a stage cost and \( \Phi \) a terminal cost. The details of how \( P_t^* \) is computed (graph search, optimization, etc.) are deferred to later courses; here we only need the structural property that planning reasons ahead in time.

4.3 Execution and latency

A deliberative controller typically executes only the first element of the current plan:

\[ u_t = u_t^0 \quad \text{from the current plan } P_t. \]

Planning can be computationally expensive. Let \( T_{\mathrm{plan}} \) be the computation time for one planning cycle and \( T_s \) the control sampling period. A purely deliberative architecture is only feasible if

\[ T_{\mathrm{plan}} \leq T_s, \]

or if planning is done asynchronously while a previously computed plan is still valid. In fast dynamic tasks (e.g., quadrotor flight), this inequality often fails, motivating hybrid architectures.

Hybrid architectures combine a fast reactive layer (for servoing and safety) with one or more slower deliberative layers (for task-level reasoning). A widely used abstract structure is the three-layer architecture:

• Reactive layer: executes feedback laws at high rate.
• Executive layer: sequences and monitors tasks.
• Deliberative layer: performs global planning and reasoning.

flowchart TD
  S["Sensors y_t"] --> R["Reactive control (high rate)"]
  R --> A["Actuators"]

  S --> E["Executive (task monitor)"]
  E --> R
  E --> D["Deliberative planner (low rate)"]
  D --> E

  classDef fast fill:#ffffff,stroke:#000000,stroke-width:1px;
  classDef slow fill:#ffffff,stroke:#000000,stroke-dasharray: 3 3;
  class R fast;
  class D slow;
        

5.1 Time-scale separation

Let \( T_s \) be the sampling period of the reactive layer, and let \( T_{\mathrm{exec}} \) and \( T_{\mathrm{plan}} \) be the effective update periods of the executive and deliberative layers. A typical design enforces

\[ T_s \ll T_{\mathrm{exec}} \ll T_{\mathrm{plan}}, \]

so that low-level control appears effectively continuous from the point of view of higher layers. For instance, one might have \( T_s = 5 \,\mathrm{ms} \), \( T_{\mathrm{exec}} = 100 \,\mathrm{ms} \), \( T_{\mathrm{plan}} = 2 \,\mathrm{s} \).

5.2 Closed-loop behavior

In a hybrid architecture, the overall policy can be written as

\[ u_t = \pi_R\big(y_t, g_t\big), \qquad g_t = G\big(m_t, q_t\big), \]

where \( g_t \) is a goal or reference, and \( q_t \) is the discrete task state managed by the executive (e.g., which subtask is currently active). Stability and performance analyses then reduce to properties of the reactive controller tracking slowly varying references \( g_t \), together with correctness of the higher-level task logic.

Regardless of architecture, safety requirements impose bounds on the maximum allowable reaction time. Suppose a mobile robot with speed \( v_{\max} \) must stop before hitting an obstacle detected at distance \( d_{\mathrm{safe}} \). If the robot requires a physical braking distance \( d_{\mathrm{brake}} \), then the maximum allowable delay between observation and actuation is

\[ T_{\max} = \frac{d_{\mathrm{safe}} - d_{\mathrm{brake}}}{v_{\max}}. \]

For a reactive architecture with control period \( T_s \) and computation time \( T_{\mathrm{comp}}^{\mathrm{R}} \) per cycle, a simple bound on worst-case reaction time is

\[ T_{\mathrm{react}}^{\mathrm{R}} \approx T_s + T_{\mathrm{comp}}^{\mathrm{R}}. \]

We require \( T_{\mathrm{react}}^{\mathrm{R}} \leq T_{\max} \).

For a hybrid architecture, low-level safety behaviors (e.g., obstacle avoidance) should depend only on the reactive loop, so that

\[ T_{\mathrm{react}}^{\mathrm{hybrid}} \approx T_{\mathrm{react}}^{\mathrm{R}} \]

is independent of planning time. The deliberative layer may be slow, but must not be in the critical safety path. This separation is one of the key motivations for hybrid architectures.

In this introductory course we focus on structural patterns rather than full robot stacks. The following sketches show how one might express reactive and hybrid logic in different languages. Sensor values are abstracted as high-level quantities (e.g., front distance).

7.1 Python: simple reactive wall avoidance

class ReactiveWallAvoider:
    """
    Simple reactive controller:
      - If front distance d_front < d_safe: turn in place.
      - Else: drive forward.
    """
    def __init__(self, d_safe=0.7):
        self.d_safe = d_safe

    def compute_action(self, obs):
        """
        obs: dictionary with key "d_front" (meters).
        Returns a dict with linear and angular velocity.
        """
        d_front = obs["d_front"]
        if d_front < self.d_safe:
            # obstacle too close: stop and turn left
            return {"v": 0.0, "w": 1.0}
        else:
            # path clear: move forward
            return {"v": 0.4, "w": 0.0}


# Example usage
controller = ReactiveWallAvoider(d_safe=0.5)
obs = {"d_front": 0.3}
u = controller.compute_action(obs)
print("Command:", u)  # Command: {'v': 0.0, 'w': 1.0}
      

This code expresses a purely reactive policy \( u_t = \pi_R(y_t) \). In ROS Python nodes, similar logic is embedded inside a callback that receives sensor messages and publishes velocity commands.

7.2 C++: reactive behavior with parameterized gains

#include <iostream>

struct Observation {
    double d_front;  // meters
};

struct VelocityCommand {
    double v;  // linear velocity (m/s)
    double w;  // angular velocity (rad/s)
};

class ReactiveWallAvoider {
public:
    explicit ReactiveWallAvoider(double d_safe)
        : d_safe_(d_safe) {}

    VelocityCommand computeAction(const Observation& obs) const {
        VelocityCommand cmd;
        if (obs.d_front < d_safe_) {
            cmd.v = 0.0;
            cmd.w = 1.0;
        } else {
            cmd.v = 0.4;
            cmd.w = 0.0;
        }
        return cmd;
    }

private:
    double d_safe_;
};

int main() {
    ReactiveWallAvoider controller(0.6);
    Observation obs{0.4};
    VelocityCommand cmd = controller.computeAction(obs);
    std::cout << "v = " << cmd.v
              << ", w = " << cmd.w << std::endl;
    return 0;
}
      

In ROS2 C++, such a controller would typically be wrapped in a node that subscribes to range sensor topics and publishes geometry_msgs/Twist commands.

7.3 Java: interface-based architecture separation

public interface Controller {
    VelocityCommand computeAction(Observation obs);
}

public class Observation {
    public final double dFront;
    public Observation(double dFront) {
        this.dFront = dFront;
    }
}

public class VelocityCommand {
    public final double v;
    public final double w;
    public VelocityCommand(double v, double w) {
        this.v = v;
        this.w = w;
    }
}

public class ReactiveWallAvoider implements Controller {
    private final double dSafe;

    public ReactiveWallAvoider(double dSafe) {
        this.dSafe = dSafe;
    }

    @Override
    public VelocityCommand computeAction(Observation obs) {
        if (obs.dFront < dSafe) {
            return new VelocityCommand(0.0, 1.0);
        } else {
            return new VelocityCommand(0.4, 0.0);
        }
    }
}
      

The Controller interface could also be implemented by a deliberative or hybrid controller class, enabling runtime switching between architectures for comparison.

7.4 MATLAB/Simulink: reactive block and hybrid structure

In MATLAB, a reactive controller can be implemented as a function:

function [v, w] = reactiveWallAvoider(d_front, d_safe)
% Simple reactive controller:
%   if d_front < d_safe: stop and turn
%   else: drive forward
if d_front < d_safe
    v = 0.0;
    w = 1.0;
else
    v = 0.4;
    w = 0.0;
end
end
      

In Simulink, this logic can be placed inside a MATLAB Function block. A hybrid architecture can be modeled by:

• Using a Stateflow chart for the executive layer (discrete task states and transitions).
• Using continuous-time blocks (transfer functions, state-space) or discrete controllers for the reactive layer that tracks references (e.g., target velocities) produced by the executive.

The key idea is that the reactive part runs at the base sample time of the model, while the Stateflow chart for deliberation can have a slower sample time, reflecting the time-scale separation in Section 5.

We modeled robot autonomy architectures as policies mapping sensor histories to actions and classified them into reactive, deliberative, and hybrid forms. Reactive architectures rely on direct sensor–actuator couplings and are fast and robust but limited in partially observable tasks. Deliberative architectures maintain internal models and plan over future trajectories but face computational and latency constraints. Hybrid architectures reconcile these perspectives by separating time scales, placing safety and servoing in fast reactive layers and task-level reasoning in slower layers. These concepts will underpin later discussions of planning, learning, and advanced autonomy algorithms.