A robot is distinguished from pure automation by the presence of a closed operational loop that adapts actions to sensed conditions. The canonical abstraction is:

flowchart TD
  W["World + Task"] --> S["Sense (Sensors)"]
  S --> P["Perceive / Preprocess"]
  P --> D["Think (Decision / Control)"]
  D --> A["Act (Actuators)"]
  A --> W
  D -->|goals/constraints| D
  S -->|new measurements| P
        

The loop runs repeatedly at a sampling period \( T_s \). Sensing converts physical processes into digital signals, thinking converts signals into decisions, and acting converts decisions into physical forces/motions.

We will make this abstract loop mathematically precise by representing: (i) the plant (robot + environment), (ii) sensors, (iii) the decision algorithm, and (iv) actuators, all in discrete time.

Let \( t_k = kT_s \) denote discrete sampling instants, where \( k \in \mathbb{Z}_{\ge 0} \). We define:

• \( \mathbf{x}_k \in \mathbb{R}^n \): plant (robot+world) state at time \( t_k \)
• \( \mathbf{u}_k \in \mathbb{R}^m \): actuator command applied over \( [t_k, t_{k+1}) \)
• \( \mathbf{y}_k \in \mathbb{R}^p \): sensor measurement at time \( t_k \)

A general discrete-time plant model is

\[ \mathbf{x}_{k+1} = f(\mathbf{x}_k, \mathbf{u}_k, \mathbf{w}_k), \qquad \mathbf{y}_k = h(\mathbf{x}_k, \mathbf{v}_k), \]

where \( \mathbf{w}_k \) represents process disturbances (unmodeled effects) and \( \mathbf{v}_k \) measurement noise. Even without advanced robotics tools, these two equations already encode the sense–think–act loop:

• Sense: produce \( \mathbf{y}_k \) from the physical state \( \mathbf{x}_k \).
• Think: compute \( \mathbf{u}_k \) from current or past measurements.
• Act: apply \( \mathbf{u}_k \) to change \( \mathbf{x}_{k+1} \).

In many introductory analyses we linearize sensing as:

\[ \mathbf{y}_k = \mathbf{C}\mathbf{x}_k + \mathbf{v}_k, \qquad E[\mathbf{v}_k]=\mathbf{0},\; E[\mathbf{v}_k\mathbf{v}_k^\top]=\mathbf{R}. \]

The matrix \( \mathbf{C} \) selects what the robot can observe. Measurement noise is unavoidable due to physical limits and electronics.

If the underlying physical signal is continuous-time \( \mathbf{y}(t) \), sampling gives \( \mathbf{y}_k = \mathbf{y}(t_k) \). According to Nyquist intuition from control/signal theory, to avoid aliasing for a bandlimited signal with max frequency \( \omega_{\max} \), one requires:

\[ T_s < \frac{\pi}{\omega_{\max}}. \]

In robotics, \( \omega_{\max} \) is set by the fastest dynamics you must react to (e.g., motor vibrations, contact impacts).

Signal-to-noise ratio (SNR). For scalar measurement \( y_k = c^\top x_k + v_k \) with noise variance \( \sigma_v^2 \) and signal variance \( \sigma_s^2 \),

\[ \mathrm{SNR} = \frac{\sigma_s^2}{\sigma_v^2}. \]

Larger SNR yields more reliable “sense” stages, reducing burden on the thinking stage.

The thinking stage maps information to action. At this point in the course, we avoid advanced robotics planning and keep a general feedback-policy view:

\[ \mathbf{u}_k = \pi(\mathbf{y}_{0:k}, \mathbf{r}_{0:k}), \]

where \( \pi \) is a policy (algorithm) and \( \mathbf{r}_k \) is a reference or goal signal. A common special case from Linear Control is memoryless feedback:

\[ \mathbf{u}_k = g(\mathbf{y}_k, \mathbf{r}_k). \]

For a linear plant approximation \( \mathbf{x}_{k+1}=\mathbf{A}\mathbf{x}_k+\mathbf{B}\mathbf{u}_k \) and measurement \( \mathbf{y}_k=\mathbf{C}\mathbf{x}_k \), a linear state feedback thinking law is:

\[ \mathbf{u}_k = -\mathbf{K}\mathbf{x}_k + \mathbf{K}_r \mathbf{r}_k. \]

Closed-loop dynamics. Substitution yields:

\[ \mathbf{x}_{k+1} = (\mathbf{A}-\mathbf{B}\mathbf{K})\mathbf{x}_k + \mathbf{B}\mathbf{K}_r\mathbf{r}_k. \]

The sense–think–act loop is thus a feedback interconnection that shapes the matrix \( \mathbf{A}-\mathbf{B}\mathbf{K} \).

Stability criterion (discrete-time). The origin of \( \mathbf{x}_{k+1}=(\mathbf{A}-\mathbf{B}\mathbf{K})\mathbf{x}_k \) is asymptotically stable if and only if all eigenvalues \( \lambda_i \) of \( \mathbf{A}-\mathbf{B}\mathbf{K} \) satisfy \( |\lambda_i| < 1 \).

Proof (standard):

Let \( \mathbf{A}_c = \mathbf{A}-\mathbf{B}\mathbf{K} \). By Jordan decomposition, \( \mathbf{A}_c = \mathbf{V}\mathbf{J}\mathbf{V}^{-1} \). Then \( \mathbf{x}_k = \mathbf{A}_c^k \mathbf{x}_0 = \mathbf{V}\mathbf{J}^k\mathbf{V}^{-1}\mathbf{x}_0 \). If all eigenvalues satisfy \( |\lambda_i| < 1 \), then every Jordan block power decays to zero, implying \( \mathbf{J}^k \to \mathbf{0} \) and thus \( \mathbf{x}_k \to \mathbf{0} \). Conversely, if any \( |\lambda_i| \ge 1 \), the corresponding block does not decay, so some initial conditions yield non-vanishing or diverging trajectories. ∎

This proof shows why the thinking stage is critical: it is mathematically responsible for stability and performance.

Acting transforms digital commands into forces/torques/velocities. We keep an abstract linear actuator interface:

\[ \mathbf{u}_k^{\text{phys}} = \mathbf{G}\mathbf{u}_k + \boldsymbol{\eta}_k, \]

where \( \mathbf{G} \) models actuator gains and \( \boldsymbol{\eta}_k \) actuator noise or saturation effects.

In practice, actuators are often bandwidth-limited. A simple discrete first-order actuator model is:

\[ \mathbf{u}_{k+1}^{\text{phys}} = \alpha\,\mathbf{u}_k^{\text{phys}} + (1-\alpha)\mathbf{u}_k, \qquad 0 < \alpha < 1. \]

This shows that even if thinking computes “perfect” commands, the act stage can introduce lag that affects closed-loop stability.

Collecting the pieces yields a structural view of the full loop:

flowchart TD
  X["State x_k"] --> H["Sensor: y_k = C x_k + v_k"]
  H --> PI["Policy pi(.)"]
  PI --> U["Command u_k"]
  U --> G["Actuator dynamics"]
  G --> F["Plant: x_{k+1} = f(x_k, u_k, w_k)"]
  F --> X
        

If we use linear approximations for plant and actuator, the overall closed-loop can still be written in augmented state form. For example, with actuator state \( \mathbf{z}_k=\mathbf{u}_k^{\text{phys}} \),

\[ \begin{bmatrix} \mathbf{x}_{k+1}\\ \mathbf{z}_{k+1} \end{bmatrix} = \begin{bmatrix} \mathbf{A} & \mathbf{B}\\ \mathbf{0} & \alpha \mathbf{I} \end{bmatrix} \begin{bmatrix} \mathbf{x}_k\\ \mathbf{z}_k \end{bmatrix} + \begin{bmatrix} \mathbf{0}\\ (1-\alpha)\mathbf{I} \end{bmatrix}\mathbf{u}_k. \]

The sense–think–act loop is thus a discrete-time dynamical system with feedback. Future chapters will fill in robotics-specific details of \( f \), \( h \), and \( \pi \).

We now implement a toy sense–think–act loop for a scalar discrete plant:

\[ x_{k+1} = a x_k + b u_k + w_k,\qquad y_k = x_k + v_k, \]

with feedback controller \( u_k = -K y_k + r_k \). This is not a robotics-specific model; it is the generic loop skeleton.

7.1 Python (NumPy + control-style simulation)

import numpy as np

# Plant parameters
a, b = 1.02, 0.05
K = 1.5
Ts = 0.01
N = 2000

# Noise std devs
sigma_w = 0.02
sigma_v = 0.05

x = 0.0
xs, ys, us = [], [], []

for k in range(N):
    r = 1.0  # step reference
    v = np.random.randn() * sigma_v
    y = x + v                       # Sense
    u = -K * y + r                  # Think
    w = np.random.randn() * sigma_w
    x = a * x + b * u + w           # Act/Plant
    xs.append(x); ys.append(y); us.append(u)

print("Final state:", xs[-1])
      

Useful robotics-adjacent Python libraries for later: numpy, scipy, control (feedback analysis), and robotics toolboxes (introduced in later chapters).

7.2 C++ (Eigen for linear algebra)

#include <iostream>
#include <random>

int main() {
    double a = 1.02, b = 0.05, K = 1.5;
    int N = 2000;

    std::default_random_engine gen(0);
    std::normal_distribution<double> wdist(0.0, 0.02);
    std::normal_distribution<double> vdist(0.0, 0.05);

    double x = 0.0;
    for(int k=0; k<N; ++k){
        double r = 1.0;
        double v = vdist(gen);
        double y = x + v;        // Sense
        double u = -K*y + r;     // Think
        double w = wdist(gen);
        x = a*x + b*u + w;       // Act/Plant
    }
    std::cout << "Final state: " << x << std::endl;
    return 0;
}
      

Typical C++ robotics stacks later add: Eigen (linear algebra), real-time middleware, and ROS2 (introduced in Chapter 12).

7.3 Java (basic loop + Apache Commons Math style)

import java.util.Random;

public class SenseThinkActDemo {
    public static void main(String[] args) {
        double a = 1.02, b = 0.05, K = 1.5;
        int N = 2000;
        Random rng = new Random(0);

        double sigmaW = 0.02, sigmaV = 0.05;
        double x = 0.0;

        for (int k = 0; k < N; k++) {
            double r = 1.0;
            double v = rng.nextGaussian() * sigmaV;
            double y = x + v;           // Sense
            double u = -K * y + r;      // Think
            double w = rng.nextGaussian() * sigmaW;
            x = a * x + b * u + w;      // Act/Plant
        }
        System.out.println("Final state: " + x);
    }
}
      

Java robotics work later often uses: EJML / Apache Commons Math for matrices, and higher-level frameworks.

7.4 MATLAB / Simulink (script + block mapping)

% Scalar sense-think-act loop in MATLAB
a = 1.02; b = 0.05; K = 1.5;
N = 2000;
sigmaW = 0.02; sigmaV = 0.05;

x = 0;
xs = zeros(N,1); ys = zeros(N,1); us = zeros(N,1);

for k = 1:N
    r = 1.0;
    v = sigmaV * randn;
    y = x + v;          % Sense
    u = -K*y + r;       % Think
    w = sigmaW * randn;
    x = a*x + b*u + w;  % Act/Plant
    xs(k)=x; ys(k)=y; us(k)=u;
end

disp(xs(end))
      

Simulink mapping: you can implement the same loop with blocks: “Unit Delay” (state), “Gain” (a, b, K), “Sum” (feedback), and “Random Source” (w, v). The discrete-time structure mirrors the formal equations in Sections 2–6.

Problem 1 (Closed-loop stability): Consider the scalar discrete-time plant \( x_{k+1} = a x_k + b u_k \) with measurement \( y_k = x_k \). The controller is \( u_k = -K y_k \). Derive the closed-loop update and find the range of \( K \) that makes the origin asymptotically stable.

Solution:

Substituting \( u_k=-Kx_k \) gives \( x_{k+1} = (a-bK)x_k \). Stability requires \( |a-bK| < 1 \). Therefore,

\[ -1 < a-bK < 1 \;\;\Longrightarrow\;\; \frac{a-1}{b} < K < \frac{a+1}{b} \quad (b > 0). \]

Problem 2 (Effect of sampling time): Suppose a continuous-time signal \( y(t)=\sin(\omega t) \) is sampled with period \( T_s \). Show that if \( T_s > \pi/\omega \), then the sampled sequence is indistinguishable from a lower-frequency sinusoid.

Solution:

Sampling gives \( y_k = \sin(\omega kT_s) \). For any integer \( q \),

\[ \sin(\omega kT_s) = \sin\!\big((\omega - 2\pi q/T_s)kT_s\big). \]

If \( T_s > \pi/\omega \), then \( \omega > \pi/T_s \) so we can choose \( q=1 \) to obtain an aliased frequency \( \omega_a = \omega - 2\pi/T_s \) inside \( (0,\pi/T_s) \). Hence the discrete samples cannot distinguish \( \omega \) from \( \omega_a \). ∎

Problem 3 (Noise propagation): Let the scalar closed-loop system be \( x_{k+1} = (a-bK)x_k - bK v_k + w_k \), where \( w_k \sim \mathcal{N}(0,\sigma_w^2) \), \( v_k \sim \mathcal{N}(0,\sigma_v^2) \) are independent. Assuming stability, compute the steady-state variance \( \sigma_x^2 = \lim_{k\to\infty} E[x_k^2] \).

Solution:

Let \( \phi = a-bK \). Then \( x_{k+1} = \phi x_k + \epsilon_k \) where \( \epsilon_k = -bK v_k + w_k \). Since \( v_k \) and \( w_k \) are independent,

\[ \sigma_\epsilon^2 = E[\epsilon_k^2] = b^2K^2\sigma_v^2 + \sigma_w^2. \]

For stable AR(1), the steady variance satisfies \( \sigma_x^2 = \phi^2 \sigma_x^2 + \sigma_\epsilon^2 \), so

\[ \sigma_x^2 = \frac{\sigma_\epsilon^2}{1-\phi^2} = \frac{b^2K^2\sigma_v^2 + \sigma_w^2}{1-(a-bK)^2}. \]

Problem 4 (Policy memory): Give a concrete example of a policy \( \pi(\mathbf{y}_{0:k}) \) that uses memory (depends on past measurements), and explain why memory can improve performance even without changing sensors or actuators.

Solution:

A simple example is a moving-average decision: \( u_k = -K \bar{y}_k \) where

\[ \bar{y}_k = \frac{1}{M}\sum_{i=0}^{M-1} y_{k-i}. \]

This reduces measurement noise variance by a factor \( 1/M \) (for i.i.d. noise), yielding smoother commands and less actuator chattering. The improvement is achieved in the “think” stage alone. ∎

We defined the robot sense–think–act cycle as a discrete-time feedback loop, modeled sensing, decision policies, and actuator dynamics, and proved the standard closed-loop stability criterion. The core idea is that robotics begins with this adaptive loop; later chapters will specify how real robots implement each block with particular sensors, mechanisms, and software stacks.