Chapter 14: Human–Robot Interaction (HRI) Basics

Lesson 2: Teleoperation vs. Shared Autonomy

This lesson compares teleoperation and shared autonomy from a control-theoretic perspective. We build linear models for human–robot closed loops, introduce authority blending between human and autonomous controllers, and analyze stability and performance in a simple 1-DOF example. We conclude with small simulation snippets (Python, C++, Java, MATLAB) that implement a basic shared-autonomy law.

1. Conceptual Overview

In teleoperation, a human directly commands the robot in real time, typically over some communication channel and through an interface device (joystick, haptic device, keyboard, etc.). In shared autonomy, an autonomous controller and a human operator simultaneously influence the robot. The amount of human vs. autonomous authority is modulated by a blending or arbitration policy.

Let the robot be modeled as a linear time-invariant (LTI) system with state \( \mathbf{x}_r(t) \in \mathbb{R}^n \), control input \( \mathbf{u}(t) \in \mathbb{R}^m \), and output \( \mathbf{y}(t) \in \mathbb{R}^p \):

\[ \dot{\mathbf{x}}_r(t) = \mathbf{A}_r \mathbf{x}_r(t) + \mathbf{B}_r \mathbf{u}(t), \quad \mathbf{y}(t) = \mathbf{C}_r \mathbf{x}_r(t). \]

A human operator is modeled as a causal mapping \( \mathcal{H}: (\mathbf{r}, \mathbf{y}) \mapsto \mathbf{u}_h \) that transforms a task-level reference \( \mathbf{r}(t) \) and observed feedback \( \mathbf{y}(t) \) into a command signal \( \mathbf{u}_h(t) \). An autonomous controller is similarly a mapping \( \mathcal{A}: (\mathbf{r}, \mathbf{y}) \mapsto \mathbf{u}_a \).

In this abstraction:

  • Teleoperation: the robot executes \( \mathbf{u}(t) = \mathbf{u}_h(t) \).
  • Full autonomy: the robot executes \( \mathbf{u}(t) = \mathbf{u}_a(t) \).
  • Shared autonomy: the robot executes a combination of \( \mathbf{u}_h(t) \) and \( \mathbf{u}_a(t) \).
flowchart TD
  H["Human operator (reference r, perception)"]
  UI["Interface / input device"]
  NET["Comm / middleware"]
  COMB["Command combiner"]
  AUTO["Autonomous controller A(r, y)"]
  CTRL["Low-level robot controller"]
  ROB["Robot + environment"]
  SENS["Sensors / perception"]
  
  H --> UI --> NET --> COMB
  AUTO --> COMB
  COMB --> CTRL --> ROB --> SENS --> H
        

Teleoperation corresponds to the case where the command combiner forwards only the human command. Shared autonomy corresponds to a combiner that merges human and autonomous inputs in a state- and context-dependent manner.

2. Teleoperation as a Feedback Control System

A simple yet useful model for a human operator in teleoperation is a linear dynamic controller acting on tracking error. Let the human maintain a reference \( \mathbf{r}(t) \) (e.g., desired end-effector pose) and observe \( \mathbf{y}(t) \); define the tracking error \( \mathbf{e}(t) = \mathbf{r}(t) - \mathbf{y}(t) \). A linear human model is

\[ \dot{\mathbf{x}}_h(t) = \mathbf{A}_h \mathbf{x}_h(t) + \mathbf{B}_h \mathbf{e}(t), \quad \mathbf{u}_h(t) = \mathbf{C}_h \mathbf{x}_h(t), \]

where \( \mathbf{x}_h(t) \in \mathbb{R}^{n_h} \) is an internal state of the operator (e.g., adaptation, prediction, neuromuscular dynamics). Pure teleoperation applies \( \mathbf{u}(t) = \mathbf{u}_h(t) \) to the robot.

By stacking robot and human states, \( \mathbf{x}(t) = \begin{bmatrix} \mathbf{x}_r(t) \\ \mathbf{x}_h(t) \end{bmatrix} \), the closed-loop dynamics become

\[ \dot{\mathbf{x}}(t) = \underbrace{ \begin{bmatrix} \mathbf{A}_r & \mathbf{B}_r \mathbf{C}_h \\ -\mathbf{B}_h \mathbf{C}_r & \mathbf{A}_h \end{bmatrix}}_{\mathbf{A}_{\text{cl}}} \mathbf{x}(t) + \underbrace{ \begin{bmatrix} \mathbf{0} \\ \mathbf{B}_h \end{bmatrix}}_{\mathbf{B}_{\text{cl}}} \mathbf{r}(t). \]

In this linear setting, classical notions from control engineering apply:

  • Internal stability: all eigenvalues of \( \mathbf{A}_{\text{cl}} \) must lie in the open left half-plane.
  • Tracking performance: the mapping \( \mathbf{r} \mapsto \mathbf{y} \) should have adequate bandwidth and gain properties.
  • Robustness to delay: communication delays can be included by augmenting the model; stability margins typically shrink as delay grows.

In practice, the human dynamics are highly nonlinear and time-varying, but even simple linear models are valuable for comparing pure teleoperation to shared-autonomy alternatives.

3. Shared Autonomy as Authority Blending

In shared autonomy, both the human controller and an autonomous controller produce candidate commands. The robot executes a blended input:

\[ \mathbf{u}(t) = \alpha(t) \,\mathbf{u}_h(t) + \bigl(1 - \alpha(t)\bigr) \,\mathbf{u}_a(t), \]

where the authority coefficient \( \alpha(t) \in [0,1] \) determines how much control authority is assigned to the human vs. autonomy at time \( t \). Typical cases:

  • \( \alpha(t) = 1 \): pure teleoperation.
  • \( \alpha(t) = 0 \): full autonomy.
  • \( 0 < \alpha(t) < 1 \): shared autonomy, continuous blending of commands.

A simple state-dependent policy is

\[ \alpha(t) = \alpha\bigl(\mathbf{x}_r(t), \mathbf{r}(t)\bigr), \quad 0 \leq \alpha(\cdot,\cdot) \leq 1, \]

where \( \alpha \) decreases (more autonomy) when the robot state approaches unsafe configurations or when human commands conflict with task constraints.

We can embed the autonomous controller as another linear system, for example

\[ \dot{\mathbf{x}}_a(t) = \mathbf{A}_a \mathbf{x}_a(t) + \mathbf{B}_a \begin{bmatrix} \mathbf{r}(t) \\ \mathbf{y}(t) \end{bmatrix}, \quad \mathbf{u}_a(t) = \mathbf{C}_a \mathbf{x}_a(t), \]

and stack the full state \( \mathbf{x}(t) = [\mathbf{x}_r^\top(t)\; \mathbf{x}_h^\top(t)\; \mathbf{x}_a^\top(t)]^\top \). For a fixed \( \alpha \), the overall closed-loop remains LTI and can be analyzed using standard eigenvalue and transfer-function tools. When \( \alpha(t) \) switches or varies with state, the system becomes time-varying or hybrid, and tools like Lyapunov functions for switched systems are used.

flowchart TD
  ST["State x_r, human cmd u_h, auto cmd u_a"] --> ALPHA["Compute authority alpha in [0,1]"]
  ALPHA --> BLEND["u_blend = alpha*u_h + (1-alpha)*u_a"]
  BLEND --> SAFE["Check safety / limits"]
  SAFE -->|safe| SEND["Execute u_blend on robot"]
  SAFE -->|"violates constraints"| PROJ["Modify cmd (e.g., saturation, projection)"] --> SEND
        

The safety check often enforces actuator limits or keeps the system inside a prescribed safe set; mathematically, this can be seen as a projection of the blended command onto a constraint set of admissible inputs.

4. 1-DOF Example: Teleoperation vs. Shared Autonomy

To obtain explicit formulas, consider a one-dimensional robot with position \( x(t) \in \mathbb{R} \) and control input \( u(t) \in \mathbb{R} \) modeled as a single integrator:

\[ \dot{x}(t) = u(t). \]

4.1 Pure Teleoperation

In pure teleoperation, the human directly commands \( u(t) = u_h(t) \). Assume the human command is uniformly bounded:

\[ |u_h(t)| \leq U_{\max}, \quad \forall t \geq 0. \]

The solution of \( \dot{x}(t) = u_h(t) \) is

\[ x(t) = x(0) + \int_{0}^{t} u_h(\sigma)\,\mathrm{d}\sigma. \]

Using the bound on \( u_h \),

\[ |x(t)| \leq |x(0)| + \int_{0}^{t} |u_h(\sigma)|\,\mathrm{d}\sigma \leq |x(0)| + U_{\max}\, t. \]

Thus, even for bounded human commands, the position can drift without bound as \( t \to \infty \). This illustrates why pure teleoperation can be problematic for safety or workspace constraints.

4.2 Shared Autonomy with Linear Blending

Now introduce an autonomous stabilizing controller \( u_a(t) = -k\, x(t) \) with gain \( k > 0 \), and a constant blending factor \( \alpha \in [0,1) \). The applied command is

\[ u(t) = \alpha\, u_h(t) + (1 - \alpha)\, u_a(t) = \alpha\, u_h(t) - (1 - \alpha) k\, x(t), \]

so the closed-loop dynamics become

\[ \dot{x}(t) = -(1 - \alpha) k\, x(t) + \alpha\, u_h(t). \]

This is a stable first-order system driven by the human command. The homogeneous solution is

\[ x_{\text{hom}}(t) = \mathrm{e}^{-(1-\alpha)k t} x(0), \]

and by variation of constants, the full solution is

\[ x(t) = \mathrm{e}^{-(1-\alpha)k t} x(0) + \int_{0}^{t} \mathrm{e}^{-(1-\alpha)k (t - \sigma)} \alpha\, u_h(\sigma)\,\mathrm{d}\sigma. \]

Assuming again \( |u_h(t)| \leq U_{\max} \) for all \( t \geq 0 \), we obtain the bound

\[ \begin{aligned} |x(t)| &\leq \mathrm{e}^{-(1-\alpha)k t} |x(0)| + \alpha U_{\max} \int_{0}^{t} \mathrm{e}^{-(1-\alpha)k (t - \sigma)} \,\mathrm{d}\sigma \\ &= \mathrm{e}^{-(1-\alpha)k t} |x(0)| + \alpha U_{\max} \frac{1 - \mathrm{e}^{-(1-\alpha)k t}}{(1-\alpha)k}. \end{aligned} \]

Taking \( t \to \infty \), we get

\[ \limsup_{t \to \infty} |x(t)| \leq \frac{\alpha}{(1-\alpha)k} U_{\max}. \]

Hence, in this simplified model:

  • The state \( x(t) \) remains bounded for all bounded human commands.
  • The asymptotic bound shrinks as \( k \) increases or \( \alpha \) decreases.
  • As \( \alpha \to 1^{-} \) (pure teleoperation), the bound can become arbitrarily large.

For a constant human command \( u_h(t) \equiv \bar{u}_h \), the equilibrium \( x^* \) satisfies

\[ 0 = -(1-\alpha)k\, x^* + \alpha\, \bar{u}_h \quad \Rightarrow \quad x^* = \frac{\alpha}{(1-\alpha)k}\,\bar{u}_h. \]

Thus, the autonomous controller pulls the system toward \( x = 0 \), while the human command shifts the equilibrium in a controlled way that remains bounded for \( \alpha < 1 \).

5. Discrete-Time Shared Autonomy Law

On a digital robot controller, we often discretize the dynamics using a sampling period \( T > 0 \). With Euler forward discretization of the integrator \( \dot{x}(t) = u(t) \), we obtain

\[ x_{k+1} = x_k + T\, u_k, \]

where \( x_k \approx x(kT) \) and \( u_k \approx u(kT) \). For shared autonomy with linear blending,

\[ u_k = \alpha\, u_{h,k} + (1-\alpha)\, u_{a,k}, \quad u_{a,k} = -k\, x_k, \]

and a simple saturation to enforce actuator limits \( |u_k| \leq U_{\max} \) is

\[ u_k^{\text{sat}} = \max\bigl(-U_{\max}, \min(U_{\max}, u_k)\bigr). \]

The update becomes \( x_{k+1} = x_k + T\, u_k^{\text{sat}} \). Below we implement this discrete-time law in different programming languages for a simulated 1-DOF robot.

6. Simulation Snippets for Shared Autonomy

We now give minimal simulation code for the discrete-time shared-autonomy example. In a real robot, the same logic would be integrated with a robotics middleware (e.g., ROS/ROS2) that handles topics such as /joy (human input) and /cmd_vel (robot velocity command).

6.1 Python Example


import numpy as np

T = 0.01      # sampling period [s]
T_end = 10.0  # simulation horizon
k_gain = 1.0  # autonomous feedback gain
alpha = 0.4   # human authority in [0,1)
U_max = 2.0   # actuator saturation

N = int(T_end / T)
t = np.linspace(0.0, T_end, N + 1)
x = np.zeros(N + 1)

def human_command(time):
    # Simple piecewise-constant input: push right for 0-5s, left for 5-10s
    if time < 5.0:
        return 1.0
    else:
        return -1.0

for k in range(N):
    u_h = human_command(t[k])
    u_a = -k_gain * x[k]
    u = alpha * u_h + (1.0 - alpha) * u_a
    # Saturation
    u_sat = max(-U_max, min(U_max, u))
    # Euler forward integration of x_dot = u
    x[k + 1] = x[k] + T * u_sat

# x and t now contain the simulated trajectory under shared autonomy
print("Final position:", x[-1])
      

6.2 C++ Example


#include <iostream>
#include <cmath>

double human_command(double time) {
    if (time < 5.0) {
        return 1.0;
    } else {
        return -1.0;
    }
}

int main() {
    const double T = 0.01;
    const double T_end = 10.0;
    const double k_gain = 1.0;
    const double alpha = 0.4;
    const double U_max = 2.0;

    const int N = static_cast<int>(T_end / T);
    double x = 0.0;
    double t = 0.0;

    for (int k = 0; k < N; ++k) {
        double u_h = human_command(t);
        double u_a = -k_gain * x;
        double u = alpha * u_h + (1.0 - alpha) * u_a;

        // Saturation
        if (u > U_max) u = U_max;
        if (u < -U_max) u = -U_max;

        x = x + T * u;
        t = t + T;
    }

    std::cout << "Final position: " << x << std::endl;
    return 0;
}
      

6.3 Java Example


public class SharedAutonomySim {

    static double humanCommand(double time) {
        if (time < 5.0) {
            return 1.0;
        } else {
            return -1.0;
        }
    }

    public static void main(String[] args) {
        double T = 0.01;
        double T_end = 10.0;
        double k_gain = 1.0;
        double alpha = 0.4;
        double U_max = 2.0;

        int N = (int) (T_end / T);
        double x = 0.0;
        double t = 0.0;

        for (int k = 0; k < N; k++) {
            double u_h = humanCommand(t);
            double u_a = -k_gain * x;
            double u = alpha * u_h + (1.0 - alpha) * u_a;

            // Saturation
            if (u > U_max) u = U_max;
            if (u < -U_max) u = -U_max;

            x = x + T * u;
            t = t + T;
        }

        System.out.println("Final position: " + x);
    }
}
      

6.4 MATLAB / Simulink Example


T = 0.01;
T_end = 10.0;
k_gain = 1.0;
alpha = 0.4;
U_max = 2.0;

N = floor(T_end / T);
t = linspace(0, T_end, N + 1);
x = zeros(1, N + 1);

human_command = @(time) (time < 5.0) * 1.0 + (time >= 5.0) * (-1.0);

for k = 1:N
    u_h = human_command(t(k));
    u_a = -k_gain * x(k);
    u = alpha * u_h + (1 - alpha) * u_a;
    % Saturation
    u = max(-U_max, min(U_max, u));
    x(k + 1) = x(k) + T * u;
end

plot(t, x);
xlabel('Time [s]');
ylabel('Position x(t)');
title('1-DOF Shared Autonomy Simulation');
grid on;
      

A corresponding Simulink model can be built using:

  • Blocks implementing the human command (e.g., Step or Signal Builder),
  • A Gain block for the autonomous law u_a = -k_gain * x,
  • A Sum block implementing u = alpha * u_h + (1 - alpha) * u_a,
  • A Saturation block for actuator limits,
  • An Integrator block for dx/dt = u.

This small example illustrates how shared autonomy can be implemented with only linear control primitives and simple nonlinearities (saturation).

7. Problems and Solutions

Problem 1 (Teleoperation Drift): Consider the pure teleoperation system \( \dot{x}(t) = u_h(t) \) with \( |u_h(t)| \leq U_{\max} \) for all \( t \geq 0 \). Show that, for any finite time \( t \), \( |x(t)| \leq |x(0)| + U_{\max} t \), and explain why this implies possible unbounded drift as \( t \to \infty \).

Solution: The solution is \( x(t) = x(0) + \int_{0}^{t} u_h(\sigma)\,\mathrm{d}\sigma \). Hence,

\[ |x(t)| \leq |x(0)| + \int_{0}^{t} |u_h(\sigma)|\,\mathrm{d}\sigma \leq |x(0)| + \int_{0}^{t} U_{\max}\,\mathrm{d}\sigma = |x(0)| + U_{\max} t. \]

The bound grows linearly with time, so there is no uniform bound on \( |x(t)| \) as \( t \to \infty \). In other words, even if the human command is bounded, the position can wander arbitrarily far over long time scales.

Problem 2 (Boundedness Under Shared Autonomy): For the shared-autonomy system \( \dot{x}(t) = -(1-\alpha)k x(t) + \alpha u_h(t) \) with \( \alpha \in [0,1) \), \( k > 0 \), and \( |u_h(t)| \leq U_{\max} \), derive an explicit bound on \( |x(t)| \) and show that \( x(t) \) remains bounded for all \( t \geq 0 \).

Solution: As derived in Section 4, the solution is

\[ x(t) = \mathrm{e}^{-(1-\alpha)k t} x(0) + \int_{0}^{t} \mathrm{e}^{-(1-\alpha)k (t - \sigma)} \alpha\, u_h(\sigma)\,\mathrm{d}\sigma. \]

Using \( |u_h(\sigma)| \leq U_{\max} \), we obtain

\[ |x(t)| \leq \mathrm{e}^{-(1-\alpha)k t} |x(0)| + \alpha U_{\max} \frac{1 - \mathrm{e}^{-(1-\alpha)k t}}{(1-\alpha)k}. \]

The right-hand side is uniformly bounded for all \( t \geq 0 \), and the exponential terms vanish as \( t \to \infty \), so \( \limsup_{t \to \infty} |x(t)| \leq \frac{\alpha}{(1-\alpha)k} U_{\max} \). Therefore, the state is bounded for all time, unlike in pure teleoperation.

Problem 3 (Equilibrium Under Constant Human Command): Consider the same shared-autonomy system with a constant human command \( u_h(t) \equiv \bar{u}_h \). Find the equilibrium \( x^* \) and study the dependence on \( \alpha \) and \( k \).

Solution: At equilibrium, \( \dot{x}(t) = 0 \), so

\[ 0 = -(1-\alpha)k\, x^* + \alpha\, \bar{u}_h \quad \Rightarrow \quad x^* = \frac{\alpha}{(1-\alpha)k} \,\bar{u}_h. \]

For fixed \( \bar{u}_h \) and \( k \), this equilibrium:

  • Approaches \( 0 \) as \( \alpha \to 0 \) (full autonomy suppresses the human command).
  • Grows without bound as \( \alpha \to 1^{-} \), recovering the unbounded drift of teleoperation.
  • Shrinks as \( k \) increases, corresponding to a stronger stabilizing autonomous feedback.

Problem 4 (Discrete-Time Stability Condition): For the discrete-time shared-autonomy update \( x_{k+1} = x_k + T\, u_k \), \( u_k = \alpha u_{h,k} + (1-\alpha)u_{a,k} \), \( u_{a,k} = -k x_k \), and \( \alpha \in [0,1) \), derive the homogeneous update (with \( u_{h,k} \equiv 0 \)) and find a condition on \( T \), \( \alpha \), and \( k \) for asymptotic stability.

Solution: With \( u_{h,k} \equiv 0 \), we have \( u_k = (1-\alpha)(-k x_k) = -(1-\alpha)k x_k \), so

\[ x_{k+1} = x_k - T(1-\alpha)k\, x_k = \bigl(1 - T(1-\alpha)k\bigr) x_k. \]

This is a scalar linear system \( x_{k+1} = a x_k \) with \( a = 1 - T(1-\alpha)k \). Asymptotic stability requires \( |a| < 1 \), i.e.

\[ |1 - T(1-\alpha)k| < 1 \quad \Leftrightarrow \quad -1 < 1 - T(1-\alpha)k < 1. \]

The right inequality \( 1 - T(1-\alpha)k < 1 \) is always satisfied if \( T(1-\alpha)k > 0 \). The left inequality \( -1 < 1 - T(1-\alpha)k \) yields \( T(1-\alpha)k < 2 \). Therefore, a sufficient condition is

\[ 0 < T(1-\alpha)k < 2. \]

For fixed \( k \) and \( \alpha \), this gives an upper bound on the sampling period \( T \) that guarantees stability of the autonomous part of the discrete-time closed loop.

8. Summary

In this lesson we contrasted teleoperation with shared autonomy using simple linear models. Teleoperation can be viewed as a human-in-the-loop controller directly closing the feedback around a robot plant. Shared autonomy introduces an additional autonomous controller and an authority blending mechanism that combines human and autonomous commands. Even in a 1-DOF integrator example, we see that teleoperation may exhibit unbounded drift under bounded inputs, whereas an appropriately designed shared-autonomy scheme yields bounded trajectories and tunable equilibria. Discrete-time realizations show how sampling and saturation constraints enter, and basic stability conditions can be derived using familiar linear control tools.

Subsequent HRI lessons will build on this foundation to incorporate human factors, usability, and trust, which are crucial when applying these control-theoretic concepts to real human users.

9. References

  1. Sheridan, T.B. (1995). Teleoperation, telerobotics and telepresence: A progress report. Control Engineering Practice, 3(2), 205–214.
  2. Anderson, R.J., & Spong, M.W. (1989). Bilateral control of teleoperators with time delay. IEEE Transactions on Automatic Control, 34(5), 494–501.
  3. Lawrence, D.A. (1993). Stability and transparency in bilateral teleoperation. IEEE Transactions on Robotics and Automation, 9(5), 624–637.
  4. Niemeyer, G., & Slotine, J.-J.E. (2004). Telemanipulation with time delays. International Journal of Robotics Research, 23(9), 873–890.
  5. Spong, M.W., & Vidyasagar, M. (1989). Robust linear compensator design for nonlinear robot motion control. IEEE Journal of Robotics and Automation, 5(4), 484–490.
  6. Javdani, S., Srinivasa, S.S., & Bagnell, J.A. (2015). Shared autonomy via hindsight optimization. Robotics: Science and Systems (RSS), 1–9.
  7. Dragan, A.D., & Srinivasa, S.S. (2013). A policy-blending formalism for shared control. International Journal of Robotics Research, 32(7), 790–805.
  8. Fisac, J.F., Bajcsy, A., Herbert, S., Wang, C., Tomlin, C.J., & Dragan, A.D. (2019). Probabilistically safe robot planning with confidence-based human predictions. Robotics: Science and Systems (RSS), 1–10.
  9. Löfberg, J. (2003). Minimax approaches to robust model predictive control. Linköping Studies in Science and Technology, Thesis (for robust blending and constraint handling ideas).
  10. Hsu, P., & Tomizuka, M. (1992). Robust tracking in robot manipulators under joint and base disturbances. Journal of Dynamic Systems, Measurement, and Control, 114(1), 28–36.