Variational analysis of sensory feedback mechanisms in powerstroke–recovery systems

Abstract

Although the raison d’etre of the brain is the survival of the body, there are relatively few theoretical studies of closed-loop rhythmic motor control systems. In this paper we provide a unified framework, based on variational analysis, for investigating the dual goals of performance and robustness in powerstroke–recovery systems. To demonstrate our variational method, we augment two previously published closed-loop motor control models by equipping each model with a performance measure based on the rate of progress of the system relative to a spatially extended external substrate—such as a long strip of seaweed for a feeding task, or progress relative to the ground for a locomotor task. The sensitivity measure quantifies the ability of the system to maintain performance in response to external perturbations, such as an applied load. Motivated by a search for optimal design principles for feedback control achieving the complementary requirements of efficiency and robustness, we discuss the performance–sensitivity patterns of the systems featuring different sensory feedback architectures. In a paradigmatic half-center oscillator-motor system, we observe that the excitation–inhibition property of feedback mechanisms determines the sensitivity pattern while the activation–inactivation property determines the performance pattern. Moreover, we show that the nonlinearity of the sigmoid activation of feedback signals allows the existence of optimal combinations of performance and sensitivity. In a detailed hindlimb locomotor system, we find that a force-dependent feedback can simultaneously optimize both performance and robustness, while length-dependent feedback variations result in significant performance-versus-sensitivity tradeoffs. Thus, this work provides an analytical framework for studying feedback control of oscillations in nonlinear dynamical systems, leading to several insights that have the potential to inform the design of control or rehabilitation systems.

1 Introduction

Physiological systems underlying vital behaviors such as breathing, walking, crawling, and feeding, must generate motor rhythms that are not only efficient, but also robust against changes in operating conditions. Although central neural circuits have been shown to be capable of producing rhythmic motor outputs in isolation from the periphery (Brown 1911, 1914; Harris-Warrick and Cohen 1985; Pearson 1985; Smith et al. 1991), the role of sensory feedback should not be underestimated. Sensory feedback can play a crucial role in stabilizing motor activity in response to unexpected conditions. For example, modeling work suggests that walking movements can be stably restored after spinal cord injury by enhancing the strengths of the afferent feedback pathways to the spinal central pattern generator (CPG) (Markin et al. 2010; Spardy et al. 2011). Feedback control can also improve the performance and efficiency of movements. For instance, in a model of feeding motor patterns in the marine mollusk Aplysia californica, seaweed intake can be increased by strengthening the gain of sensory feedback to a specific motor neural pool (Wang et al. 2022).

We are interested in understanding how sensory feedback contributes to control and stabilization within a specific class of rhythmic motor behaviors, namely, behaviors in which an animal (or robot) repeatedly engages and disengages with the outside world (see Fig. 1, top). We refer to the phase of the motion during which the animal is in contact with an external substrate as the power stroke, and the component during which the animal is disengaged as the recovery phase. The decomposition of a repetitive movement into powerstroke and recovery applies naturally to many motor control systems, including locomotion (Jahn and Votta 1972) and swallowing (Shaw et al. 2015); a similar dynamical structure also appears in mechanical stick–slip systems (Galvanetto and Bishop 1999) as well as abstract two-stroke relaxation oscillators (Jelbart and Wechselberger 2020). In the motor control context, when the animal is in contact with an external substrate or load opposing the motion, we say the animal makes “progress" (food is consumed, distance is traveled, oxygen is absorbed) relative to the outside world. During the recovery phase, the animal disconnects from the external component, and repositions relative to the substrate in order to prepare for the next power stroke. Consider, for example, the ingestive behavior of Aplysia (Shaw et al. 2015; Lyttle et al. 2017; Wang et al. 2022). When the animal’s grasper is closed on a stipe of seaweed, it drags the food into the buccal cavity; meanwhile, the food applies a mechanical load on the grasper. Then the grasper opens, releasing its grip on the food. The grasper moves in the absence of the force exerted by the seaweed and returns to the original position to begin the next swallowing cycle.

Fig. 1

Schematic of dynamics and analysis of a powerstroke–recovery system. Top left: The system trajectory sketched in a 3D space, for illustration. The vertical axis represents a body variable (e.g., foot) and the xy-plane represents brain variables. A specific model system can have multiple brain and body variables. The system trajectory makes an entry to a constraint surface (blue shaded parallelogram) at the landing point A, followed by motion confined to move along the surface (red wave) and then a liftoff at point B back into the unconstrained space. The motion of the system is thereby partitioned into a powerstroke phase and a recovery phase. Top right: A system of differential equations governs the dynamics of the powerstroke–recovery system. The brain and body variables are denoted by $\textbf{a}$ and $\textbf{x}$, respectively. During recovery the body variable is unconstrained, while it is confined to a lower-dimensional manifold during powerstroke. An external load, represented by $\kappa$, is imposed on the system when the body is in contact with the hard surface. Bottom: Two objectives in the control of the system’s response to a sustained external perturbation away from a default value $\kappa _0$. Performance, $Q_0$, measures the average rate of task progress of the system (e.g., food consumed, distance traveled), defined to be the total progress $y_0$ over a cycle divided by the cycle period $T_0$, when $\kappa =\kappa _0$ (see Eq. (2)). Sensitivity, $S_0$, measures the (infinitesimal) robustness of the system performance against the perturbation, defined to be the derivative of the performance with respect to the perturbation parameter evaluated at $\kappa _0$. It accounts for the interaction of both the shape and timing effects of the perturbation on the trajectory: $y_1$ is the linearized shift in the shape of the trajectory while $T_1$ is the linearized shift in the trajectory timing (see Eq. (12)). The two quantities can be captured by recently developed variational analysis tools—the infinitesimal shape response curve (iSRC) and the local timing response curve (lTRC), respectively (Wang et al. 2021)

In this paper, we present a novel analysis of feedback control for powerstroke–recovery systems. To quantitatively evaluate the behavior of a system controlled by different feedback mechanisms, we measure the sensitivity (or robustness) and performance (or efficiency) (see Fig. 1, bottom). The complementary objectives of sensitivity and performance have been studied in a variety of motor control systems, from both empirical and theoretical perspectives (Lee and Tomizuka 1996; Yao et al. 1997; Ronsse et al. 2008; Hutter et al. 2014; Lyttle et al. 2017; Sharbafi et al. 2020; Mo et al. 2023). There are a myriad of ways to interpret performance and robustness used by engineers, biologists, neuroscientists, and applied mathematicians. Here we define the performance of a powerstroke–recovery system to be the total progress divided by the period of the rhythm (i.e., the average rate of progress), and the sensitivity to be the ability of the system to maintain performance in response to some specific external perturbation, such as an increased mechanical resistance while pulling on a load, or increased slope while walking. That is, we take the sensitivity to be the derivative of the performance with respect to the external perturbation parameter. As a step towards first-principles—based design of sensory feedback mechanisms, we aim to understand what aspects of sensory feedback contribute to the coexistence of high performance and low sensitivity.

The ubiquity of powerstroke–recovery systems, and the importance of the dual goals of robustness and efficiency, motivate us to develop analytical tools for systematically studying both quantities simultaneously. In this work we apply mathematical tools based on variational analysis to evaluate the two objectives applicable for any powerstroke–recovery system. The key quantities in our analysis are the infinitesimal shape response curve (iSRC) and local timing response curve (lTRC) recently established and validated by Wang et al. (2021, 2022) and generalized in Yu and Thomas (2022); Yu et al. (2023). The iSRC describes, to first order, the distortion of an oscillator trajectory—the shape response—under a sustained perturbation. In contrast, the lTRC captures the effect of the perturbation on the timing of the oscillator trajectory within any defined segments of the trajectory (such as the powerstroke and recovery phases). Both the iSRC and the lTRC complement the more widely known infinitesimal phase response curve (iPRC), which quantifies the effect of a transient perturbation on the global limit cycle timing (Brown et al. 2004; Izhikevich and Ermentrout 2008; Ermentrout and Terman 2010; Schultheiss et al. 2012; Zhang and Lewis 2013).

Based on the iSRC and lTRC approach, we propose a general framework to investigate robust and efficient control through diverse feedback architectures. We apply our method to two neuro-mechanical models, each possessing a natural powerstroke–recovery structure but different in their levels of details and perturbations. The first model is based on an abstract CPG-feedback-biomechanics system introduced in Yu and Thomas (2021) which studied the relative contributions of feedforward and feedback control (an idea going back to Kuo 2002). We extend this model to incorporate an externally applied load, enabling us to define quantitative measures of both performance and robustness, as the system alternately grasps and releases the external substrate. The second model, due to Markin et al. (Markin et al. 2010; Spardy et al. 2011), represents a locomotor system with a single-joint limb, and features more detailed CPG circuitry as well as more realistic afferent feedback pathways. We modify the Markin model so that the limb “walks” up an “incline”; modifying the angle of the incline introduces a parametric perturbation that allows us to define performance and robustness.

The activity of sensory feedback pathways is difficult to measure in many physiological systems; for this reason sensory feedback is the “missing link" for understanding the design and function of many biological motor systems. Specifically, in many experimental biological systems, the dynamics of the isolated CPG and the form of the muscle activation in response to descending motor signals is well characterized, while the precise form of the sensory feedback remains unkown. From a practical perspective, descending motor signals are generally carried by large-diameter axons from which it is easier to record high-quality (high signal-to-noise) traces, relative to ascending sensory signals, which are generally carried by much smaller diameter axons with poor signal-to-noise properties.

Given a particular specification of central neural circuit, descending output, and biomechanical response elements, but with the precise form of sensory feedback unknown, we can think of the pursuit of performance and/or robustness as an optimization (or dual optimization) problem on the space of sensory feedback functions. However, this is an infinite dimensional space of inputs to a highly nonlinear system. That is, the mapping from sensory feedback function to system trajectories to system performance and/or robustness is highly nonlinear and possibly non-convex. It may have multiple nonequivalent optima. Therefore, to restrict the problem to a manageable scope, in the specific cases studied here we restrict attention to sensory feedback functions with a prescribed, but plausible form, such as a sigmoid specified by a threshold and slope parameters, or multiple channels with different relative gain parameters, and study the restricted optimization problem there. Moreover, for simplicity, in the models we consider here, we restrict attention to sensory feedback that either monotonically increases or monotonically decreases with variables such as muscle length, tension, and/or velocity.

Our analysis of these two systems—one more abstract and the other more realistic—illustrates a technical framework for studying performance and sensitivity of powerstroke–recovery motor systems, leading to several insights that have the potential to inform the design of control or rehabilitation systems. For example, (i) the excitation–inhibition property of feedback signals determines the sensitivity pattern while the activation–inactivation property determines the performance pattern; (ii) the strong nonlinearity of feedback activation with respect to biomechanical variables may contribute to achievable performance–sensitivity optima; (iii) force-dependent feedback can prevent the performance/robustness tradeoffs commonly occuring with the length-dependent feedback. These findings may yield important information for future work modeling biphasic rhythm generation, in that they provide insights that could guide the design of feedback systems to accomplish well-balanced efficient and robust powerstroke–recovery activities in biological and robotic experiments.

The broad aim of this paper is to establish variational methods for multi-objective optimization of powerstroke–recovery systems. To this end, we organize the paper as follows.

• We introduce and give a general definition of a “power- stroke–recovery” motor control system at the beginning of Sect. 2. Although such systems are ubiquitous in the physiology of motor control, to the best of our knowledge they have previously not been systematically defined nor studied.

• We illustrate the particular features of recently established variational tools (such as the local timing response curve) as applied to powerstroke–recovery systems, in the rest of Sect. 2. For these systems, the quantities of interest (performance and sensitivity) take on specific forms that we describe in detail.

• To show the breadth of applicability of the general analysis, we apply it to example model systems embodying a variety of feedback mechanisms, namely

  • Several versions of a nominal half-center oscillator model, each with a different feedback architecture (Sect. 3)

  • A detailed, physiologically-grounded hindlimb model incorporating multiple types of feedback operating in parallel within one model (Sect. 4).

• In Sect. 5 we summarize the framework, as well as the main observations and insights that we obtain from the models; we also discuss limitations, connections to previous literature, and possible implications of our results for biology and engineering as well as future directions.

2 Mathematical formulation

The motor systems we consider integrate central neural circuitry, biomechanics, and sensory inputs from the periphery to form a closed-loop control system. The model systems we study fall within the following general framework (Shaw et al. 2015; Lyttle et al. 2017; Yu and Thomas 2021):

$$\begin{aligned} \frac{d\textbf{a}}{dt}=\textbf{f}(\textbf{a})+\textbf{g}(\textbf{a},\textbf{x}),\quad \frac{d\textbf{x}}{dt}=\textbf{h}(\textbf{a},\textbf{x})+ \textbf{j}(\textbf{x},\kappa ). \end{aligned}$$

(1)

Here, $\textbf{a}$ and $\textbf{x}$ are vectors representing the neural activity variables and mechanical state variables, respectively; the vector field $\textbf{f}(\textbf{a})$ represents the intrinsic neural dynamics of the central pattern generator when isolated from the rest of the body; $\textbf{h}(\textbf{a},\textbf{x})$ captures the biomechanical dynamics driven by the central inputs; $\textbf{g}(\textbf{a},\textbf{x})$ carries the sensory feedback from the periphery, which modulates the neural dynamics; $\textbf{j}(\textbf{x},\kappa )$ is an externally applied load to the mechanical variables controlled by a load parameter $\kappa$.¹

In the powerstroke–recovery systems, we assume the load interacts with the mechanics only during the powerstroke phase. The portion of the trajectory comprising the powerstroke phase is specified separately for each model. For the purposes of this paper we assume the vector fields $\textbf{f},\textbf{g},\textbf{h},\textbf{j}$ are sufficiently smooth (e.g., twice differentiable). Specifically, we assume that any nonsmoothness in the vector fields is limited to a finite number of transition surfaces—for example at points marking the powerstroke–recovery transitions—and that the limit cycle trajectories are themselves piecewise differentiable, with at most a finite number of points of nondifferentiability.

For many naturally occurring control systems, the mechanical variables $\textbf{x}$ may include both the position and velocity of different body components, as well as muscle activation variables. The sensory feedback function $\textbf{g}$ may be difficult to ascertain experimentally. For example, the feedback could have an excitatory effect on the neural dynamics, or an inhibitory effect, or a mixture at different points within a single movement; it may depend not only on neural outputs but also the length, velocity, or tension of the mechanical components; it may arise from multiple channels each with different gain. Given the broad varieties of the possible feedback functions, we restrict the scope of our investigation to some biologically plausible forms for the specific models we consider in Sects. 3 and 4.

2.1 Performance and sensitivity

Suppose for $\kappa \in \mathcal {I}\subset \mathbb {R}$, system (1) has an asymptotically stable limit cycle solution $\gamma _\kappa (t)$ with period $T_\kappa$. Let $q_\kappa$ represent the rate at which the system advances relative to the outside world, and let $y_\kappa$ represent the total progress achieved over one limit cycle, i.e.,

$$\begin{aligned} y_\kappa =\int _0^{T_\kappa } q_\kappa (\gamma _\kappa (t))\,dt. \end{aligned}$$

We note that in general, the instantaneous performance measure q may depend both on the system variables $\textbf{x}$ and on the control parameter $\kappa$.

We consider the task performance of the system, denoted by Q, to be the progress divided by the limit-cycle period, or equivalently, the mean value of the rate of progress averaged around the limit cycle, defined as follows

$$\begin{aligned} Q(\kappa )=\frac{1}{T_\kappa }\int _0^{T_\kappa } q_\kappa (\gamma _\kappa (t))\,dt=\frac{y_\kappa }{T_\kappa }. \end{aligned}$$

(2)

For a powerstroke–recovery system, we choose the time coordinate so that $t=0$ coincides with the beginning of the powerstroke phase, and write $T_\kappa ^\text {ps}$ for the duration of the power stroke. We adopt the convention that during the recovery phase, the position with respect to the outside world is held fixed ($q_\kappa \equiv 0$),² and denote $T_\kappa ^\text {re}$ as the recovery phase duration ($T_\kappa ^\text {ps}+T_\kappa ^\text {re}=T_\kappa$). In such a system, we write the performance (2) as

$$\begin{aligned} Q(\kappa )=\frac{1}{T_\kappa }\int _0^{T_\kappa ^\text {ps}} q_\kappa (\gamma _\kappa (t))\,dt. \end{aligned}$$

(3)

Assume that $\kappa _0\in \mathcal {I}$, which represents the unperturbed load. When the system is subjected to a small static perturbation on the load, $\kappa _0\rightarrow \kappa _\epsilon =\kappa _0+\epsilon \in \mathcal {I}$, the original limit cycle trajectory $\gamma _{\kappa _0}$ is shifted to a new trajectory $\gamma _{\kappa _\epsilon }$, and its ability to resist the external change to maintain the performance is considered as a measure of robustness for the system. Since the sensory feedback pathways regulate the system dynamics, it would be desirable to obtain a feedback function $\textbf{g}$ so that the system is most robust against the load change, i.e.,

$$\begin{aligned} \textbf{g}^*(\textbf{a},\textbf{x})=\underset{\textbf{g}(\textbf{a},\textbf{x})}{{\text {argmin}}}\,|Q(\kappa _\epsilon )-Q(\kappa _0)|. \end{aligned}$$

(4)

Suppose we can expand $Q(\kappa _\epsilon )$ around $\kappa _0$:

$$\begin{aligned} Q(\kappa _\epsilon )=Q(\kappa _0)+\epsilon \frac{\partial Q}{\partial \kappa }(\kappa _0)+O(\epsilon ^2). \end{aligned}$$

Then the minimization problem (4) to the first order reduces to

$$\begin{aligned} \textbf{g}^*(\textbf{a},\textbf{x})=\underset{\textbf{g}(\textbf{a},\textbf{x})}{{\text {argmin}}}\,\left| \epsilon \frac{\partial Q}{\partial \kappa }(\kappa _0)\right| =\underset{\textbf{g}(\textbf{a},\textbf{x})}{{\text {argmin}}}\,\left| \frac{\partial Q}{\partial \kappa }(\kappa _0)\right| . \end{aligned}$$

We quantify the sensitivity of the original system to be $S=\left| \frac{\partial Q}{\partial \kappa }(\kappa _0)\right|$, which describes the (infinitesimal) response of the task performance to the external perturbation on the load. When $S=0$ with a certain feedback function, $Q(\kappa _\epsilon )\approx Q(\kappa _0)$, which implies a strong ability of the system to maintain performance homeostasis.³ Define a (linear) functional $J: q\rightarrow S$, and the problem falls into a functional minimization problem, $\min _\textbf{g}J[\textbf{g}]$, which attains the minimum with some feedback function $\textbf{g}^*$ when the functional derivative $\partial J/\partial \textbf{g}^*=0$. Finding $\textbf{g}^*$ when the underlying system has a limit cycle (as opposed to the more often studied case of fixed-point homeostasis) is an open problem that we do not attempt to solve here. Instead, we focus on a limited range of $\textbf{g}$ functions with practical significance and investigate the constrained optimization problem for the optimal feedback structure within that range.

2.2 Variational analysis

Variational analysis using the infinitesimal shape response curve (iSRC) and the local timing response curve (lTRC) is the key tool in our derivation of the performance sensitivity $S=\left| \frac{\partial Q}{\partial \kappa }(\kappa _0)\right|$. In this section, we present a brief review of the theory and then provide two analytical methods to calculate the sensitivity. More mathematical details are given in Appendix C.

Consider system (1) written in the form

$$\begin{aligned} \frac{d\textbf{z}}{dt}=\textbf{F}_\kappa (\textbf{z}), \end{aligned}$$

(5)

where $\textbf{z}=(\textbf{a},\textbf{x})^\intercal$ and $\textbf{F}_\kappa (\textbf{z})$ is the corresonding vector field parameterized by the load $\kappa \in \mathcal {I}$. For convenience, we write trajectories $\gamma _{\kappa _\epsilon }$ as $\gamma _{\epsilon }$ and $\gamma _{\kappa _0}$ as $\gamma _{0}$; we write other quantities similarly. Expanding the perturbed trajectory yields

$$\begin{aligned} \gamma _\epsilon (\tau _\epsilon (t))=\gamma _0(t)+\epsilon \gamma _1(t)+O(\epsilon ^2). \end{aligned}$$

(6)

The timing function $\tau _\epsilon (t)$ rescales the perturbed trajectory to match the unperturbed trajectory, so that the series (6) is uniform with respect to the time coordinate. As discussed in Wang et al. (2021), $\tau _\epsilon (t)$ can be any smooth, monotonically increasing function mapping the interval $[0,T_0)$ to the interval $[0,T_\epsilon )$. We discuss the choice of $\tau _\epsilon$ further below. The linear shift in the shape of the unperturbed trajectory, $\gamma _1(t)$, is referred to as the iSRC. It satisfies a nonhomogeneous variational equation (Wang et al. 2021; Yu and Thomas 2022)

$$\begin{aligned} \frac{d\gamma _1(t)}{dt}=D\textbf{F}_0(\gamma _0(t))\gamma _1(t)+\nu _1(t)\textbf{F}_0(\gamma _0(t))+\frac{\partial \textbf{F}_\kappa (\gamma _0(t))}{\partial \kappa }\bigg |_{\epsilon =0}, \nonumber \\ \end{aligned}$$

(7)

where $\nu _1(t)=\frac{\partial ^2\tau _\epsilon (t)}{\partial \epsilon \partial t}\big |_{\epsilon =0}$ measures the local timing sensitivity to the perturbation. The initial condition for the iSRC Eq. (7) is

$$\begin{aligned} \gamma _1(0)=\lim _{\epsilon \rightarrow 0}\frac{\textbf{p}_\epsilon -\textbf{p}_0}{\epsilon }, \end{aligned}$$

(8)

where $\textbf{p}_\epsilon$ and $\textbf{p}_0$ represent the intersection points of the trajectories with a smooth Poincaré section transverse to both the perturbed and unperturbed limit cycles, so that $\gamma _1(0)$ indicates the linear displacement of the unperturbed intersection point. For more details about the iSRC, see Appendix C.2, Wang et al. (2021) and Yu and Thomas (2022).

To solve the iSRC Eq. (7), the lTRC is built to yield the timing sensitivity $\nu _1$ local to each phase of the motion. Approximate the perturbed phase durations by

$$\begin{aligned} T_\epsilon ^\text {ps}&=T_0^\text {ps}+\epsilon T_1^\text {ps}+O(\epsilon ^2),\\ T_\epsilon ^\text {re}&=T_0^\text {re}+\epsilon T_1^\text {re}+O(\epsilon ^2). \end{aligned}$$

Wang et al. (2021) and Yu et al. (2023) developed a formula to calculate the first-order approximation for the duration change in phase $i\in \{\text {ps},\text {re}\}$, given by

$$\begin{aligned} T_1^i= & \eta ^i(\textbf{z}_0^\text {in})\cdot \frac{\partial \textbf{z}_\kappa ^\text {in}}{\partial \kappa }\bigg |_{\epsilon =0}-\eta ^i(\textbf{z}_0^\text {out})\cdot \frac{\partial \textbf{z}_\kappa ^\text {out}}{\partial \kappa }\bigg |_{\epsilon =0}\nonumber \\ & +\int _{t^\text {in}}^{t^\text {out}}\eta ^i(\gamma _0(t))\cdot \frac{\partial \textbf{F}_\kappa (\gamma _0(t))}{\partial \kappa }\bigg |_{\epsilon =0}\,dt. \end{aligned}$$

(9)

Here, $\textbf{z}_0^\text {in}$ and $\textbf{z}_0^\text {out}$ denote the unperturbed entry point to, and exit point from, the specific phase, respectively. Vector $\eta ^i$, defined to be the gradient of the remaining time of the trajectory until exiting phase i, is referred to as the lTRC for phase i. It satisfies the adjoint equation

$$\begin{aligned} \frac{d\eta ^i}{dt}=-D\textbf{F}_0(\gamma _0(t))^\intercal \eta ^i, \end{aligned}$$

with a boundary condition

$$\begin{aligned} \eta ^i(\textbf{z}_0^\text {out})=-\frac{n^\text {out}}{(n^\text {out})^\intercal \textbf{F}_0(\textbf{z}_0^\text {out})}, \end{aligned}$$

where $n^\text {out}$ is a normal vector of the exit boundary surface at $\textbf{z}_0^\text {out}$. When the vector field $\textbf{F}$ changes discontinuously across the surface defining the boundary between two regions, the Jacobian $D\textbf{F}$ should be evaluated as a one-sided limit, taken from the interior of the local region. With a linear time scaling for (6) (and setting $t=0$ to be the start of the power stroke), i.e.,

$$\begin{aligned}&\text {for}\quad t\in [0,T_0^\text {ps}),\quad \tau _\epsilon (t)=\tau _\epsilon ^\text {ps}(t):=\frac{T_\epsilon ^\text {ps}}{T_0^\text {ps}}t\\&\text {for}\quad t\in [T_0^\text {ps},T_0),\quad \tau _\epsilon (t)=\tau _\epsilon ^\text {re}(t):=\tau _\epsilon ^\text {ps} (T_0^\text {ps})\\&\quad +\frac{T_\epsilon ^\text {re}}{T_0^\text {re}}(t-T_0^\text {ps}), \end{aligned}$$

the local timing sensitivity function in (7) reduces to

$$\begin{aligned} \nu _1^\text {ps}=\frac{T_1^\text {ps}}{T_0^\text {ps}},\quad \nu _1^\text {re}=\frac{T_1^\text {re}}{T_0^\text {re}}, \end{aligned}$$

which can be obtained by using Eq. (9) for each phase. See Appendix C.1, Wang et al. (2021), and Yu et al. (2023) for more details about the lTRC formulation.

The variational analysis above allows us to analyze the performance sensitivity of powerstroke–recovery systems. In Yu and Thomas (2022) we provided a formula for the sensitivity of any averaged quantity with respect to an arbitrary control parameter, as long as the quantity of interest does not have explicit dependence on the control parameter. We generalize the approach of Yu and Thomas (2022) to allow for dependence of the instantaneous performance $q_\kappa (\textbf{x})$ on both the state $\textbf{x}$ and the parameter $\kappa$, and obtain

$$\begin{aligned} & \frac{\partial Q}{\partial \kappa }(\kappa _0)=\frac{1}{T_0^\text {ps}}\int _0^{T_0^\text {ps}}\Bigg [\beta _0\Bigg (\nabla q_{0}(\gamma _0(t))\cdot \gamma _1(t)\nonumber \\ & \quad +\frac{\partial q_\kappa (\gamma _0(t))}{\partial \kappa }\bigg |_{\epsilon =0}\Bigg )+\beta _1q_0(\gamma _0(t))\Bigg ]\,dt. \end{aligned}$$

(10)

The first term in the integral arises from the impact of the perturbation on the shape of the trajectory ($\gamma _1$) as well as directly on the quantity of interest ($\partial q_\kappa /\partial \kappa )$. Here $\beta _0$ denotes the proportion of the powerstroke duration within the period ($\beta _0=T_0^\text {ps}/T_0$). The second term indicates the impact of the perturbation on the timing of the trajectory, in that $\beta _1$ represents the linear shift in $\beta _0$ in response to the perturbation, which can be analytically evaluated by

$$\begin{aligned} \beta _1=\frac{\partial \beta _\kappa }{\partial \kappa }\bigg |_{\epsilon =0}=\frac{T_1^\text {ps}T_0-T_0^\text {ps}T_1}{T_0^2}. \end{aligned}$$

The derivation of formula (10) is given in Appendix C.3.

Given the special structure of powerstroke–recovery systems, we can derive a more succinct expression for $\partial Q/\partial \kappa$. For any value of $\kappa ,$ the second definition in (2) gives

$$\begin{aligned} \frac{\partial Q}{\partial \kappa }(\kappa )&=\frac{1}{T_\kappa ^2}\left( \frac{\partial y_\kappa }{\partial \kappa }T_\kappa -y_\kappa \frac{\partial T_\kappa }{\partial \kappa }\right) \nonumber \\&=\frac{y_\kappa }{T_\kappa }\left( \frac{1}{y_\kappa }\frac{\partial y_\kappa }{\partial \kappa }-\frac{1}{T_\kappa }\frac{\partial T_\kappa }{\partial \kappa }\right) \nonumber \\&=Q(\kappa )\left( \frac{1}{y_\kappa }\frac{\partial y_\kappa }{\partial \kappa }-\frac{1}{T_\kappa }\frac{\partial T_\kappa }{\partial \kappa }\right) . \end{aligned}$$

(11)

Recall at $\kappa =\kappa _\epsilon$, we write $y_\epsilon$ and $T_\epsilon$ as shorthand for $y_{\kappa _\epsilon }$ and $T_{\kappa _\epsilon }$. We can expand the perturbed progress $y_\epsilon$ and period $T_\epsilon$ around $\epsilon =0$ as

$$\begin{aligned} y_\epsilon&=y_0+\epsilon y_1+O(\epsilon ^2),\\ T_\epsilon&=T_0+\epsilon T_1+O(\epsilon ^2), \end{aligned}$$

where $y_1$ is approximately given by the net change of the mechanical component of the iSRC $\gamma _1$ (cf. Eqs. (7) and (8)) within the powerstroke phase, and $T_1$ is the linear shift in the total period, readily given by $T_1=T_1^\text {ps}+T_1^\text {re}$ (cf. Eq. (9)). Therefore, Eq. (11) at $\kappa =\kappa _0$ becomes

$$\begin{aligned} \frac{\partial Q}{\partial \kappa }(\kappa _0)=Q_0\left( \frac{y_1}{y_0}-\frac{T_1}{T_0}\right) , \end{aligned}$$

(12)

which, like (10), incorporates both the shape and timing effects of the perturbation in two distinct terms. Equation (12) also suggests that the sensitivity can be directly given by the absolute difference between the first-order timing change and shape change induced by the perturbation. When the two effects completely offset each other, the system achieves “perfect" robustness.

The two expressions given by (10) and (12) for calculating the sensitivity of the task performance for powerstroke–recovery systems allow us to compare different sensory feedback mechanisms in pursuit of an efficient and robust motor pattern. In the following sections we develop two illustrative examples: an abstract CPG-motor model introduced in Yu and Thomas (2021), and an unrelated realistic locomotor model studied in Markin et al. (2010) and Spardy et al. (2011). The two examples show a variety of differences in their model construction, but our analytic framework is broad enough to address both and give useful insights. Simulation codes required to produce each figure are available at https://github.com/zhuojunyu-appliedmath/Powerstroke-recovery. Instructions for reproducing each figure in the paper are provided (see the README file at the github site). Figures for this paper were produced using MatLab release R2023b.

3 Application: HCO model with external load

In Yu and Thomas (2021), we studied a simple closed-loop model combining neural dynamics and biomechanics, as sketched in Fig. 2. The CPG sytem comprises a half-center oscillator (HCO) with two conductance-based Morris–Lecar neurons (Morris and Lecar 1981; Skinner et al. 1994). Outputs from the HCO drive a simple biomechanical system, which follows a Hill-type kinetic model based on experimental data from the marine mollusk Aplysia californica (Yu et al. 1999). Sensory feedback from the periphery couples the body and brain dynamics, allowing the system to interact with the changing outside world, and to modulate the central neural activities adaptively. However, the previous study of this model did not explore the performance with respect to a physical task; rather, the CPG followed an autonomous clocklike pattern. To perform a more meaningful analysis for understanding principles of closed-loop motor control, here we augment the model from Yu and Thomas (2021) by incorporating a mechanical load exerted in a specific direction with recurrent engagement and disengagement with the system, which enables us to apply our quantitative measures of progress and sensitivity of the system.

Fig. 2

Schematic of components and basic behavior of the HCO model system, adapted from Yu and Thomas (2021). The CPG circuit of the system comprises a half-center oscillator, represented by mutually inhibitory cell 1 and cell 2. Output from each neuron drives its ipsilateral muscle pulling a limb. The muscle stretch and contraction in turn produce reflex commands to a feedback receptor, which sends an inhibitory signal to its contralateral neuron. The limb interacts with an external substrate, which imposes a mechanical load opposing the limb movement. Inhibitory connections end with a round ball, and excitatory connections end with a triangle. In a single movement cycle, the powerstroke phase (panel A to panel B) occurs when the body (red dashed rectangle) moves forward while the foot is fixed, subjected to the load $F_\ell$ opposite to the movement direction (blue arrow). The recovery phase follows (panel B to panel C), during which the body is fixed and the foot is lifted off the substrate and repositions for the next power stroke (green arrow)

3.1 The equations of the HCO model

The model equations we consider are as follows. For $i,j=1,2$ and $j\ne i$,

$$\begin{aligned} C\frac{dV_i}{dt}= & I_\text {ext}-g_\text {L}(V_i-E_\text {L})-g_\text {Ca}M_\infty (V_i)(V_i-E_\text {Ca})\nonumber \\ & -g_\text {K}N_i(V_i-E_\text {K})\nonumber \\ & -g_\text {syn}^\text {CPG}S_\infty ^\text {CPG}(V_j)\left( V_i-E_\text {syn}^\text {CPG}\right) \nonumber \\ & -g_\text {syn}^\text {FB}S_\infty ^\text {FB}(L_j)(V_i-E_\text {syn}^\text {FB}),\nonumber \\ \frac{dN_i}{dt}= & \lambda _N(V_i)(N_\infty (V_i)-N_i),\nonumber \\ \frac{dA_i}{dt}= & \tau ^{-1}\{U(V_i)-[\beta +(1-\beta )U(V_i)]A_i\},\nonumber \\ \frac{dx}{dt}= & \frac{1}{b}(F_2-F_1+r\kappa F_\ell ). \end{aligned}$$

(13)

Variable $V_i$ denotes the membrane voltage for HCO neuron cell i, and $N_i$ is the gating variable for the potassium current in cell i. The two neuron cells are coupled by fast inhibitory synapses, and the coupling function is given by

$$\begin{aligned} S_\infty ^\text {CPG}(V_j)=\frac{1}{2}\left( 1+\tanh \left( \frac{V_j-E_\text {thresh}}{E_\text {slope}}\right) \right) , \end{aligned}$$

which closely approximates a Heaviside step function with $E_\text {thresh}$ denoting the synaptic threshold.

In the third equation of (13), $A_i\ge 0$ represents the activation of the ith muscle. The neural outputs from the HCO drive the associated muscle, modeled as

$$\begin{aligned} U(V_i)=1.03-4.31\exp {(-0.198(V_i/2))},\quad V_i\ge 16. \end{aligned}$$

The biomechanics is represented by the movement of an object (nominally, a pendulum or limb), with each side connected to one of the two muscles. The object position relative to the center of mass of the organism, denoted as x, is controlled by the muscle forces $F_1$ and $F_2$ acting on it. An external load $F_\ell$ is exerted on the object only during the powerstroke phase, as specified by the indicator variable r defined to be

$$\begin{aligned} r=\left\{ \begin{array}{ll} 1,& \quad \text {power stroke},\\ 0,& \quad \text {recovery}. \end{array}\right. \end{aligned}$$

We assume that the powerstroke phase is at work when $V_1$ is in the active state ($V_1> E_\text {thresh}$), whereas the load is absent from the system when $V_1$ is inhibited ($V_1\le E_\text {thresh}$). Parameter $\kappa$ describes the strength of the load, which is considered as the perturbation parameter for this model.

The system completes an intact closed loop through the sensory feedback induced by the biomechanics on the CPG in the form of feedback currents, e.g.,

$$\begin{aligned} g_\text {syn}^\text {FB}S_\infty ^\text {FB}(L_j)\left( V_i-E_\text {syn}^\text {FB}\right) , \end{aligned}$$

where $L_j$ is the length of muscle j, and the function $S_\infty ^\text {FB}(L_j)$ describes the feedback synaptic activation. We assume that the feedback conductance has fast dynamics, following a sigmoid function. As discussed in Yu and Thomas (2021), the feedback synaptic architecture affords eight variations, depending on whether the feedback is (i) inhibitory or excitatory, (ii) activated by muscle contraction or muscle stretch, and (iii) modulatory on the contralateral or ipsilateral neuron. For example, when the feedback current is inhibitory to its contralateral neuron and activated when the muscle is contracted, then for the $V_i$-equation we set $E_\text {syn}^\text {FB}=-80$ mV and

$$\begin{aligned} S_\infty ^\text {FB}(L_j)=\frac{1}{2}\left( 1-\tanh \left( \frac{L_j-L_0}{L_\text {slope}}\right) \right) . \end{aligned}$$

(14)

Fig. 2 illustrates the system controlled by the inhibitory-contralateral-decreasing feedback mechanism, and Fig. 3 shows a typical solution for the system. In contrast, setting $E_\text {syn}^\text {FB}=80$ mV for the excitatory feedback current, or setting the sigmoid function to be increasing for the muscle-stretch activated case, or changing $L_j$-dependence to $L_i$-dependence for the ipsilateral mechanism, would specify other possible mechanisms. We compare the performance and sensitivity of all different realizations of each of the eight variations of the feedback control scheme below. The force terms $F_{1,2}$, as well as additional details about the functions in system (13), parameter values used for simulations, and simulation codes are given in Appendix A.1.

Fig. 3

A typical solution for the system (13) in the absence of perturbation ($\kappa _0=1$), plotted over two periods with $T_0=3055$ ms. The system is governed by the inhibitory-contralateral feedback mechanism, with a decreasing sigmoid activation function $S_\infty ^\text {FB}$ given by (14), and threshold $L_0=10$ (panel E). Blue trace: cell/muscle 1. Red trace: cell/muscle 2. The gray shaded regions represent the powerstroke phase, defined by the active state of cell 1 ($V_1>E_\text {thresh}$, panel A), with duration $T_0^\text {ps}=1544$ ms; the white regions represent the recovery phase with duration $T_0^\text {re}=1511$ ms. The system makes progress during the powerstroke phase at rate $q=-dx/dt$, while it maintains its position during the recovery phase (compare panels D and F)

3.2 Analysis of the HCO model

In order to establish measures of performance and sensitivity, we assume that the system advances only during the powerstroke phase. That is, the rate at which the system makes progress is given by

$$\begin{aligned} q=-r\frac{dx}{dt}, \end{aligned}$$

as indicated in Fig. 3D, F. Therefore, by (3), the performance (i.e., the average rate of progress) is

$$\begin{aligned} Q(\kappa )&=\frac{1}{T_\kappa }\int _0^{T_\kappa }\left( -r\frac{dx}{dt}\right) \,dt\nonumber \\&=-\frac{1}{T_\kappa b\ell }\int _0^{T_\kappa ^\text {ps}}(F_2-F_1+\kappa F_\ell )\,dt. \end{aligned}$$

(15)

When a sustained small perturbation is applied to the load, $\kappa _0\rightarrow \kappa _0+\epsilon$, the solution trajectory shifts in both its shape and timing, and the performance of the perturbed system is consequently different from the performance of the unperturbed system. As a reference, Fig. 4A, B, E, F compare the trajectories of the unperturbed solution with $\kappa _0=1$ for the system shown in Fig. 3 and the perturbed solution with $\kappa _\epsilon =2$, and Fig. 4C, D, G, H illustrate the iSRC $\gamma _1$ of the unperturbed trajectory, specified by Poincaré section $\{V_1=0,\,dV_1/dt>0\}$. Note that for visual convenience, the large perturbation ($\epsilon =1$) is applied here, but in our actual analysis the perturbation magnitude should be small ($|\epsilon |\ll 1$). Our analysis yields several observations about the role of the inhibition-contralateral-decreasing sensory feedback in regulating the system’s response to the perturbation, as discussed in detail below.

Fig. 4

The iSRC analysis for system (13). A, B, E, F: Time series of the unperturbed trajectory (solid, same as Fig. 3) and perturbed trajectory with load $\kappa _\epsilon =2$ (dashed), both of which are initiated at the start of their respective powerstroke phase. The perturbed trajectory is uniformly time-rescaled by $t_\epsilon =\frac{T_0}{T_\epsilon }t$ to compare with the unperturbed trajectory, where $T_0=3055$ and $T_\epsilon =2831$. The shaded regions represent the powerstroke phase for the unperturbed case, whereas the vertical green dotted lines represent the time at which the phase for the perturbed case switches from power stroke to recovery, which is advanced due to the perturbation. C, D, G, H: The components for the iSRC $\gamma _1(t)$ of the unperturbed trajectory, with the initial condition defined by the Poincaré section $\{V_1=0,\,dV_1/dt>0\}$. The negative responses in the $V_1, N_1, A_1$ directions at the end of power stroke are consistent with the earlier transition to the recovery phase shown in panels A, B, C. The accumulating positive response of x results from the direct effect of perturbation, which shows a stronger resistance to the limb movement

With the perturbation (increased load), the transition from the power stroke to recovery is advanced, i.e., the powerstroke phase is shorter. As indicated in Fig. 4F, the immediate effect of the perturbation is the positive displacement and slower change rate in the x-variable, which occurs because the object is being pulled by the stronger load opposite to its movement direction. Correspondingly, muscle 2, whose length is $L_2=10-x$, is more contracted than it is in the unperturbed case. The sensory feedback current injected to neuron 1, with synaptic activation given by (14), is therefore larger and gives more inhibition to the active neuron 1. As a result, the active $V_1$ crosses the synaptic threshold and terminates the powerstroke phase at an earlier time, as indicated in Fig. 4A. The iSRC in each direction is consistent with the associated trajectory comparison. The significant negative peak in the $V_1$-component at the end of powerstroke phase (panel C) suggests that $V_1$ of the perturbed solution at the rescaled time already decreases to the synaptic threshold and jumps down to the inhibited state, indicating the transition out of the power stroke is advanced. Since $\kappa$ directly impacts dx/dt, we see a different effect on x (panel H) than the other variables.

Both efficiency (high performance) and robustness (low sensitivity) are important features of motor control systems interacting with the outside world. The performance for the perturbed system in Fig. 4 is smaller than the unperturbed system ($Q_\epsilon =0.87\times 10^{-3}$, $Q_0=1.23\times 10^{-3}$), due to the larger magnitude in the progress decrease relative to that in the period. To measure the ability of maintaining the performance, we quantify the sensitivity of the original system in response to an infinitesimal sustained perturbation, following (10), to be

$$\begin{aligned} \frac{\partial Q}{\partial \kappa }(\kappa _0)= & \frac{1}{T_0^\text {ps}}\int _0^{T_0^\text {ps}}\Bigg [-\beta _0\Bigg (\nabla \Bigg (\frac{dx}{dt}\Bigg )\cdot \gamma _1(t)+\frac{F_\ell }{b\ell }\Bigg )\nonumber \\ & -\beta _1\Bigg (\frac{dx}{dt}\Bigg )\Bigg ]\,dt. \end{aligned}$$

(16)

One can also estimate the sensitivity by (12), where $y_1$ corresponds to the x-component of $\gamma _1$. That is,

$$\begin{aligned} \frac{\partial Q}{\partial \kappa }(\kappa _0)=Q_0\left( \frac{\gamma _{1,x}}{\int _0^{T_0^\text {ps}}(dx/dt)\,dt}-\frac{T_1}{T_0}\right) . \end{aligned}$$

To evaluate the joint goals of high performance and low sensitivity, and to investigate how they are affected by sensory feedback, we will simultaneously study the two measures plotted together, while manipulating the shape of the feedback activation function. Specifically, we will vary the steepness parameter $L_\text {slope}$ and position/half-threshold parameter $L_0$ of the sigmoid synaptic feedback activation function $S_\infty ^\text {FB}$. Figure 5 shows the results for all eight sensory feedback mechanisms.

Fig. 5

Performance–sensitivity patterns of all eight sensory feedback architectures in the HCO model. A Contralateral feedback. B Ipsilateral feedback. Each panel includes four subarchitectures: excitatory feedback (upper ensembles) versus inhibitory feedback (lower ensembles), and inactivating feedback (dots) versus activating feedback (+ signs). Specifically, the dot trace indicates decreasing synaptic feedback activation $S_\infty ^\text {FB}$ (muscle-contraction activated feedback), while the plus trace indicates increasing activation $S_\infty ^\text {FB}$ (muscle-stretch activated feedback). The half-threshold position parameter $L_0$ of $S_\infty ^\text {FB}$ is varied over $\{9, 10, 11\}$, colored by blue, black, and red, respectively. A green dot at the center of each ensemble indicates the system with constant feedback ($S_\infty ^\text {FB}\equiv 0.5$). Starting from this constant feedback case, the performance–sensitivity curve of each mechanism becomes distinct as the shape of $S_\infty ^\text {FB}$ becomes steeper (i.e., $L_\text {slope}$ decreases) until it approaches a Heaviside step function ($L_\text {slope}\rightarrow 0$). The contralateral feedback mechanism with increasing (resp. decreasing) sigmoidal activation $S_\infty ^\text {FB}$ with $L_0=10+\theta$ is functionally equivalent to the ipsilateral mechanism with decreasing (resp. increasing) sigmoidal activation $S_\infty ^\text {FB}$ with $L_0=10-\theta$, reducing the eight feedback architectures to four fundamentally different classes. Blue arrow (panel A top) marks the most advantageous configuration among those tested, and black arrow marks the sub-optimal configuration with high performance and low sensitivity

The eight superficially distinct feedback architectures can be reduced to four fundamentally different mechanisms in terms of their performance and sensitivity. The contralateral mechanism of muscle-stretch activated (increasing) current with threshold $L_0=10+\theta$ ($\theta \in \mathbb {R}$), is equivalent to the ipsilateral mechanism of muscle-contraction activated (decreasing) current with threshold $L_0=10-\theta$. Specifically, substituting the contralateral-increasing feedback activation with $L_0=10+\theta$ to the $V_1$-equation of system (13) yields

$$\begin{aligned} \frac{dV_1}{dt}&=\cdots -g_\text {syn}^\text {FB}S_\infty ^\text {FB}(L_2)(V_1-E_\text {syn}^\text {FB})\\&=\cdots -\frac{g_\text {syn}^\text {FB}}{2}\left( 1+\tanh {\left( \frac{10-x-(10+\theta )}{L_\text {slope}}\right) }\right) \\&\quad \left( V_1-E_\text {syn}^\text {FB}\right) \\&=\cdots -\frac{g_\text {syn}^\text {FB}}{2}\left( 1+\tanh {\left( \frac{-x-\theta }{L_\text {slope}}\right) }\right) \left( V_1-E_\text {syn}^\text {FB}\right) \\&=\cdots -\frac{g_\text {syn}^\text {FB}}{2}\left( 1-\tanh {\left( \frac{x+\theta }{L_\text {slope}}\right) }\right) \left( V_1-E_\text {syn}^\text {FB}\right) \\&=\cdots -\frac{g_\text {syn}^\text {FB}}{2}\left( 1-\tanh {\left( \frac{10+x-(10-\theta )}{L_\text {slope}}\right) }\right) \\&\quad \left( V_1-E_\text {syn}^\text {FB}\right) , \end{aligned}$$

where the last equation is exactly the case for the ipsilateral-decreasing feedback with $L_0=10-\theta$. Note that for the unloaded model in Yu and Thomas (2021), only the inhibition–excitation property of the feedback makes a fundamental difference regarding the stability and robustness of the system. However, when we incorporate mechanical interactions with an external substrate, the activating property of the feedback must be taken into account. In the following, we will discuss the performance and sensitivity for the four contralateral feedback mechanisms, which we call inhibition-increasing (II), inhibition-decreasing (ID), excitation-increasing (EI), and excitation-decreasing (ED). The other four ipsilateral feedback mechanisms can be applied accordingly.

3.3 Performance and sensitivity of the HCO model

Our analysis of the HCO model, subject to an applied external load, leads to the following observations from Fig. 5:

• 1. Excitatory feedback is advantageous over inhibitory feedback in terms of performance.

• 2. The qualitative patterns of performance are reversed with respect to the steepness of synaptic activation function in the activating and inactivating feedback mechanisms.

• 3. The qualitative patterns of sensitivity are reversed with respect to the steepness of synaptic activation function in the excitatory and inhibitory feedback mechanisms.

• 4. When the sigmoid activation function $S_\infty ^\text {FB}$ is approximately linear over the working range of the limb, the performance–sensitivity changes approximately linearly with the slope of $S_\infty ^\text {FB}$.

• 5. As the working range of the limb extends beyond the linear regime of $S_\infty ^\text {FB}$, the performance–sensitivity curve can become strongly nonlinear and even non-monotonic, leading to well-defined simultaneous optima in both performance and sensitivity.

We discuss each of these points in turn below.

Excitatory sensory feedback outperforms inhibitory feedback. This conclusion is evident from Fig. 5, in the higher location of the performance–sensitivity pattern for each of the excitation mechanisms relative to that for all of the inhibition mechanisms. To understand the advantage of excitatory over inhibitory feedback in this model system, Fig. 6 compares the trajectories of two systems, one with excitatory constant feedback and the other with inhibitory constant feedback. These systems correspond respectively to the two green dots in Fig. 5A. In the excitatory system, the extra excitation to $V_i$ due to the feedback has two opposing effects. On the one hand, it advances the time at which the active neuron “jumps down” to the inhibited state and thus shortens both the powerstorke phase and the total period T, relative to the system with constant inhibitory feedback. Panel F of Fig. 6 shows the projection of the trajectory on the $(V_1, N_1)$ plane with points plotted at constant time intervals. Note the rapid change in the voltage component relative to the slower gating variable. The significant timing change results from the exponential deceleration of the dynamics of the active neuron when approaching the jump-down point, as shown by the contraction of points before jumping in panel F. Although the $(V_1, N_1)$ projection of the two trajectories during the active state does not differ much spatially, the difference in time needed to cover the small spatial difference is significant. On the other hand, the constant excitatory drive also reduces the net progress y of the system per cycle. In particular, the progress y of the system declines due to the shorter movement time (panel E). The net performance ($Q=y/T$, Eq. (2)) is determined by both the timing T and shape y of the trajectory. For the excitatory system, the resulting performance is in fact larger than for the inhibitory system, because the relative change in period is larger than the relative decrease in progress. Therefore, regardless of the structure of the feedback pathway, any system equipped with the excitatory sensory feedback is always advantageous over the system with the inhibitory sensory feedback in terms of their performance.

Fig. 6

Comparison of trajectories for the constant inhibitory feedback system (solid lines) and constant excitatory feedback system (dashed lines), plotted over one period. The only difference in the setting for the two systems is the synaptic feedback reversal potential, $E_\text {syn}^\text {FB}=+80$ or $-80$ mV, respectively. The gray shaded region represents the powerstroke phase of the inhibitory system; the vertical green dotted line denotes the transition time of the excitatory system out of the power stroke, and the vertical green dashed line denotes the end of its recovery phase. Panel F shows the projection of the trajectories to the $(V_1, N_1)$ phase plane, where the successive dots are equally spaced in time. The inhibitory trajectory is in dark blue (generally passing through more negative voltages) and the excitatory trajectory in light blue (generally higher voltages). The excitatory feedback current raises the voltage of the active neuron and greatly shortens both the powerstroke and recovery phases (panels A, F). The limb makes smaller progress due to the shorter powerstroke duration (panels D, E), but the decrease in magnitude is smaller than the decrease in the period. Hence, the performance of the excitatory-feedback case is larger than the inhibitory-feedback case

The performance patterns in the decreasing and increasing feedback mechanisms are qualitatively reversed with respect to the steepness of $S_\infty ^\text {FB}$. For example, as $S_\infty ^\text {FB}$ changes from constant to approaching a Heaviside step function, the performance of the II mechanism with $L_0=11$ (red dots in Fig. 5A bottom) monotonically increases while the performance of the corresponding ID mechanism (red + in Fig. 5A bottom) decreases monotonically. Figure 7 illustrates two systems controlled by the ID mechanism with $L_0=11$ but different $L_\text {slope}$ values. When $S_\infty ^\text {FB}$ is more shallow (dashed), the inhibitory feedback current to cell 1 is less intense, due to the smaller synaptic activation over the contraction regime of muscle 2 (panel E). This makes neuron 1 (the neuron driving the powerstroke) more active, switching earlier to the inhibited state, and thus giving a shorter duration power stroke (panel A). The progress of the limb over the shorter powerstroke phase is however larger, which is opposite to the excitatory situation in Fig. 6. This outcome occurs because the effect of the duration decrease is not comparable to the effect of the faster velocity around the end of power stroke. The progress velocity over the powerstroke phase is controlled by the force generated by muscle 1, which becomes stronger due to the increased muscle activation $A_1$ (panel C). Therefore, by decreasing the steepness of $S_\infty ^\text {FB}$ for the ID mechanism, the muscles act on the limb in a stronger and faster fashion, leading to the enhanced performance. In contrast, by applying a similar analysis, we observe that the corresponding II mechanism, for which the muscle-stretch activated feedback current gives more inhibition to the active neuron 1, has the opposite effect (not shown). This example illustrates how two mechanisms with the opposite activating property of the sensory feedback can have qualitatively distinct performance changes when the sensitivity of the feedback pathway to the biomechanics is varied.

Fig. 7

Comparison of the dynamics of two systems controlled by the ID mechanism with $L_0=11$ and $L_\text {slope}=0.5$ (solid traces) versus $L_\text {slope}=2$ (dashed traces). The gray shaded region denotes the powerstroke phase of the first (steeper slope) system ($T_0^\text {ps}=1547$); the green vertical dotted line represents the end of the powerstroke phase for the second system ($T_0^\text {ps}=1511$), and the green vertical dashed line represents the end of its full cycle. During the powerstroke phase, the less steep synaptic activation $S^\text {FB}_\infty$ (panel E) reduces the feedback inhibition to $V_1$, leading to greater activation of the neuron driving the powerstroke (“neuron 1”), and earlier transition into the recovery phase (panel A). The activation of muscle 1 becomes stronger due to the larger $V_1$ (panel C), which results in a stronger force being generated by muscle 1. This enhanced force pulls the object forward more quickly, especially around the end of power stroke (panel D). Due to the faster movement, the object makes greater progress in an even shorter time (panel F), thereby outperforming the system with the more steep $S_\infty ^\text {FB}$ curve

The sensitivity profiles of the excitatory and inhibitory systems to an applied load are qualitatively reversed with respect to the steepness of the synaptic activation function. Consider the two contralateral-decreasing mechanisms with $L_0=9$ (see the two blue dot curves in Fig. 5A). As $L_\text {slope}$ becomes smaller, the sensitivity of the ED mechanism first decreases to almost 0 (perfect robustness) and then increases, whereas the sensitivity of the corresponding ID mechanism first increases to the maximum and then decreases. Recall that following Eq. (12), the sensitivity can be written as

$$\begin{aligned} S=\left| \frac{\partial Q}{\partial \kappa }(\kappa _0)\right| =Q_0\left| \frac{y_1}{y_0}-\frac{T_1}{T_0}\right| . \end{aligned}$$

This expression contains two terms—$y_1/y_0$ accounts for the effect of the perturbation on the system progress (shape), while $T_1/T_0$ accounts for the effect on the period (timing). To the extent that the two effects are large and of opposite signs, the system becomes more sensitive; in contrast, the cancellation of the two effects leads to a robust system. Figure 8 shows the two effects in the ED mechanism (red) and ID mechanism (blue) with $L_0=9$ and $L_\text {slope}$ varied over (0.6, 40). We observe that the stronger mechanical load exerts positive effects on both the timing and shape for the system with the excitatory sensory feedback mechanism, by prolonging the limit cycle period and increasing the progress of the limb (positive red curves), but the shape effect is more profound. As $L_\text {slope}$ decreases, the improvement in the progress becomes less significant, while the timing changes more dramatically, which leads to a reduction in the sensitivity. The sensitivity minimum is attained when the two effects completely offset each other at $L_\text {slope}\approx 0.6$, where the system is most robust against the perturbation. In contrast, the perturbation allows the system with the inhibitory feedback to make larger progress in a shorter time (negative blue solid and positive blue dashed); this possibility was mentioned in the discussion of Fig. 7. Although making greater progress in a shorter time in response to perturbation improves performance, it is not beneficial to the robustness of the system in terms of maintaining performance homeostasis. Moreover, as $L_\text {slope}$ decreases, both effects become stronger, inducing the system to be even more sensitive. Apart from this example, the excitation–inhibition property of the sensory feedback in this model always offers qualitatively reversed sensitivity patterns when we consider the steepness variation of the feedback activation curve. The variational analysis serves as a tool to examine the coordinated effects of perturbation on the trajectory geometry and timing, and to identify mechanisms with superior robustness.

Fig. 8

Effects of the increased load on the period ($T_1/T_0$, solid) and on the progress ($y_1/y_0$, dashed) of the trajectories for the ED systems (red) and ID systems (blue) with $L_0$ fixed at $L_0=9$ and $L_\text {slope}$ varied over (0.6, 40). (Note this is a subset of the ranges plotted in Fig. 5, limited to the region in which sensitivity varies monotonically.) The absolute difference between the two effects accounts for the sensitivity of the system. In the excitatory system, the positive $T_1/T_0$ and $y_1/y_0$ indicate that the perturbed system spends longer time to make larger progress. The decrease of $L_\text {slope}$ reduces $y_1/y_0$ while $T_1/T_0$ is increased, so the sensitivity decreases, until perfect robustness is attained at $L_\text {slope}\approx 0.6$, i.e., where the two red curves meet each other. In the inhibitory system, $T_1/T_0$ is negative, indicating that the perturbation shortens the working period. The effects on the progress and period both become stronger with the decrease of $L_\text {slope}$, so the system becomes more sensitive to the perturbation

When $S_\infty ^\text {FB}$

is approximately linear over the working range of the muscles and limb, the performance–sensitivity curve changes approximately linearly with $L_\text {slope}$. As an illustration, Fig. 9 zooms in on the patterns shown in Fig. 5A with $L_\text {slope}$ varied among $(2,\infty )$, over which $S_\infty ^\text {FB}$ is approximately linear within the possible range of muscle length. On this scale, a fundamental ambiguity arises on account of the dual goals of performance and robustness. Without specifying a relative weighting between these two quantities, there is no well justified way to choose which of the three traces in the upper ensemble, all moving up and to the left, are preferred. The red, black, and blue traces all simultaneously increase performance while decreasing sensitivity, relative to the constant feedback case (green dot). Among the lower ensemble, the blue curve can be rejected as its components exhibit a tradeoff: the robustness is enhanced with decreased performance. Moreover, within the linear regime, the performance and sensitivity can both be improved indefinitely by increasing the feedback gain. The possibility of disambiguating the two effects and finding a globally optimal solution arises only when the parameters are varied enough for nonlinear effects to come into play, as shown next.

The nonlinearity of the synaptic feedback activation function allows the existence of optimal combinations of performance and sensitivity. As the linear regime of $S_\infty ^\text {FB}$ extends beyond the working range of muscle length and the curvature of the sigmoid $S_\infty ^\text {FB}$ becomes more pronounced, some sharp turning points arise in the performance–sensitivity curve of each feedback mechanism, either in terms of performance or sensitivity or both. In general, along a continuous curve in the (sensitivity S, performance Q) plane, indexed by a parameter $\mu$ (here, the sigmoid steepness parameter $L_\text {slope}$), there will be an optimal region marked at one end by the condition $(\partial S/\partial \mu =0, \partial Q/\partial \mu \not =0)$ and at the other end by $(\partial S/\partial \mu \not =0, \partial Q/\partial \mu =0)$. Depending on the relative weight given to S and Q, the optimal value of $\mu$ will place the system somewhere within this segment of the $(S(\mu ), Q(\mu ))$ curve. In the case of the HCO system, the optimal curve among those tested ($L_0=9$, top blue dots in Fig. 5A) makes a hairpin turn in the upper left corner of the plot (blue arrow), leading to relatively unambiguous identification of the optimal value of the slope parameter $L_\text {slope}$. In such cases, these optimal points, or narrowly identified optimal regions, indicate the possibility of well-defined simultaneous optima in both of the performance and sensitivity patterns. Thus our results suggest the possibility to realize these joint goals through adjusting the structure of the sensory feedback mechanisms more broadly. Note that the nearby black trace with $L_0=10$ shows a qualitatively different result on the sensitivity, which gives rise to a sub-optimal mechanism (black arrow). In Appendix A.2 we show the higher-resolution pattern by considering smaller increments of the position parameter $L_0$ and slope parameter $L_\text {slope}$ around the optimal and sup-optimal points.

Fig. 9

Detail from Fig. 5A, expanded to show the region of approximate linearity. Colors as in Fig. 5A. The steepness parameter $L_\text {slope}$ of the synaptic feedback activation function $S_\infty ^\text {FB}$ is varied over $(2,\infty )$. Correspondingly, the gain of the sigmoid ranges from zero to 0.5. With the small gain, the sigmoid $S_\infty ^\text {FB}$ is almost linear over the working range of the muscle length, and the performance–sensitivity curve for each case changes approximately linearly with respect to $L_\text {slope}$

The preceding observations have provided several insights that could support the design of sensory feedback pathways, such as the selection of excitatory versus inhibitory feedback currents, and activation versus inactivation with muscle contraction, as well as the shape of the feedback activation curve, to promote efficiency and robustness in other more realistic HCO-motor models with analogous configurations.

4 Application: Markin hindlimb model with imposed slope

The second closed-loop powerstroke–recovery example we consider is based on a neuromechanical model proposed by Markin et al. (2010), as sketched in Fig. 10. The model consists of a spinal central pattern generator controlling the movement of a single-joint limb. The CPG sends output via efferent activation of two antagonist (flexor and extensor) muscles to a mechanical limb segment, which in turn generates afferent feedback signals to the CPG. The model system performs rhythmic locomotion, comprising a stance phase, during which the limb is in contact with the ground, and a swing phase, during which the limb moves without ground contact. Thus the system falls within the class of powerstroke–recovery systems.

Fig. 10

Schematic of Markin hindlimb model with imposed slope, adapted from Markin et al. (2010). The spinal CPG, consisting of Rhythm Generator (RG) and Pattern Formation (PF) half-center oscillators, as well as Interneurons (In-E/F), receives tonic supra-spinal drive, and generates alternating activation driving corresponding flexor and extensor Motor neurons (Mn). An additional circuit including more interneurons (Int and Inab-E) is incorporated to provide disynaptic excitation to Mn-E. The Mn cells activate antagonistic flexor and exentors muscles, which control the motion of a single-joint limb. Sensory feedback from muscle activation, which comes in three types (Ia, Ib, and II), provides excitation to the ipsilateral neurons of the CPG. Inhibitory connections end with a round ball, and excitatory connections end with a black arrow. The limb stands on the ground with slope specified by $\kappa$, which we take as our perturbation parameter. During a single locomotion cycle, the ground reaction force is active on the limb in the stance phase where the limb velocity $\dot{q}$ is positive (red arrow) but not in the swing phase where $\dot{q}<0$ (blue arrow)

Everyday experience teaches us that normal walking movements are robust against gradual changes in the slope of the terrain, although the detailed shape and timing of the limb trajectory changes as one ascends (or descends) a steeper or shallower incline. As a proxy for this form of parametric perturbation of rhythmic walking movements, we extend Markin et al’s model to include an incline parameter, simulating the effects of a change in the ground slope. We use this parameter in our analysis of performance and sensitivity, to define the sensitivity of the system’s progress relative to the external substrate (the ground) in response to this environmental change.

4.1 The equations of the Markin model

Following Markin et al. (2010), we model the central pattern generator as a multi-layer circuit, consisting of a half-center rhythm generator (RG) containing flexor neurons (RG-F) and extensor neurons (RG-E). These RG neurons project to pattern formation (PF) neurons (PF-F and PF-E, respectively) and to inhibitory interneurons (In-F and In-E) which mediate reciprocal inhibition between the flexor and extensor half-centers. Given sufficient tonic supra-spinal drive, the CPG generates rhythms of alternating activation of flexor and extensor neurons, and the output of PF neurons induces alternating activity in flexor and extensor motor neurons (Mn-F and Mn-E). An additional circuit of interneurons (Int and Inab-E) provides a disynaptic pathway from the extensor side to Mn-E.

The dynamics of the RG, PF, and Mn neurons are each described by two first-order ordinary differential equations, governing each cell’s membrane potential $V_i$ and the slow inactivation gate $h_i$ of a persistent sodium current:

$$\begin{aligned} C\frac{dV_i}{dt}= & -I_\text {NaP}(V_i,h_i)-I_\text {K}(V_i)-I_\text {Leak}(V_i)\nonumber \\ & -I_\text {SynE}(V_i)-I_\text {SynI}(V_i),\nonumber \\ \frac{dh_i}{dt}= & (h_\infty (V_i)-h_i)/\tau _h(V_i). \end{aligned}$$

(17)

Here, $I_\text {NaP}$, $I_\text {K}$, and $I_\text {Leak}$ refer to the persistent sodium current, potassium current, and leak current, respectively, described by

$$\begin{aligned} I_\text {NaP}(V_i,h_i)&=g_\text {NaP}m_\text {NaP}(V_i)h_i(V_i-E_\text {Na}),\\ I_\text {K}(V_i)&=g_\text {K}m^4_\text {K}(V_i)(V_i-E_\text {K}),\\ I_\text {Leak}(V_i)&=g_\text {L}(V_i-E_\text {L}). \end{aligned}$$

Excitatory and inhibitory currents to neuron i are respectively represented by $I_\text {SynE}(V_i)$ and $I_\text {SynI}(V_i)$, given by

$$\begin{aligned} I_\text {SynE}(V_i)= & g_\text {synE}(V_i-E_\text {synE}) \nonumber \\ & \left( \sum _{j}a_{ji}f(V_j)+c_id+\sum _{k}w_{ki}s_k\text {fb}_k\right) ,\nonumber \\ I_\text {SynI}(V_i)= & g_\text {synI}(V_i-E_\text {synI})\sum _{j}b_{ji}f(V_j). \end{aligned}$$

(18)

The nonlinear function f describes the output activity of neuron j, defined to be

$$\begin{aligned} f(V_j)=\left\{ \begin{array}{ll} 1\Big / \left( 1+\exp {\left( -\frac{V_j-V_{1/2}}{k}\right) }\right) ,& \quad V_j\ge V_\text {th},\\ 0,& \quad \text {otherwise}, \end{array}\right. \end{aligned}$$

(19)

where $V_\text {th}$ is the synaptic threshold. Parameter $a_{ji}$ defines the weight of the excitatory synaptic input from neuron j to neuron i, while $b_{ji}$ defines the weight of the inhibitory input from j to i; $c_i$ represents the weight of the excitatory drive d to neuron i; $w_{ki}$ defines the synaptic weight of afferent feedback $\text {fb}_k$ to neuron i, with the feedback strength $s_k$. Details on the feedback terms $\text {fb}_k$ will be provided at the end of this section.

The dynamics of the interneurons, In-F, In-E, Int, and Inab-E, are each described by a single first-order equation:

$$\begin{aligned} C\frac{dV_i}{dt}=-I_\text {Leak}(V_i)-I_\text {SynE}(V_i)-I_\text {SynI}(V_i), \end{aligned}$$

(20)

where the currents are in the same form as above. As shown in Fig. 10, the source of excitatory inputs $I_\text {SynE}$ to these interneurons comes from RG, supra-spinal drive, and sensory feedback. Note that In-E and In-F in particular do not receive any inhibitory input or excitatory supra-spinal drive, so the right-hand side of their voltage equation has $I_\text {SynI}=0$ and $c_i=0$.

The motor neurons, Mn-F and Mn-E, respectively activate two antagonistic muscles, the flexor (F) and extensor (E), controlling a simple single-joint limb. The limb motion is described by a second-order differential equation:

$$\begin{aligned} I\ddot{q}= & \frac{1}{2}mgl_s\cos {q}-b\dot{q}+M_F(q,\dot{q},V_\text {Mn-F},t)\nonumber \\ & +M_E(\pi -q,-\dot{q},V_\text {Mn-E},t)+M_\text {GR}(q,\kappa ), \end{aligned}$$

(21)

where q represents the angle of the limb with respect to the horizontal. The first term accounts for the moment of the gravitational force; $M_F$ and $M_E$ are the moments of the muscle forces; $M_\text {GR}$ denotes the moment of the ground reaction force which is active only during the stance phase, given by

$$\begin{aligned} M_\text {GR}(q,\kappa )=\left\{ \begin{array}{ll} -M_\text {GRmax}\cos {(q-\kappa )},& \quad \dot{q}\ge 0\,\,\text {(stance/power stroke)},\\ 0,& \quad \dot{q}<0\,\,\text {(swing/recovery)}. \end{array}\right. \end{aligned}$$

(22)

Note that the stance (powerstroke) phase is defined when the limb angular velocity $v=\dot{q}$ is nonnegative, while the swing (recovery) phase occurs when the velocity is negative. We expand on the original model from Markin et al. (2010) by introducing parameter $\kappa$ to describe the slope of the ground whereon the limb stands. Thus $\kappa$ will play the role of the load parameter subjected to perturbations for this model.

Fig. 11

A typical solution for the Markin model, in A RG output, B PF output, C In output, D Mn output, E Int (blue) and Inab-E (red) output, F limb angle, G Ia-F feebback activity, H Ia-E feedback activity, I II-F feedback activity, and J Ib-E feedback activity. The flexor side is colored by blue and the extensor side is colored by red. The gray shaded region indicates the stance phase (the powerstroke) during which the limb angle is positive in velocity (panel F), and the white region indicates the swing phase. The stance and swing phases are slightly shifted relative to the extensor and flexor active phases in the CPG. Note that for rat hindlimb walking, the extensor muscle dominates the stance/powerstroke phase Markin et al. (2010)

The feedback signals from the extensor and flexor muscle afferents provide excitatory inputs to the RG, PF, In, and Inab-E neurons. Muscle afferents provide both length-dependent feedback (type Ia from both muscles and type II from the flexor) and force-dependent feedback (type Ib from the extensor). Linear combinations of feedback terms $\text {fb}_k\in \{\text {Ia-F, II-F, Ia-E, Ib-E}\}$, written as $\sum _{k}w_{ki}s_k\text {fb}_k$ in (18), are fed into each side of the model—Ia-F and II-F feedback go to the flexor neurons and Ia-E and Ib-E to the extensor neurons. The feedback terms are in the form

$$\begin{aligned} \begin{aligned} \text {Ia}&=k_\text {vIa}\text {sign}(v^m)|v_\text {norm}|^{\rho _v}+k_\text {dIa}d_\text {norm}+k_\text {nIa}f(V_\text {Mn})+C_\text {Ia},\\ \text {II}&=k_\text {dII}d_\text {norm}+k_\text {nII}f(V_\text {Mn})+C_\text {II},\\ \text {Ib}&=k_\text {FIb}F_\text {norm}. \end{aligned} \end{aligned}$$

(23)

For more details about the mathematical formulations, functional forms, and parameter values of the entire model, see Appendix B.

4.2 Analysis of the Markin model

Figure 11 shows the time courses of the output of neurons $f(V_i)$, limb angle q, and feedback activity $\text {fb}_k$ in the default flat-ground system where $\kappa =0$. Unlike the HCO model, where the active states of neurons overlap with the powerstroke and recovery phases, here the extensor and flexor active phases are slightly shifted relative to the stance and swing phases Spardy et al. (2011). The excitatory feedback of types Ia and II increase during the silent phase of the associated neuron receiving the signal, reaching a peak just before the target neuron becomes active; these feedback signals then decreases during the active phase of the neuron. In contrast, the Ib-E feedback signal, which is solely dependent on the extensor muscle force, is active only when the extensor neurons are active. It remains low until the onset of activation in the extensor units, at which point it jumps up to a high level and then recedes. The different types of the feedback pathways induce distinct performance–sensitivity patterns, as we will discuss in Sect. 4.3.

For this model, we define the performance to be the average distance the limb moves along the ground during the stance phase, given by

$$\begin{aligned} Q(\kappa )=\frac{l_s}{T_\kappa }\int _0^{T_\kappa ^\text {st}}-\frac{d}{dt}\left( \cos {(q(\gamma _\kappa (t))-\kappa })\right) \,dt. \end{aligned}$$

(24)

Here $T_\kappa ^\text {st}$ denotes the stance duration and $l_s$ is the limb length. Figure 12 compares the time series of the perturbed system subjected to a small change in the ground slope ($\kappa _\epsilon =\epsilon =0.01$) with the unperturbed system. As indicated by the green dotted vertical line, the perturbation prolongs the stance phase and delays the transition to the swing phase, in that the system decelerates because of the steeper ground slope (panel B). Consequently, the progress becomes smaller (panel D), and the performance is further deteriorated due to a longer period ($Q_0=0.1506$, $Q(\epsilon )=0.1375$). Note that this model and the HCO model have different conditions for transitioning between the powerstroke and recovery phases. In the HCO model the transition is determined by the neuron activation, while in the biophysically-grounded Markin model the transition is determined by the mechanical condition $\dot{q}=0$. This difference accounts for the opposite responses of the transition timing to analogous environmental challenges: when increasing the difficulty of the task the HCO powerstroke-to-recovery transition moves earlier, while the Markin model’s transition is delayed. Put another way, the powerstroke of the perturbed HCO system contracts relative to the net period, while in the Markin model it expands (cf. Fig. 4).

Fig. 12

Comparison of time series of A output of RG neurons, B limb angle, C output of Mn neurons, and D progress, for the unperturbed trajectory (solid, same as Fig. 11) and perturbed trajectory (dashed) with ground slope $\kappa _\epsilon =0.01$. Both are initiated at the start of their respective stance phase. The perturbed trajectory is uniformly time-rescaled by $t_\epsilon =\frac{T_0}{T_\epsilon }t$, where $T_0=1035$ and $T_\epsilon =1104$. The shaded regions represent the stance phase for the unperturbed case, whereas the vertical green dotted lines represent the onset of the swing phase for the perturbed case, which is delayed due to the steeper ground. Compare Fig. 4

In the following, we study how the performance–sensitivity pattern is affected by the strength of each afferent feedback pathway, represented by $s_k$ where $k\in \{\text {Ia-F, II-F, Ia-E, Ib-E}\}$ in (18). Figure 13 shows the patterns as one of the four feedback strengths is varied, with the other three strengths fixed at the normal strength $s_k=1$. Before we discuss the patterns in Sect. 4.3, note that the possible range of each strength allowing for stable progressive locomotor oscillations is remarkably different:

$$\begin{aligned} & s_\text {Ia-F}\in [0.59,\,2.67],\quad s_\text {Ia-E}\in [0.64,\,4.82],\quad \\ & s_\text {II-F}\in [0,\,4.99],\quad s_\text {Ib-E}\in [0,\,7.68]. \end{aligned}$$

In particular, the system can maintain stable oscillations upon cutting off the II-F or Ib-E feedback pathway, but cannot sustain oscillations without the Ia-type feedback. We finish this subsection by examining the mechanism underlying the failure of movement as the strength parameter is out of the range.

Fig. 13

Performance–sensitivity patterns of the Markin model for different feedback strengths. The black dot represents the default system with the normal feedback strength (all $s_k=1$). In each curve, the strength for only one feedback pathway is varied, with the other three strengths fixed at one. The number at each end indicates the maximal or minimal strength allowing for stable progressive locomotor oscillations. The purple arrow marks the most advantageous configuration among those tested

The Markin model system features fast–slow dynamics (Rubin and Terman 2002), as the persistent sodium inactivation time constant $\tau _h(V_i)$ in (17) is large over the relevant voltage range, so h evolves on a slower timescale than $V_i$. The activity of neuron i can be therefore determined from the location of the intersection of its nullclines in the $(V_i,h_i)$ phase plane. To illustrate the mechanism underlying the transition between phases, Fig. 14 shows the nullcline configuration and the corresponding positions of RG and In neurons in the default system around the CPG transition. Starting from the extensor-inhibited state (panels A), because the V-nullcline and h-nullcline of RG-E intersect at a silent stable fixed point, RG-E cannot escape from the silent state and trade dominance on its own. However, In-E, due to the increasing excitatory inputs from Ia-E and Ib-E feedback (Fig. 11), is able to cross the synaptic threshold first and begin to inhibit RG-F (panels B). This inhibition raises the V-nullcline of RG-F, followed by the decrease of RG-F voltage, weakening its excitation to the downstream In-F. This reduction of excitation results in the decrease of In-F voltage. Consequently, RG-E receives less inhibition from In-F, lowering the V-nullclines of RG-E, such that the critical point moves to the middle branch of its V-nullcline, and hence, RG-E can reach the left knee and jump to the right branch. This transfer of active and silent states indicates that the In cells and excitatory sensory feedback dominate the transitions in the CPG. In the case without sufficiently strong feedback inputs to the silent In cell, the ipsilateral RG neuron will become deadlocked in silence, and thus the locomotion will fail.

Fig. 14

Escape of In triggers CPG transitions. Top: Nullclines and positions of RG neurons (dots) in the $(V_\text {RG}, h_\text {RG})$ phase plane. Bottom: Voltages of In neurons (diamonds) at corresponding times. Blue indicates flexor V-nullcline, voltage and/or h values. Red indicates extensor nullcline and values. The h-nullcline is shown in green. The vertical magenta line represents the synaptic voltage threshold $V_\text {th}$. Arrows mark the direction the neurons are moving. A When the extensor neurons are at the inhibited state, the h-nullcline intersects the inhibited V-nullcline of RG-E at the left branch. Excitation from feedback allows In-E to reach the threshold first, independently of RG-E. B When In-E jumps above the threshold it begins to inhibit RG-F, which raises the V-nullcline of RG-F. The downstream In-F, receiving less excitation from RG-F, thus reduces its voltage and gives less excitation to RG-E, which lowers the V-nullcline of RG-E. As a result, RG-E lies above the left knee of its V-nullcline, so it jumps across the threshold and becomes active, switching the dominance in the CPG (not shown)

Disentangling the effects of changing the strength of an afferent feedback pathway is nontrivial, because of the multiplicity of pathways impacting the activities of many neurons and limb within the system. For instance, Fig. 15 shows the feedback and neuron dynamics in the cases $s_\text {Ia-F}=0.63$ (dashed) and $s_\text {Ia-F}=0.6$ (solid), which is approaching the minimal allowable strength of Ia-F. Varying the strength of the Ia-F pathway alone leads to a chain of changes in all system components including the extensor feedback, and significantly impacts the transition timing. Although the magnitude of the feedback signals does not differ much in the two systems, yet the system with smaller $s_\text {Ia-F}$ accumulates feedback excitation more slowly. This circumstance is due to the interplay of spatial and timing measures of distance in the fast–slow dynamical systems (Terman et al. 1998; Rubin and Terman 2002; Yu et al. 2023), in which a small spatial difference translates into a significant temporal extension. Decreasing the feedback strength $s_\text {Ia-F}$ further (below 0.59) makes it impossible for the extensor feedback to jump up; consequently the In-E neuron will not be facilitated to escape from the inhibited state (not shown) and the movement will stall. This outcome may seem counterintuitive, since reducing the flexor feedback strength should make the flexor In more difficult to escape; however, its direct effect on the mechanics in turn influences the extensor feedback to an analogous (Ia-E) or even more significant (Ib-E) extent (see Eq. (23)). We observe similar situations beyond the nonzero extrema of other feedback strengths, in which either In-F or In-E fails to escape from the silent state and compromises the whole locomotor oscillation.

Fig. 15

Effects of reduced sensory feedback gain $s_\text {Ia-F}$ to In cells near the collapse point. Dashed curves: $s_\text {Ia-F}=0.63$. Solid curves: $s_\text {Ia-F}=0.6$. For all plots, red traces indicate extensor and blue traces indicate flexor quantities. The vertical green dotted and dashed lines indicate the end of stance and swing phases for the case $s_\text {Ia-F}=0.63$. The shaded and white regions represent the stance and swing phases, respectively, for the case $s_\text {Ia-F}=0.6$, both of which are significantly prolonged. Further reduction of $s_\text {Ia-F}$ below 0.59 leads to stalling (convergence to a stable fixed point). Panels G and H plot the weighted feedback inputs ($\sum _{k}w_{ki}s_{k}\text {fb}_k$) fed into each In cell. Although the feedback magnitude does not differ much between the two cases, the time to reach the peak is significantly different, which affects the extensor-flexor transition and the following stance-swing transition

4.3 Performance and sensitivity of the Markin model

When analyzing the abstract HCO model, we considered both ipsilateral and contralateral feedback, both excitatory and inhibitory. In contrast, each of the three sensory feedback pathways in the Markin model is ipsilateral and excitatory. Although the two models are quite different in their level of details, feedback variations, and the type of perturbation, we compare and contrast the two models as far as we can in the Discussion. In this section, by varying the strength of each feedback pathway, we study the performance–sensitivity patterns of the Markin model as shown in Fig. 13 and yield the following observations.

Trade-offs between performance and sensitivity often occur as the afferent feedback strength changes As the strength parameter is varied, the system shows a performance–sensitivity tradeoff when the curve either moves up and to right (performance improves while robustness decreases) or moves down and to left (robustness increases while performance declines). Specifically, the system is not able to generate simultaneously efficient and robust movement with the whole flexor-feedback (Ia-F and II-F) variations as well as with the large Ia-E and small Ib-E variations. For example, Fig. 16 compares the performance of the default system to the system with a stronger Ia-F pathway ($s_\text {Iaf}=1.1$). Similarly to Fig. 15, the increase in the Ia-F strength advances the stance-swing transition, leading to a shortened working period and reduced progress. Since the change in the shape is more significant than the change in the timing, the performance ($Q_0=y_0/T_0$) declines. Note that in the HCO systems featuring either excitatory feedback or inhibitory feedback, the dominance between the shape and timing is reversed (cf. Fig. 6). These examples indicate the interplay of temporal and spatial effects in the performance measure. Likewise, the sensitivity measure, given by (10) or (12), also consists of two factors—the timing response to the perturbation and the shape response. Although the system with strengthened Ia-F feedback is inferior to the default system in terms of performance, it benefits from enhanced robustness. In contrast, weakening the Ia-F strength contributes to an improved performance at the cost of responding more sensitively to external perturbations. With such a performance–sensitivity tradeoff, the system cannot simultaneously achieve the dual goals of efficiency and robustness, especially when varying the flexor-feedback pathway(s) alone.

Fig. 16

Comparison of the dynamics of two systems with $s_\text {Iaf}=1$ (solid traces) versus $s_\text {Iaf}=1.1$ (dashed traces). The other three feedback strengths are fixed at one. The shaded region and white region denote the stance phase and swing phase, respectively, for the first system. The vertical green dotted and dased lines represent the end of the stance phase and swing phase, respectively, for the second system, both of which are advanced due to the faster decay and accumulation of feedback excitation to In neurons (panel C). The limb therefore makes less progress during the shorter stance phase (panel D). Since the temporal decrease in the total period is less dominant than the spatial decrease in the progress, the performance (progress/period) of the system with the larger Ia-F strength is inferior to the default system (0.1457 vs. 0.1505)

Force-dependent feedback gain can optimize both performance and robustness simultaneously. We observe that the force-dependent feedback pathway (type Ib from the extensor) impacts performance and robustness very differently than the length-dependent feedback pathways (types Ia and II). In contrast to the length-dependent feedback, the force-dependent feedback is only active during the active phase of the associated motor neuron, reflecting the fast–slow dynamics of the CPG (see Eq. (23) and Fig. 11). Notably, the performance–sensitivity tradeoff disappears with $s_\text {Ib-E}$ variations, and an optimal value of $s_\text {Ib-E}$ arises, giving the best combination of the pair (S, Q) (purple arrow in Fig. 13). Around the optimal point, the performance saturates while the sensitivity changes dramatically, and at $s_\text {Ib-E}\approx 5.5$, the sensitivity attains zero, indicating that the system achieves infinitesimal homeostasis (Yu and Thomas 2022) concurrent with peak performance. We verify this perfect robustness by plotting the coordinated effects of the slope perturbation on the system trajectory’s shape and timing in Fig. 17 as $s_\text {Ib-E}$ varies over (1, 7). For values of $s_\text {Ib-E}$ above the optimum, the robustness decreases dramatically. Indeed, the underlying oscillation fails when $s_\text {Ib-E}$ reaches about 7.7, presumably due to a global bifurcation. As the $s_\text {Ib-E}$ parameter approaches this point, the trajectory period and amplitude might be insensitive to the parameter variation but the inherent sensitivity to perturbations may dramatically increase. This circumstance is also observed in the HCO model when (subcritical) pitchfork bifurcations occur (Yu and Thomas 2021). On the contrary, with a strong Ia-E pathway (left corner of Fig. 13), the system maintains robustness while its performance significantly decreases, which also appears around a (subcritical) Hopf bifurcation as shown in Yu and Thomas (2021). Although a detailed bifurcation analysis of oscillation termination is beyond the scope of this paper, our observations demonstrate the difficulty of achieving the joint goals of high performance and low sensitivity in a realistic rhythmic physiological system. The variational method, by incorporating both temporal and spatial factors, offers the possibility of an analytic framework that may assist to identify the optimal sensory feedback control mechanism in regulating the system’s efficiency and robustness.

Fig. 17

Effects of the Ib-E feedback strength $s_\text {Ib-E}\in (1,7)$ on the timing ($T_1/T_0$, red) and on the shape ($y_1/y_0$, blue) of the solution trajectories for the Markin model. The purple dot represents the zero-sensitivity value, where $s_\text {Ib-E}=5.5$, which also corresponds to near-peak performance (cf. purple arrow in Fig. 13)

5 Discussion

Efficiency and robustness are important aspects of control systems in both engineering and biological contexts. The application of conceptual and mathematical ideas from control theory in the physiological sciences has a long history (Baylis 1966; Grodins et al. 1967; Khoo 2018). In motor control, viewed through the lens of systems engineering, the concepts of robustness and efficiency can help quantify different aspects of system performance underlying biological fitness. The roles of feedforward, reciprocal, and feedback motor pathways in contributing to robustness and efficiency must be investigated on a case-by-case basis. In many instances, control theorists have observed tradeoffs between efficiency and robustness, although the definitions of these terms may vary from one context to another (Kuo 2002; Boulet and Duan 2007; Ronsse et al. 2008; Vasconcelos et al. 2009; Alfaro et al. 2010; Sariyildiz and Ohnishi 2013). These authors describe a fundamental tradeoff between efficiency and robustness, in the sense that the system improves efficiency at the cost of becoming increasingly fragile to a slight change in any parameter. Hence, an important aspect of controller design is to achieve an acceptable balance between the goals of increasing performance versus maintaining robustness to external changes, when these objectives are in conflict. Throughout this paper, we interpret efficiency as quantifying how well a system performs, and robustness as quantifying the stability of efficiency with respect to parametric perturbations.

A remarkable feature of rhythmic motor systems underlying a wide range of animals’ physiological behaviors (crawling, breathing, scratching, swallowing) is the closed-loop control structure, which integrates brain, body, and environment via sensory feedback (Chiel and Beer 1997; Diekman et al. 2017; Lyttle et al. 2017; Chen et al. 2021; Korkmaz et al. 2021). In particular, feedback control plays a significant role in regulating the robustness and efficiency of these rhythmic movements.⁴ Through afferent feedback pathways, sensory information can communicate current demands, thereby allowing the system to update the corresponding movement on short time scales. In this way, sensory feedback may improve performance; however, robustness remains a critical issue. For example, Kuo (2002) and Yu and Thomas (2021) discussed the fundamental properties of the trade-off between efficiency and robustness for a combined feedback–feedforward model for rhythm generation. These papers demonstrated that a system with pure feedback control, analogous to a chain reflex system, can compensate for unexpected disturbances, but shows poor robustness to imperfect sensors, and that the best trade-off requires a proper design incorporating both feedback and feedforward pathways.

In the present paper, we propose a unified framework to investigate how the architecture of sensory feedback influences the efficiency and robustness of rhythmic systems. In the particular context of a general powerstroke–recovery system, where the system repeatedly engages and disengages with the outside world, we measure the efficiency of the system, i.e., the task performance, in terms of the physical progress of the system over the working period (or equivalently, the mean rate of progress). This measure of the task performance is adopted from Lyttle et al.’s study of the feeding behavior of the sea slug Aplysia californica, which considered mean seaweed consumption rate (cm/sec) (Lyttle et al. 2017; Wang et al. 2022). This measure can be generalized to any powerstroke–recovery systems in which the progressive activity is important. We probe the robustness of the system by studying its ability to maintain task performance despite external perturbations. Following Yu and Thomas (2022), we generalize the analytic formula for this robustness (or sensitivity) measure by using the infinitesimal shape response curve (iSRC) and the local timing response curve (lTRC) developed in Wang et al. (2021). These tools provide a mathematically grounded numerical quantification of the system’s response to perturbations. By simultaneously studying the two objectives, we compare the performance–sensitivity patterns obtained in different feedback architectures, and identify optimal designs achieving the joint requirements of high performance and low sensitivity.

Here, we consider the control problem with two specific neuromechanical systems that produce powerstroke–recovery oscillations. In the paradigmatic half-center oscillator (HCO)-motor model with an external load, adapted from Yu and Thomas (2021), we conduct a comprehensive analysis of different feedback mechanisms in the hopes that we uncover insights that may apply more broadly. We study both contralateral and ipsilateral feedback architectures, both excitatory and inhibitory feedback modes, and both activating and inactivating pathways, in terms of their performance–sensitivity patterns. First, we find that exchanging excitatory and inhibitory architectures reverses the effect of increasing feedback gain on the sensitivity. Second, we observe that the performance response is reversed depending on whether the feedback is activated or inactivated by muscle contractions. More interestingly, as the sigmoid activation of the feedback signal (as a function of limb position) becomes increasingly nonlinear over the working range of the limb, a well-defined simultaneous optimum in performance and sensitivity arises. These findings may guide the selection of modeling frameworks to capture experimental observations or design aims for more realistic brain-body-environment systems with analogous architectures in future work.

In the hindlimb locomotor model (Markin et al. 2010), based on experimental measurements from spindle primary afferents in the cat, we modify the model by imposing a ground slope, and then manipulate the gain parameters for each of four sensory feedback pathways. We find that the force-dependent (type Ib-E) feedback can simultaneously optimize both performance and robustness, while the length-dependent (types Ia-E, Ia-F, and II-F) feedback variations result in marked performance-versus-sensitivity trade-offs. Indeed, as discussed in Yu and Thomas (2021), the situation that the performance is increased by sacrificing the robustness is also observed in the abstract HCO-motor model when the system approaches a subcritical pitchfork bifurcation; and also, the situation that the robustness is improved at the cost of losing efficiency occurs when the system approaches a subcritical Hopf bifurcation.

However, it is difficult to pursue further comparisons between the two models. In the HCO model, we consider only one feedback pathway, solely dependent on the muscle length. In contrast, the locomotor model possesses multiple feedback channels, each of which relies on several components including muscle length, muscle velocity, motor neuron output, or muscle force. As argued in Katz (2023), although focusing on a specific model can advance understanding of the neural basis of behavior in species with similar developmental and physical constraints, doing so may overlook insights that could be obtained by considering convergent evolution as a framework leading to more general principles. Our HCO model can provide a mechanistic understanding leading to predictions about structure-function relationships, and allows us to compare different kinds of architectures that are “equivalent” only at a more general level. On the other hand the detailed Markin model is just one specific architecture where perhaps alternative architectures could have evolved to serve the same purpose. Thus, although the analogy between the HCO-motor and the Markin-motor model is limited, both models are worth studying from a conceptual point of view. In particular, we note that the Ia- and II-type afferent feedback in Markin’s more realistic locomotor model is ipsilateral, excitatory, and increasing with muscle length; the performance–sensitivity pattern for this combination of elements nominally falls into the upper ensemble of the HCO model’s ipsilateral patterns (Fig. 5B, upper star traces). In the HCO setting, these patterns are always advantageous over other patterns in terms of performance, and give rise to the only (performance, sensitivity) optimum. This superiority could be interpreted as suggesting that our modeling analysis might be consistent with the action of natural evolution by which species adapt to environmental demands.

Apart from the restricted scope of feedback functions considered in our examples, it would be desirable to have greater insights into the way any form of sensory feedback shapes the attributes of motor control systems, both in terms of performance and in terms of robustness. However, given the complexity and difficulty to ascertain sensory feedback signals experimentally, it is natural to restrict attention to monotonically increasing or decreasing sensory feedback signals, such as an excitatory or inhibitory conductance in a central pattern generator that increases or decreases steadily as a function of limb position, muscle stretch, muscle velocity, or other biomechanical variables. A realistic example is the responses of cat spindle (group Ia and group II) afferents, which monotonically change with muscle length and velocity signals, and tendon organ (group Ib) afferents, which scales in approximate proportion to muscle forces (Prochazka and Gorassini 1998; Prochazka 1999). Another example is the chemosensory pathway in the respiratory control system, which has been modeled by a sigmoidal relationship between the arterial partial pressure of oxygen and the conductance representing external drive to the CPG (Diekman et al. 2017). Moreover, biological signals are typically limited in range. For instance, the firing rate of a neuron is bounded below by zero and typically bounded above by a maximal firing rate which is related, for example, to the neuron’s absolute refractory period.

The muscle dynamics of the HCO model was inspired in part by Aplysia’s I2 muscle (Yu et al. 1999). A nominal feeding model, featuring grasper-retraction (powerstroke) and grasper-protraction (recovery), has been well studied by Chiel and colleagues (Shaw et al. 2015; Lyttle et al. 2017; Wang et al. 2022). In their model, the proprioceptive feedback input is assumed to be affine linear with the grasper position—decreasing for the protraction neural pool while increasing for the retraction neural pool. Depending on the grasper position and proprioceptive neutral position, the feedback signal can switch between excitation and inhibition during a single swallowing cycle. In our HCO model, we only consider the same form of feedback functions for all neurons and the excitation–inhibition property does not change during the whole movement. These differences point out an interesting and important direction for further study—whether mixed inhibitory and excitatory feedback in a single pathway, or heterogeneous activating/inactivating feedback across different pathways—could provide more realistic models for a broader range of specific motor systems.

Finally, we apply a fast–slow decomposition analysis to the Markin model when discussing the failure of oscillations as the feedback gain parameter reaches the collapse point. Although the stance and swing phases in the locomotion system do not differ much concerning their time scale, it is nevertheless possible that in some powerstroke–recovery systems, the powerstroke, under a heavy load, could proceed on a slow timescale while the recovery could proceed on a fast timescale. In this case one might connect powerstroke–recovery systems to two-stroke relaxation oscillators as studied in depth by Jelbart and Wechselberger (2020), an interesting direction for future work. Another avenue for future inquiry would be to consider the scaling of robustness and sensitivity structures, as studied here, in motor systems spanning a range of length and time scales (Sutton et al. 2023). The approaches we use here should be applicable to these systems as well.

Appendices

A HCO model details

1.1 A.1 Model details

For completeness, we list here the full equations of the HCO model with an externally applied load, as introduced in Sect. 3 and originally proposed in Yu and Thomas (2021). For cell $i,j=1,2$ and $j\ne i$,

$$\begin{aligned} \begin{aligned} C\frac{dV_i}{dt}&=I_\text {ext}-g_\text {L}(V_i-E_\text {L})-g_\text {Ca}M_\infty (V_i)(V_i-E_\text {Ca})\\&\quad -g_\text {K}N_i(V_i-E_\text {K})\\&\quad -g_\text {syn}^\text {CPG}S_\infty ^\text {CPG}(V_j)\left( V_i-E_\text {syn}^\text {CPG}\right) \\&\quad -g_\text {syn}^\text {FB}S_\infty ^\text {FB}(L_{i,j})(V_i-E_\text {syn}^\text {FB}),\\ \frac{dN_i}{dt}&=\lambda _N(V_i)(N_\infty (V_i)-N_i), \end{aligned} \end{aligned}$$

where

$$\begin{aligned} S_\infty ^\text {CPG}(V_j)&=\frac{1}{2}\left( 1+\tanh \left( \frac{V_j-E_\text {thresh}}{E_\text {slope}}\right) \right) ,\\ S_\infty ^\text {FB}(L_{i,j})&=\frac{1}{2}\left( 1\pm \tanh \left( \frac{L_{i,j}-L_0}{L_\text {slope}}\right) \right) ,\\ M_\infty (V_i)&=\frac{1}{2}\left( 1+\tanh \left( \frac{V_i-E_1}{E_2}\right) \right) ,\\ N_\infty (V_i)&=\frac{1}{2}\left( 1+\tanh \left( \frac{V_i-E_3}{E_4}\right) \right) ,\\ \lambda _N(V_i)&=\phi _N\left( \cosh \left( \frac{V_i-E_3}{2E_4}\right) \right) . \end{aligned}$$

The force of muscle i is given by

$$\begin{aligned} F_i(t)=F_0\cdot a_i(t)\cdot LT(L_i(t)). \end{aligned}$$

Here,

$$\begin{aligned} LT(L_i)&=-\frac{3\sqrt{3}}{1250}(L_i-1)(L_i-5)(L_i-15),\quad 5\le L_i\le 15,\\ L_1&=10+x,\quad L_2=10-x,\quad -5\le x\le 5, \end{aligned}$$

and

$$\begin{aligned} a_i(t)=g[A_i(t)-a_0]_+,\quad 0<a_i\le 1, \end{aligned}$$

where

$$\begin{aligned} \frac{dA_i}{dt}&=\tau ^{-1}\{U(V_i)-[\beta +(1-\beta )U(V_i)]A_i\},\\ U(V_i)&=1.03-4.31\exp {(-0.198(V_i/2))},\quad V_i\ge 16. \end{aligned}$$

Note that in the U-equation, we leave implicit a conversion factor from mV to Hz. The limb position is controlled by

$$\begin{aligned} \frac{dx}{dt}=\frac{1}{b}(F_2-F_1+r\kappa F_\ell ), \end{aligned}$$

where

$$\begin{aligned} r=\left\{ \begin{array}{ll} 1,& \quad \text {power stroke},\\ 0,& \quad \text {recovery}. \end{array}\right. \end{aligned}$$

Table 1 specifies the fixed parameter values used for simulations, and Table 2 lists the feedback parameter combination for each figure of the HCO application. The simulation codes are available from https://github.com/zhuojunyu-appliedmath/Powerstroke-recovery. Instructions for reproducing each figure and table in the paper are provided (see the README file at the github site).

Table 1 Fixed parameter values of the HCO model

Table 2 Feedback parameter combinations for the HCO figures

1.2 A.2 Fine structure of performance and sensitivity near the optimum

In the top ensemble of Fig. 5A, the solid blue ($L_0=9$) and black ($L_0=10)$ curves show qualitatively different results on the sensitivity, giving rise to an optimal point and a sub-optimal point for simultaneously achieving high performance and low sensitivity. In order to resolve the fine structure of the two objective functions near these two points, we consider smaller increments of change in the feedback parameters ($L_0, L_\text {slope}$) around the two points, as shown in Figs. 18 and 19. Over this range, the sensitivity of the system changes dramatically (Fig. 18, panel A) while the performance maintains a high level (panel B, where the relative difference between maximum and minimum performance is within 1%), and thus, the key to achieving the two-objective optimization is to reduce sensitivity. The green parabolic path traces the systems with almost zero sensitivity, which is superimposed on the performance surface (panel C). The feedback parameter combinations lying on this path endow the system with perfect robustness and high efficiency. An optimal mechanism we identified in Fig. 5A, located on this path (blue arrow), is one possibility. The black arrow marks the sub-optimal point discussed previously, where the sensitivity is slightly higher than on the curve of optima.

On the one hand, the performance–sensitivity patterns shown in Fig. 5 are a simplification because the real situation is complicated due to the parabolic zero-sensitivity curve. On the other hand, by considering multiple sensory feedback architectures (contralateral vs ipsilateral, activating vs inactivating, excitatory vs inhibitory), the figure enables us to exclude some architectures and suggests the possibility of attaining the efficiency-robustness goal within a narrow range of feedback variations.

Fig. 18

Sensitivity and performance near the optimum and sub-optimum identified in Fig. 5A (blue and black arrows), with smaller increments in the feedback parameters $(L_\text {slope}, L_0)$. The sensitivity varies significantly (panel A), while the performance maintains a high level, where the relative difference between maximum and minimum performance is within 1% (panel B). The points on the green parabola in panel A have almost zero sensitivity, which is superimposed on the performance surface (panel C). The system with the feedback parameter combination lying on the green “optima path”, achieves the joint goals of low sensitivity and high performance, and the optimal configuration tested in Fig. 5A, marked by the blue arrow, is located on this path. The black arrow corresponds to the sub-optimal point tested in Fig. 5A, which is inferior in sensitivity

Fig. 19

Performance–sensitivity patterns near the optimum and sub-optimum identified in Fig. 5A (blue and black arrows), with smaller increments in the feedback parameters $(L_\text {slope}, L_0)$ (as in Fig. 18). The blue and black curves correspond to the two paths shown in the top ensemble of Fig. 5A, with the blue dot and black dot representing the optimum and sub-optimum configuration, respectively. The magenta curve approximates the Pareto (dual optimality) front among the cases tested (Pareto et al. 1964)

B Markin model details

Based on Markin et al. (2010) and Spardy et al. (2011), the following provides modeling details not specified in Sect. 4, especially on the biomechanics and feedback dynamics. The limb motion is described by a second-order differential equation:

$$\begin{aligned} I\ddot{q}= & \frac{1}{2}mgl_s\cos {q}-b\dot{q}+M_F(q,\dot{q},V_\text {Mn-F},t)\nonumber \\ & +M_E(\pi -q,-\dot{q},V_\text {Mn-E},t)+M_\text {GR}(q), \end{aligned}$$

where $I=ml_s^2/3$ is the moment of inertia of the limb with respect to the suspension point and b is the angular viscosity in the hinge joint. The first term accounts for the moment of the gravitational force; $M_F$ and $M_E$ are the moments of the muscle forces (see below); $M_\text {GR}$ represents the moment of the ground reaction force which is active only during the stance phase, given by

$$\begin{aligned} M_\text {GR}(q)=\left\{ \begin{array}{ll} -M_\text {GRmax}\cos {(q-\kappa )},& \quad \dot{q}\ge 0\text { (stance / power stroke)},\\ 0,& \quad \dot{q}<0\text { (swing / recovery)}. \end{array}\right. \end{aligned}$$

The parameter $\kappa$ describes the slope where the limb stands on, which is considered as the load parameter subjected to perturbations for this model. This parameter is not included in the original model Markin et al. (2010); in effect in the original model $\kappa \equiv 0$.

In the muscle model, the muscle length is calculated as

$$\begin{aligned} L_F&=\sqrt{a_1^2+a_2^2-2a_1a_2\cos {q}},\\ L_E&=\sqrt{a_1^2+a_2^2-2a_1a_2\cos {(\pi -q)}}, \end{aligned}$$

and the moment arm is given by

$$\begin{aligned} h_F=(a_1a_2\sin {q})/L_F,\quad h_E=(a_1a_2\sin {(\pi -q)})/L_E, \end{aligned}$$

for the flexor msucle and extensor muscle, respectively. Muscle velocities are defined as

$$\begin{aligned} v_F^m=vh_F,\quad v_E=-vh_E, \end{aligned}$$

where $v=\dot{q}$ denotes the limb angular velocity. The total force in each muscle follows the Hill-type model:

$$\begin{aligned} F=F_\text {max}(f(V)F_lF_v+F_p), \end{aligned}$$

Constant $F_\text {max}$ is the maximal isometric force; f(V) is the output of the corresponding motorneuron. The force-length dependence $F_l$ is given by⁵

$$\begin{aligned} F_l=\exp {\left( -\left| \frac{l^\beta -1}{w}\right| \right) ^\rho }, \end{aligned}$$

where l is the normalized muscle length corresponding to $F_l = 1$, i.e., $l = L/L_\text {opt}$. The force-velocity dependence $F_v$ is given by

$$\begin{aligned} F_v=\left\{ \begin{array}{ll} \frac{b_1-c_1v^m}{v^m+b_1},& \quad v^m<0,\\ \frac{b_2-c_2(l)v^m}{v^m+b_2},& \quad v^m\ge 0, \end{array}\right. \end{aligned}$$

where $c_2(l)=-5.34l^2+8.41l-4.7$. The passive force $F_p$ is calculated as follows:

$$\begin{aligned} F_p&=3.5\ln {\left( \exp {\left( \frac{l-1.4}{0.05}\right) }+1\right) }\\&\quad -0.02\left( \exp {\left( -18.7(l-0.79)\right) }-1\right) . \end{aligned}$$

Given the above, the muscle moments are given by $M_F=F_Fh_F$ and $M_E=-F_Eh_E$.

To complete the model, the afferent feedback terms fed into the CPG neurons are in the form

$$\begin{aligned} \text {Ia}&=k_{\text {vIa}}\text {sign}(v^m)|v_\text {norm}|^{\rho _v}+k_\text {dIa}d_\text {norm} +k_\text {nIa}f(V_\text {Mn})+C_{\text {Ia}},\\ \text {II}&=k_\text {dII}d_\text {norm}+k_\text {nII}f(V_\text {Mn})+C_\text {II},\\ \text {Ib}&=k_\text {FIb}F_\text {norm}. \end{aligned}$$

Here, $v_\text {norm}$ denotes the normalized muscle velocity ($v^m/L_\text {th}$); $d_\text {norm}$ denotes the normalized muscle length ($(L-L_\text {th})/L_\text {th}$ if $L\ge L_\text {th}$ and 0 otherwise); $F_\text {norm}$ denotes the normalized muscle force ($(F-F_\text {th})/F_\text {max}$ if $F\ge F_\text {th}$ and 0 otherwise). The other functions in the CPG model are

$$\begin{aligned} m_\text {K}(V)&=1/\left( 1+\exp {\left( -\frac{V+44.5}{5}\right) }\right) ,\\ m_\text {NaP}(V)&=1/\left( 1+\exp {\left( -\frac{V+47.1}{3.1}\right) }\right) ,\\ h_\infty (V)&=1/\left( 1+\exp {\left( \frac{V+51}{4}\right) }\right) ,\\ \tau _h(V)&=600/\cosh {\left( \frac{V+51}{8}\right) }. \end{aligned}$$

The values of the weight parameters for synaptic connections are provided in Table 3, and the other parameter values are listsed in Table 4. Simulation codes required to produce each figure are available at https://github.com/zhuojunyu-appliedmath/Powerstroke-recovery.

Table 3 Synaptic connection weights of the Markin model

Table 4 Parameter values of the Markin model

C Variational analysis

The tools in the variational analysis were developed in Wang et al. (2021) and generalized in Yu and Thomas (2022) and Yu et al. (2023). Consider a parameterized continuous-time dynamical system defined on a domain $\Omega \subset \mathbb {R}^n$,

$$\begin{aligned} \frac{d\textbf{z}}{dt}=\textbf{F}_\kappa (\textbf{z}), \end{aligned}$$

(25)

where $\textbf{z}\in \Omega$, $\kappa \in \mathcal {I}\subset \mathbb {R}$, and $\textbf{F}_\kappa (\cdot )$ is either smooth or piecewise smooth in both $\textbf{z}$ and $\kappa$. Suppose $\textbf{F}_\kappa (\cdot )$ admits a family of hyperbolic and asymptotically attracting limit cycles for $\kappa \in \mathcal {I}$ including $\kappa _0$, which we consider as the unperturbed limit cycle $\gamma _0(t)$. We assume that the domain is partitioned into two subdomains $\Omega =R^\text {I}\cup R^\text {II}$ with the subdomain boundaries transverse to the flow of the unperturbed limit cycle, and that $\textbf{F}_\kappa (\cdot )$ is smooth within each subdomain.

1.1 C.1 Local timing response curve (lTRC)

For $\textbf{z}\in R^i$, $i=\text {I}, \text {II}$, let $\Gamma ^i(\textbf{z})$ be the time remaining until the unperturbed trajectory beginning at $\textbf{z}$ exits the region. By construction, along $\gamma _0$,

$$\begin{aligned} \frac{d\Gamma ^i}{dt}(\textbf{z})=-1,\quad \textbf{z}\in R^\text {i}. \end{aligned}$$

Hence, by the chain rule

$$\begin{aligned} \textbf{F}_0(\textbf{z})\cdot \nabla \Gamma ^i(\textbf{z})=-1, \end{aligned}$$

(26)

where we write $\textbf{F}_0(\cdot )$ for $\textbf{F}_{\kappa _0}(\cdot )$. Define the local timing response curve (lTRC) for subdomain $R^i$ to be $\eta ^i(t):=\nabla \Gamma ^i(\textbf{z}(t))$. Differentiating both sides of (26) with respect to t yields

$$\begin{aligned} \frac{d\eta ^i}{dt}=-D\textbf{F}_0(\gamma _0(t))^\intercal \eta ^i, \end{aligned}$$

(27)

where $D\textbf{F}_0$ denotes the Jacobian of $\textbf{F}_0$. Let $\textbf{z}_0^{\text {out},i}$ denote the exit point of tracjectory $\gamma _0$ from region $R^i$ and $n^{\text {out},i}$ denote a normal vector of the exit boundary of $R^i$ at $\textbf{z}_0^{\text {out},i}$. Following (26),

$$\begin{aligned} \textbf{F}_0\left( \textbf{z}_0^{\text {out},i}\right) \cdot \nabla \Gamma ^i\left( \textbf{z}_0^{\text {out},i}\right) =-1, \end{aligned}$$

which gives the boundary condition of $\eta ^i$,

$$\begin{aligned} \eta ^i\left( \textbf{z}_0^{\text {out},i}\right) =-\frac{n^{\text {out},i}}{(n^{\text {out},i})^\intercal \textbf{F}_0\left( \textbf{z}_0^{\text {out},i}\right) }. \end{aligned}$$

(28)

The adjoint Eq. (27) together with the boundary condition (28) defines the lTRC within region $R^i$.

One application of the lTRC is to calculate the duration the trajectory spent in each region (phase). For a small positive $\epsilon$, consider $\kappa _\epsilon =\kappa _0+\epsilon \in \mathcal {I}$ and the corresponding perturbed trajectory $\gamma _\epsilon .$ We assume that

$$\begin{aligned} T^i_\epsilon =T^i_0+\epsilon T_1^i+O(\epsilon ^2), \end{aligned}$$

where $T_0^i$ and $T_\epsilon ^i$ represent the duration of the unperturbed trajectory and the perturbed trajectory spent in phase i, respectively, and $T_1^i$ is therefore the linear shift in the phase duration in response to the perturbation. In Yu et al. (2023), we provide a general expression for $T_1^i$:

$$\begin{aligned} T_1^i= & \eta ^i(\textbf{z}_0^{\text {in},i})\cdot \frac{\partial \textbf{z}_\kappa ^{\text {in},i}}{\partial \kappa }\bigg |_{\epsilon =0} -\eta ^i(\textbf{z}_0^{\text {out},i})\cdot \frac{\partial \textbf{z}_\kappa ^{\text {out},i}}{\partial \kappa }\bigg |_{\epsilon =0}\nonumber \\ & +\int _{t^{\text {in},i}}^{t^{\text {out},i}}\eta ^i(\gamma _0(t)) \cdot \frac{\partial \textbf{F}_\kappa (\gamma _0(t))}{\partial \kappa }\bigg |_{\epsilon =0}\,dt. \end{aligned}$$

(29)

Here, at time $t^{\text {in},i}$ the unperturbed limit cycle $\gamma _0$ enters phase i and at time $t^{\text {out},i}$ it exits the phase; $\textbf{z}_\kappa ^{\text {in},i}$ and $\textbf{z}_\kappa ^{\text {out},i}$ represent the entry point to and exit point from the phase i, respectively, for the limit cycle trajectory with $\kappa$. The expression shows that the first-order shift in the phase duration consists of three terms: the first term accounts for the impact of the perturbation on the entry point to the region; the second term arises from the impact on the exit point; the integral term shows the impact on the vector field during the transit from ingress to egress.

1.2 C.2 Infinitesimal shape response curve (iSRC)

Consider the perturbed trajectory $\gamma _\epsilon$ and the unperturbed trajectory $\gamma _0$. Within each subdomain $R^i$, we expand $\gamma _\epsilon$:

$$\begin{aligned} \gamma _\epsilon (\tau ^i_\epsilon (t))=\gamma _0(t)+\epsilon \gamma _1(t)+O(\epsilon ^2), \end{aligned}$$

where $\tau ^i_\epsilon (t)$ is introduced as a rescaled time coordinate so that the expansion is uniform with respect to t. It satisfies

$$\begin{aligned} \tau ^i_\epsilon (0)=0,\quad \tau ^i_\epsilon (t+T_0^i)=\tau ^i_\epsilon (t)+T^i_\epsilon \end{aligned}$$

The vector function $\gamma _1(t)$ is defined as the infinitesimal shape response curve (iSRC), which is piecewise-specified with period $T_0$. In region $R^i$, the iSRC satisfies a nonhomogenous variational equation

$$\begin{aligned} \frac{d\gamma _1^i(t)}{dt}=D\textbf{F}_0(\gamma _0(t))\gamma _1^i(t)+\nu _1^i(t)\textbf{F}_0(\gamma _0(t))+\frac{\partial \textbf{F}_\kappa (\gamma _0(t))}{\partial \kappa }\bigg |_{\epsilon =0}, \nonumber \\ \end{aligned}$$

(30)

where $\nu _1^i=\frac{\partial ^2\tau ^i_\epsilon (t)}{\partial \epsilon \partial t}\big |_{\epsilon =0}$ measures the (local) timing sensitivity to the perturbation in region $R^i$. In the special case of $\tau ^i_\epsilon$ with a linear scaling, i.e., $\tau _\epsilon ^i(t)=(T_\epsilon ^i/T_0^i)t$, we have $\nu _1^i=T_1^i/T_0^i$ independent of t, where $T_1^i$ is given by (29).

The iSRC Eq. (30) requires an initial condition, which is

$$\begin{aligned} \gamma _1^i(t^{\text {in},i})=\lim _{\epsilon \rightarrow 0}\frac{\textbf{p}_\epsilon ^i-\textbf{p}_0^i}{\epsilon }, \end{aligned}$$

where $\textbf{p}_\epsilon ^i$ and $\textbf{p}_0^i$ represent the intersection points of the limit cycle trajectories with the boundary surfaces between regions. Generally, changing the surfaces (initial conditions) results in the consequent iSRC functions related by a simple phase shift, which has no effect on the sensitivity given by (10). See Lemma 1 in Yu and Thomas (2022) or Lemma 2.3 in Wang et al. (2021)) for proof.

1.3 C.3 Derivation of sensitivity formula (10)

This section presents the derivation of Eq. (10). It is a general formula to calculate the sensitivity of an averaged quantity with respect to the change in any parameter, applicable to the powerstroke–recovery systems. The derivation follows the same exposition given in Yu and Thomas (2022) but with some modifications.

Fig. 20

Discretization of limit cycles by points comprising the limit cycles’ orbits satisfying (25), illustrating the construction for comparing the average of a quantity around the two orbits. Figure modified from Yu and Thomas (2022). The loop $\textbf{x}$ (blue trace) represents the unperturbed limit cycle with the default parameter value $\kappa _0$; the loop $\textbf{y}$ (red trace) represents the perturbed limit cycle with $\kappa _\epsilon =\kappa _0+\epsilon$. Each orbit consists of two phases—powerstroke (ps) and recovery (re). During the powerstroke, each orbit is divided into $n_1$ steps of equal time $T^\text {ps}_x/n_1$ or $T^\text {ps}_y/n_1$, respectively, giving points ($\textbf{x}^\text {ps}_1, \textbf{x}^\text {ps}_2,\ldots ,\textbf{x}^\text {ps}_{n_1-1}$) and ($\textbf{y}^\text {ps}_1, \textbf{y}^\text {ps}_2,\ldots ,\textbf{y}^\text {ps}_{n_1-1}$). During the recovery, each orbit is divided into $n_2$ steps of equal time $T_x^\text {re} /n_2$ or $T_y^\text {re} /n_2$, respectively, giving points ($\textbf{x}^\text {re}_1, \textbf{x}^\text {re}_2,\ldots ,\textbf{x}^\text {re}_{n_2-1}$) and ($\textbf{y}^\text {re}_1, \textbf{y}^\text {re}_2,\ldots ,\textbf{y}^\text {re}_{n_2-1}$). The phase transition points are marked as $\textbf{x}^{\text {rp}},\textbf{x}^{\text {pr}},\textbf{y}^{\text {rp}},\textbf{y}^{\text {pr}}$ correspondingly

Denote the unperturbed limit cycle (with $\kappa _0$) by $\textbf{x}$ and the perturbed limit cycle (with $\kappa _\epsilon =\kappa _0+\epsilon$) by $\textbf{y}$, respectively. Denote the periods by

$$\begin{aligned} T_x=T_x^\text {ps}+T_x^\text {re},\quad T_y=T_y^\text {ps}+T_y^\text {re}. \end{aligned}$$

During the powerstroke, each limit cycle orbit is divided into $n_1$ steps of equal time $T_x^\text {ps}/n_1$ or $T_y^\text {ps}/n_1$; during the recovery, each orbit is divided into $n_2$ steps of equal time $T_x^\text {re}/n_2$ or $T_y^\text {re}/n_2$. Let $n=n_1+n_2.$ Mark off the points as in Fig. 20. Then,

$$\begin{aligned} \textbf{x}_k^\text {ps}&=\textbf{x}\left( \frac{k}{n_1}T^\text {ps}_x\right) ,\quad \textbf{y}_k^\text {ps}=\textbf{y}\left( \frac{k}{n_1}T^\text {ps}_y\right) ,\quad k=0,1,\ldots ,n_1,\\ \textbf{x}_k^\text {re}&=\textbf{x}\left( T^\text {ps}_x+\frac{k}{n_2}T^\text {re}_x\right) ,\quad \textbf{y}_k^\text {re}=\textbf{y}\left( T^\text {ps}_y+\frac{k}{n_2}T^\text {re}_y\right) ,\\&\quad k=0,1,\ldots ,n_2. \end{aligned}$$

Note that at the powerstroke-to-recovery (pr) transition state,

$$\begin{aligned} \textbf{x}^{\text {pr}}=\textbf{x}^\text {ps}_{n_1}=\textbf{x}^\text {re}_{0},\quad \textbf{y}^{\text {pr}}=\textbf{y}^\text {ps}_{n_1}=\textbf{y}^\text {re}_{0}, \end{aligned}$$

and at the recovery-to-powerstroke (rp) transition state,

$$\begin{aligned} \textbf{x}^{\text {rp}}=\textbf{x}^\text {ps}_0=\textbf{x}^\text {re}_{n_2},\quad \textbf{y}^{\text {rp}}=\textbf{y}^\text {ps}_0=\textbf{y}^\text {re}_{n_2}. \end{aligned}$$

Let $Q_x$ be the average of the quantity $q(\textbf{x})$ around the trajectory $\textbf{x}$, i.e.,

$$\begin{aligned} Q_x= & \frac{1}{T_x}\int _0^{T_x}q(\textbf{x}(t))\,dt\nonumber \\= & \frac{1}{T_x}\left[ \int _0^{T_x^\text {ps}}q(\textbf{x}(t))\,dt+\int _{T_x^\text {ps}}^{T_x}q(\textbf{x}(t))\,dt\right] , \end{aligned}$$

(31)

and similarly $Q_y$. Suppose q and the limit cycles are piecewise smooth, where the discontinuity occurs at the transitions between the two phases. As $n_1,n_2\rightarrow \infty$, following (31), since the set of discontinuous points has measure zero, we have

$$\begin{aligned} Q_x&=\frac{1}{T_x}\left[ \frac{T_x^\text {ps}}{n_1}\sum _{k=0}^{n_1-1}q(\textbf{x}_k^\text {ps})+\frac{T_x^\text {re}}{n_2}\sum _{k=0}^{n_2-1}q(\textbf{x}_k^\text {re})\right] +O\left( \frac{1}{n^2}\right) \\&=\frac{1}{n_1}\sum _{k=0}^{n_1-1}\frac{T_x^\text {ps}}{T_x}q(\textbf{x}_k^\text {ps})+\frac{1}{n_2}\sum _{k=0}^{n_2-1}\frac{T_x^\text {re}}{T_x}q(\textbf{x}_k^\text {re})+O\left( \frac{1}{n^2}\right) , \end{aligned}$$

with a similar expression for $Q_y$. Then,

$$\begin{aligned} Q_y-Q_x&=\frac{1}{n_1}\sum _{k=0}^{n_1-1}\left( \frac{T_y^\text {ps}}{T_y}q(\textbf{y}_k^\text {ps}) -\frac{T_x^\text {ps}}{T_x}q(\textbf{x}_k^\text {ps})\right) \\&\quad +\frac{1}{n_2}\sum _{k=0}^{n_2-1}\left( \frac{T_y^\text {re}}{T_y}q(\textbf{y}_k^\text {re})-\frac{T_x^\text {re}}{T_x}q(\textbf{x}_k^\text {re})\right) +O\left( \frac{1}{n^2}\right) . \end{aligned}$$

Let the fraction of the powerstroke duration be $\beta$, i.e.,

$$\begin{aligned} \beta =\frac{T^\text {ps}}{T},\quad 1-\beta =\frac{T^\text {re}}{T}. \end{aligned}$$

Expand $\beta _y$ and $q(\textbf{y}_k)$ around $\epsilon =0$:

$$\begin{aligned} \beta _y&=\beta _x+\epsilon \beta _1+O(\mu ^2),\\ q(\textbf{y}^\text {ps}_k)&=q(\textbf{x}^\text {ps}_k)+\nabla q(\textbf{x}^\text {ps}_k)\cdot (\textbf{y}^\text {ps}_k-\textbf{x}^\text {ps}_k)+\epsilon \frac{\partial q(\textbf{x}_k^\text {ps})}{\partial \kappa }\\&\quad +O(\epsilon ^2),\quad k=1,2,\ldots n_{1}-1,\\ q(\textbf{y}^\text {re}_k)&=q(\textbf{x}^\text {re}_k)+\nabla q(\textbf{x}^\text {re}_k)\cdot (\textbf{y}^\text {re}_k-\textbf{x}^\text {re}_k)+O(\epsilon ^2),\quad \\&\quad k=1,2,\ldots n_{2}-1, \end{aligned}$$

where $\beta _1$ represents the linear shift in $\beta ^x$ under the perturbation. Since the quantity q may be explicitly affected by the perturbation during the powerstroke, then there is an additional term $\frac{\partial q}{\partial \kappa }$ in the expansion of $q(\textbf{y}^\text {ps}_k)$. Note that the gradient $\nabla q$ is not well-defined at the discontinuous points. Then,

$$\begin{aligned} Q_y-Q_x&=\frac{1}{n_1}\sum _{k=1}^{n_1-1}\left[ \beta ^yq(\textbf{y}_k^\text {ps})-\beta ^xq(\textbf{x}_k^\text {ps})\right] \\&\quad +\frac{1}{n_2}\sum _{k=1}^{n_2-1}\left[ (1-\beta ^y)q(\textbf{y}_k^\text {re})-(1-\beta ^x)q(\textbf{x}_k^\text {re})\right] \\&\quad +\frac{1}{n_1}\left[ \beta ^yq(\textbf{y}^\text {rp})-\beta ^xq(\textbf{x}^\text {rp})\right] \\&\quad +\frac{1}{n_2}\left[ (1-\beta ^y)q(\textbf{y}^\text {pr})-(1-\beta ^x)q(\textbf{x}^\text {pr})\right] \\&\quad +O\left( \frac{1}{n^2}\right) \\&=\frac{1}{n_1}\sum _{k=1}^{n_1-1}\left[ \beta ^x\nabla q(\textbf{x}_k^\text {ps})\cdot (\textbf{y}^\text {ps}_k -\textbf{x}^\text {ps}_k)+\epsilon \beta ^x\frac{\partial q(\textbf{x}_k^\text {ps})}{\partial \kappa }\right. \\&\quad \left. +\epsilon \beta _1q(\textbf{x}^\text {ps}_k)\right] \\&\quad +\frac{1}{n_2}\sum _{k=1}^{n_2-1}\left[ (1-\beta ^x)\nabla q(\textbf{x}_k^\text {re})\cdot (\textbf{y}^\text {re}_k-\textbf{x}^\text {re}_k)-\epsilon \beta _1q(\textbf{x}^\text {re}_k)\right] \\&\quad +\frac{1}{n_1}\left[ \beta ^x(q(\textbf{y}^\text {rp})-q(\textbf{x}^\text {rp}))+\epsilon \beta _1q(\textbf{y}^\text {rp})\right] \\&\quad +\frac{1}{n_2}\left[ (1-\beta ^x)(q(\textbf{y}^\text {pr})-q(\textbf{x}^\text {pr})) -\epsilon \beta _1q(\textbf{y}^\text {pr})\right] \\&\quad +O(\epsilon ^2)+O\left( \frac{1}{n^2}\right) . \end{aligned}$$

Using the iSRC $\gamma _1$ of $\textbf{x}$, we obtain

$$\begin{aligned}&Q_y-Q_x\\&\quad =\frac{\epsilon }{n_1}\sum _{k=1}^{n_1-1}\left[ \beta ^x\nabla q(\textbf{x}_k^\text {ps})\cdot \gamma _1(\textbf{x}^\text {ps}_k)+\beta ^x\frac{\partial q(\textbf{x}_k^\text {ps})}{\partial \kappa }+\beta _1q(\textbf{x}^\text {ps}_k)\right] \\&\qquad +\frac{\epsilon }{n_2}\sum _{k=1}^{n_2-1}\left[ (1-\beta ^x)\nabla q(\textbf{x}_k^\text {re})\cdot \gamma _1(\textbf{x}^\text {re}_k)-\beta _1q(\textbf{x}^\text {re}_k)\right] \\&\qquad +\frac{1}{n_1}\left[ \beta ^x(q(\textbf{y}^\text {rp})-q(\textbf{x}^\text {rp}))+\epsilon \beta _1q(\textbf{y}^\text {rp})\right] \\&\qquad +\frac{1}{n_2}\left[ (1-\beta ^x)(q(\textbf{y}^\text {pr})-q(\textbf{x}^\text {pr})) -\epsilon \beta _1q(\textbf{y}^\text {pr})\right] +O(\epsilon ^2)\\&\qquad +O\left( \frac{1}{n^2}\right) . \end{aligned}$$

Dividing both sides by $\epsilon$ yields

$$\begin{aligned}&\frac{Q_y-Q_x}{\epsilon }\\&\quad =\frac{1}{n_1}\sum _{k=1}^{n_1-1}\left[ \beta ^x\nabla q(\textbf{x}_k^\text {ps})\cdot \gamma _1(\textbf{x}^\text {ps}_k)+\beta ^x\frac{\partial q(\textbf{x}_k^\text {ps})}{\partial \kappa }+\beta _1q(\textbf{x}^\text {ps}_k)\right] \\&\qquad +\frac{1}{n_2}\sum _{k=1}^{n_2-1}\left[ (1-\beta ^x)\nabla q(\textbf{x}_k^\text {re})\cdot \gamma _1(\textbf{x}^\text {re}_k)-\beta _1q(\textbf{x}^\text {re}_k)\right] \\&\qquad +\frac{\beta ^x}{\epsilon n_1}\left( q(\textbf{y}^\text {rp})-q(\textbf{x}^\text {rp})\right) +\frac{\beta _1}{n_1}q(\textbf{y}^\text {rp})\\&\qquad +\frac{1-\beta ^x}{\epsilon n_2}\left( q(\textbf{y}^\text {pr})-q(\textbf{x}^\text {pr})\right) -\frac{\beta _1}{n_2}q(\textbf{y}^\text {pr})\\&\qquad +O(\epsilon )+O\left( \frac{1}{n^2}\right) . \end{aligned}$$

Taking the limits $\epsilon \rightarrow 0$ and $n_1,n_2\rightarrow \infty$, we obtain

$$\begin{aligned}&\frac{\partial Q}{\partial \kappa }(\kappa _0)\\&\quad =\frac{1}{T_x^\text {ps}}\int _0^{T^\text {ps}_x}\left[ \beta ^x\nabla q(\textbf{x}(t))\cdot \gamma _1(t)+\beta ^x\frac{\partial q(\textbf{x}_k^\text {ps})}{\partial \kappa }+\beta _1q(\textbf{x}(t))\right] \,dt\\&\qquad +\frac{1}{T^\text {re}_x}\int _{T_x^\text {ps}}^{T_x}\left[ (1-\beta ^x)\nabla q(\textbf{x}(t))\cdot \gamma _1(t)-\beta _1q(\textbf{x}(t))\right] \,dt\\&\qquad +\lim _{\begin{array}{c} \epsilon \rightarrow 0\\ n_1\rightarrow \infty \end{array}}\frac{\beta ^x}{\epsilon n_1}\left( q(\textbf{y}^\text {rp})-q(\textbf{x}^\text {rp})\right) \\&\qquad +\lim _{\begin{array}{c} \epsilon \rightarrow 0\\ n_2\rightarrow \infty \end{array}}\frac{1-\beta ^x}{\epsilon n_2}\left( q(\textbf{y}^\text {pr})-q(\textbf{x}^\text {pr})\right) . \end{aligned}$$

For the last two terms, use the directional derivative in the direction (say $\textbf{u}$) tangent to the recovery–powerstroke boundary, and expand $q(\textbf{y}^\text {rp})$ and $q(\textbf{y}^\text {pr})$ to be

$$\begin{aligned} q(\textbf{y}^\text {rp})&=q(\textbf{x}^\text {rp})+\epsilon \nabla _\textbf{u}q(\textbf{x}^\text {rp})+O(\epsilon ^2)\\ q(\textbf{y}^\text {pr})&=q(\textbf{x}^\text {pr})+\epsilon \nabla _\textbf{u}q(\textbf{x}^\text {pr})+O(\epsilon ^2) \end{aligned}$$

Then,

$$\begin{aligned} \lim _{\begin{array}{c} \epsilon \rightarrow 0\\ n_1\rightarrow \infty \end{array}}\frac{1}{\epsilon n_1}\left( q(\textbf{y}^\text {rp})-q(\textbf{x}^\text {rp})\right)&=\lim _{\begin{array}{c} \epsilon \rightarrow 0\\ n_1\rightarrow \infty \end{array}}\frac{1}{\epsilon n_1}\left( \epsilon \nabla _\textbf{u}q(\textbf{x}^\text {rp})+O(\epsilon ^2)\right) \\&=\lim _{n_1\rightarrow \infty }\frac{\nabla _\textbf{u}q(\textbf{x}^\text {rp})}{n_1}=0,\\ \lim _{\begin{array}{c} \epsilon \rightarrow 0\\ n_2\rightarrow \infty \end{array}}\frac{1}{\epsilon n_2}\left( q(\textbf{y}^\text {pr})-q(\textbf{x}^\text {pr})\right)&=\lim _{n_2\rightarrow \infty }\frac{\nabla _\textbf{u}q(\textbf{x}^\text {pr})}{n_2}=0. \end{aligned}$$

Therefore, we obtain the sensitivity of the average Q for the unperturbed system:

$$\begin{aligned} & \frac{\partial Q}{\partial \kappa }(\kappa _0)=\frac{1}{T_0^\text {ps}} \int _0^{T_0^\text {ps}}\left[ \beta _0\left( \nabla q_{0}(\gamma _0(t))\cdot \gamma _1(t) +\frac{\partial q_\kappa (\gamma _0(t))}{\partial \kappa }\bigg |_{\epsilon =0}\right) \right. \nonumber \\ & \quad \left. +\beta _1q_0(\gamma _0(t))\right] \,dt\nonumber \\ & \quad +\frac{1}{T^\text {re}_0}\int _{T_0^\text {ps}}^{T_0}\left[ (1-\beta _0)\nabla q_0(\gamma _0(t))\cdot \gamma _1(t)-\beta _1q_0(\gamma _0(t))\right] \,dt. \nonumber \\ \end{aligned}$$

(32)

Note that when q represents the change rate of progress, then for the models considered here (and for Shaw et al.’s Aplysia feeding model (Shaw et al. 2015; Lyttle et al. 2017; Wang et al. 2022)) q vanishes during the recovery, indicating that $q_0(\gamma _0(t))\equiv 0$ for $t\in (T_0^\text {ps},T_0]$ and that the second integral in (32) reduces to zero. Formula (10) is derived.