Abstract
Current control of prosthetic hands is ineffective when grasping deformable, irregular, or heavy objects. In humans, grasping is achieved under spinal reflexive control of the musculotendon skeletal structure, which produces a hand stiffness commensurate with the task. We hypothesize that mimicking reflex on a prosthetic hand may improve grasping performance and safety when interacting with human. Here, we present a design of compliant controller for prosthetic hand with a neuromorphic model of human reflex. The model includes 6 motoneuron pools containing 768 spiking neurons, 1 muscle spindle with 128 spiking afferents, and 1 modified Hill-type muscle. Models are implemented using neuromorphic hardware with 1 kHz real-time computing. Experimental tests showed that the prosthetic hand could sustain a 40 N load compared to 95 N for an adult. Stiffness range was adjustable from 60 to 640 N/m, about 46.6% of that of human hand. The grasping velocity could be ramped up to 14.4 cm/s, or 24% of the human peak velocity. The complaint control could switch between free movement and contact force when pressing a deformable beam. The amputee can achieve a 47% information throughput of healthy humans. Overall, the reflex-enabled prosthetic hand demonstrated the attributes of human compliant grasping with the neuromorphic model of spinal neuromuscular reflex.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Upper limb amputation would lead to permanent loss of hand function in amputees. The needs for recovery, compensation and replacement of their hand functionality have motivated generations of neural prosthetic technologies.44,59,65,82 Despite the advancement of neural prosthetic hands in the past decades, rejection of prosthetic hands remains high.6,66 A primary challenge for existing prosthetic hands is that they lack the compliant properties in the grasp control, such that the grasping behavior is indifferent to the stiffness or brittleness of the object. As a result, the amputee may find it difficult to attend to the mechanical subtlety of grasped objects.13,29,59 Therefore, an inevitable question of transcending prosthetic hand is how human-like compliant control can be reified on the device.
When interacting with real-world objects, a prosthetic hand is expected to adapt its behavior commensurate with the object, especially for objects that are deformable, crispy, or irregular in shape. In able-bodied individuals, nuances of an object are perceived through an interplay among visual, tactile, and proprioceptive information32; In amputees, however, both tactile and proprioceptive feedback are compromised at their stumps,12,19,91 leaving visual feedback an indirect source for the calculation of grasping force.71 In this scenario, if signals about hand gesture can be engineered into biomimetic forms such as spike trains, it may be possible to emulate the flow of proprioceptive information,61 which can be integrated into an artificial controller mimicking human reflex. This is a fundamentally different approach for prosthetic control compared to most existing applications, which typically model the controllers as linear-time-invariant systems.43,46,57
The role of reflex in human movement has been demonstrated in a long history of scientific literature.31,52,63,68,76 Computational models remain central to the reconstruction of reflex for prosthetic hand. Mammalian experiments have revealed principles instrumental for neural control of movements: motor units with patterned recruitment order,37 spinal-level neural circuitry,1,67 dynamics of muscle spindle,54 muscle with viscoelastic properties90 and so on. In case of stretch reflex, the stretching is sensed by the muscle spindle, transduced into excitatory spikes in the afferents, and it eventually increases the muscle force to resist the stretch. This reinforcement is added on to the inherent stiffness of muscles to further adjust force control.60 Reverse engineering of muscle spindle started from the pioneering work on animal neurography,54 followed by computational modeling of spindle,56 the functional implication,48 the addition of spiking interface and real-time emulation,62 and recently a “model-in-the-loop” application that demonstrated its sufficiency in limb control.61
One of the challenges with hand prosthesis is how to faithfully execute movement intention of the amputee. EMG signals have been generally adopted to control the prosthetic hand in a wide range of schemes, including proportional control,3,33,34,72,79 regression control,40,58,81 on-off control,80 finite state machine control,15,46 pattern recognition-based control,4,22,77,86,88 postural control,73,74,89 etc. An alternative approach for myocontrol is to decode the latent control variable behind raw EMG using biology-inspired models. In most cases, the EMG signal was modeled as amplitude-modulated band-limited noise,16,18 and therefore the EMG signal was filtered and decoded for myocontrol by algorithms that extract the amplitude envelope.39 Sanger demonstrated a new Bayesian model for EMG that described the motor intent as a stochastic process with jumping70 for myocontrol applications.7
This study proposes a new methodology of real-time control for cable-driven prosthetic hands using a human-like reflex model. In specific, we seek to reconstruct biomimetic reflex on a prosthetic hand, which presumably enables a natural translation between motor command and hand movement. The biomimetic reflex includes spiking neurons, skeletal muscles, muscle spindles, spinal circuitries, etc. All components in the reflex have been derived from well-established previous experiments and modeling work. The real-time closed-loop execution of reflex is enabled by neuromorphic hardware, a customized architecture of VLSI (very-large-scale-integrated-circuit) that parallelizes large-scale model computation. The goal of this study is to verify the feasibility of model-based control for hand prosthesis. We hypothesize that with the help of biomimetic reflex, a cable-driven prosthetic hand could exhibit rich behaviors of human reflexes in response to constraints from the environment and task. Results will justify whether the model-based control approach may have significant benefits for the motor performance of prosthetic hand.
Materials and Methods
In this study, we proposed a model-based control strategy for prosthetic hand, which was inspired by the neuromuscular control of upper-extremities in human. The model emulated a monosynaptic reflex loop by implementing elements known to be important in biological neural control. Surface EMG from wrist muscles was used to activate a torque motor, which pulled a cable to create hand movements. Wrist flexors were chosen because those muscles were still functioning after fore-arm amputation, and wrist flexors were easily recorded using sEMG for myocontrol.25,27,30,36 Using this methodology, we were able to build a prosthetic hand platform that enabled us to ask whether introducing principles from human sensorimotor control may benefit the performance of hand prosthesis.
The following parts are presented in the following order: first the build-up of the cable-driven prosthetic as the mechanical plant; second the detailed descriptions of the model-based controller implementing biomimetic reflex, especially how components were modeled following the flow of signals; third how the model was emulated on neuromorphic hardware.
Cable-Driven Prosthetic Hand Operated by Wrist-Flexor EMG
Prosthetic hands with cable-driven designs are favored for model-based control. This is because cable-driven mechanisms are compatible with tendon-driven fingers in back-drivability; also the translational movement on a cable is compatible with the lengthening of a muscle.85 Without loss of generality, we demonstrate model-based control using an open-source design of cable-driven hand (InMoov).41 The hand was 3D-printed and assembled for under 100 U.S. dollars. Other candidate designs of cable-driven hands include Ada hand, Brunel, Dextrus, etc.9,11,14,64,78; pneumatically-driven hands are also acceptable.20,83 Hands with gear or linkage transmission,17,45,50,53,84 however, are not directly applicable for model-based control, because simulating pull-only muscle tension can be non-trivial.
In our implementation, hand joints were pulled using high density Polyethylene (PE) cables as proxies of human tendon, each of which simultaneously flexed the metacarpophalangeal (MCP), Distal Interphalangeal (DIP), and proximal interphalangeal (PIP) joints. The cables were attached to the shaft of a DC torque motor (PD2-C42, Nanotec Electronics GmbH & Co.KG, Germany) communicated through a CANopen interfacing card (ECAN-IT, Guangcheng Technology Co., Ltd., Shenyang, China). The motor generated a torque resulting in a tension on the cable. The main control loop of the prosthetic hand was coordinated on a PC (Intel Core i7-8700CPU, 3.20 GHz, 16 GB Memory, Microsoft Windows 10 64-bit) at 100Hz sampling rate. Only finger flexion was activated by the user; finger extension was passively applied by an extending spring in each joint. Figure 1 shows an illustration and the work scene of the cable-driven prosthetic hand.
Model-Based Controller with Biomimetic Reflex
The neuromuscular components of human hand control at the level of spinal-cord are illustrated in Fig. 2a. We focus on the spinal level of human nervous system because the supra-spinal structures are not lesioned through amputation. In a typical joint (e.g. the metacarpophalangeal joint) rotated by an antagonistic pair of muscles, muscle contraction produces the force transferred via tendons attached to the joint. Muscle contraction is determined first by the spiking excitations from motoneurons (alpha and gamma), and the contraction is also modulated by proprioceptive feedback from muscle spindles.
The overall architecture of the controller is shown in Fig. 2b. Colored blocks represent the key models from neurophysiology of reflex, meaning that their dynamic behavior should reflect a mammalian reflex loop, which includes a pool of motoneurons, a skeletal muscle, and associated muscle spindles. Raw EMG signals are filtered into an alpha motor command activating the motoneuron pool, which is then converted into a force of contraction through muscle model, followed by a DC motor to establish the calculated force; meanwhile, the calculation of muscle force is constantly adjusted by a biomimetic spindle.
Definitions and Responsibilities of Models in Biomimetic Reflex
Details about a biomimetic reflex are shown in Fig. 2c. The alpha motor command is filtered from sEMG on the wrist flexor (normally flexor carpi ulnaris, FCU) from the amputee. The motor command enters the biomimetic reflex loop as an excitatory post-synaptic current (EPSC), which is distributed to several pools of spiking motoneurons. Spiking outputs from the motoneurons are subsequently converted to excitatory drive to Hill-type models of skeletal muscle. The muscle model calculates a muscle force that will eventually be established by the DC motor. Information about muscle lengthening is sent back to a model of muscle spindle, which subsequently produces excitatory afferents that loop back to the motoneuron pool.
The mathematical definitions of models are listed in Table 1. It is required that adjacent models must communicate with compatible types of signal. And all interfacing signals should be measurable in reality, such that the validity of model-based control can be verified against literature.
Model for Alpha Motor Command
Alpha motor command is the entrance to the model-based controller (Fig. 2b). We propose to decode the alpha motor command by filtering the wrist flexor EMG from the amputee. Wrist flexors are superficial in anatomy so they are widely used in myocontrol applications.25,27,30,36 The filtering was done using a nonlinear Bayesian algorithm that has demonstrated its advantage in myocontrol applications.7
Model of Spiking Neuron and Motoneuron Pool
Spiking neurons are implemented using the Izhikevich approximation42 to the classic Hodgkin–Huxley model.38 In the biomimetic reflex, we include 768 alpha-motoneurons divided into 6 pools representing motor units of 6 various sizes (Fig. 2c). The recruitment order in motoneuron pools naturally emerge due to Henneman’s size-principle.37 Since each motoneuron pool contains 128 neurons, their EPSCs are superimposed with a Gaussian noise to allow 128 neurons fire at similar rate but with different timing. The number of motoneuron was determined by balancing the innervation number of a typical mammalian muscle8 and the maximum number allowed by the hardware.61 More details about neuron modeling are listed in Appendix.
Model of Skeletal Muscle with Access to Proprioception
Skeletal muscles are implemented using a Hill-type model (Fig. 3a). The formulation is based on a version with active twitch by Shadmehr and Wise,75 and a modification that can receive spike train as the input.61 Therefore, the muscle model converts α-motoneuron spikes into muscle force, depending on the muscle’s history of length and lengthening velocity. All 6 motoneuron pools are connected to the same type of muscle fiber that was capable of twitching at 30 ms time constant. It is possible, given more on-chip resources, to differentiate muscle fibers by diversifying the time constants of twitch.26
The force-length and force-velocity properties are introduced by multiplying a length-sensitive term (\( F_{L} \)) and a velocity-sensitive term (\( F_{V} \)) to the active muscle force (\( F_{A} \)), as is described by Zajac.90 In specific, the force-length term is given by:
The force-velocity term is given by:
Other details about the Hill-type model of muscle are provided in the Appendix.
Model of Proprioceptive Information Based on Motor Rotation
Muscle length and lengthening velocity are critical information in the biomimetic reflex: they are the source of proprioception that should be explicitly provided to muscle spindle, and they must be accessible by the muscle itself to engage force-length and force-velocity properties. Both the length of musculotendinous unit (\( L_{\text{mtu}} \)) and length of contractile element (\( L_{\text{ce}} \)) and are captured in the biomimetic reflex.
\( L_{\text{mtu}} \) is equivalent to the translational displacement of the cable, which is measured by a rotational transducer on the DC motor (Fig. 3b):
meaning that the instantaneous \( L_{\text{mtu}} \left( t \right) \) equals the length of the initial cable minus the cable length coiled on the winder. \( L_{\text{mtu}} \left( 0 \right) = 39.7\,{\text{cm}} \) denotes the initial length of the cable, as determined by the placement of the motor on anatomical socket; \( L_{\text{mtu}} \left( 0 \right) \) is also similar to the length of a typical FCU MTU (38.3 cm).24\( r = 0.25\,{\text{cm}} \) denotes the radius of the cable winder, which is chosen to allow for adequate travel on the cable (~ 1.57 cm for 360° of rotation) and maintain the ability of weight bearing (30 N at the fingertip).
\( L_{\text{ce}} \) is the key input to the muscle spindle model as given by:
When the finger is moving freely without contact of object, \( \Delta L_{\text{ce}} \) and \( \Delta L_{\text{mtu}} \) are both equivalent to the translational displacement on the cable, and therefore it is straightforward to acquire \( \Delta L_{\text{ce}} \left( t \right) \) from motor rotation. During contact, nevertheless, \( \Delta L_{\text{ce}} \) is simultaneously affected by the active muscle force A, passive damping b, the parallel elastic component \( K_{\text{pe}} \), and most importantly the deformation of the object. It is no longer trivial to acquire \( \Delta L_{\text{ce}} \). Our approximation of \( L_{\text{ce}} \) is described in the next section.
Approximation of Length of Contractile Element
When the fingers are in contact with an object, we ignore the effect of muscle damping due to relative slow lengthening of the muscle (Fig. 3b). According to the force-length curve,55 the active component in Hill-type muscle model is equivalent of a spring with adjustable stiffness \( K_{\alpha } \) (Fig. 3c). Therefore, the length of contractile element is given by:
The adjustable stiffness \( K_{\alpha } \) can be inferred from the force-length relationship as follows. We restrict the range to \( L_{\text{mtu}} \le L_{\text{mtu}}^{0} \), such that only positive \( K_{\alpha } \)was effective.
Therefore, the real length of \( L_{\text{ce}} \) can be obtained as:
Model of Muscle Spindle and Proprioceptive Feedback
Proprioceptive feedback was provided by implementing the well-known model56 for muscle spindle. The \( L_{\text{ce}} \) provided to the spindle model is normalized to the resting length \( L_{\text{MTU}}^{0} = 23.6\,\,{\text{cm}} \) for FCU.24 The model characterized how nuclear bag and nuclear chain fibers produce Group Ia and Group II afferents; and influences from gamma motoneurons were also modeled. In the closed-loop control, only Group Ia afferents were included in the feedback loop, because the connections from Group II are not as clearly defined in monosynaptic circuitry. See Appendix for details on the spindle model.
Real-Time Emulation of Biomimetic Reflex on Neuromorphic Hardware
The time-consuming components in the model-based controller are programmed on neuromorphic hardware, constructed under a customized VLSI architecture (implemented on Xilinx Spartan-6 FPGA). The neuromorphic design has been shown its advantage that allows large volume of spiking neurons to be evaluated in parallel without sacrificing speed.61 In addition, numerical floating-point calculations required by models are difficult to run at > 1 kHz on CPU. At least two Spartan-6 chips are required to enable an afferented muscle, one for motoneuron and muscle, the other for muscle spindles and their spiking afferents. The chips are connected by wires transmitting in raw spikes (Fig. 1c). These calculations are accelerated using combinatorial logics on FPGA. As a result, the biomimetic reflex managed to install a total of 768 motoneurons, 1 muscle spindle with 128 spiking afferents, and a Hill-type muscle model, all running with > 1 kHz refresh rate (Fig. 1c).
Experiments for Verification of Hand Functionality
Three categories of experiments were conducted to validate the functionality of reflex-enabled control of prosthetic hand:
-
1.
Basic functionality without human operation
-
2.
Human-operated performance without object interaction
-
3.
Human-operated performance with object interaction
Hand aperture was monitored using 240 fps slow-motion videos with a ruler in the scene. Details about the experiments are explained below in the Results. Signals were processed using MATLAB R2016b (MathWorks, Natick, MA). All statistical analyses were conducted using R (version 3.5.2).
Results
Experiment 1: Basic Functionality Without Human Operation
Hand Aperture is Linearly Controlled by Alpha Motor Command
We first tested the ability of the biomimetic prosthetic hand for maintenance of an opening aperture (\( H_{a} \)). The hand was fully extended (\( H_{a} = 100\% \)) at the beginning. Then a certain level of alpha command was sent to the biomimetic model, which closed the hand to a steady-state aperture. The relationship between \( H_{a} \) and alpha is shown in Fig. 4. Each data point was measured when the hand had stabilized at an aperture, then the hand was released to full extension for the next level of alpha command. When three fingers (thumb, index, middle) were pulled by the torque motor (Fig. 4a), the hand fully closed when the alpha motor command reached 8.4 mA. The linear relationship between \( H_{a} \) and alpha was significant (\( p < 0.05 \), \( R^{2} = 0.978 \)). When all five fingers were pulled (Fig. 4b), it required 11.5 mA alpha motor command to fully close the hand following a linear correlation (\( p < 0.05 \), \( R^{2} = 0.988 \)). In both situations the hand was not moving with an alpha command less than 5 mA, most likely due to the initial torque required for the motor to move. Figure 4c shows that the opening aperture \( H_{a} \) has a linear relationship with \( L_{\text{mtu}} \) (\( p < 0.05 \), \( R^{2} = 0.980 \)).
Experiment 2: Human-Operated Performance Without Object Interaction
Hand Closing at Different Velocities Using Real EMG
In Experiment 2, we continued to verify whether a subject could close the prosthetic hand at different velocities, i.e. varying the speed when driving the hand from the resting posture to full closure. A healthy subject (male, age 30) controlled the prosthetic hand by contraction of the flexor carpi ulnaris muscle, wrist flexion was translated into hand closing on the prosthetic hand. In all three parts of experiment 2, the healthy subject was flexing his wrist with his fingers relaxed (Fig. 5a, top panel).
EMG was filtered using non-linear Bayesian filter and scaled to 0–20 mA. The subject was instructed to fully close the hand with either low, medium or high speed. As can be seen from Fig. 5a, the subject was capable of closing the hand swiftly in 0.87 s (purple), moderately in 3.11 s (orange), and more slowly in 6.06 s (blue). Raw EMGs recorded from the FCU of the subject are also shown. Alpha motor commands filtered from raw EMG using nonlinear Bayesian filter are also shown. Note the abrupt changes in the alpha motor commands, which are featured characteristics enabled by the Bayesian estimation.7
Quasi-static Stiffness at Different Gestures
We verified whether the hand could establish various stiffness at a chosen gesture. This was measured by having the finger producing a pressing force at the fingertip, and then recording the change in force after a small displacement. Wrist flexion was translated to the flexion of index finger (Fig. 5b). In specific, the model-based controller received an alpha motor command such that the finger was pressing against a force gauge (Model ELK-20, Elecall Co., Ltd., Wenzhou, China). Subsequently, the force gauge was elevated by 5mm and the change of force (\( dF \)) was recorded. The ratio of \( dF/dH_{a} \) was calculated as the stiffness. We sampled three values of alpha motor command (6, 7, 8 mA) at 9 different apertures. As can be seen from Fig. 5b, the finger established a range of stiffness between 60 and 640 N/m, compared to 23 and 527 N/m as reported in human index finger.10 A comparison on the relative flexibility of stiffness, defined as \( \left( {K_{\text{high}} - K_{\text{low}} } \right)/K_{\text{low}} \), shows that the prosthetic hand achieved 46.6% of the flexibility in stiffness of a human index finger.
Weight Bearing Ability
The weight bearing ability of the prosthetic hand was tested. Wrist flexion was translated to whole hand closing against a rope loop around the 4 fingers except for the thumb (Fig. 5c). A video demonstrating how the hand picked up a 2kg weight is available in the Supplementary Materials. The fingers were pulled together by a ring rope. We measured the maximal force when the ring rope was pulled off the hand. As can be seen from Fig. 5c, the hand could maintain up to 44 N of force, compared to 95.6 N maximum force of precision-grip force and 400 N maximum force of power-grip reported in human.87 The linear correlation between weight and alpha motor command was significant (\( p < 0.01 \), \( R^{2} = 0.976 \)).
Experiment 3: Human-Operated Performance with Object Interaction
Contacting a Deformable Object
In Experiment 3, the index finger of the prosthetic hand was activated to press down a bendable bean. The finger was initially hovering at 1 cm above a bendable beam. Thereafter, a step alpha motor command was issued to the biomimetic model, leading to a series of events: change in motor torque (t1), change in muscle length (t2), contact of object (t3), end of deformation (t4), settlement of force (t5). An illustration of the five events can be seen in Fig. 6a. We issued a step signal of alpha (a = 6.5 mA) to the biomimetic model, and we measured the time of all five events given hovering distance of either 1 cm (Fig. 6b).
Due to limited bandwidth for data monitoring in hardware-in-the-loop applications, not all internal statuses were recorded. We simulated the contact of deformable beam in Simulink using the same model of reflex. Multi-scale information was available about spindle output, motoneuron spike raster, EMG, etc. A similar lineup of events (t1–t5) were identified from simulated data (Fig. 6c). The timing of events is shown in Table 2.
“Press-Without-Break” Performance in Amputee
We tested how an amputee performed with the prosthetic hand in a “press-without-break” task. Using the above setup, the monitor displayed a moving bar, whose height (D) represented the finger pressure (Fig. 7a). In each trial, the subject was requested to raise the moving bar to a green target zone (W) as quickly as possible. A trial was considered successful once the moving bar had stayed in the target zone for 1 s. Above the target zone there was a prohibited zone in red. The amputee was informed that entering the prohibited zone meant breaking the virtual object, which should be avoided at all times. We chose a breakage threshold (lower bound of prohibited zone) of 4.4 N. Visual feedback was constantly available to subjects.
Each trial was associated with an Index of Difficulty (ID), formulated as \( {\text{ID}} = { \log }_{2} \left( {2D/w} \right) \). A total of 6 different IDs were introduced in the experiment. Table 3 shows the target distances (D) and target width (W) corresponding to each ID.
We calculated the information throughput (TP) as a metric to quantify the performance,5 which reflects the speed-accuracy relationship commonly seen in human-machine interactions23:
where N = 6 is the number of IDs, and CT is the time for task completion in each trial.
The force and normalized EMG of a successful trial from an amputee (male, age 50, fore-arm amputation, consented as approved by the Ethics Committee of Human and Animal Experiments of the Med-X Research Institute of Shanghai Jiao Tong University) are shown in Fig. 7b; data of a successful trial using his contralateral hand (index finger, no prosthetic hand) are shown in Fig. 7c. The information throughput was 1.561 bit/s using prosthetic hand, compared to 3.160 bit/s using the contralateral hand (Fig. 7d).
Discussion
We have designed a model-based controller that enables biomimetic reflex on cable-driven prosthetic hands. The biomimetic reflex comprises of spiking neurons, skeletal muscles, muscle spindles, and monosynaptic spinal circuitry. The model-based controller interfaces between surface EMG and an electric motor that pulls the cables. We have also shown how to reconstruct proprioceptive information based on readouts from the motor. The closed-loop behavior of the reflex-enabled prosthetic hand has been tested in six experiments. Results suggest that the hand could achieve expected behaviors, including moving with different destinations or velocities, pressing against a bendable beam, grasping paper or steel cups, etc. This study verified the feasibility of enabling biomimetic reflex in the real-time control of cable-driven prosthetic hand.
Previous work has demonstrated the capability of simulated monosynaptic reflex for replication of human-like movements in health and disease,51,61 which comprises of spiking neurons, muscle spindles, synapses, skeletal muscles, and monosynaptic spinal circuitry. The “virtual reflex” provides the foundation for model-based control in this study, but it is repurposed for translation of motor command into expected muscle force and hand stiffness. We adhere to a criterion that all parameters with biomimetic reflex should be traceable to mammalian neurophysiology. In future applications, however, parameters can deviate from normal physiology if proven more advantageous in motor performance. One advantage of neuromorphic hardware is that it allows for flexible trial-and-error against various model configurations. Models in the form of differential equations (e.g. spindle) are computationally expensive and generally difficult to run in real-time,2,62 making it difficult to be integrated in the closed-loop control of prosthetic hand. Acceleration of model evaluation is another advantage from neuromorphic hardware.
The model-based controller aims to provide an informational context compatible with the amputee. The model attempts that the alpha motor commands (inferred from EMG) operate through a machinery (motoneurons, muscles, spindles, etc.) as if there were no amputation. Both the healthy subject and amputee were given 10 minutes to familiarize with the prosthetic control, during which time the subjects may look at the prosthetic hand and the object, and adjust their commands if they saw any failure. In order to clarify the contribution of model-based control separate from the visual feedback, it might be necessary to perturb the vision in future experiments.
This study has by no means exhausted the voluminous possibilities of non-biomimetic approaches for prosthetic control (15,21,33,35,36,69,73,77,88 for an incomplete list). In fact, most of existing algorithms focused on the feedforward components, or introducing more loops to allow for complex dynamics in the closed-loop behavior. This study adopted the most parsimonious architecture for closed-loop control,47 and reduced the question to whether computationally intensive—but more biomimetic—models could enhance the performance. One explanation for the observed improvements in performance with biomimetic models is that they maintain a neurocompatibility between the device and amputee.49 Alpha motor commands from the peripheral nervous system are supposed to activate a machinery with muscle properties, spindle feedback, etc. A controller with neuromorphic models, therefore, provided such a virtual environment that is compatible with the original intention of subjects before amputation. It is also noteworthy that all neuromorphic models implemented in this study are, in essence, mathematical formulations of high-order nonlinear systems.48,56,61 Biomimetic models do not contradict the linear ones in principle of control, but rather they showed where the complexity should reside. In particular, impedance control measures position and generates force, which shares the same input-output structure as our model-based controller, and therefore impedance control strategies are reasonable contenders when benchmarking the model-based controller.
Only one afferented muscle was enabled on the index finger for its flexion, leaving the extension passively stretched by a spring. This setup suffices in engineering to produce whole-hand grasping when all five fingers are all configured similarly. But it lacks the biological realism that both flexion and extension should be separately actuated by a pair of antagonistic muscles. In practice, the flexion-only setup requires a constant EMG activity whenever the amputee wants to exert a force, meaning it also loses the capability of resetting the resting posture. Thus, it might be crucial to enable two autonomous muscles for each joint, such that the controller both achieves higher level of biomimicry and solves the practical problem of posture resetting. The model-driven strategy of control is not limited to cable-driven prosthetic hands. For non-cable-driven hands, the proprioceptive information can be reconstructed in a similar way from the angle transducers. It is noteworthy, however, that most non-cable-driven designs use non-backdrivable transmissions (e.g. gears or link-mechanisms), and therefore the controller would not be directly applicable without addressing the back-drivability beforehand.
Abbreviations
- \( L_{\text{mtu}} \) :
-
Length of musculotendinous unit
- \( L_{\text{ce}} \) :
-
Length of contractile element
- \( H_{\text{a}} \) :
-
Opening aperture of prosthetic hand
- \( \theta_{\text{motor}} \) :
-
Rotational angle of DC motor
- \( K_{\text{se}} \) :
-
The series elastic component
- \( K_{\text{pe}} \) :
-
The parallel elastic component
- \( K_{\alpha } \) :
-
The active component in modified Hill muscle model
- \( F_{L} \) :
-
Length-sensitive term, muscle model
- \( F_{V} \) :
-
Velocity-sensitive term, muscle model
- \( F_{A} \) :
-
Active muscle force
- \( V_{\text{m}} \) :
-
Maximum contraction velocity of muscle
- \( L_{t} \) :
-
Length of passive element
- \( F_{0} \) :
-
Peak active force
- \( F_{1} \) :
-
Active force at maximum contraction velocity of muscle
- FCU:
-
Flexor carpi ulnaris
- EPSC:
-
Excitatory post-synaptic current
- VLSI:
-
Very large scale integrated circuit
- ID:
-
Index of difficulty
- CT:
-
Completion time
- TP:
-
Throughput
References
Alstermark, B., T. Isa, L.-G. Pettersson, and S. Sasaki. The C3–C4 propriospinal system in the cat and monkey: a spinal pre-motoneuronal centre for voluntary motor control. Acta Physiol. 189:123–140, 2007.
Bäuml, B., and G. Hirzinger. When hard realtime matters: software for complex mechatronic systems. Robot. Auton. Syst. 56:5–13, 2008.
Belyea, A. T., K. B. Englehart, and E. J. Scheme. A proportional control scheme for high density force myography. J. Neural Eng. 15:046029, 2018.
Benatti, S., B. Milosevic, E. Farella, E. Gruppioni, and L. Benini. A prosthetic hand body area controller based on efficient pattern recognition control strategies. Sensors 17:869, 2017.
Bertucco, M., and T. D. Sanger. Speed-Accuracy testing on the Apple iPad® provides a quantitative test of upper extremity motor performance in children with dystonia. J. Child Neurol. 29:1460–1466, 2014.
Biddiss, E. A., and T. T. Chau. Upper limb prosthesis use and abandonment: a survey of the last 25 years. Prosthet. Orthot. Int. 31:236–257, 2007.
Borish, C. N., A. Feinman, M. Bertucco, N. G. Ramsy, and T. D. Sanger. Comparison of speed-accuracy tradeoff between linear and nonlinear filtering algorithms for myocontrol. J. Neurophysiol. 119:2030–2035, 2018.
Buchthal, F., and H. Schmalbruch. Motor unit of mammalian muscle. Physiol. Rev. 60:90–142, 1980.
Carrozza, M. C., C. Suppo, F. Sebastiani, B. Massa, F. Vecchi, R. Lazzarini, M. R. Cutkosky, and P. Dario. The SPRING hand: development of a self-adaptive prosthesis for restoring natural grasping. Auton. Robots 16:125–141, 2004.
Carter, R. R., P. E. Crago, and M. W. Keith. Stiffness regulation by reflex action in the normal human hand. J. Neurophysiol. 64:105–118, 1990.
Catalano, M. G., G. Grioli, E. Farnioli, A. Serio, C. Piazza, and A. Bicchi. Adaptive synergies for the design and control of the Pisa/IIT SoftHand. Int. J. Robot. Res. 33:768–782, 2014.
Chai, G., X. Sui, S. Li, L. He, and N. Lan. Characterization of evoked tactile sensation in forearm amputees with transcutaneous electrical nerve stimulation. J. Neural Eng. 12:066002, 2015.
Cipriani, C., C. Antfolk, M. Controzzi, G. Lundborg, B. Rosen, M. C. Carrozza, and F. Sebelius. Online myoelectric control of a dexterous hand prosthesis by transradial amputees. IEEE Trans. Neural Syst. Rehabil. Eng. 19:260–270, 2011.
Cipriani, C., M. Controzzi, and M. C. Carrozza. The SmartHand transradial prosthesis. J. NeuroEng. Rehabil. 8:1–14, 2011.
Cipriani, C., F. Zaccone, S. Micera, and M. C. Carrozza. On the shared control of an EMG-controlled prosthetic hand: analysis of user-prosthesis interaction. IEEE Trans. Robot. 24:170–184, 2008.
Clancy, E. A., S. Bouchard, and D. Rancourt. Estimation and application of EMG amplitude during dynamic contractions. IEEE Eng. Med. Biol. Mag. 20:47–54, 2001.
Controzzi, M., F. Clemente, D. Barone, A. Ghionzoli, and C. Cipriani. The SSSA-MyHand: a dexterous lightweight myoelectric hand prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 25:459–468, 2017.
D’Alessio, T., and S. Conforto. Extraction of the envelope from surface EMG signals. IEEE Eng. Med. Biol. Mag. 20:55–61, 2001.
D’Anna, E., G. Valle, A. Mazzoni, I. Strauss, F. Iberite, J. Patton, F. M. Petrini, S. Raspopovic, G. Granata, R. Di Iorio, M. Controzzi, C. Cipriani, T. Stieglitz, P. M. Rossini, and S. Micera. A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback. Sci. Robot. 4:eaau8892, 2019.
Deimel, R., and O. Brock. A novel type of compliant and underactuated robotic hand for dexterous grasping. Int. J. Robot. Res. 35:161–185, 2016.
Deng, H., G. Zhong, X. Li, and W. Nie. Slippage and deformation preventive control of bionic prosthetic hands. IEEE/ASME Trans. Mechatron. 22:888–897, 2017.
Englehart, K., and B. Hudgins. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 50:848–854, 2003.
Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol. 47:381–391, 1954.
Fridén, J., R. M. Lovering, and R. L. Lieber. Fiber length variability within the flexor carpi ulnaris and flexor carpi radialis muscles: implications for surgical tendon transfer. J. Hand Surg. 29:909–914, 2004.
Fu, Q., F. Shao, and M. Santello. Inter-limb transfer of grasp force perception with closed-loop hand prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 27:927–936, 2019.
Fuglevand, A. J., D. A. Winter, and A. E. Patla. Models of recruitment and rate coding organization in motor-unit pools. J. Neurophysiol. 70:2470–2488, 1993.
Gailey, A. S., S. B. Godfrey, R. E. Breighner, K. L. Andrews, K. D. Zhao, A. Bicchi, and M. Santello. Grasp performance of a soft synergy-based prosthetic hand: a pilot study. IEEE Trans. Neural Syst. Rehabil. Eng. 25:2407–2417, 2017.
Glowatzki, E., and P. A. Fuchs. Transmitter release at the hair cell ribbon synapse. Nat. Neurosci. 5:147–154, 2002.
Godfrey, S. B., M. Rossi, C. Piazza, M. G. Catalano, M. Bianchi, G. Grioli, K. D. Zhao, and A. Bicchi. SoftHand at the CYBATHLON: a user’s experience. J. NeuroEng. Rehabil. 14:124, 2017.
Godfrey, S. B., K. D. Zhao, A. Theuer, M. G. Catalano, M. Bianchi, R. Breighner, D. Bhaskaran, R. Lennon, G. Grioli, M. Santello, A. Bicchi, and K. Andrews. The SoftHand pro: functional evaluation of a novel, flexible, and robust myoelectric prosthesis. PLoS ONE 13:e0205653, 2018.
Granit, R. The functional role of the muscle spindles—facts and hypotheses. Brain 98:531–556, 1975.
Graziano, M. S. A. Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. Proc. Natl. Acad. Sci. 96:10418–10421, 1999.
Hahne, J. M., F. Biebmann, N. Jiang, H. Rehbaum, D. Farina, F. C. Meinecke, K.-R. Muller, and L. C. Parra. Linear and nonlinear regression techniques for simultaneous and proportional myoelectric control. IEEE Trans. Neural Syst. Rehabil. Eng. 22:269–279, 2014.
Hahne, J. M., S. Dahne, H.-J. Hwang, K.-R. Muller, and L. C. Parra. Concurrent adaptation of human and machine improves simultaneous and proportional myoelectric control. IEEE Trans. Neural Syst. Rehabil. Eng. 23:618–627, 2015.
Hahne, J. M., M. Markovic, and D. Farina. User adaptation in myoelectric man-machine interfaces. Sci. Rep. 7:1–10, 2017.
Hahne, J. M., M. A. Schweisfurth, M. Koppe, and D. Farina. Simultaneous control of multiple functions of bionic hand prostheses: performance and robustness in end users. Sci. Robot. 3:eaat3630, 2018.
Henneman, E. Relation between size of neurons and their susceptibility to discharge. Science 126:1345–1347, 1957.
Hodgkin, A. L., and A. F. Huxley. A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117:500–544, 1952.
Hogan, N. A review of the methods of processing EMG for use as a proportional control signal. Biomed. Eng. 11:81–86, 1976.
Hwang, H.-J., J. M. Hahne, and K.-R. Müller. Real-time robustness evaluation of regression based myoelectric control against arm position change and donning/doffing. PLoS ONE 12:e0186318, 2017.
InMoov | open-source 3D printed life-size robotat. http://inmoov.fr/.
Izhikevich, E. M. Simple model of spiking neurons. IEEE Trans. Neural Netw. 14:1569–1572, 2003.
Jiang, N., K. B. Englehart, and P. A. Parker. Extracting simultaneous and proportional neural control information for multiple-DOF prostheses from the surface electromyographic signal. IEEE Trans. Biomed. Eng. 56:1070–1080, 2009.
Jones, L. A., and S. J. Lederman. Human Hand Function. New York: Oxford University Press, p. 290, 2006.
Kyberd, P. J. Assessment of functionality of multifunction prosthetic hands. J. Prosthet. Orthot. 29:103–111, 2017.
Kyberd, P. J., O. E. Holland, P. H. Chappell, S. Smith, R. Tregidgo, P. J. Bagwell, and M. Snaith. MARCUS: a two degree of freedom hand prosthesis with hierarchical grip control. IEEE Trans. Rehabil. Eng. 3:70–76, 1995.
Lan, N., P. E. Crago, and H. J. Chizeck. Feedback control methods for task regulation by electrical stimulation of muscles. IEEE Trans. Biomed. Eng. 38:1213–1223, 1991.
Lan, N., and X. He. Fusimotor control of spindle sensitivity regulates central and peripheral coding of joint angles. Front. Comput. Neurosci. 6:66, 2012.
Lan, N., C. M. Niu, M. Hao, C.-H. Chou, and C. Dia. Achieving neural compatibility with human sensorimotor control in prosthetic and therapeutic devices. IEEE Trans. Med. Robot. Bionics 1:122–134, 2019.
Lenzi, T., J. Lipsey, and J. W. Sensinger. The RIC arm—a small anthropomorphic transhumeral prosthesis. IEEE/ASME Trans. Mechatron. 21:2660–2671, 2016.
Li, S., C. Zhuang, M. Hao, X. He, J. C. Marquez, C. M. Niu, and N. Lan. Coordinated alpha and gamma control of muscles and spindles in movement and posture. Front. Comput. Neurosci. 9:122, 2015.
Liddell, E. G. T., and C. Sherrington. Reflexes in response to stretch (myotatic reflexes). Proc. R. Soci. B: Biol. Sci. 96:212–242, 1924.
Luchetti, M., A. G. Cutti, G. Verni, R. Sacchetti, and N. Rossi. Impact of Michelangelo prosthetic hand: findings from a crossover longitudinal study. J. Rehabil. Res. Dev. 52:605–618, 2015.
Matthews, P. B. C. Muscle spindles and their motor control. Physiol. Rev. 44:219–288, 1964.
Mcmahon, T. A. Muscles, Reflexes, and Locomotion. Princeton: Princeton University Press, 1984.
Mileusnic, M. P., I. E. Brown, N. Lan, and G. E. Loeb. Mathematical models of proprioceptors. I. Control and transduction in the muscle spindle. J. Neurophysiol. 96:1772–1788, 2006.
Muceli, S., and D. Farina. Simultaneous and proportional estimation of hand kinematics from EMG during mirrored movements at multiple degrees-of-freedom. IEEE Trans. Neural Syst. Rehabil. Eng. 20:371–378, 2012.
Muceli, S., N. Jiang, and D. Farina. Extracting signals robust to electrode number and shift for online simultaneous and proportional myoelectric control by factorization algorithms. IEEE Trans. Neural Syst. Rehabil. Eng. 22:623–633, 2014.
Napier, J. R. The prehensile movements of the human hand. J. Bone Jt. Surg. Br. 38-B:902–913, 1956.
Nichols, T. R., and J. C. Houk. Improvement in linearity and regulation of stiffness that results from actions of stretch reflex. J. Neurophysiol. 39:119–142, 1976.
Niu, C. M., K. Jalaleddini, W. J. Sohn, J. Rocamora, T. D. Sanger, and F. J. Valero-Cuevas. Neuromorphic meets neuromechanics, part I: the methodology and implementation. J. Neural Eng. 14:025001, 2017.
Niu, C. M., S. K. Nandyala, and T. D. Sanger. Emulated muscle spindle and spiking afferents validates VLSI neuromorphic hardware as a testbed for sensorimotor function and disease. Front. Comput. Neurosci. 8:141, 2014.
Nordin, A. D., W. Z. Rymer, A. A. Biewener, A. B. Schwartz, D. Chen, and F. B. Horak. Biomechanics and neural control of movement, 20 years later: what have we learned and what has changed? J. NeuroEng. Rehabil. 14:91, 2017.
Open Bionics. Open Bionics Labs, 2014. https://openbionicslabs.com.
Ortiz-Catalan, M., E. Mastinu, P. Sassu, O. Aszmann, and R. Brånemark. Self-contained neuromusculoskeletal arm prostheses. N. Engl. J. Med. 382:1732–1738, 2020.
Østlie, K., I. M. Lesjø, R. J. Franklin, B. Garfelt, O. H. Skjeldal, and P. Magnus. Prosthesis rejection in acquired major upper-limb amputees: a population-based survey. Disabil. Rehabil.: Assist. Technol. 7:294–303, 2012.
Pierrot-Deseilligny, E., and D. Burke. The Circuitry of the Human Spinal Cord: Its Role in Motor Control and Movement Disorders. Cambridge: Cambridge University Press, 2005.
Prochazka, A., F. Clarac, G. E. Loeb, J. C. Rothwell, and J. R. Wolpaw. What do reflex and voluntary mean? Modern views on an ancient debate. Exp. Brain Res. 130:417–432, 2000.
Ray, Z., and E. D. Engeberg. Human-Inspired reflex to autonomously prevent slip of grasped objects rotated with a prosthetic hand. J. Healthc. Eng. 1–11:2018, 2018.
Sanger, T. D. Bayesian filtering of myoelectric signals. J. Neurophysiol. 97:1839–1845, 2007.
Saunders, J. A., and D. C. Knill. Visual feedback control of hand movements. J. Neurosci. 24:3223–3234, 2004.
Schmalfuss, L., J. Hahne, D. Farina, M. Hewitt, A. Kogut, W. Doneit, M. Reischl, R. Rupp, and D. Liebetanz. A hybrid auricular control system: direct, simultaneous, and proportional myoelectric control of two degrees of freedom in prosthetic hands. J. Neural Eng. 15:056028, 2018.
Segil, J. L., S. A. Huddle, and R. F. F. F. Weir. Functional assessment of a myoelectric postural controller and multi-functional prosthetic hand by persons with trans-radial limb loss. IEEE Trans. Neural Syst. Rehabil. Eng. 25:618–627, 2017.
Segil, J. L., and R. F. F. F. Weir. Design and validation of a morphing myoelectric hand posture controller based on principal component analysis of human grasping. IEEE Trans. Neural Syst. Rehabil. Eng. 22:249–257, 2014.
Shadmehr, R., and S. P. Wise. A mathematical muscle model. In: Supplementary Documents for “Computational Neurobiology of Reaching and Pointing”. Cambridge: MIT Press, 2005.
Sherrington, C. S. The Integrative Action of the Nervous System. New Haven: Yale University Press, 1911.
Simon, A. M., L. J. Hargrove, B. A. Lock, and T. A. Kuiken. The target achievement control test: evaluating real-time myoelectric pattern recognition control of a multifunctional upper-limb prosthesis. J. Rehabil. Res. Dev. 48:619–627, 2011.
Sivakumar, K., and C. Priyanka. Grasping objects using shadow dexterous hand with tactile feedback. Int. J. Innov. Res. Sci. Eng. Technol. 4:3108–3116, 2015.
Smith, L. H., T. A. Kuiken, and L. J. Hargrove. Real-time simultaneous and proportional myoelectric control using intramuscular EMG. J. Neural Eng. 11:066013, 2014.
Smith, L. H., T. A. Kuiken, and L. J. Hargrove. Myoelectric control system and task-specific characteristics affect voluntary use of simultaneous control. IEEE Trans. Neural Syst. Rehabil. Eng. 24:109–116, 2016.
Smith, L. H., T. A. Kuiken, and L. J. Hargrove. Evaluation of linear regression simultaneous myoelectric control using intramuscular EMG. IEEE Trans. Biomed. Eng. 63:737–746, 2016.
Taylor, C. L., and R. J. Schwarz. The anatomy and mechanics of the human hand. Artif. Limbs 2:22–35, 1955.
Tian, M., Y. Xiao, X. Wang, J. Chen, and W. Zhao. Design and experimental research of pneumatic soft humanoid robot hand. In: Robot Intelligence Technology and Applications 4, edited by J.-H. Kim, F. Karray, J. Jo, P. Sincak, and H. Myung. Cham: Springer, 2017, pp. 469–478.
Townsend, W. The BarrettHand grasper—programmably flexible part handling and assembly. Ind. Robot Int. J. 2000. https://doi.org/10.1108/01439910010371597.
Valero-Cuevas, F. J. Fundamentals of neuromechanics. Biosyst. Biorobot. 8:194, 2016.
Vidovic, M. M.-C., H.-J. Hwang, S. Amsuss, J. M. Hahne, D. Farina, and K.-R. Muller. Improving the robustness of myoelectric pattern recognition for upper limb prostheses by covariate shift adaptation. IEEE Trans. Neural Syst. Rehabil. Eng. 24:961–970, 2016.
Weir, R. F. Design of artificial arms and hands for prosthetic applications. In: Standard Handbook of Biomedical Engineering & Design, edited by M. Kutz. New York: McGraw-Hill, 2002.
Wurth, S. M., and L. J. Hargrove. A real-time comparison between direct control, sequential pattern recognition control and simultaneous pattern recognition control using a Fitts’ law style assessment procedure. J. NeuroEng. Rehabil. 11:91, 2014.
Xu, K., H. Liu, Y. Du, and X. Zhu. Design of an underactuated anthropomorphic hand with mechanically implemented postural synergies. Adv. Robot. 28:1459–1474, 2014.
Zajac, F. E. Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. Crit. Rev. Biomed. Eng. 17:359–411, 1989.
Zollo, L., G. Di Pino, A. L. Ciancio, F. Ranieri, F. Cordella, C. Gentile, E. Noce, R. A. Romeo, A. Dellacasa Bellingegni, G. Vadalà, S. Miccinilli, A. Mioli, L. Diaz-Balzani, M. Bravi, K.-P. Hoffmann, A. Schneider, L. Denaro, A. Davalli, E. Gruppioni, R. Sacchetti, S. Castellano, V. Di Lazzaro, S. Sterzi, V. Denaro, and E. Guglielmelli. Restoring tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands. Sci. Robot. 4:eaau9924, 2019.
Acknowledgments
This work was supported in part by a key Grant from the Natural Science Foundation of China (No. 81630050), and a Grant of the National Key R&D Program of China by the Ministry of Science and Technology of China (No. 2017YFA0701103).
Conflict of interest
The authors declare that they have no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Additional information
Associate Editor Xiaoxiang Zheng oversaw the review of this article.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 2 (MP4 1923 kb)
Supplementary material 3 (MP4 1391 kb)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Niu, C.M., Luo, Q., Chou, Ch. et al. Neuromorphic Model of Reflex for Realtime Human-Like Compliant Control of Prosthetic Hand. Ann Biomed Eng 49, 673–688 (2021). https://doi.org/10.1007/s10439-020-02596-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10439-020-02596-9