Introduction

Movement augmentation technology promises to increase a user’s effective mechanical degrees-of-freedom (DoFs). For instance, a surgeon could use a supernumerary robotic limb to perform laparoscopy without an assistant1, or a user-interface could enable a person to control their phone while their hands carry heavy luggage. However, an increase in DoFs comes with a corresponding need to control them. For practical DoF augmentation to be achieved it is therefore necessary that (i) the user is able to control the extra DoFs with a functional level of accuracy; and (ii) this control has minimal interference on existing natural motion behaviour. While movement augmentation has been demonstrated for a number of different applications2, the limits and mechanisms for its practical application are still unclear.

A number of different control schemes have been developed to support human DoF augmentation3. These typically make use of autonomous augmentation4,5, in which robot DoFs are controlled without the user’s volitional control, or augmentation by transfer6,7, where body DoFs not relevant to the task are substituted to control the extra DoFs. While such approaches can enable task specific DoF augmentation, their application is limited. Here, autonomous augmentation requires that the robot can predict the operator’s desired actions (which can lead to poor functional accuracy with respect to the user’s intent). Meanwhile, augmentation by transfer impacts natural motion unless it is restricted to use cases that do not use the substituted body DoFs.

Augmentation by extension, which uses body signals that are not involved in motion, could enable DoF augmentation that is application independent3. While invasive methods for simultaneous brain computer interfaces (BCI) and natural movement control have been investigated8,9 and could potentially be used as augmentation by extension schemes, non-invasive approaches such as electroencephalogram (EEG)10 or motor unit recording from high-density surface electromyography (EMG)11,12,13 can be more accessible and safer.

An alternative possible non-invasive augmentation by extension approach is to use the redundancy naturally present within the musculoskeletal system through the task-intrinsic muscular null-space14. The human body possesses more muscles than effective mechanical DoFs such that there is redundancy if multiple muscle activation patterns can be used to produce the same task-space force. This redundancy could potentially provide natural and measurable signals for human movement augmentation. Here, it has been shown that after remapping muscle activation to new muscle synergies, humans can (with sufficient exploration time) adapt their muscle activation behaviour in response to either simple remappings requiring combinations of existing muscle synergies, or the more challenging case requiring completely new synergies15. Moreover, it has been found that muscle activation patterns that fall into the muscle-to-force ‘null-space’ during isometric movement can be used as an additional control signal that is regulated simultaneously to the generation of isometric force16. These results suggest the potential for using muscle redundancy as a source of command signals for extra DoFs. However, while it has been shown that simultaneous null-space and reaching actions can be performed when the redundancy is from the muscles of the entire limb to the end-effector forces16, it is difficult to evaluate if natural body behaviour has been altered due to difficulty measuring all of the joint angles and/or torques involved across the entire limb and it is unclear if these results are only possible with the large levels of redundancy present across the entire upper limb.

In this study we focused on a single joint, as it could easily be restrained and its alterations could be monitored, and we investigated whether the activities of muscles acting on this joint could be implemented to control DoF augmentation. For this purpose, the wrist joint, with its 2 revolute DoFs, was chosen as it also possesses a muscle-to-force null space that has a low enough dimension such that it can be directly provided to the participant as visual feedback. Moreover, its enrollment during impedance modulation when interacting with the environment17, suggests the potential for skillful control of the wrist joint muscle-to-force null-space. Using EMG signals recorded from 4 wrist muscles (ECRL, ECU, FCR, FCU) we conducted an experiment with 2 sessions considering if: (i) humans can alter their natural co-contraction behaviour to produce atypical muscle activation patterns that do not produce force; and (ii) they can simultaneously vary both task-oriented forces and activity that is in the null-space of their force production. This used an incremental training paradigm to set targets for the modulation of muscle activation patterns in order to optimize the participants’ learning process18.

Methods

Participants

The experiment was approved by the Research Governance and Integrity Team at Imperial College London (Reference: 21IC6935). It was performed in accordance with all relevant guidelines and regulations in accordance with the Declaration of Helsinki. The experiment was carried out by 10 participants (5 female, 5 male) without known sensorimotor impairment aged 24–36. All participants partook in both experiment sessions. 1 participant’s data from the concurrent target reaching session was removed due to a computer crash during the session. Each participant had no prior experience with the experimental paradigm and gave their informed consent before participating. All participants self-reported to be right-handed and had normal or corrected to normal vision.

Experimental setup

The experiment was conducted using a fixed handle (Fig. 1a,19) attached to a 6 DoF force torque sensor (Nano SI-25, ATI Industrial Automation). The seated participants had their right arm fixed to the interface at the wrist, forearm and elbow with their forearm and wrist placed with \(0^\circ\) pronation–supination such that flexion–extension would lead to left/right motion and radial-ulnar deviation would lead to up/down motion. Surface electromyography (EMG) was recorded using the g.GAMMASYS system (g.tec) from 4 wrist muscles: flexor carpi radialis (FCR); flexor carpi ulnaris (FCU); extensor carpi radialis longus (ECRL); extensor carpi ulnaris (ECU) as well as the interossei muscles to detect finger movement. These 4 wrist muscles are primarily responsible for 2 mechanical DoFs (flexion–extension and radial-ulnar deviation). The recorded muscles were located manually at the suggested sites described in20 together with palpation. During electrode placement, the effect of interference from finger movement was attempted to be controlled for by asking participants to move their fingers while EMG activation was observed. Furthermore, participants were asked to move their wrist in extension, flexion, radial and ulnar deviation directions to confirm the detection of EMG signals of the corresponding recording sites. The interface and EMG recording was operated at 1000 Hz, where all EMG signals used in calibration and the main experiment were first high pass filtered with a cut-off frequency of 20 Hz, then rectified and finally low-pass filtered with a 5 Hz cut-off frequency (all second-order Butterworth filters).

Figure 1
figure 1

Experiment setup and design. (a) Participants were attached to an ergonomic handle mounted to a force torque sensor and were required to reach for targets using the recorded force signal and/or surface EMG recordings. (b) The experimental protocol for the 2 experiment sessions, where participants completed both sessions in a randomised order. A visualisation for each block is shown in (c–f). Depending on the block, participants controlled the position (indicated by a purple cross on the yellow cursor) and size (d)-(f) of the cursor to match different targets (shown in blue). In the NSTR (e), the cursor position was controlled by activation in the EMG-to-force null-space, where natural co-contraction was mapped to negative X-axis motion. In all other blocks force control was used. Possible target locations are shown in the Force/EMG calibration (c) and null-space target reaching blocks (e).

Full size image
Figure 2
figure 2

Performance during null-space target reaching. (a) Number of participants that attempted the target location. 6 out of 10 participants met the criteria to move past the first target. (b) Distribution of successful trials for each participant. A successful trial is shown by a coloured bar, where each colour represents a different target location (with colours matching those shown in (c)). (c) Overlaid cursor trajectories from the participant that reached the most targets (Subject 6) during the exploration and target reaching tasks. The black empty circles indicate the target position. Trajectories from failed and successful trials in the target reaching task are shown in grey and coloured lines, respectively. Note that trajectories are only shown up to the time instant at which the target was reached in the successful trial cases. (d) Histogram showing the angles reached for each target location. Angular coordinate show the reached angles from exploration (black: reached angles with the same constraints as in the target reaching task; grey: reached angles that violated the force magnitude constraint) or target reaching task (in colour). Width shows the frequency that the angles were reached. The angle range was larger than the distance between targets, suggesting a potential to move to reach more angles. Colour representation for each target: Red (Target 1); Orange (Target 2); Green (Target 3); Cyan (Target 4); Blue (Target 5); Purple (Target 6).

Full size image

Experiment design

Due to non-negative muscle activation constraint, for an n-DoF joint, at least n + 1 muscles would be required so that force/torque can be produced in every possible direction. However mechanically, any single desired force in 2 dimensions can be produced with only 2 muscles being simultaneously active (such that they are pulling), corresponding to the two muscles whose positive span the required force lies within. This means that a 1-DoF joint torque actuated by 2 muscles acting in opposing directions has a null-space as there are infinite solutions to any desired torque21 because of co-activation. Similarly, in our 2-DoF wrist joint actuated by 4 measured muscles, for any desired task-related force there would still be 2 other muscles that could possibly be actuated (with a corresponding co-activation compensation) by exploiting the redundancy with respect to the desired task-related force.

To investigate the ability of participants to exploit the redundancy present within their wrist system, the vector of 4 recorded EMG signals was therefore (at each time step across the experiment) projected onto a 2-dimensional task-space component, defined to be the set of all forces produced by wrist flexion–extension and radial-ulnar deviation (computed using the method of22) and a 2-dimensional null-space component, defined using the set of all EMG activations that did not result in force/torque (as in16). The experiment was conducted across 2 different cursor reaching sessions: null-space target reaching (NSTR) and concurrent target reaching (CTR), which are defined below. During the NSTR session, the participants’ ability to control only the null-space component of their EMG activation pattern without producing force was evaluated. Here, participants were required to produce a null-space activation of a fixed magnitude in different locations of the null-space, which would require them to alter their EMG activation patterns so as to deviate from their natural co-contraction behaviour without producing force. The non-negativity constraint acts to limit the possible set of EMG activities such that this set is a cone within the “null-space” (with dimension equal to that of the “null-space”) rather than the entire space. In the NSTR session, the cone for each participant was computed beforehand (see Supplementary Fig. S3) and from this we confirm that the requested target locations were always within the null-space cone and therefore our participants could theoretically (given only the mechanical aspects of the system) have been able to reach for these locations. In contrast in the CTR session, the participants ability to produce null-space activity concurrently to force/torque activation was evaluated by requiring participants to use force control to move a cursor to virtually reach for targets in different locations of the task-space while generating the correct amount of null-space activation using their natural co-contraction behaviour.

The experiment took place over 2 days with no more than 1 day of break in between. Participants completed 1 session each day and were randomly (with equal probability) assigned to start with either the NSTR or the CTR session. Figure 1b depicts the experiment protocol for each session. Here, each (approximately 90 min session) consisted of the same series of experimental blocks: an initial force/EMG calibration, a null-space calibration, a main experiment block and a final post session force/EMG calibration to confirm that there was no changes induced by the main block. The sessions therefore only differed in the different main experiment blocks.

Figure 3
figure 3

Movement characteristics during NSTR. (a) The range of different angles reached for the exploration, null-space reaching and across the entire session. (b–d) Motion characteristics of the null-space movement. Each participant is encoded with a certain colour. Each coloured bar represents the performance of a single participant within the trials of a given target, with the middle being the median and the length being the interquartile range. Black horizontal bars over the coloured bars depict the median performance of all participants for the same target. (b) Motion efficiency was computed as the ratio of the Euclidean distance from the cursor’s initial position to the target and the sampled motion’s path length. Most participants show high efficiency suggesting relatively direct motion to the target. (c) The time to success is relatively constant across all the targets. (d) Number of angle corrections from the initial direction (for successful trials this stops when the target is reached) is also relatively constant. (e) Task-space force magnitude when the cursor was at the correct target position for each trial for an individual participant. Saturated colours depict successful trials in the success condition. The transparent colours illustrate the force produced when the cursor is at the target for unsuccessful trials. Each colour is coded for an individual target. Note that the force is only shown until the threshold of 5% MVC. Force magnitude does not increase with trial or target, indicating that the participants were not using higher force levels to compensate for the null-space movement.

Full size image
Figure 4
figure 4

Performance during concurrent target reaching (Fig. 1f). (a) An illustration for the labels for targets in different locations. Note that for visual convenience the target size is not the same as was used in the session. (b) Success rate of different target locations and target sizes. The black horizontal lines indicate the median success rate. (c,d) Overlaid trajectories in the task-space for Subjects 5(c) and 3(d). Different colours depict the trajectories for different targets. (e) Amount of null-space movement in different target positions and sizes. Solid lines and the shades denote the mean and the standard deviation respectively of null-space movement by time. Black horizontal solid lines mark the target size and the black horizontal dashed lines denote the tolerance range for null-space movement. Each colour denotes 1 participant. (f) Target reaching time in the task- and null-space for different target sizes. (g) Percentage of null-space movement (median ± median absolute deviation) during Phase 1 (light colours) and Phase 2 (dark colours) for small (blue) and large (brown) targets. Negative values depict a reduction of the magnitude of null-space movement which can be due to overshooting in the null-space during Phase 1.

Full size image

Tasks

Force/EMG calibration

The participant specific maximum voluntary force (MVF) and the mapping from EMG activation to task-space force (or pulling vector matrix) were computed. Throughout this process, the participants were instructed to face the monitor which displayed a yellow cursor representing the measured X and Y axis force, which based on their configuration they were informed matched to extension–flexion and radial-ulnar deviation motions, respectively. They were then instructed to one-at-a-time provide a maximum exertion of force in 1 of 16 equally spaced (every \(22.5^\circ\)) directions (Fig. 1c), where the reference direction was displayed on the monitor with a dashed line and progressed anticlockwise starting initially from leftward (flexion) force production. Each direction was given 4 s of time and the MVF for each direction was computed as the maximum recorded force during the final 2 s. The participant specific scaling MVF (sMVF) was then set to be the minimum recorded MVF value across the 16 directions and the force visualisation was subsequently scaled by the sMVF such that this magnitude of force would move the cursor to the boundaries of the monitor. With this scaling the pulling vector matrix was calibrated for using a similar procedure to22. Participants were asked to produce and hold (for 2 s) their force at 17 different target locations, corresponding to 1 target at rest with [0, 0] N force and the remaining 16 targets evenly spaced along a circle with radius 0.2 sMVF. This 0.2 sMVF force level was used for task-space target locations throughout the remainder of the experiment. For each target, the average of the force \({\textbf{f}} = [f_x,\,f_y]^T\) and rectified EMG activity \(\varvec{\alpha } = [\alpha _{FCR},\,\alpha _{FCU},\,\alpha _{ECRL},\,\alpha _{ECU}]^T\) was recorded across the 2 s holding period. After multiple rounds of targets, the pulling matrix \({\textbf{H}} \in {\mathbb {R}}^{2\times 4}\) was computed in a manner consistent with23 through ordinary least squares (see Supplementary Fig. S1 for a visualisation of the pulling vector matrix) assuming the linear relationship

$$\begin{aligned} {\textbf{f}} = {\textbf{H}}\varvec{\alpha }. \end{aligned}$$
(1)

The initial calibration and post session calibration blocks contained an identical process, however, to account for participants not having familiarity with the calibration procedure, the initial calibration was composed of 2 rounds of MVF production and 6 rounds of EMG calibration target reaches, while the post calibration consisted of only 1 round of MVF production and 4 rounds of target reaches.

Null-space calibration

The null-space with respect to the produced force was computed and the components of its visualisation were oriented. First the null-space projection matrix \({\textbf{N}}\) was determined such that its rows consisted of orthonormal basis vectors that were orthogonal to the subspace projected by the pulling matrix \({\textbf{H}}\). Then, a scaling

$$\begin{aligned} \tilde{{\textbf{N}}} = \kappa {\textbf{N}} \end{aligned}$$
(2)

was applied to ensure that the resulting null-space coordinates

$$\begin{aligned} \tilde{{\textbf{n}}} = \tilde{{\textbf{N}}}\varvec{\alpha }, \end{aligned}$$
(3)

had a similar magnitude to that of the produced force (for the same muscle activity levels), where \(\Vert \tilde{{\textbf{n}}} \Vert = 1\) would have similar EMG activity magnitude to that of 1 N of produced force and any null-space coordinate with magnitude \(\Vert \tilde{{\textbf{n}}} \Vert = \beta\) could because of the linear scaling be interpreted as requiring a muscle activity level that was similar to that required to produce \(\beta\) N of force. This scaling was determined so that its magnitude was equal to the largest of the scaling from EMG activity to the principle force directions (flexion–extension or radial-ulnar deviation), such that \(\kappa = \max {\{\Vert h_1 \Vert , \Vert h_2 \Vert \}}\), where \(h_i\) denotes the ith row of the pulling matrix \({\textbf{H}}\). With a null-space mapping computed, the cursor was augmented such that in addition to controlling its position through the sMVF scaled X and Y axis force, participants also were in control of the cursor radius through the null-space component \(\tilde{{\textbf{n}}}\)’s magnitude.

To determine the natural direction that co-contraction took within the null-space, participants were then instructed to “naturally” stiffen their wrist such that the cursor remained at the origin position while its size grew to match a target ring with \(\Vert \tilde{{\textbf{n}}} \Vert = 3\) (Fig. 1d). Participants were required to repeat this task 6 times, where for each repetition the last 2 s of EMG activity was recorded (see Supplementary Fig. S2 for a visualisation of the average EMG activity in each of the 6 repetitions) and used to compute the average co-contraction null-space activity \(\mathbf {n_c}\). Participants were then asked to perform 1 final calibration trial in which they were instructed to completely rest. This was used to determine their offset null-space activity \(\mathbf {n_0}\). From the null-space offset and the average co-contraction null-space activity \(\mathbf {n_c}\), the null-space component \(\tilde{{\textbf{n}}}\) was transformed such that it was \({\textbf{0}}\) at rest and natural co-contraction corresponded to horizontal motion on the screen. This meant that the final null-space activity \({\textbf{n}}\) was given by

$$\begin{aligned} {\textbf{n}} = {\textbf{R}}_c\left( \tilde{{\textbf{N}}}\varvec{\alpha }-\mathbf {n_0}\right) , \end{aligned}$$
(4)

where \({\textbf{R}}_c\) corresponds to the matrix that rotates the vector \(\mathbf {n_c}\) to \([1,0]^T\).

Null-space target reaching

Since this session required virtual reaching of targets in the null-space without significant task-space force production, the visualisation was changed such that the \(X-Y\) position on the monitor corresponded to the negative of the null-space activity \({\textbf{n}}\) with natural co-activation leading to leftward motion, and the radius of the cursor was proportional to the magnitude of the recorded force. Before the participants were asked to attempt 80 null-space target reaches, they were first given 2 min of exploration time to freely explore and get familiar with the altered visualisation without specific instruction. For each of the 80 targets, the participant had 7 s to control a cursor through their EMG activity to reach for the target location in the null-space. A target reach was considered successful if the participant reached its location and held at that location for 1 s without leaving the target or producing more than a predefined force threshold (0.04 sMVF). The cursor was considered in the target if the null-space error \({\textbf{e}}_{{\tilde{n}}}\) between the target and cursor location was less than a predefined threshold (\(\Vert {\textbf{e}}_{{\tilde{n}}}\Vert \le 0.4\)). If the cursor satisfied this condition while being below the target threshold, the target colour would change to green to indicate that both conditions were met.

The target was initially located to correspond to the natural co-contraction measured during null-space calibration. Therefore it was placed at \([2,0]^T\) (which due to the flipped coordinates was visualised on the left side of the origin). Each successive target reach would use the same location until the participant demonstrated that they could consistently reach the target, which was defined to be when they could successfully reach at least 80% of targets over a moving window of ten trials. If this condition was met, a new target location would be triggered corresponding to an anticlockwise rotation of the previous location by \(5^\circ\) (Fig. 1e). This was set such that each new target would require increasing the null-space activity in the direction that was orthogonal to natural co-contraction, while being small enough so that participants needed small adaptation for successive targets. At best the participants could reach for targets in ten different target locations within the 80 reaching trials.

Concurrent target reaching

The visualisation was the same as in the null-space calibration, as participants were similarly asked to virtually reach for task-space targets while concurrently reaching for an additional DoF through their natural co-contraction behaviour. The participants were required to make 90 target reaches from the centre to the target, where as in the NSTR, they had 7 s to reach for the target location and would be considered successful if they held at that location for 1 s. In this task, targets were defined both in terms of their task-space location (visualised as the target’s position on the screen) and the natural co-contraction coordinate of the null-space (visualised as the target’s size). Here, force control was used as a direct task-space interface, while only the natural null-space EMG component (i.e. natural co-contraction) was used to discourage changes in null-space behaviour across the experiment.

Figure 1f illustrates the visual feedback for the participants. 9 different task-space locations were considered corresponding to the origin and 8 equally spaced points on a circle with magnitude 0.2 sMVF, while 2 different sizes were considered corresponding to null-space activity \(\Vert {\tilde{\textbf{n}}}\Vert = 3\) and \(\Vert {\tilde{\textbf{n}}}\Vert = 6\). Target sizes and locations were randomised and could change for each target reach. If the cursor was within the tolerances in reaching the target, the target changed colour to be green. Before each reach, participants were required to return to the centre position to initiate the next trial.

Data analysis

The participant behaviour was evaluated in terms of the task performance and motion characteristics for each experimental session. In the NSTR, task performance consisted of the number of different target locations that were attempted (number of targets) as well as the participant’s displayed angular range of motion within the null-space (reaching range). The reaching range was individually computed for the exploration and the main experiment as well as the union of the data from both stages. In contrast, since the CTR had a fixed number of trials for each target location, task performance was evaluated through the percentage of successful trials (success rate) for each target.

The participant’s motion characteristics were evaluated in the NSTR through 4 metrics: (i) the motion efficiency computed as the ratio of the direct straight line motion to the total distance travelled in the null-space for a successful target reach; (ii) the time to success defined as the time interval between the start of the trial and the time of a successful reach; (iii) the number of corrections which was estimated by first low-pass filtering the null-space trajectories (second-order Butterworth; 3 Hz cut off) and then computing the number of peaks in the speed profile (using the MATLAB findpeaks function); and (iv) the force magnitude, computed as the root mean square of the measured force across the trial.

In the CTR session, the motion characteristics were instead computed through the time to success as well as the motion concurrency which was computed as the percentage of the total null-space motion that was performed at the same time as the task-space motion. This percentage calculation accounted for the impact of the unintentional null-space activity that would occur during natural force production by removing that component before the percentage was subsequently computed. Here, we used the null-space activity from the last 2 s of the Force/EMG calibration for the equivalent target position reach during the calibration (for each individual participant) as the activation baseline (unintentional null-space activation). This baseline activity was then used to predict how much null-space movement each participant would have generated when reaching for that individual task-space target without any additional co-contraction. The remaining null-space movement from the baseline to the target size was then separated into 2 phases: Phase 1 running from movement onset until the target in task-space was reached; and Phase 2 running from when the task-space target was reached (end of Phase 1) to the null-space target reach. The sum of null-space movement from Phase 1 and Phase 2 was equivalent to the difference of the target size and the baseline. If the needed null-space movement was less than 15 % of the target size, it was considered to be too small of an effort for any robust measurement and therefore removed from the analysis (8 and 6 data points removed from small and big target size, respectively).

Statistical analysis

Bayesian methods were used for hypothesis testing. The Bayes Factor \(BF_{10}\) measures the level of support for \(H_1\) (alternative hypothesis) over \(H_0\) (null hypothesis) and ranges from 0 to \(+\infty\). A value greater than 1 indicates that \(H_1\) is more strongly supported by the data, whereas a value below 1 indicates stronger support for \(H_0\). To interpret the Bayes Factor value, we followed previously established heuristics24 according to which values of \(BF_{10}\) between 1 and 3 would be considered anecdotal evidence, values between 3 and 10 moderate evidence, values between 10 and 30 strong, and values between 30 and 100 very strong evidence for \(H_1\). Likewise, values of 1/\(BF_{10}\) in the above ranges would be considered as evidence in favour of \(H_0\), for example values of \(BF_{10}\) between 1/10 and 1/3 would be considered moderate evidence for \(H_0\).

To compare the range of movement angles that participants could generate between their initial null-space exploration and the main null-space target reaching, the Bayesian Wilcoxon signed rank test25 was applied as a Bayesian alternative of the non-parametric Wilcoxon signed rank sum test. A Cauchy distribution was used as prior distribution25. To assess the robustness of the Bayes factor concerning the prior scale, we computed the Bayes factor for various scale parameter values (\(\gamma =1/\sqrt{2}\), \(\gamma\) =1, \(\gamma\) =1.4). Consistent Bayes factor values across different prior scales signify the robustness of the test outcome. For all other metrics in the NSTR, due to each participant reaching a different amount of target locations such that there are inconsistencies in the amount of data for each target, only descriptive stats are presented.

The Bayesian repeated measures ANOVA26,27,28 was applied to compare the means of multiple dependent samples as a Bayesian variant of the repeated measures ANOVA. A Cauchy distribution was used as prior distribution. To assess the robustness of the Bayes factor concerning the prior scale, we computed the Bayes factor for various scale parameter values (\(\gamma\) =0.5, \(\gamma =1/\sqrt{2}\), \(\gamma\) =1) for fixed effects. This was used to analyse (1) the dependence of the success rate and latency in the CTR on target location and size and (2) the dependence of the activity in the null-space during the CTR on the target size and task phase. In these two-way repeated measures ANOVA designs (with interaction) each effect is included in multiple models. We, therefore, report the inclusion Bayes Factors (\(BF_{inclusion}\)) that provides the relative support for all models that include an effect compared to all models excluding it27,28. Post-hoc Bayesian paired t-tests were performed to identify pair-wise differences.

All hypothesis tests were performed using JASP 0.17.1 (https://jasp-stats.org).

Results

Null-space target reaching

Task performance

Participants showed a mix of abilities for successfully reaching for the null-space targets. Here as shown in Fig. 2a only 6 out of 10 participants were able to show sufficient consistency in their reaching to progress beyond the first target. Of those participants that did progress beyond that target, only 2 were able to progress to attempt the fifth target, while the remaining 4 did not progress beyond the third target. Here it is noted that due to the target size, the first 3 targets could have been successfully reached using the same strategy as the first target. This therefore indicates that the majority of the participants did not show a consistent ability to change their null-space activity.

The overall participant ability is further illustrated through their trial-by-trial success (Fig. 2b). This shows that participants were in general unable to successfully reach for the first target within the initial trials. This was despite the target being set to match their average ‘natural’ co-contraction behaviour across the null-space calibration. Since participants were only able to proceed beyond the second target if they were able to adapt such that they could consistently produce the necessary null-space activation to reach and hold the first target, the use of the number of targets attempted as a metric is limited in that it does not provide understanding of any modulation that may have taken place without successful reaching of the first target (for example if the participant was able to modulate their null-space activity but could not hold it at the required location).

For the best performing participant (Subject 6), Fig. 2c shows their overall null-space trajectories for each target, in which the successful trials are shown coloured and the non-successful trials shown in grey. From these trials it can be observed that this participant showed an ability to move within a large range in the null-space exploration. For the first target, they were able to restrict this motion such that it was mainly horizontal (corresponding to ‘natural’ co-contraction). While similar trajectories were then used for the second and third targets, the participant was able to redirect their motion for the fourth and subsequent targets. However, by the fifth target they showed signs of difficulty for generating further changes in their motion angle. This suggests that in this case they may have reached the limit of their range of controlled motions.

To further investigate the range of null-space activation that the participants were able to generate without the production of force, Fig. 2d shows a histogram of the different angles within the null-space that were reached without violation of the force and null-space magnitude constraints for each participant. Note that the exploration component shows both the reached angles with (black) and without (grey) considering the constraint on the force magnitude, respectively. From the exploration shown in this figure it can be observed that while participants could often reach a wide range of angles in the null-space, this was typically coupled to the production of force. Furthermore, in the reaching stage participants showed a mixed ability to successfully reach for the different targets. Here, their distribution of motion appears to have a trend that differs for each target that they attempted to reach. However, more data would be needed for each target to validate this trend.

Motion characteristics

The characteristics of the participant motion is further illustrated through Fig. 3. Here, Fig. 3a indicates that participants possessed some range of motion within the null-space exploration. This range of motion appears related to the range of reaching angles observed during target reaching, where there was weak evidence to suggest that there was no difference between the range of angles reached in exploration and in target reaching (\(BF_{10} = 0.38, 0.3, 0.22\) for normal, wide and ultra-wide prior with Cauchy prior distribution set to \(\gamma\) = 0.707, 1 and 1.4, Bayesian Wilcoxon signed-rank test).

When considering the successful trials, the motion efficiency (Fig. 3b) shows that in general participant motion was direct towards the target for all locations, where the median efficiency across participants (who reached that target) was greater than 0.75 for all targets. The time to success (Fig. 3c) and the number of corrections made (Fig. 3d) show no clear trend of change across the target locations. However, it should be noted that the small number of participants that reached the targets with larger reaching angles limits this analysis.

Finally, while participants showed some force production in their null-space activation, most participants were able to keep it below the threshold, and within successful trials they do not appear to have an increase in the magnitude of force produced (Fig. 3e) with respect to the different target locations.

Concurrent target reaching

To test the feasibility of performing null-space target reaching simultaneously with movement in the task-space, participants used their 2-dimensional wrist force to reach for a target at 1 of the 9 locations, with target T9 corresponding to no force needed (Fig. 4a). Concurrently they were required to use their null-space activation projected onto their natural co-contraction to reach the given target size (\(\Vert {\textbf{n}}\Vert = 3\) or \(\Vert {\textbf{n}}\Vert = 6\)), which corresponded to the 2 different levels of null-space activation. Only when both target position (task-space) and size (null-space) were reached jointly would the trial be considered as successful. The participant success rate depended on target size and location (Bayesian repeated measures ANOVA, \(BF_{inclusion}\) = 3.0, 9.9, 6.6 (\(\gamma\) =0.5); 2.3, 6.2, 6 (\(\gamma =1/\sqrt{2}\)); 1.5, 2.6, 4.3 (\(\gamma\) =1) for target size, location and interaction of size and location) though the target size did not have a strong effect. The success rates were higher when reaching for the small targets and lower for the large targets in the ulnar directions (T6–8) (Fig. 4b).

Trajectories from the cursor position show that all target positions could be reached regardless of the target position and size, suggesting that there were no difficulties for the participants to move in the task-space (Fig. 4c,d). However, while some participants showed a relatively stable trajectory towards targets regardless of the target size (Fig. 4c), other participants showed much more variance in the task-space when higher co-contraction was required (Fig. 4d). This could be due to the different level of activity that different participants generated in the null-space while reaching for targets in the task-space. Null-space activity generated during target reaching for all participants suggest different reasons for failing to reach the targets. When the target was small, participants mainly failed the task due to generating too much null movement (T5–T8). For the larger sized targets, unsuccessful trials were caused by either insufficient null movement (T9) or also too much of the null movement (T6, T7) (Fig. 4e).

The latencies to successfully reach the target in task- and null-space evaluation clearly show that the target position was typically reached before the correct target size was obtained using null-space activity (Fig. 4f). The latencies depended on space and target size (Bayesian repeated measures ANOVA, \(BF_{inclusion}>100\) for space, target size and interaction of size and space for Cauchy prior distribution set to \(\gamma = 0.5, 1/\sqrt{2}\) and 1) but there was no evidence for a difference between large and small targets for the task-space (post-hoc Bayesian paired t-tests with adjustment for multiple comparison; posterior odds > 20 for all pairwise comparison except between \(\Vert {\tilde{\textbf{n}}}\Vert =3\) and \(\Vert {\tilde{\textbf{n}}}\Vert =6\) for the task-space: posterior odds \(=0.25\); uncorrected \(BF_{10}=0.6\)).

To investigate whether target reaching in the null- and task-space took place sequentially or concurrently, we split the null-space activity into 2 phases: Phase 1 lasted from movement onset until when the target in task-space was reached while Phase 2 lasted from when the target was reached in the task-space until when the target reach was completed (i.e. the target was reached in both task- and null-space). We used the EMG reading of the same target positions as in T1–8 taken during the calibration for each individual participant and predicted how much null-space movement they would have generated in reaching for the target in task-space without any additional co-contraction. This null-space movement was used as the baseline null-space movement, and the additional required null-space movement to reach the target size was then considered as the total null-space movement needed. From here we computed the proportion of null-space movement generated by the 2 phases as the contribution of the null-space movement to reach the target for the corresponding phase. For both target sizes, the contribution from Phase 1 was higher than from Phase 2 (Bayesian repeated measures ANOVA, \(BF_{inclusion}=10,0.7,1.2\) (\(\gamma\) =0.5) ; 8.5, 0.6,1.1 (\(\gamma =1/\sqrt{2}\)); 6.3, 0.6, 1.1 (\(\gamma\) =2) for phase, size and interaction), suggesting that target reaching mainly took place concurrently in the null- and task-space, and that the participants then fine-tuned the target size after reaching the correct target location (Fig. 4g).

Discussion

We tested the capability of users to exploit the redundancy naturally present within the wrist’s musculoskeletal system as a source of signals for DoF augmentation. For successful DoF augmentation it is required that users can reliably control additional DoFs with minimal interference on their existing natural motion behaviours. In 2 sessions, we therefore evaluated if users could (i) alter their wrist muscle activation patterns to reach for targets positioned in the task-intrinsic muscular null-space; and (ii) simultaneously vary their activity in the null-space while performing isometric force reaching tasks. Our results show limited ability for participants to reliably control their behaviour within the muscular null-space as well as no clear improvement in the range of their null-space activation as a result of training. In contrast, while their performance was direction and target size dependent, the results show some potential for concurrent target reaching in the task- and null-space when natural co-contraction magnitude is used as a null-space input.

Participants struggled to reach different targets in the null-space

Despite participants showing a range of null-space angles that they could manoeuvre the cursor to, only 2 out of 10 participants (Subjects 5 and 6) were able to progress beyond the third target in the NSTR task. Here because of the chosen target size, the fourth target would be the first target for which an initial strategy of moving to the first target’s centre location (as in a natural co-contraction) would be unsuccessful. The poor performance of most participants shows that they could not adapt their muscle activation patterns for reliable null-space target reaching within the given number and length of trials. This inability to reliably reach for targets in the null-space could have been caused by: (i) limitations in the pulling vector/null-space computation; (ii) variations in the participants’ natural co-contraction behaviour making it that the first target itself was difficult to reach; (iii) participants not having enough repetitions and/or time to adapt their behaviour; and/or (iv) the participants lacking the necessary capability to perform fine control of their muscle activation patterns.

Due to the presence of the non-negativity constraint on muscle activation (muscles can only pull), the set of possible null-space vectors that can be reached through muscle activity corresponds to the subset \({\mathscr {N}}\) of the entire null-space such that

$$\begin{aligned} {\mathscr {N}} = \{{\textbf{n}} = {\textbf{R}}_c\left( \tilde{{\textbf{N}}}\varvec{\alpha }-\mathbf {n_0}\right) , \varvec{\alpha }>{\textbf{0}}\}. \end{aligned}$$
(5)

It is therefore possible that the participants were not able to reach for null-space targets if the targets were outside of the reachable subset \({\mathscr {N}}\). Here, computation of the set \({\mathscr {N}}\) for each participant showed that all participants were never required to reach for a target beyond \({\mathscr {N}}\) (see Supplementary Fig. S3) and therefore that they were not limited by the non-negativity constraint. One additional possible limitation in the use of pulling vectors is that they form a linear approximation of the mapping between muscle and force space. Here, while the computed pulling matrices did not perfectly reproduce the observed force measurements and for some participants displayed atypical mapping for the FCR muscle (see Supplementary Fig. S1), our observed quality of fit (see Supplementary materials) was similar to that of other successful EMG-to-force reproductions22. This, in addition to the participants being able to use the null-space projection for both the CTR session and null-space calibration, implies that participants were able to overcome the limitations in the pulling vector/null-space computation.

4 out of 10 participants did not progress beyond the first target. This suggests that they were not able to reliably reproduce the natural co-contraction behaviour that they executed in the calibration. This might reflect that the natural co-contraction behaviour of each single participant possessed a sufficiently large variation such that while they might have been able to reach the target associated to their natural co-contraction, they could not hold the necessary muscle activation for the required 1 s. However, this is unlikely as some participants (for example Subjects 1, 3 and 9) displayed a consistent bias in their null-space activation within target reaching (Fig. 2d) and never produced motion in the same direction as their original mean natural co-contraction activity. This bias may have been the result of an unobserved small postural change that occurred within the 2 min exploration phase29 that could have led to a small mismatch between the calibration computed pulling matrix (and null-space bases) and that of the participant in their new posture. While the participant’s arm was fixed to the interface and they were instructed to place their arm in a neutral position, given that only the flexion–extension and radial-ulnar deviation were detected and associated to the X-Y visualisation, undetected changes in forearm pronation–supination might lead to discrepancies between the instructed actions and the directions of force that they would produce. Alternatively it could result from the change of the visualisation and the associated instruction when going from producing ‘natural’ co-contraction to change the cursor size during the calibration to the target reaching for which co-contraction now resulted in 2-dimensional motion. In either case, the participants did not show an ability to compensate for the presence of a bias nor for the natural variation in their wrist co-contraction.

The similar angle ranges (Fig. 3a) in exploration and target reaching suggest that participants did not gain additional capability to reach angles in the null-space throughout the 80 trials of target reaching. This indicates that at least over the time-scale of the experiment that there was no learning effect in the range of null-space activation behaviours. It has been previously suggested that adaptation to produce atypical muscle activation patterns requires long trial duration to enable strategy exploration within a trial15. While we do observe a longer time to success (Fig. 3c) and more corrections (Fig. 3d) than would be observed in conventional physical target reaching tasks, this is likely related to participants struggling to use our novel EMG co-contraction based controller to produce the necessary directions of reaching movement (which leads to multiple corrections per trial) and to control the variability in their magnitude of co-contraction. It is worth noting that the time to success is still much less then the maximum reaching time of 7 s, where the high motion efficiency (Fig. 3b) suggests that participants made quick corrections when they were successful, rather than using the full 7 s which was large enough that in some cases participants could return to the origin and make a second reaching attempt within the allotted time period.

Was there any difference that explained the better performance of the 2 most successful participants? There was no clear feature (for example experience with the device and/or demographics) that separated the 2 most successful participants from the other participants. However, with only 2 participants further data would be required to determine if there is a particular cause for these improved results, where for example these participants may simply possess a greater flexibility and natural control over their muscle activation patterns.

Participants could simultaneously vary their co-contraction with task-space reaching

In the CTR session, we tested whether participants could vary the magnitude of their co-contraction in the null-space while simultaneously producing task-space force. The relatively high success rates shown in Fig. 4b and even higher performance when consider only target reaching without holding (see Supplementary Fig. S6) suggests that participants were capable of performing the task. However, their success rate was target position dependent. Among all the target positions, the success rate was lowest when the targets required force ulnar deviation. When isometric wrist movement in task-space was performed, it was also found to produce unintended natural null-space activity due to non-negativity of muscle activity. This natural null-space activity was found to be nonlinear across the 2D space and participant dependent. The null-space trajectories for different target positions in Fig. 4e suggests that the unintended natural null-space was larger in the ulnar direction (T6–8), causing the lower success rate in these directions. A more sophisticated nonlinear calibration that accounts for this natural null-space activity may be able to minimise this interference and enable greater success across these target locations.

The task- and null-space target reaching sub-components showed a clear difference in their time to reach the targets, as was previously observed in16. Here, with no constraints on the sequence at which participants performed the sub-components, the time required to reach the target in the task-space was shorter than that of the null-space. Despite this, the motion concurrency analysis (Fig. 4f,g) showed that most of the gross motion in the task- and null-spaces occurred simultaneously. The additional time to reach the null-space target may therefore indicate that participants possessed a less accurate motor control in their null-space activity than in their task-space actuation. Here, it is worth noting that participants have a lifetime of experience in performing task-space motions with fine-motor control. In contrast while co-contraction is used in natural movement (e.g., to “stiffen up” in response to disturbances30), the control of co-contraction within natural movement may take place less often and require lower accuracy than that of task-oriented force control. With greater experience it is therefore possible that the participants may be able to account for the observed performance differences.

Application considerations

Our analysis evaluated the ability to concurrently coordinate co-contraction (as a signal for DoF augmentation) and task-space force while the wrist was placed in an isometric configuration. While an isometric configuration was chosen to limit confounding variables, such as the muscle pulling vectors changing with the wrist position, it is noted that some muscle contractions such as pronation/supination may have still be able to alter force sensor readings. The choice of an isometric interface represents an evaluation of a potential command interface for movement augmentation that does not consider the simultaneous generation of motion, for which performance may be worse given that previous studies have shown that humans possess difficulties when performing other regulation tasks concurrent to motion generation (such as force regulation31). The approximately isometric wrist behaviour of such an interface would, however, be consistent with a subset of supernumerary limb applications where the natural limbs are constrained while the supernumerary limb moves independently. This includes when the supernumerary limb is used to open a door while the natural limbs hold a heavy box32 and aircraft fuselage assembly33 where the natural limbs could support the door while the supernumerary limb attaches it to a frame. Alternatively, the isometrically produced forces together with the muscle activity in the null space could be used to control computers, mobile devices or supernumerary limbs via an interface without direct physical contact. This would enable users to control more DoFs than with the wrist forces alone.

Our null-space reaching results suggest that while users possess variability in their null-space behaviour, most lack the flexibility and control necessary for controlling all of the DoFs possible within the wrist complex’s muscle-to-force null-space. This apparent inability to exploit novel muscle activation patterns differs from findings considering the entire upper limb15, where it has been observed that with sufficient time to make online motion corrections participants could learn to exploit atypical arm-wide muscle activation patterns for end-point reaching tasks. This suggests that while there is some potential for the use of musculoskeletal redundancy across an entire limb as a source of commands for movement augmentation, it does not appear to be suitable at the level of the wrist joint, at least not within a short practice period on a single day. As a result, the number of possible signals that are directly available through the muscular null-space may be lower then previously thought.

It is however worth noting that while the wrist system was chosen partly due to it being a single joint with low (and easily visualised) degree of redundancy that also allowed us to exclude the influence of difficult to measure angles in multi-joint systems, due to difficulties in obtaining a reliable sEMG recording without cross-talk, the study only used 4 out of 5 wrist muscles after the exclusion of the extensor carpi radialis brevis (ECRB). Furthermore, while no evidence of adaptation was found, the study only considered 1 session. The lack of an ECRB measurement limits the degree of redundancy that can be exploited within the wrist-complex such that there is only a 2-dimensional muscle-to-force null-space, while the use of a single session limited the potential for possible motor learning. These 2 factors restricted the study’s ability to observe variation in muscle activation patterns, such that evaluation with all muscles (potentially through intra-muscular recording) or the consideration of other joints (such as the shoulder or the hip) that possess higher dimensional muscle-to-force nulls-spaces, are worth investigating over a multi-day experiment.

Our concurrent reaching results, in addition to those previously found with the upper limb16, indicate that variation of the magnitude of co-contraction may represent a mechanism for non-invasive augmentation control that can be manipulated in parallel with natural movement. However, in both our results and those of16, while participants could in general reach the different desired null-space magnitudes, the ability to hold that level of muscle activation was not as consistent and observably participant dependent. Furthermore, it is worth noting that because of a linear model assumption, as well as the non-negativity property of muscle activity, task oriented motion will always lead to activities in the 2D null-space. Here, the magnitude of co-contraction is also known to vary during natural motion for information exchange purposes17, such that natural co-contraction has a function and cannot necessarily be considered as a mechanism for augmentation by extension without there being new different muscle activation patterns observed.

Despite these concerns, the use of the muscular null-space is likely less application dependent than approaches that have considered the kinematic null-space34, for which the task-relevant kinematics changes for each different task. Since the primary cause of failure was the presence of involuntary null-space activation, the use of a participant dependent non-linear null-space that adapts the required co-contraction levels based on this involuntary activation could offer a mechanism of reducing any negative interference with natural motion behaviour while increasing individual participant success rate.

How might our results generalise to other possible muscle redundancy based augmentation mappings? Here, while we only made use of an isometric mapping, other studies have previously considered concurrent movement and co-contraction control schemes by investigating the voluntary control of impedance35,36,37,38. Similar to our results, these works suggest that the ability to modulate impedance, which occurs without changing task-space force such that it is in the muscle-to-force null-space of the task, is quite limited35,36,37, where in the difficult task of concurrent motion and co-contraction modulation, most observed modulation is constrained to magnitude scaling, which correspond to a single direction of null-space activation. Furthermore, these results again suggest that larger redundancy might be needed, given that the only scenarios that show more complex impedance modulation30,38 were also reliant on redundancy coming from the entire upper limb. One novel mapping for future investigation might be the use of velocity control rather then position control. Here, velocity control has previously been shown to result in more efficient energy usage for DoF augmentation as it does not require continuous activation39. However, it should be noted that such a scheme may suffer from similar limitations if participants cannot properly control the direction of their null space activation.

Conclusion

Our results do not find evidence for a general ability to vary the wrist’s null-space muscle activation patterns. Furthermore, we do observe that the wrist’s null-space activation patterns are coupled to task-oriented motions. Together these factors suggest limited potential for the use of the redundancy present within the wrist’s musculoskeletal system as a non-invasive source of signals for DoF augmentation.