Abstract
Purpose
Endomicroscopy (EM) provides high resolution, non-invasive histological tissue information and can be used for scanning of large areas of tissue to assess cancerous and pre-cancerous lesions and their margins. However, current robotic solutions do not provide the accuracy and force sensitivity required to perform safe and accurate tissue scanning.
Methods
A new surgical instrument has been developed that uses a cable-driven parallel mechanism (CPDM) to manipulate an EM probe. End-effector forces are determined by measuring the tensions in each cable. As a result, the instrument allows to accurately apply a contact force on a tissue, while at the same time offering high resolution and highly repeatable probe movement.
Results
0.2 and 0.6 N force sensitivities were found for 1 and 2 DoF image acquisition methods, respectively. A back-stepping technique can be used when a higher force sensitivity is required for the acquisition of high quality tissue images. This method was successful in acquiring images on ex vivo liver tissue.
Conclusion
The proposed approach offers high force sensitivity and precise control, which is essential for robotic EM. The technical benefits of the current system can also be used for other surgical robotic applications, including safe autonomous control, haptic feedback and palpation.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
The da Vinci (Intuitive Surgical Inc., CA, USA) is a notable surgical robot which is being used to facilitate complex minimally invasive surgery (MIS). Although the da Vinci is admittedly the most successful surgical robotic system to date, it suffers from a number of limitations. For instance, the lack of haptic feedback has not been satisfactorily resolved despite multiple attempts [1]. Additionally, the intrinsic accuracy of the system has been shown to be around 1 mm [2], which is insufficient for applications that require sub-millimetre accuracy, such as tissue scanning for optical biopsy. While higher accuracy has been achieved using the da Vinci, such approaches require additional markers and calibration steps [3].
On the other hand, there is a demand from the surgeons to be able to utilize the imaging modality intraoperatively for accurate identification of malignant tissue to minimize unnecessary damage to healthy tissue. Optical biopsy using endomicroscopy (EM) provides the highest resolution images among current modalities and is potentially applicable to robotic surgeries as shown in [4]. EM relies on sub-millimetre diameter optical probes to obtain histological information. For accurate and reliable assessment of pre-cancerous lesions and their margins, large areas of tissue must be scanned and accurately reconstructed. The use of a large number of spatially distributed points is one way to cover wide tissue areas. At the same time, the force applied on the tissue by the scanning probe must remain within a very tight range for optimal signal acquisition and safety [5]. For the same reasons, the probe must be able to comply with tissue morphological variations. Research is being conducted and outcomes have been reported by measuring the contact force using a force sensor placed at the tip of an instrument [6], by using force adaptive device [7], or by using computer vision techniques such as edge detection or intensity recognition [3].
An alternative and simpler design, based on the use of a cable-driven parallel manipulator (CDPM), is proposed here for robotic EM. The mechanism is used to accurately control an EM probe and at the same time accurately measure tissue contact forces. The contact forces are derived simply by measuring the tension of the controlling cables/tendons. CDPMs exhibit several desirable qualities: high force transmission, large workspace, easy workspace reconfigurability, high dynamic capabilities, and low cost [8]. To demonstrate its potential and favourable characteristics, the proposed instrument is retro-fitted on a modified da Vinci robot instrument. Partial robotic hepatectomy is used as an exemplar target procedure, where EM can be very useful in identifying the malignant tissue margins intraoperatively [9, 10]. Therefore, the system is validated on bovine liver tissue. The following sections provide a description of the developed system and its principle of operation and conclude with experimental validation of its capabilities.
Instrument description
The concept of CDPM is introduced first followed by the design and specifications of the proposed instrument.
Cable-driven parallel robotic manipulator
The proposed instrument is based on a patented concept introduced in [11], named ‘CYCLOPS’, which describes a robotic attachment for endoscopes (Fig. 1). The CYCLOPS is based on a six-cable CDPM used to control an over-tube that can accommodate commercially available surgical instruments. The CDPM is supported within a deployable rigid or inflatable scaffold, which then becomes rigid or quasi-rigid upon deployment or inflation.
The CYCLOPS exhibits several desirable properties, which are inherent to most CDPM approaches. These include, dexterous manipulation, high accuracy, stability, simplicity and force sensitivity. The system is currently undergoing pre-clinical in vivo validation with a focus on colorectal Endoscopic Submucosal Dissection (ESD) [12]. Here, we use EM as an exemplar application to further highlight the advantages of the concept.
Conceptual design of the proposed instrument
Unlike the six tendons per instrument used in the CYCLOPS, the proposed instrument uses four tendons to actuate a cylindrical over-tube with an outer diameter of 3.0 mm. The over-tube is used to accommodate an EM probe with an outer diameter of 2.8 mm. The four tendons can actuate the probe in three planar degrees of freedom (DoF). Figure 2 shows a schematic illustration of the concept.
Force calculation and sensitivity
With reference to Fig. 2, the four-tendon configuration can be used to control the position of the probe and simultaneously sense the forces applied on it. The forces on the tip of the probe are estimated by measuring the tension \(T_i\) for each tendon i, as shown in Fig. 3a and (1). The calculated contact force \({ CF}_\mathrm{{cal}}\) is derived based on the instrument pose and the measured tensions, as shown in Fig. 3b and (2). The force sensitivity can be estimated by comparing the calculated forces \({ CF}_\mathrm{{cal}}\) with ‘ground truth’ forces CF, measured with a load cell placed at the interface with the probe tip. Discrepancies between \({ CF}_\mathrm{{cal}} \) and CF are expected due to static friction, elasticity, and hysteresis in the mechanism. \(T_0 \) is an initial preset tension to keep the over-tube stable before applying the external force.
Depending on the tissue type, the contact forces required for EM, can be below 1.0 N [13]. Therefore, the reduction of friction in the mechanical parts involved is essential to increase the forces sensitivity of the system. To reduce friction on the rounded corners where the tendons change direction on the scaffold, or where they interface with the load cells, we use Polytetrafluoroethylene (PTFE) tubes. Using pulleys is an alternative, which however takes more space and introduces more complexity. Moreover, through experimentation we have established that the difference between pulleys and PTFE tubes in terms of friction is negligible. For the proposed device, the tendons in use are 0.19 mm diameter UHMW-PE spectra wires (PowerPro, Shimano Inc., Japan), which can withstand up to 13 kgf.
Mechanical design of proposed instrument
All parts are mounted on a modified da Vinci instrument. Figure 4 shows the instrument. Four DC servo motors, 2232S024BX4 CCD 3830 (FAULHABER Inc., Daimlerstrabe, Germany) are used to control the tendons. With 3000 increments per revolution, a 1:25 gear ratio and a 10 mm tendon spool, the theoretical positional accuracy for each tendon is \(0.4~\upmu \hbox {m}\). Load cells (LCL-005, OMEGA Engineering, Inc, Manchester, UK) are used, which can measure up to 2.27 kgf with \(2~\hbox {mV/V} \pm 20\%\) rated output by a 5VDC excitation voltage. The system is controlled by a laptop PC in C++. Most structures have been manufactured in Poly-Lactic Acid (PLA) by a commercial 3D printer (Ultimaker2 Extended, Ultimaker Inc., Netherlands).
Experimental validation
The accuracy of force sensing is discussed by comparing the calculated contact force with the measured contact force in a static situation. The relation between contact forces and acquired EM images is evaluated in a simple 1 DoF axial movement. Subsequently, concurrent force sensing and image acquisition in a 2 DoF controlled movement is presented.
Measurement of contact force
For experimental validation a testing rig has been created. The mechanism to test the contact force is shown in Fig. 5. A linear actuator, L2818S0604-T5X5 (Nanotec Electronic Inc., Feldkirchen, Germany) is placed at the bottom of the testing mechanism to simulate up–down contacting movement. Another load cell (LCL-005) is placed on the tip of the linear actuator and used to provide the ground truth. The position of the over-tube is set at the centre of the scaffold, where the tensions in all cables should be equal to satisfy the static equilibrium condition. The linear actuator is programmed to perform 10 consecutive 2 mm displacements, along the longitudinal axis of the probe. Both ground truth and calculated contact forces are acquired and evaluated.
Relation of contact force and EM image quality
To assess the capabilities of the system when applied to EM, the optimal applied force for adequate image quality should be assessed and compared to the force sensitivity of the proposed mechanism.
Experimental environment
The Cellvizio endomicroscope (Mauna Kea Technologies Inc., Paris, France) is used. An optical probe is inserted from the back-end of the instrument and placed into the over-tube to which the probe is fixed (Fig. 6a). A Coloflex probe with a confocal length of 55–65 \(\upmu \hbox {m}\) is used. The field-of-view diameter of the probe is approximately 0.5 mm.
Target tissue
A piece of healthy bovine liver was used as a sample tissue. As the normal liver does not have noticeable features on the surface, a piece of lens-cleaning paper (Thorlabs Inc., NJ, USA) was placed on top of the liver. The dimension of the tissue is approximately \(20\times 20\times 8~\hbox {mm}\), and both the tissue and the paper were dyed with 0.2% acriflavine hydrochloride solution to get a fluorescent effect (Fig. 6b).
Optimum contact force
As in EM there is a direct relation between the applied force and the image quality achieved, the optimum contact force for the tissue needs to be identified. This contact force can be determined by the ground truth load cell which is placed underneath the liver tissue (Fig. 7).
Control method for image quality assessment
The over-tube is controlled autonomously by 0.01 mm incremental steps in the vertical direction. When the instrument detects contact with an object, the motors are stopped in holding mode. Contact is sensed through a rapid increase in the first derivative on the sensed cable tension values. After this point, the over-tube is moved backwards until the contact force becomes zero again. During this autonomous movement, ground truth force data and EM images are acquired and used for evaluation (Fig. 7). To assess whether the stiffness of the surface affects the relation between contact force and image quality, this experiment is conducted both on a rigid surface (PLA) as on liver tissue, each covered with lens testing paper.
Autonomous 2DoF control
Autonomous control is essential in a clinical case. Manual control would be difficult to achieve stable sub-mm accuracy required for vertical and horizontal control of the EM probe, based on a confocal length of 55–65 \(\upmu \hbox {m}\) and a field of view of approx. 0.5 mm, respectively. The liver tissue is used for testing the performance in 2 DoF movement using the inverse kinematics of the CDPM, based on a pre-defined surface scanning pathway. All pathways have been preliminarily programmed in \(\hbox {C++}\). The testing mechanism is the same as in “Relation of contact force and EM image quality” section. The difference is that we use the second derivative of tension data to detect contact force. Existing 2 DoF dynamic effects introduce changes in the first derivative even without any surface contact, and therefore additional threshold based on the second derivative is required to increase robustness of force detection. Threshold values P, \(Q_i\), and \(R_i\) (3, 4, 5) are used. P is a difference of the first derivative comparing the sum of tensions of tendon 1 and 2 with the sum of tensions of tendon 3 and 4. \(Q_i\) is the increase ratio of the first derivative of each tendon. \(R_i \) is the difference of the second derivative compared to a reference. For the tension T, the subscript i refers to the tendon number ranging from 1 to 4, and subscript t refers to the time at which the sample was collected: t being the current sample and \(t_0 \) and \(t_1 \) reference values at which the system was unloaded.
If the detected contact force is higher than the contact force required for obtaining an optimum quality image due to the diminished force sensitivity caused by the static friction, controlled step-back of the probe can be used to reach the required optimum force. The amount of stepping back needs to be determined empirically. During the 2 DoF autonomous movement, the ground truth force data and EM images are acquired for evaluation (Fig. 8). Six horizontal points are used to conduct effectively and repeatedly the same experiments. Consecutive mosaic images can be acquired between each horizontal point. At each point, the same control method as described in the previous section is applied, as shown in Fig. 8.
Results
Results of contact force measurements
Figure 9 shows the plots acquired by the procedure described in “Measurement of contact force”. The top graph illustrates how the four tensions change depending on the position of the end-effector. The bottom graph is the comparison between the ground truth contact force and the calculated contact force. The dotted area A shows that the calculated contact force is \(14.7\,\pm \, 2.0\%\) lower than the ground truth contact force. The dotted areas B1 and B2 show the hysteresis effects, in which the tensions did not return to the initial tension even after removing the external force applied by the actuator. It is considered that this discrepancy is due to static friction between the tendons and the PTFE and the elasticity of the structure. However, the overall performance is significantly stable over repeated motions, which demonstrates the high repeatability of the mechanism.
Results on the relation between contact force and EM image quality
Figure 10 shows EM images obtained using different contact forces. The contact force was measured by a load cell placed under the liver tissue. The image at 0.05 N shows the most noticeable contrast with clear edge features and was therefore selected as the optimal contact force value during subsequent experiments with the same liver tissue.
Figure 11 illustrates detection of the contact force and a relation between the contact force and images using the testing paper on a rigid surface. The moving average method was used to reduce the noise of the four tension values. Subsequently, by using the least square method, gradient values were derived as the first derivative. The red line shows the ground truth contact force which was measured on the rigid surface where the testing lens paper was placed. It is noticeable that the contact force was detected at approximately 0.2 N, at which point motor actuation was stopped. 0.2 N is the minimum contact force which the instrument could detect. This minimum force depends mainly on threshold conditions required to robustly detect forces. At the 0.2 N detecting point, the EM image shows certain features of the testing paper, but the image is blurred (Fig. 11(i)), as this value is higher than the earlier determined 0.05 N force required for acquiring the highest quality EM images. From this point, the EM probe was gradually stepped back from the paper at 0.01 mm steps, until the optimum force of 0.05 N was reached and the highest quality image was acquired (Fig. 11(ii)). The number of back-steps required to reach this optimum force was recorded. The same 0.2 N force sensitivity has been found while performing the 1 DoF movement on the liver tissue.
Results of autonomous 2 DoF control task
For the 2 DoF case, the average minimum detectable contact force on healthy liver tissue was approximately 0.6 N, which was larger than the 0.2 N found in the 1 DoF case due to the differences of the detecting algorithm. Similar to the 1 DoF controlled case, the minimum detectable contact force is higher than the required optimum contact force. Therefore, continuous images were taken while stepping back. The pathway in 2 DoF is shown in Fig. 8, and Fig. 12 illustrates the first three points of the 2 DoF movement extracting tension data and the first derivative data. CF (ground truth contact force measured on the object surface) in Fig. 12 shows that approximately 0.6 N contact force was detected during each stepping-in motion. Below the graph, representative images are shown. At the detecting point, EM image shows certain features of the testing paper, but the image was blurred (Fig. 12(i)). When the contacting force was approximately 0.05 N, the highest quality images were taken (Fig. 12(ii)). Additionally, clear images for mosaicking were also taken during the horizontal movement of the probe to the next point (Fig. 12(iii)). The 2 DoF point scanning continued for six points in total. Three additional cases showed minor differences in average detected contact force (0.62 N) when compared to the first case, and the optimum contact force (0.04 N) when compared to the determined force (0.05 N). However, there were no visible negative effects on the EM image quality.
Discussion
This paper shows a first proof-of-concept using a CDPM for safe and adequate contact force to acquire high-resolution EM images. While the detectable force sensitivity is high, the used healthy liver sample has shown to require an even higher force sensitivity for high-quality images. However, the proposed back-stepping technique effectively shows that the positional accuracy of the system is sufficiently high to acquire high-quality images. Additional bench-marking is required to fully validate the positional feed-forward accuracy of the end-effector. Hardware and software improvements can be implemented to further decrease friction and increase the detection sensitivity. It is noteworthy to mention that current results are found without any additional modeling in which the theoretical optimum tension distribution, dynamic and friction effects are taken into account. Implementing such models would further increase the force sensitivity of the system.
The large variability between different tissue pathologies, different tissue types, and different patients will make an adequate solution that is based on contact force alone as a metric for image quality unlikely. The combination of the current method and existing computer vision techniques would enable the system to autonomously and safely obtain high-quality EM images over large tissue sections. In such a case, no a priori information about the tissue is required to attain the optimum contact force for adequate images.
Further hardware developments will include a system with additional cables to allow for 5 Dof scanning of larger and complex surface area, while using the same algorithm. The force sensitivity is expected to reduce slightly, as the forces are distributed over more loadcells. Additional integration with the da Vinci and hybrid control methods can be used to increase the workspace of the instrument further. For clinical applicability, a deployable scaffold as used in the current CYCLOPS system needs to be incorporated.
While the research focuses primarily on EM imaging, it should be noted that the high force sensitivity can also be used for other clinical applications, including accurate control of a laser scalpel or a micro ultrasound probe. Haptic feedback and palpation are two other functionalities that the current system can be used for, giving the surgeon enhanced situational awareness and diagnostic tools.
Conclusion
An integration of a CDPM and a da Vinci instrument was introduced providing a viable solution for the safe and precise autonomous EM scanning. 0.2 and 0.6 N force sensitivity were found for 1 and 2 DoF image acquisition methods, respectively. In case of insufficient force sensitivity for specific tissues, the system is accurate enough to acquire high-quality images by gradually stepping back. As force sensing and accurate control have been technical challenges since the emergence of the surgical robots, this new instrument could offer an option not only for robotic EM but also for other applications which require force sensing and precise control.
References
Vitiello V, Lee S-L, Cundy TP, Yang GZ (2013) Emerging robotic platforms for minimally invasive surgery. IEEE Rev Biomed Eng 6:111–126
Kwartowitz DM, Herrell SD, Galloway RL (2007) Update: toward image-guided robotic surgery: determining the intrinsic accuracy of the daVinci-S robot. Int J Comput Assist Radiol Surg 1(5):301–304
Zhang L, Ye M, Giataganas P, Hughes M, Yang GZ (2017) Autonomous scanning for endomicroscopic mosaicing and 3D fusion. In: IEEE international conference on robotics and automation, pp 3587–3593
Jabbour JM, Saldua MA, Bixler JN, Maitland KC (2011) Confocal endomicroscopy: instrumentation and medical applications. Ann Biomed Eng 40(2):378–397
Latt WT, Newton RC, Visentini-Scarzanella M, Payne CJ, Noonan DP, Shang J, Yang GZ (2011) A hand-held instrument to maintain steady tissue contact during probe-based confocal laser endomicroscopy. IEEE Trans Biomed Eng 58(9):2694–2703
Noonan DP, Payne CJ, Shang J, Sauvage V, Newton R, Elson D, Darzi A, Yang GZ (2010) Force adaptive multi-spectral imaging with an articulated robotic endoscope. In: International conference on medical image computing and computer-assisted intervention, pp 245–252
Giataganas P, Hughes M, Yang GZ (2015) Force adaptive robotically assisted endomicroscopy for intraoperative tumour identification. Int J Comput Assist Radiol Surg 10:825–832
Lim WB, Yang G, Yeo SH, Mustafa SK, Chen IM (2019) A generic tension-closure analysis method for fully-constrained cable-driven parallel manipulators. In: IEEE international conference on robotics and automation, pp 2187–2192
Qiu J, Chen S, Chengyou D (2016) A systematic review of robotic-assisted liver resection and meta-analysis of robotic versus laparoscopic hepatectomy for hepatic neoplasms. Surg Endosc 30:862–875
Schneider C, Johnson SP, Walker-Samuel S, Gurusamy K, Clarkson MJ, Thompson S, Song Y, Totz J, Cook RJ, Desjardins AE, Hawkes DJ, Davidson BR (2016) Utilizing confocal laser endomicroscopy for evaluating the adequacy of laparoscopic liver ablation. Lasers Surg Med 48:299–310
Mylonas GP, Vitiello V, Cundy TP, Darzi A, Yang GZ (2014) CYCLOPS: A Versatile Robotic Tool for Bimanual Single-access and natural-orifice endoscopic surgery. In: IEEE international conference on robotics and automation, pp 2436–2442
Oude Vrielink T, Zhao M, Darzi A, Mylonas GP (2018) ESD CYCLOPS: a new robotic surgical system for GI surgery. Int Conf Rob Autom (in press)
Giannarou S, Ye M, Gras G, Leibrandt K, Marcus HJ, Yang GZ (2016) Vision-based deformation recovery for intraoperative force estimation of tool-tissue interaction for neurosurgery. Int J Comput Assist Radiol Surg 11:929–936
Acknowledgements
We would like to thank Alexandros Kogkas, Ming Zhao, Fernando Avila-Rencoret and Dr Stamatia Giannarou for their help with this research, and Lin Zhang and Khushi Vyas for their support with the da Vinci and Cellvizio systems, respectively.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 53674 KB)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Miyashita, K., Oude Vrielink, T. & Mylonas, G. A cable-driven parallel manipulator with force sensing capabilities for high-accuracy tissue endomicroscopy. Int J CARS 13, 659–669 (2018). https://doi.org/10.1007/s11548-018-1717-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-018-1717-7