This work presents a preoperative microrobotic surgical simulation and planning application. The ... more This work presents a preoperative microrobotic surgical simulation and planning application. The main contribution is to support computer-aided minimally invasive surgery (MIS) procedure using untethered microrobots that have to navigate within the arterial networks. We first propose a fast interactive application (with endovascular tissues) able to simulate the blood flow and microrobot interaction. Secondly, we also propose a microrobotic surgical planning framework, based on the anisotropic Fast Marching Method (FMM), that provides a feasible pathway robust to biomedical navigation constraints. We demonstrate the framework performance in a case study of the treatment of peripheral arterial diseases (PAD).
This paper presents the endovascular navigation of a ferromagnetic microdevice using magnetic res... more This paper presents the endovascular navigation of a ferromagnetic microdevice using magnetic resonance imaging (MRI)-based predictive control. The concept was studied for the future development of microrobots designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow three-dimensional (3-D) navigation of a microdevice in blood vessels, namely: (i) vessel path extraction, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the navigation path of the microrobot into the blood vessel is extracted using the Fast Marching Method from the pre-operation images (3-D MRI imaging) to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the pre-computed path, a Model Predictive Controller is proposed for robust time-multiplexed navigation along a 3-D path in the presence of pulsative flow. The simulation results suggest the validation of the proposed image processing and control algorithms.
Microinjection is the highly efficient delivery method of exogenous materials into cells, and it ... more Microinjection is the highly efficient delivery method of exogenous materials into cells, and it has been widely used in biomedical research areas such as transgenics and genomics. However, this direct injection task is time consuming and laborious, resulting in low throughput and poor reproducibility. This paper describes a telerobotic shared control (TSC) framework for the microinjection with high manipulation efficiencies, in which a micromanipulator is controlled by the shared motion commands of both the human operator (direct manipulation) and the autonomous controller. To determine the optimal gains between the operator and the controller, we proposed a quantitative evaluation method using Fitts' and steering laws. The results showed that a 40%-60% weighting on the human operator produced better performance for both speed and accuracy of task completion, and suggested that some level of automation or human involvement is important for microinjection tasks.
This paper presents real-time MRI-based control of a ferromagnetic microcapsule for endovascular ... more This paper presents real-time MRI-based control of a ferromagnetic microcapsule for endovascular navigation. The concept was studied for future development of microdevices designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow navigation of a microdevice in blood vessels, namely: (i) vessel path planner, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the position recognition of the microrobot into the blood vessel is extracted using Frangi vesselness filtering from the pre-operation images. Then, a set of minimal trajectory is predefined, using FMM, to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the pre-computed path, a GPC is proposed for robust time-multiplexed navigation along a 2D path in presence of pulsative flow. The simulation results suggest the validation of the proposed image processing and control algorithms. A series of disturbances introduced in the presence and absence of closed-loop control affirms the robustness and effectiveness of this predictive control system.
The propulsion of nano-ferromagnetic objects by means of MRI gradients is a promising approach to... more The propulsion of nano-ferromagnetic objects by means of MRI gradients is a promising approach to enable new forms of therapy. In this work, necessary techniques are presented to make this approach work. This includes path planning algorithms working on MRI data, ferromagnetic artifact imaging and a tracking algorithm which delivers position feedback for the microdevice and a propulsion sequence to enable interleaved magnetic propulsion and imaging. Using a dedicated software environment integrating path-planning methods and real-time tracking, a clinical MRI system is adapted to provide this new functionality for potential controlled interventional targeted therapeutic applications. Through MRI-based sensing analysis, this paper aims to propose a framework to plan a robust pathway to enhance the navigation ability to reach deep locations in human body. The proposed approaches are validated with different experiments.
This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based p... more This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based predictive control. The concept was studied for future development of microrobot designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow 3D navigation of a microdevice in blood vessels, namely: (i) vessel path extraction, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the navigation path of the microrobot into the blood vessel is extracted using Fast Marching Method (FMM) from the pre-operation images (3D MRI imaging) to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the precomputed path, a Model Predictive Controller (MPC) is proposed for robust time-multiplexed navigation along a 3D path in presence of pulsative flow. The simulation results suggest the validation of the proposed image processing and control algorithms.
This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based p... more This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based predictive control. The concept was studied for future development of microrobot designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow 3D navigation of a microdevice in blood vessels, namely: (i) vessel path extraction, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the navigation path of the microrobot into the blood vessel is extracted using Fast Marching Method (FMM) from the pre-operation images (3D MRI imaging) to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the pre-computed path, a Model Predictive Controller (MPC) is proposed for robust navigation along a 3D path. The simulation results suggest the validation of the proposed image processing and control algorithms.
In this paper we propose a new on-line sensor self-calibration framework. The approach is to cons... more In this paper we propose a new on-line sensor self-calibration framework. The approach is to consider the sensor/robot interaction that links the sensor signal variations to the robot velocity. By on-line calibration, we mean only the actual measurements are used to perform calibration under the condition that the interaction matrix is analytically known. This allows us to propose a very simple and versatile formulation of sensor parameter calibration. Various sensors can be considered, and calibration from different sensory data may be considered within the same process. Intrinsic and extrinsic parameters estimation are formulated as a nonlinear minimization problem the jacobian of which can be expressed analytically from the sensor model. Simulations and experiments are presented for a camera observing four points, showing good results in the case of separated intrinsic and extrinsic calibration, and illustrating the possible limitations in the case of simultaneous estimation.
We address the problem of multi-sensor-based navigation in a cluttered environment for a non-holo... more We address the problem of multi-sensor-based navigation in a cluttered environment for a non-holonomic robot. To perform such a successful and safe navigation, three controllers realizing, respectively, nominal vision-based navigation, obstacle bypassing and occlusion avoidance have been designed using the task function approach. Then it suffices to sequence them to realize the complete mission. To guarantee the control continuity when switching between two successive controllers, two sequencing approaches have been used and compared. Simulation results validate our work.
This paper deal with the problem of executing a vision-based task in an unknown environment. Duri... more This paper deal with the problem of executing a vision-based task in an unknown environment. During such a task, two unexpected events may occur: the image data loss due to a camera occlusion and the robot collision with obstacles. We first propose a method allowing to compute the visual data when they are totally lost, before addressing the obstacle avoidance problem. Then, we design a sensor-based control strategy to perform safely vision-based tasks despite complete loss of the image. Simulation and experimental results validate our work.
In this paper, we address the problem of estimating image features whenever they become unavailab... more In this paper, we address the problem of estimating image features whenever they become unavailable during a vision-based navigation task. The method consists in analytically integrating the relation linking the visual features motion in the image to the 3D camera motion. Simulation results validate our work.
This paper presents a redundancy-based scheme allowing to avoid both occlusions and obstacles for... more This paper presents a redundancy-based scheme allowing to avoid both occlusions and obstacles for a mobile robot performing a vision-based task in a cluttered environment. We consider the model of a cart-like robot equipped with ultrasonic sensors and a camera mounted on a pan-platform. The proposed method relies on the continuous switch between several controllers depending on the environment. Experimental results validating this approach are given at the end of the paper.
In this paper, we address the problem of estimating image features whenever they become unavailab... more In this paper, we address the problem of estimating image features whenever they become unavailable during a vision-based task. The method consists in using numerical algorithm to compute the lacking data and allows to treat both partial and total visual features loss. Simulation and experimental results validate our work for two different visual-servoing navigation tasks. A comparative analysis allows to select the most efficient algorithm.
This paper presents a sensor-based controller allowing to visually drive a mobile robot towards a... more This paper presents a sensor-based controller allowing to visually drive a mobile robot towards a target while avoiding visual features occlusions and obstacle collisions. We consider the model of a cart-like robot equipped with proximetric sensors and a camera mounted on a pan-platform. The proposed method relies on the continuous switch between three controllers realizing respectively the nominal vision-based task, the obstacle by passing and the occlusion avoidance. Simulation results are given at the end of the paper.
This work presents a preoperative microrobotic surgical simulation and planning application. The ... more This work presents a preoperative microrobotic surgical simulation and planning application. The main contribution is to support computer-aided minimally invasive surgery (MIS) procedure using untethered microrobots that have to navigate within the arterial networks. We first propose a fast interactive application (with endovascular tissues) able to simulate the blood flow and microrobot interaction. Secondly, we also propose a microrobotic surgical planning framework, based on the anisotropic Fast Marching Method (FMM), that provides a feasible pathway robust to biomedical navigation constraints. We demonstrate the framework performance in a case study of the treatment of peripheral arterial diseases (PAD).
This paper presents the endovascular navigation of a ferromagnetic microdevice using magnetic res... more This paper presents the endovascular navigation of a ferromagnetic microdevice using magnetic resonance imaging (MRI)-based predictive control. The concept was studied for the future development of microrobots designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow three-dimensional (3-D) navigation of a microdevice in blood vessels, namely: (i) vessel path extraction, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the navigation path of the microrobot into the blood vessel is extracted using the Fast Marching Method from the pre-operation images (3-D MRI imaging) to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the pre-computed path, a Model Predictive Controller is proposed for robust time-multiplexed navigation along a 3-D path in the presence of pulsative flow. The simulation results suggest the validation of the proposed image processing and control algorithms.
Microinjection is the highly efficient delivery method of exogenous materials into cells, and it ... more Microinjection is the highly efficient delivery method of exogenous materials into cells, and it has been widely used in biomedical research areas such as transgenics and genomics. However, this direct injection task is time consuming and laborious, resulting in low throughput and poor reproducibility. This paper describes a telerobotic shared control (TSC) framework for the microinjection with high manipulation efficiencies, in which a micromanipulator is controlled by the shared motion commands of both the human operator (direct manipulation) and the autonomous controller. To determine the optimal gains between the operator and the controller, we proposed a quantitative evaluation method using Fitts' and steering laws. The results showed that a 40%-60% weighting on the human operator produced better performance for both speed and accuracy of task completion, and suggested that some level of automation or human involvement is important for microinjection tasks.
This paper presents real-time MRI-based control of a ferromagnetic microcapsule for endovascular ... more This paper presents real-time MRI-based control of a ferromagnetic microcapsule for endovascular navigation. The concept was studied for future development of microdevices designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow navigation of a microdevice in blood vessels, namely: (i) vessel path planner, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the position recognition of the microrobot into the blood vessel is extracted using Frangi vesselness filtering from the pre-operation images. Then, a set of minimal trajectory is predefined, using FMM, to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the pre-computed path, a GPC is proposed for robust time-multiplexed navigation along a 2D path in presence of pulsative flow. The simulation results suggest the validation of the proposed image processing and control algorithms. A series of disturbances introduced in the presence and absence of closed-loop control affirms the robustness and effectiveness of this predictive control system.
The propulsion of nano-ferromagnetic objects by means of MRI gradients is a promising approach to... more The propulsion of nano-ferromagnetic objects by means of MRI gradients is a promising approach to enable new forms of therapy. In this work, necessary techniques are presented to make this approach work. This includes path planning algorithms working on MRI data, ferromagnetic artifact imaging and a tracking algorithm which delivers position feedback for the microdevice and a propulsion sequence to enable interleaved magnetic propulsion and imaging. Using a dedicated software environment integrating path-planning methods and real-time tracking, a clinical MRI system is adapted to provide this new functionality for potential controlled interventional targeted therapeutic applications. Through MRI-based sensing analysis, this paper aims to propose a framework to plan a robust pathway to enhance the navigation ability to reach deep locations in human body. The proposed approaches are validated with different experiments.
This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based p... more This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based predictive control. The concept was studied for future development of microrobot designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow 3D navigation of a microdevice in blood vessels, namely: (i) vessel path extraction, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the navigation path of the microrobot into the blood vessel is extracted using Fast Marching Method (FMM) from the pre-operation images (3D MRI imaging) to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the precomputed path, a Model Predictive Controller (MPC) is proposed for robust time-multiplexed navigation along a 3D path in presence of pulsative flow. The simulation results suggest the validation of the proposed image processing and control algorithms.
This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based p... more This paper presents an endovascular navigation of a ferromagnetic microdevice using a MRI-based predictive control. The concept was studied for future development of microrobot designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow 3D navigation of a microdevice in blood vessels, namely: (i) vessel path extraction, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the navigation path of the microrobot into the blood vessel is extracted using Fast Marching Method (FMM) from the pre-operation images (3D MRI imaging) to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the pre-computed path, a Model Predictive Controller (MPC) is proposed for robust navigation along a 3D path. The simulation results suggest the validation of the proposed image processing and control algorithms.
In this paper we propose a new on-line sensor self-calibration framework. The approach is to cons... more In this paper we propose a new on-line sensor self-calibration framework. The approach is to consider the sensor/robot interaction that links the sensor signal variations to the robot velocity. By on-line calibration, we mean only the actual measurements are used to perform calibration under the condition that the interaction matrix is analytically known. This allows us to propose a very simple and versatile formulation of sensor parameter calibration. Various sensors can be considered, and calibration from different sensory data may be considered within the same process. Intrinsic and extrinsic parameters estimation are formulated as a nonlinear minimization problem the jacobian of which can be expressed analytically from the sensor model. Simulations and experiments are presented for a camera observing four points, showing good results in the case of separated intrinsic and extrinsic calibration, and illustrating the possible limitations in the case of simultaneous estimation.
We address the problem of multi-sensor-based navigation in a cluttered environment for a non-holo... more We address the problem of multi-sensor-based navigation in a cluttered environment for a non-holonomic robot. To perform such a successful and safe navigation, three controllers realizing, respectively, nominal vision-based navigation, obstacle bypassing and occlusion avoidance have been designed using the task function approach. Then it suffices to sequence them to realize the complete mission. To guarantee the control continuity when switching between two successive controllers, two sequencing approaches have been used and compared. Simulation results validate our work.
This paper deal with the problem of executing a vision-based task in an unknown environment. Duri... more This paper deal with the problem of executing a vision-based task in an unknown environment. During such a task, two unexpected events may occur: the image data loss due to a camera occlusion and the robot collision with obstacles. We first propose a method allowing to compute the visual data when they are totally lost, before addressing the obstacle avoidance problem. Then, we design a sensor-based control strategy to perform safely vision-based tasks despite complete loss of the image. Simulation and experimental results validate our work.
In this paper, we address the problem of estimating image features whenever they become unavailab... more In this paper, we address the problem of estimating image features whenever they become unavailable during a vision-based navigation task. The method consists in analytically integrating the relation linking the visual features motion in the image to the 3D camera motion. Simulation results validate our work.
This paper presents a redundancy-based scheme allowing to avoid both occlusions and obstacles for... more This paper presents a redundancy-based scheme allowing to avoid both occlusions and obstacles for a mobile robot performing a vision-based task in a cluttered environment. We consider the model of a cart-like robot equipped with ultrasonic sensors and a camera mounted on a pan-platform. The proposed method relies on the continuous switch between several controllers depending on the environment. Experimental results validating this approach are given at the end of the paper.
In this paper, we address the problem of estimating image features whenever they become unavailab... more In this paper, we address the problem of estimating image features whenever they become unavailable during a vision-based task. The method consists in using numerical algorithm to compute the lacking data and allows to treat both partial and total visual features loss. Simulation and experimental results validate our work for two different visual-servoing navigation tasks. A comparative analysis allows to select the most efficient algorithm.
This paper presents a sensor-based controller allowing to visually drive a mobile robot towards a... more This paper presents a sensor-based controller allowing to visually drive a mobile robot towards a target while avoiding visual features occlusions and obstacle collisions. We consider the model of a cart-like robot equipped with proximetric sensors and a camera mounted on a pan-platform. The proposed method relies on the continuous switch between three controllers realizing respectively the nominal vision-based task, the obstacle by passing and the occlusion avoidance. Simulation results are given at the end of the paper.
Uploads
Papers by David Folio