Smart ROVLATIS: Control Architecture

Edin Omerdic, Daniel Toal, Sean Nolan, Hammad Ahmad

Mobile & Marine Robotics Research Centre

ECE Department, University of Limerick, Limerick, Ireland
(e-mail:{edin.omerdic}{daniel.toal}{sean.nolan}{hammad.ahmad}@ul.ie)

Abstract: This paper describes the control architecture of smart ROV LATIS. Smart ROVLATIS is a novel,
multi-mode of operation marine robotics vehicle, developed at Mobile & Marine Robotics Research
Centre (MMRRC), for operational flexibility in high-resolution near seabed survey from shallow inshore
waters out to the continental shelf edge. The vehicle can operate in surface-tow mode, surface-thrusted
mode and ROV operation mode. Special features of the system include: deployment interoperability for
small inshore boats and larger research vessel; built-in auto-tuning of low-level controllers; fault tolerant
thruster control in 6 DOF; onboard computer control enabling real-time disturbance reaction and topside
augmented reality system support. The vehicle can be used as a target platform in various scenarios
(research and scientific projects, inspections, seabed survey missions, vision experiments etc.).
Keywords: smart ROV, fault-tolerant control, control allocation, augmented reality.

bathymetric, sidescan and video imagery/maps. The team

1. INTRODUCTION
have worked with leased ROVs integrating precision
In recent years, autonomous underwater vehicles (AUVs) navigation and imaging sonar on the vehicles and
have been the focus of much research by the academic ship/topside controls (Grehan et al., 2006). Other activities
community (Lapierre and Jouvencel, 2008; 2007; Silvestre et include development of survey platforms (autonomous
al., 2009; Stutters et al., 2008) and major advances have been underwater vehicles (AUVs), ROVs and pontoons) for high-
made in Navigation, Guidance and Control of AUVs. resolution seabed mapping (Molnar et al., 2005, Toal et al.,
Elements of this AUV technology are now being adapted for 2006). The team has further developed real-time Virtual
use on new generations of ROV. (Bingham et al., 2006; Kim Underwater Laboratory (VUL) (Omerdic et al., 2006) and
et al., 2007) describe advances in ROV navigation strategies. real-time high-resolution sidescan sonar simulators (Riordan
(Saul and Tena, 2007; De Souza and Maruyama, 2007) detail and Toal, 2008) for use in laboratory testing, training and
ROV station keeping systems and (Rife and Rock, 2006) offshore operations support. The idea to integrate all these
detail the design and validation of a control law for the technologies into a unique system (MPPT Ring) has been
observation of Deep-Ocean Jellyfish with the MBARI proposed in (Omerdic et al., 2008).
Ventana ROV. (Caccia, 2006) describes vertical motion
Based on the experiences gained in these projects, the
control of the ROMEO ROV using laser triangulation and
challenges met, the solutions developed, the ideas proposed
optical correlation methods. (Negahdaripour and Pezhman,
and confirmed in simulation environment and the inherently
2006) describe a vision system on an ROV for automated
high costs associated with marine technology and offshore
ship-hull inspection, based on computing the necessary
operations, the MMRRC decided to develop a flexible multi-
information for positioning, navigation, and mapping of the
mode of operation survey class vehicle, ROVLATIS, with the
hull from stereo images. Other advanced functionality such as
main purpose to serve as a platform to test new approaches
automated fault management; monitoring of system
and applications in the field of ocean renewable energy.
components for faults and automatic fault compensation is
Design aspects and results from offshore test trials with
also being ported from AUVs to ROVs. (Soylu et al., 2007)
ROVLATIS have been already published (Toal et al., 2008,
details a thruster fault-tolerant control systems that will, in
Omerdic and Toal, 2009). This paper is focused on
the event of a single or multiple thruster failure, reallocate
description of the system control architecture, highlighting
thruster power across the remaining thrusters in order to
specific implementation issues.
maintain ROV course/trajectory.
The outline of the paper is given in the following. Short
Over the last five years, researchers in the Mobile & Marine
description of the overall hardware architecture is given in
Robotics Research Centre (MMRRC) at the University of
the second section. The third section describes the control
Limerick have been engaged in science collaborative and
architecture. Finally, the fourth section presents results of test
engineering led seabed survey projects, including technical -
trials.
design, integration and offshore support (Grehan et al., 2005,
Grehan et al., 2006) and survey operations carrying out
detailed survey projects acquiring high-resolution
 2. HARDWARE ARCHITECTURE
Control Cabin
Fig. 1 illustrates the overall hardware architecture, whose full
VIDEO
description can be found in Omerdic and Toal (2009). Short
Video Data Processing
overview of the hardware architecture is given in the Overlay
following.
Topside components include control cabin and winch with MULTIBEAM
Multibeam Data Processing Local GPS
umbilical. The Control Cabin provides housing for power
supply and control/supervision centre with computers, Time Synchro
monitors and auxiliary equipment. Four Ethernet networks SURVEY
have been implemented: CONTROL NETWORK ( , Sidescan Data Processing
navigation and control data), SURVEY NETWORK ( , INS- Adaptive Multi-Sonar
aiding sensors & sidescan), VIDEO NETWORK ( , video Controller Support Vessel
data) and MULTIBEAM NETWORK ( , multibeam data).
CONTROL USBL
The umbilical has 6/4 copper cores for AC/DC power and 8 Control
single mode optical fibres (currently 4 of them are used ± Visualisation Nav Devices
each network has a dedicated fibre core). Simulation (optional)
AC
Wetside components include 5 wet bottles, 8 thrusters, 4 PSU Power Source
cameras, 6 lights, multibeam, sidescan, sound velocity probe,
obstacle avoidance sensors, INS and a set of aiding sensors. DC AC

A set of aiding sensors is connected with Inertial Navigation Power Box

System (INS) through the Survey Bottle, including Doppler Winch
Velocity Log (DVL), depth sensor, USBL transponder and Signal Box Slip Ring
local GPS receiver (available only for surface operations). Ethernet ± FO Converters Tension Control
These sensors are used by INS to improve accuracy of ROV
position estimation. The output of INS is directly sent to the Umbilical
Control Bottle for control purposes through a serial interface. Umbilical
At the same time, sidescan data, raw data from sensors and Water Line
navigation data processed by INS are sent topside via the
SURVEY NETWORK.
DC Umbilical Bottle
All control-related components are mounted inside the Thruster Bottle Power Junction Box
Control Bottle. The PC EBX945 (mini PC) receives and DC Power Filters
Thrusters

processes ROV navigation data, transforms data into system HT/VT Interface
AC FO Cores
states using standard control frames (body-fixed and earth-
fixed), bundles data into clusters and shares it over the
CONTROL NETWORK. NI CompactRIO is used as a real-
Power Bottle
Multibeam

time controller and I/O interface with thrusters and lights.

Cameras
DC Power Supply Units
3. CONTROL ARCHITECTURE Ethernet - FO Converters
RS-232/485 Converter
In contrast to most common control architectures used in
modern ROV industry (where ROV is equipped with basic DC DC
I/O modules, while control synthesis is performed topside),
ROVLATIS is equipped with a full real-time embedded control Control Bottle Survey Bottle
system, which performs all necessary data processing and Ethernet Serial Server Ethernet Serial Servers
synthesis online, aboard the vehicle in real-time. Ethernet Switch Ethernet Switch
Relay Module Relay Module
The block diagram of the ROVLATIS control system is shown PC EBX945
in Fig. 2, while physical layout of software modules CompactRIO
(software distribution) is presented in Fig. 3. The ROVLATIS Light Interface
control system utilises control allocation approach, where the Thruster Interface INS Sidescan
actuator selection task (mapping of the total control efforts Aiding Sensors:
onto individual actuator settings) is separated from the o DVL
regulation task (design of total control efforts) in the control o Depth
design. The Mission Builder module, Arbitration module and Lights o USBL
Synthesis module perform the regulation task, while the o GPS2
Control Allocation module performs the actuator selection
task. Fig. 1. ROVLATIS overall hardware architecture.
The main objective of the Mission Builder module is to Navigation Data
transform the mission objective, pilot inputs and measured (Support Vessel)
navigation data into the desired ROV trajectory, i.e. to
formulate the trajectory planning problem. A description of
the Arbitration module is given in the following. In order to
achieve the desired trajectory under real-time constraints, a ControlPC
set of task executors (Exclusive Behaviours (only one active
at a time) and Collaborative Behaviours (many active at a App1: GUI App2: Master
Instrument Display Shared Variable Manager
time)) have been developed. Each of these task executors is
ROV Pilot Input Interface Remote Power Control
competing to control actuators. The control buffer concept
Operation Mode Selector Interface with Support
has been developed to provide transparency and easy fusion Vessel
Autotuning Setup
of different task executor demands. Each task executor LLC Monitoring Mission Planning
produces its own control cluster inside the control buffer.

Navigation Data Navigation Data Pilot Input

(ROV) (Support Vessel) Interface VisualisationPC

App: VR
Ethernet Switch Augmented Reality Display
Interface & Data Management (CONTROL NETWORK) Thruster Saturation Bound
Visualisation

Map Way Points Ethernet ļ)LEUH2SWLF

Mission Builder Builder Database
Water Line

Mission Planner Trajectory Planner Fibre Optic ļEthernet EBX945

App: PCEBX945
Arbitration Ethernet Switch Shared Variable Server
Collaborative Behaviours Autotuning
(CONTROL NETWORK)
Arbitration
Standard Control Cluster Coordinator

Exclusive Behaviours
Exclusive Standard NI CompactRIO
Mode Navigation Data
Switch Emergency Control Cluster App: IO_Host (RT Controller) (ROV)
Emergency Low-Level Controllers
Control Allocation
AUV
Operation
Mode App: IO_FPGA (FPGA)
ROV Mode Control Cluster
Switch ROV I/O Interface with Thrusters,
Leak Detectors and Lights

Fusion Obstacle Avoidance Control Cluster

Fig. 3. Software modules ± physical layout.

Winner Control Cluster

Fault Accommodation

HT Saturation Bounds VT Saturation Bounds Fault

Diagnosis
Disturbances

ROV
Low-Level Controllers
IJd nd Propulsion Rigid-Body
IJ LLC + Control Allocation
System Dynamics
IJVJ + Synthesis Control Allocation
Faults
Fig. 2. Control architecture.
A control cluster (see Fig. 4) consists of Virtual Joystick performance in the case of the time-varying SP vector.
components (to mimic direct controls generated by a virtual Vector SP Offset is used to avoid integrator saturation
pilot) and a set of settings for Low-Level Controllers (set problems. For example, the PID depth control algorithm has
points, feed-forward inputs and on/off switches to pure performance during the transition stage, since the
enable/disable individual controllers). The Virtual Joystick ROVLATIS is slightly positively buoyant. However, much
components are normalised surge, sway and heave forces and better performance is obtained using the PD control
roll, pitch and yaw moments. The Coordinator performs the algorithm with SP Offset set to a positive value (0.1m for
fusion of these control clusters into the Standard Control example shown in Fig. 6) to compensate steady-state error
Cluster. due to positive buoyancy of the vehicle. In this case, depth
change is fast, smooth, without overshoot and without steady-
The Operation Mode Switch is used to switch between AUV state error, as shown in Fig. 6. However, this approach works
Mode and ROV Mode. In AUV Mode, control algorithms are well only in the case of smooth sensor signals, like in the
exclusively used to create control actions (write values and case of ROVLATIS, where the depth sensor measurements are
set switches inside control clusters), while in ROV Mode the fused with the INS estimations using the Kalman filter.
pilot has full freedom to generate these actions. However, the
SLORW VKRXOG EH DZDUH QRW WR ³ILJKW´ ZLWK HQDEOHG ORZ-level
controller. For example, if the low-level heading controller is
enabled to keep a set point (desired heading) of 60°, any yaw
moment created by the pilot using an input device, such as a
joystick or gamepad, is considered as a disturbance by the
heading controller. Therefore, the controller will create
corresponding actions to reject this disturbance and keep the
heading at 60°.
The Exclusive Mode Switch is used to switch between
Standard and Emergency Control Clusters in AUV Mode. In
the case of any leakage (water penetration inside any bottle),
two Leakage Answer Modes are available. In Auto-Answer Fig. 4. Control Cluster.
mode the main state machine will activate the Emergency
State setting the Exclusive Mode Switch to Emergency,
setting the Operation Mode Switch to AUV Mode and
initiating automatic ROV recovery to the surface. In Manual-
Answer Mode the Operation Mode Switch is set to ROV
Mode and the pilot is informed of the leakage, but no other
automatic action is undertaken.
Finally, the output control cluster is (optionally) blended with
the Obstacle Avoidance Control Cluster to create the Winner
&RQWURO &OXVWHUWKHXOWLPDWH ³ERVV´ ZLWKH[FOXVLYHULJKWVWR
control the actuators. Fig. 5. Internal structure of LLC loop.

Inside the Synthesis module the Winner Control Cluster is )LQDOO\ LQ VSHFLDO ³$XWRWXQLQJ´ RSHUDWLRQ PRGH WKH
unbundled into Virtual Joystick components (vector IJVJ ) controller output is generated using a relay (see Fig. 5).
Between successive ROV missions, it is likely that some of
and the Low-Level Controller (LLC) cluster, which is used as the onboard instruments/sensors/equipment will be
one of inputs to the LLC loop (Vectors SP and FF in Fig. 5). added/removed/replaced, leading to changes in dynamic
Other inputs include ROV navigation data (vectors PV and properties of the ROV (mass, moments of inertia, drag
dPV/dt) and other parameters (vectors SP Offset, FF , KFF, properties, etc). Controllers optimally tuned for a particular
KP, KI, KD and Relay Amplitude). vessel configuration will not give the optimal performance in
the case of a change in configuration. Autotuning of LLC is
There is a single controller for each degree of freedom an advanced feature of the control system, yielding optimal
(DOF). Surge and Sway controllers are velocity controllers, controller performance, regardless of configuration changes.
while Heave, Roll, Pitch and Yaw are position controllers. It is recommended that the autotuning is performed at the
Each controller generates a manipulated variable MV to be beginning of a mission.
applied to drive actuators, in order to keep process variable
PV as close as possible to set point SP. Individual outputs are Two types of autotuning algorithms have been developed.
bundled into a vector of normalised forces and moments Velocity controllers are tuned by recording and utilising the
force-speed static characteristics. Autotuning of position
IJ LLC . If a controller is disabled (not active), the controllers, described in the following, utilises self-
corresponding MV is set to zero. Otherwise, the controller oscillations. Autotuning algorithms described in (Miskovic et
output is calculated as a normalised output of a modified PID al., 2006) have been expanded for 4 DOF controllers: Heave,
controller. Feed-forward (FF) input improves the tracking Roll, Pitch and Yaw. The autotuning process involves the
following steps: (1) Generate self-oscillations; (2) Wait for iteration method is activated in the second step. In this way,
transient stage to finish; (3) Measure amplitude and period of the hybrid approach is able to allocate the exact solution,
steady-state oscillations; and (4) Find new values of optimal in the l2 sense, inside the entire attainable command
controller gains using tuning rules. A novel set of tuning set.
rules for underwater applications has been developed, which
provides the optimal performance of low-level controllers in
the case of configuration changes and the presence of
disturbances (waves & sea currents).

Fig. 8. Control Allocation Express VI.

Fig. 6. Real-time monitoring of LLC signals. 4. ROVLATIS TEST TRIALS

Demo trials of ROVLATIS were undertaken off the west of
,UHODQG¶V&RQQHPDUDFRDVWOLQHIURP)HEUXDU\WR0DUFK
2009 ZLWK 59 ³&HOWLF ([SORUHU´ DQG in Limerick Dock in
the period from 1st ± 3rd July 2009 (with small ship
³6KDQQRQ 2QH´. The trials included alignment of INS and
DVL, auto-tuning of low-level controllers, testing of the
adaptive multi-sonar controller and fault-tolerant control
system etc. Selected results are reported here.

Fig. 7. Autotuning of Heave (Depth) controller.

Autotuning of the Heave LLC is shown in Fig. 7. The ROV is
oscillating (moving up and down) around a setpoint of 30m.
The final output IJ d of Synthesis module, which represents
the total control efforts to be exerted by thrusters, is obtained
by summation and normalisation of vectors IJVJ and IJ LLC .

The Control Allocation module in Fig. 2 performs the

actuator selection task. The thruster fault-tolerant control
system consists of two subsystems: (1) The Fault Diagnosis
Subsystem (FDS) uses fault detector units to monitor the
states of thrusters (Omerdic and Roberts, 2003). (2) The Fault
Accommodation Subsystem (FAS) uses information provided
by the FDS to accommodate faults and perform control
reallocation. The output of FAS controls the HT/VT
Saturation Bounds sliders. A hybrid control allocation
approach is implemented inside the LabVIEW Control
Allocation Express VI (Fig. 8). A hybrid approach for control Fig. 9. 3D Real-Time Augmented Reality Display: Path
allocation (Omerdic et al., 2004) is based on integration of following with faulty horizontal thruster.
the pseudoinverse and the fixed-point iteration method. It is
implemented as a two-step process. The pseudoinverse In order to test the performance of a fault-tolerant control
solution is found in the first step. Then the feasibility of the system, a number of test trials have been performed in
solution is examined analysing its individual components. If Limerick Dock. The main task imposed on ROV control
violation of actuator constraint(s) is detected, the fixed-point system was the path following with faulty thruster. The path
was defined with a set of way-points. Different faulty Molnar, L., Omerdic, E., and Toal, D. (2005). Design of
situations were artificially generated by changing values of advanced AUV for deployment close to seabed and other
HT/VT sliders in Fig. 8. Fig. 9 shows results from one of hazards. Ocean 2005 Europe IEEE Conference, Brest,
these tests. In this particular case, the front left horizontal France.
thruster was disabled and the ROV was commanded to Negahdaripour, S. and Pezhman, F. (2006) An ROV
follow the rectangular path defined by way points WP1 - Stereovision System for Ship-Hull Inspection. IEEE
WP4 in surface mode of operation, with constant forward Journal of Oceanic Engineering, 31, 551-564.
speed of 0.4 m/s. As indicated in Fig. 9, an excellent tracking Omerdic, E., Riordan, J., Molnar, L., and Toal, D. (2006).
performance has been achieved with only three working Virtual Underwater Lab: Efficient Tool for System
horizontal thrusters. Integration & UUV Control Development. 7th IFAC
Conference on Manoeuvring and Control of Marine
5. ACKNOWLEDGEMENTS Craft (MCMC 2006), Lisbon, Portugal.
Omerdic, E., Riordan, J., and Toal, D. (2008). MPPT Ring ±
The development of ROVLATIS and of the modelling and Multi-Purpose Platform Technologies for Subsea
operations support tools described has been supported by Operations. IFAC Workshop on Navigation, Guidance
funding under the Irish Marine Institute and the Marine RTDI and Control of Underwater Vehicles, Killaloe, Ireland.
Measure, Productive Sector Operational Programme, Omerdic, E. and Roberts, G. (2003). Thruster fault diagnosis
National Development Plan 2000 ± 2006 (PhD -05-004, INF- and accommodation for open-frame underwater vehicles.
06-013 and IND-05-03); Science Foundation Ireland under Control Engineering Practice, 12 (2004), pp. 1575-1598.
Grant Number 06/CP/E007 (Charles Parsons Energy Omerdic, E., Roberts, G. and Toal, D. (2004). Extension of
Research Awards 2006); HEA PRTLI 3 &4 and Enterprise Feasible Region of Control Allocation for Open-Frame
Ireland Commercialisation Fund Technology Development Underwater Vehicles. CAMS 2004, Ancona.
2007 projects ± MPPT Ring (CFTD/07/IT/313) and PULSE Omerdic, E. and D. Toal. (2009). Smart ROV LATIS: From
RT (CFTD/07/323). Design Concepts to Test Trials. In IFAC 8th Conference
on Manoeuvring and Control of Marine Craft
REFERENCES (MCMC'2009), Guarujá, Brazil.
Bingham, B., Mindell, D., Wilcox, T. and Bowen, A. (2006). Rife, J. H. and Rock, S. M. (2006) Design and Validation of a
Integrating Precision Relative Positioning Into Robotic Control Law for Observation of Deep-Ocean
JASON/MEDEA ROV Operations. Marine Technology Jellyfish. IEEE Transactions on Robotics, 22, 282-291.
Society Journal, 40, 80-89. Riordan, J., and Toal, D. (2008). Performance Optimised
Caccia, M. (2006). Laser-Triangulation Optical-Correlation Reverberation Modelling for Real-Time Synthesis of
Sensor for ROV Slow Motion Estimation. IEEE Journal Sidescan Sonar Imagery. IEEE Journal of Oceanic
of Oceanic Engineering, 31, 711-727. Engineering. In Press.
De Souza, E. C. and Maruyama, N. (2007). Intelligent UUVs: Saul, D. and Tena, I. (2007). BP's AUV Development
Some issues on ROV dynamic positioning. IEEE program, Long Term Goals - Short Term Wins. Oceans
Transactions on Aerospace and Electronic Systems, 43, 2007.
214-226. Silvestre, C., Cunha, R., Paulino, N. and Pascoal, A. (2009).
Grehan, A., Toal, D., and Brown, C. (2006). ROV A Bottom-Following Preview Controller for
Investigations of Cold Water Coral Habitats in the Autonomous Underwater Vehicles. IEEE Transactions
Porcupine/Rockall off the west coast of Irelad. The Irish on Control Systems Technology, 17, 257-266.
Scientist Yearbook, No 14, pp 96. Soylu, S., Buckham, B. J. and Podhorodeski, R. P. (2007).
Grehan, A., Wilson, M., Guinan, J., Riordan, J., Molnar, L., Robust Control of Underwater Vehicles with Fault-
Omerdic, E., Ullgren, J., Le Guilloux, E., Toal, D., and Tolerant Infinity-Norm Thruster Force Allocation.
Brown, C. (2005). ROV Investigations of Cold-Water Oceans 2007.
Coral Habitats along the Porcupine Bank Margin, West Stutters, L., Honghai, L., Tiltman, C. and Brown, D. J.
Coast of Ireland. 3rd International Symposium on Deep- (2008). Navigation Technologies for Autonomous
Sea Corals, Miami, Florida, USA. Underwater Vehicles. IEEE Transactions on Systems,
Kim, K., Lee, P.-M., Lee, K. C.-M., Jun, B.-H., Choi, K. H.- Man, and Cybernetics, Part C: Applications and
T., Lee, J.-H. and Ryu, S.-C. (2007). Underwater Reviews, 38, 581-589.
navigation based on the multiple sensor fusion for the Toal, D., Omerdic, E., Nolan, S., and Al-Shamma'a, A.
deep-sea UUV system, HEMIRE and HENUVY. (2006). Intervention AUVs for permanent seabed
OCEANS 2007 - Europe. observatories, addressing some of the key technical
Lapierre, L. and Jouvencel, B. (2008). Robust Nonlinear challenges. In Proceedings of SSC '06, IEEE Fourth
Path-Following Control of an AUV. Oceanic International Workshop on Scientific Use of Submarine
Engineering, IEEE Journal of, 33, 89-102. Cables and Related Technologies 2006, Dublin.
Miskovic, N., Z. Vukic, M. Barisic and B. Tovornik (2006). Toal, D., Omerdic, E., Nolan, S., Riordan, J., and Molnar, L.
Autotuning Autopilots for Micro ROVs. Proceedings of (2008). A Flexible Multi-Mode of Operation Survey
14th Mediterranean Conference on Control and Platform for Surface and Underwater Operations. IFAC
Automation (MED06), Ancona. Workshop on Navigation, Guidance and Control of
Underwater Vehicles, Killaloe, Ireland.