Remote Operations Of Robot

A Seminar Report

Abstract
The use of teleoperated robots is increasing substantially. The possibility of controlling these machines remotely in dangerous or inaccessible environments like outer space, deep water, nuclear or biologically chemically toxic environments, mines, construction sites and fires proved to be a powerful resource. However the successful requirements for telerobotics operations are still broad and imprecise. With a broad spectrum of immersive displays and network bandwidths increasing every day these requirements are becoming extremely precious for the design and implementation of telerobotics systems. Telerobotics actually result from the merger of two originally separate areas that are teleoperation and robotics. Indeed, autonomous robotics are not yet fully developed, the robot must now be operated remotely by a human operator. We must therefore take into account the principles developed in teleoperation. However, as the robot can perform basic tasks independently, it is called telerobotics rather than teleoperation. Telerobotics has applications wherever man has difficulty working directly (hostile environment, too far or too small places) and the tasks are sufficiently complex or unpredictable to be an obstacle to full automation. The main condition for the development of telerobotics is the ability to compete with the direct intervention of humans or the use of an automated highly specialized. In the first case, the advantage of telerobotics is firstly the replacement of human labour painful or dangerous by another, more secure and more comfortable. In the second case, we must show the interest of a material more versatile than the automatic system dedicated to the proposed application.

Table of Contents Chapters

1. Introduction
1.1 Interfaces 1.2 The Human Role in Telerobotics 1.3 Applications

2. History 3. Literature survey

3.1 Difference between Telerobotics and Autonomous Robots 3.2 Wireless Remote Control

4. General Structure of a Telerobotic System

4.1 Operator Environment 4.2 Remote Environment

5. Case Study 6. Conclusion 7. References

1. Introduction
Telerobotics is the area of robotics concerned with the control of robots from a distance, chiefly using wireless connections (like Wi-Fi, Bluetooth, the Deep Space Network, and similar), "tethered" connections, or the Internet. It is a combination of two major subfields, teleoperation and telepresence. Telerobots require an additional component to wired connections, also remote sensory feedback. Most telerobotic systems require a remote camera or some sort of interactive component to sense what is happening on the remote end of the system. WiFi may be used within the confines of your home or office. This could control the robot and provide a video feed. A bomb disposal robot may use bluetooth or a similar, secure, high-powered radio frequency to control and view the robot in dangerous environments from a safe distance. Devices that are located in very distant locations and/or harsh environments may need to rely on radio telemetery for basic instructions, but nonvisual feedback to guide the vehicle; the distance between the controller and the device may be too great to allow signals to travel between the communicators effectively. The field of telerobotics grew out of the need to perform operations where it is difficult to place a human being, due to constraints such as cost, safety or time. Telerobotic systems need to be able to perform tasks that a human would normally do. Due to limitations in robot autonomy, this often has to be achieved by using human operators to control the remote robot (via a communication link). Such a system is a telerobot. The human operator is responsible for high level control such as planning and perception, while the robot performs the low level instructions at the remote site. An important property of a telerobotic system is the method by which it is controlled. Conway (Conway, 1990) defines the following classification of telerobotic control schemes: Direct continuous control The remote device follows the inputs from the controller; also known as master/slave control. Shared continuous control Control is at a higher level than direct position servoing; i.e. The device may vary from course if it encounters an obstacle. Discrete command control The controller is able to carry out discrete commands without intervention. This implies a higher level of capability in the remote portion of the controller as it must be able to carry out the command without help. Supervisory control The remote device operates in a largely autonomous mode and only interacts with the human when it encounters an unexpected situation. Learning control The remote device is given intelligence that allows it to learn from human inputs and sensor information and subsequently to generate behaviour in similar situations without human intervention. An aspect of web telerobot systems that affects the choice of control scheme is time delay. The Internet protocols do no guarantee a maximum delay for a message to be carried across a

network link, which means that the control scheme must work under variable (and possibly large) time delays. Continuous control is not well suited, as it is prone to instability problems under time delay ( Sheridan,1993). Shared continuous control is less sensitive to these problems and has been demonstrated over the Internet (Tzyn-Jong and Brady,1998; Brady and Tzyn-Jong,1998,1999), but only on short, high bandwidth Internet links. Discrete command control schemes and above are free of any time delay based instability problems as all closed loop control is performed locally. They are therefore the most appropriate choice for web telerobotic systems.

1.1 Interfaces
A telerobotic interface can be as simple as a common MMK (monitor-mouse-keyboard) interface. While this is not immersive, it is inexpensive. Telerobotics driven by internet connections are often of this type. A valuable modification to MMK is a joystick, which provides a more intuitive navigation scheme for planar robot movement. Dedicated telepresence setups utilize a head mounted display with either single or dual eye display, and an ergonomically matched interface with joystick and related button, slider, trigger controls. Future interfaces will merge fully immersive virtual reality interfaces and port real-time video instead of computer-generated images. Another example would be to use an omnidirectional treadmill with an immersive display system so that the robot is driven by the person walking or running. Additional modifications may include merged data displays such as Infrared thermal imaging, real-time threat assessment, or device schematics.

1.2 The Human Role in Telerobotics

Telerobotic systems allow human operators to properly interact with a telerobot to telemanipulate objects located in a remote environment. This means that human actions are extended to remote locations allowing the execution of complex tasks and avoiding risky situations for the human operator. In a telerobotic system the human operator plays an important role. He perceives information from the remote environment through the human system interface and acts accordingly by sending commands to the remote devices. Thus the human system interface has two important functions; first, it has to excite the operator senses so as to show the status of the executed task in the remote environment and second, it has to process the operator commands in order to properly control remote devices. Multi-modal1 commands are generated by the operator at his or her working site by means of the human system interface using motion, force, voice or symbolic inputs. Such commands are transmitted to the telerobot in order to perform the remote task. Sensors are placed at the remote site to gather data from the task which is then transmitted back to the operator and displayed by the multi-modal human system interface. Designing of multi-modal human system interfaces is one of the key challenges in telerobotics. Vision, audition, and haptics are senses excited by the multi-modal interface. It is thereby important to take into account human perception capabilities so as to obtain a better interaction. Telepresence is one of the key factors that enhances performance of a telerobotic system. Telepresence means that the information about the remote environment is displayed to the operator in a natural manner, which implies a feeling of presence at the remote site. A good degree of telepresence guarantees the feasibility of the required manipulation task.

1.3 Applications
Telerobotics for Space With the exception of the Apollo program most space exploration has been conducted with telerobotic space probes. Most space-based astronomy, for example, has been conducted with

telerobotic telescopes. The Russian Lunokhod-1 mission, for example, put a remotely-driver rover on the moon, which was driven in real time (with a 2.5-second lightspeed time delay) by human operators on the ground. Robotic planetary exploration programs use spacecraft which are programmed by humans at ground stations, essentially achieving a long time-delay form of telerobotic operation. Recent noteworthy examples include the Mars exploration rovers (MER) and the Curiosity rover. In the case of the MER mission, the spacecraft and the rover operated on stored programs, with the rover drivers on the ground programming each day's operation. The International Space Station (ISS) uses a two-armed telemanipulator called Dextre. More recently, a humanoid robot Robonaut has been added to the space station for telerobotic experiments. NASA has proposed use of highly-capable telerobotic systems for future planetary exploration using human exploration from orbit. In a concept for Mars Exploration proposed by Landis, a precursor mission to Mars could be done in which the human vehicle brings a crew to Mars, but remains in orbit rather than landing on the surface, while a highly-capable remote robot is operated in real time on the surface. Such a system would go beyond the simple long time delay robotics and move to a regime of virtual telepresence on the planet. One study of this concept, the Human Exploration using Real-time Robotic Operations (HERRO) concept, suggested that such a mission could be used to explore a wide variety of planetary destinations. Marine Applications Marine remotely operated vehicles (ROVs) are widely used to work in water too deep or too dangerous for divers. They repair offshore oil platforms and attach cables to sunken ships to hoist them. They are usually attached by a tether to a control center on a surface ship. The wreck of the Titanic was explored by an ROV, as well as by a crew-operated vessel. Telemedicine Additionally, a lot of telerobotic research is being done in the field of medical devices, and minimally invasive surgical systems. With a robotic surgery system, a surgeon can work inside the body through tiny holes just big enough for the manipulator, with no need to open up the chest cavity to allow hands inside. Other Telerobotic Applications Remote manipulators are used to handle radioactive materials. Telerobotics has been used in installation art pieces; Telegarden is an example of a project where a robot was operated by users through the Web.

2. History
Teleoperator systems were developed in the mid- 1940s to create capabilities for handling highly radioactive material. Teleoperators allowed a human operator to handle radioactive material from a workroom separated by a one meter thick, radiation-absorbing concrete wall from the radioactive environment. The operator could observe the task scene through radiation resistant viewing ports in the wall. The development of teleoperators for the nuclear industry culminated in the introduction of bilateral force-reflecting master-slave manipulator systems. In these very successful systems, the slave arm at the remote site is mechanicallyor electrically coupled to the geometrically identical or similar master arm handled by the operator and follows the motion of the master arm. The coupling between the master and slave arms is a two-way coupling: inertia or work forces exerted on the slave arm can back-drive the master arm, enabling the operator to feel the forces that are acting on the slave arm. Force information available to the operator is an essential requirement for dexterous control of remote manipulators, since general-purpose manipulation consists of a series of well controlled contacts between handling device and objects and also implies the transfer of forces and torques from the handling device to objects. The first web telerobots came into existence in 1994. These were the robots of Goldberg (Goldberg

et al., 1995a) and Taylor (Taylor and Trevelyan, 1995). Since then the number of web telerobots has been growing rapidly. Although initially considered a novelty, improvements in the underlying Internet technology have meant that more and more devices are being controlled or monitored via the web. This chapter documents some of the different web telerobots that have been developed and discusses how they have been implemented. This thesis continues the Australias Telerobot on the Web project started by Taylor (Taylor, 1999) in 1994. At the start of Taylors work there were no web telerobots in existence. By the time his system came online in 1994. Goldbergs (Goldberg et al., 1995a) Mercury project was also in operation. Taylors system allowed users to control a 6 degree of freedom arm located above a table which had wooden blocks placed on it. Users could send move requests to the robot via a web page which included images of robot workspace. The system used the common gateway interface to generate dynamic web pages which showed the new state of the robot after each move was made. The system was extremely popular and can be viewed as one of the pioneering web telerobot systems. A very similar system is now installed in the Carnegie Science museum. The system is discussed in more detail in the next chapter. Web Interface for Telescience (WITS) NASA has developed a web interface for Telescience (WITS) for controlling remote vehicles on planets such as Mars and Saturn. WITS is being developed for use in the 2003 and 2005 rover missions to Mars for distributed rover command generation. WITS was also available in demonstration from for the 1997 Mars Pathfinder mission. The Pathfinder interface only allowed control and observation of a simulated rover but used real data from the Mars mission. Berkeley Goldberg et al. (1995b,a, 2000) have produced a number of Internet controlled devices. The first of these was the Raiders Robot believed to be the first robot controlled via a web browser. The robot was located over a dry-earth surface allowing users to direct short bursts of compressed air onto the surface using the pneumatic system. Thus robot operators could excavate regions within the environment by positioning the arm, delivering a burst of air, and viewing the image of the newly cleared region. The system used CGI to interface between the webserver and physical devices. The Raiders project is no longer running. Xavier Xavier (Simmons et al, 1999) is a mobile robot at Carnegie Mellon University (CMU). As it is a mobile robot it is powered by batteries, which need to be recharged and are expensive to replace. As a result, the robot is online for only a few hours a day (19:00 to 21:00 GMT). The web interface was conceived as a way of testing navigation algorithms of the robot. Initially intended as a short test period, the project has now been running for 4 years. This is due to the surprising popularity of the project.

3. Literature survey 3.1 Difference between Telerobotics and Autonomous Robots

Telerobotics is similar to remote control in that the operator is controlling the device without physically touching it. In the case of telerobotics, however; there is no physical connection to the remotely operated system, at all. The WiFi controlled robots like the Mars rover vehicles and police bomb disposal robots are all controlled by wireless methods. Telerobots require an additional component to wired connections, also remote sensory feedback. Most telerobotic systems require a remote camera or some sort of

interactive component to sense what is happening on the remote end of the system. WiFi may be used within the confines of your home or office. This could control the robot and provide a video feed. A bomb disposal robot may use bluetooth or a similar, secure, high-powered radio frequency to control and view the robot in dangerous environments from a safe distance. Devices that are located in very distant locations and/or harsh environments may need to rely on radio telemetery for basic instructions, but non-visual feedback to guide the vehicle; the distance between the controller and the device may be too great to allow signals to travel between the communicators effectively. Autonomous systems operate by making decisions based on programming and sensory feedback. Line following robots, color sorters, and even the Roomba vacuum cleaner robots are all autonomous robots. They are programmed to make decisions based on sensory feedback of their surroundings. Many robots have a combination of two or all three of the styles listed here. The Mars rover is telerobotic, but may have a collection component where it seeks out a particular type of soil to sample, without human interaction, and reports back. There are telerobots designed to independently map out locations without human feedback but have a wired or wireless connection to occasionally phone in information. So dont get too concerned if you find it difficult to name a robot that is solely of one type or another. For the purposes of the merit badge, be aware of the three types of robots, and be able to explain some of their similarities and differences. If you name a Mars rover as a telerobot, but later learn of features that make it autonomous as described above, its okay. It can be both. Thats one of the things that makes robotics fun!

3.2 Wireless Remote Control

IR IR remote control is the best known form of wireless remote control. It's cheap and reliable, but limited to line-of-sight communication. Complete IR-receiver modules, like the TSOP1736, are available cheaply and can be interfaced with most controllers without much extra components. TV remote controls using RC5 (Phillips) can be used with such modules. If you want a faster data link, IRDA components could boost it significantly. Bluetooth and Wifi have replaced it on modern laptops, but IRDA components are still available. RF RF is widely known in model race cars, Wifi, and various other applications. These days complete RF transmitter/receiver modules are available at reasonable low prices. These modules are very easy to use and have ranges of around 100m depending on their environment. RF remote controls for high end model race cars have larger range but are much more expensive and limited in their use. While it is definitely possible to build RF transmitters from scratch, this is not advisable. Radio frequencies are strictly governed and building a transmitter that uses the wrong frequency quickly leads to a fine or worse. Know what you're allowed to do, before building one of these. It is possible to use a wireless telephone to provide an RF connection to your robot. The major restriction being data rates limited to 9.6kbaud or so. Speech Recognition In essence speech recognition is a form of remote control. Probably one of the hardest forms of remote control, but also one of the most impressive ones. Although today there are modules that contain a full speech recognition system capable of learning a dozen commands, those systems are still very limited as they can't handle sentences (just commands), need to be trained before they are useful and usually can only be used by one person.

Sound Sound can also be used as remote control, generating a tone of a particular frequency isn't hard, building a receiver to detect this tone isn't too difficult either. Sounds like whistling and clapping hands have been used for remote control before (e.g. the keyring which makes a sound when you whistle).

4. General Structure of a Telerobotic System

A telerobotic system is comprised of two main parts; the operator environment and the remote environment. Both environments are linked by a communication channel that transmits commands from the operator to the remote devices and sends back information of the remote task to the operator. The operator environment is made up of a multi-modal human system interface, which the operator uses in order to control the remote devices. The remote environment consists of teleoperated devices, sensors and objects that take part in the teleoperation task. Each environment contains processing modules which have double functions: first, to transform data transmitted by the communications channel and second, to execute the corresponding local control loops. A central issue related to the design and operation of telerobotic systems is the degree of coupling between the human operator and the remote robot. It is generally classified as weak or strong. If the operator gives symbolic commands to the robot by pushing buttons and watching the resulting action in the remote environment, its coupling is rather weak. Some degree of intelligence is required for a remote robot to execute such symbolic commands. The coupling is comparably strong for the kinesthetic modality in a bilateral teleoperation scenario. Commonly, the motion (and/or force) of the human operator is measured, communicated and used as a set-point for teleoperator motion (and/or force) controller. On the other hand, forces (motions) of the teleoperator in the remote environment are sensed, communicated and fed back to the human operator through the multi-modal human system interface. The degree of coupling is thereby related to the control distribution between operator and remote robot controller. Literature on telerobotics distinguishes among shared, cooperative, supervisory, and bilateral control.

4.1 Operator Environment

The human system interface plays an important role in a telerobotic system. It provides input devices that are used to generate operator commands and display devices that are used to monitor the interaction between remote robot and environment. Telerobot commands are generated by input devices that identify the operator actions. According to the control mode, commands have to be processed to a greater or lesser degree before they are transmitted to the remote environment. For example, when an operator executes a guidance task using a master-slave system, i.e. with strong coupling, motion (force) commands are continuously processed. They could be scaled or transformed to different coordinates. This is an example for a rather simple processing. More complex processing would be required if commands were symbolic, like e.g.picking an object. Symbolic commands have to be transformed - on operator or remote site - to the corresponding sequence of remote device actions. Simultaneously, multi-modal sensor information is received from the remote environment. This multi-modal feedback consists of 2D or 3D visual, mono/stereo acoustic, haptic (force, motion, tactile, temperature) and symbolic information which is generated by feedback information processors and displayed by the corresponding interface devices. The purpose of a feedback device is to excite the operators senses in order to show him the remote task status. Force feedback master-arms and stereoscopic screens are typical examples of devices used as human system interfaces. The former informs the operator about applied contact forces during telemanipulation,

while the latter gives a 3D visual impression of the remote environment.

4.2 Remote Environment

When the operator commands reach the remote environment, the task processor transforms them into actions. Once again, the complexity of data processing depends on the type of command and the degree of coupling. Complex data processing is required when the operator and the telerobot are weakly coupled, i.e. in cases where the robot has some degree of autonomy or when the robot only receives symbolic commands. Simple data processing is required when the operator and the telerobot are strongly coupled. The information captured by sensors is used in obtaining data from the remote task and sending them to the operator environment via the communication channel. Computer vision recognition and object localization algorithms are good examples of sensor processing. They obtain information from the objects located at the remote site and thereby define spatial positions of telemanipulated objects. Local control loops that are executed at the remote site, ensure the motion (force) tracking of the robot. Trajectories are provided by the operator or generated from symbolic commands.

5. Case Study of Remote Surgery Introduction

Remote surgery (also known as telesurgery) is the ability for a doctor to perform surgery on a patient even though they are not physically in the same location. It is a form of telepresence. A robot surgical system generally consists of one or more arms (controlled by the surgeon), a master controller (console), and a sensory system giving feedback to the user. Remote surgery combines elements of robotics, cutting edge communication technology such as high-speed data connections and elements of management information systems. While the field of robotic surgery is fairly well established, most of these robots are controlled by surgeons at the location of the surgery. Remote surgery is essentially advanced telecommuting for surgeons, where the physical distance between the surgeon and the patient is immaterial. It promises to allow the expertise of specialized surgeons to be available to patients worldwide, without the need for patients to travel beyond their local hospital.

Surgical Systems

Surgical robot systems have been developed from the first functional telesurgery system-ZEUS-to the da Vinci Surgical System, which is currently the only commercially available surgical robotic system. In Israel a company was established by Professor Moshe Schoham, from the faculty of Mechanical Engeenering at the Technion. Used mainly for on-site surgery, these robots assist the surgeon visually, with better precision and less invasiveness to patients. The Da Vinci Surgical System has also been combined to form a Dual Da Vinci system which allows two surgeons to work together on a patient at the same time. The system gives the surgeons the ability to control different arms, switch command of arms at any point and communicate through headsets during the operation.

Costs
Marketed for $975,000, the ZEUS Robot Surgical System was less expensive than the da Vinci Surgical System, which cost $1 million. The cost of an operation through telesurgery is not precise but must pay for the surgical system, the surgeon, and contribute to paying for a years worth of ATM technology which runs between $100,000-$200,000.

The Lindbergh Operation

One of the earliest remote surgeries was conducted on 7 September 2001 across the Atlantic Ocean, with a surgeon (Dr. Jacques Marescaux) in New York performing a cholecystectomy on a 68-yearold female patient 6,230 km away in Strasbourg, France named Operation Lindbergh. That operation, called Project Lindbergh after Charles Lindberghs pioneering transatlantic flight from New York to Paris, was conducted over a dedicated fiberoptic link to ensure guaranteed connectivity and minimal lag.

Technology
The speed of remote surgery is made possible through ATM technology, or Asynchronous Transfer Mode. Asynchronous Transfer Mode is a technology designed for the high-speed transfer of voice, video, and data through public and private networks using cell relay technology. Cell relay technology is the method of using small fixed length packets or cells to transfer data between computers or network equipment and determines the speed at which information is transferred. ATM technology has a maximum speed of 10 Gbit/s (Gigabits per second). This developed technology provides opportunities for more transatlantic surgeries similar to the Operation Lindbergh. During a surgery, the robot arm can use a different angle during a laparoscopic surgery than a tool in the surgeons hand, providing easier movement. The da Vinci Surgical System, using "Endowrist" instruments, allows the surgeon seven degrees of rotation and a range of motion far greater than the human hand while filtering out the hands natural tremor.

Applications
Since Operation Lindbergh, remote surgery has been conducted many times in numerous locations. To date Dr. Anvari, a laparoscopic surgeon in Hamilton, Canada, has conducted numerous remote surgeries on patients in North Bay, a city 400 kilometres from Hamilton. Even though he uses a VPN over a non-dedicated fiberoptic connection that shares bandwidth with regular telecommunications data, Dr. Anvari has not had any connection problems during his procedures. Rapid development of technology has allowed remote surgery rooms to become highly specialized. At the Advanced Surgical Technology Centre at Mt. Sinai Hospital in Toronto, Canada, the surgical room responds to the surgeons voice commands in order to control a variety of equipment at the surgical site, including the lighting in the operating room, the position of the operating table and the surgical tools themselves. With continuing advances in communication technologies, the availability of greater bandwidth and more powerful computers, the ease and cost effectiveness of deploying remote surgery units is likely to increase rapidly. The possibility of being able to project the knowledge and the physical skill of a surgeon over long distances has many attractions. There is considerable research underway in the subject. The armed forces have an obvious interest since the combination of telepresence, teleoperation, and telerobotics can potentially save the lives of battle casualties by providing them with prompt attention in mobile operating theatres. Another potential advantage of having robots perform surgeries is accuracy. A study conducted at Guys Hospital in London, England compared the success of kidney surgeries in 304 dummy patients conducted traditionally as well as remotely and found that those conducted using robots were more successful in accurately targeting kidney stones

Problems with remote surgery

The first noticeable problem is what to do if the communication link goes down half way during the surgery or the robot fails?

A necessary solution to such a problem would be to have medical professionals actually at the location of where the operation is being carried out in case something like that does happen so that they can take over. Engineers would also be needed at both ends in case technical difficulties arise. The temperamental nature of electronic equipment means that it would not be safe to fully depend on it as it is not ready for such critical use. The second point is more of a challenge than a problem. Due to the distance between the surgeon and the surgery, there is bound to be some time delay (termed lag) between the surgeon moving the control and the robot doing something. This is because it takes time for the signal to reach the surgeon from the surgery and vice versa (due to the length of the communication link). In addition to this, there would also be a lag between the robot doing something and the updated view of the surgery being received by the surgeon. This time delay would need to be as low as possible (e.g. under 330ms) to ensure that the surgeon's activities are carried out immediately and that he/she has access to the patient's status in real time. This ensures that the surgeon is in complete control. For example, in the event of complications arising, the surgeon must be made aware of them immediately, so they can be dealt with straight away. This reiterates the need for real-time communication. Due to the fact that technology is susceptible to failures, telesurgery is mainly used for research purposes and not as much in mainstream.

6. Conclusion
The telerobotics is a great achievement in the field of robotics. Advances in telerobotics are in high gear here on Earth, enabling scientists to plumb the deepest oceans, extract resources from dangerous mines and even carry out high-precision surgery from thousands of miles away. The remote operations of robot have contributed a great help to the remote, space and marine applications. Now researchers are considering ways to adopt and adapt telerobotics for more far-reaching duties in outer space. The research is going on for further development of robotics application through network.

7. References
[1] Tyler Schilling. Wiley, Telerobotic Applications, (2000) [2] Thomas B. Sheridan, Telerobotics, Automation, and Human Supervisory Control, MIT Press (1992) [3] Manuel Ferre, Martin Buss, Rafael Aracil, Claudio Melchiorri, Carlos Balaguer, Advances in Telerobotics (Springer Tracts in Advanced Robotics), Springer (2007) [4] en.wikipedia.org/wiki/Telerobotics [5] en.wikipedia.org/wiki/Remote_surgery