Teleoperated multi-robot group for technical inspections

Wojciech Moczulski, Marcin Januszka, Wawrzyniec Panfil, Piotr Przystałka, Mirosław Targosz, Wojciech Skarka

Institute of Fundamentals of Machinery Design, Silesian University of Technology, Gliwice

Abstract: In this paper, the authors discuss results of the research concerning a group of multitasking mobile robots that use advanced technologies. The main goal of the paper is to illustrate functionality of the robot group, wireless communication and control system based on Wi-Fi standard as well as architectures of the selected subsystems. The developed robots allow aiding humans in accomplishing tasks in an environment that may be dangerous. The group consists of teleoperated robots: a transporting robot, an exploring robot, and small monitoring robots. Teleoperated robots can be used most often as moving sensor devices. The group of robots is capable of monitoring and carrying out measurements of selected physical quantities, which can occur within the territory of any object, and then transmitting the data to the user. Additionally, elaborated exploring robot can survey an area of terrain with visual inspection and take samples of soil.

Keywords: teleoperated robots, inspection mobile robot, control system, Wi-Fi

DOI: 10.14313/PAR_207/134

1. Introduction

Nowadays, there are many attempts to develop mobile robots assisting in the implementation of human dangerous tasks (e.g. inspection in hazardous areas). In Poland a largest manufacturer of mobile robots is the Industrial Research Institute for Automation and Measurements PIAP, which produces, among others: commercial pyrotechnics robots Expert and Inspector, inspection robot Scout and reconnaissance-pyrotechnics robot IBIS [8]. The world’s leading manufacturer of mobile robots is the Remotec company, producing little small inspection robot MiniAndros and large Wheelbarrow robot [8]. Commercial versions of inspection or intervention robots [8]: Cybernetix RM35 and Castor, PW Allen-Vanguard HOBO and Defender, ABP Bison and Cyclops can be found on the European market Most of these mobile robots are teleoperated. Teleoperation indicates control of robots from a distance and is one of the first domains of the robotics. One of the earliest examples of teleoperation was in 1951 when Goertz [3] developed a mechanical master-slave manipulator arm for work with radioactive material. Nowadays, despite the rapid development of autonomous robots, a robot teleoperation is still widely used in mobile robotics. Examples of robot teleoperation in different environments are presented in [1, 2].

In the recent years many mobile robot platforms use wireless technology to communicate with controlling operators or other robots [12]. Many mobile robots have been equipped with wireless technology such as Wi-Fi, Bluetooth, Wireless LAN etc [5]. Generally, there is a great interest in multi-robot systems [9–11].

The paper briefly presents the results (earlier stages of implementation are presented in [6, 7]) of the research project on a group of teleoperated mobile robots for technical inspections. As a part of the research described in this paper (compared to [6, 7]) has been the implementation of robotic executive systems and improved operators’ interface to support these real systems. The validation tests were carried out with the use of an improved version of the robots.

2. A group of robots

The goal of the project has been the research, development, and implementation of a team of teleoperated multitasking mobile robots that use advanced technologies. The developed robots allow aiding humans in accomplishing tasks in an environment that may be dangerous. The group (Fig. 1) consists of remote controlled robots: a transporting robot (called Transporter), an exploring robot (called Explorer), and small monitoring robots (called Pathfinders).

a)

b)

Fig. 1. A teleoperated multi-robot group: a) concept of a collaboration, b) the physical realisation of the multi-robot group
Rys. 1. Grupa zdalnie sterowanych robotów: a) koncepcja współdziałania, b) fizyczna realizacja grupy robotów

The team is able to supervise large objects by wireless transmitting data collected by sensors and video streams. The group of robots is capable of monitoring and carrying out measurements of selected physical quantities that can occur within the territory of any object and then transmitting the data to the user. Teleoperated robots can be used most often as moving sensor devices.

Both Transporter and Explorer share the same tracked chassis that allows them to move across uneven terrain including staircases. The robots are capable of operating up to 5 hours using their own batteries. Transporter is equipped with a robotic arm and a compartment that can house up to four Pathfinders or a dangerous item of up to 5 kg mass. Elaborated Explorer robot can survey an area of terrain with visual inspection and take samples of soil. Additionally, Explorer is equipped with a special system for sensor mounting that allows attaching several sensors including gas and smoke detectors, and many others, depending on the kind of mission the robot is taking part in.

The last robot called Pathfinder plays the role of a small inspecting robot. It is equipped with a digital camera, a microphone and a loudspeaker, and also with a temperature sensor. The main idea implemented in this robot was to develop a low-cost remote controlled robotic system capable of inspecting inaccessible areas and of allowing voice contact with a human remaining in this area. Such robots can be operated in dangerous environment, because the cost is relatively low.

Software for controlling all the teleoperated robots has been developed, with two-level architecture which is composed of low-level software (physical layer), upper-level software and user interface. The system allows assigning tasks to the robots operating in the group, especially in the case of rescue operations where the operator works under stress conditions and needs support in deciding which robots can undertake activities required to accomplish the complete operation.

3. Control and communication system

This section presents a teleoperation system to control mobile robots remotely. In elaborated teleoperation system it is able to control all robots independently by a group of operators or to control each robot only by one operator by switching between them. The quality of robots’ teleoperation system greatly depends on its control and communication systems. Navigation of Explorer and Transporter robots is supported by GPS-based units, that allow the robot to return to the last position where wireless communication with the operator was possible.

3.1 Control system

In this subsection the software development platform, main idea and architecture of the control system are presented.

3.1.1. Software platform

After a comprehensive study of the available software platforms intended for development of the mobile robots’ control systems it was decided to apply the Microsoft Robotics Developer Studio (MRDS) [4]. This software package consists of a few tools especially useful for developing robotic applications (also distributed), i.e.:

– Concurrency and Coordination Runtime (CCR) – ensures controlled asynchronous communication between MRDS services and nodes,

– Decentralized Software Services (DSS) – allows for easy distribution of the software modules (MRDS services) among many nodes of the control system.

Moreover, MRDS contains a very sophisticated simulator, which allowed the software developers to perform preliminary tests of the control system including: robots’ behavior in simulated environment conditions similar to the real conditions, control algorithm tests, and also control system architecture of initial tests.

3.1.2. Control system of the robots’ group – main idea

In general the group of robots consists of the two Pathfinders, one Explorer and one Transporter. Each of the above-mentioned robots is remotely controlled by an operator using UI application running on an individual computer. The control system of the mobile robots group is distributed in both hardware and software layers. It uses wireless communication standard 802.11 b/g/n (section 3.2) employing TCP/IP protocol on which MRDS is based. Every existing node (device) in control system net has its own unique and predefined IP address – it allows for easy searching for nodes in the net.

The control system, developed according to the proposed architecture (subsection 3.1.3), is modular, scalable and extensible. The base functionality of the control system can be easily extended, the control system allows to control almost unlimited number of robots. All of the robots can operate individually and in group. At the beginning (after the system starts individual robot) the control system of the robot works regardless of the others. At any time the control system of the robot can be included in the control system of the group – the other members (their operators) of the group are informed about its status (on/off) and (if possible) its localization. There have not been taken into account the group behaviour (e.g. moving in formation) – the robots are just teleoperated.

3.1.3. Control system architecture

The architecture of the high-level control system of every robot was divided into three layers:

– User application layer – consists of user interface panels (example of the UI is presented in Fig. 2), communicates with peripheral devices (LCD, keyboard, mouse, joystick) and controls logic layer, Control logic layer – contains MRDS services responsible for control logic of the control system, e.g. robots’ movement control, streaming and recording video and audio signals, control robots’ peripheral devices (manipulator, soil probe, gas sensors), etc. Contents of this layer (package of the MRDS services) is dependent mainly on functionality of the robots and the whole system. This layer communicates with user interface and control logic layer – there is no direct communication between the above mentioned layers,

– Intermediate layer – communicates with control logic layer and main equipment of the robots (video cameras, internal robot state sensors, environment detection sensors, lights, main and auxiliary drives. For these purposes control system of Pathfinder uses FoxBoard control unit, Transporter and Explorer apply PC communicating with the above-mentioned devices using CAN and USB interfaces. In the case of Transporter/Explorer robots this layer contains mainly MRDS services responsible for direct control of the PC with robots’ equipment.

Fig. 2. Example of UI running on the operator’s PC allowing to control Transporter and Explorer robots
Rys. 2. Przykład graficznego interfejsu użytkownika uruchomionego na komputerze PC operatora pozwalającego na sterowanie robotami Transporter i Explorer

–

–

User interface and control logic layers of the individual robot control system are present on the operator control unit (PC), whereas intermediate layer is placed on the operator control unit (Pathfinder) or robot control unit (Transporter, Explorer).

3.2 Communication system

The communication system proposed in this paper enables wireless remote control of a group of robots by operators in real time simultaneously. For such systems, it is required that the transmission of control and audio-video signals must be realized with the minimum latency. Moreover, in the case of audio and video data streaming, it is necessary to achieve the maximum raw data rate. An additional challenge is to guarantee the appropriate transmission range between communicating nodes, especially for missions accomplished in indoor environments. Due to these assumptions, it is settled that the architecture of a communication system is based on IEEE 802.11n standard (150 Mbit/s, 2.4 GHz). The hardware layer of the communication system of each robot is designed using BulletM2 device (2.4 GHz 802.11b/g/n, max. 28 dBm TX-Power, –40 °C to +80 °C). Three basic wireless network topologies are developed, as shown in Fig. 3a–c. The first way of communication is based on a star topology (Fig. 3a). The communication between the operators and robots is followed by the access point. This solution is recommended for the mission which does not require the significant dispersion of robots and operators. The second solution is based on a repeater bridge (Fig. 3b). This method of communication is required when the long-distance data transmission between robots and operators is needed or when there is a large attenuation of the communication signal within the inspection area. The last topology is a classic point-to-point connection (Fig. 3c). Such data transmission is involved for high-speed communication between a single operator and a mobile robot.

Fig. 3. Schemes of three basic wireless network topologies used for communication purposes
Rys. 3. Schematy trzech podstawowych topologii sieci bezprzewodowej użytych do celów komunikacji

4. Validation

A validation process included tests of mobility and test of all the systems. Laboratory and field tests of the team under differing environmental and climate circumstances have proven high mobility of the robots. Typical duration of the mission is 3–5 hours for the Explorer or Transporter robot and 2 hours for inspecting robots, depending on the ambient temperature and humidity. Moreover, most of the achieved values of communication system parameters exceeded the assumed ones. Typical communication and teleoperation range for the group of robots which operate in buildings is up to 50 m (see Fig. 4) and in open areas up to 500 m.

Fig. 4. Results of communication and teleoperation tests in in-door and out-door scenarios
Rys. 4. Wyniki testów systemów komunikacji i zdalnego sterowania robotów w budynku i na terenie otwartym

5. Conclusions and future work

Nowadays global trends in the field of mobile robotics outline the need for the development of mobile robots that can work in a group and can be wirelessly controlled by the operators. The paper deals with a brief description of a team of teleoperated mobile robots for inspecting large-area technical objects such as large open-air stores, closed areas of factories, airfields, military objects, and many others.

The main conclusions of the authors are formulated as follows:

– the use of Wi-Fi allows to achieve satisfactory communication ranges with low cost of the system;

– the validation tests confirmed the large functionality of presented teleoperated robots;

– the presented team of robots is currently complete and provides functionality equivalent to other commercial robots;

– robot teleoperation commonly requires an operator‘s full attention to achieve task goals.

There is no doubt that the field of teleoperated robotics provides a rich area for future developments. The authors are going to continue the work with special attention paid to industrial implementation of the group of robots.

Acknowledgements

This project has been carried out in the framework of Strategic Programme “Innovative Systems of Technical Support for Sustainable Development of Economy” (Innovative Economy Operational Programme) coordinated by Institute for Sustainable Technologies – National Research Institute (ITeE-PIB). The authors wish to express their gratitude to the staff of ITeE-PIB for their collaboration at all stages of the project.

Bibliography

1. Dong-So K., Jee-Hwa R., Pan-Mook L., Suk-Won H., Design of a Teleoperation Controller for an Underwater Manipulator, IEEE International Conference on Robotics & Automation, San Francisco, California, 3114–3119, April 2000, DOI: 10.1109/ROBOT.2000.845142.

2. Ermolov I.L., Levenkov A.V., Poduraev J.V., Choi S.J., Internet Control of Mobile Robots for Pipe Inspection/Repair, Proceedings of the 4th International Workshop on Computer Science and Information Technologies, 18–20 September, 2002.

3. Goertz R., Thompson R., Electronically controlled manipulator, Nucleonics, vol. 12 (11): 46–47, 1954.

4. Johns K., Taylor T., Professional Microsoft Robotics Developer Studio, Wiley Publishing, Inc. Indianapolis, Indiana 2008.

5. Kahar S., Sulaiman R., Prabuwono A.S., Ahmad N.A., Ashri Abu Hassan M., A Review of Wireless Technology Usage for Mobile Robot Controller, PCSIT vol. 34: 7–12, 2012.

6. Moczulski W., Skarka W., Targosz M., Panfil W., Przystałka P., Wyleżoł M., Januszka M., Pająk D., Wielozadaniowe mobilne roboty do inspekcji obiektów technicznych, Advance in Robotics. T. 1. Electronics vol. 182: 199–206, K. Tchoń, C. Zieliński (Eds.), Publishing House of the Warsaw University of Technology, Warsaw 2012.

7. Moczulski W., Skarka W., Januszka M., Giesko T., Mężyk J., Mizak W., Pająk D., Panfil W., Przystałka P., Targosz M., Wiględa R., Wyleżoł M., Projekt grupy wielozadaniowych robotów mobilnych wykorzystujących zaawansowane technologie, Maintenance Problems, 3(82), 123–129, Radom 2011.

8. Trojnacki M., Szynkarczyk P., Andrzejuk A., Tendencje rozwoju mobilnych robotów lądowych, Pomiary Automatyka Robotyka, 6: 11–14, 2008.

9. Turek W., Cetnarowicz K., Zaborowski W., Software Agent Systems for Improving Performance of Multi-Robot Groups, [in:] Fundamenta Informaticae, vol. 112, 103–117, IOS Press, Amsterdam 2011, DOI: 10.3233/FI-2011-581.

10. Cetnarowicz K., Agenci softwere’owi w systemach wielorobotowych, [in:] Inteligencja wokół nas: współdziałanie agentów softwarowych, robotów, inteligentnych urządzeń, praca zbiorowa/pod red. Stanisława Ambroszkiewicza [et al.], Akademicka Oficyna Wydawnicza EXIT, Warszawa 2010.

11. Skrzypczyński P., Multi-agent software architecture for autonomous robots: a practical approach, Management and Production Engineering Review, vol. 1, no. 4, 2010, 55–66.

12. Kasiński A., Skrzypczyński P., Communication mechanism in a distributed system of mobile robots, [in:] Distributed Autonomous Robotic Systems 5 (H. Asama et al., eds.), Tokyo, Springer 2002, 51–60. DOI: 10.1007/978-4-431-65941-9_6.

Grupa teleoperowanych inspekcyjnych robotów mobilnych

Streszczenie: W artykule przedstawiono badania dotyczące opracowania grupy inspekcyjnych robotów mobilnych stosujących zaawansowane technologie. Głównym celem pracy jest zaprezentowanie funkcjonalności systemu wielorobotowego (wraz z podsystemami), a także jego architektury, systemu komunikacji opartego na technologii Wi-Fi. Podstawowym zadaniem opracowanej grupy robotów mobilnych jest wspomaganie ludzi w prowadzeniu działań inspekcyjnych oraz eksploracyjnych w warunkach, które są dla nich niebezpieczne, tj. zagrażają ich zdrowiu lub życiu. W skład grupy robotów wchodzą: robot transportowy, robot eksploracyjny oraz kilka mniejszych robotów inspekcyjnych. Roboty te mogą być postrzegane jako mobilne urządzenia pomiarowe. Grupa takich robotów umożliwia dokonywanie pomiarów wybranych wielkości fizycznych (np. stężenie szkodliwych gazów, pomiar temperatury, detekcja ruchu) w zadanym obszarze, a następnie na przesyłanie użytkownikowi (operatorowi) wyników tych pomiarów. Dodatkowo, robot eksploracyjny – poza inspekcją wizyjną – może pobierać próbki gleby.

Słowa kluczowe: roboty teleoperowane, roboty inspekcyjne, systemy sterowania, sieci Wi-Fi

Artykuł recenzowany, nadesłany 17.01.2014 r., przyjęty do druku 25.03.2014 r.

Wojciech Moczulski, Prof. PhD DSc.

He is a professor in the Institute of Fundamentals of Machinery Design at Silesian University of Technology at Gliwice. His researches are focused on: design and operation of the machines, computer science, particularly in machinery diagnostics and application of methods and means of Artificial Intelligence. He is author and co-author of over 200 publications. He is the holder of scholarship of Humboldt’s Foundation at University in Paderborn (Germany). He managed researches at Wichita State University and University of North Carolina in Charlotte (USA) and many others. Since 2005 he has been Associated Editor of ’Engineering Applications of Artificial Intelligence’. Since 2002 he has been serving as Organizing Committee Chair of the International Symposium on Methods of Artificial Intelligence (AI-METH). He is founder and member of the Central Board of the Polish Society of Technical Diagnostics.

e-mail: wojciech.moczulski@polsl.pl

Marcin Januszka, PhD Eng.

He is assistant professor in Institute of Fundamentals of Machinery Design, Silesian University of Technology. His research interests include augmented reality, virtual reality, CAD, mobile robotics. His current research topic is application of augmented reality in designing, diagnostics and maintenance of mobile robots. Author or co-author of over 30 peer-reviewed publications. He is member of Polish Association of Disseminating Computer Engineering Systems (ProCAx).

e-mail: marcin.januszka@polsl.pl

Wawrzyniec Panfil, PhD Eng.

He is an assistant professor in Institute of Fundamentals of Machinery Design, Silesian University of Technology. He deals with designing and operation of machines, autonomous mobile robots systems and applications of augmented reality in machinery diagnostics and maintenance.

e-mail: wawrzyniec.panfil@polsl.pl

Piotr Przystałka, PhD Eng.

He is an assistant professor in Institute of Fundamentals of Machinery Design at Silesian University of Technology. He received his PhD from the Faculty of Mechanical Engineering in 2009 with his dissertation on the methodology of neural modeling in fault diagnosis with the use of the chaos theory. His research focus is on fault diagnosis and fault-tolerant control, rapid prototyping of control systems and mobile robotics.

e-mail: piotr.przystalka@polsl.pl

Mirosław Targosz, MSc Eng.

PhD student in Institute of Fundamentals of Machinery Design, Silesian University of Technology. He deals with designing of machines, optimization of powertrain units for mobile robots and electric vehicles.

e-mail: mirosław.targosz@polsl.pl

Wojciech Skarka, PhD DSc Eng.

He is associate professor in the Institute of Fundamentals of Machinery Design at Silesian University of Technology at Gliwice. He is the Head of the Division of Mobile Systems in the Institute. His research is focused on: design and operation of machines, computer aided design and knowledge based-engineering, particularly in mining machinery and aviation industry. He deals with mobile systems such as mobile robots, electric cars, unmanned aerial system. Author or co-author of over 100 publications. He is a member of the Board of Polish Association of CAx systems users. He is the winner of electric cars race in Silverstone – Greenpower Corporate Challenge and takes part in energy efficient cars race – Shell Eco-marathon in Rotterdam.

e-mail: wojciech.skarka@polsl.pl