2023 Impacts_of_Wireless_on_Robot_Control_The_Network_Hardware-in-the-Loop_Simulation_Framework_and_

.pdf

School

The University of Sydney *

*We aren’t endorsed by this school

Course

2023

Subject

Mechanical Engineering

Date

Oct 30, 2023

Type

pdf

Pages

Uploaded by KidFreedomFlamingo27

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 19, NO. 9, SEPTEMBER 2023 9255 Impacts of Wireless on Robot Control: The Network Hardware-in-the-Loop Simulation Framework and Real-Life Comparisons Honghao Lv , Student Member, IEEE , Zhibo Pang , Senior Member, IEEE , Koushik Bhimavarapu , Member, IEEE , and Geng Yang , Member, IEEE Abstract — As many robot applications become more re- liant on wireless communications, wireless network latency and reliability have a growing impact on robot control. This article proposes a network hardware-in-the-loop (N- HiL) simulation framework to evaluate the impacts of wire- less on robot control more efficiently and accurately, and then improve the design by employing correlation analysis between communication and control performances. The N- HiL method provides communication and robot developers with more trustworthy network conditions, while the huge efforts and costs of building and testing the entire physical robot system in real life are eliminated. These benefits are showcased in two representative latency-sensitive applica- tions: 1) safe multirobot coordination for mobile robots, and 2) human-motion-based teleoperation for manipula- tors. Moreover, we deliver a preliminary assessment of two new-generation wireless technologies, the Wi-Fi6 and 5G, for those applications, which has demonstrated the effec- tiveness of the N-HiL method as well as the attractiveness of the wireless technologies. Index Terms — 5G, hardware-in-the-loop, multirobot coor- dination, teleoperation, Wi-Fi 6, wireless control. Manuscript received 18 October 2022; accepted 25 November 2022. Date of publication 8 December 2022; date of current version 24 July 2023. This work was supported in part by the Swedish Foundation for Strategic Research under Grant APR20-0023; in part by the National Natural Science Foundation of China under Grant 51975513; in part by the Natural Science Foundation of Zhejiang Province, China under Grant LR20E050003; in part by the Major Research Plan of National Natural Science Foundation of China under Grant 51890884; and in part by the Major Research Plan of Ningbo Innovation 2025 under Grant 2020Z022. The work of Honghao Lv was supported by the China Scholarship Council. Paper no. TII-22-4339. (Corresponding author: Zhibo Pang.) Honghao Lv is with the State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang Uni- versity, Hangzhou 310027, China, and also with the Department of Intelligent Systems, Royal Institute of Technology, 114 28 Stockholm, Sweden (e-mail: lvhonghao@zju.edu.cn). Zhibo Pang is with the Department of Automation Technology, ABB Corporate Research, 722 26 Vasteras, Sweden, and also with the De- partment of Intelligent Systems, Royal Institute of Technology, 114 28 Stockholm, Sweden (e-mail: zhibo@kth.se). Koushik Bhimavarapu is with the Department of Automation Tech- nology, ABB Corporate Research, 722 26 Vasteras, Sweden (e-mail: koushik.bhimavarapu@se.abb.com). Geng Yang is with the State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang Uni- versity, Hangzhou 310027, China (e-mail: yanggeng@zju.edu.cn). Color versions of one or more figures in this article are available at https://doi.org/10.1109/TII.2022.3227639. Digital Object Identifier 10.1109/TII.2022.3227639 I. I NTRODUCTION T HE growing wireless network technologies and the cloud and edge computing toward Industrial 4.0 enable various wireless-network-controlled robot platforms that are practically applied to industrial production [1], [2]. Intuitively, wireless networks enable higher flexibility of the robotic system and simplify design and installation processes, and diminish mainte- nance needs, especially for mobile robots [3], [4]. And wireless networks make it possible to implement a remote operation, because the controllers and actuators are deployed over a wire- less link, such as the teleoperation of the manipulators [5], [6]. However, wireless communication imposes challenges in time-critical robot control scenarios, where wireless networks involved extra latency compared to wired connection [7]. To ensure the robustness and reliability of the robotic system, simulation and testing for the communication are required dur- ing the design stage and deployment process of an industrial robotic platform [8]. Introducing latency or other communi- cation characteristics into the controller codes to imitate the profiles of the real network is a common solution for the sim- ulation [9]. However, the practical communication condition, particularly the long-term stable performance of a wireless network, is extremely difficult to be modeled and simulated [10]. Li and Savkin [11] simulated unmanned aerial vehicle (UAV) navigation based on the wireless sensor network using the professional simulation software V-REP in the industrial in- ternet of things applications. However, a stable and invulnerable network condition is assumed in the simulation environments. Despite the simulation method of [11] is able to do long-term stability tests but has no generalized capabilities to evaluate different network conditions and communication uncertainty. The model-based simulation methods are commonly used for validating the auto guide vehicles (AGVs) coordination, such as in [12] and [13], while both cannot be used to evaluate the communication uncertainty. Hardware-in-loop (HiL) simulation refers to the simulation technology, which simulates one part of the whole system with computer modeling while using physical modeling or an actual system for the other part [14]. In the past few years, research efforts have been made in HiL simulation for robot system design and optimization [15], [16]. Zhou et al. [14] proposed an HiL simulation method for underwater acoustic This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/

9256 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 19, NO. 9, SEPTEMBER 2023 TABLE I C OMPARISONS B ETWEEN THE P ROPOSED A PPROACH AND O THER R ELATED S IMULATION W ORKS FOR R OBOT C ONTROL Fig. 1. Diagram of the proposed network hardware-in-the-loop framework. communication, but without applying it to robots or other devices. Marco et al. proposed an on-ground HiL simulation method for prevalidation and optimization of on-orbit robotic missions. However, the technical requirements of the proposed approach are the real-time computations and the negligible latency given by Assumption 1 in [16]. An HiL simulation system for manipulator control in [17] was built to improve the system stability by deploying the self-designed PID controller. Lamping et al. [18] worked on a multiagent UAV system based on robot operation system (ROS), and they experimented with control and supervision algorithms on multiple UAVs using HiL simulation. The majority of these HiL simulation studies for robotic applications aimed to provide a flexible and easy-to-use interface for a controller design [19]. Markus et al. [20] from ABB and NOKIA investigated the capabilities of 5G and LTE for supporting industrial robotic applications using the traffic- model-based communication simulation methods. However, the simulation results are constrained to the specific scenario de- fined in this article and do not have the ability to evaluate the impact of network uncertainty. One common constraint of HiL simulation for robotics, mechatronics, and control is the limited modeling of the communication condition, especially for the growing wireless network controlled robots [21]. Lv et al. [22] presented an HiL simulation framework and conducted a preliminary study on wireless robot control using Wi-Fi 6. But it lacks the systematic evaluation of the network performance, and the simultaneous correlation between communication and controlisnotelaborated.Thesummarizedcomparisonsarelisted in Table I . To cover the gap in the literature, a network hardware-in-the- loop(N-HiL)simulationframeworkshowninFig. 1 isdeveloped in this article for evaluating the impacts of the wireless network on robot control systems, considering the scalability of the net- work interface, and the friendliness to both the communication and robot developers in the field. The major contributions can be summarized as follows: 1) A novel N-HiL simulation framework is proposed to evaluate the impacts of wireless on robot control, which provides more trustworthy results, while the huge efforts and costs of testing the entire physical robot system in real life are eliminated. 2) Two typical latency-sensitive robot control cases, safe multirobot coordination and human-motion-based tele- operation, are investigated, and the performance compar- isons of Wi-Fi 6 and 5G wireless networks are conducted. 3) The correlation between the communication and control is investigated using the proposed N-HiL method, and a selection rule is given for choosing the wireless networks

LV et al.: IMPACTS OF WIRELESS ON ROBOT CONTROL 9257 Fig. 2. Diagram of the overall methodology compared with previous methods: (a) traditional methodology and (b) N-HiL methodology pro- posed in this article. with different profiles for robot control applications in the design stage. The rest of this article is organized as follows. In Section II, the N-HiLframeworkisproposedandthemethodologyisdescribed. Two case studies are presented in Section III. Sections IV and V provide the experiments and validations of the coordination case and teleoperation case using the N-HiL framework. Finally, Section VI concludes this article. II. N ETWORK H ARDWARE - IN - THE -L OOP F RAMEWORK A. Overall Methodology The proposed N-HiL framework aims to provide a reliable and trustworthy methodology to evaluate the impacts of wireless on robot systems. The controller design is the essential part for designing a robot control system, where the simulation in the de- sign stage could provide a controller tuning according to the con- trol performance assessment. For most robotic simulation cases using the traditional methodologies, assumptions under ideal communication conditions or stable networks are considered, as shown in Fig. 2(a) , which provides an over-optimistic network model and an interpretable tracking behavior between the output and command by ignoring the network-communication uncer- tainty. Compared to the previous methods which integrate the network modeling in the controller for simulation, the proposed N-HiL framework injects the realistic network hardware into the simulation as shown in Fig. 2(b) . In addition, the control adjust- ment is associated with the communication adjustment based on the performance assessment both for the network and control in the N-HiL simulation framework. Driven by the system design from two representative use cases, the statistical evaluation met- rics for the control and communication are proposed. The results from the N-HiL simulation are coanalyzed to adjust the control strategies and network strategies collaboratively, investigating the impacts of wireless on robot control. The proposed N-HiL framework has the following expected benefits: 1) exposing the issues of the real wireless network condition that cannot be mimicked by statistical modeling; 2) eliminating the efforts and costs of building the entire physical robot system for assessment; 3) providing an effective approach for studying the interplay between the wireless and control strategy; and 4) enabling the long-term stability test of the communication for a robot system. B. System Architecture The system architecture of the proposed N-HiL framework includes four parts, as shown in Fig. 1 : simulated controller, simulatedrobot platform, networksniffer hardware, andthereal- life network environment. The essential element of the N-HiL framework is the unob- trusive network sniffer hardware interface deployed based on an Ethernet multichannel probe. The sniffer interface is deployed to connect the simulated controller and the simulated robot plat- form. The controller includes the robot kinematic forward model or the dynamic model, which is used to compute the discrete high-frequency control command in real time. The simulated robot is driven by the command transferred by the network hardware interface. The sniffer program keeps on sniffing the raw packet from the sniffer port continuously in real time to capture the packets and calculate the latency of the packets as the unobtrusive latency tester (ULT). The downstream latency of the packets that are transmitted from source to destination is calculated using the time stamping of destination output DOWN_OUT minus the time stamping of source input UP_IN. The upstream latency of the packets that are transmitted from destination to source is calculated using the time stamping of source output UP_OUT minus the time stamping of destination input DOWN_IN. The wired Ethernet and the wireless network, Wi-Fi6 and 5G, are implemented as shown in the floor plan of Fig. 1 . In particular, three test conditions of Wi-Fi 6, short-range, medium-range, and long-range, are set for the distance be- tween two access point (AP) nodes (hereafter represented as Wi-Fi 6 @ short/medium/long). The 5G station used here is an internal industrial solution provided by Ericsson AB. For the wired Ethernet network, the interface is con- nected using an Ethernet cable between port DOWN_OUT and port UP_OUT. For the wireless network framework, the master AP of Wi-Fi 6 and the user plane function of the 5G station are connected to port DOWN_OUT, while the slave AP and the 5G user equipment are connected to port UP_OUT. III. I MPLEMENTATION OF THE N-H I L F RAMEWORK A. Case Study Overview To investigate the impacts of wireless on robot control, the corresponding case studies are conducted by deploying the proposed N-HiL simulation framework. Two of the key concerns for wireless robot systems in practical application are safety and accuracy. Multirobot coordination is chosen as the first case study, which is a representative core problem in mobile robot fleets. Most multirobot coordination solutions assume good communication conditions [23], [24]. The unreliable commu- nication will lead to the delay of the real-time status update or control command loss of each robot, which final causes collision and unsafety [25]. The multirobot coordination does need to

9258 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 19, NO. 9, SEPTEMBER 2023 Fig. 3. N-HiL simulation framework for safe multirobot coordination. deploy the test on the proposed N-HiL framework since it is a key to pretest for avoiding collision and ensuring safety [26]. Moreover, the manipulator teleoperation is chosen as the sec- ond test case, which is another typical communication-sensitive robot application [27]. In order to achieve accurate and smooth control of a multijoint manipulator remotely, the high-frequency discrete joint commands need to be accurately transferred from the remote side to the robot controller [28]. The packet loss and latency introduced by the communication will change the motion status suddenly, which will greatly affect the control performance. B. Case Study 1: Safe Multirobot Coordination In this case, a centralized multirobot coordination platform from Orebro University is deployed, in which robots are driven by generic second-order dynamics and coordinated by heuristic robot ordering policies [12]. As shown in Fig. 3 , the motion planner is used to calculate the path and trajectory of each robot. The modeling method for the coordination avoids the need for a priority discretization of the environment and allows robots to “follow each other” through CSs (overlap of multirobot trajectory envelops). The coordinator decides the robot order in the CS to avoid collisions and ensure safety. The simulated robot forward model is driven by the motion command from the con- troller, and the real-time multirobot motion status is displayed in a graphical user interface. The modeling method we used in this article has been validated formally and experimentally in simulation and with real robots [29]. The robot trajectory tracker tracks the robot trajectories and sends the real-time robot status to the controller. The controller and the robot forward model, as the simulated robot system, are achieved by Java, running on Ubuntu 20.04 PC. Here, two agents are designed to forward the robot command and update the real-time robot status, as illustrated in Fig. 3 . Agent 1 is responsible to forward the robot motion command from the controller to the simulated robot model, which is done over two independent UDP sockets. Agent 2 is responsible to forward the current robot status from the driven robot model to the controller. These two agents are programmed by Java and deployed on a Windows 10 PC. The network hardware interface Fig. 4. N-HiL simulation framework for human-motion-based teleoper- ation. is deployed between the simulated robot and the agents. To more closely match the real AGVs, the proposed N-HiL framework has the feasibility to add more simulated agents with more APs connections and add more sniffer channels to measure the wireless network performance. C. Case Study 2: Human-Motion-Based Teleoperation In this case, a human–robot motion transfer teleoperation system is deployed to evaluate the impacts of wireless using the N-HiL framework. Here, the captured motion data from the operator are used to control the YuMi robot. The external guide motion (EGM) interface from ABB provides a motion control interface with high frequency (up to 250 Hz), which enables real-time human–robot motion mapping. As shown in Fig. 4 , the software Axis Neuron captures human motion and sends the data to the robot controller via ROS serial protocol from Windows 10 to Ubuntu 20.04. The controller is responsible for processing and converting the human motion data to robot tool center point data using the designed mapping strategy [30]. The robot tool center point data are used to calculate the real-time joint configuration by the inverse kinematic solver. The motion controller program is connected to the motion server running on the simulated robot model via TCP protocol. The motion server is responsible to control normal motion like linear motion and joint motion using the given robot target. The UDP User Communication device is defined for the task of each arm, which receives the joint values at a high rate. The EGM motion is activated/deactivated by the motion server. When the EGM motion is activated, the generated joint value queue will be sent using Google Protocol Buffers (Protobuf) based on the UDP protocol. In this case, the communication between the human motion capture part on the Windows 10 system and the ubuntu 20.04 is connected by Ethernet. The network hardware interface is injected between the controller running on a ubuntu 20.04 PC and the simulated YuMi robot in RobotStudio on a Windows 10 PC. RobotStudio is a mature commercial simulation software provided by ABB and has been used for many years in the industry. It is used in this article for simulating the YuMi robot provides the same configuration of

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version