ACM Transactions on Human-Robot Interaction

Nonverbal sound offers great potential to enhance robots’ interactions with humans, and a growing body of research has begun to explore nonverbal sound for tasks such as sound source localization, explicit communication, and improving sociability. However, nonverbal sound has a broad interpretation and design space that can draw from areas such as machine learning, music theory, and foley. We sought to identify and compare use cases and approaches for nonverbal sound in human-robot interaction through a systematic review. A search of sound and robotics-related publisher databases yielded 148 peer-reviewed articles presenting systems, studies, and taxonomies. Differences in taxonomy and overlap of terminology with adjacent research fields such as speech, gaze, and gesture posed difficulties for the search, which we attempted to address through a multi-stage search process. Based on the reviewed articles, we developed a pair of taxonomies using scientific communication principles and analyzed study designs and measures for the creation of nonverbal robot sound. We discuss recommendations for the field, including the use of the new taxonomies; methods for design, generation, and validation; and paths for future research. Roboticists may benefit from incorporating nonverbal sound as a key component in multimodal human-robot interaction.

Brian J. Zhang

Naomi T. Fitter

Nonverbal Sound in Human-Robot Interaction: A Systematic Review

人机交互中的非言语声音：系统综述

Remote robot manipulation with human control enables applications in which safety and environmental constraints are adverse to humans (e.g., underwater, space robotics and disaster response) or the complexity of the task demands human-level cognition and dexterity (e.g., robotic surgery and manufacturing). These systems typically use direct teleoperation at the motion level and are usually limited to low-DOF arms and two-dimensional (2D) perception. Improving dexterity and situational awareness demands new interaction and planning workflows. We explore the use of human–robot teaming through teleautonomy with assisted planning for remote control of a dual-arm dexterous robot for multi-step manipulation, and conduct a within-subjects experimental assessment (n = 12 expert users) to compare it with direct teleoperation with an imitation controller with 2D and three-dimensional (3D) perception, as well as teleoperation through a teleautonomy interface. The proposed assisted planning approach achieves task times comparable with direct teleoperation while improving other objective and subjective metrics, including re-grasps, collisions, and TLX workload. Assisted planning in the teleautonomy interface achieves faster task execution and removes a significant interaction with the operator’s expertise level, resulting in a performance equalizer across users. Our study protocol, metrics, and models for statistical analysis might also serve as a general benchmarking framework in teleoperation domains. Accompanying video and reference R code: https://people.csail.mit.edu/cdarpino/THRIteleop/

Claudia Pérez-D’Arpino

Rebecca P. Khurshid

Julie A. Shah

Experimental Assessment of Human-Robot Teaming for Multi-Step Remote Manipulation with Expert Operators

专家操作员多步远程操作的人机协作实验评估

This article proposes a novel stochastic-skill-level-based shared control framework to assist human novices to emulate human experts in complex dynamic control tasks. The proposed framework aims to infer stochastic-skill-levels (SSLs) of the human novices and provide personalized assistance based on the inferred SSLs. SSL can be assessed as a stochastic variable that denotes the probability that the novice will behave similarly to experts. We propose a data-driven method that can characterize novice demonstrations as a novice model and expert demonstrations as an expert model, respectively. Then, our SSL inference approach utilizes the novice and expert models to assess the SSL of the novices in complex dynamic control tasks. The shared control scheme is designed to dynamically adjust the level of assistance based on the inferred SSL to prevent frustration or tedium during human training due to poorly imposed assistance. The proposed framework is demonstrated by a human subject experiment in a human training scenario for a remotely piloted urban air mobility vehicle. The results show that the proposed framework can assess the SSL and tailor the assistance for an individual in real time. The proposed framework is compared to practice-only training (no assistance) and a baseline shared control approach to test the human learning rates in the designed training scenario with human subjects. A subjective survey is also examined to monitor the user experience of the proposed framework.

Sooyung Byeon

Joonwon Choi

Yutong Zhang

Inseok Hwang

Stochastic-Skill-Level-Based Shared Control for Human Training in Urban Air Mobility Scenario

基于随机技能水平的共享控制方法在城市空中交通场景中的人类训练

Robotic search often involves teleoperating vehicles into unknown environments. In such scenarios, prior knowledge of target location or environmental map may be a viable resource to tap into and control other autonomous robots in the vicinity towards an improved search performance. In this paper, we test the hypothesis that despite having the same skill, prior knowledge of target or environment affects teleoperator actions, and such knowledge can therefore be inferred through robot movement. To investigate whether prior knowledge can improve human-robot team performance, we next evaluate an adaptive mutual-information blending strategy that admits a time-dependent weighting for steering autonomous robots. Human-subject experiments show that several features including distance travelled by the teleoperated robot, time spent staying still, speed, and turn rate, all depend on the level of prior knowledge and that absence of prior knowledge increased workload. Building on these results, we identified distance travelled and time spent staying still as movement cues that can be used to robustly infer prior knowledge. Simulations where an autonomous robot accompanied a human teleoperated robot revealed that whereas time to find the target was similar across all information-based search strategies, adaptive strategies that acted on movement cues found the target sooner more often than a single human teleoperator compared to non-adaptive strategies. This gain is diluted with number of robots, likely due to the limited size of the search environment. Results from this work set the stage for developing knowledge-aware control algorithms for autonomous robots in collaborative human-robot teams.

Rafal Krzysiak

Sachit Butail

Human prior knowledge estimation from movement cues for information-based control of mobile robots during search

基于移动机器人搜索过程中运动线索的人类先验知识估计与信息控制

The increasing human-robot interactions across service domains require an examination of their moral challenges, particularly from the user’s perspective. This study explores how efficiency and morality intersect in deploying service robots during the customer encounter. Utilizing a mixed-methods approach, we examine behavioral and attitudinal customers responses to service robot morality. In a field study, we uncover a trade-off between efficiency-oriented and moral robots, varying based on product involvement (Study 1). Customers prefer moral robots for high-involvement products and efficiency-oriented robots for low-involvement products. In a subsequent laboratory study (Study 2), we focus on a high-involvement retail scenario to elucidate the underlying mechanisms of customer responses to moral service robots. We demonstrate that experienced robot morality benefits relational interactions by increasing customer trust. However, in transactional interactions where customers prioritize robot functionality, experienced robot morality can lead to drawbacks. By demonstrating the inadequacy of a one-size-fits-all approach to robot morality, we underscore the importance of adopting a customer-centric design for moral service robots and show that the effectiveness of their placement depends largely on the type of service encounter. These results can serve to guide managers and robot designers towards aligning robot morality with customer demands, facilitating positive human–robot interactions.

Mona Mareen Kegel

Ruth Maria Stock-Homburg

Tuure Tuunanen

Service Robot Morality in Customer-Robot Interactions: A Mixed-Methods Study

客户-机器人互动中的服务机器人道德：一项混合方法研究

The integration of robots into daily life has increased significantly, spanning applications from social-care to industrial settings with collaborative robots. Ensuring a safe, secure environment and equitable workload distribution in human-robot collaborations is crucial. Trust is a key factor in these environments, essential for enhancing collaboration and achieving tasks while maintaining safety. Under-trusting robots can hinder productivity, while over-trusting them can lead to accidents. A comprehensive literature review of 100 publications from 2003 to 2023 analyzes trust and its influencing factors in industrial and social-care contexts. Findings reveal that in industrial settings, robot-related factors are more influential, whereas in social-care, human and environmental factors play a significant role. Furthermore, the review delves into gauging trust through observable behavior, while also exploring various trust evaluation methodologies. Results show that trust can be gauged through human behaviors, physical cues, and physiological measurements. Concerning trust evaluation methodologies, traditional questionnaires have limitations, opening new opportunities for machine learning and sensor-based approaches to real-time trust evaluation, as trust is a dynamic cognitive value that evolves over time. Notably, 97% of the reviewed articles were published in the last decade, underscoring a growing interest in Human-Robot Interaction and trust within the scientific community.

Giulio Campagna

Matthias Rehm

A Systematic Review of Trust Assessments in Human-Robot Interaction

人机交互中信任评估的系统综述

With the increasing performance of text-to-speech systems and their generated voices indistinguishable from natural human speech, the use of these systems for robots raises ethical and safety concerns. A robot with a natural voice could increase trust, which might result in over-reliance despite evidence for robot unreliability. To estimate the influence of a robot's voice on trust and compliance, we design a study that consists of two experiments. In a pre-study ( \(N_{1}=60\) ) the most suitable natural and mechanical voice for the main study are estimated and selected for the main study. Afterward, in the main study ( \(N_{2}=68\) ), the influence of a robot's voice on trust and compliance is evaluated in a cooperative game of Battleship with a robot as an assistant. During the experiment, the acceptance of the robot's advice and response time are measured, which indicate trust and compliance respectively. The results show that participants expect robots to sound human-like and that a robot with a natural voice is perceived as safer. Additionally, a natural voice can affect compliance. Despite repeated incorrect advice, the participants are more likely to rely on the robot with the natural voice. The results do not show a direct effect on trust. Natural voices provide increased intelligibility, and while they can increase compliance with the robot, the results indicate that natural voices might not lead to over-reliance. The results highlight the importance of incorporating voices into the design of social robots to improve communication, avoid adverse effects, and increase acceptance and adoption in society.

Dennis Becker

Lukas Braach

Lennart Clasmeier

Teresa Kaufmann

Oskar Ong

Kyra Ahrens

Connor Gäde

Erik Strahl

Di Fu

Stefan Wermter

Influence of robots’ voice naturalness on trust and compliance

机器人语音自然度对信任与服从的影响

The goal of this paper is to investigate the use of communication robots as tools to increase the safety of first responders (specifically, law enforcement officers) during social interactions or first-response situations. This paper functions as the first stage of our investigation into communication robots for use by first responders. The law enforcement officers (LEOs) in our study were provided with a mobile communication robot to determine if such robots could be of use in the public domain for the benefit of both civilians and LEOs. As part of our study, LEOs participated in training sessions where they worked with non-weaponized teleoperated communication robots and completed pretest-posttest surveys. The surveys captured LEOs’ perceptions and attitudes towards the robot and examined the impact of robot design features on LEOs’ acceptance of the communication robot. This paper discusses the significance and potential use of communication robots as tools in the future of law enforcement. Our future works include conducting additional studies involving affected stakeholders (e.g., citizen communities) and assessing/mitigating any potential negative impact of communication robots.

短名	J. Hum.-Robot Interact.
Journal Impact	4.32
国际分区	ROBOTICS(Q2)

ISSN	2573-9522
h-index	22

涉及主题	计算机科学人工智能机器人人机交互心理学生物哲学工程类经济数学物理管理认识论人机交互神经科学操作系统政治学法学社会学社会心理学量子力学医学语言学程序设计语言地理移动机器人认知心理学系统工程
出版信息	出版商: Association for Computing Machinery (ACM)，出版周期: ，期刊类型: journal，开源期刊: 否
基本数据	创刊年份: 2018，原创研究文献占比: 97.83%，自引率:4.80%， Gold OA占比: 14.29%

ACM Transactions on Human-Robot Interaction

期刊引文格式

有 1 位以上作者的期刊

有 2 位作者的期刊

有 3 位作者的期刊

有 5 位以上作者的期刊

书籍引用格式

学位论文引用格式

网页引用格式

专利引用格式

手工熬夜修改参考文献？研飞自动匹配期刊，一键轻松成稿，支持 Word/WPS

投稿经验分享

Nonverbal Sound in Human-Robot Interaction: A Systematic Review

Experimental Assessment of Human-Robot Teaming for Multi-Step Remote Manipulation with Expert Operators

Stochastic-Skill-Level-Based Shared Control for Human Training in Urban Air Mobility Scenario

Human prior knowledge estimation from movement cues for information-based control of mobile robots during search

Service Robot Morality in Customer-Robot Interactions: A Mixed-Methods Study

A Systematic Review of Trust Assessments in Human-Robot Interaction

Influence of robots’ voice naturalness on trust and compliance

Human prior knowledge estimation from movement cues for information-based control of mobile robots during search

Service Robot Morality in Customer-Robot Interactions: A Mixed-Methods Study

Teleoperated Communication Robot: A Law Enforcement Perspective