Assessing user specifications of robot behaviour for material transport tasks

Abstract

Robots are an established component of many existing manufacturing processes. In the majority of cases, these robots operate in segregated areas, separated from human workers. However, as robots' capabilities in sensing and autonomy improve, they are expected to increasingly operate in human environments, and interact with novice, untrained users. When robots operate in human or shared environments, their tasks and behaviours need to be specified. This is typically performed by a human operator or supervisor, who may specify constraints on robot behaviour to make the robot more predictable or align its behaviour with user expectations. However, these constraints may impact robot task performance.

This thesis develops a user interface to obtain robot specifications, and proposes metrics for quantifying the specification quality, and to investigate how users create robot specifications. The metrics relate the robot's performance in the specified environment to its performance in a fully-unconstrained environment, and capture the trade-offs that users make in ensuring that the robot accomplishes its tasks while minimizing the loss of performance.

The proposed approach is evaluated in a series of user studies. The first user study sought to understand how novice users provide specifications for an autonomous robot operating in a shared warehouse environment, and to validate the metrics by applying them to user-created specifications. The metrics were then modified based on the results of the pilot study, and employed in a second, larger study. The second study trialed a modified interface and interaction scheme that implemented an interactive preference learning system, aimed at modifying specifications to improve robot performance. The modified metrics were then used to assess the quality of specifications following the preference learning system.

The two studies show that inexperienced users create a wide variety of behaviour-limiting specifications, and that they generally have difficulty creating efficient specifications, or assessing their own performance. Furthermore, the preference learning process succeeds in improving specification quality by making them more efficient and more similar between different users. Moreover, users that created specifications of worse initial quality benefit the most from the interactive learning process, as those specifications see a larger improvement.