Recognition of Human Activities in Hybrid Teamwork with Industrial Robot Systems

The effort that is required to establish automation imposes a limitation to traditional production environments. This yields a lack of flexibility, which gains significance due to the changing consumer market. To overcome these limitations, a fundamental shift has been initiated towards a new era known as Industry 4.0. This shift is supported by rapidly evolving technological opportunities that enable novel forms of hybrid teamwork between humans and robots with significant implications for operators. Hybrid teamwork refers to the interaction of operators and robot systems in order to complete a work task in a robot cell. Cells are coherent environments that include robot systems with end-effectors, sensors and devices such as monitors. Due to the traditional isolation of robots within the cells, the design is often technology-centered rather than human-centered. However, the novel forms introduce more dynamic and uncertainty into teamwork, and consequently the way people and robots approach a given task must adapt to the evolving conditions. The challenge is to enable shared work tasks to be completed with as few agreements and interruptions as possible. Therefore, robot systems need to recognize human activities to interact, coordinate actions and avoid waiting times as well as confusion. In order to enable this, a (re)design of traditional environments towards increased flexibility and human-centered teamwork is required. In this thesis, I present my research on hybrid teamwork in industrial robot cells. This involves the recognition of human activities such as gestures or body movements to coordinate robot system actions. However, in order to establish hybrid teamwork, a number of research gaps still need to be addressed. This involves the analysis of human and hybrid teams in terms of internal and external relationships, the desired behavior of robot systems, and frequent human activities in typical teamwork situations. With regard to the realization of robot cells, a holistic view without shortcuts in the implementation and usable domain-specific datasets are missing. Therefore, I define industrial requirements, provide operator training strategies, integrate cognitive skills, and establish realistic domain-specific datasets as a foundation for robot training. Finally, the state-of-the-art in human activity recognition does not provide solutions that meet the requirements of Industry 4.0. The approaches lack flexibility as they are difficult to transfer, especially between initially unknown sources, and require large domain specific datasets to ensure acceptable reliabilities. I provide approaches that enable high performance under industrial conditions, cross-modality fusion of unknown sources, and weakly-supervised learning from unprepared samples. For the evaluation and training, generic datasets are used to improve transferability, along with realistic and limited domain-specific datasets to prove applicability. The finally introduced weakly-supervised fusion increases reliability by incorporating additional, but initially unknown, modalities in conjunction with flexibility and reliability by incorporating additional unprepared samples. Overall, this work contributes to the improvement of hybrid teamwork and identifies challenges of human-centered design in an industrial context.

Level thesis:degree_level

thesis.doctoral

Grantor dc:publisher

Universität Bielefeld

Year

2022

Author dc:creator

• Pohlt, Clemens

Repository record source_url

https://pub.uni-bielefeld.de/record/2963524

OAI identifier oai:identifier

oai:pub.uni-bielefeld.de:2963524