Papers related to the human-centered solutions

Here you can find our latest publications about the developed human-centered frameworks and algorithms. You can find all of our papers at the site of the research group:

Knowledge Graph-Based Framework to Support Human-Centered Collaborative Manufacturing in Industry 5.0

The importance of highly monitored and analyzed processes, linked by information systems such as knowledge graphs, is growing. In addition, the integration of operators has become urgent due to their high costs and from a social point of view. An appropriate framework for implementing the Industry 5.0 approach requires effective data exchange in a highly complex manufacturing network to utilize resources and information. Furthermore, the continuous development of collaboration between human and machine actors is fundamental for industrial cyber-physical systems, as the workforce is one of the most agile and flexible manufacturing resources. This paper introduces the human-centric knowledge graph framework by adapting ontologies and standards to model the operator-related factors such as monitoring movements, working conditions, or collaborating with robots. It also presents graph-based data querying, visualization, and analysis through an industrial case study. The main contribution of this work is a knowledge graph-based framework that focuses on the work performed by the operator, including the evaluation of movements, collaboration with machines, ergonomics, and other conditions. In addition, the use of the framework is demonstrated in a complex use case based on an assembly line, with examples of resource allocation and comprehensive support in terms of the collaboration aspect between shop-floor workers.

Extension of HAAS for the Management of Cognitive Load

The rapid advancement of technology related to Industry 4.0 has brought about a paradigm shift in the way we interact with assets across various domains. This progress has led to the emergence of the concept of a Human Digital Twin (HDT), a virtual representation of an individual’s cognitive, psychological, and behavioral characteristics. The HDT has demonstrated potential as a strategic tool for enhancing productivity, safety, and collaboration within the framework of Industry 5.0. In response to this challenge, this paper outlines a process for tracking human cognitive load using the galvanic skin response as a physiological marker and proposes a novel method for managing cognitive load based on the extended Human Asset Administration Shell (HAAS). The proposed HAAS framework integrates real-time data streams from wearable sensors, user skills, contextual information, task specifics, and environmental and surrounding conditions to deliver a comprehensive understanding of an individual’s cognitive state, physical wellness, and skill set. Through the incorporation of skills set, physical, physiological, and psychological variables, and task parameters, the developed HAAS framework enables the identification, management, and development of individual capabilities, thereby facilitating individualized training and knowledge exchange. The applicability of the developed framework is proved by an experiments in the Operator 4.0 laboratory with the detailed HAAS parameters.

Extending factory digital Twins through human characterisation in Asset Administration Shell

This paper extends the traditional factory digital twins by incorporating human characterisation in Asset Administration Shell (AAS). The extension lays the basis for human-centred control and management, as demonstrated by employing a prototype of the extended AAS in two proposed use cases. Referred to Industry 5.0, an accurate digital representation of humans as a basis of the data-based decision support to improve operators’ well-being and resilience. The AAS is extended to include dedicated digital models accommodating a set of properties to describe the human operators and its interactions with the surrounding shop-floor resources. Two reference use cases have been designed in the context of a complete lab-scale manufacturing system: equipment and devices have been modelled according to the AAS standard, exposing information via MQTT, and have been integrated with the proposed AAS definition of human operators. Operators have been equipped with wearable sensors and a dashboard providing them with feedback from the manufacturing environment and notifications about changes. As part of the extension process, some ethical and regulation concerns are discussed, highlighting that the extended AAS is mature enough to support the inclusion of human operators, but regulations struggle to keep up with technological advances.

Indoor Positioning-based Occupational Exposures Mapping and Operator Well-being Assessment in Manufacturing Environment

This research was motivated by the need for detailed information about the spatial and contextualized distribution of occupational exposures, which can be used to improve the layout of the workspace. To achieve this goal, the study emphasizes the need for position-related information and contextualized data. To address these concerns, the study proposes the use of Indoor Positioning System (IPS) sensors that can be further developed to establish a set of metrics for measuring and evaluating occupational exposures. The proposed IPS-based sensor fusion framework, which combines various environmental parameters with position data, can provide valuable insights into the operator’s working environment. For this, we propose an indoor position-based comfort level indicator. By identifying areas of improvement, interventions can be implemented to enhance operator performance and overall health. The measurement unit installed on a manual material handling device in a real production environment and collected data using temperature, noise, and humidity sensors. The results demonstrated the applicability of the proposed comfort level indicator in a wire harness manufacturing setting, providing location-based information to enhance operator well-being. Overall, the proposed framework can be used as a tool to monitor the industrial environment, especially the well-being of shop floor operators.

Assessing human worker performance by pattern mining of Kinect sensor skeleton data

The human worker is an in-disposable factor in manufacturing processes. Traditional observation methods to assess their performance is time-consuming and expert-dependent, while it is still impossible to diagnose the detailed movement trajectory with the naked eye. Industry 4.0 technologies can innovate that process with smart sensors paired with data mining techniques for automated operation and develop a database of frequent movements for corporate reference and improvement. This paper proposes an approach to automatically assess worker performance with skeleton data by applying pattern mining methods and supervised learning algorithms. A use case is performed on an electrical assembly line to validate the approach, with the skeleton data collected by Kinect sensor v2. By using supervised learning, the movements of workers in each workstation can be segmented, and the line performance can be assessed. The work movement motifs can be recognized with pattern mining. The mined results can be used to further improve the production processes in terms of work procedures, movement symmetry, body utilization, and other ergonomics factors for both short and long-term human resource development. The promising result motivates further utilization of easy-to-adopt technology in Industry 5.0, which facilitates human-centric data-driven improvements.

Current development on the Operator 4.0 and transition towards the Operator 5.0: A systematic literature review in light of Industry 5.0

Technology-driven Industry 4.0 (I4.0) paradigm combined with human-centrism, sustainability, and resilience orientation, forms the Industry 5.0 (I5.0) paradigm, providing support for the workforce and enabling the Operator 4.0 (O4.0) approach. The I5.0 focuses can face unforeseen challenges, as the applicability and readiness of I4.0 solutions are still not well discussed in the literature. Therefore, structuring existing knowledge of O4.0 to prepare for the smooth transition toward Operator 5.0 (O5.0) is crucial. A systematic literature review is performed in the Scopus database, considering publications up to 31 December 2022. Bibliography Network Analysis (BNA), text mining techniques (i.e., Latent Dirichlet Allocation (LDA), BERTopic), and knowledge graph (KG) were deployed on the retrieved abstracts. The full-text examination is carried out over papers matched by LDA and BERTopic. From the BNA result of 279 relevant papers, clusters of active researchers and topics were found, while text-mining results revealed trending and missing research directions. The extracted details from the full text of 81 papers reflected the coverage and development levels of O4.0 types with the preparation for resilience, human-centrism, and sustainability. Achieved results suggest that though the O5.0 transition is inevitable, I4.0 technologies are not ready with sufficient human factor integration. Missing research orientations including integrated sustainability from the human perspective, or system resilience, concerning drivers and restrainers for technology adoption. To prepare for O5.0, discussed O4.0 drivers can help to shape the favorable conditions, and the restrainers should be mitigated before adopting human-centric technologies. Further study including grey literature is necessary to exploit more industrial and policy-making perspectives.

Heart Rate Variability Measurement to Assess Acute Work-Content-Related Stress of Workers in Industrial Manufacturing Environment—A Systematic Scoping Review

Background: Human workers are indispensable in the human–cyber-physical system in the forthcoming Industry 5.0. As inappropriate work content induces stress and harmful effects on human performance, engineering applications search for a physiological indicator for monitoring the well-being state of workers during work; thus, the work content can be modified accordingly. The primary aim of this study is to assess whether heart rate variability (HRV) can be a valid and reliable indicator of acute work-content-related stress (AWCRS) in real time during industrial work. Second, we aim to provide a broader scope of HRV usage as a stress indicator in this context. Methods: A search was conducted in Scopus, IEEE Xplore, PubMed, and Web of Science between 1 January 2000 and 1 June 2022. Eligible articles are analyzed regarding study design, population, assessment of AWCRS, and its association with HRV. Results: A total of 14 studies met the inclusion criteria. No randomized control trial (RCT) was conducted to assess the association between AWCRS and HRV. Five observational studies were performed. Both AWCRS and HRV were measured in nine further studies, but their associations were not analyzed. Results suggest that HRV does not fully reflect the AWCRS during work, and it is problematic to measure the effect of AWCRS on HRV in the real manufacturing environment. The evidence is insufficient for a reliable conclusion about the HRV diagnostic role as an indicator of human worker status. Conclusion: This review is valuable in the Operator 4.0 paradigm, calling for more trials to validate the use of HRV to measure AWCRS on human workers.

The human-centric Industry 5.0 collaboration architecture

While the primary focus of Industry 4.0 revolves around extensive digitalization, Industry 5.0, on the other hand, seeks to integrate innovative technologies with human actors, signifying an approach that is more value-driven than technology-centric. The key objectives of the Industry 5.0 paradigm, which were not central to Industry 4.0, underscore that production should not only be digitalized but also resilient, sustainable, and human-centric. This paper is focusing on the human-centric pillar of Industry 5.0. The proposed methodology addresses the need for a human-AI collaborative process design and innovation approach to support the development and deployment of advanced AI-driven co-creation and collaboration tools. The method aims to solve the problem of integrating various innovative agents (human, AI, IoT, robot) in a plant-level collaboration process through a generic semantic definition, utilizing a time event-driven process. It also encourages the development of AI techniques for human-in-the-loop optimization, incorporating cross-checking with alternative feedback loop models. Benefits of this methodology include the Industry 5.0 collaboration architecture (I5arc), which provides new adaptable, generic frameworks, concepts, and methodologies for modern knowledge creation and sharing to enhance plant collaboration processes.

Self-improving situation awareness for human–robot-collaboration using intelligent Digital Twin

The situation awareness, especially for collaborative robots, plays a crucial role when humans and machines work together in a human-centered, dynamic environment. Only when the humans understands how well the robot is aware of its environment can they build trust and delegate tasks that the robot can complete successfully. However, the state of situation awareness has not yet been described for collaborative robots. Furthermore, the improvement of situation awareness is now only described for humans but not for robots. In this paper, the authors propose a metric to measure the state of situation awareness. Furthermore, the models are adapted to the collaborative robot domain to systematically improve the situation awareness. The proposed metric and the improvement process of the situation awareness are evaluated using the mobile robot platform Robotino. The authors conduct extensive experiments and present the results in this paper to evaluate the effectiveness of the proposed approach. The results are compared with the existing research on the situation awareness, highlighting the advantages of our approach. Therefore, the approach is expected to significantly improve the performance of cobots in human–robot collaboration and enhance the communication and understanding between humans and machines.

Demonstration Laboratory of Industry 4.0 Retrofitting and Operator 4.0 Solutions: Education towards Industry 5.0

One of the main challenges of Industry 4.0 is how advanced sensors and sensing technologies can be applied through the Internet of Things layers of existing manufacturing. This is the so-called Brownfield Industry 4.0, where the different types and ages of machines and processes need to be digitalized. Smart retrofitting is the umbrella term for solutions to show how we can digitalize manufacturing machines. This problem is critical in the case of solutions to support human workers. The Operator 4.0 concept shows how we can efficiently support workers on the shop floor. The key indicator is the readiness level of a company, and the main bottleneck is the technical knowledge of the employees. This study proposes an education framework and a related Operator 4.0 laboratory that prepares students for the development and application of Industry 5.0 technologies. The concept of intelligent space is proposed as a basis of the educational framework, which can solve the problem of monitoring the stochastic nature of operators in production processes. The components of the intelligent space are detailed through the layers of the IoT in the form of a case study conducted at the laboratory. The applicability of indoor positioning systems is described with the integration of machine-, operator- and environment-based sensor data to obtain real-time information from the shop floor. The digital twin of the laboratory is developed in a discrete event simulator, which integrates the data from the shop floor and can control the production based on the simulation results. The presented framework can be utilized to design education for the generation of Industry 5.0. 

Human-centered knowledge graph-based design concept for collaborative manufacturing

With the increasing importance of highly connected and monitored processes supported by industrial information systems, such as knowledge graphs, the integration of the operator has become urgent due to its high cost and is also to be appreciated from a social point of view. The facilitation of collaboration between humans and machines is a fundamental target for Industrial Cyber-Physical Systems, as the workforce is the most agile and flexible manufacturing resource. Furthermore, the design of such a framework requires effective systems to utilise resources and information. This paper aims to provide recommendations of ontologies and standards that can support monitoring work conditions, scheduling, planning and supporting the operator and the possibilities to formalise the classic work instructions to analyse the unique activities. The main contributions of the work are that it proposes a design work-frame of a knowledge graph where the work performed by the operator is in the scope, including the evaluation of movements, collaboration with machines, work steps, ergonomics and other conditions. The paper highlights that activity recognition technologies can enhance the utilisable data in a knowledge graph for a smart factory. With this approach, the future goal may be to automate the entire data collection and knowledge exploration processes, which can facilitate the support of the human-digital twin and the implementation of augmented reality technologies in the Industry 5.0 concept.

Intelligent Collaborative Manufacturing Space for Augmenting Human Workers in Semi-Automated Manufacturing Systems

Manufacturing companies are facing two major trends affecting their business operations: "automatization" and "collaboration". Companies have realized that they still need humans on the shop floor beside the availability of high levels of automation solutions in the market. This realization has created a new Industrial Revolution known as "Industry 5.0". While the primary concern in Industry 4.0 is about achieving high levels of full automation, Industry 5.0 focuses on creating synergies between humans and autonomous machines in semi-automated manufacturing systems toward flexible, resilient, and sustainable systems. The critical element of human-automation synergies is a better understanding of the excellent cooperation between humans and making a better collaboration between humans and autonomous machines inspired by it. The proposed Intelligent Collaborative Manufacturing Space (ICMS) aims to create a framework for supporting collaborations based on smart sensor networks and data science techniques. Four main elements or sub-spaces characterize this "Intelligent Workspace": (i) the Working Space, (ii) the Monitoring Space, (iii) the Modelling Space, and (iv) the Decision Space. The ICMS is a framework envisioned for supporting the effective collaboration between humans and automated and semi-automated production assets based on activity recognition and prediction paired with machine learning optimization algorithms. A methodology for developing ICMSs is described in detail in this paper. 

Trajectory Prediction of Moving Workers for Autonomous Mobile Robots on the Shop Floor

In partially automated manufacturing, humans work together with mobile robots. Trajectory prediction, i.e. predicting future positions of human workers, improves collaboration and coexistence between humans and robots on the shop floor. In this paper, we discuss the interrelated research questions of how human motion trajectories can be predicted and how mobile robots such as Autonomous Mobile Robots and Automated Guided Vehicles can take such predictions into account in their pathfinding and navigation. On the robot side, advanced D* pathfinding algorithms allow robots to take dynamic obstacles into account. For trajectory prediction, the position of human workers is determined by an Ultra-Wideband-based Real-Time Locating System. A trajectory prediction framework is introduced to support the implementation and use of pattern- and planning-based trajectory prediction algorithms. The evaluation is based on scenarios from the addressed problem area of manufacturing.

Hypergraph-based analysis and design of intelligent collaborative manufacturing space

A method for hypergraph-based analysis and the design of manufacturing systems has been developed. The reason for its development is the need to integrate the human workforce into Industry 4.0 solutions. The proposed intelligent collaborative manufacturing space enhances collaboration between the operators as well as provides them with valuable information about their performance and the state of the production system. The design of these Operator 4.0 solutions requires a problem-specific description of manufacturing systems, the skills, and states of the operators, as well as of the sensors placed in the intelligent space for the simultaneous monitoring of the cooperative work. The design of this intelligent collaborative manufacturing space requires the systematic analysis of the critical sets of interacting elements. The proposal is that hypergraphs can efficiently represent these sets, moreover, studying the centrality and modularity of the resultant hypergraphs can support the formation of collaboration and interaction schemes and the formation of manufacturing cells. A fully reproducible illustrative example presents the applicability of this concept. 

Heart Rate Variability Measurement to Assess Work-Related Stress of Physical Workers in Manufacturing Industries - Protocol for a Systematic Literature Review

Although Industry 4.0 automation has replaced the human workforce by machines in industries, physical workers are indispensable in many facilities. One primary source of stress in the manufacturing workplace is inappropriate work content (e.g., unreasonable workload and work pace). With workers facing physical activities or machine operations, work-related stress can affect their performance and cause productivity, quality, and safety problems. Thus, industrial managers constantly seek ways to ease stress among their workforce, as Healthy Operator is proposed as an essential pillar of the Operator 4.0 concept. As the first step of stress reduction is to identify the stress (if possible, its early signs), heart rate variability (HRV) is widely measured, with different methods and technologies adopted to assess the stress levels of workers. Recent technological advancements have developed many non-invasive measurement techniques and systems, offering more convenience and flexibility than traditional clinical methods. Since human centricity is a strategic focus of the forthcoming Industry 5.0 initiative proposed by the European Union, these measurement practices should be disseminated and shared to foster better stress identification and reduction. A systematic literature review is needed to deliver a comprehensive update in the field. Besides synthesizing the relevant development, the review study aims to motivate industrial managers to adopt a similar approach and provide helpful guidance on what to expect with the Heart Rate Variability measurement for the workplace stress assessment. A detailed protocol for the systematic literature review is given. 

Architecture of a Human-Digital Twin as Common Interface for Operator 4.0 Applications

At collaborative workspaces, humans and robots share the shop floor and work closely together. Operator 4.0 is a wide research topic and its solutions aim at the creation of Human-centered Cyber-Physical Systems that improve operators’ capabilities. Such applications require a bi-directional flow of information and need data, models and simulations of machines as well as humans. To realize a common interface for information, the concept of Digital Twin is promising. This paper therefore discusses the adaption of conventional Digital Twin architectures and presents a derived Human-centered Digital Twin (H-DT) architecture designed for operators in production and intralogistics. 

Trajectory Prediction of Humans in Factories and Warehouses with Real-Time Locating Systems

Flexible intralogistics systems use automated guided vehicles (AGV) to transport goods. In assembly and warehouses, AGVs and human workers often work side by side. For optimal navigation, AGVs must consider human movement and estimate future positions of workers. Using real-time locating systems (RTLS) to improve human-robot collaboration enables more energy-efficient and safer AGV wayfinding strategies. This paper gives a summary on the topics RTLS, AGV wayfinding and trajectory prediction and introduces the momentum-based approach to predicting future worker positions in factories and warehouses. The results show that ultra-wideband-based RTLS are very well suited for trajectory prediction in the production sector. 

Worker movement diagram based stochastic model of open paced conveyors

Human resources are still utilized in many manufacturing systems, so the development of these processes should also focus on the performance of the operators. The optimization of production systems requires accurate and reliable models. Due to the complexity and uncertainty of the human behavior, the modeling of the operators is a challenging task. Our goal is to develop a worker movement diagram based model that considers the stochastic nature of paced open conveyors. The problem is challenging as the simulator has to handle the open nature of the workstations, which means that the operators can work ahead or try to work off their backlog, and due to the increased flexibility of the moving patterns the possible crossings which could lead to the stopping of the conveyor should also be modeled. The risk of such micro-stoppings is calculated by Monte-Carlo simulation. The applicability of the simulator is demonstrated by a well-documented benchmark problem of a wire-harness production process. 

Enabling Technologies for Operator 4.0: A Survey

The fast development of smart sensors and wearable devices has provided the opportunity to develop intelligent operator workspaces. The resultant Human-Cyber-Physical Systems (H-CPS) integrate the operators into flexible and multi-purpose manufacturing processes. The primary enabling factor of the resultant Operator 4.0 paradigm is the integration of advanced sensor and actuator technologies and communications solutions. This work provides an extensive overview of these technologies and highlights that the design of future workplaces should be based on the concept of intelligent space. 

BSc/MSc thesis

Design of Experiments to evaluate the collaborative work between human and robot

Industrial development is evolving towards an increased focus on human-machine collaboration, exemplified by the emergence of Industry 5.0. This new industrial paradigm prioritizes solutions centered around humans and emphasizes resilience and sustainability. A critical aspect of this collaboration is the need for robots to possess more advanced cognitive abilities, allowing for safe co-working environments with humans. Advancements in technology have enabled various methods for monitoring human behavior. Analyzing these behavioral patterns enhances efficiency and collaboration. 

My thesis introduces a setting where monitoring the operator and the robot is possible. It is achieved through a camera, an indoor positioning system, and, in the future, wearable sensors. Indicators, for example, the utilization of participants or waiting times, can represent the collaboration.

Measurements were taken with a specific focus, and the results were analyzed comprehensively—the thesis aimed to establish a connection between human activities and collaborative robots while considering human needs. The process used to achieve this objective is demonstrated through two complex games, and the design of experiments, which were then used to create and do more comprehensive experiments, is also presented.

Sensor fusion-based occupational environmental exposures mapping assessment using a mobile measuring unit

Workers in manufacturing companies are changing jobs more frequently nowadays. The main causes of this are that the operators spend a lot of time working on uncomfortable (noisy, poor air quality) shop floors that are not specifically made for them. An unsuitable production environment can put operators under increased stress, which can have a significant impact on their workload and thus productivity. Therefore, it would be crucial to measure the shop-floor's environmental impact in a cost-effective way.

To achieve this goal, the study emphasizes the need for position-related information and contextualized data. To address these concerns, the study proposes the use of Indoor Positioning System (IPS) sensors that can be further developed to integrate mapping techniques and establish a set of metrics for measuring and evaluating occupational exposures. The proposed IPS sensor fusion framework, which combines various environmental parameters with position data, can provide valuable insights into the operator's working environment. By identifying areas of improvement, interventions can be implemented to enhance operator performance and overall health.

Modelling and estimating operator activity

To plan production and estimate the costs of the production process, manufacturing companies use so-called process or base activity times. These activity times are often estimated or based on field measurements and are dependent on the subjectivity of the engineer who estimates the time effort of the processes. The main subject of the thesis is the development of a data collection and processing system that fits into the Operator 4.0 concept and can be adapted to a brownfield Industry 4.0 manufacturing systems.

Due to the stochastic nature of the problem - different workplace layouts, a wide spectrum of production processes, different work environments, different work habits, additional stress due to measurement and data collection - the objectives include the creation of a generally applicable flexible system.

After reviewing the applicable technologies with a view to making them as adaptable as possible, I concluded that visual information gathering is the easiest way to monitor existing production systems, because it allows both workers and machines to be observed and evaluated.

Optical flow-based displacement detection, clustering of displaced points, tracking of displaced objects with object tracking algorithms and machine learning-based technologies are the most suitable for modelling workers activity and for qualitative and quantitative performance evaluation.

In this study, the analysis of the manufacturing of a product in a one-operator and a two-operator work model at the investigated workplace is presented. By processing the data, the activity performed by the workers and their performance can be measured using indicators assigned to job roles. In addition to human performance, machine utilisation, number of pieces produced and cycle times can also be measured using the framework.

TDK (Students' Scientific Conference)

Workspace design and monitoring framework to evaluate the collaboration between human-machine

Industrial development is marked by a pronounced shift towards human-machine collaboration, a trend exemplified by the beginning of Industry 5.0. This evolving industrial paradigm prioritizes human-centered solutions, emphasizing resilience and sustainability. A vital part of this collaboration is the need for robots to possess more complex cognitive abilities, facilitating safe co-working environments with humans. Technological advances have enabled diverse methods for monitoring human behavior. Analyzing these behavioral patterns enhances efficiency and collaboration.

This study introduces a setting where monitoring the operator and the robot is possible. It is achieved through a camera, an indoor positioning system, and wearable sensors. The collaboration can be represented by indicators (utilization of participants, level of stress based on physiological measurements of the person).

Targeted measurements were conducted, and the results underwent comprehensive analysis. The objective is to bridge the gap between human activities and collaborative robots and design their service to human needs. The developed methodology is presented in a complex demonstration game and a design of experiments is shown.

Development of a discrete event simulation model for human-machine collaborative work

The advent of collaborative robots has made it technically feasible for humans and machines to work together in a shared workspace. A fundamental spirit of the fifth industrial revolution is to develop human-centred solutions and processes. This is necessary because full automation has its limits, both economically and technologically.

The aim of my research is to develop a discrete event simulation-based toolkit to efficiently and purposefully investigate the efficiency and feasibility of these collaborative processes. In the developed simulation model, I use key indicators to investigate collaborative and decoupled manufacturing between humans and machines. The simulation will make it possible to investigate more cost-effective improvements without interrupting real production. My work will show how to reduce the resource requirements of manufacturing and maximise the utilisation of robotic arms and workers. For a specific automotive supplier, I will prepare a case study for a specific workflow, where I will evaluate the workload of a given collaborative robot and operators.

Risk-based maintenance schedule based on Markov-chains

The equipment and tools used in manufacturing have a lifetime, during which they need to undergo a number of maintenance operations. When we want to optimise the number of maintenance operations, we have to take into account which costs more: frequent maintenance or the cost of repairing the breakdown and the downtime it causes. In complex manufacturing systems, especially in the automotive industry, this scheduling task challenges traditional optimization algorithms.

Optimising the number of maintenance operations is therefore a key productivity driver to avoid downtime caused by unexpected failures, minimise downtime caused by maintenance and maintain acceptable product quality. The problem is that we do not know when a part or all of the machine will fail.

In my work, I work with machine event log data where only machine events are recorded, no diagnostic data. I use this information to build a Markov model of machine manufacturing and changeover processes. My goal is to create a model that can optimize the scheduling of preventive maintenance. I will use a Markov chain to handle the risk probabilities, the optimization and a population-based algorithm to solve the optimization.

Enabling Technologies for Operator 4.0: A Survey

Industrial development is increasingly focusing on human-machine collaboration. As Industry 5.0 is being introduced, more human-centered solutions are presented. The key elements of Industry 5.0 are resilience and sustainability, and it is also more human-centric, concentrating on collaboration between people and machines. Collaborative work requires that the robots have more complex cognitive abilities. These abilities allow humans and robots to work together safely near each other.

However, safety is only one part of human-robot collaboration. Scheduling the tasks of the operator and robot creates an environment where collaboration is safely achievable. Sharing activities and defining and scheduling these are complex tasks requiring monitoring and recognizing the operator’s actions. Thanks to technological advancements, several methods exist to monitor human behavior, such as cameras, indoor positioning systems, and wearable sensors. The patterns can be analyzed, and the result can be used the enhance efficiency and collaboration. 

This work presents a workspace where monitoring the operator and robot is achievable using a camera, an indoor positioning system, and wearable sensors. Targeted measurements were made, and the results were analyzed. The developed methodology and optimization tools are presented in a complex demonstration game, and finally, will be detailed on how they can be applied in a real industrial environment.

Estimation of the operator comfort level and the layout information based on sensor fusion techniques

The most significant problem for manufacturing companies today is human resource shortages. The problem with industrial workplaces is that the operators work long hours on uncomfortable shop floors (noisy, bad condition of air quality) that are not designed individually for the operators. We can get information about these parameters by measuring the environmental data on the shop floor. These aspects have an impact on production performance. The so-called comfort level of the operator can be determined by fusing different sensors. After all, the more comfortable the environment is, the more human worker can be productive, and the company can have a more comfortable workspace.

Our research aims to develop a framework that uses sensor fusion techniques to create a detailed layout. A mobile robot is designed to evaluate the comfort level of the shop floor zones. The mobile robot has different sensors. Thus we developed a so-called mobile measuring unit. We use temperature, noise, and humidity sensors to measure the environmental parameters to derive the comfort level. Besides, the indoor positioning system tracks all resources (including humans) during production. We will use 3D LiDAR in addition to the indoor positioning system to achieve more accurate positioning. Also, the robot provides a visual image (camera) of the inspected area, which will fuse with the LiDAR point cloud to provide a more accurate visual monitoring of the production. The developed mobile measure unit with the sensor-fusion algorithm is tested in the Operator 4.0 laboratory at the University of Pannonia.

Workspace design for human-machine collaboration monitoring

The human-machine collaboration is coming to the forefront of industrial development. As the concept of Industry 5.0 is being introduced, more human-centered solutions are proposed, as opposed to the more machine-centric solutions in Industry 4.0. The main focus of Industry 5.0 is flexibility, sustainability, and the mentioned human-machine col- laboration. To create a more human-centric workplace, the workers’ behavior should be monitored. Technological advances have allowed new methods to monitor human behavior in the past years. For example, people’s motions can be tracked using the help of camera systems or indoor positioning systems. The patterns can be analyzed, and the result can be used the improve efficiency and collaboration.

However, this has several shortcomings. The goal is to prepare the collaborative robot based on human activities so that they serve people.

This work presents a workspace where the operator and robot can be observed using a camera and an indoor positioning system during their collaboration. The developed monitoring space and tools are presented in a complex demonstration game, and finally, it will be detailed on how they can be applied in a real industrial environment.