The Industry 4.0 movement involves the concerted operation of modern technologies – the internet of things (IOT), artificial intelligence, robotics, etc. – to capture and contextualize data, and then automate manufacturing processes to reduce waste and optimize efficiency.
Few technologies are more essential to the hands-on manufacturing employee than mixed reality, most commonly encountered in the form of virtual reality (VR) and augmented reality (AR) headsets and glasses. Mixed reality takes the data captured by IoT and AI systems and helps workers visualize it in real time on the job.
“Mixed reality goes beyond simple manufacturing design concepts and enables the integration of analytics and real-time model feedback, including predictive artificial intelligence,” said Yiwen Rong, vice president of product development at uSens, which develops hand and vision tracking tools for mixed reality.
Making data visual
Industry 4.0 is all about information and connectivity. Mixed reality helps connect contextualized data to the worker through seamless, intuitive visual cues.
“Many of the major uses for mixed reality in manufacturing will take place in the factory [through] the overlaying of instructions and diagrams, the leveraging of computer vision for quality assurance, and possibly the incorporation virtual mentors/supervisors,” said Todd Richmond, IEEE fellow and director of the Mixed Reality Lab at USC.
Oftentimes, machine-learning algorithms are employed to analyze IoT data and then flag any anomalies or make recommendations to decision-makers. But still, that leaves a gap between company leaders and workers who could benefit greatly in their day-to-day jobs from access to that valuable data.
“I see AR as the user interface for IoT,” Gaia Dempsey, co-founder of mixed reality company DAQRI, told Business News Daily. “It doesn’t do you very much good to collect data and analyze but not make it actionable to folks actually in the field. AR lets you do that.”
For example, Dempsey said, inspection workers walking through a plant will be able to access unique sets of metadata associated with each machine, visually obtaining information about energy consumption, uptime and downtime, and how individual components of the device are working. Moreover, mixed reality enables inspection and maintenance staff alike to consult with a remote expert in the event they run into something they are unfamiliar with or if they require a second opinion.
“If there’s an issue visualized in AR, you can notify the appropriate people,” Dempsey said. “If something breaks in one of these machines, that’s instantly $25,000, so just one prevention and you have more than paid for the device. And you’ve not just prevented a component from breaking but prevented downtime that impacts the overall productivity of the entire company.”
Creating virtual instructions
Mixed reality also allows for the creation of virtual cues and instructions that other workers can follow. Dropping a pin or highlighting a specific location can signal other workers that an area or specific device needs their attention, and these virtual cues are seamlessly connected across the entire system, so other mixed reality devices on the network will display them in real time.
“Mixed reality can be used to plot virtual objects within the real world,” said Bas de Vos, director of IFS Labs, the creative think tank of business software supplier IFS. “It brings possibilities like showing work instructions for interacting with an actual asset. Scenarios like this … can help us to easily determine the best solution for a problem.”
Mixed reality can also be used to preview things like new furniture or machines. In this way, purchasers can understand where things might fit or how they’ll look before buying. In manufacturing, this applies to potential new devices or renovations that might be made to the production facility.
“Does this new machine fit the location in the factory? Or what would it look like if we added a door to this wall?” de Vos said.
Training and education
In addition, mixed reality has applications when it comes to training and continuing education. “On-the-job” training can be redefined in a virtual environment, making it easier and safer for new workers to practice their jobs ahead of time. Experiential learning methods are also proven to be more impactful than sitting in a classroom or reading from a textbook.
“Prior to the introduction of [mixed reality] technology, the primary training methods took place on physical equipment or in the classroom from a book or video,” said Annie Eaton, CEO of Futurus. “When training on equipment, there are safety risks and costs expended on equipment wear and tear. And when using classroom training, the employee doesn’t get the benefit of physical process training. New technologies like mixed reality can help solve these issues by implementing compelling and interactive training programs.”
Engaging, physical education programs have been proven to be more effective for employees when it comes to absorbing and retaining knowledge, as well. Mixed reality can put new workers in the driver’s seat – sometimes literally – without risking productivity or equipment. And that means a more effective training session so employees are better prepared when they begin their jobs for real.
Adding to the treasure trove of data
Mixed reality isn’t just a passive observer of data in the workplace either. Mixed reality devices like helmets, headsets and glasses are IoT-enabled Bluetooth devices that communicate with other devices as well. This means they, too, can capture data through their own sensors and upload it through the network. Workers equipped with these devices then become mobile sensors that help refine and perfect your data set, enabling more accurate predictions and better-informed decision-making.
“It’s not just displaying information from IoT strategy but also collecting data,” Dempsey said. “You can associate any kind of sensor you want and use glasses and helmets to take readings off that sensor and then pipe that data back into your predictive model.”
How companies incorporate these capabilities into a larger strategy can determine how great a competitive advantage they reap, Rong said. It’s not about the best machine anymore but the most effective use of information technology.
“The new image [of manufacturing] … is one where the most successful factories are those that are able to make the best analysis from advanced data metrics captured through every part of the manufacturing process to boost production and increase standards.”
Mixed reality is a critical piece of that puzzle, and enterprises are already beginning to adopt the technology.
“Manufacturers are already embracing AR technology and seeing ROI, because [they] … can combine multiple technologies and create their own content,” Rong said. “Enterprises are willing to make an investment into AR headsets to improve the efficiency of their employees and will buy the hardware even for a single application.”
According to the International Data Corporation (IDC), global shipments of mixed reality headsets are expected to reach nearly 100 million by 2021, a whopping compound annual growth rate of 58 percent over a five-year period. When it comes to AR headsets specifically, the commercial sector is leading the way. Of the 162,458 AR devices shipped in 2016, 68 percent were for commercial purposes, according to the IDC.
Still, mixed reality is in its early stages, and the development is anything but stagnant. According to Richmond, there are still bugs to work out, and researchers will be eagerly studying initial feedbacks to the user experience in a bid to retool and improve their systems.
“While AR will gain traction over the next five years, it’ll take a while for widespread adoption,” Richmond said. “The technology needs to improve – resolution, field of view, size and weight, tracking – but also the user experience is still not well understood, and it will take a lot of experimentation to develop the ‘language of immersion’ so that AR and VR experiences are useful and lead to better outcomes.”
While mass adoption might still be a few years off, one thing is for certain: Mixed reality is an imperative component of Industry 4.0 and possibly the most important for connecting the boots on the ground with the data in the cloud.