Blog

Looking to embark on Industry 4.0 journey? Start (Modernize OT) and Accelerate (Leverage AI): A Two-step IIoT Blueprint

(with customer stories)

 

By Amit Jain

 

 

Only 1 in 5 manufacturers are demonstrating high levels of maturity in both smart products and smart factories

        Source: KPMG: i4.0 maturity assessment study

It is quite evident that Industry 4.0 initiatives are underway across the factory floors, distribution centers and supply chain for product design, product automation and delivery.

This is to drive high levels of business process efficiency, reduce downtime and boost productivity. However, initiatives and ambitions don't necessarily translate to successful execution. It is estimated that only 1% of the data collected by companies is used. In our opinion, two fundamental issues are holding businesses back:

  1. OT complexity:  There is huge OT complexity around legacy sensors and SCADA systems. The data needs to be tapped, transformed and visualized across different IIoT silos. Couple this with the fact that one has to manage 10s or 100s of those factory floors, each of which is mission critical.

  2. IoT and AI operationalization at industrial scale: The core personality of IoT and AI applications is going to be very different requiring rich runtime and data services - cloud-native support, continuous integration and continuous delivery (CI/CD) etc. for the developers, so the question becomes - what is the operating stack that can support these modern data science applications?

Embark on the Industry 4.0 journey with a two-step blueprint...

Step-1 is to modernize OT by removing complexity and providing visibility, which becomes the foundational step for factory automation.

Once businesses have unprecedented visibility of their operations, Step-2 involves leveraging AI for acting upon those insights in an automated manner (predictive maintenance) and applying machine vision capabilities.  

Ride on Nutanix Xi IoT: Intelligent Edge Computing Platform

The Nutanix Xi IoT platform is a software-defined solution that delivers local computing, machine learning, and intelligence for your edge devices, converging the edge and your choice of cloud into one seamless and delightful application development platform. The Xi IoT Edge Platform eliminates OT complexity, streamlines collection of data, and elevates IT to focus on the business logic for data processing. The edge PaaS supports easy-to-use developer APIs, reusable data pipelines, kubernetes apps (containers-as-a-service), serverless functions and pluggable machine learning architecture to enable rapid development and global deployment of modern IoT and AI apps. 

With Xi IoT Edge Platform, we were able to demonstrate to our customers that Step-1 entails a simple Machine Condition Monitoring (MCM) use case to start the journey but indeed a very impactful one to see the outcomes in a few weeks time frame. Gradually, one can build towards more intermediate (Predictive Maintenance) to advanced AI (Product Quality Inspection, Smart Supply Chain etc.) use cases.

Here are more details on the two-step blueprint and the customer stories:

Step #1: Modernize OT for Factory Automation and Machine Condition Monitoring (MCM)

This is to simplify floor operations and provide visibility for enabling factory automation. You need a unified data processing platform to derive insights out of the data streaming from various sensors or stored in data historians. So, the first step in the journey is solving for Machine Condition Monitoring (MCM) i.e. provide streamlined machine data acquisition, collection, transformation and centralized visualization. This is the foundational step for building Industry 4.0 capabilities. It is sort of an easy step to start the journey but a very impactful one to see the outcomes in a very short period of time.

Let’s see some real-world deployment proof points. 

Customer Story #1: MCM Type-1

Problem Statement:

On the automotive parts manufacturing floor, where injection molding machines already have sensors and Omron PLCs, however there is no mechanism to tap the streaming data and generate insights. The requirement is to increase machine up-time and provide operation visibility by reading temperature and pressure data from a motor that is part of an injection molding machine. 

The factory managers need to be able to monitor this data centrally for increased visibility and reduced reaction time. Note that a minor change in temperature and pressure parameters generates a heat profile that can cause part defects.

Deployment and Benefits:

The shop floor (edge) deployment encompasses data collection from an injection molding machine with delivery to a cloud based data lake for building multi-tenant dashboarding capability. 

Nutanix Xi IoT Edge software platform is deployed as bare metal on Supermicro x86 edge hardware (i7 CPU, 8GB RAM) and is connected to Omron PLC via Ethernet.  For security, the respective OT and IT networks are connected via separate physical ports, the former being on the private subnet. Xi IoT’s Infrastructure and Application Lifecycle Manager provides management simplicity for planet-scale operations of hundreds to thousands of edge sites. Xi IoT Edge platform provides zero-touch setup and management of edge devices, so operators are able to reduce the risk of IoT security breaches due to human error and increase overall management efficiency. Xi IoT Edge platform enables secure multi-tenancy through user isolated Projects, allowing admins to configure and allocate resources for use by multiple teams on the shared infrastructure. 

Data variety is handled by having necessary protocol drivers within the Xi IoT Edge software stack. The basic popular protocols like MQTT, RTSP etc. are natively supported within the stack, while the rest of other protocols leverage the partner software from Litmus Automation. Moreover, the Xi IoT Edge platform is extensible, via data ifc (interface), so that any other protocol driver can easily be ported onto the Xi IoT Edge platform. In this particular data acquisition setup, FINS protocol is used to connect to Omron PLCs, which relays the temperature and pressure data. Data tags with appropriate scaling factor are mapped to create the right human readable and understandable values.

Data collection feeds the steaming temperature and pressure data into a common Xi IoT Edge in-memory message bus, which scales very well to handle the massive influx of data volume. The streaming data is also stored directly onto the platform-managed persistent Pub/Sub data service (Kafka). Any Time-series database (TSDB) can be configured onto the software stack as well.

The streaming data is sampled, filtered, correlated and transformed leveraging Functions (written in Python | Node.js | Golang etc.). This makes it very easy for operators (can be a third-party ISV as well) to focus just on the business logic without worrying about the underlying stitching and the runtime libraries. Think of functions as LEGO building blocks, whereby operators can apply multiple, cascaded functions to transform the incoming data. In the post process phase, the data is sent across the cloud to AWS S3, leveraging the Xi IoT Edge Data Pipeline

AWS S3 becomes the basis for building custom designed dashboards. The centralized dashboards help the managers to evaluate KPIs across various plants and recommend any process or equipment changes. It also reduces the time to review compliance checks. The customer was able to reduce the raw material defects Here is an example dashboard from our partner Intellithink.

The solution was rolled out under 2 weeks with active collaboration of the OT team. The customer was able to drive up the OEE (Overall Equipment Effectiveness) and reduce the raw material wastage by 15-20%.

Customer Story #2: MCM Type-2

Problem Statement:

A parts manufacturing plant floor, with existing OT and IT systems but with IoT silos and fragmented lifecycle management. The IIoT software, Kepware Kepserverex or Aveva Wonderware, run as bare metal Windows applications on standalone Industrial PCs (IPC). Each IIoT software has its own data stream going to separate IT systems. Multiple shop floors per site with many IPCs creates hardware and software proliferation, which in turn leads to fragemened device and app lifecycle management. One has to login into each box to do the lifecycle management of these applications as well as the configuration and device management of corresponding sensors. 

Deployment and Benefits:

The deployment was done in two phases. Phase-I consolidated the data from various IPCs into a unified data processing platform so that there is a common data bus which can correlate the data from multiple incoming streams to provide meaningful insights. There are well-defined APIs (open, Apache class) to tap (read/write) the data off the bus and send across for a customized dashboard. Here is an example video of how Xi IoT was securely connected to Kepserverex software for ingesting streaming sensor data. 

Phase-II involved re-platforming, without disrupting the OT state, by moving the Kepserverex and Wonderware apps as VMs on top of Nutanix Edge platform, encompassing Xi IoT and HCI (Hyper-converged infrastructure) . This consolidation yields immediate capex and opex benefits. The IT design of 1-click centralized Life Cycle Management can now be easily done for OT assets and IIoT software applications. 

Besides, Nutanix Edge HCI (Hyper-converged Infrastructure) consolidates server and storage services into a simple, software-defined stack and allows you to scale on your terms while providing a giant pool of compute and storage resources, similar to cloud consumption economics. The built-in AHV hypervisor comes at no extra cost, removing expensive virtualization lock-in concerns. Security is always top of mind when discussing modern platforms. Nutanix Flow provides application-centric microsegmentation to secure east-west traffic, which is very important for OT and IT security. Data can be persistently stored into Nutanix Files, Volumes or Object storage. Moreover, Nutanix Xi IoT Edge offers Kubernetes/CaaS apps (Container-as-a-Service) for new data science apps and Serverless functions for simple business logic. 

With successful completion of Step-1, the businesses now have unprecedented visibility of their operations, so here is the Step-2 to act upon those insights in an automated manner (predictive maintenance) and/or collect the data through ubiquitous sensor (camera) as well as apply machine vision capabilities.

Step2: Leverage AI for predictive maintenance, product quality inspection, supply chain automation etc. use cases

Let’s see some real-world deployment proof points. 

Customer Story #3: Predictive Maintenance

Problem Statement: At a sheet stamping plant with high-speed press machine and requirement for high quality pressed product. The use case is to avoid abnormal holes caused by a defective punch and malfunctioning process. 

Deployment and Benefits:

Solution is based on following paradigm - When a machine isn’t working as planned, it vibrates and sounds different than it does when under normal operation. The system compares the measured Acoustic Emission (AE) signals and vibration measurements with the baseline of stored normal profiles. The system takes action only if the changes in the signals indicate a defect or wear and tear.

Change in the patterns detect process faults and identify tool failures early, while helping to increase monitoring accuracy and optimize operator adjustments of the machine and tooling. 

The solution was deployed in partnership with Tokyo Electron Device (TED). The project had two phases. Phase-I: Training and deployment - TED’s CX-M training application leverages the existing historian with time-series AE signals and vibration data to build a ML (Machine Learning) model. Once the model is built, it is uploaded to the Xi IoT platform AI Inference Service for model inventory and versioning through the management plane. The Xi IoT management plane then pushes the model to factory plants worldwide, where Xi IoT Edge is already running, as part of Step-1 modernization.

Phase-II Inferencing: XI IoT Edge stack with the model is deployed on the factory floor. Xi IoT built-in drivers connect to the interfacing machine sensors. Streaming data flows into the Xi IoT NATS bus and the model is invoked. The resulting inference about the state of the machine is provided to the line worker and factory operator. Email notification and other alerts are configured as well through the CX-M software. 

Note that you can bring your own organically developed model as appropriate for your business operations. Or if you are an ISV, your application can easily be ported onto our platform, so that you can focus on the model accuracy and the latency characteristics, while leaving the Infrastructure and Application Lifecycle management at scale to us.

Customer Story #4: Product Quality Inspection

Problem Statement:

The customer belongs to CPG (Consumer Packaged Goods) vertical, specifically food packaging. The use case is to ensure the quality of packaging for meat products in an automated way. The customer’s previous method was to perform statistical sampling through manual inspection.

he implications of error in the manual approach were very high as even a single meat product becoming rotten, owing to bad packaging, would pose a health hazard risk for potential consumers and as well lead to costly recall of the entire batch.

Deployment and Benefits:

Quality check via image analytics and real-time inference 

The OT team set up an industrial-grade Basler camera, mounted a few feet from the conveyor belt. The camera scans the packaged object and sends high-resolution streaming images over the GigE Vision interface. The camera is connected to the Xi IoT Edge, which is running as VM on existing virtualized hardware at the production site. The native GigE vision connector on the Xi IoT Edge ingests the frames, performs a simple conversion, and places them onto the in-memory data bus.

The IT team leveraged the data pipeline construct to pick up those frames from the bus and apply a function, which captures the business logic written in Python. This function also invokes the custom ML model (customer took under two months to build a trained model), which was developed by the customer and imported into the Xi IoT Edge Model Library. The industry-standard Tensorflow framework and the run-time environment is built natively into the platform, making the model itself very easy to import. The ML model then uses the Xi “Infer” API to calculate the “quality threshold” based on the package image. An Nvidia T4 Inferencing GPU was leveraged for real-time decisions. The business logic captures the two possible workflows — Actuation and Re-training.

Quality control via robotic arm actuation

Workflow A (as in the image above): If the image is detected with high confidence and the quality threshold is deemed to be low, then the Function outputs to a MQTT topic which is subscribed to by Kepware Kepserverex software, which is running as VM on Nutanix Edge platform. Note that OT team leveraged a Xi IoT Edge generated certificate for secure MQTT communication between KEPServerEX and Xi IoT Edge. The Kepserverex converts the message into a PLC-specific format that actuates a robotic arm to pick up the faulty packaged product from the conveyor belt and put it into a separate bin!

Improving the quality check mechanism via re-training the model

Workflow B (as in the image above): If the confidence level itself in the detection of the image is low, then it means that either the package is of a new type or perhaps the model itself is not rich enough to capture the different facets of a previously known package. In either case, the model needs to be re-trained on these new data sets. The global data pipeline construct is used to send the image data from the local edge to the central (to aggregate over multiple edge sites) GCP Cloud Store instance.

Note that this was the customer’s own public data cloud. In this case, it was GCP, but could have been another public cloud like AWS, or even a private cloud. Google AI Platform was used to re-train the model on this new dataset and the new model was uploaded to the cloud repository. The customer’s CI/CD integration with the Xi IoT Platform made the automatic redeployment of the updated model onto the Edge very simple and seamless.

The decoupling of ML models from the actual business logic enables agility in model updates as well as pushing over low-bandwidth links. Model versioning and inventory management is provided by Xi Edge.  

Moreover, the processing of the model and compiling it for the right hardware is handled automatically by the system, thus decoupling the development (in the framework of choice by the developer) from deployment (on the HW of choice by the operator). This abstraction from HW leveraging “infer” API allows the AI apps to be developed once and deployed many times on various substrates.

In this case, the ML model was developed organically by the customer, however if you are an ISV, your application can easily be ported onto the Xi IoT Edge platform, so that you can focus on the model accuracy and latency, while leaving the Infrastructure and Application Lifecycle management at scale to us.

Customer Story #5: Smart Warehouse and Supply Chain Optimization

Problem Statement:

At a  French Multinational company, logistics is at the heart of keeping warehouses functioning. Tracking tens of thousands of pallets moving through these large warehouses can be quite challenging. However, improving existing processes without disrupting already running factories and warehouses is not always possible. That’s why it’s important to start with alerting and monitoring solutions that can coexist with existing environments. Until now, the warehouse and supply chain managers had to rely on manual processes to detect & track pallets and identify quality & compliance issues. 

Deployment and Benefits:

With Hardis Group’s Vision Insight powered by Xi IoT, the french multinational company gets easy traceability of assets through real-time image monitoring and recognition. Xi IoT Edge is deployed as VM on existing virtualized appliance (as the customer had other legacy VMs). Streaming video data from existing CCTV cameras is ingested via RTSP protocol into the Xi IoT Edge platform through data pipelines and passed to Vision Insights for AI inferencing at the edge. The AI model leverages the built-in TensorFlow 2.0 library within Xi IoT Edge platform. The model is unpacked from the rest of the application so that any new version of the model can be deployed at ease without disrupting the rest of the application. XI IoT AI Inferencing Service provides versioning and inventorying of the AI models. 

The front-end and back-end application is packed as kubernetes yaml and leverages Containers-as-a-service of the Xi IoT Edge Platform. Xi IoT Edge platform provides support for onboarding Kubernetes Apps at the right abstraction for developers, so that they can provision, run, manage and monitor k8s Apps as pods of containers without worrying about underlying k8s orchestration, patching, scaling etc. 

Developers can just input the Kubernetes formatted YAML, which will securely pull the images from any Container Registry. This workflow can be easily integrated with their custom CI/CD implementation.

Xi IoT Edge simplifies ongoing management and operations of deployed apps through features like real-time monitoring remotely, logging, and alerting. As an example, with a few clicks (or API calls), a user can monitor an app’s deployment status (including all containers within), across an entire global network of edges. If software or hardware diagnostics are needed at a particular service domain or edge where an app is misbehaving, a log bundle can be easily generated. If a user is actively testing and debugging, or just needs to take a quick peek at a transient error in production, real-time logs from within any application containers can also be easily viewed. And this is all available from Xi IoT infrastructure and lifecycle management without the need to login directly to the system where the apps are running. Messages stream securely over an encrypted channel and are viewable only by authenticated clients (such as an existing user logged on to the Xi IoT Platform).

Any insights are sent via built-in data connector to GCP Cloud Storage, where visualization dashboards are built. This enables the creation of a digital twin of the warehouse and real-time analysis of the situation, which can include comprehensive visibility into a specific logistics process flow, the number of pallets positioned, wet or damaged package detection, or operator errors, such as not utilizing proper safety equipment. Any anomalies in the image recognition are sent to the Hardis Group’s cloud for deep learning purposes, and the entire edge platform is managed by Xi IoT management.

The capability to seamlessly work with existing CCTV camera deployments provides for a quick path to ROI. Moreover, you can start small and scale fast with a single to multi-site deployment option. 

Ready-to-deploy cloud environment

Note that Nutanix Xi IoT Edge also offers a hosted cloud environment. This allows data scientists or developers to easily get started without having to wait for the Edge Hardware to be provisioned. There are some pre-built apps, some from our partners, in the Xi IoT Library. And because we know it's difficult to prototype edge data in the cloud, we’ve also enabled built-in YouTube 8M video data sources and even built Xi IoT Sensor app for iOS and Android based phones that turn cameras into instant video feeds. We’ve also made viewing test output simple by providing simple to leverage, native, HTTP Live Streams functionality. This makes prototyping and testing new ML models a breeze. Then when you’re ready, simply swap the sample input for a production RTSP or GigE-vision stream and deploy to your edge.

Edge Hardware options

We understand that hardware selection can be quite challenging to fit various requirements. Note that Xi IoT Edge software can run as bare metal on a specialized hardware or can run as VM on any virtualization substrate (private cloud or any public cloud). And we can assist in qualifying the right hardware for your needs.

Get started now...

In a nutshell, you read how Nutanix Xi IoT Edge platform can enable you to start the Industry 4.0 journey with a two-step IIoT blueprint, starting from OT modernization to Leveraging AI. While we discussed many use cases in this blog, a similar framework could be leveraged for multiple other use cases or different manufacturing settings, as deemed fit for your own Industry 4.0 journey. 

Our commitment to you is that we will hold your hand throughout this journey - from hardware selection, right design choices with our platform, bringing in the right partner (as appropriate), kickstarting a pilot to expanded production deployments and subsequent day-n operations and management! If you are a CxO or representing OT or IT teams, don’t hesitate to contact your local Nutanix sales team or reach out directly to the Xi IoT team by sending an email to iot [at] nutanix [dot] com. Or better start right away with the free trial at nutanix.com/iot. Either way, let’s start and accelerate on this journey together!

 

© 2020 Nutanix, Inc. All rights reserved. Nutanix, the Nutanix logo, and all Nutanix product and service names are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. All other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s). This post may contain links to external websites that are not part of Nutanix.com. Nutanix does not control these sites and disclaims all responsibility for the content or accuracy of any external site. Our decision to link to an external site should not be considered an endorsement of any content on such a site. Certain information contained in this post may relate to or be based on studies, publications, surveys and other data obtained from third-party sources and our own internal estimates and research. While we believe these third-party studies, publications, surveys and other data are reliable as of the date of this post, they have not independently verified, and we make no representation as to the adequacy, fairness, accuracy, or completeness of any information obtained from third-party sources.