Augmented factory offers fresh user experience for IoT enabled factories via location based UI and camera vision.
In the IoT manufacturing scenario, the factory data are exploding as more and more information is collected and stored daily. It’s difficult to identify the alarmed device among many identical ones, so we came up some ideas with augmented reality function to give maintenance engineers an easy and intuitive way to inspect factory, to collect essential data effectively and locate the problem efficiently.
We call it “Augmented factory”. It is an information center of connected factory, an application on handheld device with indoor location (CMX) and augmented reality function. There’re two views in this application, map view and augmented reality view. We also provide user an easy and quick way to shift from map view to the augmented reality view by the handheld device.
When user holds the handheld device horizontally, it will show the map view. Map view can help user to know the indoor location and get related information efficiently.
In map view, if user keeps moving, it will only display the overall status and moving arrows. If user stops at a device, system will automatically display its UI widow to give more detailed information. User shall be able to do some interactions on this UI window.
When user lifts up the handheld device vertically, it will show the augmented reality view. Augmented feature can help maintenance engineers to identify the alarmed device among many identical ones quickly.
In the AR view, it shall by default display the brief status of visible devices and components. As user moves the camera view, the brief status shall change accordingly. If user touches one of the brief statuses, system will display the full UI for the device/components to do more interactions, such as showing trouble shooting guide and the progress of the trouble shooting in augmented reality view.
We have also deployed a CMX server and an arm server for our project. The CMX server is used to return map data, location data to mobile clients. So clients can show the floor map and user location on its screen. The arm server is connected to the control panels of robotic arms via USB cables. It is used to receive control requests sent from mobile clients and write data to the control panels to control the servos of robotic arms.
We placed 3 iBeacon devices close to our robotic arms, each beacon is close to one arm. When the user comes close to an arm, the RSSI (Received Signal Strength Indication) will become strong. So we can know which arm is the closest one and show user the interface of the closest arm on the map.
Augmented factory can be used in two ways: In factory scenario, it can be used to help equipment inspectors locate the equipment in error and fix the issue. Users just have to upload their factory map to CMX and change the server configuration. Modules of this project can also be used separately by developers to build their own project. E.g. They may use CMX module or iBeacon module to build a new application.
