Automation, data analytics, cloud, artificial intelligence, machine learning – these transformative technologies offer manufacturers a host of potential operational and commercial benefits. 

However, across the manufacturing industry, most organisations are still in the early stages of digital transformation, and are not always in a position to handle and exploit the above technologies, or rapidly process the data required to benefit from them. 

As Thomas Degen, Solutions Engineer Industries at KX, notes, “We have seen lots of modernization from a digitization perspective happening on the shop floors. But there is still the gap around what Industry 4.0 outlines as connecting the physical productization with enterprise IT across the whole value chain up to the end customer, including partners. This is still a vision which hasn’t happened.”

There are a few reasons for this: first off, infrastructure is not yet in place to modernize the industry in this way. 

Degen explains, “A big digital transformation project at a steel factory saw the furnace blast controller migrated to a new platform, which made it the most vulnerable point in the whole factory. The server got caged after one incident where it was brought down after a technician added a USB stick to the device.”

The reality is that industrial Digital Transformation projects bridging OT & Enterprise IT take a long time and many large organizations have only just started on this journey. In the last year, I’ve been working with big system integrators who are helping large enterprises, from food and beverage to the automotive sector transform their systems and processes. If the big players are only just starting to roll out Manufacturing Execution Systems to connect their shop floors on a central system, you can imagine what it means for other manufacturers.”

Manufacturers have a clear understanding that faster access to better quality data is critical to improving operations and lowering costs. However, they face a key hurdle in getting the technology and processes aligned to enable the operational shift to continuous intelligence

One of the biggest challenges – and opportunities – for manufacturers is how they manage all of the data they are now producing: how to process, analyse and act on insights in real-time. The key to this is combining the vast amounts of historical, time-series data with the real-time insights.

With sensors dotted around manufacturing operations, collecting information on light, vision and sound, motion and vibration, humidity and moisture, location, gravity and temperature, there will be meta data particular to that organization. Degen says:

“This is the first thing you need to get through an integration hook, which could be an IoT gateway, an API, an SDK access to this kind of system. You need to pull this in a normalization layer to get at this data in a way that makes sense and can be used upstream for further analytics and processing. It needs to be in a data format that is known to the standard enterprise IT system upstream, the back-end systems, the ERP systems, the SAPs.

“From the integration aspect, IoT gateway providers play a role. For heterogeneous shop floors, you still need to provide IoT gateways that speak cross-manufacturer and can normalize the data for further upstream usage. This is the biggest problem with the sensor data that is getting captured, and the upstream process that eventually analyses it to be acted upon.”

Rolling out an integrated data platform is certainly worth the effort, offering a much richer, deeper and truer understanding of the manufacturing process. Artificial intelligence and machine learning-based data modelling lets firms predict the future before it happens, automate key decisions, and enable an operating model of Continuous Operational Intelligence.

However, it is not enough to just have access to the data; manufacturers need to have this data available at speed, for faster product production, faster time to market and faster response to customers in the field. 

Manufacturers want to store their data in the most efficient way, and get access to it at speed to analyse and react to events in real-time – but this is not easy where there is such a mix of data. 

“This makes it hard for manufacturers to decide which way to go. In the IT world, you have to accommodate for all these kind of techniques – real time, historical analysis, intraday capture and review – different data marts, different technologies to do these real-time analytics versus historical analytics. This usually implies different technologies that are not easy to handle across the whole value chain from the manufacturer perspective,” Degen explains.

“This is one of the issues that manufacturers looking to implement data-driven, real-time decision making face. It’s a problem addressed by KX. The KX streaming analytics problem combines both real-time, streaming and historical data to allow for complex event processing on the shop floor, even if different data marts must be stored. Rich visualisations on dashboards allows for insight across various data points, facilitating data-driven decision making.”

Artificial intelligence and machine learning technologies are ideally placed to offer manufacturers value from their historical data, often collected over decades. This data can be revalidated and reassessed to identify patterns in operations via machine-learning models and specific algorithms.

Degen explains, “These are deployed into operational technology monitoring services to identify if the production line, the shop floors are producing well, or are malfunctioning and causing anomalies, where we are supposed to send a technician to the site, and eventually open up and create a repair ticket for resolution. This is the kind of need that could help manufacturers get advanced use of all the data that has been captured by them already in the past.”

As the speed of change and digitisation continues to accelerate, manufacturers must act now to ensure they remain competitive in globalized markets. There is no better or more efficient resource usage, including human beings, than to automate with additional digital services around the chain of productivity and production lines. 

Digital transformation in the manufacturing industry should always start from a point where the business identifies certain gaps – needing to roll out goods with change requests in a fast, more efficient way; or being able to react to requests from new markets to adapt their offerings. This can only be achieved with higher automation and better digitization and real-time analytics is a critical part of this strategy.


About KX:


KX, a global leader in real time streaming analytics, is part of First Derivatives plc, a global technology provider with 20 years of experience working with some of the world’s largest finance, technology, automotive, manufacturing and energy institutions. KX Streaming Analytics, built on the kdb+ time-series database, is an industry leading high-performance, in-memory computing, streaming analytics and operational intelligence platform. It delivers the best possible performance and flexibility for high-volume, data-intensive analytics and applications across multiple industries. The Group operates from 14 offices across Europe, North America and Asia Pacific and employs more than 2,500 people worldwide.

Find out more at, follow on Twitter, connect on LinkedIn and watch on YouTube.