Radix IoT in Action: Scaling Systems with Mango

More and more we are seeing the need for larger IIoT (Industrial Internet of Things) applications where you might have 100 or 1000 smaller systems all feeding data into one large cloud-based system. One of the reasons Mango is such a great choice for these is that it really can scale.

Real life example | Distributed data acquisition system

Here is one real life example where we have a distributed data acquisition system with 35 Mango servers across the globe.  Each server has between 10,000 and 23,000 data points and data updates on average every 2 seconds with some updates as fast as 100ms. Data points are mostly Modbus IP.

Each server uses the Mango Persistent TCP publisher to push its real time data and sync its historical data to one large server where all the data is available for users. Every sample of data is stored which makes for a very large database which increases about 250GB each day.

In total there are about 400,000 data points on the large server and it’s consistently writing around 300,000 values per second with burst write speeds over 3 million per second to the database.

Server RAM is heavily used for buffering the data streams.  When this much data is coming in there needs to be a sophisticated system to buffer the data and then write it into the database in batches. If the disk is busy you need to be able to hold all the data in the queue and then write as fast as possible when the disk is free.

The remote Mango systems also have a robust system of buffering data so in the event of an internet connection outage or if the central server is not available all data will be buffered on the remote system. This means in addition to being able to handle the constant volume of data coming in from regular updates every 2 seconds the system also needs to be able to keep up after a server reboot or a remote system has built up a large buffer.

In the event of a reboot on the central server all remote systems will buffer their data.  As soon as the system is back up they will connect their publishers and start sending real time values immediately and will simultaneously start dumping their buffers to the central server. This can result in a stream of data 4-6 times the normal volume of data and can last for several hours as all the buffered data is synced.

In this example Mango as a large central IIoT application has been more than capable of receiving the steady stream of data and also handling the hammering of system reboots and network loss recoveries.

At the end of our trials and through ongoing improvements and tweaks in the Mango NoSQL database we were pleased to find our final benchmarks greatly exceeded all other tests. In addition to exceptional performance, the cost savings were much larger than the other databases, so in the end using the Mango NoSQL database was an obvious choice.

Conclusion

Although Mango is perfect for small projects, it really shines as a large scale IIoT platform and users will be impressed with the performance and the extremely low cost to operate and maintain the system. In this real-life operating system there are 400,000 data points but we expect the system to scale much larger without problems. With Mango and the REST API it will be possible to have a cluster of Mango servers all bridged by a single UI so the system can scale out to millions of data points.