When you build an IoT device, you need to verify than the main goal can be reached: ours is analytics capabilities. Without timeseries analytics, no valuable service can be build!
So far we've been doing all of that, with a simple PI, Warp10 and a first stage analysis with Warscript.
But in our IoT device, we have to manage the BLE connection, expose a REST API, receive messages from Beddit sensor, decode and send them into Warp10’s Ingress endpoint, serve a progressive webapp, authenticate the user, etc...
Even with a quad core processor Raspberry PIs, efficiency is key.
A classical engineering design would use a multi-threaded approach coordinated with locks. It is proven to to work, but such approach has scalability problems, been there, done that...
This time we have opted for a Reactive approach based on Eclipse Vert.x. Vert.x is a toolkit for building applications on top of the JVM, another cool Open Source project of the Eclipse foundation.
Vert.x tool-kit implements the reactor pattern as its core and is well adapted for the C10K like problems. You could think, cool but where is the relationship between the C10K problem and IoT?
We will not handle 10K clients at the same time on the Rapsberry PI, but solving the C10K implies to be resource efficient which is one of the properties IoT devices must have.
Verticles are chunks of code that get deployed and run by Vert.x. Our project is devided into 4 Verticles each having a different role:
- The MainVerticle, the first launched manages the deployment/undeployment of the other Verticles
- The Bluetooth verticle manage the BLE aspects such scanning, pairing and device communications
- The StreamProcessing verticle talks with the Beddit sensor protocol and transcode dataframes into Warp10 input format, at the end records every second received datapoints into Warp10
- The WebServer verticle, exposes a web service API and resources
But, all this “happy crowd” needs to communicate in a “smart way”, and for IoT devices, a smart communication is an asynchronous communication.
Vert.x integrates a light-weight messaging system named event-bus which allows different parts of the application to communicate in a loosely coupled way. We uses the event-bus as an “asynchronous” timing belt, allowing the different verticles to exchange messages.
For example, the Bluetooth verticle during the night receives datagrams (BLE notifications ) from the sensor. When it happens, the verticles must be ready to receive another message with a small latency. In this situation each received message is published via the event bus to a decoder that will decode it without blocking data acquisition, as the data acquisition thread has been freed from the decoding task.
This mechanism allow to spread the work load on the different cores of the PI CPU in a very efficient manner. If one software component needs more resources during a short time, its impact on other components is very limited.
Yes… and no! Manage BLE device is effortless on Raspbian since all can be done with command line toots likes hcitool and gatttool. Many resources in the Maker community talks about Bluetooth, for example this one from Adafruit is very comprehensive. But how can we spawn, kill and read and write data streams from Vert.x without blocking anything?
Once again, the Vert.x community has a solution for that: this github repository caught our attention, it is just a complete extension for manage child processes form Vert.x written by Julien Viet, Vert.x core developer and team leader. There was no possible alternative, we were charmed once again!
After a couple few bugs corrected (located between my chair and my keyboard), all this “watchmaking” receives, decodes and records with a very small memory and cpu footprint around 150 datapoints per second.
Definitively, Vert.x is the best IoT toolkit!
Furthermore, if you prefer to read the source, just go on our Github repository!