Transmuting Data, pt 2

See also transmuting data, part 1

This example shows how we dynamically calculate and render the effectiveness of a given fuelCell by comparing the gas consumption versus the electricity production.

The problem is both gas consumption and electricity production are stored as incremental values.
The anticipated formula will calculate the yield for a given time range where :
yield = normalisationConstant * energyProductionDelta / (gasConsumptionDelta * constant)


Luckily, we can easily transform this data by applying a rule, written in our in-house developed rule format:

We assign this rule to a virtual item trace, and request some data using a daily interval delta range.

The resulting graph is this:

It is based upon this original data, incremental values for which it is hard to see, by human eye, the ups and downs in the yield:

Transmuting Data

A big part of what we do at Bynubian is to apply some wizardry on complex data streams we capture.

An example is to visualize the constant stream of Events coming into and leaving our MQTT server.

That data is very frequent and massive in size, and is being captured as raw, timestamped incremental counts of sent and received events.

The above psychedelic figure is a rendering achieved by transmuting these data streams as follows:

1. The events, which are captured multiple times per minute, are reduced to 15 minute averages.
2. We transform the incremental 15 minute averages into delta values by creating a virtual data sequence that calculates, using our own rule engine for data streams, the difference between the current value and the previously stored value.
3. On the front end we receive the aggregated data, and once again transform the second series (received events) by inverting it (multiply by -1). Again this uses our internal rule-engine, but now as a javascript library.
4. We also calculate another virtual data sequence that is the subtraction of the delta of sent items and the delta of received items, represented by the pink line.
5. We plot the data on a timeseries rendered by echarts. The trace above the X axis respresents the number of events sent at that given time. The trace below the X axis represents the number of events received at that given time.
6. The vertical green line is a very short lived interruption of our MQTT server due to Intel Spectre fixes.