msg2 Frontend Notes

Out of date since we switched to flotr2 some time ago. TODO: Update to current solution

Dygraph

Dygraph uses the following format:

Time (Date object)	Values Series 1	Values Series 2	...	Value Series N
1.9.1983 23:00:00	0.000042	null	...	3.18
1.9.1983 23:00:11	0.000042	0.0023	...	null
1.9.1983 23:01:00	0.000042	NaN	...	null
1.9.1983 23:01:00	null	2.3	...	null
1.9.1983 23:01:05	null	null	...	8.4

• The table is represented as a list of lists.
• Each entry consists of a Date object and a float for every time series
• Time differences between entries do not need to follow a fixed period
• Entries should be sorted by their timestamps
• There may be multiple entries per time stamp iff. there are is exactly one value for each time series in the union of these entries
• If time series has no value at this point a time its float is set to null or NaN
  • A null sequence in the series column will be drawn as a linear interpolation between the regular values at its start and end
  • A NaN in sequence of null values or regular values indicates that there should be no interpolation between the two regular values surrounding the NaN (see http://dygraphs.com/options.html#connectSeparatedPoints)

Examples:

Time (Date object)	Values Series 1
1.9.1983 23:00:00	1
1.9.1983 23:01:00	null
1.9.1983 23:02:00	null
1.9.1983 23:03:00	4
1.9.1983 23:04:00	4
1.9.1983 23:05:00	4

4 -|      ,-o-o-o
   |     / 
   |   / 
1 -|o-´
   +--------------
    |       |
  23:00   23:03

Time (Date object)	Values Series 1
1.9.1983 23:00:00	1
1.9.1983 23:01:00	null
1.9.1983 23:02:00	NaN
1.9.1983 23:03:00	4
1.9.1983 23:04:00	4
1.9.1983 23:05:00	4

4 -|       o-o-o
   | 
   | 
1 -|o
   +------------
    |       |
  23:00   23:03

• To guarantee that a fixed time period is displayed regardless of the first and last value available a header and footer entry can be inserted containing NaN for every time series

Example:

Time (Date object)	Values Series 1	Values Series 2	...	Value Series N
1.9.1983 23:59:59	NaN	NaN	...	NaN
1.9.1983 23:00:00	0.000042	null	...	3.18
1.9.1983 23:00:11	0.000042	0.0023	...	null
1.9.1983 23:01:00	0.000042	NaN	...	null
1.9.1983 23:01:00	null	2.3	...	null
1.9.1983 23:01:05	null	null	...	8.4
1.9.1983 23:01:06	NaN	NaN	...	NaN

SensorValueStore

A time series value store implemented in typescript provide the data in a dygraph compatible format.

The value store only stores values for specific time period from now() - storageperiod to now(). Values inserted for any point in the further future or will be ignored.

If a time series a only null entry for a period longer the timeout a NaN will be inserted at the end of this period. The NaN entry is removed automatically if new values are provide for the timeout period.

A header and a footer containing only NaNs is inserted automatically to force dyraph to display the full storage period, independent of the availability of values.

Init

The initially the store will look like this:

Time (Date object)
now() - period
now()

Adding a sensor

If a sensor is added a new column containing a null value is inserted in every entry. The header an footer are updated with an additional NaN entry.

Time (Date object)	Old Sensor	New Sensor
now() - period	NaN	NaN
...	3	null
...	5	null
...	8	null
now()	NaN	NaN

Removing a sensor

For removing a sensor the corresponding column is removed from every entry.

Time (Date object)	Removed Sensor	New Sensor
now() - period	NaN	NaN
...	3	13
...	5	21
...	8	34
now()	NaN	NaN

Adding values

New values can be added to the store as a list of time stamp, sensor and value three-tuples. First the time stamp in the footer entry is updated to now() to ensure all values can be inserted between header and footer entry. In a second step the list gets sorted by the timestamps and all tuples with timestamps to far in the future or past are dropped. The sorted list can be merged with the existing sorted internal data storage in a single linear iteration over the existing data. If there is no existing entry for a time stamp a new entry has to added, otherwise the null in the existing entry can be exchanged.

The time stamp of the oldest value inserted is stored for the next update operation.

Finally an update operation should be executed on the store to drop any value with timestamps to far in the past and to update the timeout entries where necessary.

Example:

Current data:

Time (Date object)	SensorX	SensorY
now() - period	NaN	NaN
TimeA	3	null
TimeB	5	null
TimeD	8	null
now()	NaN	NaN

List with new Values:

Time (Date object)	Sensor	Value
TimeA	SensorY	4
TimeB	SensorY	8
TimeC	SensorX	16

Resulting data:

Time (Date object)	SensorX	SensorY
now() - period	NaN	NaN
TimeA	3	4
TimeB	5	8
TimeC	16	null
TimeD	8	null
now()	NaN	NaN

Updating

Since the store should only contain values ranging from now() - period to now() a regular update operation is required to remove old values and to adjust the timestamps in the header and footer entries. This done by adjusting the header and footer time stamp first and then removing every element with a time stamp older then now() - period.

This operation also takes care of removing and inserting the NaN entries for series with a timeout. A series is considered timed out if there the difference between the last two non-null values for it is bigger than timeout. In this case a NaN entry has to be inserted directly after the last non-null entry for the series.

It may be necessary to remove a timeout if new values are added to the timed out sensor. Both can be achieved by a linear iteration over the stored entries, starting one timeout in the past from the oldest entry added since the last update operation. Timeout entries further in the past can not be affected by the new values. The time stamp of this entry should be stored by the add operation. It may be null if no values have been added since the last update.

During iteration the index of the last non-null value for each series is kept in a an array. If the a other non-null has been found for the series its time stamp can compared to the time stamp of the entry indicated by the array. In case of a timeout a NaN entry can be place directly after the index stored in the array. (The other indexes stored in the array may need to be incremented due to the new element.) Regardless of the timeout, the array can be updated with the current index. If an old entry timeout entry is encountered in the iteration it's validity can be checked by looking ahead to find the next non-null value for its series and comparing the timestamps.

Examples:

Only removing old values - before :

Time (Date object)	Old Sensor	...
now() - period - delta	NaN	...
...	3	...
...	5	...
...	8	...
now() - delta	NaN	...

Only removing old values - after :

Time (Date object)	Old Sensor	...
now() - period	NaN	...
...	5	...
...	8	...
now()	NaN	...