Skip to content

Commit 6c1f1e4

Browse files
bnoguchibnoguchi
committed
Initial working commit with Number, Date, Clock, LengthAnnealingClock
1 parent 234f8c1 commit 6c1f1e4

File tree

14 files changed

+659
-0
lines changed

14 files changed

+659
-0
lines changed

‎Makefile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
test:
2+
./node_modules/.bin/mocha $(TESTS)
3+
4+
TESTS := $(shell find test/ -name '*.js')
5+
6+
.PHONY: all test clean

‎README.md

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
Lineage
2+
=======
3+
4+
A data versioning library.
5+
6+
## Introduction
7+
8+
Lineage is a JavaScript library for managing versions of data, especially
9+
helpful when reasoning about state in distributed systems. It provides several
10+
data types that can act as versions and also provide
11+
12+
Lineage:
13+
14+
- Defines a [protocol](https://github.com/codeparty/protocoljs) for
15+
incrementing, comparing, and merging versions.
16+
- Provides several data types that can act as versions.
17+
- Enables you to add your own data types that can act as versions.
18+
19+
## Installation
20+
21+
```
22+
$ npm install lineage
23+
```
24+
25+
## Basics
26+
27+
The most basic version data type you can use is a Number or Date.
28+
29+
## Numbers as versions
30+
31+
```javascript
32+
var lineage = require('lineage')
33+
, protocol = lineage.protocol;
34+
35+
// Require this to extend Number with the version protocol provided by lineage.
36+
require('lineage/lib/types/number');
37+
38+
var oldVersion = 1;
39+
var newVersion = protocol.incr(oldVersion); // => 2
40+
41+
var comparison = protocol.compare(oldVersion, newVersion);
42+
console.log(comparison === protocol.consts.LT); // true
43+
44+
comparison = protocol.compare(oldVersion, oldVersion);
45+
console.log(comparison === protocol.consts.EQ); // true
46+
47+
comparison = protocol.compare(newVersion, oldVersion);
48+
console.log(comparison === protocol.consts.GT); // true
49+
```
50+
51+
## Dates as versions
52+
53+
Dates act similarly to numbers except version incrementing a Date just means
54+
returning the Date right now.
55+
56+
```javascript
57+
var lineage = require('lineage')
58+
, protocol = lineage.protocol;
59+
60+
// Require this to extend Date with the version protocol provided by lineage
61+
require('lineage/lib/types/number');
62+
63+
var oldVersion = new Date();
64+
65+
setTimeout( function () {
66+
var newVersion = protocol.incr(oldVersion); // => <Date>
67+
var comparison = protocol.compare(oldVersion, newVersion);
68+
console.log(comparison ==== protocol.consts.LT); // true
69+
70+
comparison = protocol.compare(oldVersion, oldVersion);
71+
console.log(comparison, protocol.consts.EQ); // true
72+
73+
comparison = protocol.compare(newVersion, oldVersion);
74+
console.log(comparison === protocol.consts.GT); // true
75+
}, 1);
76+
```
77+
78+
## Vector Clocks
79+
80+
For distributed systems, the ideal data structure to use is a vector clock,
81+
also known as a [Lamport Clock](http://en.wikipedia.org/wiki/Lamport_timestamps).
82+
83+
Lineage provides vector clocks out of the box.
84+
85+
Why vector clocks are critical when you want to version data in a distributed
86+
system is more involved, and this README will point the reader instead to the
87+
[canonical paper on Lamport
88+
clocks](http://research.microsoft.com/en-us/um/people/lamport/pubs/time-clocks.pdf).
89+
90+
What a vector clock does behind the scenes is more straightforward. Vector
91+
clocks extend the concept of the Number as a counter that represents the
92+
version. Vector clocks are a set of counters, where each counter is associated
93+
with the actor/client/node that updated the vector clock. Every time a
94+
particular actor updates a vector clock, the counter it is associated with is
95+
incremented.
96+
97+
```JavaScript
98+
var lineage = require('lineage')
99+
, protocol = lineage.protocol
100+
, Clock = require('lineage/lib/types/clock');
101+
102+
var oldVersion = new Clock(); // => <Clock>
103+
104+
// Compared to Number as a version, Clocks as versions must associate every
105+
// update to a version with an actor id. Here that actor id is 'actor-A'
106+
var newVersion = protocol.incr(oldVersion, 'actor-A');
107+
108+
// Vectors updated by an actor compare to other vectors just how you would
109+
// expect them to.
110+
var comparison = protocol.compare(oldVersion, newVersion);
111+
console.log(comparison === protocol.consts.LT); // true
112+
113+
comparison = protocol.compare(oldVersion, oldVersion);
114+
console.log(comparison === protocol.consts.EQ); // true
115+
116+
comparison = protocol.compare(newVersion, oldVersion);
117+
console.log(comparison === protocol.consts.GT); // true
118+
```
119+
120+
Up until this point, it would appear as though Clock versions compare to each
121+
other in the same way that Number versions compare to each other. So why even
122+
have a different data structure to use for versions?
123+
124+
Things become more interesting when we involve another actor updating the
125+
version -- in this case, we have a distributed system.
126+
127+
TODO - finish the README
128+
129+
Suppose 1 person in the US and 1 person in China both receive an identical calendar.
130+
131+
### MIT License
132+
Copyright (c) 2012 by Brian Noguchi and Nate Smith
133+
134+
Permission is hereby granted, free of charge, to any person obtaining a copy
135+
of this software and associated documentation files (the "Software"), to deal
136+
in the Software without restriction, including without limitation the rights
137+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
138+
copies of the Software, and to permit persons to whom the Software is
139+
furnished to do so, subject to the following conditions:
140+
141+
The above copyright notice and this permission notice shall be included in
142+
all copies or substantial portions of the Software.
143+
144+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
145+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
146+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
147+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
148+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
149+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
150+
THE SOFTWARE.

‎lib/consts.js

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
exports.LT = -1;
2+
exports.EQ = 0;
3+
exports.GT = 1;
4+
exports.CONCURRENT = null;

‎lib/lineage.js

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
module.exports = {
2+
protocol: require('./protocol')
3+
, consts: require('./consts')
4+
};

‎lib/protocol.js

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
var protocol = require('protocoljs');
2+
3+
module.exports = protocol({
4+
incr: ('Increments the version'
5+
, [protocol, ('agent', Object)])
6+
, compare: ('Compares 2 versions of the same type'
7+
, [protocol, ('otherVersion', protocol)])
8+
, merge: ('Merges 2 versions to create a version that is causally ahead of both input versions'
9+
, [('agent', Object), protocol, ('otherVersion', Object)])
10+
});

‎lib/types/clock.js

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
// Vector Clock (aka Logical Clock, or Lamport Clock)
2+
3+
var VersionProtocol = require('../protocol')
4+
, consts = require('../consts')
5+
, LT = consts.LT
6+
, GT = consts.GT
7+
, EQ = consts.EQ
8+
, CONCURRENT = consts.CONCURRENT;
9+
10+
module.exports = Clock;
11+
12+
function Clock () {
13+
this.vector = {};
14+
}
15+
16+
Clock.prototype.protocolId = 'lineage:clock';
17+
18+
VersionProtocol(Clock, {
19+
incr: function (clock, agentId) {
20+
var vec = clock.vector;
21+
if (! (agentId in vec)) return vec[agentId] = 1;
22+
return ++vec[agentId];
23+
}
24+
, compare: function (clock, otherClock) {
25+
var allKeys = {}, vec;
26+
for (var i = arguments.length; i--; ) {
27+
vec = arguments[i].vector;
28+
for (var agentId in vec) allKeys[agentId] = true;
29+
}
30+
31+
var counter, otherCounter, mem;
32+
for (agentId in allKeys) {
33+
counter = clock.vector[agentId] || 0;
34+
otherCounter = otherClock.vector[agentId] || 0;
35+
36+
if (counter < otherCounter) {
37+
if (mem == GT) return CONCURRENT;
38+
mem = LT;
39+
} else if (counter > otherCounter) {
40+
if (mem == LT) return CONCURRENT;
41+
mem = GT;
42+
} else if (counter == otherCounter) {
43+
if (mem != LT && mem != GT) mem = EQ;
44+
}
45+
}
46+
return mem;
47+
}
48+
, merge: function (agentId, clock, otherClock) {
49+
var newClock = new Clock();
50+
51+
var vec = newClock.vector = greedyZip(clock.vector, otherClock.vector, function (a, b) {
52+
return Math.max(a || 0, b || 0);
53+
});
54+
if (vec[agentId]) {
55+
++vec[agentId];
56+
} else {
57+
vec[agentId] = 1;
58+
}
59+
return newClock;
60+
}
61+
});
62+
63+
// Given 2 Objects a and b, iterate through their keys. For each key, take the
64+
// corresponding value a[k] and the corresponding value b[k], and apply the
65+
// function fn -- i.e., fn(a[k], b[k]). Assign the result to the same key k on
66+
// a new Object we eventually return.
67+
function greedyZip (a, b, fn) {
68+
var out = {}
69+
, seen = {};
70+
for (var k in a) {
71+
seen[k] = true;
72+
out[k] = fn(a[k], b[k]);
73+
}
74+
for (k in b) {
75+
if (k in seen) continue;
76+
out[k] = fn(a[k], b[k]);
77+
}
78+
return out;
79+
}

‎lib/types/date.js

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
var VersionProtocol = require('../protocol')
2+
, consts = require('../consts')
3+
, LT = consts.LT
4+
, GT = consts.GT
5+
, EQ = consts.EQ;
6+
7+
module.exports = Date;
8+
9+
VersionProtocol(Date, {
10+
incr: function (date, agent) {
11+
var newDate = new Date();
12+
if (+newDate === +date) {
13+
return new Date(+date + 1);
14+
}
15+
return newDate;
16+
}
17+
, compare: function (dateA, dateB) {
18+
if (dateA < dateB) return LT;
19+
if (dateA > dateB) return GT;
20+
return EQ;
21+
}
22+
, merge: function (agent, dateA, dateB) {
23+
var now = new Date;
24+
if (now > dateA && now > dateB) return now;
25+
return new Date(Math.max(dateA, dateB));
26+
}
27+
});

‎lib/types/length_annealing_clock.js

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
var VersionProtocol = require('../protocol')
2+
, consts = require('../consts')
3+
, LT = consts.LT
4+
, GT = consts.GT
5+
, EQ = consts.EQ
6+
, CONCURRENT = consts.CONCURRENT
7+
8+
, Clock = require('./clock');
9+
10+
module.exports = LengthAnnealingClock;
11+
12+
function LengthAnnealingClock (max) {
13+
this.vectorClock = new Clock();
14+
15+
this.max = max;
16+
17+
// [agentId, timestamp] pairs sorted from earliest to most recent
18+
this.lruAgents = [];
19+
}
20+
21+
function indexOf (array, fn) {
22+
for (var i = 0, l = array.length; i < l; i++) {
23+
if (fn(array[i])) return i;
24+
}
25+
return -1;
26+
};
27+
28+
LengthAnnealingClock.prototype.protocolId = 'lineage:length_annealing_clock';
29+
30+
VersionProtocol(LengthAnnealingClock, {
31+
incr: function (clock, agentId) {
32+
var max = clock.max
33+
, vec = clock.vectorClock.vector
34+
, lruAgents = clock.lruAgents
35+
, index = indexOf(lruAgents, function (pair) {
36+
return pair[0] === agentId;
37+
})
38+
, counter;
39+
40+
if (~index) {
41+
lruAgents.splice(index, 1);
42+
lruAgents.push([agentId, +new Date()]);
43+
} else {
44+
vec[agentId] = 0;
45+
}
46+
lruAgents.push([agentId, +new Date()]);
47+
if (lruAgents.length >= max) {
48+
var agentIdToRm = lruAgents.shift()[0];
49+
delete vec[agentIdToRm];
50+
}
51+
return ++vec[agentId];
52+
}
53+
, compare: function (clock, otherClock) {
54+
return VersionProtocol.compare(clock.vectorClock, otherClock.vectorClock);
55+
}
56+
, merge: function (agentId, clock, otherClock) {
57+
var lruAgents = clock.lruAgents
58+
59+
, otherLruAgents = otherClock.lruAgents
60+
61+
, max = Math.max(clock.max, otherClock.max)
62+
63+
, mergedClock = new LengthAnnealingClock(max)
64+
, mergedLruAgents = mergedClock.lruAgents
65+
;
66+
67+
mergedClock.vectorClock = VersionProtocol.merge(agentId, clock.vectorClock, otherClock.vectorClock);
68+
69+
// Merge the lru agents
70+
var uniqueAgents = [[agentId, +new Date()]];
71+
var agentLists = [lruAgents, otherLruAgents];
72+
for (var k = agentLists.length; k--; ) {
73+
var agentList = agentLists[k]
74+
, pair, index, currAgentId, timestamp;
75+
for (var i = 0, l = agentList.length; i < l; i++) {
76+
pair = agentList[i];
77+
currAgentId = pair[0];
78+
if (currAgentId === agentId) {
79+
continue;
80+
}
81+
index = indexOf(uniqueAgents, function (pair) {
82+
return pair[0] === currAgentId;
83+
});
84+
if (~index) {
85+
// Update the timestamp to the max timestamp per agentId
86+
timestamp = pair[1];
87+
uniqueAgents[index][1] = Math.max(timestamp, uniqueAgents[index][1]);
88+
} else {
89+
uniqueAgents.push(pair);
90+
}
91+
}
92+
}
93+
94+
// Sort the agents in chronological order and assign to the clock
95+
mergedClock.lruAgents = uniqueAgents.sort( function (pairA, pairB) {
96+
return pairA[1] - pairB[1];
97+
});
98+
return mergedClock;
99+
}
100+
});

0 commit comments

Comments
 (0)