Skip to content

Commit a3d2683

Browse files
joaomlnetojoaomlneto
committed
Merge branch 'main' into tendermint-main-fix-byzzfuzz-partitions
2 parents e3315fa + a4fdb90 commit a3d2683

File tree

148 files changed

+15402
-799
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

148 files changed

+15402
-799
lines changed

‎.idea/compiler.xml

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.idea/kotlinc.xml

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.idea/misc.xml

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.vscode/settings.json

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"java.configuration.updateBuildConfiguration": "interactive"
3+
}

‎Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM node:20.18.1-alpine AS nodejs-base
1+
FROM node:20.18.3-alpine AS nodejs-base
22

33
# Generate the OpenAPI docs and build the Java application
44
FROM gradle:8 AS java-builder

‎README.md

+17-62
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,22 @@
1-
# ByzzBench
1+
# ByzzBench - a BFT Protocol Benchmarking Suite
22

3-
BFT Protocol Benchmarking Suite
3+
[![Build Docker Image](https://github.com/joaomlneto/byzzbench/actions/workflows/docker-build.yml/badge.svg)](https://github.com/joaomlneto/byzzbench/actions/workflows/docker-build.yml)
4+
5+
> ByzzBench is a benchmark suite designed to evaluate the performance of testing algorithms in detecting bugs in
6+
> Byzantine
7+
> Fault Tolerance (BFT) protocols. It is designed to be modular and extensible, allowing for easy integration of new
8+
> protocols and scenarios.
9+
>
10+
> ByzzBench is designed for a standardized implementation of BFT protocols and their execution in a controlled testing
11+
> environment, controlling the nondeterminism in concurrency, network and process faults in the protocol execution,
12+
> enabling the functionality to enforce particular execution scenarios and thereby facilitating the implementation of
13+
> testing algorithm for BFT protocols.
414
515
This is a Gradle monorepo that contains the following modules:
616

7-
- `simulator`: The core benchmarking suite. Currently also includes the protocol implementations.
8-
- `webui`: A web interface for the benchmarking suite.
17+
- `simulator`: The core benchmarking suite, written in Java/Spring Boot. It currently also includes the protocol
18+
implementations.
19+
- `webui`: A web interface for the benchmarking suite, written in React/NextJS.
920

1021
## Prerequisites
1122

@@ -32,8 +43,8 @@ Installing JDK:
3243

3344
1. Through Eclipse Adoptium:
3445

35-
- Download the version you need (JDK-21)
36-
- When installing, select "Set or override JAVA_HOME variable"
46+
- Download the version you need (JDK-21)
47+
- When installing, select "Set or override JAVA_HOME variable"
3748

3849
2. Through Windows Package Manager - "winget":
3950

@@ -157,59 +168,3 @@ See additional documentation in the [docs](docs) directory.
157168
- [Implementing new BFT Protocols](docs/implementing-protocols.md)
158169
- [Reproducing Schedules](docs/reproducing-schedules.md)
159170
- [User Interface](docs/user-interface.md)
160-
161-
## Simulator Structure
162-
163-
```mermaid
164-
---
165-
title: Simulator Components
166-
---
167-
classDiagram
168-
class Event {
169-
-int eventId
170-
}
171-
class Transport {
172-
}
173-
class MessageEvent {
174-
-String senderId
175-
-String recipientId
176-
-MessagePayload message
177-
-MessageStatus status
178-
}
179-
class TimeoutEvent {
180-
}
181-
class MessageStatus {
182-
<<Enumeration>>
183-
QUEUED
184-
DELIVERED
185-
DROPPED
186-
}
187-
class MessagePayload {
188-
<<Interface>>
189-
+String getType()
190-
}
191-
class Replica {
192-
<<Abstract>>
193-
+String getType()
194-
}
195-
class CommitLog {
196-
<<Abstract>>
197-
addEntry()
198-
}
199-
class TotalOrderCommitLog {
200-
addEntry()
201-
}
202-
class PartialOrderCommitLog {
203-
addEntry()
204-
}
205-
Event <|-- MessageEvent
206-
Event <|-- TimeoutEvent
207-
MessageEvent -- MessageStatus
208-
MessageEvent -- MessagePayload
209-
CommitLog -- Replica
210-
Transport o-- Event
211-
Transport o-- Replica
212-
Replica --> Event: emits, receives
213-
CommitLog <|-- TotalOrderCommitLog
214-
CommitLog <|-- PartialOrderCommitLog
215-
```

‎buildSrc/src/main/kotlin/byzzbench.java-common-conventions.gradle.kts

+1-1
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ dependencies {
1919
}
2020

2121
// Use JUnit Jupiter for testing.
22-
testImplementation("org.junit.jupiter:junit-jupiter:5.11.4")
22+
testImplementation("org.junit.jupiter:junit-jupiter:5.12.0")
2323

2424
testRuntimeOnly("org.junit.platform:junit-platform-launcher")
2525
}

‎docs/implementing-protocols.md

+57
Original file line numberDiff line numberDiff line change
@@ -58,3 +58,60 @@ check whether safety invariants of distributed consensus are broken.
5858
order, and this can lead to non-deterministic behavior. Use our own `DeterministicCompletableFuture`, which will
5959
execute the tasks in the order they were submitted to it.
6060

61+
## Components
62+
63+
The diagram below includes the relevant ByzzBench components for implementing a new BFT protocol.
64+
65+
```mermaid
66+
---
67+
title: Simulator Components
68+
---
69+
classDiagram
70+
class Event {
71+
-int eventId
72+
}
73+
class Transport {
74+
}
75+
class MessageEvent {
76+
-String senderId
77+
-String recipientId
78+
-MessagePayload message
79+
-MessageStatus status
80+
}
81+
class TimeoutEvent {
82+
}
83+
class MessageStatus {
84+
<<Enumeration>>
85+
QUEUED
86+
DELIVERED
87+
DROPPED
88+
}
89+
class MessagePayload {
90+
<<Interface>>
91+
+String getType()
92+
}
93+
class Replica {
94+
<<Abstract>>
95+
+String getType()
96+
}
97+
class CommitLog {
98+
<<Abstract>>
99+
addEntry()
100+
}
101+
class TotalOrderCommitLog {
102+
addEntry()
103+
}
104+
class PartialOrderCommitLog {
105+
addEntry()
106+
}
107+
Event <|-- MessageEvent
108+
Event <|-- TimeoutEvent
109+
MessageEvent -- MessageStatus
110+
MessageEvent -- MessagePayload
111+
CommitLog -- Replica
112+
Transport o-- Event
113+
Transport o-- Replica
114+
Replica --> Event: emits, receives
115+
CommitLog <|-- TotalOrderCommitLog
116+
CommitLog <|-- PartialOrderCommitLog
117+
```

‎gradle/wrapper/gradle-wrapper.jar

122 Bytes
Binary file not shown.

‎gradle/wrapper/gradle-wrapper.properties

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
distributionBase=GRADLE_USER_HOME
22
distributionPath=wrapper/dists
3-
distributionUrl=https\://services.gradle.org/distributions/gradle-8.12-bin.zip
3+
distributionUrl=https\://services.gradle.org/distributions/gradle-8.13-bin.zip
44
networkTimeout=10000
55
validateDistributionUrl=true
66
zipStoreBase=GRADLE_USER_HOME

‎gradlew

+1-1
Original file line numberDiff line numberDiff line change
@@ -205,7 +205,7 @@ fi
205205
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
206206

207207
# Collect all arguments for the java command:
208-
# * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
208+
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
209209
# and any embedded shellness will be escaped.
210210
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
211211
# treated as '${Hostname}' itself on the command line.

‎schedules/hbft known violation.json

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"events":[{"type":"ClientRequest","eventId":2,"createdAt":1736960554.6270425,"status":"DELIVERED","recipientId":"B","senderId":"C0","timestamp":1736960554627,"payload":{"timestamp":1736960554627,"operation":"C0/1","signed":false,"signedBy":null,"type":"DefaultClientRequest"},"deliveredAt":null},{"type":"ClientRequest","eventId":9,"createdAt":1736960554.6280425,"status":"DELIVERED","recipientId":"B","senderId":"C1","timestamp":1736960554628,"payload":{"timestamp":1736960554628,"operation":"C1/1","signed":false,"signedBy":null,"type":"DefaultClientRequest"},"deliveredAt":null},{"type":"Message","eventId":16,"createdAt":1736960556.0801437,"status":"DELIVERED","recipientId":"A","senderId":"B","timestamp":0,"payload":{"viewNumber":1,"sequenceNumber":1,"digest":"CtMEMkjjICiJ0zfwSspofgpOE5M=","request":{"operation":"C0/1","timestamp":1736960554627,"clientId":"C0","signed":false,"signedBy":null,"type":"REQUEST"},"signed":true,"signedBy":"B","type":"PREPARE"},"deliveredAt":null},{"type":"Message","eventId":18,"createdAt":1736960556.0801437,"status":"DELIVERED","recipientId":"D","senderId":"B","timestamp":0,"payload":{"viewNumber":1,"sequenceNumber":1,"digest":"CtMEMkjjICiJ0zfwSspofgpOE5M=","request":{"operation":"C0/1","timestamp":1736960554627,"clientId":"C0","signed":false,"signedBy":null,"type":"REQUEST"},"signed":true,"signedBy":"B","type":"PREPARE"},"deliveredAt":null},{"type":"Message","eventId":21,"createdAt":1736960556.0811455,"status":"DELIVERED","recipientId":"D","senderId":"B","timestamp":0,"payload":{"viewNumber":1,"sequenceNumber":1,"digest":"CtMEMkjjICiJ0zfwSspofgpOE5M=","request":{"operation":"C0/1","timestamp":1736960554627,"clientId":"C0","signed":false,"signedBy":null,"type":"REQUEST"},"replicaId":"B","speculativeHistory":{"history":{},"empty":true,"greatestSeqNumber":0,"requests":{}},"signed":true,"signedBy":"B","type":"COMMIT"},"deliveredAt":null},{"type":"Message","eventId":31,"createdAt":1736960562.4083266,"status":"DELIVERED","recipientId":"D","senderId":"A","timestamp":0,"payload":{"viewNumber":1,"sequenceNumber":1,"digest":"CtMEMkjjICiJ0zfwSspofgpOE5M=","request":{"operation":"C0/1","timestamp":1736960554627,"clientId":"C0","signed":false,"signedBy":null,"type":"REQUEST"},"replicaId":"A","speculativeHistory":{"history":{"1":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"}},"empty":false,"greatestSeqNumber":1,"requests":{"1":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"}}},"signed":true,"signedBy":"A","type":"COMMIT"},"deliveredAt":null},{"type":"ClientRequest","eventId":13,"createdAt":1736960554.6280425,"status":"DELIVERED","recipientId":"D","senderId":"C1","timestamp":1736960554628,"payload":{"timestamp":1736960554628,"operation":"C1/1","signed":false,"signedBy":null,"type":"DefaultClientRequest"},"deliveredAt":null},{"type":"Timeout","eventId":42,"createdAt":1736960574.9117787,"status":"DELIVERED","description":"REQUEST1736960554628","nodeId":"D","timeout":10,"expiresAt":10.001,"task":{},"recipientId":"D","deliveredAt":null},{"type":"Timeout","eventId":22,"createdAt":1736960556.9483867,"status":"DELIVERED","description":"REQUEST1736960554628","nodeId":"B","timeout":10,"expiresAt":10.002,"task":{},"recipientId":"B","deliveredAt":null},{"type":"MutateMessage","eventId":52,"senderId":"B","recipientId":"C","payload":{"eventId":24,"mutatorId":"hbft-prepare-sec-dec"},"createdAt":1736960584.0889707,"status":"DELIVERED","deliveredAt":null},{"type":"Message","eventId":24,"createdAt":1736960556.9483867,"status":"DELIVERED","recipientId":"C","senderId":"B","timestamp":0,"payload":{"viewNumber":1,"sequenceNumber":1,"digest":"vK5bH85oOq0vaso+TSfV+PLPXFE=","request":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"},"signed":true,"signedBy":"B","type":"PREPARE"},"deliveredAt":null},{"type":"Message","eventId":47,"createdAt":1736960576.5642834,"status":"DELIVERED","recipientId":"C","senderId":"D","timestamp":0,"payload":{"newViewNumber":2,"speculativeHistoryP":null,"speculativeHistoryQ":null,"requestsR":{"1":{"operation":"C0/1","timestamp":1736960554627,"clientId":"C0","signed":false,"signedBy":null,"type":"REQUEST"}},"replicaId":"D","signed":true,"signedBy":"D","type":"VIEW-CHANGE"},"deliveredAt":null},{"type":"MutateMessage","eventId":56,"senderId":"B","recipientId":"C","payload":{"eventId":50,"mutatorId":"hbft-view-change-remove-first-request"},"createdAt":1736960613.561314,"status":"DELIVERED","deliveredAt":null},{"type":"MutateMessage","eventId":57,"senderId":"B","recipientId":"C","payload":{"eventId":50,"mutatorId":"hbft-view-change-decrement-last-request-seqNum"},"createdAt":1736960617.884014,"status":"DELIVERED","deliveredAt":null},{"type":"Message","eventId":50,"createdAt":1736960579.971162,"status":"DELIVERED","recipientId":"C","senderId":"B","timestamp":0,"payload":{"newViewNumber":2,"speculativeHistoryP":null,"speculativeHistoryQ":null,"requestsR":{"1":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"}},"replicaId":"B","signed":true,"signedBy":"B","type":"VIEW-CHANGE"},"deliveredAt":null},{"type":"Message","eventId":62,"createdAt":1736960619.5324345,"status":"DELIVERED","recipientId":"A","senderId":"C","timestamp":0,"payload":{"newViewNumber":2,"viewChangeProofs":[{"newViewNumber":2,"speculativeHistoryP":null,"speculativeHistoryQ":null,"requestsR":{"1":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"}},"replicaId":"B","signed":true,"signedBy":"B","type":"VIEW-CHANGE"},{"newViewNumber":2,"speculativeHistoryP":null,"speculativeHistoryQ":null,"requestsR":{"1":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"}},"replicaId":"C","signed":true,"signedBy":"C","type":"VIEW-CHANGE"},{"newViewNumber":2,"speculativeHistoryP":null,"speculativeHistoryQ":null,"requestsR":{"1":{"operation":"C0/1","timestamp":1736960554627,"clientId":"C0","signed":false,"signedBy":null,"type":"REQUEST"}},"replicaId":"D","signed":true,"signedBy":"D","type":"VIEW-CHANGE"}],"checkpoint":{"sequenceNumber":0,"history":null},"speculativeHistory":{"history":{"1":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"}},"empty":false,"greatestSeqNumber":1,"requests":{"1":{"operation":"C1/1","timestamp":1736960554628,"clientId":"C1","signed":false,"signedBy":null,"type":"REQUEST"}}},"signed":true,"signedBy":"C","type":"NEW-VIEW"},"deliveredAt":null}],"brokenInvariants":[],"scenarioId":"hbft","finalized":false}

‎schedules/hbft-bug-or-potential-violation.json

+1
Large diffs are not rendered by default.

‎schedules/hbft-random-0-0-violation.json

+1
Large diffs are not rendered by default.

‎simulator/build.gradle.kts

+16-10
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ plugins {
22
java
33
id("org.springframework.boot") version "3.3.5"
44
id("io.spring.dependency-management") version "1.1.7"
5-
id("org.graalvm.buildtools.native") version "0.10.4"
5+
id("org.graalvm.buildtools.native") version "0.10.5"
66
//id("com.vaadin") version "24.4.7"
77
id("com.github.psxpaul.execfork") version "0.2.2"
88
id("org.springdoc.openapi-gradle-plugin") version "1.9.0"
@@ -25,16 +25,22 @@ repositories {
2525
mavenCentral()
2626
}
2727

28-
extra["vaadinVersion"] = "24.6.0"
28+
//extra["vaadinVersion"] = "24.6.4"
2929

3030
dependencies {
3131
implementation("org.springframework.boot:spring-boot-starter-web")
32+
implementation("org.springframework.boot:spring-boot-starter-data-jpa")
3233
compileOnly("org.projectlombok:lombok")
3334
annotationProcessor("org.springframework.boot:spring-boot-configuration-processor")
3435
annotationProcessor("org.projectlombok:lombok")
3536
testImplementation("org.springframework.boot:spring-boot-starter-test")
3637
testRuntimeOnly("org.junit.platform:junit-platform-launcher")
3738
implementation("org.springframework.boot:spring-boot-starter-validation")
39+
runtimeOnly("com.h2database:h2")
40+
implementation("org.xerial:sqlite-jdbc:3.49.1.0")
41+
42+
// sqlite dialect
43+
//implementation("org.hibernate:hibernate-community-dialects:5.6.3.Final")
3844

3945
//implementation("org.springdoc:springdoc-openapi-ui:1.8.0")
4046
// either API (just documentation) or API + UI (documentation + Swagger UI)
@@ -43,19 +49,19 @@ dependencies {
4349
//implementation("jakarta.xml.bind:jakarta.xml.bind-api:4.0.2") // required for webmvc-ui
4450
//compileOnly("javax.servlet:javax.servlet-api:4.0.1")
4551

46-
implementation("com.fasterxml.jackson.core:jackson-core:2.18.2")
47-
implementation("com.fasterxml.jackson.core:jackson-annotations:2.18.2")
48-
implementation("com.fasterxml.jackson.core:jackson-databind:2.18.2")
49-
implementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.18.2")
50-
implementation("com.fasterxml.jackson.module:jackson-module-parameter-names:2.18.2")
51-
implementation("com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.18.2")
52+
implementation("com.fasterxml.jackson.core:jackson-core:2.18.3")
53+
implementation("com.fasterxml.jackson.core:jackson-annotations:2.18.3")
54+
implementation("com.fasterxml.jackson.core:jackson-databind:2.18.3")
55+
implementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.18.3")
56+
implementation("com.fasterxml.jackson.module:jackson-module-parameter-names:2.18.3")
57+
implementation("com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.18.3")
5258
}
5359

54-
dependencyManagement {
60+
/*dependencyManagement {
5561
imports {
5662
mavenBom("com.vaadin:vaadin-bom:${property("vaadinVersion")}")
5763
}
58-
}
64+
}*/
5965

6066
tasks.withType<Test> {
6167
useJUnitPlatform()

‎simulator/src/main/java/byzzbench/simulator/BaseScenario.java

+11-1
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ public abstract class BaseScenario implements Scenario {
3737
/**
3838
* The timekeeper for the scenario.
3939
*/
40+
@JsonIgnore
4041
protected final transient Timekeeper timekeeper;
4142
/**
4243
* The scheduler for the scenario.
@@ -175,7 +176,7 @@ public final void setupScenario() {
175176
this.markReplicaFaulty(replicaIds.get(i));
176177
}
177178

178-
this.getClients().values().forEach(Client::initialize);
179+
//this.getClients().values().forEach(Client::initialize);
179180
this.getNodes().values().forEach(Node::initialize);
180181
this.scheduler.initializeScenario(this);
181182
}
@@ -236,6 +237,15 @@ public final void runScenario() {
236237
*/
237238
@Override
238239
public final boolean invariantsHold() {
240+
// Write to file which invariant is violated
241+
// BufferedWriter writer = new BufferedWriter(new FileWriter("", true));
242+
// for (ScenarioPredicate invariant : this.invariants) {
243+
// if (!invariant.test(this)) {
244+
// writer.write(this.id + " " + invariant.getClass().getSimpleName() + "\n");
245+
// writer.close();
246+
// }
247+
// }
248+
239249
return this.invariants.stream().allMatch(invariant -> invariant.test(this));
240250
}
241251

‎simulator/src/main/java/byzzbench/simulator/BaseScenarioFactory.java

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
public abstract class BaseScenarioFactory implements ScenarioFactory {
1515
@Getter(AccessLevel.PROTECTED)
1616
private final SchedulerFactoryService schedulerFactoryService;
17+
@Getter
1718
private final ByzzBenchConfig byzzBenchConfig;
1819
private final ObjectMapper mapper;
1920

0 commit comments

Comments
 (0)