Skip to content

Commit a0cff1c

Browse files
BioPhotonBioPhoton
committed
wip
1 parent d8a763b commit a0cff1c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+2562
-1061
lines changed

‎bench/access-file-bail/index.ts

-54
This file was deleted.

‎bench/access-file/README.md

+87-115
Original file line numberDiff line numberDiff line change
@@ -1,187 +1,159 @@
1-
# Find Pattern in File - Benchmarks
1+
<!-- START HEADER -->
2+
# access-file
3+
<!-- END HEADER -->
24

3-
This benchmark explores various techniques available in **Node.js** for efficiently finding patterns within a file.
5+
Description: TBD
46

57
---
68

79
## Benchmark Requirements
810

9-
- **Search Content:** The search content should get returned as *string**
10-
- **Line:** The content should be accessible line by line
11+
- TBD
1112

1213
---
1314

14-
## Overview
15+
<!-- START OVERVIEW -->
16+
The node:fs.readFileSync benchmark is the fastest.
1517

16-
| **File** | **fs.readFileSync** | **fs/promises.readFile** | **readline** | **Fastest Speed Advantage** |
17-
|-------------------|---------------------|--------------------------|--------------|--------------------------------|
18-
| `10-loc.txt` | **6.23 µs** | 512.18 µs | 481.15 µs | **77.28x faster** than closest |
19-
| `100-loc.txt` | **8.08 µs** | 45.42 µs | 459.36 µs | **5.62x faster** than closest |
20-
| `1000-loc.txt` | **29.52 µs** | 581.85 µs | 781.66 µs | **19.71x faster** than closest |
21-
| `10000-loc.txt` | **564.30 µs** | 1.22 ms | 2.92 ms | **2.17x faster** than closest |
22-
| `100000-loc.txt` | 9.32 ms | **8.20 ms** | 14.28 ms | **1.14x faster** than closest |
23-
| `1000000-loc.txt` | **25.03 ms** | 41.92 ms | 56.63 ms | **1.68x faster** than closest |
2418

25-
---
26-
27-
### How Much Faster?
19+
## file access; loc: 100;
2820

29-
- **Small Files (10-1000 LOC):**
30-
- `fs.readFileSync` is **5.6x to 77x faster**, showing that synchronous reading is extremely efficient for small
31-
files.
21+
| alias | avg (min … max) | p75 | p99 | speed |
22+
| ------------------------ | ----------------------------------------- | ------------ | ------------ | ------------- |
23+
| **node:fs.readFileSync** | 7841.36 ns/iter (7640.02 … 8198.60) | 7969.66 ns | 8151.47 ns | 🔥 fastest |
24+
| node:fs/promise.readFile | 125984.07 ns/iter (99334.00 … 392917.00) | 129583.00 ns | 242250.00 ns | 16.07x slower |
25+
| node:readline | 232471.74 ns/iter (192542.00 … 556833.00) | 240958.00 ns | 329459.00 ns | 29.65x slower |
3226

33-
- **Medium Files (10,000 LOC):**
34-
- The speed difference reduces, but `fs.readFileSync` remains **2.17x faster** than its async counterpart.
27+
## file access; loc: 1000;
3528

36-
- **Large Files (100,000 LOC):**
37-
- Surprisingly, `fs/promises.readFile` slightly outperforms `fs.readFileSync`, indicating that asynchronous
38-
operations can start to optimize better for larger file sizes.
29+
| alias | avg (min … max) | p75 | p99 | speed |
30+
| ------------------------ | ----------------------------------------- | ------------ | ------------ | ------------- |
31+
| **node:fs.readFileSync** | 26773.27 ns/iter (26543.04 … 27086.21) | 26905.63 ns | 26973.61 ns | 🔥 fastest |
32+
| node:fs/promise.readFile | 70881.15 ns/iter (59850.19 … 73539.74) | 73092.68 ns | 73466.15 ns | 2.65x slower |
33+
| node:readline | 646319.26 ns/iter (572083.00 … 770583.00) | 662459.00 ns | 744791.00 ns | 24.14x slower |
3934

40-
- **Massive Files (1,000,000 LOC):**
41-
- `fs.readFileSync` regains performance advantage, being **1.68x faster** than the async variant, and **2.26x faster
42-
** than `readline`.
35+
## file access; loc: 10000;
4336

44-
---
37+
| alias | avg (min … max) | p75 | p99 | speed |
38+
| ---------------------------- | -------------------------------------------- | ------------- | ------------- | ------------- |
39+
| **node:fs/promise.readFile** | 376297.66 ns/iter (338625.00 … 504542.00) | 383958.00 ns | 452875.00 ns | 🔥 fastest |
40+
| node:fs.readFileSync | 533772.21 ns/iter (478959.00 … 741250.00) | 543834.00 ns | 684750.00 ns | 1.42x slower |
41+
| node:readline | 4402708.96 ns/iter (3836916.00 … 4858125.00) | 4534834.00 ns | 4808333.00 ns | 11.70x slower |
4542

46-
## File Content Access Methods
47-
48-
---
43+
## file access; loc: 100000;
4944

50-
Overview
45+
| alias | avg (min … max) | p75 | p99 | speed |
46+
| ---------------------------- | ----------------------------------------------- | -------------- | -------------- | ------------- |
47+
| **node:fs/promise.readFile** | 2691327.25 ns/iter (2555917.00 … 3081917.00) | 2728375.00 ns | 3069541.00 ns | 🔥 fastest |
48+
| node:fs.readFileSync | 4212338.30 ns/iter (4005833.00 … 4749417.00) | 4285792.00 ns | 4608083.00 ns | 1.57x slower |
49+
| node:readline | 37398813.81 ns/iter (36849334.00 … 38135625.00) | 37536625.00 ns | 37965208.00 ns | 13.90x slower |
5150

52-
- `node:fs.readFileSync`
53-
- `node:fs.readFile`
54-
- `node:fs.createReadStream` with `node:readline.createInterface`
51+
<!-- END OVERVIEW -->
5552

5653
---
5754

58-
### 1. Synchronous File Reading
59-
60-
- **Method:** `fs.readFileSync`
61-
- **Description:** Reads the entire file synchronously into memory before processing.
62-
- **Source:** [node-fs.readFileSync.ts](./src/node-fs.readFileSync.ts)
63-
64-
```javascript
65-
import * as fs from "node:fs";
55+
<!-- START CASES -->
56+
## node-fs.promise.readFile.ts
57+
_[node-fs.promise.readFile.ts](access-file/src)_
58+
```ts
59+
import * as fs from "node:fs/promises";
6660

67-
export default function accessContent(filePath: string): void {
68-
const content = fs.readFileSync(filePath, "utf8");
69-
content.split("\n")
70-
.forEach(line => (line === 'not-in-file'));
61+
export default function accessFile(filePath: string): Promise<string> {
62+
return fs.readFile(filePath, "utf8");
7163
}
72-
```
73-
74-
### 2. Asynchronous File Reading
7564

76-
- **Method:** `fs.promises.readFile`
77-
- **Description:** Reads the entire file asynchronously into memory before processing.
78-
- **Source:** [node-fs.promise.readFile.ts](./src/node-fs.promise.readFile.ts)
65+
```
7966

80-
```javascript
81-
import * as fs from "node:fs/promises";
67+
## node-fs.readFileSync.ts
68+
_[node-fs.readFileSync.ts](access-file/src)_
69+
```ts
70+
import * as fs from "node:fs";
8271

83-
export default async function accessContent(filePath): Promise<void> {
84-
const content = await fs.readFile(filePath, "utf8");
85-
content.split("\n")
86-
.forEach(line => (line === 'not-in-file'));
72+
export default function accessFile(filePath: string): string {
73+
return fs.readFileSync(filePath, "utf8");
8774
}
88-
```
8975

90-
### 3. Streaming File Reading (Line by Line)
91-
92-
- **Method:** `fs.createReadStream` with `readline.createInterface`
93-
- **Description:** Processes the file line by line without loading the entire file into memory.
94-
- **Source:** [node-readline](./src/node-readline.ts)
76+
```
9577

96-
```javascript
78+
## node-readline.ts
79+
_[node-readline.ts](access-file/src)_
80+
```ts
9781
import * as fs from "node:fs";
9882
import * as readline from "node:readline";
9983

100-
export default async function accessContent(filePath: string): Promise<void> {
84+
export default async function* accessFile(filePath: string): AsyncGenerator<string> {
10185
const stream = fs.createReadStream(filePath);
10286
const rl = readline.createInterface({input: stream});
103-
for await (const line of rl) {
104-
line === 'not-in-file';
87+
88+
try {
89+
for await (const line of rl) {
90+
yield line;
91+
}
92+
} finally {
93+
rl.close();
10594
}
10695
}
96+
10797
```
10898

99+
<!-- END CASES -->
100+
109101
---
110102

111103
## Benchmark Results
112104

113-
```shell
114-
> npx tsx --tsconfig=../tsconfig.perf.json file-access
115-
105+
<!-- START DATA -->
106+
```bash
116107
clk: ~3.33 GHz
117108
cpu: Apple M2 Max
118-
runtime: node 22.12.0 (arm64-darwin)
109+
runtime: node 23.9.0 (arm64-darwin)
119110

120111
benchmark avg (min … max) p75 / p99 (min … top 1%)
121112
------------------------------------------- -------------------------------
122-
• file access; loc: 10;
123-
------------------------------------------- -------------------------------
124-
node:fs.readFileSync 6.23 µs/iter 6.26 µs 6.83 µs ▂█▆▃▂▁▁▁▁▁▁
125-
node:fs/promise.readFile 512.18 µs/iter 661.42 µs 1.36 ms ▂█▄▂▃▂▁▁▁▁▁
126-
node:readline 481.15 µs/iter 533.42 µs 978.21 µs ▁▁▂█▅▂▂▂▂▁▁
127-
128-
summary
129-
node:fs.readFileSync
130-
77.28x faster than node:readline
131-
82.26x faster than node:fs/promise.readFile
132-
133113
• file access; loc: 100;
134114
------------------------------------------- -------------------------------
135-
node:fs.readFileSync 8.08 µs/iter 8.09 µs 8.38 µs ▃▅▇█▅▃▁▁▃▃▂
136-
node:fs/promise.readFile 45.42 µs/iter 45.30 µs 46.21 µs ▅▃█▅▁▁▁▁▃▁▃
137-
node:readline 459.36 µs/iter 515.54 µs 931.92 µs ▁▁▂█▅▂▂▂▂▁▁
115+
node:fs.readFileSync 7.93 µs/iter 7.98 µs 8.13 µs ▃▇▄▇▆█▂▁▁▂▂
116+
node:fs/promise.readFile 124.55 µs/iter 130.13 µs 201.29 µs ▃▅█▇▃▂▁▁▁▁▁
117+
node:readline 233.24 µs/iter 241.75 µs 341.08 µs ▁███▄▂▂▁▁▁▁
138118

139119
summary
140120
node:fs.readFileSync
141-
5.62x faster than node:fs/promise.readFile
142-
56.83x faster than node:readline
121+
15.71x faster than node:fs/promise.readFile
122+
29.42x faster than node:readline
143123

144124
• file access; loc: 1000;
145125
------------------------------------------- -------------------------------
146-
node:fs.readFileSync 29.52 µs/iter 29.07 µs 29.82 µs ▂▂█▁▄▂▁▁▁▁▂
147-
node:fs/promise.readFile 581.85 µs/iter 720.67 µs 1.35 ms ▁▇█▂▂▃▂▂▁▁▁
148-
node:readline 781.66 µs/iter 837.54 µs 1.36 ms ▁▁▁▄█▃▂▂▂▂▁
126+
node:fs.readFileSync 27.72 µs/iter 27.87 µs 28.31 µs ▅▅█▅▅▁▅▁▁▁▅
127+
node:fs/promise.readFile 71.42 µs/iter 73.95 µs 74.04 µs ▃▁▁▁▁▁▁▁▁▅█
128+
node:readline 644.37 µs/iter 662.00 µs 743.96 µs ▁▁▃▅██▅▃▂▁▁
149129

150130
summary
151131
node:fs.readFileSync
152-
19.71x faster than node:fs/promise.readFile
153-
26.48x faster than node:readline
132+
2.58x faster than node:fs/promise.readFile
133+
23.24x faster than node:readline
154134

155135
• file access; loc: 10000;
156136
------------------------------------------- -------------------------------
157-
node:fs.readFileSync 564.30 µs/iter 568.13 µs 1.44 ms ▂▁█▃▁▁▁▁▁▁▁
158-
node:fs/promise.readFile 1.22 ms/iter 1.38 ms 1.74 ms ▁▅█▇▆▆▇▄▃▂▁
159-
node:readline 2.92 ms/iter 3.00 ms 3.23 ms ▁▃▅▄▆█▇▅▃▂▁
137+
node:fs.readFileSync 566.27 µs/iter 576.13 µs 757.79 µs ▂██▄▂▂▂▂▁▁▁
138+
node:fs/promise.readFile 381.38 µs/iter 389.38 µs 491.50 µs ▂▄██▄▂▁▁▁▁▁
139+
node:readline 4.48 ms/iter 4.64 ms 4.94 ms ▁▃▃▃▅▇█▇▃▃▁
160140

161141
summary
162-
node:fs.readFileSync
163-
2.17x faster than node:fs/promise.readFile
164-
5.17x faster than node:readline
142+
node:fs/promise.readFile
143+
1.48x faster than node:fs.readFileSync
144+
11.75x faster than node:readline
165145

166146
• file access; loc: 100000;
167147
------------------------------------------- -------------------------------
168-
node:fs.readFileSync 9.32 ms/iter 9.65 ms 9.97 ms ▁▁▁▁▁▁▁▂▄█▂
169-
node:fs/promise.readFile 8.20 ms/iter 8.78 ms 9.28 ms ▃▄▂▂▃▃▄▆█▄▁
170-
node:readline 14.28 ms/iter 14.46 ms 15.08 ms ▂▆█▆▅▃▄▂▂▂▂
148+
node:fs.readFileSync 4.48 ms/iter 4.59 ms 5.82 ms ▂█▅▄▂▁▂▁▂▁▁
149+
node:fs/promise.readFile 2.78 ms/iter 2.82 ms 3.45 ms ▃▇█▅▃▁▂▂▂▂▁
150+
node:readline 39.24 ms/iter 39.38 ms 42.57 ms ▂▅█▄▂▂▂▁▂▁▂
171151

172152
summary
173153
node:fs/promise.readFile
174-
1.14x faster than node:fs.readFileSync
175-
1.74x faster than node:readline
176-
177-
• file access; loc: 1000000;
178-
------------------------------------------- -------------------------------
179-
node:fs.readFileSync 25.03 ms/iter 24.87 ms 34.75 ms ▂█▂▁▁▁▁▁▁▁▁
180-
node:fs/promise.readFile 41.92 ms/iter 45.11 ms 46.60 ms ▅▄▁▁▁▃▃▄█▇▃
181-
node:readline 56.63 ms/iter 56.98 ms 57.20 ms ▃▁▃▃▆▆▅▆▃█▃
154+
1.61x faster than node:fs.readFileSync
155+
14.13x faster than node:readline
182156

183-
summary
184-
node:fs.readFileSync
185-
1.68x faster than node:fs/promise.readFile
186-
2.26x faster than node:readline
187157
```
158+
159+
<!-- END DATA -->

‎bench/access-file/index.ts

+4-12
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import * as path from "node:path";
22
import {bench, compact, do_not_optimize, group, run, summary} from 'mitata';
3-
import {fileName, runAndSave} from "../../src/lib/utils.ts";
3+
import {fileName, runWithConfig} from "../../src/lib/utils.ts";
44
import {LOC_RANGES, VOLUME_TEST_DIR} from "../../src/lib/constants.ts";
55
import readFileSync from "./src/node-fs.readFileSync.ts";
66
import readFilePromise from "./src/node-fs.promise.readFile.ts";
@@ -17,17 +17,13 @@ LOC_RANGES.forEach((loc) => {
1717
{
1818
name: 'node:fs.readFileSync',
1919
fn: () => {
20-
for (const line of readFileSync(targetFile)) {
21-
do_not_optimize(line);
22-
}
20+
do_not_optimize(readFileSync(targetFile));
2321
}
2422
},
2523
{
2624
name: 'node:fs/promise.readFile',
2725
fn: async () => {
28-
for await (const line of readFilePromise(targetFile)) {
29-
do_not_optimize(line);
30-
}
26+
do_not_optimize(await readFilePromise(targetFile));
3127
}
3228
},
3329
{
@@ -47,8 +43,4 @@ LOC_RANGES.forEach((loc) => {
4743
});
4844
});
4945

50-
const __dirname = path.dirname(new URL(import.meta.url).pathname);
51-
await runAndSave(path.join(process.cwd(), '.bench', `${__dirname}-report.json`), () => run({
52-
throw: true,
53-
format: 'json',
54-
}));
46+
await runWithConfig(run)

‎bench/access-file/src/node-fs.promise.readFile.ts

+2-13
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,5 @@
11
import * as fs from "node:fs/promises";
22

3-
export default async function* accessContent(filePath: string, bail = false): AsyncGenerator<string> {
4-
const content = await fs.readFile(filePath, "utf8");
5-
let start = 0;
6-
7-
while (start < content.length) {
8-
const end = content.indexOf('\n', start);
9-
if (end === -1) {
10-
yield content.slice(start); // Yield last line if no newline is found
11-
break;
12-
}
13-
yield content.slice(start, end);
14-
start = end + 1;
15-
}
3+
export default function accessFile(filePath: string): Promise<string> {
4+
return fs.readFile(filePath, "utf8");
165
}

‎bench/access-file/src/node-fs.readFileSync.ts

+2-7
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,5 @@
11
import * as fs from "node:fs";
22

3-
export default function* accessContent(filePath: string, bail = false): Generator<string> {
4-
const content = fs.readFileSync(filePath, "utf8");
5-
const lines = content.split("\n");
6-
7-
for (const line of lines) {
8-
yield line;
9-
}
3+
export default function accessFile(filePath: string): string {
4+
return fs.readFileSync(filePath, "utf8");
105
}

0 commit comments

Comments
 (0)