Skip to content

Commit 8c9e05b

Browse files
Link to tiktoken crate used
1 parent 814ef8d commit 8c9e05b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

crates/bpe/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@ On average it is about ~4 faster, since the short-cuts usually pay off.
183183

184184
## Benchmarks
185185

186-
We ran several benchmarks to compare performance between different encoders and with the tiktoken library:
186+
We ran several benchmarks to compare performance of different encoders and the [tiktoken-rs](https://crates.io/crates/tiktoken-rs) library (a wrapper around OpenAI's tiktoken implementation):
187187

188188
- The first measures encoding runtime for our different encoders and the tiktoken Rust implementation.
189189
This shows a ~3.5x performance improvement for our fastest correct encoder compared to the tiktoken library.

0 commit comments

Comments
 (0)