You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: crates/bpe/README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -183,7 +183,7 @@ On average it is about ~4 faster, since the short-cuts usually pay off.
183
183
184
184
## Benchmarks
185
185
186
-
We ran several benchmarks to compare performance between different encoders and with the tiktoken library:
186
+
We ran several benchmarks to compare performance of different encoders and the [tiktoken-rs](https://crates.io/crates/tiktoken-rs) library (a wrapper around OpenAI's tiktoken implementation):
187
187
188
188
- The first measures encoding runtime for our different encoders and the tiktoken Rust implementation.
189
189
This shows a ~3.5x performance improvement for our fastest correct encoder compared to the tiktoken library.
0 commit comments