Webassembly

[do-me]

It would be great to compile model2vec-rs to webassembly. I know that transformers.js already supports running most models in the browser but I have the suspicion that it might be slower than this optimized Rust-based implementation.
Also, it would be cool to a minimal library without all the dependencies like ONNX etc.

Would you consider this within the scope of this repo?

[Pringled]

Hey @do-me! Unfortunately I think Tokenizers is not WASM friendly; I did find this: https://github.com/mithril-security/tokenizers-wasm but it's pretty outdated. Not an expert on this though, so if you have any thoughts/suggestions let me know!

[do-me]

I opened a ticket in TokenDagger meanwhile. Alternatively we could simply use the transformers.js tokenizer I guess.