Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JavaScript Raw engine to the playground #115

Open
slevithan opened this issue Jan 24, 2025 · 5 comments
Open

Add JavaScript Raw engine to the playground #115

slevithan opened this issue Jan 24, 2025 · 5 comments

Comments

@slevithan
Copy link

Currently, the grammar/theme playground allows switching between the Oniguruma and JavaScript engines, and for the JavaScript engine it allows toggling the forgiving option.

It would be helpful for experimentation and debugging if you could also choose Shiki's JavaScript Raw engine.

Less important, but it might also be helpful if for the non-raw JavaScript engine you could also set the target option's value (auto, ES2025, ES2024, ES2018).

@antfu
Copy link
Member

antfu commented Jan 25, 2025

The raw engine requires the grammar to be pre-compiled, which doesn't really apply to this repo as the grammar's regex here is in string format. If we compile them there, it's essentially the same as the JavaScript engine, raw engine is more for smaller bundle size.

Less important, but it might also be helpful if for the non-raw JavaScript engine you could also set the target option's value (auto, ES2025, ES2024, ES2018).

This sounds good to have

@slevithan
Copy link
Author

Maybe it could be a "precompiled" checkbox for the JavaScript engine, rather than listed as a separate engine? Yes, it would be the same engine, but including an option for precompiled grammars would allow checking two specific things:

  1. The "Highlighting finished in" timer on the bottom.
  2. Whether it was able to run at all (i.e., there wasn't some problem with the precompiled grammar that led to it throwing or otherwise failing).

doesn't really apply to this repo

That's fair. But I'm assuming it could be made to work using the existing @shikijs/langs-precompiled without changes in this repo?

In any case, no worries at all if you think it's better to not include it!

@antfu
Copy link
Member

antfu commented Jan 25, 2025

The point of the playground is to run the latest grammars and themes from the source. Using @shikijs/langs-precompiled loose the point and will be misaligned.

  1. After the first run, the second run of JavaScript engine would be the same as precompiled, as we cache the regex instance
  2. The same as the JavaScript engine, if it doesn't run, it won't run as precompiled.

@slevithan
Copy link
Author

slevithan commented Jan 25, 2025

I think you're right to leave it out -- it's not designed to be my personal debugging playground 🙂

However, note that while In theory your points 1 and 2 above are true, in practice they might not be.

  1. The cached run doesn't include regex construction (not transpilation) time for the full grammar. This might be tricky to report on anyway, but ideally the precompiled version would report it at least on the first run. This aspect should only make a significant difference for extreme cases, but I'm currently working on solving an extreme case: Poor performances on CPP language shiki#893
  2. Some precompiled grammars for languages supported by the JavaScript engine (specifically angular-html, bicep, blade, erb, html, jison, pug, svelte) don't seem to be running correctly at the moment, based on Shiki's bench script.

@antfu
Copy link
Member

antfu commented Jan 25, 2025

Yeah, but that sounds more like the issues on the Shiki side, there we are more focused on the grammar themselves. If we want to test the precompiled engine/logic, we better to it in the main shiki repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants