improve JSDocs extraction/parsing

[pi0]

(using this here for discussion because it is closer to the place where I can test things out haha)

Currently, we use untyped that itself uses jiti + babel parser to traverse the source files and extract the JSDocs comments above them from AST and then use a small parser in untyped to parse them. since untyped parser was buggy, I made another parse util in automd only to make an MVP working.

But things aren't perfect and we need to improve...

File traverse / AST parsing

We might make a change in untyped or maybe make some alternative loaders that use static syntax analyzer to traverse the source files and understand imports. (I am thinking to export it from mlly ideally and looking for a good zero dep+native AST parser at the same time: unjs/mlly#219)

More context why: While jiti+babel works well and supports runtime value integration, it has two issues at least I see today:

• If source code cannot be executed in Node.js environment, this method fails
• Babel has some edge-case issues with comment extractions. Most importantly it doesn't support function overrides
• TS Server is huge also, so not an option

Comment parsing/interpreting

Parsing JSDoc/TSDoc comments is another tricky part. Right now we have two utils that both are dummy to just work in limited situations.

I found some alternative options (yet to try):

• jsdoc-parse: uses lodash an manual string parsing
• dox
• jsdoctypeparser

Both are nice libs but they use same method of our current utils that rely on manual/regex based parsing.

I am thinking of using a first-class markdown based parser on comments that can understand full-markdown correctly in the comments syntax and then we can interpret it further for things like tags and type refs.

There are few options of markdown parser options, I was thinking to make a minimal one in unjs/omark and under the hood use faster parser option but use an standard/minimal layer like this that we can rely on and improve it under the hood. I have no ideas where to expose this jsdocs parser from. Could be automd or another unjs package that relying on the markdown parsing of omark. (exposing parse utils from md4w by @ije is one option: ije/md4w#3)