Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,7 @@ Callbacks run in source order. Edits are validated before any are applied. They

Every callback node is a dict with these common fields:

- `type`: callback name, currently `image` or `math_inline`.
- `type`: callback name, including `image`, `link`, and `math_inline`.
- `source`: the exact source text for the construct.
- `start`, `end`: half-open character offsets into the original Python string.

Expand All @@ -355,6 +355,8 @@ An `image` node has:

An image callback may return `{"url": "new destination"}`. Other image fields are read-only. Reference-style images such as `![alt][id]` are not callback targets.

A `link` node has the same `form`, `url`, and `title` fields as an image, and its callback may likewise return `{"url": "new destination"}`. Reference-style links are not callback targets.

A `math_inline` node has:

- `delimiter`: `$`, `$$`, `\(`, or `\[`.
Expand All @@ -363,7 +365,7 @@ A `math_inline` node has:

A math callback may return `{"tex": "new TeX"}` to preserve the delimiters, or a string to replace the entire construct. Dollar math is recognized only with `math="dollars"`, using the same dollar rules as rendering.

Rewriting is confined to inline-capable prose regions. Inline code, fenced and indented code blocks, raw HTML blocks, block math, and link reference definitions are left untouched. Inline images and math inside paragraphs, headings, lists, block quotes, definition bodies, footnotes, and pipe tables are supported.
Rewriting is confined to inline-capable prose regions. Inline code, fenced and indented code blocks, raw HTML blocks, block math, and link reference definitions are left untouched. Inline links, images, and math inside paragraphs, headings, lists, block quotes, definition bodies, footnotes, and pipe tables are supported.

### Callbacks

Expand Down Expand Up @@ -467,6 +469,8 @@ Use `id_prefix='md-'` to distinguish exported ids from those of the host page. E

Set `number_headings` to `'legal'`, `'decimal'`, or a `{lvlText: numFmt}` dictionary as in mdhtml2docx. When the argument is omitted, HTML, GFM, and Typst exporters use the document's frontmatter `number_headings` setting. If a reference needs a heading number and neither source supplies a scheme, numbering uses `'decimal'`.

For HTML, GFM, and Typst, `number_headings=False` explicitly disables heading numbering, overriding frontmatter and automatic numbering. Frontmatter accepts `number_headings: false`; `md2html` and `viewmd` accept `--number-headings false`. Numeric references to unnumbered headings raise an error; use `{ref=text}` to refer to their text instead. HTML's `refs='ids'` mode still displays target IDs without numbering.

For example, put `number_headings: legal` in frontmatter and run `md2html contract.md --frontmatter`. Headings use legal numbering without a separate numbering option. `viewmd contract.ipynb` also reads this setting from the notebook's frontmatter cell.

Heading numbers appear in `<span class="heading-number">` elements. Reference text includes the full context, such as "3.(c)(iii)", computed from the scheme using Word's rules.
Expand Down Expand Up @@ -540,6 +544,8 @@ References use plain text because anchor links depend on each renderer's id and

With `imgdir=`, base64 data-URI images are saved in that directory with content-hashed filenames. Their source paths become relative to the directory containing `dest`. You can commit these files for GitHub to serve.

With `link=`, a callback receives each inline link or image URL and returns its replacement, or `None` to leave it unchanged. GFM export uses `rewrite()` to update URLs before lowering other constructs, so rewrites survive heading numbering and caption formatting. `imgdir` takes precedence for base64 images. Reference-style links are unchanged. The same URL edits are available directly through `rewrite(source, {'link': callback})`, whose node callback returns `{'url': replacement}`. Pass `templates=` to either function to protect template tokens.

With `templates=`, the `tmpl(node)` callback supplies each token's replacement. For example, `mustache_code` wraps tokens in code spans for literal display. Without `tmpl`, tokens remain byte-identical.

Inline recognition uses the parser's grammar at every nesting depth. It respects code spans, links, and escapes. Text such as `use {braces} freely` remains literal.
Expand Down Expand Up @@ -571,7 +577,7 @@ Typst resolves references at compile time. `[@sec-pay]` becomes `#ref(<sec-pay>,
Reference and numbering options have these effects:

- `reftypes` supplies Typst supplements.
- `number_headings` emits a `set heading` rule using the same `SCHEMES` and Word-style full-context numbers as other exporters. With `None`, numbering is enabled when a reference needs it.
- `number_headings` emits a `set heading` rule using the same `SCHEMES` and Word-style full-context numbers as other exporters. `None` inherits input metadata, falling back to automatic numbering when a reference needs it; `False` disables heading numbering.
- Figures and tables use Typst's native numbering.
- `{ref=page}` produces a page reference, such as `page 6`, and enables page numbering.
- `{ref=text}` links the target's text.
Expand Down
2 changes: 1 addition & 1 deletion docs/DIALECT.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ The parser does not resolve numbers or require targets to exist. Converters repo

Reference targets are the id-bearing headings, paragraphs, figures, tables, spans, and definition terms. Headings resolve to a heading number ("Section 1"), figures and tables to a caption number ("Table 2"), and paragraphs only through `{ref=text}`. Spans and definition terms resolve to their own text with no prefix word, so `the [@def-term] period` reads as running prose citing the defined term, and a rendering variant that needs a number (`leaf`, `rel`) is an error for them. Converters with a link mechanism link the cited text back to its definition site; `md2gfm` renders the text alone.

The shipped exporters lower references from one shared vocabulary (`mdhtml.export`) at three levels of liveness: `mdhtml2docx` bakes REF fields Word keeps live, `mdhtml2html` bakes links with computed text, and `md2gfm` bakes plain text. Prefix words come from `REFTYPES` (`sec`, `fig`, `tbl`; extended per call with `reftypes=`) and heading numbering from `SCHEMES` (`'legal'`, `'decimal'`, or a `{lvlText: numFmt}` dict). `number_headings=None` means automatic: headings are numbered exactly when some reference needs a heading number. Scheme level 0 is the h1 document title, with an empty lvlText: it shows no number, and bumping it restarts every level below, so `%2` is the h2 counter and each document in a file that opens with its own h1 numbers from 1; a title is cited with `{ref=text}`, never by number.
The shipped exporters lower references from one shared vocabulary (`mdhtml.export`) at three levels of liveness: `mdhtml2docx` bakes REF fields Word keeps live, `mdhtml2html` bakes links with computed text, and `md2gfm` bakes plain text. Prefix words come from `REFTYPES` (`sec`, `fig`, `tbl`; extended per call with `reftypes=`) and heading numbering from `SCHEMES` (`'legal'`, `'decimal'`, or a `{lvlText: numFmt}` dict). Automatic numbering numbers headings exactly when some reference needs a heading number. For HTML, GFM, and Typst, `number_headings=None` inherits the input's metadata setting, falling back to automatic numbering; `False` disables heading numbering, and numeric references to unnumbered headings are errors. Scheme level 0 is the h1 document title, with an empty lvlText: it shows no number, and bumping it restarts every level below, so `%2` is the h2 counter and each document in a file that opens with its own h1 numbers from 1; a title is cited with `{ref=text}`, never by number.

`mdhtml2html` also offers `refs='ids'` for live-preview contexts where targets may sit outside the fragment: each reference bakes as a working link showing its target id (class `xref`), with no registry, numbering, or failure modes - and captions render as authored, since per-fragment numbers would lie. `id_prefix` namespaces the fragment's ids against a host page (the authored id kept in `data-id`), and `fn_salt` adds a further prefix to the `fn-*`/`fnref-*` footnote namespace only, keeping footnote pairs distinct across fragments that share one `id_prefix`.

Expand Down
1 change: 1 addition & 0 deletions py/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -929,6 +929,7 @@ fn export_html(
let number_headings = match number_headings {
None => None,
Some(o) if o.is_none() => None,
Some(o) if o.extract::<bool>().is_ok_and(|b| !b) => Some(NumberHeadings::Off),
Some(o) => Some(match o.extract::<String>() {
Ok(name) => NumberHeadings::Name(name),
Err(_) => {
Expand Down
6 changes: 3 additions & 3 deletions python/mdhtml/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,11 +121,11 @@ def blocks(markdown: str, *, math: str = "brackets", implicit_figures: bool = Fa
return _blocks(markdown, math=math, implicit_figures=implicit_figures, templates=_template_args(templates))


def rewrite(markdown: str, callbacks: dict, *, math: str = "brackets") -> str:
"Rewrite recognized Markdown constructs while preserving all other source text."
def rewrite(markdown: str, callbacks: dict, *, math: str = "brackets", templates=None) -> str:
"Rewrite recognized md constructs while preserving all other source text; `templates` protects template tokens."
normalized, offsets = _normalize_offsets(markdown)
edits = []
for raw in _edit_nodes(normalized, math=math):
for raw in _edit_nodes(normalized, math=math, templates=_template_args(templates)):
norm_start, norm_end = raw["start"], raw["end"]
start, end = offsets[norm_start], offsets[norm_end]
internal = {k: raw.pop(k) for k in tuple(raw) if k.startswith("_")}
Expand Down
13 changes: 7 additions & 6 deletions python/mdhtml/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,10 @@ def math_js(fn=None, **opts):
return _math_js(fn, "".join(f", {k}: {json.dumps(v)}" for k, v in opts.items()))


def _headnums(src, number_headings):
"The call's `number_headings`, else the source's frontmatter `number_headings:` (an `Mdhtml` carries its `meta`)"
return number_headings if number_headings is not None else getattr(src, "meta", {}).get("number_headings")
def _headnums(meta, number_headings):
"Explicit numbering overrides metadata; the frontmatter/CLI spelling `false` disables it."
scheme = number_headings if number_headings is not None else meta.get("number_headings")
return False if scheme == 'false' else scheme


def meta_table(meta):
Expand Down Expand Up @@ -120,11 +121,11 @@ def mdhtml2html(src, dest=None, reftypes: dict | None = None, number_headings=No
may return replacement markup for the highlighted block (None keeps it; `text` is unescaped).
Highlighting comes from the optional fastpylight package (`pip install 'mdhtml[hl]'`);
without it, code blocks render plain and a warning reports it.
`number_headings=None` takes the scheme from the source's frontmatter `number_headings:` when
`src` is `md2mdhtml`'s result (its `meta` carries the block), else numbers automatically.
`number_headings=None` inherits the input's metadata setting, else numbers automatically when
a reference needs it. `False` disables heading numbering, including automatic numbering.
Returns an `Html` str carrying `.warnings`; `dest` also writes it to a file."""
if refs not in ("resolve", "ids", "lenient"): raise ValueError(f"unknown refs mode {refs!r}")
number_headings = _headnums(src, number_headings)
number_headings = _headnums(getattr(src, 'meta', {}), number_headings)
if not isinstance(src, str): src = src.to_html()
hl_fn = None if hl is None else _hl_fn(hl)
out, warnings = _export_html(src, reftypes, number_headings, hl, toc, refs, id_prefix, fn_salt, hl_lang, code_wrap, hl_fn, auto_ids, gh_ids)
Expand Down
34 changes: 20 additions & 14 deletions python/mdhtml/md.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

from ._native import (blocks as _blocks, edit_nodes as _edit_nodes, anchors as _anchors, trailing_attr_span as _trailing_attr_span,
frontmatter_meta as _frontmatter_meta)
from .export import HeadingNums, Resolver, group_plan, ref_tokens, ref_variant
from .export import HeadingNums, Resolver, group_plan, ref_tokens, ref_variant, _headnums

__all__ = ["md2gfm"]

Expand Down Expand Up @@ -64,12 +64,11 @@ def _is_caption(line):

class _GfmExporter:
def __init__(self, reftypes, number_headings, math, implicit_figures, templates=None, tmpl=None,
raw=("md",), imgdir=None, imgbase=None, link=None):
raw=("md",), imgdir=None, imgbase=None):
self.res = Resolver(reftypes)
self.number_headings, self.math, self.implicit_figures = number_headings, math, implicit_figures
self.templates, self.tmpl = [astuple(t) if is_dataclass(t) else tuple(t) for t in templates or []], tmpl
self.raw, self.imgdir, self.imgbase = raw, imgdir, imgbase
self.link = link
self.warnings, self.inline, self.block, self.rebuilt = [], [], [], []

def run(self, src):
Expand All @@ -87,11 +86,6 @@ def run(self, src):
if n["type"] == "attrs": self.inline.append((n["start"], n["end"], ""))
elif n["type"] == "raw_inline": self.inline.append((n["start"], n["end"], n["text"] if n["format"] in self.raw else ""))
elif n["type"] == "template_token" and self.tmpl: self.inline.append((n["start"], n["end"], self.tmpl(n)))
elif n["type"] == "image" and self.imgdir and ";base64," in n["url"] and n["url"].startswith("data:"):
self.inline.append((n["_url_start"], n["_url_end"], self._extract_img(n["url"])))
elif self.link is not None and n["type"] in ("image", "link"):
new = self.link(n["url"])
if new is not None: self.inline.append((n["_url_start"], n["_url_end"], new))
for x, parsed in self.xrefs: self._xref(x, parsed)
for b in spans: self._block(b)
keep = [e for e in self.inline if not any(s <= e[0] and e[1] <= t for s, t in self.rebuilt)]
Expand Down Expand Up @@ -201,7 +195,7 @@ def _index(self, spans, nodes):
needed = any(kinds[r["target"]] == "block" and ref_variant(toks) != "text"
for _, parsed in self.xrefs for r, toks in parsed)
self.headnum = {}
if self.number_headings or needed:
if self.number_headings is not False and (self.number_headings or needed):
nums = HeadingNums(self.number_headings or "decimal")
for b in self.heads:
if not (d := nums.bump(b["level"] - 1)): continue # None beyond the scheme, '' at the title level
Expand Down Expand Up @@ -303,18 +297,30 @@ def md2gfm(src, dest=None, reftypes: dict | None = None, number_headings=None, m
lists and definitions are stripped, and raw data in the formats named by `raw` is spliced
(all other formats drop; `('md', 'html')` suits targets that render inline HTML, like GFM).
With `imgdir`, each base64 data-URI image is written to a content-hashed file in that
directory and its src rewritten relative to `dest`'s directory (or the cwd). With `link`, each inline link or image URL is passed to the callback and replaced when it returns a non-`None` string. With `templates`,
directory and its src rewritten relative to `dest`'s directory (or the cwd). With `link`, each
inline link or image URL is passed to the callback and replaced when it returns a non-`None`
string; `imgdir` takes precedence for base64 images. URL rewriting precedes GFM conversion,
preserving rewrites inside headings and captions. Reference-style links are left unchanged. With `templates`,
each template token is rewritten to whatever the
`tmpl` callable `(node) -> str` returns: the node dict carries `body`, `syntax`, `form`,
scanner classification (`kind`, `name`, `inverted`), and spans (`mustache_code` is a ready-made recipe;
without `tmpl`, tokens pass through). All other source text is preserved byte-for-byte,
the frontmatter included; `number_headings=None` takes the scheme from its `number_headings:`.
the frontmatter included; `number_headings=None` inherits its `number_headings:`; `False` disables numbering.
Returns an `Md` str carrying `.warnings`; `dest` also writes it to a file."""
if number_headings is None: number_headings = dict(_frontmatter_meta(src)).get("number_headings")
normalized, offsets = _normalize_offsets(src)
from . import rewrite
number_headings = _headnums(dict(_frontmatter_meta(src)), number_headings)
imgbase = Path(dest).parent if dest is not None else Path(".")
ex = _GfmExporter(reftypes, number_headings, math, implicit_figures, templates, tmpl,
raw=raw, imgdir=None if imgdir is None else Path(imgdir), imgbase=imgbase, link=link)
raw=raw, imgdir=None if imgdir is None else Path(imgdir), imgbase=imgbase)
if link is not None or imgdir is not None:
def url(node):
old = node['url']
if node['type'] == 'image' and imgdir is not None and old.startswith('data:') and ';base64,' in old:
new = ex._extract_img(old)
else: new = link(old) if link is not None else None
return {'url': new} if new is not None else None
src = rewrite(src, {'link': url, 'image': url}, math=math, templates=templates)
normalized, offsets = _normalize_offsets(src)
edits = ex.run(normalized)
for start, end, repl in reversed(edits): src = src[:offsets[start]] + repl + src[offsets[end]:]
res = Md(src, ex.warnings)
Expand Down
2 changes: 1 addition & 1 deletion python/mdhtml/md2html.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

RefsMode = str_enum('RefsMode', 'ids', 'lenient', 'resolve')
HlMode = str_enum('HlMode', 'spans', 'api', 'off')
NumMode = str_enum('NumMode', 'legal', 'decimal')
NumMode = str_enum('NumMode', 'legal', 'decimal', 'false')
KATEX = "https://cdn.jsdelivr.net/npm/katex@0.16.22/dist"
MERMAID = "https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs"
CACHE = Path.home() / ".cache" / "md2html"
Expand Down
11 changes: 7 additions & 4 deletions python/mdhtml/typst.py
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,9 @@ def _ref(self, tgt, tokens, override, prefix=True, plural=False):
if self.kinds[tgt] == "block":
if tgt not in self.headids:
raise ValueError(f"cross-reference #{tgt} targets a paragraph; only {{ref=text}} can render it")
self.need_nums = True
if variant != "page":
if self.number_headings is False: self.core(tgt, tokens) # shared error for a numeric reference to an unnumbered target
self.need_nums = True
args = []
if variant == "page":
args.append('form: "page"')
Expand Down Expand Up @@ -323,15 +325,16 @@ def mdhtml2typst(src, dest=None, reftypes: dict | None = None, number_headings=N
`None` drops them). `table_styles` maps a table's `custom-style` name or class (matched in that
order, case-insensitively) to extra Typst table arguments, e.g. `{'borderless table': 'stroke: none'}`.
`prelude` text is prepended before the generated setup. `number_headings=None` takes the
scheme from the source's frontmatter `number_headings:` when `src` is `md2mdhtml`'s result.
scheme from the input's metadata; `False` disables heading numbering.
Returns a `Typst` str carrying `.warnings`; `dest` also writes it to a file."""
number_headings = _headnums(src, number_headings)
number_headings = _headnums(getattr(src, 'meta', {}), number_headings)
if not isinstance(src, str): src = src.to_html()
ex = _TypstExporter(reftypes, number_headings, tmpl, table_styles)
body = ex.run(mdhtml2dom(src))
parts = [prelude.rstrip()] if prelude else []
if ex.has_math: parts.append(f'#import "{MITEX}": mi, mitex')
if number_headings is not None or ex.need_nums: parts.append(_numbering_code(number_headings or "decimal"))
if number_headings is False: parts.append('#set heading(numbering: none)')
elif number_headings is not None or ex.need_nums: parts.append(_numbering_code(number_headings or "decimal"))
if ex.need_page_nums: parts.append('#set page(numbering: "1")')
res = Typst("\n".join(parts) + ("\n\n" if parts else "") + body + "\n", ex.warnings)
if dest is not None: Path(dest).write_text(res, encoding="utf-8")
Expand Down
3 changes: 2 additions & 1 deletion src/block.rs
Original file line number Diff line number Diff line change
Expand Up @@ -864,7 +864,8 @@ fn edit_nodes_for_regions(source: &Source<'_>, regions: &[(usize, usize, RegionK
}
}
out.sort_by_key(|node| match node {
EditNode::Image { range, .. } | EditNode::Link { range, .. }
EditNode::Image { range, .. }
| EditNode::Link { range, .. }
| EditNode::Math { range, .. }
| EditNode::Xref { range, .. }
| EditNode::Attrs { range, .. }
Expand Down
Loading
Loading