Skip to content

Add Markdown importer to convert Markdown back into ContentState#155

Open
thibaudcolas wants to merge 1 commit into
mainfrom
markdown-importer
Open

Add Markdown importer to convert Markdown back into ContentState#155
thibaudcolas wants to merge 1 commit into
mainfrom
markdown-importer

Conversation

@thibaudcolas

Copy link
Copy Markdown
Member

Motivation

The exporter already supports Draft.js ContentState → Markdown via DOM.MARKDOWN and build_markdown_config(). This is a first implementation of the reverse direction: Markdown → ContentState, enabling round-trip workflows and server-side ingestion of Markdown content from external sources (file imports, API consumers, content migrations).

⚠️ This is an experimental first draft. The API is marked experimental in the docs and may change based on integrator feedback before being promoted to stable.

Description

Adds a new public function and options type, both re-exported from the top-level draftjs_exporter package:

from draftjs_exporter import markdown_to_content_state, MarkdownImporterOptions

content_state = markdown_to_content_state(markdown_text, options={"bold": "__"})

Scope of the parser

The importer is dependency-free — a custom character-by-character inline walker plus a regex-based block dispatcher. It supports the same subset of Markdown that the exporter produces, plus common CommonMark edge cases:

  • ATX headings (1-6), blockquotes (single- and multi-line), nested unordered/ordered lists (with ./) ordered list delimiters), fenced code blocks (``` and ~~~, with optional info strings), horizontal rules (---/***/___), images (![alt](src)), links ([text](url), with optional "title").
  • Inline styles: bold, italic, code, strikethrough. Custom protocols like wagtail://core.Page.89 round-trip unchanged (the importer does no URL scheme validation — that's the integrating application's responsibility, matching the exporter).
  • The 10 inline HTML tags the exporter emits for non-Markdown styles (<u>, <sup>, <sub>, <mark>, <q>, <small>, <samp>, <ins>, <del>, <kbd>) are parsed back into inlineStyleRanges.
  • Backslash escapes for all CommonMark punctuation. Nested links and styles within link text. Soft line breaks within block-level elements.

Options surface

MarkdownImporterOptions mirrors the three inline style marker options from the exporter's MarkdownOptions (bold, italic, strikethrough). Block-level variants (list markers, HR variants, fence variants, ordered-list delimiters) are not mirrored because the importer's regexes accept all forms polymorphically — any block-level keys passed through from MarkdownOptions are simply ignored. A fourth option, html_style_tags, extends the default HTML tag → style mapping and is the inverse of the exporter's STYLE_MAP.

Architectural notes

  • The importer is a peer module to the exporter (markdown/importer.py), not integrated into the rendering pipeline. It doesn't reuse the exporter's Command/EntityState/StyleState abstractions because those consume ranges; this produces them from a different source representation.
  • _merge_style_ranges is an intentional round-trip trade-off: it repairs the exporter's inline-style nesting quirks (e.g. **Bold ****_Italic_** producing two BOLD ranges) by merging adjacent or overlapping ranges of the same style.
  • Unrecognized block-level constructs fall through to unstyled at logger.debug (not warning) — plain paragraphs are the expected fall-through path, not an anomaly.

Documentation

New "Importer" section in docs/markdown.md covering quick start, options, fallback behavior, round-trip guarantees (what's preserved vs. lost), and supported Markdown features. New "Importer pipeline" section in docs/contributing/architecture.md documenting the design choices. New experimental entry in docs/ROADMAP.md.

Tests

  • tests/markdown/test_importer.py — 137 tests, including fixture-driven round-trip tests (export → import → export) and import correctness tests for every supported Markdown feature. Reuses the tests/test_exports.json fixture matrix with an explicit LOSSLESS_CASES allowlist (13 of 18 fixtures), acknowledging that Markdown is a lossy representation.
  • tests/strategies.py + tests/test_properties.py — a new TestImporterCrashSafety property test using Hypothesis that asserts markdown_to_content_state(md) returns well-formed ContentState on adversarial input (unterminated links, mismatched markers, malformed HTML tags, deep nesting). 11 pinned @example cases for known-tricky inputs.
  • pyproject.tomldisallow_untyped_calls = false override for tests.*, needed because the metaclass-generated test methods (ImporterTestMeta, RoundTripTestMeta) are untyped from mypy 2.0's perspective. Commented inline.

Test evidence

  • just lint — clean (ruff, mypy, ty).
  • just test — 563 tests passing (was 562; added the new TestImporterCrashSafety property test).
  • just test-coverage tests/markdown/test_importer.py — 100% coverage on importer.py (717 lines, 284 statements, 0 missing).

Tested on macOS, Python 3.14.5.


  • Stay on point and keep it small so it can be easily reviewed. For example, try to apply any general refactoring separately outside of the PR.
  • Consider adding unit tests, especially for bug fixes. If you don't, tell us why.
  • All new and existing tests pass, with 100% test coverage (just test-coverage).
  • Linting passes (just lint).
  • Consider updating documentation. If you don't, tell us why.
  • List the environments / platforms in which you tested your changes. macOS Python 3.14.5.

@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown

🔍 OpenCodeReview found 1 issue(s) in this PR.

  • ✅ Successfully posted inline: 1 comment(s)

Comment on lines +254 to +258
for quote_char in ('"', "'"):
if raw_url.endswith(quote_char) and " " + quote_char in raw_url:
space_pos = raw_url.rfind(" " + quote_char)
url = raw_url[:space_pos]
title = raw_url[space_pos + 2 : -1]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link title parsing is broken for titles with spaces. The logic uses raw_url.rfind(' ' + quote_char) which finds the LAST space before the closing quote. For input like url "Title With Spaces", this extracts url as the URL and Title With as the title (the Spaces" part is cut off incorrectly). Additionally, the condition " " + quote_char in raw_url doesn't properly handle titles with multiple spaces.

Suggestion:

Suggested change
for quote_char in ('"', "'"):
if raw_url.endswith(quote_char) and " " + quote_char in raw_url:
space_pos = raw_url.rfind(" " + quote_char)
url = raw_url[:space_pos]
title = raw_url[space_pos + 2 : -1]
Implement a proper parser: find the first space from the right that separates the URL from the title. A more robust approach is to find the opening quote position, then extract everything between the first quote and the last matching quote character, treating the URL as everything before the first space+quote.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant