fix(ipynb): label code fences with the notebook's own language - #2443
Open
kevin (kevin9327) wants to merge 1 commit into
Open
fix(ipynb): label code fences with the notebook's own language#2443kevin (kevin9327) wants to merge 1 commit into
kevin (kevin9327) wants to merge 1 commit into
Conversation
Code cells were always fenced as ```python, so an R, Julia or C# notebook came out claiming to be Python. Read the language nbformat records in metadata.language_info.name, falling back to the kernel's own metadata.kernelspec.language and then to python.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Converting an R notebook produces
as
The same happens to Julia, Scala, C#, Bash and every other kernel:
IpynbConverter._convertwrites the fence with a hard-codedpythoninfo string, whatever the notebook says it is. The output then tells a reader — and any downstream renderer or model — that the code is Python.nbformat records the notebook's language in
metadata.language_info.name, which is the field nbconvert reads;metadata.kernelspec.languageis the kernel's own declaration and is all some producers write. This reads the first, falls back to the second, and falls back topythonwhen a notebook declares neither, which is what the converter has always emitted.Only the first word of the declaration is used, since a fence's info string ends at the first whitespace.
Reproduction
New test file, on unmodified
main(packages/markitdown):with the failure showing the wrong label:
The two tests that pass before and after are the ones that pin the existing behaviour: a notebook declaring
python, and a notebook declaring no language at all, both still getpython.With the fix:
Verification
tests/test_module_vectors.pyincludes the end-to-endtest_notebook.ipynbvector, which requires```python— that notebook declareslanguage_info.name: "python", so it keeps its label.(The three deselected tests fail on
mainon this Windows machine for reasons unrelated to notebooks: console encoding, file-URI paths, and a missingffmpeg.)blackreports both touched files unchanged.