Render Quarto to Markdown remove author information from YAML header

[rgaiacs]

Bug description

If creator provides a valid author metadata following Author Schema and convert the .qmd file to Markdown, some information in the author metadata is discarted.

Steps to reproduce

Consider the index.qmd file:

---
title: "MWE"
author:
  - name:
      given: Jane
      family: Doe
    email: jane@mail.com
    orcid: 0000
    attributes:
      corresponding: true
  - name:
      given: Jonh
      family: Doe
    orcid: 0000
---

Lorem ipsum

and run quarto render index.qmd --to markdown --output rendered_index.md.

Expected behavior

Output is identical to the input:

---
title: "MWE"
author:
  - name:
      given: Jane
      family: Doe
    email: jane@mail.com
    orcid: 0000
    attributes:
      corresponding: true
  - name:
      given: Jonh
      family: Doe
    orcid: 0000
---

Lorem ipsum

This is what happen when using Hugo format. quarto render index.qmd --to hugo-md --output rendered_index.hugo && diff index.qmd rendered_index.hugo produces

17c17
< Lorem ipsum
\ No newline at end of file
---
> Lorem ipsum

Actual behavior

Information in the YAML header is lost:

---
author:
- family: Doe
  given: Jane
- family: Doe
  given: Jonh
title: MWE
toc-title: Table of contents
---

Lorem ipsum

Your environment

• OS: Linux 6.2.12-1-MANJARO

Quarto check output

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.2: OK
      Dart Sass version 1.55.0: OK
      Deno version 1.33.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.176
      Path: /opt/quarto-1.4.176/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.10.10
      Path: /usr/bin/python3
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with python3 -m pip install jupyter

[✓] Checking R installation...........(None)

      Unable to locate an installed version of R.
      Install R from https://cloud.r-project.org/

[cderv]

@dragonstyle I believe it requires to ask explicitly for keeping YAML

---
title: "MWE"
author:
  - name:
      given: Jane
      family: Doe
    email: jane@mail.com
    orcid: 0000
    attributes:
      corresponding: true
  - name:
      given: Jonh
      family: Doe
    orcid: 0000
format: markdown+yaml_metadata_block
---

Lorem ipsum

It kind of works when we add the format extensions as above - we still have the format key present.

Not sure why only author was there though...

Anyway, note that this is a topic that had some changes over time. See the following

See #4210 (comment) discussing the keep-yaml removal (to use the extension with + only.

quarto-cli/src/command/render/output.ts

Lines 128 to 138 in a7f26f0

	// we implement +yaml_metadata_block internally to prevent
	// gunk from the quarto rendering pipeline from showing up
	const kYamlMetadataBlock = "yaml_metadata_block";
	if (recipe.format.pandoc.to?.includes(`+${kYamlMetadataBlock}`)) {
	recipe.format.pandoc.to = recipe.format.pandoc.to.replaceAll(
	`+${kYamlMetadataBlock}`,
	"",
	);
	recipe.keepYaml = true;
	}
	}

But then there is an internal option reintroduced to fix a bug : See #5087

So if you want to tackle the topic, this context should be taken into account IMO. Otherwise, we still go on and on with this topics and our various formats

[rgaiacs]

@cderv Thanks for the reply. Append the markdown format with +yaml_metadata_block will preserve the YAML header intact. A side effect is that no key-value pair can be added to the YAML header using --metadata. A minimal working example:

Input file (index.qmd):

---
title: "MWE"
author:
  - name:
      given: Jane
      family: Doe
    email: jane@mail.com
    orcid: 0000
    attributes:
      corresponding: true
  - name:
      given: Jonh
      family: Doe
    orcid: 0000
---

Lorem ipsum

Output of quarto render mwe.qmd --to markdown --metadata="foo:bar" (key-value pair added to YAML header as expected):

---
author:
- family: Doe
  given: Jane
- family: Doe
  given: Jonh
foo: bar
title: MWE
toc-title: Table of contents
---

Lorem ipsum

Output of quarto render mwe.qmd --to markdown+yaml_metadata_block --metadata="foo:bar" (YAML header intact):

---
title: "MWE"
author:
  - name:
      given: Jane
      family: Doe
    email: jane@mail.com
    orcid: 0000
    attributes:
      corresponding: true
  - name:
      given: Jonh
      family: Doe
    orcid: 0000
---



Lorem ipsum

[cderv]

This is definitely curious 🤔 Thanks for the addition.

Indeed Pandoc will keep all meta passed in the output when +yaml_metadata_block is used.

quarto pandoc -t markdown --metadata="foo:bar" mwe.qmd 

So it seems we are loosing the feature, and we do differently by default, and with +yaml_medata_block.
We definitely have improvement to do on our Markdown output.

Though, just a head up that this could be a limitation by design as we use internal metadata a lot in Quarto and we don't want them to be included in output. That is why we reimplement +yaml_metadata_block differently. The implementation is to take the YAML block from the input and write it to the output. Which is what you see.

quarto-cli/src/command/render/output.ts

Lines 141 to 160 in a7f26f0

	if (recipe.keepYaml) {
	completeActions.push(() => {
	// read yaml and output markdown
	const inputMd = partitionYamlFrontMatter(context.target.markdown.value);
	if (inputMd) {
	const outputFile = isAbsolute(recipe.output)
	? recipe.output
	: join(dirname(context.target.input), recipe.output);
	const output = Deno.readTextFileSync(outputFile);
	const outputMd = partitionYamlFrontMatter(
	Deno.readTextFileSync(outputFile),
	);
	const markdown = outputMd?.markdown || output;
	Deno.writeTextFileSync(
	outputFile,
	inputMd.yaml + "\n\n" + markdown,
	);
	}
	});
	}

No custom metadata appended to it. This would require us to identify user provided meta vs quarto internal meta I guess. 🤔

We'll have a closer look into all this