pre-render script output (stdout) is ignored on Windows with Quarto v1.7.32

[BunjiRo-creator]

Description

Hi everyone,

I'm encountering an issue where the standard output of a pre-render script seems to be completely ignored by Quarto. My goal is to dynamically modify the book.chapters list before rendering, but the changes are not reflected in the final output.
This issue occurs even with a minimal reproducible example where the script simply prints a hardcoded JSON string to stdout.
Environment:
Quarto Version: 1.7.32 (pre-release)
OS: Windows 11
Minimal Reproducible Example (MRE)
Here are the exact files to reproduce the issue.

1. _quarto.yml

project:
  type: book

book:
  title: "Test Book"
  chapters:
    - index.qmd

pre-render:
  - run-lua.bat

2. run-lua.bat
   (A batch script to execute the Lua script with Quarto's engine)

@echo off
quarto run "%~dp0test-script.lua"

3. test-script.lua
   (This script completely ignores any input from Quarto and just prints a hardcoded JSON string to stdout.)

-- test-script.lua (Fixed JSON output version)
-- The goal is to confirm if Quarto reads stdout from a pre-render script.

local hardcoded_json_output = [[
{
  "project": {
    "type": "book"
  },
  "book": {
    "title": "SUCCESS: Output was Captured!",
    "chapters": [
      "index.qmd",
      "another.qmd"
    ]
  },
  "pre-render": [
    "run-lua.bat"
  ]
}
]]

print(hardcoded_json_output)

4. index.qmd

---
title: "Index"
---

This is the index page.

5. another.qmd

---
title: "Another Page"
---

This page should be included if the pre-render script works.

Steps to Reproduce

Create a new directory with the 5 files listed above.
Run quarto render in the terminal from that directory.

Expected Behavior

The rendered book in _book/index.html should have the title "SUCCESS: Output was Captured!" and the table of contents should include "Another Page".

Actual Behavior

The render completes successfully without any errors. However, the book title remains "Test Book" and "Another Page" is not included in the output. The stdout from the pre-render script is completely ignored.

What I've Ruled Out

This is not a filter vs. pre-render script issue. The script is correctly specified under the pre-render key.
There are no other filters: keys in any .qmd or .yml files that could cause conflicts.
The issue persists even when using a batch file to explicitly run the Lua script, and when the script outputs a simple, hardcoded JSON object. This suggests the problem lies in how Quarto captures the stdout on this specific environment.

Given that I'm using a pre-release version, is this a known issue or a potential regression? Any insights would be greatly appreciated.
Thank you for your time and for creating this great tool.

[mcanouil]

Could you share a Git repository instead of asking to create one out of many copy/paste which is error prone.

[mcanouil]

Why are you calling quarto run?
Pre/post-render scripts are ran by Quarto already and it supports Lua.

https://quarto.org/docs/projects/scripts.html#pre-and-post-render

Also, the chapters are needed for Quarto to identify the structure of the project, so it happens even before pre-post render script.

Finally, if you want to change only the chapters, you need to create a script. Create a project profiles with different list of chapters.

https://quarto.org/docs/projects/profiles.html

with Quarto v1.7.32

Given that I'm using a pre-release version, is this a known issue or a potential regression?

1.7.32 is not a pre-release.

[cderv]

Also, the chapters are needed for Quarto to identify the structure of the project, so it happens even before pre-post render script.

AFAIK project's pre-render script can be used to modify the _quarto.yml as all the project configuration will be reloaded after pre-render script. This was added while working on Website Draft feature #8509 (5bfff40) in 1.5

Documented at https://quarto.org/docs/projects/scripts.html#pre-and-post-render

The project metadata and render list will be re-computed after any pre-render scripts have executed, allowing them to modify this project data. For example, a pre-render script might generate additional qmd files or ipynb files that should be rendered.

Adding this as I was not sure your answer covers that part.

[mcanouil]

For example, a pre-render script might generate additional qmd files or ipynb files that should be rendered.

That's not entirely true, I'm afraid.

If you have a document that includes another one not yet present because generated by pre-render, Quarto will crash. Project structure is determined based on current files before pre-render. So, it won't work in all cases.

AFAIK project's pre-render script can be used to modify the _quarto.yml as all the project configuration will be reloaded after pre-render script. This was added while working on Website Draft feature #8509 (5bfff40) in 1.5

I was not fully aware of this. I'll have to try.
Also, not everything is mutable by pre/post-render scripts:

ERROR: Pre-render script resulted in a project change that is now allowed.
The project type may not be mutated by the pre-render script

[cderv]

If you have a document that includes another one not yet present because generated by pre-render, Quarto will crash.

Is this some specific case with includes shortcodes when using a shortcode that don't exist yet ?
Possible that this fails yes, it seems edge case.

What have been improve in 1.5 is the fact that project configuration are recomputed after pre-render, to make some usage possible.

And yes, some fields are immutable:

• Project type
• Project output dir
• Project root dir

But they are not related to the use case here of adding chapters.

[mcanouil]

For reference:

• Support modifying output files names in post render script #12987

[cderv]

I'm encountering an issue where the standard output of a pre-render script seems to be completely ignored by Quarto.

@BunjiRo-creator what is your expectaction with standard output of pre-render script ?

Our doc is here: https://quarto.org/docs/projects/scripts.html#pre-and-post-render and we don't say anything about stdout being processed in some ways.

So I wonder what you are expecting. From your example, it seems you are expecting the stdout to be written in _quarto.yml? Or be used as the new _quarto.yml?

Your pre-render script can modify the _quarto.yml as mentioned above (#13061 (reply in thread)), but your scripts needs to write to it.

[BunjiRo-creator]

Thank you all for the incredibly helpful discussion! Your insights were crucial in helping me understand the correct mechanism for pre-render scripts.
The core of my problem was a fundamental misunderstanding: I incorrectly assumed that pre-render scripts should pass data back to Quarto by printing JSON to stdout. As you clarified, the correct method is for the script to physically modify files on disk (like _quarto.yml or an included file), which Quarto then re-reads.
With this understanding, I was able to achieve my original goal: dynamically selecting a single chapter to render. I ended up using a Python script for this, and it now works perfectly.

Thank you all again for your time and help. This journey has been a fantastic learning experience about the inner workings of Quarto.