Errors when compiling from another directory

[jperon]

Here's an example: if I am in the /home/user folder, then compile the gabc. tex document that is in /tmp/test: lualatex -shell-escape -output-directory=/tmp/test /tmp/test/gabc.tex

• if this document contains a command \gregorioscore, I get the error The file gabc-score.gabc does not exist;
• if it contains a command \gabcsnippet, the message obtained is error: kpse prohibits write to file /tmp/test/gabc.gtmp.

I could of course chdir into the document's folder, but I encounter this problem in trying to develop a filter for pandoc, and pandoc doesn't chdir before calling lualatex.

Here is a MWE to test:

\documentclass{article}
\usepackage[autocompile]{gregoriotex}
\begin{document}
\gabcsnippet{(c4) A(f)ve.(c)}
\gregorioscore{gabc-score}
\end{document}

[rpspringuel]

The first error sounds like a conflict between the working and relative file path. GregorioTeX assumes that relative file paths are relative to the working directory and that this is the same as the directory of the file being compiled. I don't know if this assumption can be broken (it depends on how it is linked to the other error). The second error is a kpathsea error and is thus related to file security. Kpathsea won't let you write to arbitrary directories in order to prevent a malicious document from making unintended changes to your system. We can't change that, but you might be able to disable that security restriction with an option to the luatex call or an appropriate conf file. ✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝ Br. Samuel, OSB (R. Padraic Springuel) PAX ☧ ΧΡΙΣΤΟΣ

…

On Dec 4, 2017, at 12:00 PM, jperon ***@***.***> wrote: Here's an example: if I am in the /home/user folder, then compile the gabc. tex document that is in /tmp/test: lualatex -shell-escape -output-directory=/tmp/test /tmp/test/gabc.tex if this document contains a command \gregorioscore, I get the error The file gabc-score.gabc does not exist; if it contains a command \gabcsnippet, the message obtained is error: kpse prohibits write to file /tmp/test/gabc.gtmp. I could of course chdir into the document's folder, but I encounter this problem in trying to develop a filter for pandoc, and pandoc doesn't chdir before calling lualatex. Here is a MWE to test: \documentclass{article} \usepackage[autocompile]{gregoriotex} \begin{document} \gabcsnippet{(c4) A(f)ve.(c)} \gregorioscore{gabc-score} \end{document} — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[jperon]

I can confirm that, when setting the environment variable openout_any=a, the first error disappears ; the second doesn't with the MWE, but as pandoc "stays" in the original folder, it finds the score. So as is, it works.

Just in case ; wouldn't it be possible to detect when kpse prohibits to write to output-directory, and then default to a subfolder of current directory (that would be writeable) to avoid failing compilation ?

[rpspringuel]

I would have expected setting `openout_any` to eliminate the second error, not the first. My understanding of what is going on my be flawed. ✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝ Br. Samuel, OSB (R. Padraic Springuel) PAX ☧ ΧΡΙΣΤΟΣ

…

On Dec 4, 2017, at 3:28 PM, jperon ***@***.***> wrote: I can confirm that, when setting the environment variable openout_any=a, the first error disappears ; the second doesn't with the MWE, but as pandoc "stays" in the original folder, it finds the score. So as is, it works. Just in case ; wouldn't it be possible to detect when kpse prohibits to write to output-directory, and then default to a subfolder of current directory (that would be writeable) to avoid failing compilation ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[jperon]

My mistake, excuse me : the second error disappeared, not the first. You understood well !

[rpspringuel]

In that case, I think the problem with writing to the CWD when kpathsea finds output-dir to be not writable is that we'd basically be ignoring the output-dir argument, something the user probably won't like (though obviously there are exceptions). Perhaps a suitable error/warning combined with an option to prioritize compilation over respecting the output-dir could overcome this objection. ✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝ Br. Samuel, OSB (R. Padraic Springuel) PAX ☧ ΧΡΙΣΤΟΣ

…

On Dec 4, 2017, at 5:48 PM, jperon ***@***.***> wrote: My mistake, excuse me : the second error disappeared, not the first. You understood well ! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[jperon]

I think so ; on the other hand, shouldn't this behaviour be considered as a Kpathsea bug ? The output directory isn't an arbitrary one, as the user explicitly indicates it as a target. In that case, I could report it upstream (even if I don't know where).

[rpspringuel]

Well, if it is a kpathsea or luatex bug the one should be able to reproduce it without using GregorioTeX. I don't have time to dig into this, but if you can reproduce the bug with the underlying file writing functions then I'm happy to help with the upstream reporting. ✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝ Br. Samuel, OSB (R. Padraic Springuel) PAX ☧ ΧΡΙΣΤΟΣ

…

On Dec 5, 2017, at 2:47 AM, jperon ***@***.***> wrote: I think so ; on the other hand, shouldn't this behaviour be considered as a Kpathsea bug ? The output directory isn't an arbitrary one, as the user explicitly indicates it as a target. In that case, I could report it upstream (even if I don't know where). — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[jperon]

The problem doesn't occur while writing directly from lua code.

Now I remember the problem had already occured when \gabcsnippet was introduced, hence PR #494. But since the compilation of gabc scores now makes use of temporary files (.gaux), the process was reverted.

So I think the real problem comes from how gregorio detects wether kpathsea allows writing or not (probably the kpathsea_out_name_ok_silent test in src/support.c line 151). Then the problem would come from source/texk/kpathsea/tex-file.c in luatex source ; but with my knowledge in C, I can't tell anything more…

[rpspringuel]

I've come across this error in some of my own work and that has spurred me to think about a possible solution.

One thing I've come up with is the possibility of creating a GregorioTeX specific working directory in a place which kpathsea does permit writing (like the current directory). All gtex and glog files would go into this folder. We could also use it for snippets if we ever implement the feature to keep gtex files for debugging (#1206).

Does anyone have any thoughts on this idea?

[Nokel81]

I have found that this is a problem when trying to use the subfiles package.

[ghost]

I have the same problem. I have also problems adding the openout_any=a variable. I tried adding it to the LuaLaTeX call and also to the current shell. Neither works.
I have 290 files in the directory. After compiling, this will be 870 files. For obvious reasons, I don't want to move them to the cwd.
The protection from kpse is a nanny feature and shouldn't exist. But this is my own opinion about such things. A knife is also safer, when it is not sharp.

I couldn't figure out, how to add the variable, but I suppose, that someone of you knows, how to do it.

[Josh-Nehnevaj]

I'm having the kpse error when trying to access a .gabc file in a separate directory. I cannot figure how to modify the environment variables and add openout_any=a. I feel like I'm missing some very basic step to modify the environment variable, but I can just not figure it out. Any help you can offer is greatly appreciated.

[rpspringuel]

@Josh-Nehnevaj, please try #1543. That change should negate this problem.

[Josh-Nehnevaj]

@rpspringuel , That made some progress, in that I'm no loner getting the kpse error, but now the only thing that renders is the horizontal lines for the staff and the clef, a line below where it should be. When I place a copy of the .gabc file in the same folder as the .tex file everything appears as it should.

[rpspringuel]

That doesn't make sense. Can you create a simple project with the appropriate directory structure to demonstrate the error, zip it up, and upload it here.

[Josh-Nehnevaj]

This still makes no sense to me, but it looks like I was missing a ";" in my header. I was having trouble recreating the problem in a test folder unless I used the exact same files from the main project. When I added a ";" at the end of the final header line, it solved the problem in both the main project and the test project. Thanks for your patience.