Better handling of errors in backends

[pfmoore]

If a backend raises an unexpected exception, the subprocess being run by the wrapper will exit with an error, which will get reported back to the caller, but it's not as informative as it could be. We should look into capturing the actual error and traceback, and re-raising it in the calling process.

This is a particular issue when debugging pip issues, as it's hard to locate or identify what the backend might be doing, with all the layers of capturing subprocess output involved (particularly in pip's CI, where tox and pytest add extra layers).

Just noting this here for now, as I don't have any immediate ideas on how to do this, and while it's frustrating while I'm developing the PEP 517 interface for pip, it's not exactly an urgent issue.

[jcfr]

Ideally, the reported errors would also indicate which command to run, with which options and given which environment variables.

When an option like --no-clean is provided, we could even think about having an activate script allowing to enable the isolated environment and provide way to reproduce and debug problems.

[pfmoore]

I'm not sure I follow. Are you talking about what pip does? Because if so, that's not really relevant here.

[jcfr]

My original intent was to indicate that build backend error should be reported to the user in the most useful way possible.

Are you talking about what pip does? Because if so, that's not really relevant here.

And to illustrate the idea, I referenced a command used in pip and discussed how its implementation could be improved. I will move that specific discussion to the relevant issue tracker. Sorry for the noise.

[pfmoore]

Not a problem. I wanted to make sure you didn't have a specific suggestion for something that should be done in this project.

Note, however, that this issue is about totally general errors that may be raised by a backend (for example, if there's a code bug in the backend). Many times, there simply isn't a relevant end user action to be taken, apart from reporting the bug - and a raw traceback is typically sufficient for a developer to identify the issue and address it (maybe by identifying a user error that was the root cause, and providing a better error message for that case - or maybe just by fixing the bug).

The pip problem I was referring to is that in pip's test suite, there are so many layers of stderr capturing, that the traceback from the process running the backend gets lost - if the wrapper can re-raise the exception, the error gets raised one layer further up, which might make it more likely to be visible in the pip test suite output.

I genuinely doubt that a change here is likely to ever have a perceptible benefit for the end user - it's far too low level for that.

[jaraco]

setuptools has had some experience with this issue. Instead of invoking its builds in a subprocess, it instead used attempted to use an in-process context-manager sandbox to save everything, run the command, and then restore everything, while still retaining the exception (using pickle because since sys.modules was saved, the exception might be a different). This process ends up being brittle, as it makes assumptions about the compatibility of the objects in the two contexts (frequently the exceptions aren't unpickleable).

If you try to do that across a subprocess boundary, you'll run into similar problems. The subprocess will have exceptions that don't exist in the parent process. Whatever is passed between the two will most-likely need to be opaque, simple objects (strings, lists, or other things that can be readily serialized).

[vkottler]

Concrete example: pypa/setuptools#3845

The build is working hard on find_namespace_packages, but if you cancel the foreground build process you can discern that the build failed on some kind of sub-process call at best.

If the implementation is going to rely on subprocesses, we probably want caught signals to be passed on to those spawned processes or something?