mutilated pytest stack traces

[kkroening]

On Python 2.7, stack traces are garbled when using the fs pytest fixture:

test.py:

def test(fs):
    assert 0

bash:

$ virtualenv venv --no-site-packages
$ . venv/bin/activate
(venv) $ pip install pytest pyfakefs
(venv) $ pytest test.py

Output:

_____________________________________ test ______________________________________

fs = <pyfakefs.fake_filesystem.FakeFilesystem object at 0x10efdbc50>

>   ???
E   assert 0

test.py:2: AssertionError

Output without including fs fixture:

_____________________________________ test ______________________________________

    def test():
>       assert 0
E       assert 0

test.py:2: AssertionError

Any ideas? Sorry if this has already been reported somewhere (I couldn't find an issue for it).

Python version: 2.7.14
pip version: 10.0.1
virtualenv version: 15.1.0
pip freeze output:

attrs==17.4.0
funcsigs==1.0.2
more-itertools==4.1.0
pluggy==0.6.0
py==1.5.3
pyfakefs==3.4.1
pytest==3.5.1
six==1.11.0

Thanks,
Karl

[skykit-karlkroening]

I love what you've done with this library though, btw. As far as I'm concerned, this is the correct way to do file faking in python (aside from this broken stacktrace issue). Nice work 👍

[mrbean-bremen]

There are other bugs lurking there for sure ;)
I will have a look at this in the evening or at the weekend. The pytest fixture has been added by a contributor, and both of us had not much to do with pytest before - so we haven't looked into it too closely until now.

[mrbean-bremen]

I had a look, and can confirm that the problem is there since version 3.1 (when the pytest plugin has been added), and is somehow related to the setup of the Patcher. I didn't succeed so far in tracking down the reason for this, and can't promise anything soon - I'm not familiar with the intricacies of pytest - but I will check into this further.

[mrbean-bremen]

I seem to have found a solution for this under Python 3 (happened there too), but under Python 2 it still does not work. Not faking the open function fixes it, but that is not an option...

[kkroening]

Thank you for looking into this. Let me know if there's anything I can do to help

[mrbean-bremen]

Thanks - not sure how you could help, but I can explain the problem as I understand it.
The buildin linecache module is used by pytest to read the contents of the test file in case of a failed test to generate the stack trace. This happens before the tearDown of the test, so the file system modules are still stubbed out by the fake modules, thus preventing reading the correct file.
In Python 3, it is sufficient to add linecache to the skipped modules (which I have done), because it only uses os functions there. In Python 2, however, the builtin open is used for opening the file, and that cannot be fixed the same way, because __builtin__ is not explicitely imported (other than os and the rest of the patched modules).
I don't know how to fix this yet - maybe @jmcgeheeiv has an idea, as he knows the patching code best (he wrote it).

[jmcgeheeiv]

Thank you for the great triage and the Python 3 fix, @mrbean-bremen.

I will give it a try, but like @mrbean-bremen, I am not so experienced with pytest.

[mrbean-bremen]

@jmcgeheeiv - I did some reading on pytest configuration and found that this can be fixed by providing a hook function (pytest_pyfunc_call) that does the Patcher tear down. I would prefer such a solution, as it does not need any changes in pyfakefs itself. What do you think?

[jmcgeheeiv]

Well done, @mrbean-bremen.

@agosto-karlkroening, could we ask you to try this in your pytest environment and share this with us in the documentation? Just get the content in there, submit a pull request, assign it to me and I will be your tech writer.

[mrbean-bremen]

Let me make sure first that I didn't make a mistake and this really works as expected - I will have another look probably tomorrow evening.

[mrbean-bremen]

Sorry, I take it all back - I made a dumb mistake yesterday (wasn't my working temperature, obviously...).
I still think that a pytest hook would be the best way to fix this, if possible - I will check again in the evening.

[skykit-karlkroening]

@agosto-karlkroening, could we ask you to try this in your pytest environment and share this with us in the documentation?

I'm not sure I understand the proposed solution. I'm not familiar with pytest_pyfunc_call and not sure how it would interact with pyfakefs (and vice versa).

I did try pulling the latest pyfakefs and running the steps listed at the top of this issue:

pip install git+https://github.com/jmcgeheeiv/pyfakefs.git
pytest test.py

The stacktrace issue seems to be fixed for Python 3.6, but still lurking in Python 2.7.

[mrbean-bremen]

@agosto-karlkroening - sorry, as I wrote, I made a dumb mistake while testing my changes and got overexcited. You can safely ignore the last posts.
There seems to be no pytest hook that gets called at the right time, at least I didn't find one. I would have preferred such a solution, but it looks like we are out of luck here. I may think of another solution, or hopefully @jmcgeheeiv will have a better idea.

[mrbean-bremen]

Ok, I found a relatively simple (if a bit hacky) solution that works in most tested versions (except Python 3.3). Will add a PR shortly.

[mrbean-bremen]

@kkroening - can you please check if the latest master works for you now?

[skykit-karlkroening]

Looks like it's fixed!

I had to install directly from the git repo rather than through pip/pypi, but it seems to work on both py2 and py3.

Nice work! Thank you!

[mrbean-bremen]

I had to install directly from the git repo rather than through pip/pypi

Yes, we didn't make a new pypi release yet, though I think another minor release with the accumulated bug fixes would make sense. @jmcgeheeiv, what do you think?

Closing the issue, I think we can ignore Python 3.3 now that it's eol.

[jmcgeheeiv]

Well done, @mrbean-bremen. I suppose it is about time for a release.

[mrbean-bremen]

Ok, done.
@jmcgeheeiv - I made a mistake (released from the wrong virtual enviroment which didn't have the latest twine version) and corrected it in a new version (3.4.3) , so that version 3.4.2 is now obsolete - you may want to delete it on PyPi.