pyfakefs 5.3.4 + moto 4.2.13, test failed

[gnought]

The test code:

import os
from pathlib import Path

import boto3
import botocore
import cryptography
import pytest
from moto import mock_s3
from pyfakefs.fake_filesystem import FakeFilesystem


@pytest.fixture(scope="session", autouse=True)
def setup_tmpfs(fs_session: FakeFilesystem, request: pytest.FixtureRequest):
    fs_session.create_file("/zero-file")
    # compatible with moto
    for module in [boto3, botocore, cryptography]:
        module_dir = Path(module.__file__).parent
        fs_session.add_real_directory(module_dir, lazy_read=False)


@pytest.fixture
def mocked_s3():
    os.environ["AWS_ACCESS_KEY_ID"] = "testing"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "testing"
    os.environ["AWS_SECURITY_TOKEN"] = "testing"
    os.environ["AWS_SESSION_TOKEN"] = "testing"
    os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
    with mock_s3():
        client = boto3.client("s3")
        client.create_bucket(Bucket="test-bucket")
        yield client


@pytest.mark.parametrize("file", [(Path("/zero-file")), (Path("/zero-file"))])
def test_aws_s3_upload_param(mocked_s3, file):
    assert file.name in os.listdir("/"), f"{file} not exist"
    assert file.exists() is True, f"{file} not exist"
    mocked_s3.upload_file(file, "test-bucket", "s3-zero-file")

The above works flawlessly using python 3.11.7 + pyfakefs 5.3.2 + moto 4.2.13

% pytest -s test_todo.py -v
================================================================ test session starts =================================================================
platform darwin -- Python 3.11.7, pytest-7.4.4, pluggy-1.3.0 -- /dev/.venv/bin/python
cachedir: .pytest_cache
rootdir: /dev
configfile: pyproject.toml
plugins: httpserver-1.0.8, timeout-2.2.0, asyncio-0.23.3, pyfakefs-5.3.2, time-0.3.1
asyncio: mode=Mode.STRICT
collected 2 items
run-last-failure: no previously failed tests, not deselecting items.

test_todo.py::test_aws_s3_upload_param[file0] PASSED
test_todo.py::test_aws_s3_upload_param[file1] PASSED

However when upgrading pyfakefs to 5.3.4 and keeping other dependencies at the same versions, the 2nd parametrized test gives a error that it cannot find the /zero-file at the upload_file statement.

% pytest -s test_todo.py -v
================================================================ test session starts =================================================================
platform darwin -- Python 3.11.7, pytest-7.4.4, pluggy-1.3.0 -- /dev/.venv/bin/python
cachedir: .pytest_cache
rootdir: /dev
configfile: pyproject.toml
plugins: httpserver-1.0.8, timeout-2.2.0, asyncio-0.23.3, pyfakefs-5.3.4, time-0.3.1
asyncio: mode=Mode.STRICT
collected 2 items
run-last-failure: no previously failed tests, not deselecting items.

test_todo.py::test_aws_s3_upload_param[file0] PASSED
test_todo.py::test_aws_s3_upload_param[file1] FAILED

====================================================================== FAILURES ======================================================================
__________________________________________________________ test_aws_s3_upload_param[file1] ___________________________________________________________

mocked_s3 = <botocore.client.S3 object at 0x112bf3ed0>, file = PosixPath('/zero-file')

    @pytest.mark.parametrize("file", [(Path("/zero-file")), (Path("/zero-file"))])
    def test_aws_s3_upload_param(mocked_s3, file):
        assert file.name in os.listdir("/"), f"{file} not exist"
        assert file.exists() is True, f"{file} not exist"
>       mocked_s3.upload_file(file, "test-bucket", "s3-zero-file")

/dev/t/test_todo.py:89:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/dev/.venv/lib/python3.11/site-packages/boto3/s3/inject.py:145: in upload_file
    return transfer.upload_file(
/dev/.venv/lib/python3.11/site-packages/boto3/s3/transfer.py:371: in upload_file
    future.result()
/dev/.venv/lib/python3.11/site-packages/s3transfer/futures.py:103: in result
    return self._coordinator.result()
/dev/.venv/lib/python3.11/site-packages/s3transfer/futures.py:266: in result
    raise self._exception
/dev/.venv/lib/python3.11/site-packages/s3transfer/tasks.py:269: in _main
    self._submit(transfer_future=transfer_future, **kwargs)
/dev/.venv/lib/python3.11/site-packages/s3transfer/upload.py:591: in _submit
    upload_input_manager.provide_transfer_size(transfer_future)
/dev/.venv/lib/python3.11/site-packages/s3transfer/upload.py:244: in provide_transfer_size
    self._osutil.get_file_size(transfer_future.meta.call_args.fileobj)
/dev/.venv/lib/python3.11/site-packages/s3transfer/utils.py:251: in get_file_size
    return os.path.getsize(filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

filename = '/zero-file'

>   ???
E   FileNotFoundError: [Errno 2] No such file or directory: '/zero-file'

<frozen genericpath>:50: FileNotFoundError
============================================================== short test summary info ===============================================================
FAILED test_todo.py::test_aws_s3_upload_param[file1] - FileNotFoundError: [Errno 2] No such file or directory: '/zero-file'
============================================================ 1 failed, 1 passed in 2.21s =============================================================

Does it relate to recent change on the use_dynamic_patch feature?
version-5.3.2...version-5.3.4

[mrbean-bremen]

Tnanks for the report - it is most likely related to the change. I will try to find a better solution, or at least a possibility to configure the behavior. Meanwhile you may have to pin the pyfakefs version.