Custom marks don't play nice with `pytest_plugins = ('my_plugin',)` in test file

[petebman]

I've got a pytest plugin that registers a custom mark in the pytest_configure hook using config.addinivalue_line. When I load my plugin by putting pytest_plugins = ("my_plugin",) in my test file (as described here) and add the mark to one of my tests, I still get a warning PytestUnknownMarkWarning: Unknown pytest.mark.my_mark - is this a typo?

When I load my plugin by putting pytest_plugins = ("my_plugin",) in the conftest.py file, I do not get the warning.

Here's my example plugin:

import pytest

# This fixture is just so I know the plugin is working.
# If it's not installed correctly, I'll get an error when the test tries to use the foo fixture
@pytest.fixture
def foo():
  return "foo"

def pytest_configure(config):
    config.addinivalue_line("markers", "my_mark")

Here's the test file that triggers the warning:

import pytest


@pytest.mark.my_mark
def test_01(foo):
  assert foo == "foo"

Here's my environment

Package       Version
------------- -------
attrs         21.4.0
iniconfig     1.1.1
my-plugin     1.0.0
packaging     21.3
pip           20.3.4
pkg-resources 0.0.0
pluggy        1.0.0
py            1.11.0
pyparsing     3.0.7
pytest        7.0.1
setuptools    44.1.1
tomli         2.0.1

I'm aware that I can use a setuptools entrypoint to install my plugin, but that isn't an option in the more complicated environment where I encountered the problem. Likewise, the use of conftest.py to load the plugin is a bit frowned upon - we're trying declare the plugins we're using in the test files themselves so they're more stand-alone readable (among other reasons).

I suspect what's going wrong here is the warning is triggered during assertion re-writing in the test module. If I drop a breakpoint here I see the following near the end of the backtrace

. . .
.../_pytest/assertion/rewrite.py(170)exec_module()
-> exec(co, module.__dict__)
.../my_test.py(4)<module>()
-> @pytest.mark.my_mark
.../_pytest/mark/structures.py(520)__getattr__()
-> warnings.warn(

That happens before my pytest_configure hook is run, so it has no idea that my_mark is a custom mark yet.

Unfortunately, this is about the limit of my understanding so I can't be more helpful by trying stuff like "suppress this warning during assertion re-writing" to see if that fixes the problem. It's a bummer that this mark triggers a warning.

[nicoddemus]

Hi @petebman,

Yeah your assessment is correct, we are triggering the warning a bit too early (during import, which also triggers assertion rewriting).

We should delay emitting that warning, not sure at which point at the moment (would have to think about the performance implications of that).

[petebman]

Is there a way to know if pytest is currently in the "collection" phase? I can play around with some naive fixes if it's possible to get a hint. I doubt I have the knowledge to do a fix that involves moving major things around though.