Three tests fails with PYTEST_PLUGINS set to _hypothesis_pytestplugin

[mtelka]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

I tried to run pytest tests in the main git branch with the following change in tox.ini:

diff --git a/tox.ini b/tox.ini
index 61563ca2c..bc08595c3 100644
--- a/tox.ini
+++ b/tox.ini
@@ -51,6 +51,8 @@ passenv =
     TERM
     SETUPTOOLS_SCM_PRETEND_VERSION_FOR_PYTEST
 setenv =
+    PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
+    PYTEST_PLUGINS=_hypothesis_pytestplugin
     _PYTEST_TOX_DEFAULT_POSARGS={env:_PYTEST_TOX_POSARGS_DOCTESTING:} {env:_PYTEST_TOX_POSARGS_LSOF:} {env:_PYTEST_TOX_POSARGS_XDIST:}
 
     # See https://docs.python.org/3/library/io.html#io-encoding-warning

and I found that the following three tests failed:

=========================== short test summary info ============================
FAILED testing/test_helpconfig.py::test_version_verbose - assert <ExitCode.IN...
FAILED testing/test_helpconfig.py::test_version_less_verbose - assert <ExitCo...
FAILED testing/test_terminal.py::TestTerminalFunctional::test_header_trailer_info
XPASS testing/_py/test_local.py::TestLocalPath::test_make_numbered_dir_multiprocess_safe - #11603
= 3 failed, 3609 passed, 112 skipped, 11 xfailed, 1 xpassed in 312.55s (0:05:12) =

All three tests fails with the following error:

ValueError: Plugin already registered under a different name: _hypothesis_pytestplugin=<module '_hypothesis_pytestplugin' from '/data/builds/pytest/.tox/py39/lib/python3.9/site-packages/_hypothesis_pytestplugin.py'>

Probably (untested) we need to add monkeypatch.delenv("PYTEST_PLUGINS", raising=False) call to affected functions.

This seems to be related: #12616 (comment)