summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat (limited to 'guide/pytest.html')
-rw-r--r--guide/pytest.html93
1 files changed, 50 insertions, 43 deletions
diff --git a/guide/pytest.html b/guide/pytest.html
index eaf96b1..c1446d6 100644
--- a/guide/pytest.html
+++ b/guide/pytest.html
@@ -104,6 +104,55 @@ tests. If this is the case, the dependency must be stripped.</p>
</pre></div>
</div>
</section>
+<section id="disabling-plugin-autoloading">
+<span id="index-1"></span><span id="index-0"></span><h2>Disabling plugin autoloading<a class="headerlink" href="#disabling-plugin-autoloading" title="Link to this heading">¶</a></h2>
+<p>Normally, when running a test suite pytest loads all plugins installed
+on the system. This is often convenient for upstreams, as it makes it
+possible to use the features provided by the plugins (such as <code class="docutils literal notranslate"><span class="pre">async</span></code>
+test function support, or fixtures) without the necessity to explicitly
+enable them. However, there are also cases when additional plugins
+could make the test suite fail or become very slow (especially if pytest
+is called recursively).</p>
+<p>The modern recommendation for these cases is to disable plugin
+autoloading via setting the <code class="docutils literal notranslate"><span class="pre">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span></code>
+environment variable, and then explicitly enable specific plugins
+if necessary.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Previously we used to recommend explicitly disabling problematic
+plugins via <code class="docutils literal notranslate"><span class="pre">-p</span> <span class="pre">no:&lt;plugin&gt;</span></code>. However, it is rarely obvious
+which plugin is causing the problems, and it is entirely possible
+that another plugin will cause issues in the future, so an opt-in
+approach is usually faster and more reliable.</p>
+</div>
+<p>The easier approach to enabling plugins is to use the <code class="docutils literal notranslate"><span class="pre">-p</span></code> option,
+listing specific plugins. The option can be passed multiple times,
+and accepts a plugin name as specified in the package’s
+<code class="docutils literal notranslate"><span class="pre">entry_points.txt</span></code> file:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span>
+<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span><span class="o">=</span><span class="m">1</span>
+<span class="w"> </span>epytest<span class="w"> </span>-p<span class="w"> </span>asyncio<span class="w"> </span>-p<span class="w"> </span>tornado
+<span class="o">}</span>
+</pre></div>
+</div>
+<p>However, this approach does not work when the test suite calls pytest
+recursively (e.g. you are testing a pytest plugin). In this case,
+the <code class="docutils literal notranslate"><span class="pre">PYTEST_PLUGINS</span></code> environment variable can be used instead. It
+takes a comma-separated list of plugin <em>module names</em>:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span>
+<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span><span class="o">=</span><span class="m">1</span>
+<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_PLUGINS</span><span class="o">=</span>xdist.plugin,xdist.looponfail,pytest_forked
+
+<span class="w"> </span>epytest
+<span class="o">}</span>
+</pre></div>
+</div>
+<p>Please note that failing to enable all the required plugins may cause
+some of the tests to be skipped implicitly (especially if the test suite
+is using <code class="docutils literal notranslate"><span class="pre">async</span></code> functions and no async plugin is loaded). Please
+look at skip messages and warnings to make sure everything works
+as intended.</p>
+</section>
<section id="using-pytest-xdist-to-run-tests-in-parallel">
<h2>Using pytest-xdist to run tests in parallel<a class="headerlink" href="#using-pytest-xdist-to-run-tests-in-parallel" title="Link to this heading">¶</a></h2>
<p><a class="reference external" href="https://pypi.org/project/pytest-xdist/">pytest-xdist</a> is a plugin that makes it possible to run multiple tests
@@ -191,47 +240,6 @@ to strip options enabling them from <code class="docutils literal notranslate"><
</pre></div>
</div>
</section>
-<section id="explicitly-disabling-automatic-pytest-plugins">
-<h2>Explicitly disabling automatic pytest plugins<a class="headerlink" href="#explicitly-disabling-automatic-pytest-plugins" title="Link to this heading">¶</a></h2>
-<p>Besides plugins explicitly used by the package, there are a few pytest
-plugins that enable themselves automatically for all test suites
-when installed. In some cases, their presence causes tests of packages
-that do not expect them, to fail.</p>
-<p>An example of such package used to be <code class="docutils literal notranslate"><span class="pre">dev-python/pytest-relaxed</span></code>.
-To resolve problems due to the plugin, it was necessary to disable
-it explicitly:</p>
-<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span>
-<span class="w"> </span><span class="c1"># pytest-relaxed plugin makes our tests fail</span>
-<span class="w"> </span>epytest<span class="w"> </span>-p<span class="w"> </span>no:relaxed
-<span class="o">}</span>
-</pre></div>
-</div>
-</section>
-<section id="expert-disabling-plugin-autoloading-entirely">
-<h2>Expert: disabling plugin autoloading entirely<a class="headerlink" href="#expert-disabling-plugin-autoloading-entirely" title="Link to this heading">¶</a></h2>
-<p>If a test suite invokes pytest recursively (this is particularly
-the case when packaging other pytest plugins), the <code class="docutils literal notranslate"><span class="pre">-p</span></code> option
-can be insufficient to disable problematic plugins, as it does not
-get passed to the nested pytest invocations. For these packages,
-the next best thing is to use environment variables.</p>
-<p>Unfortunately, environment variables can only be used to disable
-autoloading entirely. When doing that, you need to explicitly force
-loading plugins that the test suite really needs.</p>
-<p>This is done using two envvars: <code class="docutils literal notranslate"><span class="pre">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span></code>
-to disable autoloading plugins, and <code class="docutils literal notranslate"><span class="pre">PYTEST_PLUGINS</span></code> to specify
-plugins to load. The latter takes a comma-separated list of entry point
-modules. To find the correct module names, look into
-the <code class="docutils literal notranslate"><span class="pre">entry_points.txt</span></code> inside the package’s <code class="docutils literal notranslate"><span class="pre">.egg-info</span></code> directory.</p>
-<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span>
-<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span><span class="o">=</span><span class="m">1</span>
-<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_PLUGINS</span><span class="o">=</span>xdist.plugin,xdist.looponfail,pytest_forked
-
-<span class="w"> </span>distutils_install_for_testing
-<span class="w"> </span>epytest
-<span class="o">}</span>
-</pre></div>
-</div>
-</section>
<section id="typeerror-make-test-flaky-got-an-unexpected-keyword-argument-reruns">
<h2>TypeError: _make_test_flaky() got an unexpected keyword argument ‘reruns’<a class="headerlink" href="#typeerror-make-test-flaky-got-an-unexpected-keyword-argument-reruns" title="Link to this heading">¶</a></h2>
<p>If you see a test error resembling the following:</p>
@@ -405,11 +413,10 @@ setting ignores <code class="docutils literal notranslate"><span class="pre">Dep
<li class="toctree-l2"><a class="reference internal" href="#skipping-tests-based-on-markers">Skipping tests based on markers</a></li>
<li class="toctree-l2"><a class="reference internal" href="#skipping-tests-based-on-paths-names">Skipping tests based on paths/names</a></li>
<li class="toctree-l2"><a class="reference internal" href="#avoiding-the-dependency-on-pytest-runner">Avoiding the dependency on pytest-runner</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#disabling-plugin-autoloading">Disabling plugin autoloading</a></li>
<li class="toctree-l2"><a class="reference internal" href="#using-pytest-xdist-to-run-tests-in-parallel">Using pytest-xdist to run tests in parallel</a></li>
<li class="toctree-l2"><a class="reference internal" href="#dealing-with-flaky-tests">Dealing with flaky tests</a></li>
<li class="toctree-l2"><a class="reference internal" href="#avoiding-dependencies-on-other-pytest-plugins">Avoiding dependencies on other pytest plugins</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#explicitly-disabling-automatic-pytest-plugins">Explicitly disabling automatic pytest plugins</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#expert-disabling-plugin-autoloading-entirely">Expert: disabling plugin autoloading entirely</a></li>
<li class="toctree-l2"><a class="reference internal" href="#typeerror-make-test-flaky-got-an-unexpected-keyword-argument-reruns">TypeError: _make_test_flaky() got an unexpected keyword argument ‘reruns’</a></li>
<li class="toctree-l2"><a class="reference internal" href="#importpathmismatcherror">ImportPathMismatchError</a></li>
<li class="toctree-l2"><a class="reference internal" href="#fixture-not-found">fixture ‘…’ not found</a></li>