当前位置：自学咖网 > 技术教程 > 编程最新 > pytest分布式执行插件 pytest-xdist 的高级用法
wenwen 编程最新

pytest分布式执行插件 pytest-xdist 的高级用法

pytest分布式执行插件 pytest-xdist 的高级用法

想要使用多个CPU核心来进行测试，可以使用 -n 参数( 或者 –numprocesses)
(使用8个核心来跑测试用例)

 pytest -n 8

使用 -n auto 参数可以利用电脑的所有核心来跑测试用例
测试时使用的算法可以根据--dist命令参数定制：

  • --dist load(默认选项)：给每个CPU核心随机分配用例，不保证执行顺序。
  • --dist loadscope：对于测试函数，测试按模块分组，对于测试方法，测试按类分组。每组作为一个整体分配给可用的worker。这保证了组中的所有测试都在同一进程中运行。如果的模块级或类级fixtures，这将非常有用。按类分组优先于按模块分组。
  • --dist loadfile: 测试用例按其所在文件分组。每组作为一个整体分配给可用的worker。这保证了文件中的所有测试都在同一个辅助进程中运行。
  • --dist loadgroup: 测试按xdist_group标记分组。每组作为一个整体分配给可用的执行器。这保证了具有相同xdist_ group名称的所有测试都在同一个worker中运行。
@pytest.mark.xdist_group(name="group1")
def test1():
    pass

class TestA:
    @pytest.mark.xdist_group("group1")
    def test2():
        pass-

这将确保test1和TestA:：test2将在同一个worker中运行。没有xdist_ group标记的测试在–dist=load模式下正常运行。

  • --dist no：正常的pytest执行模式，一次运行一个测试（完全没有分发）。

例子：
项目目录结构

xdist_test.py

import logging

import pytest


class TestXdist(object):

    @pytest.mark.xdist_group("group1")
    def test_one(self):
        logging.info("1")
        assert True

    @pytest.mark.xdist_group("group1")
    def test_two(self):
        logging.info("2")
        assert True

    @pytest.mark.xdist_group("group2")
    def test_three(self):
        logging.info("3")
        assert True

    @pytest.mark.xdist_group("group2")
    def test_four(self):
        logging.info("4")
        assert True

xdist_dummy_test.py

import logging

import pytest


class TestXdist(object):
    @pytest.mark.run(order=1)
    @pytest.mark.xdist_group("group1")
    def test_dummy_one(self):
        logging.info("d1")
        assert True

    @pytest.mark.xdist_group("group1")
    @pytest.mark.run(order=2)
    def test_dummy_two(self):
        logging.info("d2")
        assert True

    @pytest.mark.run(order=3)
    @pytest.mark.xdist_group("group2")
    def test_dummy_three(self):
        logging.info("d3")
        assert True

    @pytest.mark.xdist_group("group2")
    @pytest.mark.run(order=4)
    def test_dummy_four(self):
        logging.info("d4")
        assert True

(venv) ➜  pytest pytest -n auto --dist loadscope
=========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
[gw0] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw1] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw2] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw3] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw4] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw5] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw6] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw7] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
scheduling tests via LoadScopeScheduling

test2/xdist_dummy_test.py::TestXdist::test_dummy_one 
test/xdist_test.py::TestXdist::test_one 
[gw0] [ 12%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one 
[gw1] [ 25%] PASSED test/xdist_test.py::TestXdist::test_one 
test2/xdist_dummy_test.py::TestXdist::test_dummy_two 
test/xdist_test.py::TestXdist::test_two 
[gw1] [ 37%] PASSED test/xdist_test.py::TestXdist::test_two 
test/xdist_test.py::TestXdist::test_three 
[gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two 
test2/xdist_dummy_test.py::TestXdist::test_dummy_three 
[gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three 
test/xdist_test.py::TestXdist::test_four 
[gw0] [ 75%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three 
test2/xdist_dummy_test.py::TestXdist::test_dummy_four 
[gw1] [ 87%] PASSED test/xdist_test.py::TestXdist::test_four 
[gw0] [100%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four
============================================================================ 8 passed in 0.40s ============================================================================

(venv) ➜  pytest pytest -n auto --dist loadfile 
=========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
[gw0] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw1] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw2] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw3] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw4] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw5] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw6] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw7] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
scheduling tests via LoadFileScheduling

test/xdist_test.py::TestXdist::test_one 
test2/xdist_dummy_test.py::TestXdist::test_dummy_one 
[gw1] [ 12%] PASSED test/xdist_test.py::TestXdist::test_one 
[gw0] [ 25%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one 
test/xdist_test.py::TestXdist::test_two 
test2/xdist_dummy_test.py::TestXdist::test_dummy_two 
[gw1] [ 37%] PASSED test/xdist_test.py::TestXdist::test_two 
[gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two 
test2/xdist_dummy_test.py::TestXdist::test_dummy_three 
test/xdist_test.py::TestXdist::test_three 
[gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three 
[gw0] [ 75%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three 
test2/xdist_dummy_test.py::TestXdist::test_dummy_four 
test/xdist_test.py::TestXdist::test_four 
[gw1] [ 87%] PASSED test/xdist_test.py::TestXdist::test_four 
[gw0] [100%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four 

============================================================================ 8 passed in 0.38s ============================================================================


(venv) ➜  pytest pytest -n auto --dist loadgroup
=========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
[gw0] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw1] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw2] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw3] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw4] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw5] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw6] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw7] Python 3.8.9 (default, May 17 2022, 12:55:41)  -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
scheduling tests via LoadGroupScheduling

test2/xdist_dummy_test.py::TestXdist::test_dummy_three@group2 
test2/xdist_dummy_test.py::TestXdist::test_dummy_one@group1 
[gw1] [ 12%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three@group2 
test2/xdist_dummy_test.py::TestXdist::test_dummy_four@group2 
[gw0] [ 25%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one@group1 
[gw1] [ 37%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four@group2 
test2/xdist_dummy_test.py::TestXdist::test_dummy_two@group1 
[gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two@group1 
test/xdist_test.py::TestXdist::test_three@group2 
[gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three@group2 
test/xdist_test.py::TestXdist::test_four@group2 
[gw1] [ 75%] PASSED test/xdist_test.py::TestXdist::test_four@group2 
test/xdist_test.py::TestXdist::test_one@group1 
[gw0] [ 87%] PASSED test/xdist_test.py::TestXdist::test_one@group1 
test/xdist_test.py::TestXdist::test_two@group1 
[gw0] [100%] PASSED test/xdist_test.py::TestXdist::test_two@group1 

============================================================================ 8 passed in 0.40s ============================================================================
hmoban主题是根据ripro二开的主题，极致后台体验，无插件，集成会员系统
自学咖网 » pytest分布式执行插件 pytest-xdist 的高级用法

wen 普通

分享到：

相关推荐

Kafka丢数据、重复消费、顺序消费的问题

面试官：今天我想问下，你觉得Kafka会丢数据吗？ 候选者：嗯，使用Kafka时，有可能会有以下场景会丢消息 候选者：比如说，我们用Producer发消息至Broker的时候，就有可能会丢消息 候选者：如果你不想丢消息，...

MyBatis（十七）：一对多的处理

上一篇我们学习了多对一的处理，这次我们来学习一对多的处理。 一对多的处理与多对一的处理差别不大，只是有一些细微的地方需要注意。 我们还是先做准备工作，其他部分与多对一的准备工作相同，仅实体类构建需要做出改变。 一、修改实...

八、Java面向对象编程 – XiaoDi_CISP

Java面向对象 初识面向对象 面向过程 & 面向对象 面向过程思想 步骤清晰简单，第一步做什么，第二部做什么… 面对过程适合处理一些较为简单的问题 面向对象思想 物以类聚，分类的思维模式，思考问题首...