How to modularize your py.test fixtures

pytest-fixture-modularization.md

Using py.test is great and the support for test fixtures is pretty awesome. However, in order to share your fixtures across your entire module, py.test suggests you define all your fixtures within one single conftest.py file. This is impractical if you have a large quantity of fixtures -- for better organization and readibility, you would much rather define your fixtures across multiple, well-named files. But how do you do that? ...No one on the internet seemed to know.

Turns out, however, you can define fixtures in individual files like this:

tests/fixtures/add.py

import pytest

@pytest.fixture
def add(x, y):
    x + y

Then you can import these fixtures in your conftest.py:

tests/conftest.py

import pytest
from fixtures.add import add

...and then you're good to test!

tests/adding_test.py

import pytest

@pytest.mark.usefixtures("add")
def test_adding(add):
    assert add(2, 3) == 5

Because of the modularity, tests will have to be run with python -m py.test instead of py.test directly.

@christian-steinmeyer thank you for replying.
I saw that parmeterization concept i see, that applies to test method instead of fixture

Here is my fixture

import pytest
@pytest.fixture
def sum2Nbrs(a,b):
    return a+b

The test function is like

@pytest.mark.usefixtures("sum2Nbrs")
def test_2NbrsAddViaFixture(sum2Nbrs):
    print("The sume of numbers is", sum2Nbrs(2,4))

When i run this function i get

below is the error

file S:\Automation\python\tests\test_fixturesCall.py, line 7
def test_2NbrsAddViaFixture(sum2Nbrs):
file S:\Automation\python\tests\fixtures\MathsFixtures.py, line 3
@pytest.fixture
def sum2Nbrs(a,b):

E fixture 'a' not found

i would like to call fixture and not test method with parameters as shown above.

My test function should take 2 parameters and give me the sum of them

How about

import pytest
@pytest.fixture
def sum2Nbrs():
    def sum(a, b):
        return a+b

    return sum

Thanks a lot christian , this did what i wanted to achieve

Actual Code/ Fixtures

@pytest.fixture
def sum2Nbrs():
 def sum(a,b):
    return a+b
 return sum   

Calling Code/fixture

def test_2NbrsAddViaFixture(sum2Nbrs):
    print("The sume of numbers is", sum2Nbrs(2,4))

This is a good way to parameterize the fixtures by creating another function(sum) under them(sum2Nbrs).

Calling the function directly mentioning the fixture name without subfunction(sum2Nbrs(2,4) )
We did not explicit took the name sum while calling.

Thank you

Does pytest try to import all the fixtures from the pytest_plugins variable? I was wondering if the method mentioned in this post can be applied to resolve dependency issues that occur with so many fixtures being present in a single file, something like only the fixtures that are needed are imported... (I cant necessarily have multiple conftest files as I want to import and resuse fixtures across different directories)?

Does pytest try to import all the fixtures from the pytest_plugins variable?

Yes, every module in pytest_plugins is loaded and all fixtures in that module are made available to all tests.

glob("tests/fixtures/*.py") if "__" not in fixture

Can be replaced with

glob("tests/fixtures/[!__]*.py")

or even

glob("tests/fixtures/[!_]*.py")

to also skip filenames that start with single underscore.

Just a solution I've come across for anyone reading:

# conftest.py
here = os.path.dirname(os.path.realpath(__file__))

pytest_plugins = []

def _as_module(root: str, path: str) -> str:
    path = os.path.join(root, path)
    path = path.replace(here, "")
    path = path.replace(".py", "")
    path = path.replace(os.path.sep, ".")[1:]
    return "test." + path


for root, dirs, files in os.walk(here, topdown=True):
    dirs[:] = [d for d in dirs if d.startswith("test")]
    pytest_plugins += [_as_module(root, f) for f in files if f.endswith("fixtures.py")]

It allows you to have a fixtures file in any subdirectory

.../test
   __init__.py
   conftest.py
   /test_f1
      __init__.py
      fixtures.py
   
      /test_subf1
         __init__.py
         fixtures.py
   
      /test_subf2
         __init__.py
         endswith_fixtures.py
         extra_fixtures.py
   
   /test_f2
      __init__.py
      fixtures.py

@eddiebergman That's what the code in https://gist.github.com/peterhurford/09f7dcda0ab04b95c026c60fa49c2a68#gistcomment-3453453 (using pkgutil.walk_packages) already does.

@ikonst Ahh, I didn't know it supported the same layout with fixtures in each dir, very nice

You can simply create "local pytest plugins" which can be nothing more than Python files with fixtures, e.g.

* tests/unit/conftest.py:
  ```python
  pytest_plugins = [
     "tests.unit.fixtures.some_stuff",
  ]

• tests/unit/fixtures/some_stuff.py:

  import pytest
  
  @pytest.fixture
  def foo():
      return 'foobar'

Thanks for you idea @ikonst! Just made my conftest as package and add it as plugin!

Can you share how did you share the conftest as a package? Can that package be pip installed across other repo/projects?

There's no much difference between a conftest.py and a pytest plugin (only one I can think of, is that a top-level conftest can have a pytest_plugins section).

If you want to make a Python package with pytest fixtures, see https://docs.pytest.org/en/6.2.x/writing_plugins.html

Hello everyone, thank you for all the insightful comments.
If you use PyCharm, it seems that, for now, you won't be able to use the glob approach because of this issue, which cleans up the "unused import" needed with this approach.

My recommendation would be to use the hard-coded list and maintain the conftest.py file.
With this, you also do not need to import the fixture in your test file.

Otherwise, here's a summary of the above:

tests/conftest.py

from glob import glob
def _as_module(fixture_path: str) -> str:
    return fixture_path.replace("/", ".").replace("\\", ".").replace(".py", "")


pytest_plugins = [
    _as_module(fixture) for fixture in glob("tests/fixtures/[!_]*.py")
]

tests/fixtures/example.py

@pytest.fixture()
def example():
    return []

tests/unit/mytest.py

# If you did not hardcode pytest_plugins, you might need `from tests.fixtures.example import example`
def test_get_a_list(example):
    assert isinstance(example, list)

pytest_plugins

It worked like a charm, thank you.

Great gist on modularization of conftest, it works great, thanks!

And thanks to @ryanjdillon for the tip about init.py, it solved a mypy problem for me.

Another option would be to have multiple nested directories/packages containing your tests, and each directory can have its own conftest.py with its own fixtures, adding on to the ones provided by the conftest.py files in parent directories as written in the official doc here