Sphinx Setup Guide for Python Packages with a Python `src` Layout

00_sphinx_setup.md

Sphinx Setup Guide for Python Packages with a Python src Layout

This guide walks through setting up automatic Sphinx documentation for a Python package using a src layout. At the end, running make docs generates complete API reference documentation from docstrings without any manual maintenance.

Prerequisites

• Python 3.10+
• Poetry installed
• A repository with this minimum structure:

my_package/
├── src/
│   └── my_package/
│       └── __init__.py
├── pyproject.toml
└── README.md

pyproject.toml

---

1. Install Sphinx

poetry add --group dev "sphinx>=7.0,<8.0"

Sphinx 9+ requires Python 3.12. Pin to 7.x for Python 3.10 compatibility.

---

2. Run sphinx-quickstart

From the project root:

poetry run sphinx-quickstart docs

When prompted:

• Separate source and build directories: n
• Project name: your package name
• Author name: your name
• Project version: your version
• Language: en

This creates:

docs/
├── conf.py
├── index.rst
├── Makefile
└── make.bat

---

3. Configure conf.py

Replace the contents of docs/conf.py with:

import os
import sys

sys.path.insert(0, os.path.abspath("../src"))

project = "my_package"
copyright = "2026, Your Name"
author = "Your Name"
release = "0.1.0"

extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.napoleon",
]

templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
html_theme = "alabaster"
html_static_path = []

Key points:

• sys.path.insert(0, os.path.abspath("../src")) tells Sphinx where to find the package in a src layout.
• autodoc extracts docstrings from source files.
• napoleon enables Google-style docstring support.
• html_static_path = [] avoids a warning when no custom static files exist.

---

4. Update docs/index.rst

Replace the contents of docs/index.rst with:

my_package documentation
=========================
 
.. toctree::
   :maxdepth: 2
   :caption: Contents:
 
   installation
   usage
   reference/modules

.. toctree:: is a Sphinx directive that builds the navigation tree for the documentation site. Everything indented under it is a page that will appear in the sidebar and table of contents. :maxdepth: 2 controls how many heading levels deep the table of contents goes — at 2 it shows the page title and its immediate sub-sections. :caption: Contents: is the label that appears above the navigation list in the sidebar.

installation, usage, and reference/modules are paths to .rst files relative to docs/. Sphinx resolves them as docs/installation.rst, docs/usage.rst, and docs/reference/modules.rst. The .rst extension is omitted by convention.

reference/modules is the only generated entry. sphinx-apidoc always writes this file as its top-level table of contents for the package, and it in turn links to the individual per-module pages it generates. The other two are hand-written narrative pages that you maintain.

---

5. Create stub pages

Create docs/installation.rst:

Installation
============

Requirements
------------

- Python 3.10 or higher
- Poetry (for development installs)

Install from source
-------------------

.. code-block:: bash

   git clone <repo-url>
   cd my_package
   poetry install

Create docs/usage.rst:

Usage
=====

CLI
---

.. code-block:: bash

   my_package --help

API
---

Send a POST request to ``/predict`` with the required input. See the package reference for full documentation.

---

6. Update .gitignore

Add the following to .gitignore:

# Sphinx docs
docs/_build/
docs/reference/

docs/_build/ is build output. docs/reference/ contains files generated by sphinx-apidoc at build time and should never be committed.

---

7. Create a root Makefile

Create Makefile in the project root:

.PHONY: docs docs-open tests lint
 
docs:
	poetry run sphinx-apidoc -f -o docs/reference src/my_package
	poetry run sphinx-build -b html -W docs/ docs/_build/html
 
docs-open:
	poetry run sphinx-apidoc -f -o docs/reference src/my_package
	poetry run sphinx-build -b html -W docs/ docs/_build/html
	open docs/_build/html/index.html
	
docs-start:
	poetry run sphinx-apidoc -f -o docs/reference src/my_package
	poetry run sphinx-build -b html -W docs/ docs/_build/html
	start docs/_build/html/index.html

 
tests:
	poetry run pytest tests/ -v
 
lint:
	poetry run pre-commit run --all-files

Replace my_package with your actual package name.

Each target is a shortcut for a common task:

• make docs runs sphinx-apidoc to generate reference .rst files from the source tree, then runs sphinx-build to produce the HTML output in docs/_build/html. This is the command you and your team run to build the documentation locally, and the command the pre-commit hook and GitHub Actions workflow use in CI.
• make docs-open does the same as make docs and then opens the result in the browser. Use this during development to preview changes to docstrings or narrative pages.
• make tests runs the full test suite. Use this before opening a pull request.
• make lint runs all pre-commit hooks against every file. Use this to catch formatting and linting issues without making a commit. The -f flag on sphinx-apidoc forces it to overwrite any existing generated files on every run, ensuring the reference is always up to date. Without it, sphinx-apidoc skips files that already exist, which can leave stale pages behind after a module is renamed or moved.

docs-open is macOS only — Windows users should replace open with start.

---

8. Add a pre-commit hook

Create .pre-commit-config.yaml in the project root. First install pre-commit:

poetry add --group dev pre-commit
poetry run pre-commit install

Minimal .pre-commit-config.yaml:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-added-large-files
  - repo: https://github.com/psf/black
    rev: 24.3.0
    hooks:
      - id: black
  - repo: local
    hooks:
      - id: sphinx-build
        name: Build Sphinx documentation
        entry: bash -c 'poetry run sphinx-apidoc -f -q -o docs/reference src/my_package && poetry run sphinx-build -b html -W docs/ docs/_build/html'
        language: system
        pass_filenames: false
        files: \.(py|rst)$

The pre-commit-hooks entries enforce basic hygiene on every commit. black auto-formats Python files. The sphinx-build hook runs sphinx-apidoc followed by sphinx-build so it is fully self-contained — it never depends on previously generated files.

The -q flag silences sphinx-apidoc output so only errors are shown. The -W flag on sphinx-build promotes warnings to errors, so a broken docstring or missing reference blocks the commit rather than silently producing bad documentation.

If you are running pre-commit in a CI environment without Poetry, add SKIP=sphinx-build to the CI step that runs pre-commit and handle the docs build separately in your docs deployment workflow.

---

9. Verify the setup

make docs
# MacOS
open docs/_build/html/index.html
# Windows
start docs/_build/html/index.html

The package reference should appear with all modules and their docstrings.

---

10. How it works going forward

When you add a new module, no documentation files need to be updated. On the next make docs run, sphinx-apidoc finds the new module and generates its reference page automatically. When you remove a module, the reference page disappears on the next build. The only files that ever need manual updates are the narrative pages: installation.rst and usage.rst.

Adding submodules and nested packages

sphinx-apidoc recurses into subdirectories automatically, as long as each directory contains an __init__.py. For example, adding src/my_package/inference/predict.py requires two files:

src/my_package/inference/__init__.py
src/my_package/inference/predict.py

The __init__.py can be minimal:

"""Inference subpackage for my_package."""

On the next make docs run, sphinx-apidoc generates docs/reference/my_package.inference.rst and docs/reference/my_package.inference.predict.rst automatically. The module appears in the reference under my_package.inference.predict with its docstrings. No changes to docs/index.rst or any other file are required.

---

11. Publish documentation automatically with GitHub Actions

This step deploys the built documentation to GitHub Pages on every push to main.

Enable GitHub Pages

In your repository go to Settings > Pages and set the source to GitHub Actions.

Create the workflow

Create .github/workflows/docs.yaml:

name: Deploy Sphinx documentation to GitHub Pages
 
on:
  push:
    branches: ["main"]
  workflow_dispatch:
 
permissions:
  contents: read
  pages: write
  id-token: write
 
concurrency:
  group: "pages"
  cancel-in-progress: false
 
jobs:
  deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
 
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.10"
 
      - name: Install dependencies
        run: |
          pip install poetry
          poetry install --with dev
 
      - name: Generate API reference
        run: poetry run sphinx-apidoc -f -q -o docs/reference src/my_package
 
      - name: Build Sphinx documentation
        run: poetry run sphinx-build -b html -W docs/ docs/_build/html
 
      - name: Setup Pages
        uses: actions/configure-pages@v5
 
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v3
        with:
          path: docs/_build/html
 
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v5

Replace my_package with your actual package name.

The Generate API reference step is critical — it must run before Build Sphinx documentation. Without it, docs/reference/ is empty in the CI environment because generated .rst files are in .gitignore and are never committed. Omitting this step produces a site with no package reference content even though the build succeeds.

How it works

The workflow triggers on every push to main and can also be triggered manually from the Actions tab via workflow_dispatch. It installs dependencies via Poetry, runs sphinx-apidoc to generate reference .rst files from the source tree, runs sphinx-build to produce the HTML output, and deploys to GitHub Pages. The workflow does not run on pull requests — documentation is only published when changes land on main.

Viewing deployment progress

After pushing to main, go to the Actions tab in your GitHub repository. You will see the workflow named Deploy Sphinx documentation to GitHub Pages running. Click on it to expand each step. The Generate API reference and Build Sphinx documentation steps show any warnings or errors from the build. The final Deploy to GitHub Pages step shows the deployed URL when it completes.

To review past deployments go to Actions > Deployments > All Deployments. Each entry shows which PR triggered the deployment, who triggered it, and whether it succeeded. The currently active deployment is marked Active.

The deployed URL follows the pattern https://<org>.github.io/<repo>/ and also appears under Settings > Pages in the repository.

Skipping the sphinx hook in CI

The lint workflow runs pre-commit, which includes the sphinx-build hook. That hook requires Poetry and the full package environment, which is not available in the lint runner. Skip it explicitly in your lint workflow:

- name: Run pre-commit hooks
  run: SKIP=sphinx-build pre-commit run --all-files

The docs.yaml workflow is the authoritative build and deploy pipeline. The pre-commit hook is for catching broken docs locally before committing.

---

Docstring format

This setup uses Google-style docstrings via the napoleon extension:

def predict(image_path: str, model_path: str) -> dict:
    """Run inference on a single image.
 
    Args:
        image_path: Path to the input image file.
        model_path: Path to the trained model weights.
 
    Returns:
        A dict containing segmentation masks and confidence score.
 
    Raises:
        FileNotFoundError: If image_path or model_path does not exist.
 
    Example:
        >>> result = predict("image.png", "model.h5")
        >>> print(result["confidence_score"])
    """

Sphinx renders Args, Returns, Raises, and Example sections automatically from this format.

Avoid indented sub-lists inside named sections — Sphinx treats them as unexpected indentation and will fail the build. Use a Note: section with unindented bullet points instead:

    """Summary line.
 
    Note:
        Some important information:
 
        - First point
        - Second point
    """

---

Common errors

Error	Cause	Fix
toctree contains reference to nonexisting document 'reference/modules'	sphinx-apidoc has not been run yet	Run make docs instead of sphinx-build directly
autodoc: failed to import module	Package dependency not installed	Run poetry install
duplicate object description	Module documented in two places	Check for leftover .rst files in docs/ from a previous setup and delete them
html_static_path entry does not exist	_static directory missing	Set html_static_path = [] in conf.py
Unexpected indentation in docstring	Indented sub-list inside a named section	Use a Note: section with unindented bullet points

__init__.py

"""my_package — one line description of what the package does."""

pyproject.toml

[project]
name = "my_package"
version = "0.1.0"
description = ""
authors = [
    {name = "Your Name", email = "you@example.com"}
]
readme = "README.md"
requires-python = ">=3.10"
dependencies = []

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"

[dependency-groups]
dev = [
    "sphinx>=7.0,<8.0",
    "pytest>=9.0.0,<10.0.0",
    "pre-commit>=4.0.0,<5.0.0",
]