Guide for how to create a (minimal) private PyPI repo, just using Apache with directory autoindex, and pip with an extra index URL.

Private-pypi-howto

*

private_pypi_howto.md

How to set up and use a private PyPI repo

Accompanying blog post:

Splitting a Python codebase into dependencies for fun and profit

Based on:

Create a local PyPi repository using only mod_rewrite

See also:

Local PyPI Options

Setting up a private, team-wide PyPI repository

For a more advanced private PyPI, see:

devpi: PyPI server and packaging/testing/release tool

Create root directory for private PyPI

ssh [email protected]

(On remote server)

mkdir /path/to/pypi.myserver.com
mkdir /path/to/pypi.myserver.com/simple

Create self-signed SSL certificate

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout /etc/apache2/ssl/pypi.myserver.com.key \
-out /etc/apache2/ssl/pypi.myserver.com.pem

Set up Apache vhost

sudo htpasswd -c /etc/apache2/passwords_pypi pypi

sudo vi /etc/apache2/sites-available/pypi.myserver.com

(Add these lines)

---

<VirtualHost *:80>
    ServerName pypi.myserver.com

    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</VirtualHost>

<VirtualHost *:443>
    ServerName pypi.myserver.com
    DocumentRoot /data/www/pypi.myserver.com

    SSLEngine On
    SSLCertificateFile /etc/apache2/ssl/pypi.myserver.com.pem
    SSLCertificateKeyFile /etc/apache2/ssl/pypi.myserver.com.key

    <Directory /data/www/pypi.myserver.com/>
        AllowOverride None
        Options +Indexes
        IndexOptions SuppressColumnSorting
        IndexIgnore ..
        Order deny,allow
        Allow from all

        AuthType Basic
        AuthName "My Server"
        AuthBasicProvider file
        AuthUserFile /etc/apache2/passwords_pypi
        Require valid-user
    </Directory>

    LogLevel warn
    ErrorLog /var/log/apache2/pypi-error.log
    CustomLog /var/log/apache2/pypi-access.log combined
</VirtualHost>

---

cd /etc/apache2/sites-enabled
sudo ln -s ../sites-available/pypi.myserver.com pypi.myserver.com
sudo apache2ctl graceful

Create directory for new library in private PyPI

mkdir /path/to/pypi.myserver.com/simple/foobar-utils

exit

Update library's code

(On local machine)

cd /path/to/foobar-utils

vi foobar_utils.py

(Add these lines)

---

__version__ = '0.1.0'

foobar = 'Hey foo, I am a bar!'

---

vi setup.py

(Add these lines)

---

import os

import setuptools

module_path = os.path.join(os.path.dirname(__file__), 'foobar_utils.py')
version_line = [line for line in open(module_path)
                if line.startswith('__version__')][0]

__version__ = version_line.split('__version__ = ')[-1][1:][:-2]

setuptools.setup(
    name="foobar-utils",
    version=__version__,
    url="https://git.myserver.com/foobar-utils/",

    author="Mister foo",
    author_email="[email protected]",

    description="Utils for handling Foo and Bar.",
    long_description=open('README.rst').read(),

    py_modules=['foobar_utils'],
    zip_safe=False,
    platforms='any',

    install_requires=[],

    classifiers=[
        'Development Status :: 2 - Pre-Alpha',
        'Environment :: Web Environment',
        'Intended Audience :: Developers',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.3',
    ],
)

---

vi README.rst

(Add these lines)

---

foobar-utils
============

Utils for handling Foo and Bar.

---

Upload new version of library code to private PyPI

python setup.py bdist_wheel --universal
scp dist/foobar_utils-0.1.0-py2.py3-none-any.whl \
[email protected]:/path/to/pypi.myserver.com/simple/foobar-utils/

Configure pip to use private PyPI

vi ~/.pip/pip.conf

(Add these lines)

[global]
; Extra index to private pypi dependencies
extra-index-url = https://pypi:[email protected]/simple/
trusted-host = pypi.myserver.com

Use private library in a project's requirements.txt

cd /path/to/projectfoo
virtualenv .
source bin/activate

vi requirements.txt

(Add these lines)

---

foobar-utils==0.1.0

---

pip install -r requirements.txt

Nginx config I used today:

server {
    listen 443 ssl;
    server_name  {{ pypi_domain }};
    include snippets/letsencrypt-{{ pypi_domain }}.conf;
    location / {
        root /data/pypi/;
        autoindex on;
        try_files $uri $uri/ =404;
        client_body_temp_path /data/pypi.tmp/;
        dav_methods PUT;
        create_full_put_path on;
        dav_access group:rw all:r;
        limit_except GET {
                        allow {{ ci_public_ip }};
                        deny  all;
        }
    }
    index index.html;
}

We're uploading from our CI with a simple curl -XPUT https://pypi.redacted/simple/demo-0.0.2.tar.gz --data-binary @dist/demo-0.0.2.tar.gz from a whitelisted IP and it's done.