Guide for how to create a (minimal) private PyPI repo, just using Apache with directory autoindex, and pip with an extra index URL.

Private-pypi-howto

*

private_pypi_howto.md

How to set up and use a private PyPI repo

Accompanying blog post:

Splitting a Python codebase into dependencies for fun and profit

Based on:

Create a local PyPi repository using only mod_rewrite

See also:

Local PyPI Options

Setting up a private, team-wide PyPI repository

For a more advanced private PyPI, see:

devpi: PyPI server and packaging/testing/release tool

Create root directory for private PyPI

ssh [email protected]

(On remote server)

mkdir /path/to/pypi.myserver.com
mkdir /path/to/pypi.myserver.com/simple

Create self-signed SSL certificate

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout /etc/apache2/ssl/pypi.myserver.com.key \
-out /etc/apache2/ssl/pypi.myserver.com.pem

Set up Apache vhost

sudo htpasswd -c /etc/apache2/passwords_pypi pypi

sudo vi /etc/apache2/sites-available/pypi.myserver.com

(Add these lines)

---

<VirtualHost *:80>
    ServerName pypi.myserver.com

    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</VirtualHost>

<VirtualHost *:443>
    ServerName pypi.myserver.com
    DocumentRoot /data/www/pypi.myserver.com

    SSLEngine On
    SSLCertificateFile /etc/apache2/ssl/pypi.myserver.com.pem
    SSLCertificateKeyFile /etc/apache2/ssl/pypi.myserver.com.key

    <Directory /data/www/pypi.myserver.com/>
        AllowOverride None
        Options +Indexes
        IndexOptions SuppressColumnSorting
        IndexIgnore ..
        Order deny,allow
        Allow from all

        AuthType Basic
        AuthName "My Server"
        AuthBasicProvider file
        AuthUserFile /etc/apache2/passwords_pypi
        Require valid-user
    </Directory>

    LogLevel warn
    ErrorLog /var/log/apache2/pypi-error.log
    CustomLog /var/log/apache2/pypi-access.log combined
</VirtualHost>

---

cd /etc/apache2/sites-enabled
sudo ln -s ../sites-available/pypi.myserver.com pypi.myserver.com
sudo apache2ctl graceful

Create directory for new library in private PyPI

mkdir /path/to/pypi.myserver.com/simple/foobar-utils

exit

Update library's code

(On local machine)

cd /path/to/foobar-utils

vi foobar_utils.py

(Add these lines)

---

__version__ = '0.1.0'

foobar = 'Hey foo, I am a bar!'

---

vi setup.py

(Add these lines)

---

import os

import setuptools

module_path = os.path.join(os.path.dirname(__file__), 'foobar_utils.py')
version_line = [line for line in open(module_path)
                if line.startswith('__version__')][0]

__version__ = version_line.split('__version__ = ')[-1][1:][:-2]

setuptools.setup(
    name="foobar-utils",
    version=__version__,
    url="https://git.myserver.com/foobar-utils/",

    author="Mister foo",
    author_email="[email protected]",

    description="Utils for handling Foo and Bar.",
    long_description=open('README.rst').read(),

    py_modules=['foobar_utils'],
    zip_safe=False,
    platforms='any',

    install_requires=[],

    classifiers=[
        'Development Status :: 2 - Pre-Alpha',
        'Environment :: Web Environment',
        'Intended Audience :: Developers',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.3',
    ],
)

---

vi README.rst

(Add these lines)

---

foobar-utils
============

Utils for handling Foo and Bar.

---

Upload new version of library code to private PyPI

python setup.py bdist_wheel --universal
scp dist/foobar_utils-0.1.0-py2.py3-none-any.whl \
[email protected]:/path/to/pypi.myserver.com/simple/foobar-utils/

Configure pip to use private PyPI

vi ~/.pip/pip.conf

(Add these lines)

[global]
; Extra index to private pypi dependencies
extra-index-url = https://pypi:[email protected]/simple/
trusted-host = pypi.myserver.com

Use private library in a project's requirements.txt

cd /path/to/projectfoo
virtualenv .
source bin/activate

vi requirements.txt

(Add these lines)

---

foobar-utils==0.1.0

---

pip install -r requirements.txt

any idea how to use this without the user dot files? That kinda sucks when you want to have other people set up on your team right away. Obviously these packages will be sourced from the same spot. I'm new to python - just wondering if it can go into a config file in the project, just no luck finding this on google yet.

Also supplying them with environment variables, so we can do continuous delivery of our modules from a CI server, would be good info :)

https://pypi.python.org/pypi/twine

looks like this is a preferred way of deploying

Could you post the equivalent nginx config?

@brettswift Actually there are parameter to tell pip to use a different server and without TLS. example:
pip install requests --index-url=http://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com

Nginx config I used today:

server {
    listen 443 ssl;
    server_name  {{ pypi_domain }};
    include snippets/letsencrypt-{{ pypi_domain }}.conf;
    location / {
        root /data/pypi/;
        autoindex on;
        try_files $uri $uri/ =404;
        client_body_temp_path /data/pypi.tmp/;
        dav_methods PUT;
        create_full_put_path on;
        dav_access group:rw all:r;
        limit_except GET {
                        allow {{ ci_public_ip }};
                        deny  all;
        }
    }
    index index.html;
}

We're uploading from our CI with a simple curl -XPUT https://pypi.redacted/simple/demo-0.0.2.tar.gz --data-binary @dist/demo-0.0.2.tar.gz from a whitelisted IP and it's done.