An overview of some ideas for a Python application server.

app-server.textile

Application Management Server

Web-based interface (w/command-line tools) for managing:

1. Application server configuration.
   1. SSH keys, etc.
2. Application installation.
   1. Drop a .zip (or .egg, more on this below) into a folder.
   2. Upload the same via web interface.
   3. Enter the URL of a SCM system for checkout and deployment.
      1. Several deployment strategies:
         1. always-latest
         2. one-branch
         3. latest-tag
3. Application configuration.
   1. The application provides a configuration spec via INI or YAML.
      1. Mustn’t be Python code within the application; it might not be installed yet!
      2. A library of standard widgets and validation schemes must be provided.
      3. Additional validation via regular expressions.
   2. The Application Server then allows editing, validation, and saving of an INI or YAML on-disk configuration for the application.
4. Application “mounting” within: (i.e. virtual server management w/ path-based mounting)
   1. Nginx
   2. Apache
   3. Lighttpd
5. Includes several common “light-weight” applications out-of-the-box:
   1. Simple serving of static files.
   2. XML-RPC or JSON application server API. (To make the API public; available from localhost by default.)
      1. This type of API is how the web interface and command-line interfaces actually get work done.
   3. Install-via-SSH “slave” application servers.
   4. Post-commit hook service for updating applications.
6. Database-backed logging handler for trapping of all application’s log events.
   1. Log viewer, search, and notification system.
   2. MongoDB capped collections are perfect for this.

Core Operation

1. Applications are “extracted” into a pseudo-virtualenv for the application.
   1. Dependencies may be shared between applications or may be application-specific.
   2. Site packages are ignored, but the Python path includes a “shared” (server-packages) location and the application’s site-packages folder.
2. The application may request (via a spec file) writeable directories. Examples include:
   1. Sessions
   2. Caches
   3. Lock Files
3. The daemon serving each application is provided:
   1. The method of management: threading, multi-process, etc.
   2. A path to save the PID to.
   3. A path to save on-disk sockets to, or, a range of assigned port numbers.

Application Spec

A .pyapp file is a zip file containing:

1. metadata.yaml
   1. Application metadata, generated from setup.py and/or setup.ini.
   2. Better defines dependencies as shared or local.
   3. Requests resources such as writeable paths, database connections, etc.
   4. Defines paths to static resources.
   5. Defines the root WSGI application or WSGI application factory callable.
   6. Update policies.
   7. Post-install, pre/post-update, and pre-delete hooks.
   8. Recurrent tasks executed from within the context of the application.
2. configuration.yaml
   1. Runtime application configuration specification.
3. src/
   1. Extracted and installed using setup.py install.
4. deps/
   1. Like pip bundle, copies of the dependencies that are version-sensitive or for cases where you want offline installation.
   2. One subfolder per dependency, installed via the application’s setup.py install to allow for C extensions and such.

Application Usage

1. Applications have defined for them by the Application Server several environment variables:
   1. PYAPP_CONFIG_FILE — The filled-in YAML configuration.
   2. PYAPP_TMP_PATH — A directory it is safe to create temporary files within.
2. Applications are chrooted to their installation directory (pseudo-venv) and run as the application server.
3. Applications are accessed directly by the underlying web server, not proxied through the management application.

The underlying server (Paste, Flup, marrow.server.*, etc.) doesn't matter as they all speak WSGI in the end.