Skip to content

Instantly share code, notes, and snippets.

@seanh
Created August 21, 2012 09:54
Show Gist options
  • Save seanh/3414107 to your computer and use it in GitHub Desktop.
Save seanh/3414107 to your computer and use it in GitHub Desktop.
Basic Orientation to CKAN (for developers)

Basic Orientation to CKAN (for developers)

Some of the most important layers in the CKAN code architecture, going from lowest-level up to highest-level:

##Database

CKAN uses SQLAlchemy (http://www.sqlalchemy.org/) to handle communication between CKAN source code and the PostgreSQL database (http://www.postgresql.org/).

Model

The ckan/model/ package corresponds to the model in the Model-View-Controller concept (http://www.codinghorror.com/blog/2008/05/understanding-model-view-controller.html) and contains classes for each of the entities stored in CKAN's database, e.g. ckan/model/package.py contains a Package class that represents CKAN data packages, and contains the SQLAlchemy code for mapping between Package objects in the CKAN source code and their corresponding table in the database.

A note about packages, these are now called 'datasets' in the CKAN interface, but in the source code you'll still find many places where they're referred to as packages, package and dataset they mean the same thing.

Dictization

ckan/lib/dictization/ contains functions that transform CKAN model objects such as Packages, Resources, etc. into dictionaries and vice-versa. The templates that render the CKAN web interface and the JSON format used by the CKAN API both work with these dictionary representations.

Logic

ckan/logic/ contains all the functions that the WUI, API and CLI use for accessing and manipulating the data stored in CKAN. Whenever you want to get, create, update or delete some object from CKAN's model you use some function from the ckan/logic/action/ package, for example there are functions package_show(), package_create(), package_update() and package_delete() for working with packages.

All communication with the model should be done in the logic layer, and not in higher-level layers such as the controllers, which should use the logic layer instead of accessing the model themselves.

Validation

The logic action functions use schema defined in ckan/logic/schema.py to validate data that is posted to CKAN by users via the WUI, API or CLI. The schema in turn draw on validation and conversion functions defined in ckan/logic/validators.py and ckan/logic/converters.py.

Authorization

The logic action functions use the authorization functions in ckan/logic/auth/ to decide whether a user is authorized to call an action function.

API

The CKAN API v3 ('the action API') is the newest, most complete and best-documented of the CKAN API versions and the version we intend to support going forwards. Docs for the v3 API are here: http://docs.ckan.org/en/ckan-1.8/apiv3.html

The API is implemented so that as soon as function such as package_update() is added to the ckan/logic/action/ package, this function can be called from the API by posting a JSON dict (and optionally an Authorization header containing a CKAN API key) to a URL that is automatically generated from the functions name, such as /api/action/package_update. The docstrings from these ckan/logic/action/ functions are also automatically exported into the Action API documentation page as well. The idea is that we want to expose as much of CKAN's functionality as possible through the API and not just from the command-line and web user interfaces (see https://plus.google.com/112678702228711889851/posts/eVeouesvaVX for some interesting discussion about APIs).

WUI

CKAN's Web User Interface is implemented by the controller classes in ckan/controllers/. These classes call functions from ckan/logic/action/ to manipulate data, and return rendered template files from ckan/templates/. ckan/config/routing.py contains the routing definitions that control which requests get sent to which controller classes according to which URL was requested (it uses routes.groovie.org).

Currently CKAN templates use the Genshi template engine (genshi.edgewall.org) but the upcoming CKAN 2.0 release will move to the much nicer and friendlier Jinja2 by default (http://jinja.pocoo.org/). Genshi will still be supported so that existing CKAN themes don't break.

CLI

CKAN has a Command Line Interface that you can use to do lots of actions such as managing datasets and users, etc. (Do paster --plugin=ckan from the ckan dir to see what's available.) It's implemented using Paste Script: http://pythonpaste.org/script/developer.html

Plugins

CKAN contains a number of plugin interfaces that CKAN plugins can use to hook into CKAN and add or alter behaviour. For example, the IAuthFunctions plugin interface allows plugins to override CKAN's authorization functions. The different extension interfaces are documented here: http://docs.ckan.org/en/ckan-1.8/writing-extensions.html This stuff is implemented in ckan/plugins/, for example you'll find all the plugin interfaces defined in ckan/plugins/interfaces.py, and ckan/plugins/toolkit.py defines the 'plugin toolkit', a collection of useful helper functions etc. for plugins to use.

We have several CKAN plugins for adding various functionality, e.g. commenting on datasets using disqus (https://github.com/okfn/ckanext-disqus), sharing pages on social networking sites (https://github.com/okfn/ckanext-social), and Google Analytics integration (https://github.com/okfn/ckanext-googleanalytics).

Some core CKAN features are actually implemented as plugins, you can find these in ckanext in the CKAN git repo: https://github.com/okfn/ckan/tree/master/ckanext

Further Reading

CKAN Coding Standards: http://docs.ckan.org/en/latest/coding-standards.html covers how we use git, our Python, JavaScript, HTML and CSS coding standards, etc.

Release cycle: http://docs.ckan.org/en/latest/release-cycle.html

How we do i18n: http://docs.ckan.org/en/latest/i18n.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment