#About Me:
Hi, I am Rohan Jain, a 4th (final) year B.Tech undergraduate Student from Indian Institute of Technology Kharagpur. I have been using django since over a year and generally look into the code base to find about various implementations. I have made attempts to make some minor contributions and if selected this would be my first major one.
More about Me: http://www.rohanjain.in/about/ Github, IRC: crodjer
Django currently uses two methods for protection against CSRF:
Both these system introduce a difference in behaviour. Using cookie-token system provides a side effect feature which lets you access the CSRF token across multiple subdomains. But the other, referer checking looks for the request to come from the same host and hence, doesn't allow for cookie like cross subdomain behaviour.
The fact that there is a CSRF_COOKIE_DOMAIN setting shows that django supports
the cross domain capabilities in the CSRF system. So users might utilize this
while building multiple sites. When they change to https scheme, they will see
a new surprising behaviour.
The working tree is at crodjer/django over github.
###Origin and Referer Checking, Permitted domains.
Origin header, a fairly new introduction, has CSRF as one of the target. Unlike the referer header, it sends back only the domain and the path information, so the people don't need to disable these through plugins like the referer header.
With origin header absent, referer header can be used as a fallback.
Strict referer checking is already there in https scheme which is used in all major deployments. We can bring it to the non secure scheme also.
With this method a permitted domains functionality is possible, which lets explicit specification of domains which are trusted and can bypass CSRF. A prototype implementation is there at crodjer/django, csrf-enhancements branch.
Cons:
###CSRF Cookies (Time signed):
A random token generated by the server stored in the browser cookies. For verification, every non get request will need to provide a signed version of the same token. This can then be verified on the browser side.
This can be implemented by adding signing to the existing csrf token system, using the signing framework.
A conventional method of CSRF checks, all the major frameworks have similar systems.
Signing takes care of the side effects due to cross domain behaviour of cookies.
Cons:
Given these two methods, I am slightly inclined at Origin/Referer checking, which is already being completely relied upon in case of the secure requests.
Or we could loose the possibility of permitted domains functionality and implement both kinds of checks.
#Abstract
Django is a reasonably secure framework. It provides an API and development patterns which transparently take care of the common web security issues. But still there are security features which need attention. I propose to work and improved CSRF checking without any compromises and on integration of existing work on centralized token system. If time permits I will also attempt on integration of django-secure.
#Description ##CSRF Improvements
Cross-Origin Resource Sharing (CORS):
W3C has a working draft regarding CORS, which opens
up the possibility for allowing client-side request cross-origin
requests. This directly triggers in mind the capability to develop
API which can be exposed directly to the web browser. This would let
us get rid of proxies and other hacks used to achieve this.
Currently all the major browsers support this: Chrome (all versions),
Firefox (> 3.0), IE (> 7.0), Safari (> 3.2), Opera (> 12.0). Firefox
and Chrome send the origin header for both AJAX and standard from POST
requests. Introduced it here as some further parts of the post refer
to this.
###Origin checking
With CORS around need for using CSRF token can be dropped, at least in
some browsers. Ticket #16859, is an attempt for
that. But this was rejected because of neglecting the case for
presence of CSRF_COOKE_DOMAIN (Refer to the closing comment on the
ticket for details). So to handle this we need to simulate checking of
CSRF cookie domain as web browsers do it. Maybe:
reqest.META.get('HTTP_ORIGIN').endswith(settings.CSRF_COOKIE_DOMAIN)In case the server receives an origin header in the request, it will be used for an initial checking and then all the conventional checks will be done. The general security will automatically be improved with the increased market share of newer browsers which support Origin Header.
As the closing comment points it out, we can't do this with secure requests. They need to be essentially checked against the referrer or origin, at least for now. We can not be sure that some untrusted or insecure subdomain has not already set the cookie or cookie domain. To deal with this, we have to consider https separately as it is being done now. So it will be something like:
def process_view(self, request, ....):
# Same initial setup
if request.method not in ('GET', 'HEAD', 'OPTIONS', 'TRACE'):
host = request.get_host()
origin = reqest.META.get('HTTP_ORIGIN', "")
cookie_domain = settings.CSRF_COOKIE_DOMAIN
if request.is_secure():
good_referer = 'https://%s/' % host
referer = origin or request.META.get('HTTP_REFERER')
# Do the same origin checks here
# We are insecure, so care less
# A better way for this check can be used if needed
elif origin.endswith(cookie_domain):
# Safe, continue conventional checking
# Do the conventional checks hereIf the above were to be implemented, the setting CSRF_COOKIE_DOMAIN
should be deprecated for something like CSRF_ALLOWED_DOMAIN which
makes more sense.
###Multiple Allowed Domains (was Better CORS Support) Since, already introducing Origin checking, we can go one step further and try to provide better support for CORS for browsers supporting it. A tuple/list setting, which specifies allowed domains will be provided. Using this the various access control allowance response headers will be set when the request origin is from amongst the allowed domains. For CSRF check, just see if http origin is an allowed domain.
def set_cors_headers(response, origin):
response['Access-Control-Allow-Origin'] = origin
def process_response(self, request, response):
origin = reqest.META.get('HTTP_ORIGIN', "")
if origin in settings.CSRF_ALLOWED_DOMAINS:
set_cors_headers(response, origin)
def process_request(self, request, response):
# Use origin in settings.CSRF_ALLOWED_DOMAINS here instead of
# origin.endswithProbably, something similar to the above will be needed to incorporate the CORS support.
###Less restrictive secure requests
The current CSRF system is pretty much secure as it is. But CSRF protection poses too much restriction to https. It says no to all the request, without honouring any tokens. It kind of has to, thanks to the way browsers allow cookie access. A cookie accessible through subdomains mean that any subdomain secure or insecure can set the CSRF token, which could be really serious for the site security. To get around this, currently one has to completely exempt views from CSRF and may or may not handle CSRF attacks. This can be dangerous. Also if a person has a set of sites, which talk to each other through clients and decides to run it over https, it would need some modifications.
Django should behave under https similarly as it does under http
without compromising any security. So, we need to make sure that the
CSRF token is always set by a trusted site. Signing the data with the
same key, probably settings.SECRET_KEY, across the sites looks apt
for this, using django.core.signing. We can have get_token and
set_token methods which abstract the signing process.
This can be done in two ways:
Store CSRF data in sessions data in case contrib.sessions is
installed. Then the data will automatically be signed with the
secret key or will not be stored in the client as cookies at all.
In case of it being absent from installed apps, revert to custom signing
Encryption?
from django.core.signing import TimestampSigner
signer = TimestampSigner("csrf-token")
CSRF_COOKIE_MAX_AGE = 60 * 60 * 24 * 7 * 52
def get_unsigned_token(request):
# BadSignature exception needs to be handled somewhere
return signer.unsign(request.META.get("CSRF_COOKIE", None)
max_age = CSRF_COOKIE_MAX_AGE)
def set_signed_token(response, token):
response.set_cookie(settings.CSRF_COOKIE_NAME,
signer.sign(request.META["CSRF_COOKIE"]),
max_age = CSRF_COOKIE_MAX_AGE,
domain=settings.CSRF_COOKIE_DOMAIN,
path=settings.CSRF_COOKIE_PATH,
secure=settings.CSRF_COOKIE_SECURE
)
def get_token(request):
if 'django.contrib.sessions' in settings.INSTALLED_APPS:
return request.session.csrf_token
else:
return get_unsigned_token(request)
def set_token(response, token)
if 'django.contrib.sessions' in settings.INSTALLED_APPS:
request.session.csrf_token = token
else:
set_signed_token(response, token)
# Comparing to the token in the request
constant_time_compare(request_csrf_token, get_token(csrf_token))Now, doing this is not as simple as the above code block makes it look. There is a lot which can and probably will go wrong with this approach:
Even when the token is signed, other domains can completely replace the CSRF token cookie, it won't grant them access through CSRF check though. Even with signing, they just need to replay an existing good token/cookie pair, which they can get directly from the server any time they want.
This sort of couples CSRF with sessions, a contrib app. Currently nothing except some of the other contrib apps are tied up with sessions. It will break if sessions were to be removed in future or the API changed. Also, this means that if one website is using sessions CSRF, all of the other must be too. It would actually kind of be a step because of the coupling.
If this were successfully implemented, is this exposing any critical security flaws otherwise? Will it cause compatibility issues?
Encryption itself comes with its own issues. It will need high considerations.
As Paul McMillan said "This is a hard problem", I'll delegate figuring this to future me. I will look into The Tangled Web and Google's Browser Security Handbook for ideas, again suggested by Paul on the IRC.
##Centralized tokenization There are multiple places in django which use some or other kinds of tokens:
Token generation is pretty common around the framework. So, instead
of each application having its own token system, and hence needs to be
maintained separately. There should be centralized token system, which
provides an abstract API for everyone to loose. In fact, I have seen
that some apps use User.objects.make_random_password from
contrib.auth, which they can be sure of being maintained in the future
for random generation. To me this looks kind of weird.
In last djangocon, a lot of work regarding this was done over Yarko's
Fork.
I had a discussion with Yarko Tymciurak regarding this. The work is nearly ready for a merge, only some tasks left. I can work over these to insure that the already done significant work gets in django and is updated for 1.5.
I will study the changes done at djangocon and then attempt the tasks mentioned above.
##Integrating django-secure A really useful app for catching security configuration related mistakes is [carljm's django-secure][djang-secure]. It is specially useful to find out issues that might have been introduced while quick changes to settings for development. This project is popular and useful enough that it can be shipped with django. I haven't been able give this enough time yet. I can think of two ways of integrating this:
Dropping it as a contrib app
This seems pretty straight forward would require minimal amount of
changes.
Distribute around the framework:
Like CSRF, this can also be distributed framework wide and hence it
won't be optional to have. Apps can still define custom checks in
the same way when django-secure was installed as a pluggable
application.
The app might also need some changes whilst being integrated:
#Plan I think that the tasks CSRF enhancements and centralized tokenization will be enough to span through the SoC period. If after a thorough implementation and testing of these, I still have time, django-secure integration can be looked into.
Roughly this proposal can span over a maximum of 5 tasks. Each task will generally have the following steps:
a. Initial Research. Design decisions
b. Implementation with minor parallel tests.
c. Thorough and regression testing to to achieve security quality.
d. Configuration/Settings changes and handle compatibility issues.
e. Documentation.
Tasks (with most effort requiring steps in parenthesis):
I'll be using my fork of django over github. I'll probably use the following branch names: csrf-enhancements (origin checking, multiple request domains etc) centralized-tokenization (djangocon2011-sec)
##Timeline
Week 1: Task 1.a, 1.b.
Week 2: Task 1.c, 1.d
Week 3: Task 2.a, 2.b. Start task 3.a
Week 4: Task 2.c, 2.d
Week 5: Task 1.e, 2.e (Doing these together might be beneficial)
Week 6-7: Complete 3.a. Task 3.b
Week 7-9: Task 3.c, 3.d
Week 10: Task 3.e
Week 11-12: Tasks 4.abcde (max possible)
Week 13: Complete Task 4 and maybe Max of Task 5
I am sorry for writing these as if written by a bot, the deadline was so close so had to adopt this method.
##Resources
###Tasks
Ticket for origin header checking support: #16010
The most recent patch in this ticket forces the origin check without
considering CSRF_COOKIE_DOMAIN setting, which seems the reason for the
patch to be not accepted.
CSRF Improvement ticket: #16859
Adding CSRF to sessions:
In case a project has sessions app included, using of session cookie can
be beneficial, as we won't need to use an extra cookie for csrf. Also this
automatically saves for any chance of tempering with the token in whatever
session store is being used, db or cookie.
Signing of CSRF token:
Complicating CSRF cookies would mean to have two places which rely on
browsers to be maintained
In case the session app is there and the above sessions is used with csrf,
there won't be any need to do separate sigining. Otherwise, the cookie can
be signed, probably using django signing utility like it is done in
sessions signed cookie store.
Integrating django-secure
I looked into its source code and was pretty much able to understand the
app. Should it be integrated as a contirb app, maybe contrib.security?
Centralizing randomized token issuance and validation:
Some work done over it in DjangoCon 2011, at https://github.com/yarko/django.
TODOs: https://github.com/yarko/django/blob/djangocon2011-sec/README.sec
Merging, Testing and Documentation
###External
Python Security django page: Some posts about security regarding django.
Origin Header: Origin request header usable for CSRF checking. Not implemented in all browsers. Mozilla page, W3 spec
###Mailing List Post about info leakage (I am unsure about it and its old).
Django has developed many security features over time. The existing set of security features is pretty good, but there's lots of room for improvement. Much of the work in this project will be related to cleaning up existing code to make it more obviously secure, eliminate edge cases, and and improve fallback handling.
Some potential areas of work include:
While an interest in security will make these tasks more interesting, most of them don't require you to be a security expert already. Your mentor will make sure your plan is correct before you code, and carefully review your work before it is committed to trunk. Most of these tasks will be significantly easier if you already have some familiarity with Django's codebase. A successful application will have a plan which selects related areas of work, provides details, and has a good estimation of complexity for the proposed tasks. Remember that (especially for security work) a good patch often has more lines of tests than code changes. An ideal applicant will be able to demonstrate the skill with Python and attention to detail necessary to make fundamental changes to Django without breaking existing code.
Ideas that will probably not be accepted:
If you are interested in working on this project, please talk to us sooner rather than later! PaulM is usually available on IRC, and wants to help you write a really good application.
[CSRF Protection Page][csrf-proection]
Centralized Tokenization
Tokens strings are common in web projects and applications. The tokenization
library provides an abstraction over hashlib and random.
It provides two kind of tokens, RandomToken for random strings and HashToken for hashed strings:
RandomToken([length])
Class for generating randomized tokens. Has an optional argument length,
which defaults to 32.
HashToken([value, algorithm])
Class for generating hashed tokens based on an initial string. Has optional
arguments:
value (default ''): The string input, a hashed version of which is
required.algorithm (default 'sha256'): The hashing algorithm to be used. Options
are - md5, sha1, sha256.A token object provides a methods to retrieve token strings in multiple ways:
token.hex()
Token in hex character set
token.digits()
Token consisting of only digits.
token.alphanumeric()
Token consisting of alphanumeric characters (digits, uppercase alphabets,
lowercase alphabets).
token.lower_alphanumeric()
Token consisting of alphanumeric characters (digits, uppercase alphabets,
lowercase alphabets).
token.readable_alphabet()
String which has a subset of alphanumeric characters easily distinguishable by
humans from one another.
token.custom_chars(chars)
Takes in a string of custom character set in which the token string should be
returned.
Apart from these, a HashToken object also has two more methods/properties:
hash_token.update(value)
Update the value associated with hash object.
hash_token.digestmod
The hashlib constructor which returns a new hash objects. Useful to feed in
to functions like hamc.new(key[, msg[, digestmod]]).