Skip to content

Instantly share code, notes, and snippets.

View GrahamDumpleton's full-sized avatar
😎

Graham Dumpleton GrahamDumpleton

😎
View GitHub Profile
@GrahamDumpleton
GrahamDumpleton / gist:3564216
Created September 1, 2012 05:04
PyCon US 2013 Talk Proposal 1

Title

Why Apache sucks for hosting Python web applications.

Category

Web Frameworks

Sed in nulla eu est vestibulum convallis. Praesent ante magna, molestie vel porttitor vel, aliquet ut dolor. Proin molestie massa eu tellu.

I am lazy when it comes to searching though documentation on hosting providers web sites. HHG was therefore awesome for me as it brought all the important stuff I needed to know into one place. An indispensable quick reference guide.

I can be lazy when it comes to searching web documentation. HHG is awesome as it combines the important stuff I need to know in one place.

@GrahamDumpleton
GrahamDumpleton / gist:3652313
Created September 6, 2012 06:58
Nova instrumentation for New Relic
import types
import sys
from newrelic.api.object_wrapper import callable_name
from newrelic.api.error_trace import wrap_error_trace
from newrelic.api.in_function import wrap_in_function
from newrelic.api.transaction_name import wrap_transaction_name
from newrelic.api.function_trace import wrap_function_trace, FunctionTraceWrapper
from newrelic.api.external_trace import wrap_external_trace
from newrelic.api.web_transaction import WSGIApplicationWrapper
@GrahamDumpleton
GrahamDumpleton / gist:3652331
Created September 6, 2012 07:00
Agent configuration for Nova instrumentation
# Import hooks for nova.
[import-hook:nova.wsgi]
enabled = true
execute = nova-api-instrumentation:instrument_nova_wsgi
[import-hook:nova.api.openstack.wsgi]
enabled = true
execute = nova-api-instrumentation:instrument_nova_api_openstack_wsgi
@GrahamDumpleton
GrahamDumpleton / gist:3815596
Created October 2, 2012 01:16
PyCon US 2013 Talk Proposal 2

Title

How well do you really understand the WSGI specification?

Category

Web Frameworks

@GrahamDumpleton
GrahamDumpleton / gist:3909828
Created October 18, 2012 04:16
Additional ObjectWrapper methods.
def __eq__(self, other):
if isinstance(other, ObjectWrapper):
return self._nr_last_object == other._nr_last_object
return self._nr_last_object == other
def __ne__(self, other):
result = self.__eq__(other)
if result is NotImplemented:
return result
return not result
@GrahamDumpleton
GrahamDumpleton / gist:3915819
Created October 19, 2012 01:49
Tracking a one off task run from a script as a background job.
# Run this as:
#
# NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-python example.py
#
# If being run under Heroku, you can skip setting NEW_RELIC_CONFIG_FILE on the
# command line as the required environment variable settings added by the
# Heroku New Relic addon will be picked up automatically.
import time
import newrelic.agent
@GrahamDumpleton
GrahamDumpleton / gist:3930594
Created October 22, 2012 09:26
Script for validating data return for explain plan.
import MySQLdb
def doit(database, query):
connection = MySQLdb.connect(db=database)
try:
cursor = connection.cursor()
try:
cursor.execute(query)
columns = []
if cursor.description:
@GrahamDumpleton
GrahamDumpleton / gist:4034410
Created November 7, 2012 20:59
Hacking Django code to work out why database connection setup is slow.
I changed the django code in django/db/backends/postgresql_psycopg2/base.py to:
self.connection = Database.connect(**conn_params)
with newrelic.agent.FunctionTrace(newrelic.agent.current_transaction(), 'Set client encoding'):
self.connection.set_client_encoding('UTF8')
with newrelic.agent.FunctionTrace(newrelic.agent.current_transaction(), 'Set isolation level'):
self.connection.set_isolation_level(self.isolation_level)
with newrelic.agent.FunctionTrace(newrelic.agent.current_transaction(), 'connection_created signal'):
connection_created.send(sender=self.__class__, connection=self)
if new_connection:
@GrahamDumpleton
GrahamDumpleton / gist:4034982
Created November 7, 2012 22:35
Notes of set isolation level.
Update on this - turned out I was hitting this issue:
http://stackoverflow.com/questions/8198990/pgbouncer-closing-because-unclean-server-on-every-connection
We weren't getting any of the benefit of pgbouncer because pgbouncer was closing all connections when they were returned to the pool since they were still dirty. Supposedly the best way to fix this is to use autocommit in django, however that requires a careful audit of wherever you are doing db updates and wrapping that code with something like the @commit_on_success decorator. Since all of our API calls have the possibility of doing an update, I could take the easy way out and just add django.middleware.transaction.TransactionMiddleware to my list of middleware classes. Now I'm getting connection reuse from pgbouncer, and not incurring the overhead of establishing a new connection with each db request. Big improvement, and I'm no longer seeing the strange long periods of time in set_isolation_level (which were obviously due to setting up a connect