Skip to content

Instantly share code, notes, and snippets.

View GrahamDumpleton's full-sized avatar
😎

Graham Dumpleton GrahamDumpleton

😎
View GitHub Profile
@GrahamDumpleton
GrahamDumpleton / gist:4115734
Created November 20, 2012 03:17
Dealing with overridden Resource.wrap_view in TastyPie.
import newrelic.hooks.component_tastypie
...
def wrap_view(self, view):
...
wrap_view = newrelic.hooks.component_tastypie.ObjectWrapper(wrap_view,
None, newrelic.hooks.component_tastypie.outer_fn_wrapper)
@GrahamDumpleton
GrahamDumpleton / gist:4083833
Created November 16, 2012 03:41
Processing profile data.
import pstats
import glob
import sys
filenames = sys.argv[1:]
# Read all stats files into a single object
stats = pstats.Stats(filenames[0])
for i in filenames[1:]:
stats.add(i)
@GrahamDumpleton
GrahamDumpleton / gist:4083818
Created November 16, 2012 03:40
Using function profile decorator.
[function-profile:profile-1]
enabled = true
function = module:handler
filename = /tmp/profile-%(pid)s.dat
delay = 1.0
checkpoint = 5
@GrahamDumpleton
GrahamDumpleton / gist:4064096
Created November 13, 2012 05:26
Suppress requests module connection logging.
import logging
class RequestsConnectionFilter(logging.Filter):
def filter(self, record):
return False
logging.getLogger('newrelic.lib.requests.packages.urllib3.connectionpool').addFilter(RequestsConnectionFilter())
@GrahamDumpleton
GrahamDumpleton / gist:4043458
Created November 9, 2012 03:08
Context manager getattr fix.
from __future__ import with_statement
class CM1(object):
def __enter__(self):
print '__enter__'
return self
def __exit__(self, *args):
print '__exit__'
class CM2(object):
@GrahamDumpleton
GrahamDumpleton / gist:4043434
Created November 9, 2012 02:59
Context manager getattr test.
# Python 2.5, 2.6 runs okay.
# Python 2.7:
#
# Traceback (most recent call last):
#  File "test.py", line 33, in <module>
#    with cm2:
# AttributeError: __exit__
from __future__ import with_statement
# Run this as:
#
# NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-python example.py
#
# If being run under Heroku, you can skip setting NEW_RELIC_CONFIG_FILE on the
# command line as the required environment variable settings added by the
# Heroku New Relic addon will be picked up automatically.
import time
import newrelic.agent
@GrahamDumpleton
GrahamDumpleton / gist:4034982
Created November 7, 2012 22:35
Notes of set isolation level.
Update on this - turned out I was hitting this issue:
http://stackoverflow.com/questions/8198990/pgbouncer-closing-because-unclean-server-on-every-connection
We weren't getting any of the benefit of pgbouncer because pgbouncer was closing all connections when they were returned to the pool since they were still dirty. Supposedly the best way to fix this is to use autocommit in django, however that requires a careful audit of wherever you are doing db updates and wrapping that code with something like the @commit_on_success decorator. Since all of our API calls have the possibility of doing an update, I could take the easy way out and just add django.middleware.transaction.TransactionMiddleware to my list of middleware classes. Now I'm getting connection reuse from pgbouncer, and not incurring the overhead of establishing a new connection with each db request. Big improvement, and I'm no longer seeing the strange long periods of time in set_isolation_level (which were obviously due to setting up a connect
@GrahamDumpleton
GrahamDumpleton / gist:4034410
Created November 7, 2012 20:59
Hacking Django code to work out why database connection setup is slow.
I changed the django code in django/db/backends/postgresql_psycopg2/base.py to:
self.connection = Database.connect(**conn_params)
with newrelic.agent.FunctionTrace(newrelic.agent.current_transaction(), 'Set client encoding'):
self.connection.set_client_encoding('UTF8')
with newrelic.agent.FunctionTrace(newrelic.agent.current_transaction(), 'Set isolation level'):
self.connection.set_isolation_level(self.isolation_level)
with newrelic.agent.FunctionTrace(newrelic.agent.current_transaction(), 'connection_created signal'):
connection_created.send(sender=self.__class__, connection=self)
if new_connection:
@GrahamDumpleton
GrahamDumpleton / gist:3930594
Created October 22, 2012 09:26
Script for validating data return for explain plan.
import MySQLdb
def doit(database, query):
connection = MySQLdb.connect(db=database)
try:
cursor = connection.cursor()
try:
cursor.execute(query)
columns = []
if cursor.description: