Skip to content

Instantly share code, notes, and snippets.

View ncoghlan's full-sized avatar

Alyssa Coghlan ncoghlan

View GitHub Profile
@ncoghlan
ncoghlan / interactive-example.txt
Created July 24, 2018 15:28
API renames, aliasing, and cross-version pickle compatibility
>>> class C:
... def example(self): pass
...
>>> import pickle
>>> data = pickle.dumps(C.example)
>>> pickle.loads(C.example)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: a bytes-like object is required, not 'function'
>>> pickle.loads(data)
@ncoghlan
ncoghlan / it-reference.md
Last active February 16, 2018 02:10
PEP 505 alternative: the "?it" symbolic reference

Using ?it as a statement local symbolic reference

Core concept

  • (?it=expr) is a new atomic expression for an "it reference binding"
  • subsequent subexpressions (in execution order) can reference the bound subexpression using ?it (an "it reference")
  • ?it is reset between statements, including before entering the suite within a compound statement (if you want a persistent binding, use a named variable)
@ncoghlan
ncoghlan / LICENSE
Last active July 26, 2017 03:37
Auto-defined named tuples in Python 3.6+
Copyright 2017 Nicholas Coghlan. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are
permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of
conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list
of conditions and the following disclaimer in the documentation and/or other materials
@ncoghlan
ncoghlan / check_float_rounding.py
Created July 9, 2017 14:56
Checking Python's rounding behaviour
# Prompted by this article about implementing "Round half away from zero" in Go
# https://www.cockroachlabs.com/blog/rounding-implementations-in-go/
# Python uses round-half-even by default to reduce statistical bias,
# but round-half-away-from-zero can be implemented more efficiently
# (since it doesn't need to examine all the bits of the number)
# I figured the folks doing computational analysis in Python would
# have made sure this was handled correctly years ago, but it never
# hurts to check that kind of assumption :)
@ncoghlan
ncoghlan / pep-0538.rst
Last active May 9, 2017 11:39
PEP 538: Diffs for review

PEP: 538 Title: Coercing the legacy C locale to a UTF-8 based locale Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <[email protected]> BDFL-Delegate: INADA Naoki Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Dec-2016

@ncoghlan
ncoghlan / cpython_kwargs.sh
Last active December 19, 2015 18:09
This isn't really news, but just putting some numbers on why relying on passing data in kwargs doesn't generally scale well: there ends up being a lot of implicit copying to avoid accidental mutation of unowned objects. In general, if speed or memory usage matters and you potentially have a lot of entries in a map, avoid using kwargs as the sole…
# Using even one additional keyword arg forces CPython to make an additional copy of the entire original keyword dictionary
$ python3 -m timeit -s "kwds = {str(k):v for k, v in enumerate(range(1000))}" -s "def f(**kwargs): return kwargs" "f(**kwds)"
10000 loops, best of 3: 50.5 usec per loop
$ python3 -m timeit -s "kwds = {str(k):v for k, v in enumerate(range(1000))}" -s "def f(**kwargs): return kwargs" "f(other=1, **kwds)"
10000 loops, best of 3: 82.4 usec per loop
# Using a non-dict subclass forces CPython to copy it to a real dictionary as part of the call
$ python3 -m timeit -s "import collections; kwds = collections.UserDict((str(k), v) for k, v in enumerate(range(1000)))" -s "def f(**kwargs): return kwargs" "f(**kwds)"
1000 loops, best of 3: 573 usec per loop
@ncoghlan
ncoghlan / Crazy namespace, Python 2
Last active December 19, 2015 03:38
Python namespaces with side effects
>>> class Madness(dict):
... def __setitem__(self, attr, value):
... if isinstance(value, type):
... value.__name__ = attr
... dict.__setitem__(self, attr, value)
...
>>> code = """\
... class Example(object): pass
... NewName = Example
... print(Example.__name__)
@ncoghlan
ncoghlan / sort_mixed_types.py
Last active December 18, 2015 06:59
Replicate 2.x arbitrary sorting logic in Python 3
class MixedTypeKey:
"""Sort key for mixed type lists in Python 3
>>> sorted([None, ..., 1, 1j, "", (), {}, []])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: ellipsis() < NoneType()
>>> sorted([None, ..., 1, 1j, "", (), {}, []], key=MixedTypeKey)
[None, Ellipsis, 1, 1j, '', (), {}, []]
"""
@ncoghlan
ncoghlan / gist:3991579
Created November 1, 2012 03:52
Workaround github's broken comment system
Github won't let me comment on https://github.com/pculture/unisubs/commit/e41f74fa5a17573b08001cc1431b615abca930ba
so just saving this here for now.
unisubs is hitting the double import problem that is caused by Django's old default project layout and management script, which creates a broken Python import configuration. Django's default behaviour has been fixed (https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py) but projects using the old layout may still see the issue.
The only reliable solution is to fix the layout to avoid the broken configuration that allows the same module to be imported under two different names. The simplest way to do this is to:
1. Remove any sys.path manipulation from manage.py
2. Ensure that manage.py is *not* inside a Python package (there should *not* be an adjacent __init__.py file)
3. Adjust imports as needed for any changed made in steps 1 and 2
@ncoghlan
ncoghlan / helpers.py
Created October 16, 2012 12:02
Tentative Python 3.4 async example
from concurrent.futures import Future
# This first part is a helper function to wait for the first Future in a group to complete
def _wait_first(futures):
# futures must be a set as items will be removed as they complete
# we create a signalling future to return to our caller. We will copy
# the result of the first future to complete to this signalling future
signal = Future()
signal.completed = None
def copy_result(completed):