FixedOffset serialization fails

test_fo.py

"""
Save this file as test_fo.py and run it
"""

import pprint
import json
import datetime
import pickle

jsonpickle = None
bson = None

def setup_module(module):
	module.jsonpickle = __import__('jsonpickle')
	module.bson = __import__('bson.tz_util')

def test_FixedOffsetSerializable():
	fo = bson.tz_util.FixedOffset(-60*5, 'EST')
	serialized = jsonpickle.dumps(fo)
	pprint.pprint(json.loads(serialized))
	restored = jsonpickle.loads(serialized)
	print(restored._FixedOffset__offset)
	assert vars(restored) == vars(fo)

def test_timedelta():
	td = datetime.timedelta(-1, 68400)
	serialized = jsonpickle.dumps(td)
	pprint.pprint(json.loads(serialized))
	restored = jsonpickle.loads(serialized)
	assert restored == td

def test_stdlib_pickle():
	fo = bson.tz_util.FixedOffset(-60*5, 'EST')
	serialized = pickle.dumps(fo)
	pprint.pprint(serialized)
	restored = pickle.loads(serialized)
	print(restored._FixedOffset__offset)
	assert vars(restored) == vars(fo)

class FixedOffset(datetime.tzinfo):
    def __init__(self, offset):
        self.offset = datetime.timedelta(offset)

    def __getinitargs__(self):
        return self.offset,

    def utcoffset(self, dt):
        return self.offset

    def tzname(self, dt):
        return 'name'

    def dst(self, dt):
        return datetime.timedelta(0)

def test_nested_objects():
	o = FixedOffset(99)
	serialized = jsonpickle.dumps(o)
	pprint.pprint(json.loads(serialized))
	restored = jsonpickle.loads(serialized)
	assert restored.offset == datetime.timedelta(99)

def test_datetime_with_fixed_offset():
	fo = FixedOffset(-60)
	dt = datetime.datetime.now().replace(tzinfo=fo)
	serialized = jsonpickle.dumps(dt)
	pprint.pprint(json.loads(serialized))
	restored = jsonpickle.loads(serialized)
	assert restored == dt


setup_params = dict(
	install_requires=[
		'jsonpickle',
		'pymongo',
	],
	setup_requires=[
		'pytest_runner',
	],
	tests_require=[
		'pytest',
	],
)

if __name__ == '__main__':
	import sys
	sys.argv[1:1] = ['pytest']
	__import__('setuptools').setup(**setup_params)

I've started taking a look. I also noticed that test_handles_cyclical_objects was broken by d7f84a27cac0dc94fdb705a0f8e13dbf240069b2 as well. It tests tuples that contain cyclical references.

One hole in the referencing scheme that has always been an especially tricky problem are tuples, because they are immutable. The hard part is that we need to make a reference to the object available so that restoring the contents of the tuple will allow those objects to create cycles back to the actual object. Thus, the tuple must exist before its items, which is a paradox since the empty tuple can never becomes a tuple-with-items and keep the same identity.

This is hole too strong for the proxy object scheme because if a tuple ends up containing a proxy then we can't swap it out later.

But, I think there is one gnarly and magical hack that could be the basis for how to deal with tuple's immutability. Instead of returning tuples, we could return a subclass of a tuple that is duck-type enough like a tuple that, as long as we document it, might just be good enough for the 90% use case.

The trick is as follows. The reason lists are not a problem is that we can create a list, create its contents, and then extend the list with the content because a list is mutable. If we could do the same with a tuple then there's no problem.

So how can we create an object that behaves like a list, and then later is swapped in-place to becomes a tuple that prevents item assignment? This is kinda crazy, and is definitely in the stupid python tricks category, but I guess it actually has a use case here...

class List(list):
    pass

class Tuple(tuple):
    pass

# Let's create an object that we can modify, and then coerce it into becoming tuple
xenomorph = List()
# It can even reference itself
xenomorph.extend([xenomorph, xenomorph])
# And then it becomes a tuple...
xenomorph.__class__ = Tuple

I'll need to rework that test case, but this just might work, so I'm going to try that. I have some preliminary cleanup stuff that I'll push out shortly, then I'll be going down this rabbit hole. Hopefully it works, we'll see...

Ugh, nevermind, that totally doesn't work.. I'll keep digging. I was misled when I wrote some test code that made me believe that the above works, but it doesn't. doh.

TypeError: __class__ assignment: 'list' object layout differs from 'tuple'

Going to try something like this... https://mail.python.org/pipermail/python-dev/2003-January/032043.html we'll see.

# crash-python.py
class Tuple(tuple):
    __slots__ = ()

class List(list):
    __slots__ = ()

t = List()
t.__class__ = Tuple
# --- >8 --- >8 ---

$ python2.7 crash-python.py
Segmentation fault: 11

$ python3.4 crash-python.py
TypeError: __class__ assignment: 'List' object layout differs from 'Tuple'

Oh well, I'll probably try a different approach, like maybe returning an instance that behaves like a tuple, but without actually being an instance. That's unfortunate, though.