Skip to content

Instantly share code, notes, and snippets.

@cicdw
Created April 6, 2020 21:42
Show Gist options
  • Save cicdw/d5b566a9dfb4e4dd7e90a35b310df3be to your computer and use it in GitHub Desktop.
Save cicdw/d5b566a9dfb4e4dd7e90a35b310df3be to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from prefect.engine.result_handlers import ResultHandler\n",
"\n",
"\n",
"class MyHandler(ResultHandler):\n",
" def read(self, loc):\n",
" print(loc)\n",
" \n",
" def write(self, val):\n",
" print(val)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Above we created a custom Result Handler class with custom logic. When we send data handled with this handler to our database, we first deserialize it into a JSON representation:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'type': 'CustomResultHandler', '__version__': '0.10.0+116.gcade26c0'}"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from prefect.serialization.result_handlers import ResultHandlerSchema\n",
"\n",
"schema = ResultHandlerSchema()\n",
"jsonified = schema.dump(MyHandler())\n",
"jsonified"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Because this handler is completely custom, we don't know how to correctly serialize it, so we just blanket call it \"Custom\". Whenever Core receives any data which was handled with this handler, we don't know how to retrieve the underlying implementation and so we deserialize the custom payload to an empty result handler class:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<ResultHandler: ResultHandler>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"deserialized_handler = schema.load(jsonified)\n",
"deserialized_handler"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In general, we might have serialized a few universal attributes (e.g., `filename`) but otherwise don't actually know how to interact with the custom implementation. This is where the `hydrate` method comes in: we seek to preserve all of the serialized attributes that we know about and \"hydrate\" all the class methods with the user's actual implementation."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.7"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment