Skip to content

Instantly share code, notes, and snippets.

@williballenthin

williballenthin/cortex-benchmarking.ipynb

Last active May 19, 2017 09:00
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save williballenthin/12ea055f0e862adba3a1d2044e5fc97f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save williballenthin/12ea055f0e862adba3a1d2044e5fc97f to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
cortex-benchmarking.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"# cortex benchmark\n",
"\n",
"need to be able to quantitatively describe which cortex backend should be used for what task. is a `ram://` cortex really faster than `postgres://`? what about `sqlite://` vs `lmdb://`?\n",
"\n",
"want to test:\n",
" - insert, with and without many duplicates.\n",
" - tufos of varying size (number of props).\n",
" - query for existing tufos.\n",
" - `getTufosByProp()`\n",
" - range queries\n",
" - updates to existing tufos.\n",
" - deletions of tufos.\n",
" - row APIs.\n",
" \n",
" \n",
"probably mainly want to record wall time. but maybe also mem usage. disk usage."
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"## data sources\n",
"\n",
"stanford has collected a bunch of interesting graph datasets: https://snap.stanford.edu/data/\n",
"some are really big. others are simply vertices and edges.\n",
"\n",
"the reddit submission dataset tracks about 130,000 submissions to reddit. its available here: https://snap.stanford.edu/data/web-Reddit.html the metadata is 7.3 MB compressed, 17.1 MB uncompressed CSV.\n",
"\n",
"download and decompress it into the same directory as this notebook. the filename should be `redditSubmissions.csv`."
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"## data schema:\n",
"\n",
"as described on the dataset page, the schema for this data is:\n",
"\n",
" - `image_id`: id of the image, submissions with the same id are of the same image\n",
" - `unixtime`: time of the submission (unix time)\n",
" - `rawtime`: raw text of the time\n",
" - `title`: submission title\n",
" - `total_votes`: number of upvotes + number of downvotes\n",
" - `reddit_id`: id of the submission on reddit, e.g. reddit.com/14c3ls\n",
" - `number_of_upvotes`: number of upvotes\n",
" - `subreddit`: subreddit, e.g. reddit.com/r/pics/\n",
" - `number_of_downvotes`: number of downvotes\n",
" - `localtime`: local time of the submission (unix time)\n",
" - `score`: number of upvotes - number of downvotes\n",
" - `number_of_comments`: number of comments the submission received\n",
" - `username`: name of the user who submitted the image e.g. www.reddit.com/user/thatseffedup\n",
"\n",
"\n",
"### example data:\n",
"```\n",
"#image_id,unixtime,rawtime,title,total_votes,reddit_id,number_of_upvotes,subreddit,number_of_downvotes,localtime,score,number_of_comments,username\n",
"1005,1335861624,2012-05-01T15:40:24.968266-07:00,I immediately regret this decision,27,t296r,20,pics,7,1335886824,13,0,ninjaroflmaster\n",
"1005,1336470481,2012-05-08T16:48:01.418140-07:00,\"Pushing your friend into the water,Level: 99\",18,tds4i,16,funny,2,1336495681,14,0,hme4\n",
"1005,1339566752,2012-06-13T12:52:32.371941-07:00,I told him. He Didn't Listen,6,v0cma,4,funny,2,1339591952,2,0,HeyPatWhatsUp\n",
"1005,1342200476,2012-07-14T00:27:56.857805-07:00,Don't end up as this guy.,16,wjivx,7,funny,9,1342225676,-2,2,catalyst24\n",
"1005,1342485280,2012-07-17T07:34:40.225147-07:00,last one in is a rot...oh shit,9987,wpar7,5633,pics,4354,1342510480,1279,153,phillythebeaut\n",
"1005,1342499862,2012-07-17T11:37:42-07:00,Dat t[o]ngue,139,wpq35,134,whalebait,5,1342525062,129,10,clicksnd\n",
"1005,1342788847,2012-07-20T19:54:07.898464-07:00,When I try to break the ice with asmooth joke,6,wwocv,3,funny,3,1342814047,0,0,LHeeezy\n",
"1005,1343926973,2012-08-03T00:02:53-07:00,Whalebait (x-post from r/whalebait),53,xlyyr,45,PerfectTiming,8,1343952173,37,5,thatseffedup\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [],
"source": [
"import os\n",
"import csv\n",
"\n",
"def reformat_row(row):\n",
" '''\n",
" do some cleanup on a row, formatting some fields to work with synapse.\n",
" \n",
" Example::\n",
" \n",
" for row in csv.DictReader(f):\n",
" yield reformat_row(row)\n",
" '''\n",
" # the csv header labels the `image_id` field with a leading hash character.\n",
" # strip this.\n",
" if '#image_id' in row:\n",
" iid = row['#image_id']\n",
" del row['#image_id']\n",
" row['image_id'] = iid\n",
" \n",
" # `unixtime` has to be a number, or `TimeType.parse()` will be called, and\n",
" # then the format should be something like \"YYMMDDHHMMSS\".\n",
" if 'unixtime' in row:\n",
" row['unixtime'] = int(row['unixtime'] or 0)\n",
" \n",
" # there are some rows with extra columns at the end.\n",
" # remove those (synapse doesn't like props named `None`).\n",
" if None in row:\n",
" del row[None]\n",
" \n",
" # the remaining int fields can be converted by synapse just fine.\n",
" # so, don't touch those.\n",
" \n",
" return row\n",
"\n",
"def get_data(csvpath, limit=None):\n",
" '''\n",
" load reformatted rows from a CSV file.\n",
" returns a list (not generator); everything is loaded up front.\n",
" \n",
" Example::\n",
" \n",
" for row in get_data('redditSubmissions.csv'):\n",
" print(row['image_id'])\n",
" '''\n",
" # need to use textual mode for csvreader :-(\n",
" with open(DATA_FILE, 'r') as f:\n",
" reader = csv.DictReader(f, delimiter=',')\n",
" rows = [reformat_row(row) for row in reader]\n",
" if limit:\n",
" rows = rows[:min(len(rows), limit)]\n",
" return rows\n",
" \n",
"DATA_FILE = os.path.join('.', 'redditSubmissions.csv')\n",
"\n",
"# on my i7 laptop, with 10k submissions, \n",
"# tests take a few dozen seconds to a few minutes each.\n",
"DATA_ROWS = get_data(DATA_FILE, limit=10000)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"data": {
"text/plain": [
"{'image_id': '0',\n",
" 'localtime': '1333197639',\n",
" 'number_of_comments': '622',\n",
" 'number_of_downvotes': '30813',\n",
" 'number_of_upvotes': '32657',\n",
" 'rawtime': '2012-03-31T12:40:39.590113-07:00',\n",
" 'reddit_id': 'rmqjs',\n",
" 'score': '1844',\n",
" 'subreddit': 'funny',\n",
" 'title': \"And here's a downvote.\",\n",
" 'total_votes': '63470',\n",
" 'unixtime': 1333172439,\n",
" 'username': 'Animates_Everything'}"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"DATA_ROWS[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### synapse data model"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true,
"deletable": true,
"editable": true
},
"outputs": [],
"source": [
"DATAMODEL = {\n",
" 'prefix': 'syn:bench:reddit',\n",
" 'types': [\n",
" ('syn:bench:reddit:id', {'subof': 'str:lwr'}),\n",
" ],\n",
" 'forms': [\n",
" ('syn:bench:reddit:submission', {'ptype': 'syn:bench:reddit:id'}, [\n",
" ('unixtime', {\n",
" 'ptype': 'time:epoch',\n",
" 'doc': 'time of the submission',\n",
" }),\n",
" ('rawtime', {\n",
" 'ptype': 'str:lwr',\n",
" 'doc': 'raw text of the time',\n",
" }),\n",
" ('title', {\n",
" 'ptype': 'str',\n",
" 'doc': 'submission title',\n",
" }),\n",
" ('username', {\n",
" 'ptype': 'str',\n",
" 'doc': 'name of the user who submitted the image e.g. www.reddit.com/user/thatseffedup',\n",
" }),\n",
" ('subreddit', {\n",
" 'ptype': 'str:lwr',\n",
" 'doc': 'subreddit, e.g. reddit.com/r/pics/',\n",
" }),\n",
" ('image_id', {\n",
" 'ptype': 'int',\n",
" 'doc': 'id of the image, submissions with the same id are of the same image',\n",
" }),\n",
" ('score', {\n",
" 'ptype': 'int',\n",
" 'doc': 'number of upvotes - number of downvotes',\n",
" }),\n",
" ('number_of_comments', {\n",
" 'ptype': 'int',\n",
" 'doc': 'number of comments the submission received',\n",
" }),\n",
" ('total_votes', {\n",
" 'ptype': 'int',\n",
" 'doc': 'number of upvotes + number of downvotes',\n",
" }),\n",
" ('number_of_upvotes', {\n",
" 'ptype': 'int',\n",
" 'doc': 'number of upvotes',\n",
" }),\n",
" ('number_of_downvotes', {\n",
" 'ptype': 'int',\n",
" 'doc': 'number of downvotes',\n",
" }),\n",
" ]), \n",
" ],\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"## insert some data"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"first, just demonstrate the data model works:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"('d95dde162e16ee1025752e0596162034',\n",
" {'.new': True,\n",
" 'syn:bench:reddit:submission': 'rmqjs',\n",
" 'syn:bench:reddit:submission:image_id': 0,\n",
" 'syn:bench:reddit:submission:localtime': '1333197639',\n",
" 'syn:bench:reddit:submission:number_of_comments': 622,\n",
" 'syn:bench:reddit:submission:number_of_downvotes': 30813,\n",
" 'syn:bench:reddit:submission:number_of_upvotes': 32657,\n",
" 'syn:bench:reddit:submission:rawtime': '2012-03-31t12:40:39.590113-07:00',\n",
" 'syn:bench:reddit:submission:reddit_id': 'rmqjs',\n",
" 'syn:bench:reddit:submission:score': 1844,\n",
" 'syn:bench:reddit:submission:subreddit': 'funny',\n",
" 'syn:bench:reddit:submission:title': \"And here's a downvote.\",\n",
" 'syn:bench:reddit:submission:total_votes': 63470,\n",
" 'syn:bench:reddit:submission:unixtime': 1333172439,\n",
" 'syn:bench:reddit:submission:username': 'Animates_Everything',\n",
" 'tufo:form': 'syn:bench:reddit:submission'})\n"
]
}
],
"source": [
"# synapse imports.\n",
"# stolen from synapse tutorial: https://gist.github.com/williballenthin/d3de15fb701fef2259d41546c6dc377d\n",
"\n",
"# its available via pypi as `viv-synapse`, but not too often updated.\n",
"# its best to fetch from github, like:\n",
"# pip install https://github.com/vivisect/synapse/zipball/master\n",
"import synapse\n",
"import synapse.cortex\n",
"from synapse.common import tufo\n",
"\n",
"IDEN = 0\n",
"PROPS = 1\n",
"\n",
"# temporary cortex we'll use to demonstrate things are working\n",
"core = synapse.cortex.openurl('ram://')\n",
"core.addDataModel('cortex.benchmarking.does.not.exist', DATAMODEL)\n",
"\n",
"\n",
"row = DATA_ROWS[0]\n",
"tuf0 = core.formTufoByFrob('syn:bench:reddit:submission', row['reddit_id'], **row)\n",
"\n",
"from pprint import pprint\n",
"pprint(tuf0)\n",
"\n",
"assert tuf0[PROPS].get('.new') == True\n",
"assert type(tuf0[PROPS].get('syn:bench:reddit:submission:image_id')) == int\n",
"assert type(tuf0[PROPS].get('syn:bench:reddit:submission:subreddit')) == str"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"## let's benchmark"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### utils"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true,
"deletable": true,
"editable": true
},
"outputs": [],
"source": [
"import logging\n",
"\n",
"logger = logging.getLogger('bench')\n",
"logging.basicConfig(level=logging.INFO)\n",
"logger.setLevel(logging.INFO)\n",
"logging.getLogger().setLevel(logging.INFO)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [],
"source": [
"import os\n",
"import sys\n",
"import time\n",
"import psutil\n",
"import contextlib\n",
"from __future__ import print_function\n",
"\n",
"\n",
"def format_size(num, suffix='B'):\n",
" for unit in ['', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi', 'Ei', 'Zi']:\n",
" if abs(num) < 1024.0:\n",
" return \"%3.1f%s%s\" % (num, unit, suffix)\n",
" num /= 1024.0\n",
" return \"%.1f%s%s\" % (num, 'Yi', suffix)\n",
"\n",
"\n",
"@contextlib.contextmanager\n",
"def benchmark(label):\n",
" '''\n",
" timer context manager. record the amount of wall clock time taken by a block.\n",
" \n",
" Example::\n",
" \n",
" with benchmark() as t:\n",
" longrunningoperation()\n",
" \n",
" print(t[duration])\n",
" '''\n",
" print(label + '...', end='')\n",
" ret = {}\n",
" \n",
" startmem = psutil.Process(os.getpid()).memory_info().vms\n",
" startts = time.time()\n",
" yield ret\n",
" endts = time.time()\n",
" endmem = psutil.Process(os.getpid()).memory_info().vms\n",
" \n",
" ret['duration'] = endts - startts\n",
" ret['mem'] = endmem - startmem\n",
" print('done!')\n",
" print(' total time: %.1f seconds.' % (endts - startts))\n",
" print(' mem diff: %s' % format_size(endmem - startmem))"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### cortexes"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:bench:removing existing sqlite database\n",
"INFO:bench:removing existing lmdb database\n",
"INFO:bench:removing existing lmdb database (nosync)\n"
]
}
],
"source": [
"def make_cortexes():\n",
" ram_cortex = synapse.cortex.openurl('ram://')\n",
"\n",
" if os.path.exists(os.path.join('.', 'sqlite-bench.db')):\n",
" logger.info('removing existing sqlite database')\n",
" os.remove(os.path.join('.', 'sqlite-bench.db')) \n",
" sqlite_cortex = synapse.cortex.openurl('sqlite:///sqlite-bench.db')\n",
" \n",
" if os.path.exists(os.path.join('.', 'lmdb-bench.db')):\n",
" logger.info('removing existing lmdb database')\n",
" os.remove(os.path.join('.', 'lmdb-bench.db')) \n",
" lmdb_cortex = synapse.cortex.openurl('lmdb:///lmdb-bench.db?lmdb:sync=1&lmdb:metasync=1')\n",
" \n",
" if os.path.exists(os.path.join('.', 'lmdb-bench-nosync.db')):\n",
" logger.info('removing existing lmdb database (nosync)')\n",
" os.remove(os.path.join('.', 'lmdb-bench-nosync.db')) \n",
" lmdb_cortex_nosync = synapse.cortex.openurl('lmdb:///lmdb-bench-nosync.db?lmdb:sync=0&lmdb:metasync=0')\n",
"\n",
" mem_sqlite_cortex = synapse.cortex.openurl('sqlite:///:memory:')\n",
" \n",
" return [\n",
" ('ram', ram_cortex),\n",
" ('mem sqlite', mem_sqlite_cortex),\n",
" ('sqlite', sqlite_cortex),\n",
" ('lmdb', lmdb_cortex),\n",
" ('lmdb nosync', lmdb_cortex_nosync),\n",
" ]\n",
"\n",
"\n",
"def fini_cortexes(cores):\n",
" for label, core in cores:\n",
" core.fini()\n",
" \n",
" if os.path.exists(os.path.join('.', 'sqlite-bench.db')):\n",
" logger.info('removing existing sqlite database')\n",
" os.remove(os.path.join('.', 'sqlite-bench.db')) \n",
"\n",
" if os.path.exists(os.path.join('.', 'lmdb-bench.db')):\n",
" logger.info('removing existing lmdb database')\n",
" os.remove(os.path.join('.', 'lmdb-bench.db'))\n",
" \n",
" if os.path.exists(os.path.join('.', 'lmdb-bench-nosync.db')):\n",
" logger.info('removing existing lmdb database (nosync)')\n",
" os.remove(os.path.join('.', 'lmdb-bench-nosync.db')) \n",
"\n",
" \n",
"CORTEXES = make_cortexes()\n",
"\n",
"for label, core in CORTEXES:\n",
" core.addDataModel('cortex.benchmarking.does.not.exist', DATAMODEL)"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### basic insert performance"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"basic insert performance...\n",
"ram...done!\n",
" total time: 3.2 seconds.\n",
" mem diff: 38.0MiB\n",
"mem sqlite...done!\n",
" total time: 10.9 seconds.\n",
" mem diff: 1.0MiB\n",
"sqlite...done!\n",
" total time: 250.9 seconds.\n",
" mem diff: 1.9MiB\n",
"lmdb...done!\n",
" total time: 5.2 seconds.\n",
" mem diff: 680.0KiB\n",
"lmdb nosync...done!\n",
" total time: 5.4 seconds.\n",
" mem diff: 0.0B\n",
"done.\n"
]
}
],
"source": [
"print('basic insert performance...')\n",
"for label, core in CORTEXES:\n",
" with benchmark(label=label) as t:\n",
" for row in DATA_ROWS:\n",
" core.formTufoByFrob('syn:bench:reddit:submission', row['reddit_id'], **row)\n",
"print('done.')"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### insert with lots of collisions (all collisions, actually)\n",
"\n",
"here, we re-use the existing cortexes, which already contain all the rows we'll attempt to insert."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"conflicted insert performance...\n",
"ram...done!\n",
" total time: 0.9 seconds.\n",
" mem diff: 256.0KiB\n",
"mem sqlite...done!\n",
" total time: 1.6 seconds.\n",
" mem diff: 0.0B\n",
"sqlite...done!\n",
" total time: 1.9 seconds.\n",
" mem diff: 0.0B\n",
"lmdb...done!\n",
" total time: 2.7 seconds.\n",
" mem diff: 0.0B\n",
"lmdb nosync...done!\n",
" total time: 2.5 seconds.\n",
" mem diff: 0.0B\n",
"done.\n"
]
}
],
"source": [
"print('conflicted insert performance...')\n",
"for label, core in CORTEXES:\n",
" with benchmark(label=label) as t:\n",
" for row in DATA_ROWS:\n",
" core.formTufoByFrob('syn:bench:reddit:submission', row['reddit_id'], **row)\n",
"print('done.')"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### query performance\n",
"\n",
"for each submission, collect the submissions by the same user."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"query performance...\n",
"ram...done!\n",
" total time: 22.2 seconds.\n",
" mem diff: -236.0KiB\n",
"mem sqlite...done!\n",
" total time: 115.1 seconds.\n",
" mem diff: 13.8MiB\n",
"sqlite...done!\n",
" total time: 117.6 seconds.\n",
" mem diff: 1.8MiB\n",
"lmdb...done!\n",
" total time: 375.7 seconds.\n",
" mem diff: 2.0MiB\n",
"lmdb nosync...done!\n",
" total time: 375.7 seconds.\n",
" mem diff: -512.0KiB\n",
"done.\n"
]
}
],
"source": [
"def collect_by_user(core, user):\n",
" return set([t[PROPS]['syn:bench:reddit:submission'] \n",
" for t in core.getTufosByProp('syn:bench:reddit:submission:username', user)])\n",
"\n",
"print('query performance...')\n",
"for label, core in CORTEXES:\n",
" with benchmark(label=label) as t:\n",
" for tuf0 in core.getTufosByProp('syn:bench:reddit:submission'):\n",
" pprops = collect_by_user(core, tuf0[PROPS].get('syn:bench:reddit:submission:username'))\n",
"print('done.')"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### range query performance\n",
"for each submission, collect the submissions made within +/- one minute."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"query performance...\n",
"ram...done!\n",
" total time: 34.1 seconds.\n",
" mem diff: -2.3MiB\n",
"mem sqlite...done!\n",
" total time: 1.3 seconds.\n",
" mem diff: 5.0MiB\n",
"sqlite...done!\n",
" total time: 1.4 seconds.\n",
" mem diff: 1.2MiB\n",
"lmdb...done!\n",
" total time: 3.3 seconds.\n",
" mem diff: 512.0KiB\n",
"lmdb nosync...done!\n",
" total time: 3.4 seconds.\n",
" mem diff: -4.0MiB\n",
"done.\n"
]
}
],
"source": [
"def collect_by_ts(core, ts):\n",
" start = ts - 60\n",
" end = ts + 60\n",
" return set([t[PROPS]['syn:bench:reddit:submission'] \n",
" for t in core.getTufosBy('range', 'syn:bench:reddit:submission:unixtime', (start, end))])\n",
"\n",
"print('query performance...')\n",
"for label, core in CORTEXES:\n",
" with benchmark(label=label) as t:\n",
" for tuf0 in core.getTufosByProp('syn:bench:reddit:submission'):\n",
" pprops = collect_by_ts(core, tuf0[PROPS].get('syn:bench:reddit:submission:unixtime'))\n",
"print('done.')"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### update performance\n",
"for each submission, increment the number of comments."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"update performance...\n",
"ram...done!\n",
" total time: 0.6 seconds.\n",
" mem diff: -1.8MiB\n",
"mem sqlite...done!\n",
" total time: 7.3 seconds.\n",
" mem diff: 2.5MiB\n",
"sqlite...done!\n",
" total time: 178.0 seconds.\n",
" mem diff: 1.8MiB\n",
"lmdb...done!\n",
" total time: 3.0 seconds.\n",
" mem diff: -1.8MiB\n",
"lmdb nosync...done!\n",
" total time: 3.0 seconds.\n",
" mem diff: -768.0KiB\n",
"done.\n"
]
}
],
"source": [
"def inc_comment_count(core, submission):\n",
" count = submission[PROPS].get('syn:bench:reddit:submission:number_of_comments', 0)\n",
" core.setTufoProp(submission, 'number_of_comments', count + 1)\n",
" \n",
"print('update performance...')\n",
"for label, core in CORTEXES:\n",
" with benchmark(label=label) as t:\n",
" for tuf0 in core.getTufosByProp('syn:bench:reddit:submission'):\n",
" inc_comment_count(core, tuf0) \n",
"print('done.')"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### delete performance\n",
"just delete everything."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"delete performance...\n",
"ram...done!\n",
" total time: 0.8 seconds.\n",
" mem diff: -7.5MiB\n",
"mem sqlite...done!\n",
" total time: 9.5 seconds.\n",
" mem diff: -25.8MiB\n",
"sqlite...done!\n",
" total time: 204.1 seconds.\n",
" mem diff: 1.5MiB\n",
"lmdb...done!\n",
" total time: 5.7 seconds.\n",
" mem diff: -1.2MiB\n",
"lmdb nosync...done!\n",
" total time: 5.7 seconds.\n",
" mem diff: 0.0B\n",
"done.\n"
]
}
],
"source": [
"print('delete performance...')\n",
"for label, core in CORTEXES:\n",
" with benchmark(label=label) as t:\n",
" for tuf0 in core.getTufosByProp('syn:bench:reddit:submission'):\n",
" core.delTufo(tuf0) \n",
"print('done.')"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### bulk insert\n",
"here we insert submissions in chunks of 1000, using transactions. this only makes sense when we know that the target submissions do not already exist. however, the speedup in disk-backed cortexes is substantial."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"bulk insert performance...\n",
"ram...done!\n",
" total time: 2.6 seconds.\n",
" mem diff: 20.5MiB\n",
"mem sqlite...done!\n",
" total time: 4.9 seconds.\n",
" mem diff: 808.0KiB\n",
"sqlite...done!\n",
" total time: 8.5 seconds.\n",
" mem diff: 0.0B\n",
"lmdb...done!\n",
" total time: 4.8 seconds.\n",
" mem diff: 0.0B\n",
"lmdb nosync...done!\n",
" total time: 4.8 seconds.\n",
" mem diff: 0.0B\n",
"done.\n"
]
}
],
"source": [
"def chunks(l, n):\n",
" '''\n",
" Yield successive n-sized chunks from l.\n",
" via: http://stackoverflow.com/a/312464/87207\n",
" '''\n",
" for i in range(0, len(l), n):\n",
" yield l[i:i + n]\n",
"\n",
"\n",
"print('bulk insert performance...')\n",
"for label, core in CORTEXES:\n",
" with benchmark(label=label) as t:\n",
" for rows in chunks(DATA_ROWS, 1000):\n",
" with core.getCoreXact():\n",
" for row in rows:\n",
" core.formTufoByFrob('syn:bench:reddit:submission', row['reddit_id'], **row)\n",
"print('done.')"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"## the end!"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:bench:removing existing sqlite database\n",
"INFO:bench:removing existing lmdb database\n",
"INFO:bench:removing existing lmdb database (nosync)\n"
]
}
],
"source": [
"fini_cortexes(CORTEXES)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment