valyala

Optimizing postgresql table for more than 100K inserts per second

• Create UNLOGGED table. This reduces the amount of data written to persistent storage by up to 2x.
• Set WITH (autovacuum_enabled=false) on the table. This saves CPU time and IO bandwidth on useless vacuuming of the table (since we never DELETE or UPDATE the table).
• Insert rows with COPY FROM STDIN. This is the fastest possible approach to insert rows into table.
• Minimize the number of indexes in the table, since they slow down inserts. Usually an index on time timestamp with time zone is enough.
• Add synchronous_commit = off to postgresql.conf.
• Use table inheritance for fast removal of old data:

soaxelbrooke

class GzipBuffer(object):
    def __init__(self):
        self.len = 0
        self.buffer = io.BytesIO()
        self.writer = gzip.GzipFile(fileobj=self.buffer, mode='wb')

    def append(self, thing):
        self.len += 1
        self.writer.write(thing)

nlap

graph-tool on Ubuntu 14.04

Recommended: Graph-tool one-line install commands:

anaconda conda create --name gt -c conda-forge graph-tool

homebrew brew install graph-tool

obfusk

import code; code.interact(local=dict(globals(), **locals()))

audreyfeldroy

• Update HISTORY.md
• Commit the changes:

git add HISTORY.md
git commit -m "Changelog for upcoming release 0.1.1."

• Update version number (can also be minor or major)

bumpversion patch

Mortimerp9

import scala.concurrent.Await
import scala.concurrent.ExecutionContext
import scala.concurrent.Future
import scala.concurrent.blocking
import scala.concurrent.duration.Deadline
import scala.concurrent.duration.Duration
import scala.concurrent.duration.DurationInt
import scala.concurrent.duration.DurationLong
import scala.concurrent.future
import scala.concurrent.promise

samuraisam

#Copyright (c) 2010-2013 Samuel Sutch [[email protected]]
#
#Permission is hereby granted, free of charge, to any person obtaining a copy
#of this software and associated documentation files (the "Software"), to deal
#in the Software without restriction, including without limitation the rights
#to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#copies of the Software, and to permit persons to whom the Software is
#furnished to do so, subject to the following conditions:
#
#The above copyright notice and this permission notice shall be included in