Skip to content

Instantly share code, notes, and snippets.

@maaand
Last active August 29, 2015 14:07
Show Gist options
  • Save maaand/e799b1815c479dd3fbb4 to your computer and use it in GitHub Desktop.
Save maaand/e799b1815c479dd3fbb4 to your computer and use it in GitHub Desktop.
Happybase Connection Pooling (wrong way)
Wrong way :
pool = happybase.ConnectionPool(size=10, host='hbase.targeting.walmartlabs.com')
conn = pool.connection() # conn is now established and waiting for data
table = conn.table('candidate_email_sends_by_date')
...
...
# some long processing work the script is taking more than 3 mins
# above connection pool has been idle for > 3 mins, Vip will start killing them
#
table.scan()
## such scripts will run into connection reset errors
Right Way:
Use with block, so the processing thread is guaranteed to get active connection from the pool
http://happybase.readthedocs.org/en/latest/user.html#using-the-connection-pool
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment