Skip to content

Instantly share code, notes, and snippets.

@enachb
Created May 31, 2013 19:03
Show Gist options
  • Select an option

  • Save enachb/5687173 to your computer and use it in GitHub Desktop.

Select an option

Save enachb/5687173 to your computer and use it in GitHub Desktop.
insert json records into RethinkDB
import sys,json,fileinput
import rethinkdb as r
#python rdbFill.py /data/exports3/rawProd/fbComment/*
r.connect('dcd3', 28015).repl()
t = r.db('test').table('oink')
i = 0
lines = []
for line in fileinput.input():
try:
if i < 1000:
lines.append(json.loads(line))
i = i + 1
else:
i = 0
t.insert(lines).run()
lines = []
except:
print "error line: " + line
print "error:", sys.exc_info()[0]
# obj = json.loads(line)
# postId = obj["postId"]
# if len(postId) < 3:
# print "%s" % (obj["id"])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment