Skip to content

Instantly share code, notes, and snippets.

@jmvrbanac

jmvrbanac/gist:136887af0270ed48ae82f683e0ddb0d0

Created July 7, 2016 19:56
Show Gist options
  • Save jmvrbanac/136887af0270ed48ae82f683e0ddb0d0 to your computer and use it in GitHub Desktop.
Save jmvrbanac/136887af0270ed48ae82f683e0ddb0d0 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
2016-07-07 14:49:23.475 19260 140246374881024 DEBUG app.py:136 log [-] Starting up
2016-07-07 14:49:23.513 19260 140246374881024 INFO chain.py:94 log [-] Registered plugin reader in position 0
2016-07-07 14:49:23.513 19260 140246374881024 INFO chain.py:94 log [-] Registered plugin aggregator in position 1
2016-07-07 14:49:23.513 19260 140246374881024 INFO chain.py:94 log [-] Registered plugin writer in position 2
2016-07-07 14:49:23.513 19260 140246374881024 INFO chain.py:94 log [-] Registered plugin exception_handler in position 3
2016-07-07 14:49:23.513 19260 140246374881024 INFO chain.py:111 log [-] Starting chain worker
2016-07-07 14:49:23.513 19260 140246374881024 INFO chain.py:100 log [-] Initializing chain worker
2016-07-07 14:49:23.513 19260 140246374881024 DEBUG db_connectors.py:27 log [-] PostgresWriter initialize
2016-07-07 14:49:23.517 19260 140246374881024 DEBUG chain.py:68 log [-] Validating chain integrity for chain worker
2016-07-07 14:49:23.517 19260 140246374881024 DEBUG pykafka_connectors.py:57 log [-] HermesPykafkaConsumer validate
2016-07-07 14:49:23.517 19260 140246374881024 DEBUG aggregators.py:112 log [-] Aggregator validate
2016-07-07 14:49:23.517 19260 140246374881024 DEBUG db_connectors.py:40 log [-] PostgresWriter validate
2016-07-07 14:49:23.517 19260 140246374881024 DEBUG pykafka_connectors.py:60 log [-] HermesPykafkaConsumer start
2016-07-07 14:49:23.520 19260 140246374881024 DEBUG pykafka_connectors.py:84 log [-] HermesPykafkaConsumer: using partition 0
2016-07-07 14:49:23.522 19260 140246374881024 INFO pykafka_connectors.py:100 log [-] Consumer is waiting for messages...
2016-07-07 14:49:25.500 19260 140246374881024 DEBUG pykafka_connectors.py:112 log [-] Consumer got a message: offset: 19 partition: 0 message: b'ecr||137.69.117.208||-||-||-||08/Mar/2016:23:06:00 +0000||0||||[email protected]||||||GET /cccddb5fa66a4a5994e2422cdc40baef/?delimiter=%2F&max-keys=1000 HTTP/1.1||200||0||BG||ARK||323||608||0||0.088||-||S3 Browser 5-8-9 http://s3browser.com||-||-||cccddb5fa66a4a5994e2422cdc40baef||||namespace1||us-central-1||5954da05-f920-4c44-91a7-832e8495075a||s-us.objectstorage.io||-||0||||TLSv1||ECDHE-RSA-AES128-SHA||||||cccddb5fa66a4a5994e2422cdc40baef$1||1||-||-||323||608'
2016-07-07 14:49:25.501 19260 140246374881024 DEBUG pykafka_connectors.py:151 log [-] HermesPykafkaConsumer process
2016-07-07 14:49:25.501 19260 140246374881024 DEBUG aggregators.py:63 log [-] Aggregator got a message: ecr||137.69.117.208||-||-||-||08/Mar/2016:23:06:00 +0000||0||||[email protected]||||||GET /cccddb5fa66a4a5994e2422cdc40baef/?delimiter=%2F&max-keys=1000 HTTP/1.1||200||0||BG||ARK||323||608||0||0.088||-||S3 Browser 5-8-9 http://s3browser.com||-||-||cccddb5fa66a4a5994e2422cdc40baef||||namespace1||us-central-1||5954da05-f920-4c44-91a7-832e8495075a||s-us.objectstorage.io||-||0||||TLSv1||ECDHE-RSA-AES128-SHA||||||cccddb5fa66a4a5994e2422cdc40baef$1||1||-||-||323||608
2016-07-07 14:49:25.502 19260 140246374881024 INFO pykafka_connectors.py:176 log [-] Message processing time: 0.0017235
2016-07-07 14:49:25.502 19260 140246374881024 INFO pykafka_connectors.py:178 log [-] Total processing time: 0.001724
2016-07-07 14:49:25.502 19260 140246374881024 INFO pykafka_connectors.py:181 log [-] Total consumed messages: 1.0000000
2016-07-07 14:49:25.503 19260 140246374881024 INFO pykafka_connectors.py:188 log [-] Total failed messages: 0.0000000
2016-07-07 14:49:53.514 19260 140246215735040 DEBUG aggregators.py:121 log [-] completion_timeout reached for aggregator
2016-07-07 14:49:53.514 19260 140246215735040 DEBUG db_connectors.py:46 log [-] PostgresWriter process
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/john/.pyenv/versions/3.4.5/lib/python3.4/threading.py", line 911, in _bootstrap_inner
self.run()
File "/home/john/.pyenv/versions/3.4.5/lib/python3.4/threading.py", line 1177, in run
self.function(*self.args, **self.kwargs)
File "hermes/plugins/aggregators.py", line 134, in flush_all
next_plugin.process(message_context)
File "hermes/plugins/db_connectors.py", line 52, in process
with psycopg2.connect(self.connection_string) as connection:
File "/home/john/.virtualenvs/hermes/lib/python3.4/site-packages/psycopg2/__init__.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
2016-07-07 14:53:56.928 19260 140246374881024 DEBUG pykafka_connectors.py:112 log [-] Consumer got a message: offset: 20 partition: 0 message: b'ecr||137.69.117.208||-||-||-||08/Mar/2016:23:06:00 +0000||0||||[email protected]||||||GET /cccddb5fa66a4a5994e2422cdc40baef/?delimiter=%2F&max-keys=1000 HTTP/1.1||200||0||BG||ARK||323||608||0||0.088||-||S3 Browser 5-8-9 http://s3browser.com||-||-||cccddb5fa66a4a5994e2422cdc40baef||||namespace1||us-central-1||5954da05-f920-4c44-91a7-832e8495075a||s-us.objectstorage.io||-||0||||TLSv1||ECDHE-RSA-AES128-SHA||||||cccddb5fa66a4a5994e2422cdc40baef$1||1||-||-||323||608'
2016-07-07 14:53:56.928 19260 140246374881024 DEBUG pykafka_connectors.py:151 log [-] HermesPykafkaConsumer process
2016-07-07 14:53:56.928 19260 140246374881024 DEBUG aggregators.py:63 log [-] Aggregator got a message: ecr||137.69.117.208||-||-||-||08/Mar/2016:23:06:00 +0000||0||||[email protected]||||||GET /cccddb5fa66a4a5994e2422cdc40baef/?delimiter=%2F&max-keys=1000 HTTP/1.1||200||0||BG||ARK||323||608||0||0.088||-||S3 Browser 5-8-9 http://s3browser.com||-||-||cccddb5fa66a4a5994e2422cdc40baef||||namespace1||us-central-1||5954da05-f920-4c44-91a7-832e8495075a||s-us.objectstorage.io||-||0||||TLSv1||ECDHE-RSA-AES128-SHA||||||cccddb5fa66a4a5994e2422cdc40baef$1||1||-||-||323||608
2016-07-07 14:53:56.928 19260 140246374881024 INFO pykafka_connectors.py:176 log [-] Message processing time: 0.0003483
2016-07-07 14:53:56.928 19260 140246374881024 INFO pykafka_connectors.py:178 log [-] Total processing time: 0.002072
2016-07-07 14:53:56.928 19260 140246374881024 INFO pykafka_connectors.py:181 log [-] Total consumed messages: 2.0000000
2016-07-07 14:53:56.929 19260 140246374881024 INFO pykafka_connectors.py:188 log [-] Total failed messages: 0.0000000
^CAttempting to gracefully shutdown Hermes...
2016-07-07 14:55:17.732 19260 140246374881024 INFO pykafka_connectors.py:205 log [-] Consumer stop
2016-07-07 14:55:17.734 19260 140246374881024 DEBUG aggregators.py:102 log [-] Aggregator stop
2016-07-07 14:55:17.735 19260 140246374881024 DEBUG aggregators.py:121 log [-] completion_timeout reached for aggregator
2016-07-07 14:55:17.735 19260 140246374881024 DEBUG db_connectors.py:46 log [-] PostgresWriter process
2016-07-07 14:55:17.746 19260 140246374881024 DEBUG db_connectors.py:104 log [-] insert into bucket_stats_real_time ( bucket_name, cat_a, cat_b, deletes, egress, gets, ingress, internal_id, puts, sample_time, size_delta_prem, size_delta_geo, size_delta_geo_ia, size_delta_std, size_delta_std_ia,storage_class) values ('cccddb5fa66a4a5994e2422cdc40baef', 1, 0, 0, 608, 1, 0, 1, 0, '2016-03-08T23:06:00'::timestamp, 0, 0, 0, 0, 0, NULL)
2016-07-07 14:55:17.769 19260 140246374881024 DEBUG aggregators.py:135 log [-] flushed
2016-07-07 14:55:17.770 19260 140246374881024 DEBUG db_connectors.py:60 log [-] PostgresWriter stop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment