Skip to content

Instantly share code, notes, and snippets.

@rmeleromira
Created March 14, 2017 19:12
Show Gist options
  • Save rmeleromira/828f86c398b2b74ec535de17ca75fa2b to your computer and use it in GitHub Desktop.
Save rmeleromira/828f86c398b2b74ec535de17ca75fa2b to your computer and use it in GitHub Desktop.
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: Process Process-1:
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: Traceback (most recent call last):
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self.run()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self._target(*self._args, **self._kwargs)
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: p.run()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 58, in run
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: for alarm in self._consumer:
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 122, in __iter__
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self._partition()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 212, in _partition
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self._consumer = self._create_kafka_consumer(self._partitions)
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 110, in _create_kafka_consumer
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: consumer.fetch_last_known_offsets()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 110, in fetch_last_known_offsets
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: check_error(resp)
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 252, in check_error
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: raise response
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: FailedPayloadsError
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: self.doRollover()
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 136, in doRollover
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: os.rename(sfn, dfn)
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: self.doRollover()
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: OSError: [Errno 2] No such file or directory
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 136, in doRollover
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Logged from file conn.py, line 222
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: os.rename(sfn, dfn)
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: OSError: [Errno 2] No such file or directory
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Logged from file client.py, line 149
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: self.doRollover()
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 135, in doRollover
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: os.remove(dfn)
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: OSError: [Errno 2] No such file or directory: '/var/log/monasca/notification/notification.log.5'
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: Logged from file conn.py, line 222
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-3:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-4:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-2:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-1:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ Traceback (most recent call last):]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 3 times: [ File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ self.run()]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ self._target(*self._args, **self._kwargs)]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 58, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/periodic_engine.py", line 72, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 58, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for alarm in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 54, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for raw_notification in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for alarm in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for raw_notification in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='60-seconds-notifications', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='retry-notifications', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:01 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:01 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: Process Process-4:
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: Traceback (most recent call last):
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self.run()
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: p = process_type(config, *args)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/periodic_engine.py", line 42, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self._topic_name)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: max_buffer_size=None)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: raise error_class(response)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: LeaderNotAvailableError: TopicMetadata(topic='60-seconds-notifications', error=5, partitions=[])
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Process Process-1:
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Traceback (most recent call last):
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.run()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Process Process-2:
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: p = process_type(config, *args)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 40, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Traceback (most recent call last):
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: config['kafka']['alarm_topic'])
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.run()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: max_buffer_size=None)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: p = process_type(config, *args)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 40, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: config['kafka']['alarm_topic'])
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: max_buffer_size=None)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: raise error_class(response)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: raise error_class(response)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:03 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:03 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:03 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Process Process-3:
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Process Process-4:
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Traceback (most recent call last):
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Traceback (most recent call last):
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.run()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.run()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: p = process_type(config, *args)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: p = process_type(config, *args)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 46, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/periodic_engine.py", line 42, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: config['kafka']['notification_retry_topic'])
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._topic_name)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: max_buffer_size=None)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: max_buffer_size=None)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: raise error_class(response)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: raise error_class(response)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: LeaderNotAvailableError: TopicMetadata(topic='60-seconds-notifications', error=5, partitions=[])
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: LeaderNotAvailableError: TopicMetadata(topic='retry-notifications', error=5, partitions=[])
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Process Process-3:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self.run()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: p = process_type(config, *args)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 46, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: config['kafka']['notification_retry_topic'])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: max_buffer_size=None)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: raise error_class(response)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: LeaderNotAvailableError: TopicMetadata(topic='retry-notifications', error=5, partitions=[])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Process Process-1:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self.run()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: p = process_type(config, *args)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 40, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: config['kafka']['alarm_topic'])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: max_buffer_size=None)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Process Process-3:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self.run()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: p = process_type(config, *args)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 46, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: config['kafka']['notification_retry_topic'])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: max_buffer_size=None)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Process Process-1:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
~
~
~
~
~
~
~
~
~
root@monasca-api-container-08d216ae:~# less /var/log/monasca/notification/notification.log
root@monasca-api-container-08d216ae:~# less /var/log/monasca/notification/notification.log
root@monasca-api-container-08d216ae:~# apt-get install monit
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
exim4 | postfix | mail-transport-agent sysvinit-core
The following NEW packages will be installed:
monit
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 273 kB of archives.
After this operation, 826 kB of additional disk space will be used.
Get:1 http://yumrepo.sys.comcast.net/ubuntu xenial/universe amd64 monit amd64 1:5.16-2 [273 kB]
Fetched 273 kB in 0s (433 kB/s)
Selecting previously unselected package monit.
(Reading database ... 21334 files and directories currently installed.)
Preparing to unpack .../monit_1%3a5.16-2_amd64.deb ...
Unpacking monit (1:5.16-2) ...
Processing triggers for systemd (229-4ubuntu16) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up monit (1:5.16-2) ...
Processing triggers for systemd (229-4ubuntu16) ...
Processing triggers for ureadahead (0.100.0-19) ...
root@monasca-api-container-08d216ae:~# less /var/log/monasca/notification/notification.log
root@monasca-api-container-08d216ae:~# less /var/log/syslog
syslog syslog.1 syslog.2.gz syslog.3.gz syslog.4.gz syslog.5.gz syslog.6.gz syslog.7.gz
root@monasca-api-container-08d216ae:~# less /var/log/syslog
root@monasca-api-container-08d216ae:~# grep monasca-notification /var/log/syslog > notout
root@monasca-api-container-08d216ae:~# less notout
root@monasca-api-container-08d216ae:~# grep monasca-notification /var/log/syslog* > notout
root@monasca-api-container-08d216ae:~# less notout
root@monasca-api-container-08d216ae:~# grep monasca-notification /var/log/syslog.1 > notout
root@monasca-api-container-08d216ae:~# less notout
root@monasca-api-container-08d216ae:~# cat notout
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: Process Process-1:
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: Traceback (most recent call last):
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self.run()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self._target(*self._args, **self._kwargs)
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: p.run()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 58, in run
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: for alarm in self._consumer:
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 122, in __iter__
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self._partition()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 212, in _partition
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: self._consumer = self._create_kafka_consumer(self._partitions)
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 110, in _create_kafka_consumer
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: consumer.fetch_last_known_offsets()
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 110, in fetch_last_known_offsets
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: check_error(resp)
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 252, in check_error
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: raise response
Mar 13 06:25:49 monasca-api-container-08d216ae monasca-notification[62983]: FailedPayloadsError
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 06:25:49 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: self.doRollover()
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 136, in doRollover
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: os.rename(sfn, dfn)
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: self.doRollover()
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: OSError: [Errno 2] No such file or directory
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 136, in doRollover
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Logged from file conn.py, line 222
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: os.rename(sfn, dfn)
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: OSError: [Errno 2] No such file or directory
Mar 13 15:59:58 monasca-api-container-08d216ae monasca-notification[13213]: Logged from file client.py, line 149
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: self.doRollover()
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/logging/handlers.py", line 135, in doRollover
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: os.remove(dfn)
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: OSError: [Errno 2] No such file or directory: '/var/log/monasca/notification/notification.log.5'
Mar 13 16:00:00 monasca-api-container-08d216ae monasca-notification[13213]: Logged from file conn.py, line 222
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-3:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-4:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-2:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Process Process-1:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: Traceback (most recent call last):
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ Traceback (most recent call last):]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 3 times: [ File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ self.run()]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message repeated 2 times: [ self._target(*self._args, **self._kwargs)]
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 87, in start_process
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 58, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/periodic_engine.py", line 72, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 58, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: p.run()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for alarm in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 54, in run
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for raw_notification in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for alarm in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: for raw_notification in self._consumer:
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 135, in __iter__
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: message = self._consumer.get_message()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 307, in get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: return self._get_message(block, timeout, get_partition_info)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 322, in _get_message
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self._fetch()
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 381, in _fetch
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: fail_on_error=False
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='60-seconds-notifications', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 631, in send_fetch_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: KafkaProtocol.decode_fetch_response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 189, in _send_broker_aware_request
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: payload.partition)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 87, in _get_leader_for_partition
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: self.load_metadata_for_topics(topic)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: raise error_class(response)
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='retry-notifications', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae monasca-notification[13213]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:01 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:01 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:01 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: Process Process-4:
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: Traceback (most recent call last):
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self.run()
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: p = process_type(config, *args)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/periodic_engine.py", line 42, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self._topic_name)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: max_buffer_size=None)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: raise error_class(response)
Mar 13 16:03:02 monasca-api-container-08d216ae monasca-notification[17947]: LeaderNotAvailableError: TopicMetadata(topic='60-seconds-notifications', error=5, partitions=[])
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:02 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Process Process-1:
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Traceback (most recent call last):
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.run()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Process Process-2:
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: p = process_type(config, *args)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 40, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: Traceback (most recent call last):
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: config['kafka']['alarm_topic'])
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.run()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: max_buffer_size=None)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: p = process_type(config, *args)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 40, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: config['kafka']['alarm_topic'])
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: max_buffer_size=None)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: raise error_class(response)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: raise error_class(response)
Mar 13 16:03:03 monasca-api-container-08d216ae monasca-notification[17955]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:03 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:03 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:03 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Process Process-3:
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Process Process-4:
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Traceback (most recent call last):
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: Traceback (most recent call last):
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.run()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.run()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: p = process_type(config, *args)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: p = process_type(config, *args)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 46, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/periodic_engine.py", line 42, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: config['kafka']['notification_retry_topic'])
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._topic_name)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: max_buffer_size=None)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: max_buffer_size=None)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: raise error_class(response)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: raise error_class(response)
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: LeaderNotAvailableError: TopicMetadata(topic='60-seconds-notifications', error=5, partitions=[])
Mar 13 16:03:04 monasca-api-container-08d216ae monasca-notification[17966]: LeaderNotAvailableError: TopicMetadata(topic='retry-notifications', error=5, partitions=[])
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:04 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Process Process-3:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self.run()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: p = process_type(config, *args)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 46, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: config['kafka']['notification_retry_topic'])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: max_buffer_size=None)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: raise error_class(response)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: LeaderNotAvailableError: TopicMetadata(topic='retry-notifications', error=5, partitions=[])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Process Process-1:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self.run()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: p = process_type(config, *args)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 40, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: config['kafka']['alarm_topic'])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: max_buffer_size=None)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17976]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: Failed to reset devices.list on /system.slice/monasca-notification.service: Operation not permitted
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Process Process-3:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self.run()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: p = process_type(config, *args)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/retry_engine.py", line 46, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: config['kafka']['notification_retry_topic'])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: max_buffer_size=None)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Process Process-1:
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: Traceback (most recent call last):
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self.run()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self._target(*self._args, **self._kwargs)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/main.py", line 86, in start_process
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: p = process_type(config, *args)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_notification/notification_engine.py", line 40, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: raise error_class(response)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: config['kafka']['alarm_topic'])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 94, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: LeaderNotAvailableError: TopicMetadata(topic='retry-notifications', error=5, partitions=[])
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self._consumer = self._create_kafka_consumer()
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka/consumer.py", line 107, in _create_kafka_consumer
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: max_buffer_size=None)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/simple.py", line 126, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: auto_commit_every_t=auto_commit_every_t)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/consumer/base.py", line 52, in __init__
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: self.client.load_metadata_for_topics(topic)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/client.py", line 505, in load_metadata_for_topics
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: kafka_common.check_error(topic_metadata)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: File "/opt/monasca/local/lib/python2.7/site-packages/monasca_common/kafka_lib/common.py", line 255, in check_error
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: raise error_class(response)
Mar 13 16:03:05 monasca-api-container-08d216ae monasca-notification[17984]: LeaderNotAvailableError: TopicMetadata(topic='alarm-state-transitions', error=5, partitions=[])
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Main process exited, code=exited, status=17/n/a
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Unit entered failed state.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Failed with result 'exit-code'.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Service hold-off time over, scheduling restart.
Mar 13 16:03:05 monasca-api-container-08d216ae systemd[1]: monasca-notification.service: Start request repeated too quickly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment