Skip to content

Instantly share code, notes, and snippets.

@arcolife

arcolife/count_msg_state_in_range.md

Last active December 1, 2017 11:32
Show Gist options
  • Save arcolife/6d6bda31ad3685e05cb1f91c34d97d4b to your computer and use it in GitHub Desktop.
Save arcolife/6d6bda31ad3685e05cb1f91c34d97d4b to your computer and use it in GitHub Desktop.
Download ZIP
CFME evm log message counts in state
Raw
count_msg_state_in_range.md

Script: https://github.com/arcolife/cfme-tune/blob/master/scripts/count_msg_state_in_range.sh

... -t => defaults to message delivered state type: Error

... -b <> -e <> => defaults to last one hour's worth of data.

Datetime format is of type $ date +'%Y-%m-%dT%H:%M:%S'

... - c -- => optional argument

Simple count with filepath not in default location

$ ./count_msg_state_in_range.sh  -f ~/evm.log 

[12/01/17 14:15:09 IST] - INFO - Using default message delivered type - state [error]
[12/01/17 14:15:09 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 14:15:09 IST] - deleting previous garbage -> /tmp/evm.log_log_frag
[12/01/17 14:15:09 IST] - deleting previous garbage -> /tmp/evm.log_delivered
[12/01/17 14:15:09 IST] - deleting previous garbage -> /tmp/evm.log_ids
[12/01/17 14:15:09 IST] - deleting previous garbage -> /tmp/evm.log_completed
[12/01/17 14:15:09 IST] - [default mode] using last 1 hour's worth of log data to process events.
[12/01/17 14:15:09 IST] - [default mode] Assuming log datatime is in EST
[12/01/17 14:15:09 IST] - extracting logs between 2017-11-30T01:45:39 and 2017-11-30T02:45:39
[12/01/17 14:15:09 IST] - extracted MSG STATE [error] and IDs from /tmp/evm.log_delivered
[12/01/17 14:15:09 IST] - Processing complete
[12/01/17 14:15:09 IST] - Results: 
      20 [Container.perf_capture_realtime]

Count of entire evm.log

$ ./count_msg_state_in_range.sh  -f ~/evm.log -c --

[12/01/17 14:16:04 IST] - INFO - Using default message delivered type - state [error]
[12/01/17 14:16:04 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 14:16:04 IST] - deleting previous garbage -> /tmp/evm.log_log_frag
[12/01/17 14:16:04 IST] - deleting previous garbage -> /tmp/evm.log_delivered
[12/01/17 14:16:04 IST] - deleting previous garbage -> /tmp/evm.log_ids
[12/01/17 14:16:04 IST] - deleting previous garbage -> /tmp/evm.log_completed
[12/01/17 14:16:04 IST] - consuming entire evm.log
[12/01/17 14:16:04 IST] - extracted MSG STATE [error] and IDs from /tmp/evm.log_delivered
[12/01/17 14:16:16 IST] - Processing complete
[12/01/17 14:16:16 IST] - Results: 
     315 [Container.perf_capture_realtime]
      1 [EmsRefresh.refresh]

Count with no timerange supplied

..uses last 1 hour's data

$ ./count_msg_state_in_range.sh  -f ~/evm.log -t ok

[12/01/17 14:16:48 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 14:16:48 IST] - deleting previous garbage -> /tmp/evm.log_log_frag
[12/01/17 14:16:48 IST] - deleting previous garbage -> /tmp/evm.log_delivered
[12/01/17 14:16:48 IST] - deleting previous garbage -> /tmp/evm.log_ids
[12/01/17 14:16:48 IST] - deleting previous garbage -> /tmp/evm.log_completed
[12/01/17 14:16:48 IST] - [default mode] using last 1 hour's worth of log data to process events.
[12/01/17 14:16:48 IST] - [default mode] Assuming log datatime is in EST
[12/01/17 14:16:48 IST] - extracting logs between 2017-11-30T01:45:39 and 2017-11-30T02:45:39
[12/01/17 14:16:48 IST] - extracted MSG STATE [ok] and IDs from /tmp/evm.log_delivered
[12/01/17 14:16:49 IST] - Processing complete
[12/01/17 14:16:49 IST] - Results: 
      11 [ContainerProject.perf_rollup]
     14 [ContainerReplicator.perf_rollup]
     17 [ContainerService.perf_rollup]
      5 [EmsRefresh.refresh]
      1 [ExtManagementSystem.authentication_check_schedule]
      1 [ExtManagementSystem.authentication_check_types]
      1 [Host.authentication_check_schedule]
      1 [Job.check_for_evm_snapshots]
     60 [Job.check_jobs_for_timeout]
     16 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerGroup.perf_capture_realtime]
     16 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerGroup.perf_rollup]
     10 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerNode.perf_capture_realtime]
     10 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerNode.perf_rollup]
     15 [ManageIQ::Providers::Kubernetes::ContainerManager::Container.perf_capture_realtime]
     15 [ManageIQ::Providers::Kubernetes::ContainerManager::Container.perf_rollup]
      2 [ManageIQ::Providers::Openshift::ContainerManager.perf_rollup]
      4 [ManageIQ::Providers::Openshift::ContainerManager.refresh_all_ems_timer]
     20 [Metric::Capture.perf_capture_timer]
      2 [Metric::Purging.purge_realtime]
      3 [Metric::Purging.purge_realtime_timer]
      6 [MiqAeEngine.deliver]
      1 [MiqAlert.evaluate_hourly_timer]
      2 [MiqEnterprise.perf_rollup]
      6 [MiqEvent.raise_evm_event]
      2 [MiqRegion.perf_rollup]
      2 [MiqSchedule.invoke_actions]
     12 [MiqServer.log_status]
     60 [MiqServer.status_update]
     20 [MiqTask.destroy_older_by_condition]
      1 [MiqTask.update_status_for_timed_out_active_tasks]
      2 [MiqWidget.generate_content]
     12 [MiqWorker.log_status_all]
      6 [RetirementManager.check]
      1 [VmdbDatabase.capture_metrics_timer]
      2 [VmdbDatabaseConnection.log_statistics]
      1 [VmdbDatabaseMetric.purge_all_timer]
      1 [VmdbDatabaseMetric.purge_daily]
      1 [VmdbDatabaseMetric.purge_hourly]
      1 [VmdbMetric.purge_all_timer]
      1 [VmdbMetric.purge_daily]
      1 [VmdbMetric.purge_hourly]

custom time range to evm.log

$ ./count_msg_state_in_range.sh  -f ~/evm.log -b 2017-11-30T01:45:39 -e 2017-11-30T02:45:39 -t ok

[12/01/17 14:17:43 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 14:17:43 IST] - deleting previous garbage -> /tmp/evm.log_log_frag
[12/01/17 14:17:43 IST] - deleting previous garbage -> /tmp/evm.log_delivered
[12/01/17 14:17:43 IST] - deleting previous garbage -> /tmp/evm.log_ids
[12/01/17 14:17:43 IST] - deleting previous garbage -> /tmp/evm.log_completed
[12/01/17 14:17:43 IST] - extracting logs between 2017-11-30T01:45:39 and 2017-11-30T02:45:39
[12/01/17 14:17:43 IST] - extracted MSG STATE [ok] and IDs from /tmp/evm.log_delivered
[12/01/17 14:17:44 IST] - Processing complete
[12/01/17 14:17:44 IST] - Results: 
      11 [ContainerProject.perf_rollup]
     14 [ContainerReplicator.perf_rollup]
     17 [ContainerService.perf_rollup]
      5 [EmsRefresh.refresh]
      1 [ExtManagementSystem.authentication_check_schedule]
      1 [ExtManagementSystem.authentication_check_types]
      1 [Host.authentication_check_schedule]
      1 [Job.check_for_evm_snapshots]
     60 [Job.check_jobs_for_timeout]
     16 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerGroup.perf_capture_realtime]
     16 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerGroup.perf_rollup]
     10 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerNode.perf_capture_realtime]
     10 [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerNode.perf_rollup]
     15 [ManageIQ::Providers::Kubernetes::ContainerManager::Container.perf_capture_realtime]
     15 [ManageIQ::Providers::Kubernetes::ContainerManager::Container.perf_rollup]
      2 [ManageIQ::Providers::Openshift::ContainerManager.perf_rollup]
      4 [ManageIQ::Providers::Openshift::ContainerManager.refresh_all_ems_timer]
     20 [Metric::Capture.perf_capture_timer]
      2 [Metric::Purging.purge_realtime]
      3 [Metric::Purging.purge_realtime_timer]
      6 [MiqAeEngine.deliver]
      1 [MiqAlert.evaluate_hourly_timer]
      2 [MiqEnterprise.perf_rollup]
      6 [MiqEvent.raise_evm_event]
      2 [MiqRegion.perf_rollup]
      2 [MiqSchedule.invoke_actions]
     12 [MiqServer.log_status]
     60 [MiqServer.status_update]
     20 [MiqTask.destroy_older_by_condition]
      1 [MiqTask.update_status_for_timed_out_active_tasks]
      2 [MiqWidget.generate_content]
     12 [MiqWorker.log_status_all]
      6 [RetirementManager.check]
      1 [VmdbDatabase.capture_metrics_timer]
      2 [VmdbDatabaseConnection.log_statistics]
      1 [VmdbDatabaseMetric.purge_all_timer]
      1 [VmdbDatabaseMetric.purge_daily]
      1 [VmdbDatabaseMetric.purge_hourly]
      1 [VmdbMetric.purge_all_timer]
      1 [VmdbMetric.purge_daily]
      1 [VmdbMetric.purge_hourly]

Invalid range, warnings and terminations

Note:

  • Valid range is between 2017-11-29T06:16:55 and 2017-11-30T02:45:39
  • Supplide ranges vary to illustrate exception handling
$ ./count_msg_state_in_range.sh -b 2017-11-21T03:59 -e 2017-11-21T04:19 -f ~/evm.log -t ok
[12/01/17 15:28:36 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 15:28:36 IST] - WARN - Supplied BEGIN precedes log range. Recaliberating to Log's beginning
[12/01/17 15:28:36 IST] - ERROR - Invalid End time; precedes Log's beging range

$ ./count_msg_state_in_range.sh -b 2017-11-29T03:59 -e 2017-11-29T04:19 -f ~/evm.log -t ok
[12/01/17 16:48:43 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 16:48:43 IST] - WARN - Supplied BEGIN precedes log range. Recaliberating to Log's beginning
[12/01/17 16:48:43 IST] - New log_start = 2017-11-29T06:16:55
[12/01/17 16:48:43 IST] - ERROR - Invalid End time; precedes Log's beging range
 
$ ./count_msg_state_in_range.sh -b 2017-11-29T06:16:55 -e 2017-11-29T04:19 -f ~/evm.log -t ok
[12/01/17 16:48:59 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 16:48:59 IST] - ERROR - Invalid End time; precedes Log's beging range
 
$ ./count_msg_state_in_range.sh -b 2017-11-29T06:16:55 -e 2017-11-30T04:19 -f ~/evm.log -t ok
[12/01/17 16:49:08 IST] - processing log file: /home/arcolife/evm.log
[12/01/17 16:49:08 IST] - WARN - Supplied END exceeds log range. Recaliberating to Log's end
[12/01/17 16:49:08 IST] - New log_end = 2017-11-30T02:45:39
[12/01/17 16:49:08 IST] - extracting logs between 2017-11-29T06:16:55 and 2017-11-30T02:45:39
[12/01/17 16:49:10 IST] - extracted MSG STATE [ok] and IDs from /tmp/evm.log_delivered
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment