script to extract timestamp and dequeue latency for ops from an osd log

dequeue_op_latency.py

#!/usr/bin/python

from datetime import datetime
import re
import sys

class Op(object):
    def __init__(self, line):
        self.lat = float(re.search('latency (\d+.\\d+)', line).group(1))
        self.line = line
        time_format = '%Y-%m-%d %H:%M:%S.%f'
        self.time = datetime.strptime(line[:23], time_format)


def main():
    ops = []

    with open(sys.argv[1]) as f:
        for line in f.readlines():
            ops.append(Op(line))
    #ops.sort(key=lambda x: x.lat, reverse=True)
    print()
    # print('100 longest requests:')
    # for i in range(min(100, len(ops))):
    #     print(i + 1, ops[i].lat, ops[i].line)
    for op in ops:
        print(op.time.timestamp(), ',', op.lat)

if __name__ == '__main__':
    main()

Usage: filter the osd log for latency, and then run this script on it:

grep -a dequeue_op.*latency ceph-osd.45.log > dequeue_op.log
python dequque_op_latency.py dequeue_op.log > dequeue_latency.csv