Skip to content

Instantly share code, notes, and snippets.

View daverigby's full-sized avatar

Dave Rigby daverigby

View GitHub Profile
@daverigby
daverigby / path_queue_depth.cc
Created April 30, 2021 11:37
Determine the maximum number of requests (OS-level queue depth) for the given file path
#include <sys/sysmacros.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <iostream>
#include <string>
int main(int argc, char** argv) {
if (argc != 2) {

KV-Engine Bug Triage

Triage, noun: the process of determining the most important people or things from amongst a large number that require attention.

The aim of Bug Triage is to perfom an initial analysis of all new bugs, to determine is (a) they are a valid KV-Engine bug and (b) how important it is to fix.

After Triage a bug should either:

  • Be accepted as a valid KV-Engine bug, and have a suitable Priority and position in the Backlog, or
  • Be rejected as not a valid KV-Engine bug, and:
  • Assigned to the appropriate other team to investigate, or
(gdb) disas
Dump of assembler code for function _ZN8phosphor8TraceLog14getChunkTenantEv:
=> 0x00007ff6391ec420 <+0>: push %r12
0x00007ff6391ec422 <+2>: push %rbp
0x00007ff6391ec423 <+3>: mov %rsi,%r12
0x00007ff6391ec426 <+6>: push %rbx
0x00007ff6391ec427 <+7>: mov %rdi,%rbx
0x00007ff6391ec42a <+10>: sub $0x10,%rsp
0x00007ff6391ec42e <+14>: data16 lea 0x208aca(%rip),%rdi # 0x7ff6393f4f00
0x00007ff6391ec436 <+22>: data16 data16 callq 0x7ff6391de2e0 <__tls_get_addr@plt>
  • Lambda t1 (lines 97-101):
(gdb) disas /s 0x00000000004cd7ce
Dump of assembler code for function ScopedEventBaseThreadTest_keepalive_Test::<lambda()>::operator()(void) const:
/home/couchbase/couchbase/tlm/deps/packages/build/folly/follytsan-prefix/src/follytsan/folly/io/async/test/ScopedEventBaseThreadTest.cpp:
97	  std::thread t1([&] {
   0x00000000004cd74c <+0>:	push   %rbp
   0x00000000004cd74d <+1>:	mov    %rsp,%rbp
   0x00000000004cd750 <+4>:	push   %rbx
   0x00000000004cd751 <+5>:	sub    $0x2b8,%rsp
[ RUN ] ExecutorPoolEpEngineTest/1.cancel_can_schedule
[New Thread 0x7ffff1547700 (LWP 11270)]
==================
WARNING: ThreadSanitizer: data race (pid=11268)
Write of size 4 at 0x7fffffffde5c by main thread:
#0 folly::EventBase::runInEventBaseThreadAndWait(folly::Function<void ()>) /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/follytsan-prefix/src/follytsan/folly/io/async/EventBase.cpp:661 (ep-engine_ep_unit_tests+0x00000171e04e)
#1 FollyExecutorPool::unregisterTaskable(Taskable&, bool) ../kv_engine/engines/ep/src/folly_executorpool.cc:615 (ep-engine_ep_unit_tests+0x00000062f2b1)
#2 KVBucket::deinitialize() ../kv_engine/engines/ep/src/kv_bucket.cc:489 (ep-engine_ep_unit_tests+0x000000676e45)
#3 EPBucket::deinitialize() ../kv_engine/engines/ep/src/ep_bucket.cc:316 (ep-engine_ep_unit_tests+0x0000005ae8d8)
#4 EventuallyPersistentEngine::~EventuallyPersistentEngine() ../kv_engine/engines/ep/src/ep_engine.cc:6548 (ep-engine_ep_unit_tests+0x0000
/ClangBuildAnalyzer --analyze debug-build-1b.capture
Analyzing build trace from 'debug-build-1b.capture'...
**** Time summary:
Compilation (348 times):
Parsing (frontend): 1573.1 s
Codegen & opts (backend): 317.4 s
**** Files that took longest to parse (compiler frontend):
18032 ms: /Users/dave/repos/couchbase/server/source/build-debug/kv_engine/engines/ep/tests/CMakeFiles/ep-engine_ep_unit_tests.dir/module_tests/evp_store_single_threaded_test.cc.o
16909 ms: /Users/dave/repos/couchbase/server/source/build-debug/kv_engine/engines/ep/tests/CMakeFiles/ep-engine_ep_unit_tests.dir/module_tests/dcp_stream_test.cc.o

Step 1 - Gather profile

  1. Install debuginfo package (in addition to main RPM/deb)
  2. Install linux perf tools:
    • RHEL / CentOS / Amazon Linux: yum install perf.
    • Note: CentOS7 has a pretty old version of perf; which includes some issues decoding backtraces via DWARF. Prefer a newer version if possible. I have a local build of 5.11 for CentOS7 at ~/Documents/linux-perf-5.11-with-separate_debuginfo_unwind_fix
  3. Start workload to be measured
  4. Record profile
    1. x86-64:
  1. Install package libstdc++6 (Ubuntu 16.04)
  2. Add following to ~/.gdbinit
python
import sys 
sys.path.insert(0, '/usr/share/gcc-9/python')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end
https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/
-ftime-trace
@daverigby
daverigby / linux_perf_inside_docker
Created February 1, 2019 11:45
Running `linux perf` against a Docker container process
# From the host:
$ nsenter -t <PID as seen by host> /bin/bash
$ perf top / perf report etc.