Skip to content

Instantly share code, notes, and snippets.

@jvns

jvns/ruby-python-profiling-tools.md

Last active March 15, 2024 20:19
Show Gist options
  • Save jvns/b81eb039f6373bc577d7dbdd978581b5 to your computer and use it in GitHub Desktop.
Save jvns/b81eb039f6373bc577d7dbdd978581b5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
ruby-python-profiling-tools.md

Ruby and Python profilers

CPU profilers

Python

Ruby

heap analyzers / memory profilers

Ruby

blog posts about memory profiling: (from https://twitter.com/b0rk/status/957994325476036613)

Python

Frontends

various tools for injecting code into ruby / python processes or tracing

(definitely out of scope, but interesting)

ideas/thoughts by email

  • automatically save results to disk (cf https://github.com/garybernhardt/readygo)
  • give profiling results on the fly, instead of making the user wait for the end of the run
  • ability to use it in production is key
  • "I am working on a Rust thing to make Rails faster and am frustrated that I can't use the same profiling tools to measure before/after. (Instruments can't see into Ruby, stackprof can't see past FFI) "
  • "the things that ruby has been missing since forever are":
    • a way to get a stacktrace dump of all threads (which is easy)
    • information about which thread was running at the time of said dump
    • information about which threads are actively trying to get scheduled (e.g. they have work to do), threads that are waiting on IO to complete, and threads that are waiting for mutexes to become available.
@NickLaMuro
Copy link

NickLaMuro commented Dec 3, 2017 •
edited
Loading

Just thought I would also share the link for rbtrace, incase someone is looking for it:

https://github.com/tmm1/rbtrace

Also, a co-worker of my, @Fryguy, worked on https://github.com/FryGuy/memory_analyzer , which takes Aaron P.'s heap-analyzer that is terminal based (basically `IRB plus a global variable), and doesn't require node to run locally.

Best of luck with your sabbatical. Look forward to seeing what comes of it!

@twerp
Copy link

twerp commented Dec 4, 2017

Don't know how the following compare to the ones on the document already. I've only used pprofile.

pprofile is a "line-granularity, thread-aware deterministic and statistic pure-python profiler. Inspired from Robert Kern's line_profiler."

pyheat - pprofile + matplotlib = Python program profiled as an awesome heatmap

profiling package is an interactive continuous Python profiler. It is inspired from Unity 3D profiler.

pyvmmonitor_core - Open Source bits of the core utilities used in PyVmMonitor http://www.pyvmmonitor.com/

Function-Profiler - This is a Python 3.6 module to profile function performance. It records the number of calls made to select functions, and outputs statistics about their execution times.

QGProfiler is a simple, user controlled profiler. It differs from other profilers in that it provides the programmer the control of granularity with which various parts of the program are profiled. (blog post)

bprofile is a wrapper around cProfile, gprof2dot and graphviz, providing a simple context manager for profiling sections of Python code and producing visual graphs of profiling results. It works on Windows and Unix.

grav - A collection of tools to help visualise process execution.

flameprof - Flamegraph generator for python's cProfile stats.

pyprof2calltree - Profile python programs and view them with kcachegrind

lptrace is strace for Python programs. It lets you see in real-time what functions a Python program is running. It's particularly useful to debug weird issues on production. (Might not support Python 3, but there may be a fork that does.)

Pympler - Development tool to measure, monitor and analyze the memory behavior of Python objects in a running Python application. (Includes Muppy.)

memprof - A memory profiler for Python. As easy as adding a decorator

YouTube video: Watch your Python script with strace (2014)

@jfilip
Copy link

jfilip commented Dec 6, 2017

Rails panel (Chrome extension + Gem) is a pretty good tool for getting a lot of info about what's happening on a Rails request

@mdesantis
Copy link

Here there is a gdb integration: https://github.com/david942j/gdb-ruby

@williame
Copy link

Here's one suitable for profiling long-running servers: https://williamedwardscoder.tumblr.com/post/35134924139/a-novel-profiler-for-python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment