Comparison of ZSH frameworks and plugin managers

startup-time-of-zsh.md

Comparison of ZSH frameworks and plugin managers

Update 1: Add a FAQ section.

Original:

I've been trying different plugin mangers for months to find the fastest one. The content below is what I got. For the record, the conclusion is very subjective. But it's the truth for me. I would appreciate hearing your thoughts on this. Here it begins.

Display what I got after this long journey. (BTW, I has an SSD on my machine, the data should be much different on HDDs.)

❯ for i ({1..10}) time zsh -ilc echo &>/dev/null
zsh  0.09s user 0.06s system 94% cpu 0.156 total
zsh  0.09s user 0.06s system 94% cpu 0.157 total
zsh  0.09s user 0.06s system 94% cpu 0.156 total
zsh  0.09s user 0.06s system 95% cpu 0.166 total
zsh  0.09s user 0.06s system 94% cpu 0.158 total
zsh  0.09s user 0.06s system 95% cpu 0.161 total
zsh  0.09s user 0.06s system 94% cpu 0.161 total
zsh  0.09s user 0.06s system 93% cpu 0.162 total
zsh  0.09s user 0.06s system 94% cpu 0.162 total
zsh  0.09s user 0.06s system 94% cpu 0.160 total

Plugin loading times:
0.001 sec - PZT::modules/environment
0.001 sec - PZT::modules/history
0.001 sec - PZT::modules/directory
0.001 sec - PZT::modules/helper
0.001 sec - PZT::modules/git
0.003 sec - PZT::modules/spectrum
0.004 sec - PZT::modules/utility
0.001 sec - PZT::modules/osx
0.001 sec - PZT::modules/archive
0.000 sec - OMZ::plugins/docker
0.000 sec - OMZ::plugins/supervisor
0.001 sec - OMZ::plugins/torrent
0.013 sec - salmanulfarzy/spaceship-prompt
0.005 sec - seletskiy/zsh-fuzzy-search-and-edit
0.003 sec - changyuheng/zsh-interactive-cd
0.009 sec - urbainvaes/fzf-marks
0.003 sec - vifon/deer
0.001 sec - wfxr/forgit
0.002 sec - laggardkernel/git-ignore
0.010 sec - marzocchi/zsh-notify
0.001 sec - ytet5uy4/pctl
0.001 sec - MikeDacre/careful_rm (command)
0.001 sec - getcolorcode
0.003 sec - ranger-cd
0.001 sec - PZT::modules/completion
0.001 sec - zsh-users/zsh-completions
0.001 sec - zdharma/history-search-multi-word
0.002 sec - PZT::modules/autosuggestions
0.148 sec - PZT::modules/python    # Turbo Mode started
0.039 sec - PZT::modules/ruby
0.090 sec - PZT::modules/node
0.036 sec - zdharma/fast-syntax-highlighting
0.002 sec - PZT::modules/history-substring-search
0.019 sec - softmoth/zsh-vim-mode
0.014 sec - PZT::modules/fasd
0.563 sec - _init      # time-consuming scripts written by myself
0.025 sec - hlissner/zsh-autopair
0.032 sec - MichaelAquilina/zsh-you-should-use # Turbo Mode ended
Total: 1.040 sec

❯ count=$(zplg times|wc -l) && echo $((count - 2)) # calc number of plugins
37

My experience with ZSH frameworks and plugin managers:

There're basically three kinds of startup time for ZSH.

1. time taken by framework, or by plugin manger itself to parse its dialect
2. time cost in loading plugins
3. time taken by scripts written by ourselves

The results here are based on the first two kinds of time.

I should mention it here that the most of startup time is taken by some time-consuming plugins, such as

• *vm, *env initialization
  • nvm, rvm
  • rbenv, pyenv, nodenv
• init code generation (not source, or eval those codes)
  • thefuck, 120ms for init generation, 1ms for eval 👍
  • fasd, 30~40ms for init generation
• time-consuming commands
  • brew --prefix nvm, 600ms 🌚
  • brew command command-not-found-init

Frameworks

Oh-My-ZSH

The most famous framework with most plugins built-in. For example, autojump, z, fasd are all included, which let users choose whatever they want.

Prezto

The only framework does optimizations in plugins with sophisticated coding skill, which makes me realize I'm ignorant about ZSH:

• Refreshing .zcompdump every 20h
• Compiling .zcompdump as bytecode in the background
• Caching init script for fasd
• Saving *env startup time with init - --no-rehash for rbenv, pyenv, nodenv
• Removing the horribly time-consuming brew command from command-not-found

Prezto modules are loaded by its custom function called pmodload. source the script directly won't work as expected.

And it has a different style compared with oh-my-zsh:

• Avoiding environment variable pollution with zstyle
• Bundling related plugins together
  • module python in Prezto is composed of four parts: pyenv, conda, virtualenv, virtualenvwrapper

ZIM

(I have not tested this. But bytecode compiling should help it be one of the fast frameworks.)

Zsh IMproved FrameWork. According to issue #284:

• bytecode compiling
• coding style focused on efficiency

Drawback: much less plugins are included compared to oh-my-zsh and Prezto.

Plugin Managers

Antigen

The de facto official plugin manager. Antigen supports oh-my-zsh, plugins from github repos. No prezto module support.

Startup time of a clean installation is about 60ms. Startup time using conf example from the README.md is about 150ms. (Exclude the time-consuming command-not-found plugin for macOS, 6 plugins are loaded, maybe with compinit.)

Not bad, but not excellent either.

Optimizations:

• bytecode compiling for itself to reduce the 1st kind of time
• No optimization for the loading of plugins

Zgen

Zgen hitchhikes on oh-my-zsh and prezto. It clones those two repos, and loads their plugins/modules using methods from them. Zgen also supports plugins from github repos.

What makes zgen excellent is its static loading. It generates a static init script consisting of source statements for plugins and pmodload statements for prezto modules. When ZSH starts up, zgen will source the init script directly without parsing its dialect every time, which also means you need to regenerate the init script after updating ur conf. A clean startup time is 50ms.

Optimizations:

• static init, removes the 1st kind of time completely
• No optimization for the loading of plugins

Antibody

Antigen in Go. It supports oh-my-zsh plugins, plugins from github repos. No support for Prezto modules.

Note: For anyone think the plugin manager written in Go is superior to plugin manger written in ZSH. Let's make it clear, Antibody loads plugins using ZSH statement source, which means antibody ONLY reduces the 1st kind of startup time.

The born of antibody may be related to the slowness of old version antigen. Antibody tries to use Go to reduces the startup time taken by plugin manger itself, cause using Go to parse and execute its dialect antibody bundle < ~/.zsh_plugins.txt is faster.

(The slowness of antigen should only be its skeleton in the closet now.)

It adds a new loading method called static loading later. Obviously, parsing the dialect into source statements to cache it directly later, is faster than parsing the dialect every time ZSH starts up. You may guess it, this is basically what zgen is doing using ZSH. The clean startup time of both managers are very close, nearly 50ms.

I kind of feel sorry for the author of antibody, who uses a more complicated, powerful tool to solve the problem, but later found himself defeated by an simple idea.

Optimizations:

• Reduce the 1st kind of time with Go, or
• remove the 1st kind of time completely using static loading
• No optimization for the loading of plugins

A Detailed Comparison Between Antibody and Zgen

Clean startup time without any plugin loaded:

# antibody, source <(antibody init)
for i ({1..10}) time zsh -ilc echo &>/dev/null
zsh  0.03s user 0.02s system 118% cpu 0.047 total
zsh  0.03s user 0.02s system 103% cpu 0.049 total
zsh  0.03s user 0.02s system 107% cpu 0.051 total
zsh  0.03s user 0.02s system 114% cpu 0.045 total
zsh  0.03s user 0.02s system 102% cpu 0.049 total
zsh  0.03s user 0.02s system 105% cpu 0.047 total
zsh  0.03s user 0.02s system 110% cpu 0.046 total
zsh  0.03s user 0.02s system 107% cpu 0.050 total
zsh  0.03s user 0.02s system 106% cpu 0.049 total
zsh  0.03s user 0.02s system 107% cpu 0.048 total

# zgen, source "${HOME}/.zgen/zgen.zsh"
for i ({1..10}) time zsh -ilc echo &>/dev/null
zsh  0.03s user 0.02s system 110% cpu 0.046 total
zsh  0.03s user 0.02s system 106% cpu 0.046 total
zsh  0.03s user 0.02s system 106% cpu 0.048 total
zsh  0.03s user 0.02s system 109% cpu 0.051 total
zsh  0.03s user 0.02s system 105% cpu 0.048 total
zsh  0.03s user 0.02s system 102% cpu 0.051 total
zsh  0.03s user 0.02s system 106% cpu 0.047 total
zsh  0.03s user 0.02s system 107% cpu 0.047 total
zsh  0.03s user 0.02s system 111% cpu 0.049 total
zsh  0.03s user 0.02s system 106% cpu 0.047 total

Tests using example config from getantibody.github.io

# example conf for antibody dynamic loading
antibody bundle << EOF
caarlos0/jvm
djui/alias-tips
# comments are supported like this
caarlos0/zsh-mkc
zsh-users/zsh-completions
caarlos0/zsh-open-github-pr

# empty lines are skipped

# remove plugin below because of time-consuming `brew` command
# robbyrussell/oh-my-zsh path:plugins/aws

zsh-users/zsh-syntax-highlighting
zsh-users/zsh-history-substring-search
EOF

Result:

# antibody dynamic loading
for i ({1..10}) time zsh -ilc echo &>/dev/null
zsh  0.05s user 0.03s system 105% cpu 0.083 total
zsh  0.06s user 0.03s system 105% cpu 0.083 total
zsh  0.05s user 0.03s system 104% cpu 0.082 total
zsh  0.05s user 0.03s system 107% cpu 0.081 total
zsh  0.05s user 0.03s system 104% cpu 0.082 total
zsh  0.05s user 0.03s system 103% cpu 0.081 total
zsh  0.06s user 0.03s system 107% cpu 0.083 total
zsh  0.05s user 0.03s system 105% cpu 0.082 total
zsh  0.05s user 0.03s system 102% cpu 0.082 total
zsh  0.06s user 0.03s system 106% cpu 0.082 total

# antibody static loading, source .zsh_plugins.sh directly. No auto `compint`.
for i ({1..10}) time zsh -ilc echo &>/dev/null
zsh  0.05s user 0.02s system 105% cpu 0.069 total
zsh  0.05s user 0.03s system 107% cpu 0.071 total
zsh  0.05s user 0.03s system 106% cpu 0.071 total
zsh  0.05s user 0.03s system 104% cpu 0.070 total
zsh  0.05s user 0.03s system 104% cpu 0.070 total
zsh  0.05s user 0.03s system 108% cpu 0.072 total
zsh  0.05s user 0.03s system 105% cpu 0.070 total
zsh  0.05s user 0.03s system 105% cpu 0.070 total
zsh  0.05s user 0.03s system 104% cpu 0.071 total
zsh  0.05s user 0.03s system 106% cpu 0.072 total


# zgen static loading, without `compinit`, ZGEN_AUTOLOAD_COMPINIT=0
for i ({1..10}) time zsh -ilc echo &>/dev/null
zsh  0.05s user 0.03s system 105% cpu 0.075 total
zsh  0.05s user 0.03s system 104% cpu 0.077 total
zsh  0.05s user 0.03s system 106% cpu 0.074 total
zsh  0.05s user 0.03s system 104% cpu 0.074 total
zsh  0.05s user 0.03s system 105% cpu 0.076 total
zsh  0.05s user 0.03s system 103% cpu 0.074 total
zsh  0.05s user 0.03s system 108% cpu 0.076 total
zsh  0.05s user 0.03s system 104% cpu 0.075 total
zsh  0.05s user 0.03s system 107% cpu 0.077 total
zsh  0.05s user 0.03s system 102% cpu 0.081 total

# zgen source "${HOME}/.zgen/init.zsh" directly
# without `compinit`, ZGEN_AUTOLOAD_COMPINIT=0
for i ({1..10}) time zsh -ilc echo &>/dev/null
zsh  0.05s user 0.03s system 107% cpu 0.070 total
zsh  0.05s user 0.03s system 106% cpu 0.073 total
zsh  0.05s user 0.02s system 103% cpu 0.070 total
zsh  0.05s user 0.03s system 104% cpu 0.070 total
zsh  0.05s user 0.03s system 109% cpu 0.071 total
zsh  0.05s user 0.03s system 105% cpu 0.070 total
zsh  0.04s user 0.02s system 106% cpu 0.064 total
zsh  0.04s user 0.02s system 107% cpu 0.062 total
zsh  0.04s user 0.02s system 107% cpu 0.061 total
zsh  0.04s user 0.02s system 107% cpu 0.063 total

We used 7 plugins here. The result is clear, static loading saves your time. The only difference between static loading in Antibody and Zgen is, Zgen the plugin manager itself needs to be sourced, the source is run by Zgen itself, and Zgen does compint for you by default. (Disable auto compinit in Zgen with ZGEN_AUTOLOAD_COMPINIT=0)

Everything else are the same. What they do in static loading is sourcing *plugin.zsh into current shell and adding plugin folders into fpath.

Same formula, different packagings. Zgen implements static loading with only 500 lines ZSH script, with additional Prezto support.

Interlude

Now we have examined plugin mangers trying to reduce the startup time taken by themselves. But the time taken by plugin manger itself is only a fraction of the total startup time. Just as what I wrote in the beginning, a simple brew --prefix nvm cost you 600ms, thefuck --alias cost you 120ms. No mater how efficient your plugin manger is, you still get that kind of startup delay.

Next, I'll examine some other plugin managers trying to reduce the 2nd kind of time taken by plugins. Keep tuned.

Zulu

Zulu, a plugin manager uses bytecode compiling both for the plugin manager itself and all plugins being loaded. It only supports built-in plugins, no support oh-my-zsh or prezto plugins. What kind of plugin manger could call itself a plugin manger if it only uses builtin plugins? Let's be practical, zulu is just a framework.

A clean startup (without adding any plugins) time is 150ms. Ok, it's BAD. Compared with Antigen, zulu has more optimizations but a longer startup time, which means the plugin manager itself cost too much time and the implementation is not efficient.

Optimizations:

• bytecode compiling for plugin manger itself. Offset by its low efficient coding.
• bytecode compiling for plugins

Zplug

Zplug uses parallel installation/updating, hooks, cache mechanism. It also supports managing scripts and binary from Github release for you. Oh-my-zsh, Prezto and plugins from Github repos are all supported. It's very full-fledged and powerful.

The problem is, it's a plugin manger begins with good ideas, like parallel and cache, results in a very, very BAD implementation. A clean startup time is 160ms. (Just source the plugin manger itself.)

According to issue #368, issue #364, thread from reddit/r/zsh Zplug is slow?:

Zplug is slower than Oh-my-zsh, Prezto, Zgen. Antigen. Tests made by myself also confirmed the conclusion, it's slower compared with the prezto, zgen with the same plugins loaded. The time chart on zplug's Github repo is unreliable.

I'll quote what I said on issue issue #368 here:

At first, I thought it as a compatibility problem of prezto modules. But after commenting all the prezto modules, I got a startup time of some 0.6 second in total, which is basically the same startup time when i used prezto. If I enable the prezto modules I use, startup time is about 1 second.

This is ridiculous because when I used the perzto framework, it loaded more plugins and had a time-consuming initialization of pyenv.

Sorry, I didn't mean to be so rude. What irritates me is, the content from zpulg's README.md makes you hold your breath, but what I experienced is totally different from what they tout.

Optimizations:

• Negative optimization brought by terrible implementation for plugin manager itself.
• Parallel loading, cache mechanism for loading of plugins

Zplugin

Zplugin, another full-fledge plugin manger with script, binary management, reports, completion management, turbo mode, services.

• reports, zplugin time list time taken by each plugin
• completion management: enable, disable specific completion easily
• Async Turbo Mode: the real killer for time-consuming plugins

Zplugin supports plugins from oh-my-zsh, Prezto, Github repos.

At first, it didn't catch my eye because it's less popular compared to zplug, which offers most of the same features. Anyway, I already explained my relationship with zplug.

A clean startup time is 50ms. Bytecode compiling is made for manger itself and all plugins. This helps zplugin get a slower startup time increase compared to plugin mangers without bytecode compiling for plugins.

Async Turbo Mode, allows you to postpone loading of a plugin to the moment when processing of .zshrc is finished and prompt is being shown. Besides, there are no drawbacks of this approach – no lags, freezes, etc.

From the data I offered at the beginning at the post. Zplugin loads my 37 plugins with a startup time of 160ms. The real time taken by all plugins is 1.040s, which means 880ms is reduced in Turbo Mode.

Optimizations:

• bytecode compiling for plugin manger itself
• bytecode compiling for plugins
• Amazing Turbo Mode kills the time-consuming plugins like nvm, thefuck, pyenv, nodenv, etc.

FAQ

Why don't you make a benchmark?

Sorry, I won't do it, because I think it's unnecessary. According to my analytical method, any plugin manger not trying to reduce the time taken by plugins, are not qualified in the finals. Just as what I said, the startup time taken by plugin manager itself only takes a fraction of the total time. Time-consuming plugins are the main causes of a slow startup of ZSH.

Among all the plugin managers I've tested, Zplug and Zplugin are the only two trying to solve the hardest part, which brings them to the final round. Although Zplug's implementation is bad, I still hold the opinion that its parallel mechanism is more innovative than static loading and bytecode compiling.

I'd like to apologize to any plugin manager developer who feels hurt by my word. I do respect their works and they are the men who make the community of ZSH bigger than any other shell. (The community here, is based on the number of plugins and plugin mangers of a interactive shell. awesome-zsh-plugins)

While, I'm just a ZSH user like most other guys, and always want the best. Users don't care which one is better among those mediocre products. Competition is cruel.