Off the beaten path

What is `&path` used for?

Vim uses :help 'path' to define the root directories from where to search non-recursively for files.

It is used for:

gf, gF, <C-w>f, <C-w>F, <C-w>gf, <C-w>gF,
:find, :sfind, :tabfind,
[i, ]i, [I, ]I, [<C-i>, ]<C-i>, <C-w>i, :isearch, :ilist, :ijump, :isplit, :psearch
[d, ]d, [D, ]D, [<C-d>, ]<C-d>, <C-w>d, :dsearch, :dlist, :djump, :dsplit,
<C-n> and <C-p> in insert mode if you have i and/or d in &complete,
<C-x>i and <C-x>d in insert mode,
findfile(), globpath(), getcompletion().

Yes, there are lots of things that depend on &path, and that is why it is a good idea to weight the consequences when changing its value.

Ants in these pants

Navigating to another file is an extremely common action when programming and Vim has a number of commands for that. :help :edit, for example, will let you edit a single file matching a simple glob, like:

:edit **/fo*/ba*/main.rb

:help :args will let you edit all the files you feed to it:

:args **/fo*/ba*/*.rb

and those two commands are just the primary representatives of large families…

:help :find is similar to :edit in that it will only edit one file, but it differs from both :edit and :args in one big way: it looks for the given glob in default directories defined in &path, the subject of this document.

Suppose…

your working directory is the root of this tree,

+ dir1/
  + file1
  + file2
+ dir2/
  + subdir1/
    + subsubdir1/
      + file3     <-- current buffer
  + subdir2/
    + file4
    + file5
    + file6
  + subdir3/
    + file7
    + file8
  + file9
  + file10
+ dir3/
  + file11
  + file12
  + file13
+ file14
+ file15

you are editing dir2/subdir1/subsubdir1/file3,
and you do :find fi<Tab>.

It should work like this:

If &path includes ., then Vim will start searching from dir2/subdir1/subsubdir1/, the directory of the current file, and return dir2/subdir1/subsubdir1/file3.

If &path includes ,,, then Vim will start searching from the the working directory and return file14 and file15.

If &path includes .,,, then two searches are performed: one from dir2/subdir1/subsubdir1/ and the other from the working directory, returning dir2/subdir1/subsubdir1/file3, file14, and file15.

If &path also includes dir3/, then Vim is going to perform a third search, from dir3/, and also find the three files it contains.

The default value includes .,, so we have our bases covered. It is only a matter of adding directories to the list to make this option useful.

The lazy way

From there, it might be tempting to add ** to &path in order to force Vim to search recursively. In practice, this will make Vim perform 7 searches, one for every subdirectory under the working directory, effectively finding every file from 1 to 15. :find fi<Tab> is instantaneous, the sky is blue, and the birds are singing.

Case closed, right?

Our sample project is pretty small so everything is going to be quick. What if, instead of a toy demonstration project, we were in a more realistic one? Say my current project, with 1054 files scattered around 523 directories? Well, it looks like :find acco<Tab> is still instantaneous and the birds are still singing.

Now let's add the bane of every front-end project: a node_modules/ directory from the same real world project: 57994 files in 5596 directories, bringing our total to 59048 files in 6119 directories. "Heaviest object in the Universe" indeed. Let's :find acco<Tab> again…

Hmm… we got greeted by a mysterious ... for about 3 seconds before being served some result. It's not the end of the world but 3 seconds is 3 seconds.

There is another problem: all of the above assumes that we don't have :help 'wildmenu' enabled. You see, building the menu can be slow and our search is already too slow so this is not looking good. Let's enable :set wildmenu and :find acco<Tab> again…

…

OK, I have lost patience after one minute of nothing but ... and I am not even curious about how long it would have taken in total.

Yes, 3 seconds for the search plus who knows how long for building the menu really sucks. But we are not done, right? We can still use :help 'wildignore' to filter out that pesky node_modules/:

set wildignore+=*/node_modules/*

Well yes, we can do that, and it will work, for some definition of "work":

&wildignore is only applied after the search, to build the list of candidate for :help 'wildmenu',
it is only used for the wildmenu so it is only used for the :find family of commands.

In our case, we have managed to accelerate the menu building part but the search still takes 3-4 seconds, which is still way too slow, wildmenu or not. We kind of managed to make :find bearable but we still have that seemingly incompressible 3-4 seconds delay, which will be present in one way or another in some other &path-aware contexts.

In short, ** induces a performance penalty that may be acceptable in some cases and unbearable in others. We can't count on blacklisting so that solution is very suboptimal.

The short sighted way

One could think "what if I just left &path at its default value and instead used ** in my query?" but that would be both ineffective and short sighted.

Ineffective because :find **/acco<Tab> would still search everywhere, including irrelevant places, so you still get that 3-4s tax.

Short sighted because it only deals with one single use case for &path.

No. That global unconditional ** is not the solution, either in the query or in &path.

The smart way

Instead of silly catchall wildcards and semi-random &wildignore hacks, let's try to use &path as it is supposed to be used: as a white list of interesting places.

Since we are in a front-end project, there are a number of places where we will never go willingly: coverage/, dist/, node_modules/, etc. but there are others that we are going to frequent a lot: every directory under src/, maybe config/ and static/, etc. YMMV, of course.

We have seen earlier that blacklisting via &wildignore didn't affect search, which appears to be a real bottleneck, so we will turn to whitelisting instead, and add our list of interesting places to the default value:

set path+=src/**,static/,config/

OK, it looks good. Let's try to :find acco<Tab> one more time…

That is the kind of real world benefit one would expect from a single well understood and properly set built-in option: Vim now knows where to look for interesting things and we now can edit any interesting file instantly.

Case closed for good.

Other uses

So far, we have focused on :find, which is only one of the many uses of &path, but what about the others?

`gf` and friends

gf is essentially like doing :find with the filename under the cursor, without any listing involved. This means that there is no bottleneck beside the actual search, which, as we have seen already can be very slow with an improper &path. The longer it takes to find that file, the less useful gf is so it is quite important to make sure that it doesn't waste time looking for your file where it has zero chance to find it.

This is done with a properly defined &path.

Include search and definition search

When following an include, it is vital to tell Vim where to look for files in the most precise manner possible, especially if you have lots of includes or if a complex :help includexpr is involved.

This is done with a properly defined &path.

Vimscript functions

findfile() and globpath() use &path as-is by default, so they will certainly benefit from a properly defined &path, but they can also be made to use an arbitrary "path" if needed.

getcompletion() only uses &path for one completion type but that single completion type will certainly benefit from a properly defined &path.

My Vim-related gists.

richlamdev/path.md