Off the beaten path

path.md

Off the beaten path

What is &path used for?

Vim uses :help 'path' to define the root directories from where to search non-recursively for files.

It is used for:

• gf, gF, <C-w>f, <C-w>F, <C-w>gf, <C-w>gF,
• :find, :sfind, :tabfind,
• [i, ]i, [I, ]I, [<C-i>, ]<C-i>, <C-w>i, :isearch, :ilist, :ijump, :isplit, :psearch
• [d, ]d, [D, ]D, [<C-d>, ]<C-d>, <C-w>d, :dsearch, :dlist, :djump, :dsplit,
• <C-n> and <C-p> in insert mode if you have i and/or d in &complete,
• <C-x>i and <C-x>d in insert mode,
• findfile(), globpath(), getcompletion().

Yes, there are lots of things that depend on &path, and that is why it is a good idea to weight the consequences when changing its value.

Ants in these pants

Navigating to another file is an extremely common action when programming and Vim has a number of commands for that. :help :edit, for example, will let you edit a single file matching a simple glob, like:

:edit **/fo*/ba*/main.rb

:help :args will let you edit all the files you feed to it:

:args **/fo*/ba*/*.rb

and those two commands are just the primary representatives of large families…

:help :find is similar to :edit in that it will only edit one file, but it differs from both :edit and :args in one big way: it looks for the given glob in default directories defined in &path, the subject of this document.

Suppose…

• your working directory is the root of this tree,

  + dir1/
    + file1
    + file2
  + dir2/
    + subdir1/
      + subsubdir1/
        + file3     <-- current buffer
    + subdir2/
      + file4
      + file5
      + file6
    + subdir3/
      + file7
      + file8
    + file9
    + file10
  + dir3/
    + file11
    + file12
    + file13
  + file14
  + file15
  

• you are editing dir2/subdir1/subsubdir1/file3,

• and you do :find fi<Tab>.

It should work like this:

If &path includes ., then Vim will start searching from dir2/subdir1/subsubdir1/, the directory of the current file, and return dir2/subdir1/subsubdir1/file3.

If &path includes ,,, then Vim will start searching from the the working directory and return file14 and file15.

If &path includes .,,, then two searches are performed: one from dir2/subdir1/subsubdir1/ and the other from the working directory, returning dir2/subdir1/subsubdir1/file3, file14, and file15.

If &path also includes dir3/, then Vim is going to perform a third search, from dir3/, and also find the three files it contains.

The default value includes .,, so we have our bases covered. It is only a matter of adding directories to the list to make this option useful.

The lazy way

From there, it might be tempting to add ** to &path in order to force Vim to search recursively. In practice, this will make Vim perform 7 searches, one for every subdirectory under the working directory, effectively finding every file from 1 to 15. :find fi<Tab> is instantaneous, the sky is blue, and the birds are singing.

Case closed, right?

Our sample project is pretty small so everything is going to be quick. What if, instead of a toy demonstration project, we were in a more realistic one? Say my current project, with 1054 files scattered around 523 directories? Well, it looks like :find acco<Tab> is still instantaneous and the birds are still singing.

Now let's add the bane of every front-end project: a node_modules/ directory from the same real world project: 57994 files in 5596 directories, bringing our total to 59048 files in 6119 directories. "Heaviest object in the Universe" indeed. Let's :find acco<Tab> again…

Hmm… we got greeted by a mysterious ... for about 3 seconds before being served some result. It's not the end of the world but 3 seconds is 3 seconds.

There is another problem: all of the above assumes that we don't have :help 'wildmenu' enabled. You see, building the menu can be slow and our search is already too slow so this is not looking good. Let's enable :set wildmenu and :find acco<Tab> again…

…

…

…

OK, I have lost patience after one minute of nothing but ... and I am not even curious about how long it would have taken in total.

Yes, 3 seconds for the search plus who knows how long for building the menu really sucks. But we are not done, right? We can still use :help 'wildignore' to filter out that pesky node_modules/:

set wildignore+=*/node_modules/*

Well yes, we can do that, and it will work, for some definition of "work":

• &wildignore is only applied after the search, to build the list of candidate for :help 'wildmenu',
• it is only used for the wildmenu so it is only used for the :find family of commands.

In our case, we have managed to accelerate the menu building part but the search still takes 3-4 seconds, which is still way too slow, wildmenu or not. We kind of managed to make :find bearable but we still have that seemingly incompressible 3-4 seconds delay, which will be present in one way or another in some other &path-aware contexts.

In short, ** induces a performance penalty that may be acceptable in some cases and unbearable in others. We can't count on blacklisting so that solution is very suboptimal.

The short sighted way

One could think "what if I just left &path at its default value and instead used ** in my query?" but that would be both ineffective and short sighted.

Ineffective because :find **/acco<Tab> would still search everywhere, including irrelevant places, so you still get that 3-4s tax.

Short sighted because it only deals with one single use case for &path.

No. That global unconditional ** is not the solution, either in the query or in &path.

The smart way

Instead of silly catchall wildcards and semi-random &wildignore hacks, let's try to use &path as it is supposed to be used: as a white list of interesting places.

Since we are in a front-end project, there are a number of places where we will never go willingly: coverage/, dist/, node_modules/, etc. but there are others that we are going to frequent a lot: every directory under src/, maybe config/ and static/, etc. YMMV, of course.

We have seen earlier that blacklisting via &wildignore didn't affect search, which appears to be a real bottleneck, so we will turn to whitelisting instead, and add our list of interesting places to the default value:

set path+=src/**,static/,config/

OK, it looks good. Let's try to :find acco<Tab> one more time…

That is the kind of real world benefit one would expect from a single well understood and properly set built-in option: Vim now knows where to look for interesting things and we now can edit any interesting file instantly.

Case closed for good.

Other uses

So far, we have focused on :find, which is only one of the many uses of &path, but what about the others?

gf and friends

gf is essentially like doing :find with the filename under the cursor, without any listing involved. This means that there is no bottleneck beside the actual search, which, as we have seen already can be very slow with an improper &path. The longer it takes to find that file, the less useful gf is so it is quite important to make sure that it doesn't waste time looking for your file where it has zero chance to find it.

This is done with a properly defined &path.

Include search and definition search

When following an include, it is vital to tell Vim where to look for files in the most precise manner possible, especially if you have lots of includes or if a complex :help includexpr is involved.

This is done with a properly defined &path.

Vimscript functions

findfile() and globpath() use &path as-is by default, so they will certainly benefit from a properly defined &path, but they can also be made to use an arbitrary "path" if needed.

getcompletion() only uses &path for one completion type but that single completion type will certainly benefit from a properly defined &path.

---

My Vim-related gists.

Hi! Thank you very much for this gist. I'm trying to comprehend vim, sure not without troubles, but I like it at all.
I program using python, so what do you think about usage wildignore for my case?
Let me clarify, may be you know that python projects have a such structure that contains generated cache folder in each module folder and also e.g. a project that I participate has many directories for config, docs and so on files from which I also could edit.
This means if I will follow to your recomendation I have to set path variable for each project separately because in python there is no src directory but in case of wildignore I have to add there only pycache folder git and few other directories that common for any python project.

@bdrum suppose the interesting directories in some of your projects are orange, banana, and kiwi while they are pizza and paella in some others. In that case, a valid &path would be .,orange/**,banana/**,kiwi/**,pizza/**,paella/**,, which will work equally well in both kinds of projects whether the directory exists or not. With that value, you don't even have to add .git or pycache to &wildignore because a) &wildignore is only used after the search and b) Vim will never search in .git or pycache with the &path above.

So…

" in after/ftplugin/python.vim
set path=.,orange/**,banana/**,kiwi/**,pizza/**,paella/**,,

Ok, but each folder that you have supposed contains pycache, and each folder inside of one of that you supposed will also contains pycache .

Let's suppose that I have 3 different python projects with let say 3-5 modules(top level folders) each.
That means I have specify in my vimrc path that will have about 10 folders?
Moreover In this case If I will work on one project will vim search files in other projects too?

Could you add here a screenshot or a diagram of your multi-project setup so that I can have a better idea?

Yes, sure, thanks for your efforts!
And just for notice I'm interested in this because find is super slow for me right now, so I'm trying to make it fast as it should be.

There is a project

And there is its volume:
1300 python files
320 total folders
180 folders without pycache

And actually fzf work very fast, but just for interesting. What should I do in pure vim.
Ah, and sorry, also forgot to mention I use neovim...

I've got an answer to my question here - Don't use Vim

That's a very good point of view. Thanks! 👏

Is an autocmd that adds the tracked directories of the current repo such as

autocmd BufReadPost,BufNewFile *
            \ if !empty(FugitiveGitDir()) |
            \   Glcd |
            \   let &l:path = &g:path . join(systemlist('git ls-tree -d --name-only -r HEAD'), ',') |
            \ endif

sensible?

@Konfekt I think so, yes. Any method for generating an exhaustive white list is fair game in my opinion. Using one's VCS is one way, using one's language/framework's introspection features would be another way, etc.

Thank you very much for this gist! It's amazing that vim has this capability of finding files without any plugins - I am very surprised that this method is not more popular, e.g. in courses and articles for vim, where they always recommend some plugin for finding files.

As you have pointed out, if configured properly, finding files with vim is very powerful indeed.