How to use clangd C/C++ LSP in any project

clangd.md

How to use clangd C/C++ LSP in any project

tl;dr: If you want to just know the method, skip to How to section

Clangd is a state-of-the-art C/C++ LSP that can be used in every popular text editors like Neovim, Emacs or VS Code. Even CLion uses clangd under the hood. Unfortunately, clangd requires compile_commands.json to work, and the easiest way to painlessly generate it is to use CMake.

For simple projects you can try to use Bear - it will capture compile commands and generate compile_commands.json. Although I could never make it work in big projects with custom or complicated build systems.

But what if I tell you you can quickly hack your way around that, and generate compile_commands.json for any project, no matter how compilcated? I have used that way at work for years, originaly because I used CLion which supported only CMake projects - but now I use that method succesfully with clangd and Neovim.

Method summary

Basically what we need to achieve is to create a CMake file that will generate a compile_commands.json file with information about:

1. All source files
2. All include directories
3. External libraries
4. Precompiler definitions

We can do that easily without really caring about if the CMake-generate result will compile at all - we don't need to rewrite our existing build system, just hack a CMake file that will generate enough information for Clangd to work.

Prerequisities

1. CMake
2. clangd

How to

First, create a CMakeLists.txt file in the root folder of your projects, with content similar to this:

cmake_minimum_required(VERSION 3.8)
project(my_project)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

# Change path from /src if needed, or add more directories
file(GLOB_RECURSE sources
        "${CMAKE_SOURCE_DIR}/src/*.c"
        "${CMAKE_SOURCE_DIR}/src/*.cpp"
        )
# Add precompiler definitions like that:
add_definitions(-DSOME_DEFINITION)

add_executable(my_app ${sources})

# Add more include directories if needed
target_include_directories(my_app PUBLIC "${CMAKE_SOURCE_DIR}/include")

# If you have precompiled headers you can add them like this
target_precompiled_headers(my_app PRIVATE "${CMAKE_SOURCE_DIR}/src/pch.h")

(If your project already uses CMake, then you just need to add set(CMAKE_EXPORT_COMPILE_COMMANDS ON) to your main CMakeLists.txt file.)

Modify hacky CMakeLists.txt according to your project structure, and run:

cmake -S . -G "Unix Makefiles" -B cmake

which will generate the CMake output inside cmake directory. Check if compile_commands.json is there.

NOTE: You need to run that command every time you add/remove a source file in your project.

If you need more (ex. include external libraries like Boost), check out CMake documentation

Now you have two options:

1. Symlink compile_commands.json to your root project folder:

ln -s cmake/compile_commands.json .

OR

2. Create .clangd file in your root project folder, with the following contents:

CompileFlags:
  CompilationDatabase: "cmake"

Now open the project in you editor and everything should work (assuming clangd LSP is started).

Beautiful!

This was helpful, thank you.

The important thing to have in your CMakeLists.txt is set(CMAKE_EXPORT_COMPILE_COMMANDS ON), this makes cmake create compile_commands.json in the build dir and then use either a .clangd file or symlink to point clangd to it.

This is the best day of my life. Thank you so much.

It might be good to mention Bear - https://github.com/rizsotto/Bear. All you do is run bear -- (your compile command) and it gets all the commands that your build system runs and spits out a compile_commands.json for you.

For example, I ran bear -- cmake --build ./build in a repository that I needed to use clangd in and compile_commands.json was created automagically by bear in my working directory.

I found Bear a while after finding this gist, so I hope this can help some others who are in the same situation I was in.

@PowerUser64 Yeah, I know this tool - unfortunately I was never able to use it in practice. It works for small projects with simple build systems, but most big C/C++ projects that use Makefiles or some custom mambo-jumbo make it very hard to inject bear there. At work I always ended up with a hack CMake like this.

But I will add a mention that for simple projects one could try bear or similar tools.

BTW, in your example - if you use cmake there is no need to use bear. Just add set(CMAKE_EXPORT_COMPILE_COMMANDS ON) to your main CMakeLists.txt.

if you use cmake [...], just add set(CMAKE_EXPORT_COMPILE_COMMANDS ON) to your main CMakeLists.txt.

Ooh, I didn't know that! Thought the other stuff in the example was somehow required too. Might be good to mention that on its own too somewhere. Thanks for the tip!

Good idea, I've added a mention about that.

Thank you for this helpful post !!!

Thank you man! Now I can get rid of proprietary IDEs!

Is it possible to make a script that generates the CMakeLists.txt file?

@SmilingJack1 It depends. A generic script? No. A specific script that would generate a CMake for the needs of your project? Yes, as it is just a text document. Although you would need to do it on your own.

For example I did that a few times for my work projects, where there was a custom build system that was using some file format to specify which repositories should be used by a specific project - so I parsed that files, and then generated a CMake file similar to the one you see in the gist, but which paths to different repositories.

Thanks a lot for this!!

@PowerUser64 Yeah, I know this tool - unfortunately I was never able to use it in practice. It works for small projects with simple build systems, but most big C/C++ projects that use Makefiles or some custom mambo-jumbo make it very hard to inject bear there. At work I always ended up with a hack CMake like this.

But I will add a mention that for simple projects one could try bear or similar tools.

BTW, in your example - if you use cmake there is no need to use bear. Just add set(CMAKE_EXPORT_COMPILE_COMMANDS ON) to your main CMakeLists.txt.

Interesting take. In my experience, projects using Makefiles, completely irrespective of size usually allow you to insert anything in place of their compilation commands, without even touching the Makefiles containing those commands.

Just:

~/some/place/bin/clang:

#!/bin/bash
bear clang "$@" # Note: Never used bear, it may only need the compiler's arguments and not its name.

$ chmod u+x ~/some/place/bin/clang

Submakes inherit environment variables form the parent process, so:

PATH=~/"some/place/bin:${PATH}" make

This should even work for projects using any kind of build system in practice. This could theoretically be false of course, e.g. for a tool that resets its environment to heuristic defaults, like cron does, which is, granted, not a build tool.

@Nnarol In theory yes, that approach should work - but I've always run into issues when using bear and just gave up. I've used the "hack" described in this gist for almost 8 years on Linux/Windows/macOS in different projects without issues and just stuck with it.

Maybe bear is better now than it was when I've tired it, I will give it a try again and see how it goes.

Thanks a lot.

I'm using the following CMakeLists file for the raddebugger project:

cmake_minimum_required(VERSION 3.8)
project(raddebugger)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

# Change path from /src if needed, or add more directories
file(GLOB_RECURSE sources
        "${CMAKE_SOURCE_DIR}/src/*.c"
        "${CMAKE_SOURCE_DIR}/src/*.cpp"
        )

# Add precompiler definitions like that:
add_definitions(-DSOME_DEFINITION)

add_executable(raddbg ${sources})

I does NOT work.
I don't have any symbols that are outside the current file.
Neovim is indexing around 215 files after I open a project file.

Also, the command to generate a symlink doesn't work. Both in Powershell and cmd.
The compile_commands.json file has 1285 lines.

I tried both methods: copy/link the compile_commands.json and adding the .clangd file

@Takao-7 You probably also need to add this at the end:

target_include_directories(raddbg PUBLIC "{CMAKE_SOURCE_DIR}/src")

@Strus
Thanks for the suggestion.
I've added that line (or rather un-commented it), however the errors are still there.

One thing to note is that the project contains two different programs: One to generate introspection data ("metagen") and the actual program.

The errrors are visible in source files in both programs.

There is a 4Coder config file that has what looks like include search patterns, including a blacklist:

Another note:
compile_commands.json doesn't contain any header file names. This is after you suggested addtion.
The size of the file also hasn't changed.

@Takao-7 Oh man, there was a typo in the gist, which I copied 🥲 $ was missing. This is the correct line:

target_include_directories(raddbg PUBLIC "${CMAKE_SOURCE_DIR}/src")

I have checked with the following CMakeLists.txt and it works for me:

cmake_minimum_required(VERSION 3.8)
project(raddebugger)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

# Change path from /src if needed, or add more directories
file(GLOB_RECURSE raddbg_sources
        "${CMAKE_SOURCE_DIR}/src/*.c"
        "${CMAKE_SOURCE_DIR}/src/*.cpp"
        )
list(FILTER raddbg_sources EXCLUDE REGEX ".+metagen.+")
list(FILTER raddbg_sources EXCLUDE REGEX ".+metagen_base.+")

add_executable(raddbg ${raddbg_sources})
target_include_directories(raddbg PUBLIC "${CMAKE_SOURCE_DIR}/src")

I also fixed the typo in the gist now.

Indeed it's actually working :)
Thank you so much.

Do you know if there an easy way to run this automatically in Neovim?

edit:
What is the difference between "my_project" and "my_app"?
Is the first the project folder and the second the name of the executable?

It looks like there are still some errors:

The macro internal from the previous screenshot is defined in that base_core.h file on the left.

@Takao-7 base_arena.h does not have any includes, so clangd does not know where to get that definitions. Fixing that would require knowledge about how this project is being build. Maybe there is some processing magic there, maybe this file is only included in other files and other includes are included before etc. I don't know, but there is something unusual done there.

So clang doesn't keep a "database" of symbols it found throughout the code, but instead only looks through the files included in the current file?

It uses clang under the hood to parse the code and create an AST. If you would try to compile this header file, it will not compile on it's own as there are no includes there that provides definitions for all of these symbols. But, #include directive in C++ is just a copy-paste of a text into the file (#include is replaced with file contents by preprocessor). So, one explanation is that this include file is used in code like this:

#include <some_other_header_providing_all_definitions.h>
#include <base/base_arena.h>

// code

This will work and compile, but clang is not able to parse this file correctly, as this is not a standard approach.

Hej,

I'm running into another problem.
I have a folder with .c and .h files which is outside of the project folder.
Adding the folder path to "file()" will add all .c files in it correctly to the compile_commands:

file(GLOB_RECURSE sources
        "${CMAKE_SOURCE_DIR}/source/*.c"
        "${CMAKE_SOURCE_DIR}/../Basics/*.c"
        )

However when adding include directories, none of the files or folders are listed in compile_commands:

target_include_directories(DeepSeekLocalTarget
        PUBLIC "${CMAKE_SOURCE_DIR}/../Basics"
        PUBLIC "${CMAKE_SOURCE_DIR}/ThirdParty/libcurl/include"
        PUBLIC "${CMAKE_SOURCE_DIR}/ThirdParty/cJSON"
)

I can include these files directly, without a path, and I don't get any errors in neovim.
Furthermore, I can "go to" the files through the include itself.

BUT I can't find any files from the "Basics" folder when searching for files.
I can find symbols in these files when I serach for workspace symbols.

BUT I can't find any files from the "Basics" folder when searching for files.
I can find symbols in these files when I serach for workspace symbols.

@Takao-7
I don't know how you perform search in NeoVim, but I guess you have your cwd set to your project directory, and whatever you use for searching to search within cwd, which is probably the default behavior for Telescope/fzf-lua/whatever.

So you need to configure whatever you use for search to include that ../Basics directory too.

I thought that the lsp was doing the search, but it makes sense that it doesn’t.

I’m using Kickstart Neovim, so Telescope is searching for files.

Is it possible to automatically add all folders that are found by cmake to the list?
Or would I need to manually add them?

I think that if you will start nvim in the parent directory that should do the trick, assuming you don't have other plugins that affects cwd. So, if your tree looks like this:

my_projects
|- some_utils
|- project_in_c

and you want search to work in both some_utils and project_in_c, then you should:

cd my_projects
nvim            # or nvim . maybe