I am exploring Envoy to be deployed as L7 edge proxy on a bare-metal high configuration server on Linux Ubuntu 22.04. to benchmark its performance. So I wanted to build envoy and generate compile_commands.json for entire source code including dependencies and browse the code using Vim and youcompleteme.

I have never worked with bazel and have experience with cmake and Makefile. I use vim and youcompleteme which internally uses clangd LSP for auto-completion/jump-to-definition/references etc.

It was rather surprising to see that Bazel doesn't have built-in support for producing compile_commands.json like cmake(correct me if I am wrong) and that bear doesn't work with bazel

Using the instructions here to build Envoy and then generate compile_commands.json

# Build fails with clang-16 so using clang-14
bazel/setup_clang.sh /home/stonebraker/build/14_llvm/

echo "build --config=libc++" >> user.bazelrc

bazel build --verbose_failures --subcommands --compilation_mode=opt //source/... //test/... //contrib/...

Steps to generate compilation database -

# Want to include all dependecies as well hence --include_all
tools/gen_compilation_database.py --include_all

This generates compile_commands.json with ~90K lines in the workspace directory

Now I expect vim and youcomplete to work fine but find that :YcmCompleter GoToDefinition :YcmCompleter GoToReferences etc not working.
e.g If I open source/exe/main.cc and try to go to definition of Envoy::MainCommon::main(argc, argv) i.e line 24 it takes me to source/exe/main_common.h(which contains declaration of that static function) instead of source/exe/main_common.cc(which actually contains the definition). Similarly :YcmCompleter GoToReferences fails to fetch the references.

So far I've never faced any problems with my setup in any cmake-based projects(which span hundreds of thousands of lines of code and have dozens of third-party dependencies)

FYI I use the following in my .vimrc

let g:ycm_clangd_uses_ycmd_caching = 0
" Use installed clangd, not YCM-bundled clangd which doesn't get updates.
let g:ycm_clangd_binary_path = exepath("/home/stonebraker/build/14_llvm/bin/clangd")
let g:ycm_clangd_args = ['-log=verbose', '-pretty', '--clang-tidy']

Maybe I am doing something very basic wrong and so need your help(already spent 4+ hours) on this.

Some related/unrelated questions:-

1. Is tools/gen_compilation_database.py --include_all the idiomatic way to generate the compilation database for Envoy. Does it depend upon the build step as a pre-requisite? Does it automatically pick up build options like --compilation_mode=opt fired during the build step?

2. If I want to have output_base directory inside workspace directory, will it create problems?

# OutputBase directory in the working space. Possible?
bazel --output_base /home/stonebraker/envoy/build build --verbose_failures --subcommands --compilation_mode=opt //source/... //test/... //contrib/...

How to generate compile_commands in this case?

After trying a lot I finally tried bazel-compile-commands-extractor and have to say that it works way better than the tool provided with Envoy. The compile_commands.json generated by this tool has ~5Million lines(partly because it gives 1 line for each compilation switch). But it gives correct references and definitions right out of the box.

However, even that tool has problems(and I'm going to create an issue in their github) finding certain files e.g event2/event.h which is present in external/com_github_libevent_libevent/include/event2/

What am I missing here?

Regards
Tom Stonebraker