YCM reporting errors not present when code is actually built

[davidzchen]

First of all, thank you for building this wonderful tool and enabling integration with YouCompleteMe for Bazel.

However, I am running into a few issues. YouCompleteMe, using the compile_commands.json generated by bazel-compile-commands-extractor, is reporting error messages that I do not get when I run bazel build on the code. In particular, I am getting errors like these in the source files:

Out-of-line definition of 'Foo' does not match any declaration in 'foo_namespace::Foo' [member_decl_does_not_match]

...as well as "no matching constructor" errors. Curiously, this seems to be happening for all my constructors, all static methods, and some private methods.

Has anyone run into this type of issue or know what the fix is? What information should I provide to help with debugging?

Thank you in advance.

[cpsauer]

Hey David! You're very welcome. Sorry I'm just getting to this--and thanks for giving the tool a go.

I'm not sure offhand, without more info. Is clangd finding the headers? It might also be worth checking its log to make sure it's finding the command? I know there are people using it with YCM--but we're using from VSCode.

The reason for those questions is that I'd have two hypotheses about what might be going wrong. (1) Either YCM isn't finding the compile_commands.json or (2) the compile_commands.json might be wrong and need some target configuration (as in that section of the README)

[davidzchen]

No problem. Thank you for getting back to me.

I am not sure if clangd is finding the headers. How do I find the clangd logs?

I will also put together a repo with a minimal repro case and share that with you to help with debugging.

[cpsauer]

Logs: I'm not sure--we're using from VSCode, but I'll bet it says in their docs somewhere!

@stonebrakert6, are you still using this with YCM? And if so, have you seen anything like this?

[stonebrakert6]

@davidzchen You can use :YcmToggleLogs and then open the appropriate log file and check its contents to see where errors occur.
@cpsauer Yeah I am using YCM and no I don't see any problems mentioned by @davidzchen here is head of my clangd logs

I[11:19:53.661] clangd version 19.0.0git (https://github.com/llvm/llvm-project babbdad15b8049a6a78087d15a163d897f07d320)
I[11:19:53.661] Features: linux
I[11:19:53.661] PID: 7539
I[11:19:53.661] Working directory: /home/kartik/codeberg/app0
I[11:19:53.661] argv[0]: /opt/git_llvm/bin/clangd
I[11:19:53.661] argv[1]: --log=verbose
I[11:19:53.661] argv[2]: -pretty
I[11:19:53.661] argv[3]: --clang-tidy
I[11:19:53.661] argv[4]: -j=16
I[11:19:53.661] argv[5]: --background-index
I[11:19:53.661] argv[6]: --header-insertion=iwyu
I[11:19:53.661] argv[7]: --completion-style=detailed
I[11:19:53.661] argv[8]: --compile-commands-dir=/home/kartik/codeberg/app0
I[11:19:53.661] argv[9]: -header-insertion-decorators=0
V[11:19:53.661] User config file is /home/kartik/.config/clangd/config.yaml
I[11:19:53.661] Starting LSP over stdin/stdout

My system is Ubuntu 22.04 and I use vim 9.1
But I do want to highlight that I face the following problems.

Need to convert relative paths in compile_commands.json to absolute paths

I have to run a utility to convert the relative paths in compile_commands.json generated by bazel-compile-commands-extractor to absolute paths. If I don't do so, my GoToDefinition and other GoTo... commands just end up in the header file and not the exact source code definition.

I don't know why this happens and if what I do(below) is recommended or not.

Here is what I do

bazel run --config=dbg //:refresh_compile_commands && python3 ~/tools/cc_modify.py compile_commands.json

and here is the source code(in case someone needs it) for cc_modify.py(it just converts relative paths to absolute paths in compile_commands.json)

import re
import subprocess

def replace_pattern(file_name, search_pattern, replacement_pattern):
    # Read the contents of the file
    with open(file_name, 'r') as file:
        file_content = file.read()

    # Perform the replacement using regular expressions
    new_content = re.sub(search_pattern, replacement_pattern, file_content)

    # Write the updated content back to the file
    with open(file_name, 'w') as file:
        file.write(new_content)

# Define the file name
file_name = 'compile_commands.json'

# Read the contents of the file
with open(file_name, 'r') as file:
    file_content = file.read()

# Call the function to replace the first pattern
search_pattern1 = r'"(external/.*?)"'
output_base = subprocess.check_output(['bazel', 'info', 'output_base']).decode('utf-8').strip()
replacement_pattern1 = fr'"{output_base}/\1"'
file_content = re.sub(search_pattern1, replacement_pattern1, file_content)

# Call the function to replace the second pattern
search_pattern2 = r'"bazel-out/(.*?)"'
output_path = subprocess.check_output(['bazel', 'info', 'output_path']).decode('utf-8').strip()
replacement_pattern2 = fr'"{output_path}/\1"'
file_content = re.sub(search_pattern2, replacement_pattern2, file_content)

# Write the updated content back to the file
with open(file_name, 'w') as file:
    file.write(file_content)

3rd party libraries which need rules_foreign_cc don't give completions

e.g liburing which use ./configure && make i.e for which I have liburing.BUILD that looks like this

load("@rules_foreign_cc//foreign_cc:defs.bzl", "configure_make")

filegroup(
    name = "liburing_srcs",
    srcs = glob(["**"]),
)

configure_make(
    name = "liburing",
    args = [
        # "V=1",
        "-j $(nproc)",
    ],
    configure_in_place = True,
    configure_options = [
        "--cc=$$CC",
        "--cxx=$$CXX",
    ],
    env = {
        "CC": "clang",
        "CXX": "clang++",
    },
    lib_source = ":liburing_srcs",
    visibility = ["//visibility:public"],
)

doesn't give completions as the entries are not there in compile_commands.json
May be I'll open another issue(and sorry for hijacking the thread) - but is there any plan to support this?

My knowledge of bazel right now is more or less shallow as we just use cmake at our workplace and so my experience is mostly limited to my own personal projects.

[davidzchen]

I created a minimal repro case here: https://github.com/davidzchen/bazel-ycm-test

When foo/file.cc is opened in Vim, YCM highlights many lines as errors as shown below:

Here is my :YcmDebugInfo:

Printing YouCompleteMe debug information...
-- Resolve completions: Up front
-- Client logfile: /var/folders/d3/yj9wzlg96ql05cjzm329d76r0000gn/T/ycm_s03l_h6b.log
-- Server Python interpreter: /opt/homebrew/opt/[email protected]/bin/python3.12
-- Server Python version: 3.12.2
-- Server has Clang support compiled in: True
-- Clang version: clang version 17.0.1 (https://github.com/ycm-core/llvm f0e018b8aadf1be25e3af0daa7a87d505de67957)
-- No extra configuration file found
-- C-family completer debug information:
--   Clangd running
--   Clangd process ID: 70415
--   Clangd executable: ['/Users/dzc/Projects/davidzchen/dotfiles/vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clangd/output/bin/clangd', '-header-insertion-decorators=0', '-resource-dir=/Users/dzc/Projects/davidzchen/dotfiles/vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/17.0.1', '-limit-results=500']
--   Clangd logfiles:
--     /var/folders/d3/yj9wzlg96ql05cjzm329d76r0000gn/T/clangd_stderrhpnr9y6h.log
--   Clangd Server State: Initialized
--   Clangd Project Directory: /Users/dzc/Projects/davidzchen/bazel-ycm-test/foo
--   Clangd Open Workspaces: {'/Users/dzc/Projects/davidzchen/bazel-ycm-test/foo'}
--   Clangd Settings: {}
--   Clangd Compilation Command: False
-- Server running at: http://127.0.0.1:63368
-- Server process ID: 70413
-- Server logfiles:
--   /var/folders/d3/yj9wzlg96ql05cjzm329d76r0000gn/T/ycmd_63368_stdout_xebv3sf1.log
--   /var/folders/d3/yj9wzlg96ql05cjzm329d76r0000gn/T/ycmd_63368_stderr_p1l0fz26.log
-- Semantic highlighting supported: True
-- Virtual text supported: True
-- Popup windows supported: True
Press ENTER or type command to continue

Many of the errors seem to be around Abseil, though the code compiles without any issue.

[davidzchen]

FYI I did use @stonebrakert6's script to generate the compile_commands.json, but I am still getting these errors.

[davidzchen]

Is there anything I can provide to help debug this further?

[stonebrakert6]

hey @davidzchen
This is probably NOT the answer you're looking for but I am still posting it anyway.

Disclaimer

I am a newbie at bazel. So can't really explain why toolchain based solution is not working.
My dev environment is Linux and the issues I was facing with your project were much different(and way more). Don't want to mention them to avoid noise.

What I did

1. removed llvm toolchain from WORKSPACE and some other cosmetic changes.
2. I have created a PR so that you can see the diff

I have verified all this on MBP(macbook pro) by installing llvm via brew install llvm
So essentially the TLDR is
1. ./setup_clang.sh /usr/local/opt/llvm i.e path to llvm install path(top-level)
2. ./compile_commands.sh

All this has been inspired from how Envoy is built(for your reference). Do let me know if it works for you(I am assuming you are on MBP).

[davidzchen]

Thanks you very much, @stonebrakert6! I will give this a try and report back.

Yes, I am on MBP (M1 Max running macOS Sonoma 14.4.1).

[davidzchen]

Sorry for the delay.

@stonebrakert6 - I tried using the fix in another small repository that I created: https://github.com/davidzchen/primrose

However, when I try to run ./compile_commands.sh, I get the following linker error:

$ ./compile_commands.sh
WARNING: Build option --linkopt has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
INFO: Analyzed target //:refresh_compile_commands (0 packages loaded, 3111 targets configured).
ERROR: /private/var/tmp/_bazel_dzc/bcb1f62714fbca684e48483afe47c7c9/external/hedron_compile_commands~/BUILD:30:10: Linking external/hedron_compile_commands~/print_args failed: (Exit 1): cc_wrapper.sh failed: error executing CppLink command (from target @@hedron_compile_commands~//:print_args) external/bazel_tools~cc_configure_extension~local_config_cc/cc_wrapper.sh @bazel-out/darwin_arm64-fastbuild/bin/external/hedron_compile_commands~/print_args-2.params

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Homebrew clang version 19.1.1
Target: arm64-apple-darwin24.0.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.1/bin
clang-19: error: invalid linker name in argument '-fuse-ld=lld'
Target //:refresh_compile_commands failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.251s, Critical Path: 0.05s
INFO: 3 processes: 3 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

This seems to be from this line in the clang.bazelrc:

# Use llvm linker
build:clang --linkopt='-fuse-ld=lld'

I am now on macOS Sequoia 15.0.1. Do you know if anything has changed with the toolchain that makes this flag invalid?

[stonebrakert6]

hi @davidzchen
Can you paste output with --verbose_failures?

[davidzchen]

@stonebrakert6 Here is the output:

$ bazel run --config=clang --verbose_failures //:refresh_compile_commands
INFO: Analyzed target //:refresh_compile_commands (0 packages loaded, 0 targets configured).
ERROR: /private/var/tmp/_bazel_dzc/bcb1f62714fbca684e48483afe47c7c9/external/hedron_compile_commands~/BUILD:30:10: Linking external/hedron_compile_commands~/print_args failed: (Exit 1): cc_wrapper.sh failed: error executing CppLink command (from target @@hedron_compile_commands~//:print_args)
  (cd /private/var/tmp/_bazel_dzc/bcb1f62714fbca684e48483afe47c7c9/sandbox/darwin-sandbox/7/execroot/_main && \
  exec env - \
    CC=clang \
    CXX=clang++ \
    PATH=/opt/homebrew/Cellar/llvm/19.1.1/bin:/opt/homebrew/opt/ruby/bin:/opt/homebrew/opt/llvm/bin:/opt/local/bin:/opt/local/sbin:/Users/dzc/.bin:/Users/dzc/.local/bin:/usr/local/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Library/TeX/texbin:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/opt/homebrew/opt/ruby/bin:/opt/homebrew/opt/llvm/bin:/Applications/iTerm.app/Contents/Resources/utilities:/Users/dzc/go/bin:/opt/homebrew/opt/go/libexec/bin:/Library/Frameworks/Mono.framework/Commands:/Users/dzc/.antigen/bundles/robbyrussell/oh-my-zsh/lib:/Users/dzc/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/history-substring-search:/Users/dzc/.antigen/bundles/zsh-users/zsh-syntax-highlighting:/Users/dzc/.antigen/bundles/zsh-users/zsh-completions:/Users/dzc/.antigen/bundles/zsh-users/zsh-autosuggestions \
    PWD=/proc/self/cwd \
    ZERO_AR_DATE=1 \
  external/bazel_tools~cc_configure_extension~local_config_cc/cc_wrapper.sh @bazel-out/darwin_arm64-fastbuild/bin/external/hedron_compile_commands~/print_args-2.params)
# Configuration: 2839a736d42ec9044aa126828d6c8ac90a71c888639482323f767f67e2d0c90f
# Execution platform: @@platforms//host:host

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Homebrew clang version 19.1.1
Target: arm64-apple-darwin24.0.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.1/bin
clang-19: error: invalid linker name in argument '-fuse-ld=lld'
Target //:refresh_compile_commands failed to build
INFO: Elapsed time: 0.146s, Critical Path: 0.04s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

Thank you for taking a look.