Feature Request: Generate compile_commands.json from Build Log (Post-Processing Mode)

[kchevali]

Problem Statement

Bear is an excellent tool for generating this file by wrapping the build command. However, in our specific production build environment, we have a strict policy against introducing any wrappers around our core build tools (e.g., gmake) for our nightly builds. These are considered "production builds" and must remain pristine and unaltered.

Current bear Usage Limitation

While bear's wrapper functionality is powerful, it prevents us from integrating it directly into our existing build pipelines due to the aforementioned policy. This means developers currently have to manually generate compile_commands.json (if possible) or work with less precise IntelliSense, which impacts productivity.

Desired Feature: Post-Processing Mode

We are seeking a mode or functionality within bear (or a related tool/workflow) that would allow us to generate compile_commands.json by parsing an existing build log rather than acting as a real-time wrapper around the build command itself.

Question

Does bear currently offer any functionality that could achieve this "post-processing" of a build log to generate compile_commands.json? If not, would this be a feature the bear development team would consider adding or providing guidance on how to implement?

We believe this would be a valuable addition for organizations with similar build constraints.

Thank you for your time and consideration.

[rizsotto]

Hey @kchevali , thanks for raising this issue.

That is an interesting use case! I don't think this would be hard to achieve... It needs a build log parser to produce a sequence of the executed commands, which could be the input for the semantic analysis step. (The semantic analysis separated at the command line level, Bear can be invoke to do only this step.)

How to make this happen?

A few things I would mention... I am in a middle of rewrite to rust. :) The branch is called 4.0-rc... the good news is, this work would not require to integrate into Bear codebase and can be written in any language... and even the existing 3.x versions have this functionality exposed.

If you can write a program that produces a "execution events file" Bear could process that and produce the compilation database. The execution events file is an internal format of Bear. It is a JSONL file, entries looks like this (did break into multiple lines for readability).

{
  "pid": 11782,
  "execution": {
    "executable": "/usr/bin/clang",
    "arguments": ["clang", "-c", "main.c"],
    "working_dir": "/home/user",
    "environment": {
      "PATH": "/usr/bin",
      "HOME": "/home/user"
    }
  }
}

The current master branch (version 3.x) has similar input format... The processing logic is mainly focus on the execution content of the payload. The other attributes can be mocked, and won't influence the output.

The reason why I would not move the build log parser into the Bear repo, because I think the parsing logic will be build tool and build specific. I might be wrong, but my understanding is: even if you are using the same build tool (eg: make) the build output varies (can be silent, can be verbose, can be recursive, can change the current working directory, etc...) depending on how the build file is written... Would generate a lot of change requests to Bear, which I would like to avoid. :)