[Bug] LLVM module verification fails with `#dbg_declare must be a pointer or int` on LLVM 15+

[yunho-c]

I am not familiar with compilers and LLMs were used to investigate this issue—please take this with a grain of salt!

Expected behavior

TVM should successfully compile and build Relax models targeting LLVM backends when using LLVM 15+ (including LLVM 21).

Actual behavior

When building a model with fused kernels containing operations like instance_norm, LLVM module verification fails with:

LLVM module verification failed: location of #dbg_declare must be a pointer or int

This occurs specifically with fused functions like:

• fused_add*_instance_norm*_multiply*_leaky_relu*_compute_

Environment

• TVM Version: main branch (commit hash: fc2bdfe)
• LLVM Version: 21.1.6 (likely affects LLVM 15-20 also)
• Operating System: macOS 15.6 (may affect Linux also)
• Target: llvm -mtriple=arm64-apple-macosx
• Python: 3.12

Steps to reproduce

tvm_llvm_dbg_declare_bug_repro.py

import torch
import tvm
from tvm import relax
from tvm.relax.frontend.torch import from_exported_program

# Any model with InstanceNorm will trigger this
class Model(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.norm = torch.nn.InstanceNorm1d(256)
    
    def forward(self, x):
        return self.norm(x)

model = Model().eval()
example_input = torch.randn(1, 256, 100)

# Export and import to TVM
with torch.no_grad():
    exported = torch.export.export(model, (example_input,))

mod = from_exported_program(exported)

# Apply standard optimizations
mod = relax.transform.DecomposeOpsForInference()(mod)
mod = relax.transform.LegalizeOps()(mod)
mod = relax.transform.FuseOps()(mod)
mod = relax.transform.FuseTIR()(mod)

# Build - this crashes with LLVM 15+
target = tvm.target.Target("llvm")
ex = relax.build(mod, target)  # LLVM verification error here

Analysis

The issue seems to be in src/target/llvm/codegen_llvm.cc, function AddDebugInformation (lines 2311-2342).

Problem

AddDebugInformation calls insertDeclare on any llvm::Value*, including floating-point values:

void CodeGenLLVM::AddDebugInformation(llvm::Value* llvm_value, const Var& tir_var, ...) {
  // ...
  // No check if llvm_value is a pointer or integer!
  dbg_info_->di_builder_->insertDeclare(llvm_value, local_var, ...);  // ← fails on floats
}

LLVM 15+ added stricter verification for #dbg_declare intrinsics, requiring that the first operand be a pointer or integer type. When TVM emits debug info for floating-point reduction temporaries (e.g., in instance_norm fused kernels), LLVM's verifier rejects the module.

Call Sites

The function is called from:

1. VisitExpr_(const LetNode* op) - line 1722
2. VisitStmt_(const LetStmtNode* op) - lines 2203, 2210
3. VisitStmt_(const AllocateNode* op) - line 2137

When processing LetNode for float temporaries in reduction operations, AddDebugInformation receives a float llvm::Value* and attempts to attach debug info, which fails verification.

Reference

• https://llvm.org/docs/SourceLevelDebugging.html#dbg-declare

Triage

• llvm
• type: bug

[yunho-c]

I experimented with the following fixes and verified that they work (at least for the repro script):

Option 1. Skipping float/vector values

void CodeGenLLVM::AddDebugInformation(llvm::Value* llvm_value, const Var& tir_var,
                                      llvm::Instruction* insert_before) {
  llvm_value->setName(tir_var->name_hint.c_str());

#if TVM_LLVM_VERSION >= 50
  if (!di_subprogram_) return;

+ #if TVM_LLVM_VERSION >= 150
+   // LLVM 15+ requires dbg_declare to reference pointer or integer types only.
+   // Skip debug info for float/vector types to avoid verification failure.
+   if (!llvm_value->getType()->isPointerTy() && !llvm_value->getType()->isIntegerTy()) {
+       return;  // No debug info for float temporaries
+   }
+ #endif

...

Option 2. Using dbg_value for float/vector values

void CodeGenLLVM::AddDebugInformation(llvm::Value* llvm_value, const Var& tir_var,
                                      llvm::Instruction* insert_before) {
  llvm_value->setName(tir_var->name_hint.c_str());

#if TVM_LLVM_VERSION >= 50
  if (!di_subprogram_) return;

  auto dbg_dtype = GetDebugType(GetType(tir_var));
  // no invalid dtypes
  if (!dbg_dtype) return;
  auto local_var = dbg_info_->di_builder_->createAutoVariable(
      di_subprogram_, std::string(tir_var->name_hint), dbg_info_->file_, 0, dbg_dtype);

  auto* di_loc = llvm::DILocation::get(*llvm_target_->GetContext(), 0, 0, di_subprogram_);

+ #if TVM_LLVM_VERSION >= 150
+   // LLVM 15+ requires dbg_declare to reference pointer or integer types only.
+   // For non-pointer types (floats, vectors), use dbg_value instead to track
+   // the SSA value directly rather than a memory location.
+   if (!llvm_value->getType()->isPointerTy()) {
+     if (insert_before) {
+       // LLVM 20+ changed insertDbgValueIntrinsic to take BasicBlock::iterator
+       // instead of Instruction* for the insertion point.
+ #if TVM_LLVM_VERSION >= 200
+       dbg_info_->di_builder_->insertDbgValueIntrinsic(
+           llvm_value, local_var, dbg_info_->di_builder_->createExpression(), llvm::DebugLoc(di_loc),
+           llvm::BasicBlock::iterator(insert_before));
+ #else
+       dbg_info_->di_builder_->insertDbgValueIntrinsic(llvm_value, local_var,
+                                                       dbg_info_->di_builder_->createExpression(),
+                                                       llvm::DebugLoc(di_loc), insert_before);
+ #endif
+     } else {
+       dbg_info_->di_builder_->insertDbgValueIntrinsic(
+           llvm_value, local_var, dbg_info_->di_builder_->createExpression(), llvm::DebugLoc(di_loc),
+           builder_->GetInsertBlock());
+     }
+     return;
+   }
+ #endif

...

I would be happy to commit either of these and create a PR—yet still would prefer someone with LLVM expertise to have a look over this (and help decide which fix is more appropriate)! Thanks

[MasterJH5574]

I once had a PR that tries to fix this for mac, but it failed the CI and I closed it in the end #18292. Looks like it's the time to try to fix this again.

[tqchen]

seems option2 sounds like right

[MasterJH5574]

Hi @yunho-c, I also tried your patch in option 2 for both metal and cuda backends, and it seems working fine. Do you mind creating a PR with this patch? Let's see how the CI jobs go.