[SHT_LLVM_FUNC_MAP][CodeGen]Introduce function address map section and emit dynamic instruction count(CodeGen part) #124334
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Created using spr 1.3.6-beta.1
|
@llvm/pr-subscribers-llvm-binary-utilities @llvm/pr-subscribers-mc Author: Lei Wang (wlei-llvm) ChangesTest Plan: llvm/test/CodeGen/X86/function-address-map-function-sections.ll Full diff: https://github.com/llvm/llvm-project/pull/124334.diff 11 Files Affected:
diff --git a/llvm/docs/Extensions.rst b/llvm/docs/Extensions.rst
index ea267842cdc353..d94e35eeefa6ad 100644
--- a/llvm/docs/Extensions.rst
+++ b/llvm/docs/Extensions.rst
@@ -535,6 +535,30 @@ Example of BBAddrMap with PGO data:
.uleb128 1000 # BB_3 basic block frequency (only when enabled)
.uleb128 0 # BB_3 successors count (only enabled with branch probabilities)
+``SHT_LLVM_FUNC_MAP`` Section (function address map)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+This section stores the mapping from the binary address of function to its
+related metadata features. It is used to emit function-level analysis data and
+can be enabled through ``--func-map=<feature>`` option.
+
+Three fields are stored at the beginning: a version number byte for backward
+compatibility, a feature byte where each bit represents a specific feature, and
+the function's entry address. The encodings for each enabled feature come after
+these fields. The currently supported feature is:
+
+#. Dynamic Instruction Count - Total PGO counts for all instructions within the function.
+
+Example:
+
+.. code-block:: gas
+
+ .section ".llvm_func_map","",@llvm_func_map
+ .byte 1 # version number
+ .byte 1 # feature
+ .quad .Lfunc_begin1 # function address
+ .uleb128 333 # dynamic instruction count
+
+
``SHT_LLVM_OFFLOADING`` Section (offloading data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This section stores the binary data used to perform offloading device linking
@@ -725,4 +749,3 @@ follows:
add x16, x16, :lo12:__chkstk
blr x16
sub sp, sp, x15, lsl #4
-
diff --git a/llvm/include/llvm/CodeGen/AsmPrinter.h b/llvm/include/llvm/CodeGen/AsmPrinter.h
index 5291369b3b9f1d..5fe35c283cceda 100644
--- a/llvm/include/llvm/CodeGen/AsmPrinter.h
+++ b/llvm/include/llvm/CodeGen/AsmPrinter.h
@@ -414,6 +414,8 @@ class AsmPrinter : public MachineFunctionPass {
void emitBBAddrMapSection(const MachineFunction &MF);
+ void emitFuncMapSection(const MachineFunction &MF);
+
void emitKCFITrapEntry(const MachineFunction &MF, const MCSymbol *Symbol);
virtual void emitKCFITypeId(const MachineFunction &MF);
diff --git a/llvm/include/llvm/MC/MCContext.h b/llvm/include/llvm/MC/MCContext.h
index 57ba40f7ac26fc..6fc9eaafeb09e3 100644
--- a/llvm/include/llvm/MC/MCContext.h
+++ b/llvm/include/llvm/MC/MCContext.h
@@ -177,6 +177,9 @@ class MCContext {
/// LLVM_BB_ADDR_MAP version to emit.
uint8_t BBAddrMapVersion = 2;
+ /// LLVM_FUNC_MAP version to emit.
+ uint8_t FuncMapVersion = 1;
+
/// The file name of the log file from the environment variable
/// AS_SECURE_LOG_FILE. Which must be set before the .secure_log_unique
/// directive is used or it is an error.
@@ -656,6 +659,8 @@ class MCContext {
uint8_t getBBAddrMapVersion() const { return BBAddrMapVersion; }
+ uint8_t getFuncMapVersion() const { return FuncMapVersion; }
+
/// @}
/// \name Dwarf Management
diff --git a/llvm/include/llvm/MC/MCObjectFileInfo.h b/llvm/include/llvm/MC/MCObjectFileInfo.h
index fb575fe721015c..e344d4772e3fec 100644
--- a/llvm/include/llvm/MC/MCObjectFileInfo.h
+++ b/llvm/include/llvm/MC/MCObjectFileInfo.h
@@ -364,6 +364,8 @@ class MCObjectFileInfo {
MCSection *getBBAddrMapSection(const MCSection &TextSec) const;
+ MCSection *getFuncMapSection(const MCSection &TextSec) const;
+
MCSection *getKCFITrapSection(const MCSection &TextSec) const;
MCSection *getPseudoProbeSection(const MCSection &TextSec) const;
diff --git a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
index b2a4721f37b268..a00db04ef654c2 100644
--- a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
@@ -147,6 +147,11 @@ enum class PGOMapFeaturesEnum {
BrProb,
All,
};
+
+enum class FuncMapFeaturesEnum {
+ DynamicInstCount,
+};
+
static cl::bits<PGOMapFeaturesEnum> PgoAnalysisMapFeatures(
"pgo-analysis-map", cl::Hidden, cl::CommaSeparated,
cl::values(
@@ -161,6 +166,13 @@ static cl::bits<PGOMapFeaturesEnum> PgoAnalysisMapFeatures(
"Enable extended information within the SHT_LLVM_BB_ADDR_MAP that is "
"extracted from PGO related analysis."));
+static cl::bits<FuncMapFeaturesEnum> FuncMapFeatures(
+ "func-map", cl::Hidden, cl::CommaSeparated,
+ cl::values(clEnumValN(FuncMapFeaturesEnum::DynamicInstCount,
+ "dyn-inst-count", "Dynamic instruction count")),
+ cl::desc(
+ "Emit features of function address map in SHT_LLVM_FUNC_MAP section"));
+
static cl::opt<bool> BBAddrMapSkipEmitBBEntries(
"basic-block-address-map-skip-bb-entries",
cl::desc("Skip emitting basic block entries in the SHT_LLVM_BB_ADDR_MAP "
@@ -1424,6 +1436,12 @@ getBBAddrMapFeature(const MachineFunction &MF, int NumMBBSectionRanges) {
static_cast<bool>(BBAddrMapSkipEmitBBEntries)};
}
+static llvm::object::FuncMap::Features getFuncMapFeature() {
+ return {FuncMapFeatures.isSet(FuncMapFeaturesEnum::DynamicInstCount)};
+}
+
+static bool isAnyFuncMapFeature() { return FuncMapFeatures.getBits() != 0; }
+
void AsmPrinter::emitBBAddrMapSection(const MachineFunction &MF) {
MCSection *BBAddrMapSection =
getObjFileLowering().getBBAddrMapSection(*MF.getSection());
@@ -1548,6 +1566,42 @@ void AsmPrinter::emitBBAddrMapSection(const MachineFunction &MF) {
OutStreamer->popSection();
}
+void AsmPrinter::emitFuncMapSection(const MachineFunction &MF) {
+ if (!isAnyFuncMapFeature())
+ return;
+
+ MCSection *FuncMapSection =
+ getObjFileLowering().getFuncMapSection(*MF.getSection());
+ assert(FuncMapSection && ".llvm_func_map section is not initialized.");
+ const MCSymbol *FunctionSymbol = getFunctionBegin();
+ OutStreamer->pushSection();
+ OutStreamer->switchSection(FuncMapSection);
+ OutStreamer->AddComment("version");
+ uint8_t FuncMapVersion = OutStreamer->getContext().getFuncMapVersion();
+ OutStreamer->emitInt8(FuncMapVersion);
+ OutStreamer->AddComment("feature");
+ auto Features = getFuncMapFeature();
+ OutStreamer->emitInt8(Features.encode());
+
+ OutStreamer->AddComment("function address");
+ OutStreamer->emitSymbolValue(FunctionSymbol, getPointerSize());
+ if (Features.DynamicInstCount) {
+ const MachineBlockFrequencyInfo *MBFI =
+ &getAnalysis<LazyMachineBlockFrequencyInfoPass>().getBFI();
+ uint64_t DynInstCount = 0;
+ for (const MachineBasicBlock &MBB : MF) {
+ for (const MachineInstr &MI : MBB) {
+ if (MI.isDebugValue() || MI.isPseudoProbe())
+ continue;
+ DynInstCount += MBFI->getBlockProfileCount(&MBB).value_or(0);
+ }
+ }
+ OutStreamer->AddComment("dynamic instruction count");
+ OutStreamer->emitULEB128IntValue(DynInstCount);
+ }
+ OutStreamer->popSection();
+}
+
void AsmPrinter::emitKCFITrapEntry(const MachineFunction &MF,
const MCSymbol *Symbol) {
MCSection *Section =
@@ -2119,6 +2173,7 @@ void AsmPrinter::emitFunctionBody() {
MF->getContext().reportWarning(
SMLoc(), "pgo-analysis-map is enabled for function " + MF->getName() +
" but it does not have labels");
+ emitFuncMapSection(*MF);
}
// Emit sections containing instruction and function PCs.
@@ -2749,7 +2804,7 @@ void AsmPrinter::SetupMachineFunction(MachineFunction &MF) {
F.hasFnAttribute("xray-instruction-threshold") ||
needFuncLabels(MF, *this) || NeedsLocalForSize ||
MF.getTarget().Options.EmitStackSizeSection ||
- MF.getTarget().Options.BBAddrMap) {
+ MF.getTarget().Options.BBAddrMap || isAnyFuncMapFeature()) {
CurrentFnBegin = createTempSymbol("func_begin");
if (NeedsLocalForSize)
CurrentFnSymForSize = CurrentFnBegin;
diff --git a/llvm/lib/MC/MCObjectFileInfo.cpp b/llvm/lib/MC/MCObjectFileInfo.cpp
index 150e38a94db6a6..02cc207b74e40f 100644
--- a/llvm/lib/MC/MCObjectFileInfo.cpp
+++ b/llvm/lib/MC/MCObjectFileInfo.cpp
@@ -1120,6 +1120,23 @@ MCObjectFileInfo::getBBAddrMapSection(const MCSection &TextSec) const {
cast<MCSymbolELF>(TextSec.getBeginSymbol()));
}
+MCSection *MCObjectFileInfo::getFuncMapSection(const MCSection &TextSec) const {
+ if (Ctx->getObjectFileType() != MCContext::IsELF)
+ return nullptr;
+
+ const MCSectionELF &ElfSec = static_cast<const MCSectionELF &>(TextSec);
+ unsigned Flags = ELF::SHF_LINK_ORDER;
+ StringRef GroupName;
+ if (const MCSymbol *Group = ElfSec.getGroup()) {
+ GroupName = Group->getName();
+ Flags |= ELF::SHF_GROUP;
+ }
+
+ return Ctx->getELFSection(".llvm_func_map", ELF::SHT_LLVM_FUNC_MAP, Flags, 0,
+ GroupName, true, ElfSec.getUniqueID(),
+ cast<MCSymbolELF>(TextSec.getBeginSymbol()));
+}
+
MCSection *
MCObjectFileInfo::getKCFITrapSection(const MCSection &TextSec) const {
if (Ctx->getObjectFileType() != MCContext::IsELF)
diff --git a/llvm/lib/MC/MCParser/ELFAsmParser.cpp b/llvm/lib/MC/MCParser/ELFAsmParser.cpp
index b58210b3c268e9..c9df8a3a8e8e41 100644
--- a/llvm/lib/MC/MCParser/ELFAsmParser.cpp
+++ b/llvm/lib/MC/MCParser/ELFAsmParser.cpp
@@ -679,6 +679,8 @@ bool ELFAsmParser::parseSectionArguments(bool IsPush, SMLoc loc) {
Type = ELF::SHT_LLVM_LTO;
else if (TypeName == "llvm_jt_sizes")
Type = ELF::SHT_LLVM_JT_SIZES;
+ else if (TypeName == "llvm_func_map")
+ Type = ELF::SHT_LLVM_FUNC_MAP;
else if (TypeName.getAsInteger(0, Type))
return TokError("unknown section type");
}
diff --git a/llvm/lib/MC/MCSectionELF.cpp b/llvm/lib/MC/MCSectionELF.cpp
index 25e62b70b5e2a0..fa1a5eb1071934 100644
--- a/llvm/lib/MC/MCSectionELF.cpp
+++ b/llvm/lib/MC/MCSectionELF.cpp
@@ -174,6 +174,8 @@ void MCSectionELF::printSwitchToSection(const MCAsmInfo &MAI, const Triple &T,
OS << "llvm_lto";
else if (Type == ELF::SHT_LLVM_JT_SIZES)
OS << "llvm_jt_sizes";
+ else if (Type == ELF::SHT_LLVM_FUNC_MAP)
+ OS << "llvm_func_map";
else
OS << "0x" << Twine::utohexstr(Type);
diff --git a/llvm/test/CodeGen/X86/function-address-map-dyn-inst-count.ll b/llvm/test/CodeGen/X86/function-address-map-dyn-inst-count.ll
new file mode 100644
index 00000000000000..8367130071a4fc
--- /dev/null
+++ b/llvm/test/CodeGen/X86/function-address-map-dyn-inst-count.ll
@@ -0,0 +1,110 @@
+; RUN: llc < %s -mtriple=x86_64 -function-sections -func-map=dyn-inst-count | FileCheck %s
+
+
+;; Check we add SHF_LINK_ORDER for .llvm_func_map and link it with the corresponding .text sections.
+; CHECK: .section .text.foo,"ax",@progbits
+; CHECK-LABEL: foo:
+; CHECK-NEXT: [[FOO_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text.foo{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[FOO_BEGIN]] # function address
+; CHECK-NEXT: .ascii "\252\001" # dynamic instruction count
+
+
+; CHECK: .section .text.main,"ax",@progbits
+; CHECK-LABEL: main:
+; CHECK-NEXT: [[MAIN_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text.main{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[MAIN_BEGIN]] # function address
+; CHECK-NEXT: .ascii "\265\003" # dynamic instruction count
+
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo(i32 %x) !dbg !4 !prof !6 {
+entry:
+ #dbg_value(i32 %x, !7, !DIExpression(), !9)
+ call void @llvm.pseudoprobe(i64 6699318081062747564, i64 1, i32 0, i64 -1)
+ %rem = mul i32 %x, 5
+ %tobool.not = icmp eq i32 %rem, 0, !dbg !10
+ br i1 %tobool.not, label %if.end, label %if.then, !prof !12
+
+if.then: ; preds = %entry
+ %inc = add i32 0, 0
+ call void @llvm.pseudoprobe(i64 6699318081062747564, i64 2, i32 0, i64 -1)
+ #dbg_value(i32 %inc, !7, !DIExpression(), !9)
+ br label %if.end
+
+if.end: ; preds = %if.then, %entry
+ %x.addr.0 = phi i32 [ 0, %if.then ], [ 1, %entry ]
+ #dbg_value(i32 %x.addr.0, !7, !DIExpression(), !9)
+ ret i32 %x.addr.0
+}
+
+define i32 @main() #0 !dbg !13 !prof !15 {
+entry:
+ #dbg_value(i32 0, !16, !DIExpression(), !17)
+ br label %while.cond
+
+while.cond: ; preds = %if.then, %if.else, %entry
+ %i.0 = phi i32 [ 0, %entry ], [ %inc, %if.else ], [ %inc, %if.then ]
+ #dbg_value(i32 %i.0, !16, !DIExpression(), !17)
+ %inc = add i32 %i.0, 1
+ #dbg_value(i32 %inc, !16, !DIExpression(), !17)
+ %cmp = icmp ult i32 %i.0, 1600000
+ br i1 %cmp, label %while.body, label %while.end, !prof !18
+
+while.body: ; preds = %while.cond
+ %rem = urem i32 %inc, 11
+ %tobool.not = icmp eq i32 %rem, 0
+ br i1 %tobool.not, label %if.else, label %if.then, !prof !19
+
+if.then: ; preds = %while.body
+ %call = call i32 @foo(i32 0), !dbg !20
+ %0 = load volatile i32, ptr null, align 4
+ br label %while.cond
+
+if.else: ; preds = %while.body
+ store i32 0, ptr null, align 4
+ br label %while.cond
+
+while.end: ; preds = %while.cond
+ ret i32 0
+
+; uselistorder directives
+ uselistorder label %while.cond, { 1, 0, 2 }
+ uselistorder i32 %inc, { 2, 1, 0 }
+}
+
+attributes #0 = { "target-cpu"="x86-64" }
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C11, file: !1, producer: "clang version 20.0.0", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, globals: !2, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "test.c", directory: "/home", checksumkind: CSK_MD5, checksum: "920887ee2258042655d8340f78e732e9")
+!2 = !{}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 3, type: !5, scopeLine: 3, flags: DIFlagPrototyped | DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
+!5 = distinct !DISubroutineType(types: !2)
+!6 = !{!"function_entry_count", i64 20}
+!7 = !DILocalVariable(name: "x", arg: 1, scope: !4, file: !1, line: 3, type: !8)
+!8 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!9 = !DILocation(line: 0, scope: !4)
+!10 = !DILocation(line: 4, column: 9, scope: !11)
+!11 = distinct !DILexicalBlock(scope: !4, file: !1, line: 4, column: 7)
+!12 = !{!"branch_weights", i32 15, i32 5}
+!13 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line: 9, type: !14, scopeLine: 9, flags: DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
+!14 = !DISubroutineType(types: !2)
+!15 = !{!"function_entry_count", i64 1}
+!16 = !DILocalVariable(name: "i", scope: !13, file: !1, line: 10, type: !8)
+!17 = !DILocation(line: 0, scope: !13)
+!18 = !{!"branch_weights", i32 22, i32 1}
+!19 = !{!"branch_weights", i32 2, i32 20}
+!20 = !DILocation(line: 12, column: 22, scope: !21)
+!21 = !DILexicalBlockFile(scope: !22, file: !1, discriminator: 455082031)
+!22 = distinct !DILexicalBlock(scope: !13, file: !1, line: 12, column: 9)
diff --git a/llvm/test/CodeGen/X86/function-address-map-function-sections.ll b/llvm/test/CodeGen/X86/function-address-map-function-sections.ll
new file mode 100644
index 00000000000000..e83970cc86e0b5
--- /dev/null
+++ b/llvm/test/CodeGen/X86/function-address-map-function-sections.ll
@@ -0,0 +1,41 @@
+; RUN: llc < %s -mtriple=x86_64 -function-sections -func-map=dyn-inst-count| FileCheck %s
+
+$_Z4fooTIiET_v = comdat any
+
+define dso_local i32 @_Z3barv() {
+ ret i32 0
+}
+;; Check we add SHF_LINK_ORDER for .llvm_func_map and link it with the corresponding .text sections.
+; CHECK: .section .text._Z3barv,"ax",@progbits
+; CHECK-LABEL: _Z3barv:
+; CHECK-NEXT: [[BAR_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text._Z3barv{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[BAR_BEGIN]] # function address
+
+
+define dso_local i32 @_Z3foov() {
+ %1 = call i32 @_Z4fooTIiET_v()
+ ret i32 %1
+}
+; CHECK: .section .text._Z3foov,"ax",@progbits
+; CHECK-LABEL: _Z3foov:
+; CHECK-NEXT: [[FOO_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text._Z3foov{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[FOO_BEGIN]] # function address
+
+
+define linkonce_odr dso_local i32 @_Z4fooTIiET_v() comdat {
+ ret i32 0
+}
+;; Check we add .llvm_func_map section to a COMDAT group with the corresponding .text section if such a COMDAT exists.
+; CHECK: .section .text._Z4fooTIiET_v,"axG",@progbits,_Z4fooTIiET_v,comdat
+; CHECK-LABEL: _Z4fooTIiET_v:
+; CHECK-NEXT: [[FOOCOMDAT_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"oG",@llvm_func_map,.text._Z4fooTIiET_v,_Z4fooTIiET_v,comdat{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[FOOCOMDAT_BEGIN]] # function address
diff --git a/llvm/test/MC/AsmParser/llvm_section_types.s b/llvm/test/MC/AsmParser/llvm_section_types.s
index 147b1499d2b888..f3f3150ac30f1f 100644
--- a/llvm/test/MC/AsmParser/llvm_section_types.s
+++ b/llvm/test/MC/AsmParser/llvm_section_types.s
@@ -17,6 +17,8 @@
.byte 1
.section .section8,"",@llvm_lto
.byte 1
+.section .section9,"",@llvm_func_map
+.byte 1
# CHECK: Name: .section1
# CHECK-NEXT: Type: SHT_LLVM_BB_ADDR_MAP
@@ -34,3 +36,5 @@
# CHECK-NEXT: Type: SHT_LLVM_OFFLOADING
# CHECK: Name: .section8
# CHECK-NEXT: Type: SHT_LLVM_LTO
+# CHECK: Name: .section9
+# CHECK-NEXT: Type: SHT_LLVM_FUNC_MAP
|
|
@llvm/pr-subscribers-backend-x86 Author: Lei Wang (wlei-llvm) ChangesTest Plan: llvm/test/CodeGen/X86/function-address-map-function-sections.ll Full diff: https://github.com/llvm/llvm-project/pull/124334.diff 11 Files Affected:
diff --git a/llvm/docs/Extensions.rst b/llvm/docs/Extensions.rst
index ea267842cdc353..d94e35eeefa6ad 100644
--- a/llvm/docs/Extensions.rst
+++ b/llvm/docs/Extensions.rst
@@ -535,6 +535,30 @@ Example of BBAddrMap with PGO data:
.uleb128 1000 # BB_3 basic block frequency (only when enabled)
.uleb128 0 # BB_3 successors count (only enabled with branch probabilities)
+``SHT_LLVM_FUNC_MAP`` Section (function address map)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+This section stores the mapping from the binary address of function to its
+related metadata features. It is used to emit function-level analysis data and
+can be enabled through ``--func-map=<feature>`` option.
+
+Three fields are stored at the beginning: a version number byte for backward
+compatibility, a feature byte where each bit represents a specific feature, and
+the function's entry address. The encodings for each enabled feature come after
+these fields. The currently supported feature is:
+
+#. Dynamic Instruction Count - Total PGO counts for all instructions within the function.
+
+Example:
+
+.. code-block:: gas
+
+ .section ".llvm_func_map","",@llvm_func_map
+ .byte 1 # version number
+ .byte 1 # feature
+ .quad .Lfunc_begin1 # function address
+ .uleb128 333 # dynamic instruction count
+
+
``SHT_LLVM_OFFLOADING`` Section (offloading data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This section stores the binary data used to perform offloading device linking
@@ -725,4 +749,3 @@ follows:
add x16, x16, :lo12:__chkstk
blr x16
sub sp, sp, x15, lsl #4
-
diff --git a/llvm/include/llvm/CodeGen/AsmPrinter.h b/llvm/include/llvm/CodeGen/AsmPrinter.h
index 5291369b3b9f1d..5fe35c283cceda 100644
--- a/llvm/include/llvm/CodeGen/AsmPrinter.h
+++ b/llvm/include/llvm/CodeGen/AsmPrinter.h
@@ -414,6 +414,8 @@ class AsmPrinter : public MachineFunctionPass {
void emitBBAddrMapSection(const MachineFunction &MF);
+ void emitFuncMapSection(const MachineFunction &MF);
+
void emitKCFITrapEntry(const MachineFunction &MF, const MCSymbol *Symbol);
virtual void emitKCFITypeId(const MachineFunction &MF);
diff --git a/llvm/include/llvm/MC/MCContext.h b/llvm/include/llvm/MC/MCContext.h
index 57ba40f7ac26fc..6fc9eaafeb09e3 100644
--- a/llvm/include/llvm/MC/MCContext.h
+++ b/llvm/include/llvm/MC/MCContext.h
@@ -177,6 +177,9 @@ class MCContext {
/// LLVM_BB_ADDR_MAP version to emit.
uint8_t BBAddrMapVersion = 2;
+ /// LLVM_FUNC_MAP version to emit.
+ uint8_t FuncMapVersion = 1;
+
/// The file name of the log file from the environment variable
/// AS_SECURE_LOG_FILE. Which must be set before the .secure_log_unique
/// directive is used or it is an error.
@@ -656,6 +659,8 @@ class MCContext {
uint8_t getBBAddrMapVersion() const { return BBAddrMapVersion; }
+ uint8_t getFuncMapVersion() const { return FuncMapVersion; }
+
/// @}
/// \name Dwarf Management
diff --git a/llvm/include/llvm/MC/MCObjectFileInfo.h b/llvm/include/llvm/MC/MCObjectFileInfo.h
index fb575fe721015c..e344d4772e3fec 100644
--- a/llvm/include/llvm/MC/MCObjectFileInfo.h
+++ b/llvm/include/llvm/MC/MCObjectFileInfo.h
@@ -364,6 +364,8 @@ class MCObjectFileInfo {
MCSection *getBBAddrMapSection(const MCSection &TextSec) const;
+ MCSection *getFuncMapSection(const MCSection &TextSec) const;
+
MCSection *getKCFITrapSection(const MCSection &TextSec) const;
MCSection *getPseudoProbeSection(const MCSection &TextSec) const;
diff --git a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
index b2a4721f37b268..a00db04ef654c2 100644
--- a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
@@ -147,6 +147,11 @@ enum class PGOMapFeaturesEnum {
BrProb,
All,
};
+
+enum class FuncMapFeaturesEnum {
+ DynamicInstCount,
+};
+
static cl::bits<PGOMapFeaturesEnum> PgoAnalysisMapFeatures(
"pgo-analysis-map", cl::Hidden, cl::CommaSeparated,
cl::values(
@@ -161,6 +166,13 @@ static cl::bits<PGOMapFeaturesEnum> PgoAnalysisMapFeatures(
"Enable extended information within the SHT_LLVM_BB_ADDR_MAP that is "
"extracted from PGO related analysis."));
+static cl::bits<FuncMapFeaturesEnum> FuncMapFeatures(
+ "func-map", cl::Hidden, cl::CommaSeparated,
+ cl::values(clEnumValN(FuncMapFeaturesEnum::DynamicInstCount,
+ "dyn-inst-count", "Dynamic instruction count")),
+ cl::desc(
+ "Emit features of function address map in SHT_LLVM_FUNC_MAP section"));
+
static cl::opt<bool> BBAddrMapSkipEmitBBEntries(
"basic-block-address-map-skip-bb-entries",
cl::desc("Skip emitting basic block entries in the SHT_LLVM_BB_ADDR_MAP "
@@ -1424,6 +1436,12 @@ getBBAddrMapFeature(const MachineFunction &MF, int NumMBBSectionRanges) {
static_cast<bool>(BBAddrMapSkipEmitBBEntries)};
}
+static llvm::object::FuncMap::Features getFuncMapFeature() {
+ return {FuncMapFeatures.isSet(FuncMapFeaturesEnum::DynamicInstCount)};
+}
+
+static bool isAnyFuncMapFeature() { return FuncMapFeatures.getBits() != 0; }
+
void AsmPrinter::emitBBAddrMapSection(const MachineFunction &MF) {
MCSection *BBAddrMapSection =
getObjFileLowering().getBBAddrMapSection(*MF.getSection());
@@ -1548,6 +1566,42 @@ void AsmPrinter::emitBBAddrMapSection(const MachineFunction &MF) {
OutStreamer->popSection();
}
+void AsmPrinter::emitFuncMapSection(const MachineFunction &MF) {
+ if (!isAnyFuncMapFeature())
+ return;
+
+ MCSection *FuncMapSection =
+ getObjFileLowering().getFuncMapSection(*MF.getSection());
+ assert(FuncMapSection && ".llvm_func_map section is not initialized.");
+ const MCSymbol *FunctionSymbol = getFunctionBegin();
+ OutStreamer->pushSection();
+ OutStreamer->switchSection(FuncMapSection);
+ OutStreamer->AddComment("version");
+ uint8_t FuncMapVersion = OutStreamer->getContext().getFuncMapVersion();
+ OutStreamer->emitInt8(FuncMapVersion);
+ OutStreamer->AddComment("feature");
+ auto Features = getFuncMapFeature();
+ OutStreamer->emitInt8(Features.encode());
+
+ OutStreamer->AddComment("function address");
+ OutStreamer->emitSymbolValue(FunctionSymbol, getPointerSize());
+ if (Features.DynamicInstCount) {
+ const MachineBlockFrequencyInfo *MBFI =
+ &getAnalysis<LazyMachineBlockFrequencyInfoPass>().getBFI();
+ uint64_t DynInstCount = 0;
+ for (const MachineBasicBlock &MBB : MF) {
+ for (const MachineInstr &MI : MBB) {
+ if (MI.isDebugValue() || MI.isPseudoProbe())
+ continue;
+ DynInstCount += MBFI->getBlockProfileCount(&MBB).value_or(0);
+ }
+ }
+ OutStreamer->AddComment("dynamic instruction count");
+ OutStreamer->emitULEB128IntValue(DynInstCount);
+ }
+ OutStreamer->popSection();
+}
+
void AsmPrinter::emitKCFITrapEntry(const MachineFunction &MF,
const MCSymbol *Symbol) {
MCSection *Section =
@@ -2119,6 +2173,7 @@ void AsmPrinter::emitFunctionBody() {
MF->getContext().reportWarning(
SMLoc(), "pgo-analysis-map is enabled for function " + MF->getName() +
" but it does not have labels");
+ emitFuncMapSection(*MF);
}
// Emit sections containing instruction and function PCs.
@@ -2749,7 +2804,7 @@ void AsmPrinter::SetupMachineFunction(MachineFunction &MF) {
F.hasFnAttribute("xray-instruction-threshold") ||
needFuncLabels(MF, *this) || NeedsLocalForSize ||
MF.getTarget().Options.EmitStackSizeSection ||
- MF.getTarget().Options.BBAddrMap) {
+ MF.getTarget().Options.BBAddrMap || isAnyFuncMapFeature()) {
CurrentFnBegin = createTempSymbol("func_begin");
if (NeedsLocalForSize)
CurrentFnSymForSize = CurrentFnBegin;
diff --git a/llvm/lib/MC/MCObjectFileInfo.cpp b/llvm/lib/MC/MCObjectFileInfo.cpp
index 150e38a94db6a6..02cc207b74e40f 100644
--- a/llvm/lib/MC/MCObjectFileInfo.cpp
+++ b/llvm/lib/MC/MCObjectFileInfo.cpp
@@ -1120,6 +1120,23 @@ MCObjectFileInfo::getBBAddrMapSection(const MCSection &TextSec) const {
cast<MCSymbolELF>(TextSec.getBeginSymbol()));
}
+MCSection *MCObjectFileInfo::getFuncMapSection(const MCSection &TextSec) const {
+ if (Ctx->getObjectFileType() != MCContext::IsELF)
+ return nullptr;
+
+ const MCSectionELF &ElfSec = static_cast<const MCSectionELF &>(TextSec);
+ unsigned Flags = ELF::SHF_LINK_ORDER;
+ StringRef GroupName;
+ if (const MCSymbol *Group = ElfSec.getGroup()) {
+ GroupName = Group->getName();
+ Flags |= ELF::SHF_GROUP;
+ }
+
+ return Ctx->getELFSection(".llvm_func_map", ELF::SHT_LLVM_FUNC_MAP, Flags, 0,
+ GroupName, true, ElfSec.getUniqueID(),
+ cast<MCSymbolELF>(TextSec.getBeginSymbol()));
+}
+
MCSection *
MCObjectFileInfo::getKCFITrapSection(const MCSection &TextSec) const {
if (Ctx->getObjectFileType() != MCContext::IsELF)
diff --git a/llvm/lib/MC/MCParser/ELFAsmParser.cpp b/llvm/lib/MC/MCParser/ELFAsmParser.cpp
index b58210b3c268e9..c9df8a3a8e8e41 100644
--- a/llvm/lib/MC/MCParser/ELFAsmParser.cpp
+++ b/llvm/lib/MC/MCParser/ELFAsmParser.cpp
@@ -679,6 +679,8 @@ bool ELFAsmParser::parseSectionArguments(bool IsPush, SMLoc loc) {
Type = ELF::SHT_LLVM_LTO;
else if (TypeName == "llvm_jt_sizes")
Type = ELF::SHT_LLVM_JT_SIZES;
+ else if (TypeName == "llvm_func_map")
+ Type = ELF::SHT_LLVM_FUNC_MAP;
else if (TypeName.getAsInteger(0, Type))
return TokError("unknown section type");
}
diff --git a/llvm/lib/MC/MCSectionELF.cpp b/llvm/lib/MC/MCSectionELF.cpp
index 25e62b70b5e2a0..fa1a5eb1071934 100644
--- a/llvm/lib/MC/MCSectionELF.cpp
+++ b/llvm/lib/MC/MCSectionELF.cpp
@@ -174,6 +174,8 @@ void MCSectionELF::printSwitchToSection(const MCAsmInfo &MAI, const Triple &T,
OS << "llvm_lto";
else if (Type == ELF::SHT_LLVM_JT_SIZES)
OS << "llvm_jt_sizes";
+ else if (Type == ELF::SHT_LLVM_FUNC_MAP)
+ OS << "llvm_func_map";
else
OS << "0x" << Twine::utohexstr(Type);
diff --git a/llvm/test/CodeGen/X86/function-address-map-dyn-inst-count.ll b/llvm/test/CodeGen/X86/function-address-map-dyn-inst-count.ll
new file mode 100644
index 00000000000000..8367130071a4fc
--- /dev/null
+++ b/llvm/test/CodeGen/X86/function-address-map-dyn-inst-count.ll
@@ -0,0 +1,110 @@
+; RUN: llc < %s -mtriple=x86_64 -function-sections -func-map=dyn-inst-count | FileCheck %s
+
+
+;; Check we add SHF_LINK_ORDER for .llvm_func_map and link it with the corresponding .text sections.
+; CHECK: .section .text.foo,"ax",@progbits
+; CHECK-LABEL: foo:
+; CHECK-NEXT: [[FOO_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text.foo{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[FOO_BEGIN]] # function address
+; CHECK-NEXT: .ascii "\252\001" # dynamic instruction count
+
+
+; CHECK: .section .text.main,"ax",@progbits
+; CHECK-LABEL: main:
+; CHECK-NEXT: [[MAIN_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text.main{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[MAIN_BEGIN]] # function address
+; CHECK-NEXT: .ascii "\265\003" # dynamic instruction count
+
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo(i32 %x) !dbg !4 !prof !6 {
+entry:
+ #dbg_value(i32 %x, !7, !DIExpression(), !9)
+ call void @llvm.pseudoprobe(i64 6699318081062747564, i64 1, i32 0, i64 -1)
+ %rem = mul i32 %x, 5
+ %tobool.not = icmp eq i32 %rem, 0, !dbg !10
+ br i1 %tobool.not, label %if.end, label %if.then, !prof !12
+
+if.then: ; preds = %entry
+ %inc = add i32 0, 0
+ call void @llvm.pseudoprobe(i64 6699318081062747564, i64 2, i32 0, i64 -1)
+ #dbg_value(i32 %inc, !7, !DIExpression(), !9)
+ br label %if.end
+
+if.end: ; preds = %if.then, %entry
+ %x.addr.0 = phi i32 [ 0, %if.then ], [ 1, %entry ]
+ #dbg_value(i32 %x.addr.0, !7, !DIExpression(), !9)
+ ret i32 %x.addr.0
+}
+
+define i32 @main() #0 !dbg !13 !prof !15 {
+entry:
+ #dbg_value(i32 0, !16, !DIExpression(), !17)
+ br label %while.cond
+
+while.cond: ; preds = %if.then, %if.else, %entry
+ %i.0 = phi i32 [ 0, %entry ], [ %inc, %if.else ], [ %inc, %if.then ]
+ #dbg_value(i32 %i.0, !16, !DIExpression(), !17)
+ %inc = add i32 %i.0, 1
+ #dbg_value(i32 %inc, !16, !DIExpression(), !17)
+ %cmp = icmp ult i32 %i.0, 1600000
+ br i1 %cmp, label %while.body, label %while.end, !prof !18
+
+while.body: ; preds = %while.cond
+ %rem = urem i32 %inc, 11
+ %tobool.not = icmp eq i32 %rem, 0
+ br i1 %tobool.not, label %if.else, label %if.then, !prof !19
+
+if.then: ; preds = %while.body
+ %call = call i32 @foo(i32 0), !dbg !20
+ %0 = load volatile i32, ptr null, align 4
+ br label %while.cond
+
+if.else: ; preds = %while.body
+ store i32 0, ptr null, align 4
+ br label %while.cond
+
+while.end: ; preds = %while.cond
+ ret i32 0
+
+; uselistorder directives
+ uselistorder label %while.cond, { 1, 0, 2 }
+ uselistorder i32 %inc, { 2, 1, 0 }
+}
+
+attributes #0 = { "target-cpu"="x86-64" }
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C11, file: !1, producer: "clang version 20.0.0", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, globals: !2, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "test.c", directory: "/home", checksumkind: CSK_MD5, checksum: "920887ee2258042655d8340f78e732e9")
+!2 = !{}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 3, type: !5, scopeLine: 3, flags: DIFlagPrototyped | DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
+!5 = distinct !DISubroutineType(types: !2)
+!6 = !{!"function_entry_count", i64 20}
+!7 = !DILocalVariable(name: "x", arg: 1, scope: !4, file: !1, line: 3, type: !8)
+!8 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!9 = !DILocation(line: 0, scope: !4)
+!10 = !DILocation(line: 4, column: 9, scope: !11)
+!11 = distinct !DILexicalBlock(scope: !4, file: !1, line: 4, column: 7)
+!12 = !{!"branch_weights", i32 15, i32 5}
+!13 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line: 9, type: !14, scopeLine: 9, flags: DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
+!14 = !DISubroutineType(types: !2)
+!15 = !{!"function_entry_count", i64 1}
+!16 = !DILocalVariable(name: "i", scope: !13, file: !1, line: 10, type: !8)
+!17 = !DILocation(line: 0, scope: !13)
+!18 = !{!"branch_weights", i32 22, i32 1}
+!19 = !{!"branch_weights", i32 2, i32 20}
+!20 = !DILocation(line: 12, column: 22, scope: !21)
+!21 = !DILexicalBlockFile(scope: !22, file: !1, discriminator: 455082031)
+!22 = distinct !DILexicalBlock(scope: !13, file: !1, line: 12, column: 9)
diff --git a/llvm/test/CodeGen/X86/function-address-map-function-sections.ll b/llvm/test/CodeGen/X86/function-address-map-function-sections.ll
new file mode 100644
index 00000000000000..e83970cc86e0b5
--- /dev/null
+++ b/llvm/test/CodeGen/X86/function-address-map-function-sections.ll
@@ -0,0 +1,41 @@
+; RUN: llc < %s -mtriple=x86_64 -function-sections -func-map=dyn-inst-count| FileCheck %s
+
+$_Z4fooTIiET_v = comdat any
+
+define dso_local i32 @_Z3barv() {
+ ret i32 0
+}
+;; Check we add SHF_LINK_ORDER for .llvm_func_map and link it with the corresponding .text sections.
+; CHECK: .section .text._Z3barv,"ax",@progbits
+; CHECK-LABEL: _Z3barv:
+; CHECK-NEXT: [[BAR_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text._Z3barv{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[BAR_BEGIN]] # function address
+
+
+define dso_local i32 @_Z3foov() {
+ %1 = call i32 @_Z4fooTIiET_v()
+ ret i32 %1
+}
+; CHECK: .section .text._Z3foov,"ax",@progbits
+; CHECK-LABEL: _Z3foov:
+; CHECK-NEXT: [[FOO_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"o",@llvm_func_map,.text._Z3foov{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[FOO_BEGIN]] # function address
+
+
+define linkonce_odr dso_local i32 @_Z4fooTIiET_v() comdat {
+ ret i32 0
+}
+;; Check we add .llvm_func_map section to a COMDAT group with the corresponding .text section if such a COMDAT exists.
+; CHECK: .section .text._Z4fooTIiET_v,"axG",@progbits,_Z4fooTIiET_v,comdat
+; CHECK-LABEL: _Z4fooTIiET_v:
+; CHECK-NEXT: [[FOOCOMDAT_BEGIN:.Lfunc_begin[0-9]+]]:
+; CHECK: .section .llvm_func_map,"oG",@llvm_func_map,.text._Z4fooTIiET_v,_Z4fooTIiET_v,comdat{{$}}
+; CHECK-NEXT: .byte 1 # version
+; CHECK-NEXT: .byte 1 # feature
+; CHECK-NEXT: .quad [[FOOCOMDAT_BEGIN]] # function address
diff --git a/llvm/test/MC/AsmParser/llvm_section_types.s b/llvm/test/MC/AsmParser/llvm_section_types.s
index 147b1499d2b888..f3f3150ac30f1f 100644
--- a/llvm/test/MC/AsmParser/llvm_section_types.s
+++ b/llvm/test/MC/AsmParser/llvm_section_types.s
@@ -17,6 +17,8 @@
.byte 1
.section .section8,"",@llvm_lto
.byte 1
+.section .section9,"",@llvm_func_map
+.byte 1
# CHECK: Name: .section1
# CHECK-NEXT: Type: SHT_LLVM_BB_ADDR_MAP
@@ -34,3 +36,5 @@
# CHECK-NEXT: Type: SHT_LLVM_OFFLOADING
# CHECK: Name: .section8
# CHECK-NEXT: Type: SHT_LLVM_LTO
+# CHECK: Name: .section9
+# CHECK-NEXT: Type: SHT_LLVM_FUNC_MAP
|
This was referenced Jan 24, 2025
wlei-llvm
requested review from
MatzeB,
rlavaee,
WenleiHe,
boomanaiden154 and
jh7370
Created using spr 1.3.6-beta.1
Created using spr 1.3.6-beta.1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
AsmPrinter part of #124332
PR stack:
#124332 [SHT_LLVM_FUNC_MAP][ObjectYaml]Introduce function address map section and emit dynamic instruction count(ObjectYaml part)
#124333 [SHT_LLVM_FUNC_MAP][llvm-readobj]Introduce function address map section and emit dynamic instruction count(readobj part)
(This one) #124334 [SHT_LLVM_FUNC_MAP][CodeGen]Introduce function address map section and emit dynamic instruction count(CodeGen part)
Test Plan: llvm/test/CodeGen/X86/function-address-map-function-sections.ll