Repository navigation
[AMDGPU] amdgpu-agpr-alloc max field used to limit max VGPRs - #233960
Open
JoshuaGrindstaff wants to merge 3 commits into
Open
JoshuaGrindstaff wants to merge 3 commits into
JoshuaGrindstaff wants to merge 3 commits into
Conversation
JoshuaGrindstaff
force-pushed
the
amdgpu-agpr-alloc-max-codegen
branch
2 times, most recently
from
October 20, 2026 21:27
25d7a9d to
b494815
Compare
JoshuaGrindstaff
marked this pull request as ready for review
October 20, 2026 22:25
|
@llvm/pr-subscribers-backend-amdgpu Author: JoshuaGrindstaff ChangesThis is the first PR of a two PR stack #233968 IssueThe issue is described in the original PR (#218745) but I'll give a short description of it here. On gfx90a+, a kernel’s accum_offset and total vector-register count are each taken as an independent maximum over the whole call graph: the highest architectural VGPR count, aligned up, plus the highest AGPR count. Per function, getMaxNumVectorRegs still keeps VGPR + AGPR inside the occupancy budget from amdgpu-waves-per-eu and amdgpu-flat-work-group-size, but those two maxima can come from different functions. With amdgpu-agpr-alloc splitting the register file, a callee that uses many VGPRs and a sibling that uses many AGPRs can add up past the budget, so the kernel launches with a register split that silently misses the occupancy contract. ChangeThis is the codegen portion of an attributor change in which I hope to use the second field of the amdgpu-agpr-alloc to limit the number of VGPRs a function can assign. So now the behavior of the amdgpu-agpr-alloc attribute now specifies the demand/usage with the min field (similar to before) and the max specifies how much space needs to be reserved so we don't miss the occupancy contract. To reserve that space, we take it out of the Max VGPRs. Patch is 31.35 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/233960.diff 10 Files Affected:
diff --git a/llvm/docs/AMDGPUUsage.rst b/llvm/docs/AMDGPUUsage.rst
index 3de94d5617ac7..7a831a80d33fd 100644
--- a/llvm/docs/AMDGPUUsage.rst
+++ b/llvm/docs/AMDGPUUsage.rst
@@ -2812,6 +2812,10 @@ The AMDGPU backend supports the following LLVM IR attributes.
it is interpreted as the minimum register budget. The maximum will restrict
allocation to use no more than max AGPRs.
+ Max AGPR is also used to limit the maximum number of VGPRs a function can
+ allocate. Max AGPRs specifies how many AGPRs a function needs to make space
+ for by limiting the number of VGPRs it can allocate.
+
The values may be ignored if satisfying it would violate other allocation
constraints.
diff --git a/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp b/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
index 199c594c4083b..030d76d22e1a8 100644
--- a/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
@@ -693,15 +693,27 @@ GCNSubtarget::getMaxNumVectorRegs(const Function &F) const {
MinNumAGPRs = alignTo(MinNumAGPRs, 4);
MinNumAGPRs = std::min(MinNumAGPRs, TotalNumAGPRs);
+ if (MaxNumAGPRs == DefaultNumAGPR.second) {
+ MaxNumAGPRs = MaxVectorRegs / 2;
+ } else {
+ MaxNumAGPRs = alignTo(MaxNumAGPRs, 4);
+ MaxNumAGPRs = std::min(MaxNumAGPRs, TotalNumAGPRs);
+ }
}
-
// Clamp values to be inbounds of our limits, and ensure min <= max.
MaxNumAGPRs = std::min(std::max(MinNumAGPRs, MaxNumAGPRs), MaxVectorRegs);
MinNumAGPRs = std::min({MinNumAGPRs, TotalNumAGPRs, MaxNumAGPRs});
- MaxNumVGPRs = std::min(MaxVectorRegs - MinNumAGPRs, NumArchVGPRs);
- MaxNumAGPRs = std::min(MaxVectorRegs - MaxNumVGPRs, MaxNumAGPRs);
+ // We need to account for the fact that in this situation the AGPRs are not
+ // transferable so there is no reason to limit them
+ if (MaxVectorRegs > TotalNumAGPRs) {
+ MinNumAGPRs = std::max(MinNumAGPRs, MaxVectorRegs - TotalNumAGPRs);
+ }
+
+ MaxNumVGPRs = std::min(MaxVectorRegs - MaxNumAGPRs, NumArchVGPRs);
+ MaxNumAGPRs =
+ std::min({MaxVectorRegs - MaxNumVGPRs, MinNumAGPRs, MaxNumAGPRs});
assert(MaxNumVGPRs + MaxNumAGPRs <= MaxVectorRegs &&
MaxNumAGPRs <= TotalNumAGPRs && MaxNumVGPRs <= NumArchVGPRs &&
diff --git a/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers-assertion-after-ra-failure.ll b/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers-assertion-after-ra-failure.ll
index d35f04c64ac89..018888855b1e2 100644
--- a/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers-assertion-after-ra-failure.ll
+++ b/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers-assertion-after-ra-failure.ll
@@ -17,6 +17,6 @@ define void @no_free_vgprs_at_agpr_to_agpr_copy(float %v0, float %v1) #0 {
declare <16 x float> @llvm.amdgcn.mfma.f32.16x16x1f32(float, float, <16 x float>, i32 immarg, i32 immarg, i32 immarg) #1
declare noundef i32 @llvm.amdgcn.workitem.id.x() #2
-attributes #0 = { "amdgpu-agpr-alloc"="0" "amdgpu-waves-per-eu"="6,6" }
+attributes #0 = { "amdgpu-agpr-alloc"="0,0" "amdgpu-waves-per-eu"="6,6" }
attributes #1 = { convergent nocallback nofree nosync nounwind willreturn memory(none) }
attributes #2 = { nocallback nofree nosync nounwind speculatable willreturn memory(none) }
diff --git a/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers.ll b/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers.ll
index ecd7a2128ccd0..fd2363a1f9279 100644
--- a/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers.ll
+++ b/llvm/test/CodeGen/AMDGPU/agpr-copy-no-free-registers.ll
@@ -1095,6 +1095,6 @@ declare i32 @llvm.amdgcn.workitem.id.x() #2
attributes #0 = { "amdgpu-waves-per-eu"="6,6" nounwind }
attributes #1 = { convergent nounwind readnone willreturn }
attributes #2 = { nounwind readnone willreturn }
-attributes #3 = { "amdgpu-waves-per-eu"="7,7" "amdgpu-agpr-alloc"="0" nounwind }
+attributes #3 = { "amdgpu-waves-per-eu"="7,7" "amdgpu-agpr-alloc"="0,0" nounwind }
attributes #4 = { "amdgpu-waves-per-eu"="6,6" "amdgpu-flat-work-group-size"="1024,1024" nounwind }
-attributes #5 = { "amdgpu-waves-per-eu"="6,6" "amdgpu-agpr-alloc"="0" nounwind }
+attributes #5 = { "amdgpu-waves-per-eu"="6,6" "amdgpu-agpr-alloc"="0,0" nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/amdgpu-num-agpr.ll b/llvm/test/CodeGen/AMDGPU/amdgpu-num-agpr.ll
index bb4f2a68b182a..482f63b7cc4ca 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgpu-num-agpr.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgpu-num-agpr.ll
@@ -38,13 +38,12 @@ define amdgpu_kernel void @min_num_agpr_1_1() #2 {
attributes #2 = { "amdgpu-waves-per-eu"="8,8" "amdgpu-flat-work-group-size"="64,64" "amdgpu-agpr-alloc"="1,1" }
; Check parse of single entry 4, interpreted as the minimum. Total budget is 64.
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_4__amdgpu_no_agpr': desired occupancy was 8, final occupancy is 7
-; WARN: warning: inline asm clobber list contains reserved registers: v60 at line 4
+; WARN: warning: inline asm clobber list contains reserved registers: v32 at line 4
define amdgpu_kernel void @min_num_agpr_4__amdgpu_no_agpr() #3 {
call void asm sideeffect "; clobber $0","~{a0}"(), !srcloc !{i32 4}
call void asm sideeffect "; clobber $0","~{a3}"(), !srcloc !{i32 4}
- call void asm sideeffect "; clobber $0","~{v59}"(), !srcloc !{i32 4}
- call void asm sideeffect "; clobber $0","~{v60}"(), !srcloc !{i32 4}
+ call void asm sideeffect "; clobber $0","~{v31}"(), !srcloc !{i32 4}
+ call void asm sideeffect "; clobber $0","~{v32}"(), !srcloc !{i32 4}
ret void
}
@@ -106,7 +105,8 @@ define amdgpu_kernel void @min_num_agpr_63_64() #7 {
attributes #7 = { "amdgpu-waves-per-eu"="8,8" "amdgpu-flat-work-group-size"="64,64" "amdgpu-agpr-alloc"="63,64" }
-; No-op value.
+; WARN: warning: inline asm clobber list contains reserved registers: v0 at line 9
+; WARN: warning: inline asm clobber list contains reserved registers: v59 at line 9
define amdgpu_kernel void @min_num_agpr_occ8_0_64() #8 {
call void asm sideeffect "; clobber $0","~{v0}"(), !srcloc !{i32 9}
call void asm sideeffect "; clobber $0","~{v59}"(), !srcloc !{i32 9}
@@ -117,42 +117,39 @@ attributes #8 = { "amdgpu-waves-per-eu"="8,8" "amdgpu-flat-work-group-size"="64,
; Register budget 64
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ8_11_59': desired occupancy was 8, final occupancy is 7
; WARN: warning: inline asm clobber list contains reserved registers: a12 at line 10
-; WARN: warning: inline asm clobber list contains reserved registers: v52 at line 10
+; WARN: warning: inline asm clobber list contains reserved registers: v4 at line 10
define amdgpu_kernel void @min_num_agpr_occ8_11_59() #9 {
call void asm sideeffect "; clobber $0","~{a11}"(), !srcloc !{i32 10}
call void asm sideeffect "; clobber $0","~{a12}"(), !srcloc !{i32 10}
- call void asm sideeffect "; clobber $0","~{v51}"(), !srcloc !{i32 10}
- call void asm sideeffect "; clobber $0","~{v52}"(), !srcloc !{i32 10}
+ call void asm sideeffect "; clobber $0","~{v3}"(), !srcloc !{i32 10}
+ call void asm sideeffect "; clobber $0","~{v4}"(), !srcloc !{i32 10}
ret void
}
attributes #9 = { "amdgpu-waves-per-eu"="8,8" "amdgpu-flat-work-group-size"="64,64" "amdgpu-agpr-alloc"="11,59" }
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ8_12_59': desired occupancy was 8, final occupancy is 7
; WARN: warning: inline asm clobber list contains reserved registers: a12 at line 11
-; WARN: warning: inline asm clobber list contains reserved registers: v52 at line 11
+; WARN: warning: inline asm clobber list contains reserved registers: v4 at line 11
define amdgpu_kernel void @min_num_agpr_occ8_12_59() #10 {
call void asm sideeffect "; clobber $0","~{a11}"(), !srcloc !{i32 11}
call void asm sideeffect "; clobber $0","~{a12}"(), !srcloc !{i32 11}
- call void asm sideeffect "; clobber $0","~{v51}"(), !srcloc !{i32 11}
- call void asm sideeffect "; clobber $0","~{v52}"(), !srcloc !{i32 11}
+ call void asm sideeffect "; clobber $0","~{v3}"(), !srcloc !{i32 11}
+ call void asm sideeffect "; clobber $0","~{v4}"(), !srcloc !{i32 11}
ret void
}
attributes #10 = { "amdgpu-waves-per-eu"="8,8" "amdgpu-flat-work-group-size"="64,64" "amdgpu-agpr-alloc"="12,59" }
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ8_12_20': desired occupancy was 8, final occupancy is 7
; WARN: warning: inline asm clobber list contains reserved registers: a12 at line 12
-; WARN: warning: inline asm clobber list contains reserved registers: v52 at line 12
+; WARN: warning: inline asm clobber list contains reserved registers: v44 at line 12
define amdgpu_kernel void @min_num_agpr_occ8_12_20() #11 {
call void asm sideeffect "; clobber $0","~{a11}"(), !srcloc !{i32 12}
call void asm sideeffect "; clobber $0","~{a12}"(), !srcloc !{i32 12}
- call void asm sideeffect "; clobber $0","~{v51}"(), !srcloc !{i32 12}
- call void asm sideeffect "; clobber $0","~{v52}"(), !srcloc !{i32 12}
+ call void asm sideeffect "; clobber $0","~{v43}"(), !srcloc !{i32 12}
+ call void asm sideeffect "; clobber $0","~{v44}"(), !srcloc !{i32 12}
ret void
}
@@ -191,14 +188,14 @@ attributes #13 = { "amdgpu-waves-per-eu"="1,1" "amdgpu-flat-work-group-size"="64
; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ2_13_20': desired occupancy was 2, final occupancy is 1
; WARN: warning: inline asm clobber list contains reserved registers: a16 at line 15
; WARN: warning: inline asm clobber list contains reserved registers: a20 at line 15
-; WARN: warning: inline asm clobber list contains reserved registers: v240 at line 15
+; WARN: warning: inline asm clobber list contains reserved registers: v236 at line 15
define amdgpu_kernel void @min_num_agpr_occ2_13_20() #14 {
call void asm sideeffect "; clobber $0","~{a15}"(), !srcloc !{i32 15}
call void asm sideeffect "; clobber $0","~{a16}"(), !srcloc !{i32 15}
call void asm sideeffect "; clobber $0","~{a20}"(), !srcloc !{i32 15}
- call void asm sideeffect "; clobber $0","~{v239}"(), !srcloc !{i32 15}
- call void asm sideeffect "; clobber $0","~{v240}"(), !srcloc !{i32 15}
+ call void asm sideeffect "; clobber $0","~{v235}"(), !srcloc !{i32 15}
+ call void asm sideeffect "; clobber $0","~{v236}"(), !srcloc !{i32 15}
ret void
}
@@ -250,6 +247,8 @@ attributes #18 = { "amdgpu-waves-per-eu"="1,1" "amdgpu-flat-work-group-size"="64
; WARN: warning: inline asm clobber list contains reserved registers: a0 at line 20
; WARN: warning: inline asm clobber list contains reserved registers: a63 at line 20
; WARN: warning: inline asm clobber list contains reserved registers: a64 at line 20
+; WARN: warning: inline asm clobber list contains reserved registers: v64 at line 20
+; WARN: warning: inline asm clobber list contains reserved registers: v127 at line 20
; WARN: warning: inline asm clobber list contains reserved registers: v128 at line 20
define amdgpu_kernel void @min_num_agpr_occ4_0_64() #19 {
call void asm sideeffect "; clobber $0","~{a0}"(), !srcloc !{i32 20}
@@ -271,6 +270,8 @@ attributes #19 = { "amdgpu-waves-per-eu"="4,4" "amdgpu-flat-work-group-size"="64
; WARN: warning: inline asm clobber list contains reserved registers: a4 at line 21
; WARN: warning: inline asm clobber list contains reserved registers: a63 at line 21
; WARN: warning: inline asm clobber list contains reserved registers: a64 at line 21
+; WARN: warning: inline asm clobber list contains reserved registers: v64 at line 21
+; WARN: warning: inline asm clobber list contains reserved registers: v123 at line 21
; WARN: warning: inline asm clobber list contains reserved registers: v124 at line 21
define amdgpu_kernel void @min_num_agpr_occ4_1_64() #20 {
call void asm sideeffect "; clobber $0","~{a0}"(), !srcloc !{i32 21}
@@ -288,14 +289,13 @@ define amdgpu_kernel void @min_num_agpr_occ4_1_64() #20 {
attributes #20 = { "amdgpu-waves-per-eu"="4,4" "amdgpu-flat-work-group-size"="64,64" "amdgpu-agpr-alloc"="1,64" }
; 128 vector registers
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ4_32_64': desired occupancy was 4, final occupancy is 3
; WARN: warning: inline asm clobber list contains reserved registers: a32 at line 22
-; WARN: warning: inline asm clobber list contains reserved registers: v96 at line 22
+; WARN: warning: inline asm clobber list contains reserved registers: v64 at line 22
define amdgpu_kernel void @min_num_agpr_occ4_32_64() #21 {
call void asm sideeffect "; clobber $0","~{a31}"(), !srcloc !{i32 22}
call void asm sideeffect "; clobber $0","~{a32}"(), !srcloc !{i32 22}
- call void asm sideeffect "; clobber $0","~{v95}"(), !srcloc !{i32 22}
- call void asm sideeffect "; clobber $0","~{v96}"(), !srcloc !{i32 22}
+ call void asm sideeffect "; clobber $0","~{v63}"(), !srcloc !{i32 22}
+ call void asm sideeffect "; clobber $0","~{v64}"(), !srcloc !{i32 22}
ret void
}
@@ -409,13 +409,10 @@ attributes #29 = { "amdgpu-waves-per-eu"="8,10" "amdgpu-flat-work-group-size"="1
; The total vector register budget is 96
-
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ5__min_agpr_8_256': desired occupancy was 5, final occupancy is 4
-; WARN: warning: inline asm clobber list contains reserved registers: v88 at line 31
+; WARN: warning: inline asm clobber list contains reserved registers: v0 at line 31
; WARN: warning: inline asm clobber list contains reserved registers: a8 at line 31
define amdgpu_kernel void @min_num_agpr_occ5__min_agpr_8_256() #30 {
- call void asm sideeffect "; clobber $0","~{v87}"(), !srcloc !{i32 31}
- call void asm sideeffect "; clobber $0","~{v88}"(), !srcloc !{i32 31}
+ call void asm sideeffect "; clobber $0","~{v0}"(), !srcloc !{i32 31}
call void asm sideeffect "; clobber $0","~{a7}"(), !srcloc !{i32 31}
call void asm sideeffect "; clobber $0","~{a8}"(), !srcloc !{i32 31}
ret void
@@ -424,22 +421,21 @@ define amdgpu_kernel void @min_num_agpr_occ5__min_agpr_8_256() #30 {
attributes #30 = { "amdgpu-waves-per-eu"="5,5" "amdgpu-flat-work-group-size"="1024,1024" "amdgpu-agpr-alloc"="8,256" }
; The total vector register budget is 96
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ5__min_agpr_8': desired occupancy was 5, final occupancy is 4
-; WARN: warning: inline asm clobber list contains reserved registers: v88 at line 32
+; WARN: warning: inline asm clobber list contains reserved registers: v48 at line 32
; WARN: warning: inline asm clobber list contains reserved registers: a8 at line 32
define amdgpu_kernel void @min_num_agpr_occ5__min_agpr_8() #31 {
- call void asm sideeffect "; clobber $0","~{v87}"(), !srcloc !{i32 32}
- call void asm sideeffect "; clobber $0","~{v88}"(), !srcloc !{i32 32}
+ call void asm sideeffect "; clobber $0","~{v47}"(), !srcloc !{i32 32}
+ call void asm sideeffect "; clobber $0","~{v48}"(), !srcloc !{i32 32}
call void asm sideeffect "; clobber $0","~{a7}"(), !srcloc !{i32 32}
call void asm sideeffect "; clobber $0","~{a8}"(), !srcloc !{i32 32}
ret void
}
; budget is 96
-; WARN: warning: inline asm clobber list contains reserved registers: v88 at line 33
+; WARN: warning: inline asm clobber list contains reserved registers: v48 at line 33
define amdgpu_kernel void @min_num_agpr_occ5__min_agpr_8_no_agpr_references() #31 {
- call void asm sideeffect "; clobber $0","~{v87}"(), !srcloc !{i32 33}
- call void asm sideeffect "; clobber $0","~{v88}"(), !srcloc !{i32 33}
+ call void asm sideeffect "; clobber $0","~{v47}"(), !srcloc !{i32 33}
+ call void asm sideeffect "; clobber $0","~{v48}"(), !srcloc !{i32 33}
ret void
}
@@ -447,12 +443,11 @@ attributes #31 = { "amdgpu-waves-per-eu"="5,5" "amdgpu-flat-work-group-size"="10
; register budget 256
-; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'min_num_agpr_occ2__min_agpr_93': desired occupancy was 2, final occupancy is 1
-; WARN: warning: inline asm clobber list contains reserved registers: v160 at line 34
+; WARN: warning: inline asm clobber list contains reserved registers: v128 at line 34
; WARN: warning: inline asm clobber list contains reserved registers: a96 at line 34
define amdgpu_kernel void @min_num_agpr_occ2__min_agpr_93() #33 {
- call void asm sideeffect "; clobber $0","~{v159}"(), !srcloc !{i32 34}
- call void asm sideeffect "; clobber $0","~{v160}"(), !srcloc !{i32 34}
+ call void asm sideeffect "; clobber $0","~{v127}"(), !srcloc !{i32 34}
+ call void asm sideeffect "; clobber $0","~{v128}"(), !srcloc !{i32 34}
call void asm sideeffect "; clobber $0","~{a95}"(), !srcloc !{i32 34}
call void asm sideeffect "; clobber $0","~{a96}"(), !srcloc !{i32 34}
ret void
@@ -506,3 +501,149 @@ define amdgpu_kernel void @occ1_min_agpr_no_attr() #38 {
}
attributes #38 = { "amdgpu-waves-per-eu"="1,1" "amdgpu-flat-work-group-size"="256,256" }
+
+; register budget 128
+; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'num_agpr_occ4_min_4_max_4': desired occupancy was 4, final occupancy is 3
+; WARN: warning: inline asm clobber list contains reserved registers: v124 at line 40
+; WARN: warning: inline asm clobber list contains reserved registers: a4 at line 40
+define amdgpu_kernel void @num_agpr_occ4_min_4_max_4() #39 {
+ call void asm sideeffect "; clobber $0","~{v123}"(), !srcloc !{i32 40}
+ call void asm sideeffect "; clobber $0","~{v124}"(), !srcloc !{i32 40}
+ call void asm sideeffect "; clobber $0","~{a3}"(), !srcloc !{i32 40}
+ call void asm sideeffect "; clobber $0","~{a4}"(), !srcloc !{i32 40}
+ ret void
+}
+attributes #39 = { "amdgpu-waves-per-eu"="4,4" "amdgpu-flat-work-group-size"="1,1024" "amdgpu-agpr-alloc"="4,4" }
+
+; register budget 512, fully allocated and no warnings
+define amdgpu_kernel void @num_agpr_occ1_min_0_max_312() #40 {
+ call void asm sideeffect "; clobber $0","~{v255}"(), !srcloc !{i32 41}
+ call void asm sideeffect "; clobber $0","~{a255}"(), !srcloc !{i32 41}
+ ret void
+}
+attributes #40 = { "amdgpu-waves-per-eu"="1,1" "amdgpu-flat-work-group-size"="1,256" "amdgpu-agpr-alloc"="0,312" }
+
+; Register budget 64, VGPR ceiling 48, AGPR ceiling 16.
+; WARN: warning: <unknown>:0:0: failed to meet occupancy target given by 'amdgpu-waves-per-eu' in 'split_48_16': desired occupancy was 8, final occupancy is 7
+; WARN: warning: inline asm clobber list contains reserved registers: v48 at line 42
+; WARN: warning: inline asm clobber list contains reserved registers: a16 at line 42
+define amdgpu_kernel void @split_48_16() #41 {
+ call void asm sideeffect "; clobber $0","~{v47}"(), !srcloc !{i32 42}
+ call void asm sideeffect "; clobber $0","~{v48}"(), !srcloc !{i32 42}
+ call void asm sideeffect "; clobber $0","~{a15}"(), !srcloc !{i32 42}
+ call void asm sideeffect "; clobber $0","~{a16}"(), !srcloc !{i32 42}
+ ret void
+}
+attributes #41 = { "amdgpu-waves-per-eu"="8,8" "amdgpu-flat-work-group-size"="64,64" "amdgpu-agpr-alloc"="16,16" }
+
+; Register budget 64, VGPR ceiling 32, AGPR ceiling 16.
+; WARN: warning: inline asm clobber list contains reserved registers: v32 at line 43
+; WARN: warning: inline asm clobber list contains reserved registers: a16 at line 43
+define amdgpu_kernel void @split_32_16() #42 {
+ call void asm sideeffect "; clobber $0","~{v31}"(), !srcloc !{i32 43}
+ call void asm sideeffect "; clobber $0","~{v32}"(), !srcloc !{i32 43}
+ call void asm sideeffect "; clobber $0","~{a15}"(), !srcloc !{i32 43}
+ call void asm sideeffect "; clobber $0","~{a16}"(), !srcloc !{i32 43}
+ ret void
+}
+attributes #42 = { "amdgpu-waves-per-eu"="8,8...
[truncated]
|
yoonseoch
reviewed
Oct 27, 2026
| // We need to account for the fact that in this situation the AGPRs are not | ||
| // transferable so there is no reason to limit them | ||
| if (MaxVectorRegs > TotalNumAGPRs) { | ||
| MinNumAGPRs = std::max(MinNumAGPRs, MaxVectorRegs - TotalNumAGPRs); |
Contributor
There was a problem hiding this comment.
Why MaxVectorRegs - TotalNumAGPRs used toward MinNumAGPRs?
Contributor
Author
There was a problem hiding this comment.
MinNumAGPRs now limits how many AGPRs we will have available even if there is available space for them. If we know that it's unnecessary to limit the number of AGPRs then we should raise MinNumAGPRs so it doesn't get limited.
Considering it again, I think the difference should be used toward both MinNumAGPRs and MaxNumAGPRs. Also, although TotalNumAGPRs and NumArchVGPRs is currently always the same number, it more proper to use NumArchVGPRs. I need to think it over a little more.
Contributor
There was a problem hiding this comment.
I am still not so clear with checking MaxVectorRegs > TotalNumAGPRs. I guess you meant "it is unnecessary to limit the number of AGPRs", if the inequality holds. Why is so? Hasn't 'MaxVectorRegsthe same value asgetMaxNumVGPRs(F)`? Was there a function that hits this code in the lit-test?
Contributor
There was a problem hiding this comment.
Did you mean to change it to
MinNumAGPRs = std::max(MinNumAGPRs, MaxVectorRegs - TotalNumVGPRs);
?
JoshuaGrindstaff
force-pushed
the
amdgpu-agpr-alloc-max-codegen
branch
from
November 3, 2026 21:56
b494815 to
d4e930c
Compare
yoonseoch
reviewed
Nov 9, 2026
| The maximum will restrict allocation to use no more than max AGPRs. | ||
|
|
||
| Max AGPR is also used to limit the maximum number of VGPRs a function can | ||
| allocate. Max AGPRs specifies how many AGPRs a function needs to make space |
Contributor
There was a problem hiding this comment.
Max AGPRs specifies --? Max AGPR specifies
OR
--> Max AGPRs specify
I wonder if this new definition(?) of Max AGPR is to replace line 3390 above? "also" in the first sentence of 3392 implies a new usage model but not replacing 3390. Do they share the same meaning?
| // We need to account for the fact that in this situation the AGPRs are not | ||
| // transferable so there is no reason to limit them | ||
| if (MaxVectorRegs > TotalNumAGPRs) { | ||
| MinNumAGPRs = std::max(MinNumAGPRs, MaxVectorRegs - TotalNumAGPRs); |
Contributor
There was a problem hiding this comment.
Did you mean to change it to
MinNumAGPRs = std::max(MinNumAGPRs, MaxVectorRegs - TotalNumVGPRs);
?
Contributor
|
Sorry for the late response. I need some time to take a deeper look at this PR as well as the underlying issue since I almost forgot everything that we have discussed before. Lol. |
Change-Id: Id386cc14b9070f39e6366ed6d7a6c16cd5feaa68
Change-Id: I6d773d285eff398b53c62c8261130561ffebda54
Change-Id: I17f5024a0847a7593f17cdc852fa1eb517d8b152
JoshuaGrindstaff
force-pushed
the
amdgpu-agpr-alloc-max-codegen
branch
from
November 13, 2026 00:38
80952ab to
3f1f726
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the first PR of a two PR stack
#233968
#233960 <-- Current PR
mainIssue
The issue is described in the original PR (#218745) but I'll give a short description of it here. On gfx90a+, a kernel’s accum_offset and total vector-register count are each taken as an independent maximum over the whole call graph: the highest architectural VGPR count, aligned up, plus the highest AGPR count. Per function, getMaxNumVectorRegs still keeps VGPR + AGPR inside the occupancy budget from amdgpu-waves-per-eu and amdgpu-flat-work-group-size, but those two maxima can come from different functions. With amdgpu-agpr-alloc splitting the register file, a callee that uses many VGPRs and a sibling that uses many AGPRs can add up past the budget, so the kernel launches with a register split that silently misses the occupancy contract.
Change
This is the codegen portion of an attributor change in which I hope to use the second field of the amdgpu-agpr-alloc to limit the number of VGPRs a function can assign. So now the behavior of the amdgpu-agpr-alloc attribute now specifies the demand/usage with the min field (similar to before) and the max specifies how much space needs to be reserved so we don't miss the occupancy contract. To reserve that space, we take it out of the Max VGPRs.