merge amd-staging into amd-feature/wave-transform - #3766
Merged
cdevadas merged 2411 commits intoAug 5, 2026
Merged
Conversation
Add facts for SRem, if operands are known to be non-negative. Add signed bounds for `srem x, n`: * `x s>= 0` => result s>= 0 and result s<= x * `n s> 0` => result s<= n Alive2 Proofs: https://alive2.llvm.org/ce/z/e-zoAP Compile-time is in the noise https://llvm-compile-time-tracker.com/compare.php?from=60f965b1f62c0c77bcdb2997ea9bb6603aa0d002&to=ebc652af5700d569884e27812d33735d604990a1&stat=instructions:u InstCombine already has a similar fold, but with more limited reasoning. It does not trigger any changes on dtcxzyw/llvm-opt-benchmark-nightly#841 It simplifies a few times on other C/C++ workloads, including ffmpeg and OpenColorIO. Extracted end-to-end examples simplified with the change: https://clang.godbolt.org/z/1co6rvjKs This is part of an effort to improve ConstraintElimination support for IR generated by the Swift compiler, where such patterns are more common due to a number of signed runtime checks. PR: llvm#213453
…13457) This updates a number of scalar types in CallLowering to use integer.
…hen available (llvm#212845) On ARM64X targets, CRT provides separate TLS directory chunks, expecting the linker to sort it out. TLS directory uses _tls_start and _tls_end symbols to reference .tls section. Those symbols use section sorting to ensure that they are emitted at the start and end of .tls section, but that's not enough when we have two separate chunks for views: only one of them can really be the first one. Following MSVC, merge those chunks instead so that both symbol tables point to the same chunk. Additionally apply the same logic to _tls_used and _tls_index. This allows entire TLS directory to be shared between EC and native views. To achieve that, CRT additionally needs to mark each TLS callback with -arm64xsameaddress. This matches how MSVC linker and libraries work, but it requires EC and native views to use the same set of TLS callbacks. We may emit separate TLS directories in the future to make it more robust.
llvm#213450) optimizeFindIVReductions uses the step to determine if min or max is needed. Bail out if the direction of the step cannot be determined via SCEV. Fixes llvm#213424
Regular cross-team reductions have two phases: the intra-team reduction and the inter-team reduction. Atomic cross-team reductions replace the second phase with a atomic instruction which is used by the main thread of each team to directly fold the result of the intra-team reduction into the final result. Since this requires a combination of "data type" and "combine operation" for which an atomic instruction is available, only some (but very common) reductions can be transformed to atomic reductions. In cases where multiple reductions are performed on the same construct, the atomic path is only taken if all reductions can be transformed. Otherwise, we fall back to the regular cross-team reduction using a buffer with per-team slots. This is not strictly necessary, but hybrid reductions would induce more complexity with questionable benefit. Selecting an atomic path might not be the best option for every situation, which is why it is not enabled by default. Instead, it can be enabled via `-fopenmp-target-atomic-reduction`. Note that enabling the atomic path will not *force* atomic reductions. They will only be applied if possible, as described above. The performance (measured with https://github.com/ro-i/xteam-test @ c71339705091500f731e2a39f247d2660bacbdce, array size 177,777,777) is up to +15% faster (aka, more throughput) for supported reductions on a gfx942, with no noticeable regressions. Example: - sum reduction, type double: +10.22% faster - sum reduction, type uint: +15.57% faster - sum reduction, type ulong: +13.31% faster On a gfx90a, there is little to negative benefit: - sum reduction, type double: -4.32% faster (aka, slower) - sum reduction, type uint: +3.08% faster - sum reduction, type ulong: +1.68% faster Claude assisted with this patch.
An RTBridge Caller is a controller-side handle for calling a function in the runtime. Until now the abstraction assumed every such function was a trampoline -- a runtime function whose job is to invoke *another* function at an address the controller supplies (run-as-main, run-as-int, etc.) -- so every Caller carried a dedicated ExecutorAddr parameter for that target. Generalize Callers to call runtime functions of any shape. Invoking a supplied target is now just one kind of call, with the target address an ordinary leading argument rather than a built-in parameter: e.g. MainCaller becomes Caller<int64_t(ExecutorAddr, ArrayRef<std::string>)>. The SPS signatures already led with an SPSExecutorAddr for the target, so this is a pure interface change -- the SPS wrappers and all call sites are unaffected. It lets Callers model runtime functions that do the work themselves, such as the memory-access wrappers, rather than only those that dispatch to another function.
`clang++` defaults to `-stdlib=libc++` on NetBSD. When building with
both `clang` and `libcxx` included, the freshly built `clang++` fails to
find `<__config_site>`:
```
In file included from /usr/include/strings.h:68:
In file included from bin/../include/c++/v1/string.h:57:
bin/../include/c++/v1/__config:13:10: fatal
error: '__config_site' file not found
13 | #include <__config_site>
| ^~~~~~~~~~~~~~~
```
The file is present in `include/<triplet>/c++/v1`, but that isn't
searched by default. NetBSD has its own version of addLibCxxIncludePaths
which misses that directory.
This patch removes `NetBSD::addLibCxxIncludePaths` in favour of the
generic version in `Gnu.cpp`. The current code also adds
`/usr/include/c++`, although this directory only contains empty
directories in a default installation. It is only used when a bundled
version of LLVM is installed, which is not usually the case, and even
then contains a static version of `__config_site` that only applies to
`libcxxrt`.
Tested on `amd64-pc-netbsd10.1`, `x86_64-pc-solaris2.11`,
`x86_64-pc-linux-gnu`, and `x86_64-pc-freebsd15.1`.
…3914) On Windows/COFF, a dllimport call is emitted as an indirect call through an `__imp_` IAT slot (`callq *__imp_bar(%rip)`), and even a direct call to a library function is expected to bind to a thunk supplied by an import library. Today a JIT client must produce those import libraries themselves. `AutoImportGenerator` synthesizes them on demand instead. Bound to a single dynamic library via `AutoImportGenerator::Load(ES, ObjLinkingLayer, "/path/to/lib.dll")`. For each referenced export `X`, lazily synthesizes an `__imp_X` pointer slot holding `X`'s address in the library plus an `X` thunk that jumps through it, so both `__imp_`-mediated and direct references resolve. The library's export table is the authority: a name the library does not export is left unresolved, so the link fails exactly as a static link against the corresponding import library would (no silent invention of symbols). All synthesized stubs are owned by a single `ResourceTracker` (`getImportStubsResourceTracker()`), so a client can reclaim every synthesized slot/thunk in one step; subsequent imports start a fresh tracker. Relationship to `DLLImportDefinitionGenerator`: that generator resolves the underlying symbol through the JITDylib's link order. `AutoImportGenerator` is bound to one specific library and treats its export table as authoritative, giving fail-as-static-linker semantics. x86_64 and in-process execution only. - "Easy mode": every import is assumed to be a function; code and data are not distinguished, so data imports are unsupported (clients with data imports must supply an import library or use `__declspec(dllimport)`). `&X` resolves to the synthesized thunk, not the implementation in the target library. `llvm-jitlink -auto-import=<lib>` flag to attach the generator (in-process; errors if combined with out-of-process execution). Documentation in `llvm/docs/ORCv2.rst`. Partly implements github issue: llvm#190122 In the comment section of the github issue there is this comment llvm#190122 (comment) This PR implements point 3 ("Easy mode" generator)
…, NFC Reviewers: Pull Request: llvm#213538
The change is made in HexagonExpandCondsets::(predicate) function. The debug instructions are not predicable as they cannot be separated into conditional branches. So while predicating instructions in a machine basic block if we encounter any debug instructions we need to skip these instructions and continue with other instructions. The scan that collects the registers defined and used between the definition of the source register and the conditional transfer bailed out as soon as it saw a non-virtual register operand, which a DBG_VALUE can have. The transfer was then left as an unconditional A2_asrh plus an A2_tfrf instead of being folded into a single predicated A4_pasrhf, so again the generated code differed depending on whether debug info was enabled. Co-authored-by: Chandana Sinderikeri <csinderi@qti.qualcomm.com>
…m#212822) Do not call the Sema actions for absent, contains, and nullary assumption clauses after the parser has diagnosed that the clause is not allowed on the current directive. Add assertions documenting that these Sema actions must only receive clauses allowed on the current directive, and add tests covering all affected clause kinds. Fixes llvm#212780.
- Add LLVM_LIBC_ADD_FUNCTION_C_ALIAS macro to add another C alias public symbol to a function. - Add LIBC_CONF_SCANF_PROVIDE_ISOC99_ALIASES config - Add __isoc99_fscanf for generic fscanf target if LIBC_CONF_SCANF_PROVIDE_ISOC99_ALIASES is set. - Similarly: scanf, vfscanf, vscanf.
Found when looking through the generated Python file. `sig` doesn't exist in that context, it should be `idx`.
is_thread_crashed maps how each platform reports the bad access the test suite uses to simulate a crash, and had no case for Wasm, where there are no signals and a bad access raises a trap that a runtime reports as an exception. Without one it fell through to a description match that never held, so a crashed thread read as running fine.
This avoids defining an `enum`, which appears to be expensive for Clang.
…d link errors (llvm#213508) An undefined symbol error when building Flang in debug mode. This PR fix this by remove unused include file. Here is a brief error log: `/usr/bin/ld:lib/libFIRBuilder.a(MIFCommon.cpp.o): in function Fortran::evaluate::FunctionRef<Fortran::evaluate::Type<(Fortran::common::TypeCategory)4, 1> >::~FunctionRef()':`.
Firstly, by overwriting the symbol, it will have its existing flags, so there is no need to copy them back; all we need to do is mask out the other bits on the existing symbol. Secondly, copying the whole symbol just to preserve the symbol version that gets cleared by Defined::overwrite is a waste; just copy the single member to reinstate it.
llvm#213476) InstCombine canonicalizes sub nsw i8 -1, %a -> xor i8 %a, -1. Decompose the XOR as `sub nsw -1, %a` in the signed system Alive2 Proof: https://alive2.llvm.org/ce/z/f_YVjH This triggers quite rarely in C/C++ workloads (no end-to-end changes in dtcxzyw/llvm-opt-benchmark-nightly#840), found one instance in Blender. One simple end-to-end C example is https://clang.godbolt.org/z/G343shYje This is part of an effort to improve ConstraintElimination support for IR generated by the Swift compiler, where such patterns are more common due to a number of signed runtime checks. PR: llvm#213476
…vm#213303) Record the current behaviour over the types and subtargets where the fusion rules differ, f32, f16, bf16, f64, v2f32 and v2f16, with denormals both enabled and flushed, with and without the contract flags, and under -fp-contract=fast. Contributes to llvm#211092
…ounds (llvm#211960) Split evaluatePtrAddRecAtMaxBTCWillNotWrap into two stages: compute MaxOffset based on the step direction, then apply the shared MaxOffset <= DerefBytes check. Rename intermediate values to reflect what they actually represent. This restructuring makes two long-standing off-by-EltSize issues in the negative-step path explicit: * The lower-bound check is over-conservative by EltSize. * The upper-bound check under-counts by EltSize.
…2152) Lets plugins loaded with --load-dialect-plugin / --load-pass-plugin resolve MLIR and LLVM symbols against fir-opt, as mlir-opt already does. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Automated update of `LLVM_MAIN_REVISION` to match llvm/llvm-project `main` as merged into ROCm/amd-staging history. Computed revision: **591116**
…05134) Every function in `ObjectFileMachO` and `ObjectContainerMachOFileset` that iterates over load commands advances the file offset by `lc.cmdsize` after reading each command. A malformed command with cmdsize smaller than `sizeof(load_command)` (in particular cmdsize = 0) does not make forward progress, so the loop spins for ncmds iterations. With `ncmds` close to `INT_MAX` the function never returns in practice. Factor the read-and-validate step into a static template helper `ReadMachOCommand<T>` in each plugin's translation unit. It reads the 8-byte cmd/cmdsize header and returns false on EOF or on a cmdsize that is too small to make forward progress. All load-command loops now use this helper, replacing the previously duplicated GetU32 + cmdsize check. `T` may be `llvm::MachO::load_command` or any of its richer variants (uuid_command, dylib_command, thread_command, ident_command, encryption_info_command, ...). The helper only touches the leading cmd/cmdsize fields, leaving the rest of `T` for the caller to fill in. Affected loops in `ObjectFileMachO`: IsStripped, GetEncryptedFileRanges, CreateSections, ParseSymtab, GetUUID (static), GetAllArchSpecs (two loops), GetDependentModules, GetEntryPointAddress, GetNumThreadContexts, FindLC_NOTEByName, GetIdentifierString, GetVersion, FindMinimumVersionInfo And in ObjectContainerMachOFileset: ParseFileset Add unit tests (`ObjectFileMachOTest::ZeroCmdSize` and `ObjectContainerMachOFilesetTest::ZeroCmdSize`) that feed a 40-byte Mach-O with `ncmds = 0x7FFFFFFF` and `cmdsize = 0` into the relevant parsers. Without the fix the tests spin ~2 billion iterations; with the fix they return immediately. Found by lldb-target-fuzzer. Assisted-by: Claude
…lvm#211621) Part of llvm#211620. This is the first of two stacked changes implementing OpenMP `allocate` clause lowering for fixed-size intrinsic scalar `private` and `firstprivate` items on host `omp.parallel`. It carries each allocate item’s private-storage mapping through the OpenMP dialect, allocates with the requested allocator (using the runtime default for an omitted or null handle), and releases the storage during region finalization. The `align` modifier is handled by the stacked follow-up. Assisted-by: Copilot
… workflow (llvm#213871) From time to time we'll fail to fetch the test merge commit during the checkout step, e.g. see https://github.com/llvm/llvm-project/actions/runs/30626469894/job/91931527829 After a bit of research apparently the test merge commit is actually stored on the base repository, not the fork. I think this just happened to work previously because the fork repository synced objects in the background, but it's not always guaranteed to be available. So switch the checkout step to use llvm/llvm-project as the remote.
Here we have: artifact combiner creating one element unmerge and unmerge lowering of FP source using FP type for bit twiddling.
Bug in LegalizationArtifactCombiner when: DstSize < UnmergeSrcSize case can create unmerge with one element. DstSize > UnmergeSrcSize case can end up attempting to create merge with one source element and hits assert(TmpVec.size() > 1).
This is a follow-up to llvm#153683 to support OpenMP compliant pointer-attachment in `declare_mappers` via `ATTACH`-style maps. In addition to enabling attach-style maps, we also need to propagate information about which map entries are for "pointee" data, i.e. have an "attach-ptr", and thus occupy a different storage block than the base variable for which the mapper is being generated. e.g. ```c S sa[10]; ``` The entry emitted for `s.p[0:10]` is for the pointee, i.e. it does not share storage with `s`. Mapper codegen needs to know that the entry for `sa[1].p[0:10]`, for example, is not a `MEMBER_OF` the map of `sa`, as it occupies its own storage and has its own ref-count tracking etc. Flang currently passes an unconditional `PreserveMemberofFlags` bool to the OMPIRBuilder function, which should eventually be propagated to using the per-entry information so that `map(s%x)` should get MEMBER_OF during mapper codgen, but `map(s%p(1:10))` should not. Currently, the per-entry MapInfo field is set to `false` for flang, so the change is a no-op for it. I don't have enough flang expertise/testing resources, so I'll let Andrew update Flang in follow-up changes. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…ion. (#3485) On GPU targets, OMPIRBuilder marked outlined OpenMP parallel-region functions alwaysinline, forcing them to be inlined into their kernel regardless of cost. The enlarged kernel then exceeded the AMDGPU inliner's basic-block budget, so __kmpc_target_init could no longer be inlined/specialized and stayed a shared, out-of-line generic-mode runtime entry. As a result, SPMD reduction kernels that never run in generic mode still inherited the module-wide amdgpu.max_num_vgpr worst case — over-reserving registers and collapsing occupancy. A single generic-mode kernel anywhere in the module was enough to trigger this for unrelated, otherwise-lean kernels. Letting cost-based inlining decide (matching upstream) keeps these kernels small so __kmpc_target_init is inlined and register usage/occupancy return to normal.
cdevadas
requested review from
a team,
b-sumner,
chinmaydd,
david-salinas and
lamb-j
as code owners
August 5, 2026 05:55
|
PSDB Build Link: http://mlse-bdc-20dd129:8065/#/builders/10/builds/743 |
vg0204
approved these changes
Aug 5, 2026
cdevadas
deleted the
public/amd/dev/cdevadas/wave-transform/merge-from-stg-aug-5-26
branch
August 5, 2026 12:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.