Move llvmlite to LLVM 20 #1092

yashssh · 2024-10-25T10:09:02Z

~~Depends and contains changes introduced with #1091~~

List of changes in this MR

Delete legacy pass manager code
Delete typed pointer code
Update APIs that needs updating for llvm19
- Pass registry and initialization APIs support dropped: https://reviews.llvm.org/D145043
- StandardIntrumentations checks function analysis invalidation in module passes as well: https://reviews.llvm.org/D146160
- TargetParser.getHostCPUFeatures() returns a StringMap: [llvm][TargetParser] Return StringMap from getHostCPUFeatures llvm/llvm-project#97824
- CodeGenOpt::Level turned into an enum class: [NFC][CodeGen] Change CodeGenOpt::Level/CodeGenFileType into enum classes llvm/llvm-project#66295
Update refprune pass and its tests

TODO:

run with remarks feature ~~and pass timer features~~ have to be added

(edited by @gmarkall to link to relevant review for dropping initialization APIs)

gmarkall · 2024-11-05T14:18:33Z

/azp run

azure-pipelines · 2024-11-05T14:18:43Z

Azure Pipelines successfully started running 1 pipeline(s).

rj-jesus · 2025-01-13T08:31:44Z

ffi/core.cpp

    auto context = LLVMGetGlobalContext();
-    LLVMContextSetOpaquePointers(context, enableOpaquePointers);
+    // Opaque pointer support dropped https://reviews.llvm.org/D139441


I believe you meant "Typed pointer support dropped"?

You are right, thanks for spotting! Will update :)

I think we can remove these comments (and the same below).

rj-jesus · 2025-01-13T08:32:13Z

ffi/core.cpp

    LLVMContextRef context = LLVMContextCreate();
-    LLVMContextSetOpaquePointers(context, enableOpaquePointers);
+    // Opaque pointer support dropped https://reviews.llvm.org/D139441


(Same as above.)

detrout · 2025-03-03T02:21:04Z

Hi,

I found time to try and work on getting llvmlite to build with a version of llvm that's in Debian.

I took this pull request and rebased it against 0.44.0, and then collapse the branch into a single diff to apply to the a Debian 0.44.0 version of the llvmlite package.

I pushed what I've manage to get done to https://github.com/detrout/llvmlite/tree/llvm19

I noticed there's a bunch of code removed this pull request in a migration from passmanager to newpassmanager, though the llvmlite HEAD branch currently has both passmanager and newpassmanager modules. I don't know if it's intended to remove passmanager or not?

The merged code almost works (at least on Debian unstable on a x86 cpu with python 3.13.2 and llvm-19.1.7) it's down to two test failures.

======================================================================             
FAIL: test_object_file (llvmlite.tests.test_binding.TestObjectFile.test_object_file)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.13_llvmlite/build/llvmlite/tests/test_binding.py", line 2443, in test_object_file
    self.assertTrue(s.size() > 0)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
AssertionError: False is not true
                               
======================================================================
FAIL: test_lookup_current_process_symbol_fails (llvmlite.tests.test_binding.TestOrcLLJIT.test_lookup_current_process_symbol_fai
ls)                  
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.13_llvmlite/build/llvmlite/tests/test_binding.py", line 1493, in test_lookup_curre
nt_process_symbol_fails                                        
    with self.assertRaisesRegex(RuntimeError, msg):
         ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
AssertionError: RuntimeError not raised
                                                                                                                               
----------------------------------------------------------------------
Ran 379 tests in 7.699s

FAILED (failures=2, skipped=2)

I did try running the build under arm64 using qemu and funny story, all the tests passed.

Ran 379 tests in 171.947s
                               
OK (skipped=19)

detrout · 2025-03-03T18:00:25Z

I'd also managed to run the two test failures in pdb, and for the llvmlite.tests.test_binding.TestObjectFile.test_object_file

I found a possibly useful thing that there was a .text segment that had no contents. Though there were other text segments like (I think) .ltext that had some contents

detrout · 2025-03-19T05:35:10Z

I am deep into places I don't understand and could use help. This is the test failure from test_binding.py::TestObjectFile.test_object_file() on an older x86_64 cpu

I figured out how to print the assembly code.

This is with llvm-19 and there's a .section section that the llvm-15 version doesn't use

        .text
        .file   "asm_sum.c"
        .section        .ltext,"axl",@progbits
        .globl  sum
        .p2align        4, 0x90
        .type   sum,@function
sum:
        .cfi_startproc
        leal    (%rdi,%rsi), %eax
        retq
.Lfunc_end0:
        .size   sum, .Lfunc_end0-sum
        .cfi_endproc

        .type   glob,@object
        .section        .lbss,"awl",@nobits
        .globl  glob
        .p2align        2, 0x0
glob:
        .long   0
        .size   glob, 4

        .type   glob_b,@object
        .globl  glob_b
glob_b:
        .byte   0
        .size   glob_b, 1

        .type   glob_f,@object
        .section        .ldata,"awl",@progbits
        .globl  glob_f
        .p2align        2, 0x0
glob_f:
        .long   0x3fc00000
        .size   glob_f, 4

        .type   glob_struct,@object
        .section        .lbss,"awl",@nobits
        .globl  glob_struct
        .p2align        4, 0x0
glob_struct:
        .zero   24
        .size   glob_struct, 24

        .section        ".note.GNU-stack","",@progbits

This is what was generated from llvm 15.0.7 from conda.

        .text                                                                                                            
        .file   "asm_sum.c"                                                                                              
        .globl  sum                                                                                                      
        .p2align        4, 0x90                                                                                          
        .type   sum,@function                                                                                            
sum:                                                                                                                     
        .cfi_startproc                                                                                                   
        leal    (%rdi,%rsi), %eax                                                                                        
        retq                                                                                                             
.Lfunc_end0:                                                                                                             
        .size   sum, .Lfunc_end0-sum                                                                                     
        .cfi_endproc                                                                                                     
                                                                                                                         
        .type   glob,@object                                                                                             
        .bss                                                                                                             
        .globl  glob                                                                                                     
        .p2align        2                                                                                                
glob:                                                                                                                    
        .long   0
        .size   glob, 4

        .type   glob_b,@object
        .globl  glob_b
glob_b:
        .byte   0
        .size   glob_b, 1

        .type   glob_f,@object
        .data
        .globl  glob_f
        .p2align        2
glob_f:
        .long   0x3fc00000
        .size   glob_f, 4

        .type   glob_struct,@object
        .bss
        .globl  glob_struct
        .p2align        4
glob_struct:
        .zero   24
        .size   glob_struct, 24

        .section        ".note.GNU-stack","",@progbits

yashssh · 2025-03-19T05:43:32Z

Hi @detrout, thanks for looking into this! The pull request is due a rebase/merge of main for a long time now. The time I created this pull request all tests were passing which obviously isn't the case right now. May I know what are you trying to do with these changes? Meanwhile let me try running the tests again locally.

detrout · 2025-03-19T06:21:28Z

What's going on is Debian is coming close to a freeze and our release team dropped our llvm-15 for having too many unfixed bugs. which then removed llvmlite, and that removes numba, and that breaks a whole bunch of scientific python packages. We do have llvm-19 and I when I saw this pull request and thought I'd take a stab at merging it.

What I did was to take the original pull request and rebase it against the 0.44.0 release of llvmlite, to try and get something that might be reasonable for Debian to ship. So that's why I was aiming for the release version of llvmlite instead of HEAD.

At this point I made progress more out of stubbornness than any deep understanding of llvm. Though I'm learning bits as I try to figure out whats going on.

I pushed my updated debian package to https://salsa.debian.org/diane/llvmlite

Though unlike the github version that has a flattened version of the llvm-19 compatibility patch.

yashssh · 2025-03-19T06:49:08Z

What's going on is Debian is coming close to a freeze and our release team dropped our llvm-15 for having too many unfixed bugs. which then removed llvmlite, and that removes numba, and that breaks a whole bunch of scientific python packages.

I'm not sure if this helps but llvmlite v0.44 do support LLVM 16 out of the box https://llvmlite.readthedocs.io/en/latest/release-notes.html

picca · 2025-03-19T08:26:11Z

It seems that the minimum version available in Debian testing is llvm-18

yashssh · 2025-03-19T09:06:10Z

@detrout I looked the assembly assertion error and also reproduced it locally. It doesn't look like anything is wrong, just a slight update in the code layout I'm not exactly sure why. You can just update the test and move on

detrout · 2025-03-20T03:17:11Z

I added 2 commits to deal with the test failures with llvm-19 at https://github.com/detrout/llvmlite/commits/llvm19/

what would you like next? try to rebase that change set against HEAD?

yashssh · 2025-03-20T03:23:27Z

what would you like next? try to rebase that change set against HEAD?

For this PR? I don't think a rebase is required, I did merge the main branch into it. We would have to wait for the reviews to move forward. Right now numba/numba#9676 is the first hurdle before this can be merged.

rj-jesus · 2025-03-20T08:47:21Z

What's going on is Debian is coming close to a freeze and our release team dropped our llvm-15 for having too many unfixed bugs. which then removed llvmlite, and that removes numba, and that breaks a whole bunch of scientific python packages. We do have llvm-19 and I when I saw this pull request and thought I'd take a stab at merging it.

I'm happy to hear Debian's planning to ship an updated llvmlite+llvm release! 😄

What I did was to take the original pull request and rebase it against the 0.44.0 release of llvmlite, to try and get something that might be reasonable for Debian to ship. So that's why I was aiming for the release version of llvmlite instead of HEAD.

I think this should work in principle, but I'd suggest rebasing this against HEAD instead. This PR precedes a few others where we improved support for LLVM opaque pointers. Without them, some of the code paths in Numba may not work with LLVM 17+. Alternatively, you could cherry-pick these PRs onto the 0.44 release, which I believe should work as the changes are mostly self-contained. The relevant PRs are #1159, #1158 and #1160 (merged in this order). Please let me know if you'd like some help with this.

gmarkall · 2025-03-20T12:34:11Z

@yashssh Will numba/numba#9676 also be needed in Numba for it to work with LLVM >= 17? Would @detrout need to grab that and incorporate it into the Debian sources?

yashssh · 2025-03-21T08:42:07Z

@yashssh Will numba/numba#9676 also be needed in Numba for it to work with LLVM >= 17? Would @detrout need to grab that and incorporate it into the Debian sources?

Yes that's correct.

@detrout if you are cherrypicking the changes on top of last release you will also need #1163 in addition to what Ricardo mentioned (perhaps it's easier to rebase on top of main).

numba/numba#9676 introduces a switch to use either the Legacy or the New pass manager in Numba(defaulting to the new PM).

That merge request is supposed to be followed by another code cleanup change that drops support for the legacy pass manager entirely in Numba. Post those changes we can merge this change in llvmlite.

For simplifying things I have consolidated all required Numba changes in this draft pull request, which should allow you to move forward in Numba with a single patch.

detrout · 2025-03-21T16:24:38Z

My first stab at building debianianized numba 0.61.0 with this patched version of llvmlite 0.44 didn't work and I need to figure out why.

Thank you all for highlighting more pull requests that might be needed, hopefully I'll have some time soon to try and start plugging them together.

yashssh · 2025-03-26T08:50:47Z

@detrout we have rebased the required llvmlite LLVM19 changes on top of last release (v0.44) #1182 combined with necessary numba changes numba/numba#10011 should allow you to build and test everything with LLVM19. You might still have to add workarounds for the x86 failures you saw.

detrout · 2025-04-09T04:48:51Z

llvmlite build on x86_64 and on arm64 (under qemu) with the recommend numba patch numba/numba#10011 and https://github.com/numba/numba/commit/1e2fb1d1c81880a645345991eb48e39b578e119d.patch

I was able to get numba to build using llvmlite 0.44 with llvm-19. it built and ran it's tests under x86_64 but I got a timeout error for arm64 under qemu.... Though I know that numba tests take forever.

List of changes in this MR - Delete legacy pass manager code - Delete typed pointer code - Update APIs that needs updating for llvm19 - Update refprune pass and its tests

yashssh · 2025-05-28T10:54:12Z

I have rebased the changes on top of main since #1091 was merged.

gmarkall · 2025-05-29T10:40:03Z

ffi/build.py

-        elif version != 15:
-
-            msg = ("Building llvmlite requires LLVM 15, got "
+        if version not in (17, 18, 19):


I think we're not going to be trying to do any testing with LLVM 17 or 18, so it'd probably be better to mandate 19 here. If someone wants to try 17, 18, 20, or whatever on their system / distribution, they should add their own patch here.

Changed the check to only be valid for v20 and throw warning for higher LLVM versions

gmarkall · 2025-05-29T10:52:13Z

ffi/custom_passes.cpp

@@ -1192,84 +1185,21 @@ class RefNormalizePass : public PassInfoMixin<RefNormalizePass> {
    RefNormalizePass() = default;

    PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM) {
-        RefNormalize().runOnFunction(F);
+        if (RefNormalize().runOnFunction(F)) {
+            return PreservedAnalyses::none();


It seems a bit suspect that run() previously always returned PreservedAnalyses::all()... Out of interest, did you notice any particular problem during testing, or is this just added to ensure correctness in general?

I didn't notice any issue while testing. I assumed since we are making good changes to the underlying IR not preserving the analyses made more sense

gmarkall · 2025-05-29T10:54:25Z

ffi/memorymanager.h

 #define LLVMLITE_ALIGN Align
 #define GET_ALIGN_VALUE(align) align.value()
-#endif


We can probably replace uses of LLVMLITE_ALIGN and GET_ALIGN_VALUE with Align and <align>.value() now, and that would bring the code closer back to upstream. I initially needed to add these macros so that we could support different LLVM versions.

(I don't think this is a critical issue for this PR though... Happy for it to be left as-is... one day I'd like to imagine we're moved over to OrcJIT and JITLink so that this code is no longer needed)

Looks like a small change, I'll update it

gmarkall · 2025-05-29T10:59:45Z

ffi/orcjit.cpp

+        llvm::orc::SymbolMap sym_map;
+        sym_map[mangled] = symbol;


This appears to be unused - does it have some side-effect?

Don't recall how it ended up here, will remove

gmarkall · 2025-05-29T11:12:48Z

llvmlite/binding/targets.py

-        5: "Wasm",
-        6: "XCOFF",
-    }
+_object_formats = {


Historically this got out of sync with Triple.h in LLVM., I've just checked - this still matches the ObjectFormatType enum in Triple.h in LLVM 19.

gmarkall · 2025-05-30T10:55:11Z

ffi/build.py

+        if version == 20:
+            pass
+        elif version > 20:


This feels a bit funny. I think we can change it to:

Suggested change

if version == 20:

pass

elif version > 20:

if version > 20:

with a corresponding change below:

gmarkall · 2025-05-30T10:55:37Z

ffi/build.py

-
-            msg = ("Building llvmlite requires LLVM 15, got "
-                   "{!r}. Be sure to set LLVM_CONFIG to the right executable "
+        else:


Corresponding change to the above:

Suggested change

else:

elif version < 20:

gmarkall · 2025-05-30T11:37:20Z

With this PR and the package built by #1226, I get:

$ gdb --args python runtests.py 

(gdb) run

Starting program: /home/gmarkall/miniforge3/envs/test-llvmlite-20-py311/bin/python runtests.py
..s..
Program received signal SIGSEGV, Segmentation fault.
0x00007fffefe6635a in llvm::RegisterTargetMachine<llvm::X86TargetMachine>::Allocator(llvm::Target const&, llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::TargetOptions const&, std::optional<llvm::Reloc::Model>, std::optional<llvm::CodeModel::Model>, llvm::CodeGenOptLevel, bool) () from /home/gmarkall/numbadev/llvmlite/llvmlite/binding/libllvmlite.so
(gdb) bt
#0  0x00007fffefe6635a in llvm::RegisterTargetMachine<llvm::X86TargetMachine>::Allocator(llvm::Target const&, llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::TargetOptions const&, std::optional<llvm::Reloc::Model>, std::optional<llvm::CodeModel::Model>, llvm::CodeGenOptLevel, bool) () from /home/gmarkall/numbadev/llvmlite/llvmlite/binding/libllvmlite.so
#1  0x00007fffefd77dd6 in llvm::Target::createTargetMachine (this=0x7ffff6a87d20 <llvm::getTheX86_64Target()::TheX86_64Target>, 
    TT=..., CPU=..., Features=..., Options=..., RM=..., CM=..., OL=llvm::CodeGenOptLevel::Default, JIT=true)
    at /home/gmarkall/miniforge3/envs/test-llvmlite-20-py311/include/llvm/MC/TargetRegistry.h:462
#2  0x00007fffefd78a6c in LLVMPY_CreateTargetMachine (T=0x7ffff6a87d20 <llvm::getTheX86_64Target()::TheX86_64Target>, 
    Triple=0x7ffff748af50 "x86_64-unknown-linux-gnu", CPU=0x555555aaa368 <_PyRuntime+9096> "", 
    Features=0x555555aaa368 <_PyRuntime+9096> "", OptLevel=2, RelocModel=0x7fffef57cf20 "default", 
    CodeModel=0x7fffef57ccb0 "jitdefault", PrintMC=0, JIT=1, ABIName=0x555555aaa368 <_PyRuntime+9096> "")

@yashssh Have you run on x86 as well? I presume you have not encountered segfaults in the test suite so far?

gmarkall added the 2 - In Progress label Nov 5, 2024

yashssh mentioned this pull request Jan 8, 2025

Update llvmlite to be compatible with llvm-17 #1042

Closed

rj-jesus reviewed Jan 13, 2025

View reviewed changes

BwL1289 mentioned this pull request Jan 21, 2025

Continuous Discussion: LLVM update plan #1048

Open

yashssh added a commit to yashssh/llvmlite that referenced this pull request Mar 26, 2025

Apply numba#1092

b910270

yashssh added 3 - Ready for Review and removed 2 - In Progress labels May 16, 2025

Move llvmlite to llvm19

ccfbf78

List of changes in this MR - Delete legacy pass manager code - Delete typed pointer code - Update APIs that needs updating for llvm19 - Update refprune pass and its tests

yashssh force-pushed the yashwants/llvm19 branch from 2e4c738 to ccfbf78 Compare May 28, 2025 10:50

gmarkall reviewed May 29, 2025

View reviewed changes

yashssh added 2 commits May 29, 2025 04:47

Support building with LLVM20, add back codegen time passes APIs

04a1f68

Remove LLVMLITE_ALIGN macro and unused value in LLVMPY_LLJIT_Link

47e299f

yashssh changed the title ~~Move llvmlite to llvm19~~ Move llvmlite to LLVM 20 May 29, 2025

gmarkall reviewed May 30, 2025

View reviewed changes

gmarkall requested a review from esc June 3, 2025 14:20

seibert mentioned this pull request Jun 10, 2025

Does llvmlite support riscv64? #923

Open

gmarkall added this to the v0.45.0 milestone Jun 24, 2025

yashssh mentioned this pull request Jun 26, 2025

Default to static linking, override with LLVMLITE_SHARED #1234

Open

Move llvmlite to LLVM 20 #1092

Are you sure you want to change the base?

Move llvmlite to LLVM 20 #1092

Uh oh!

Conversation

yashssh commented Oct 25, 2024 • edited by gmarkall Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gmarkall commented Nov 5, 2024

Uh oh!

azure-pipelines bot commented Nov 5, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

detrout commented Mar 3, 2025

Uh oh!

detrout commented Mar 3, 2025

Uh oh!

detrout commented Mar 19, 2025

Uh oh!

yashssh commented Mar 19, 2025

Uh oh!

detrout commented Mar 19, 2025

Uh oh!

yashssh commented Mar 19, 2025

Uh oh!

picca commented Mar 19, 2025

Uh oh!

yashssh commented Mar 19, 2025

Uh oh!

detrout commented Mar 20, 2025

Uh oh!

yashssh commented Mar 20, 2025

Uh oh!

rj-jesus commented Mar 20, 2025

Uh oh!

gmarkall commented Mar 20, 2025

Uh oh!

yashssh commented Mar 21, 2025

Uh oh!

detrout commented Mar 21, 2025

Uh oh!

yashssh commented Mar 26, 2025

Uh oh!

detrout commented Apr 9, 2025

Uh oh!

yashssh commented May 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gmarkall commented May 30, 2025

Uh oh!

yashssh commented Oct 25, 2024 •

edited by gmarkall

Loading