Access Violation on Windows with new PassManager

[esc]

$ python -m llvmlite.tests
..s.s.................................s...................................................................Windows fatal exception: access violation

Current thread 0x00000fb0 (most recent call first):
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\ffi.py", line 197 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\newpassmanagers.py", line 29 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3072 in test_instcombine
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 579 in _callTestMethod
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 623 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 678 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\runner.py", line 217 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\main.py", line 274 in runTests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 97 in run_tests_real
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 112 in runTests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\main.py", line 102 in __init__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 38 in __init__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__init__.py", line 50 in run_tests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__init__.py", line 56 in main
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__main__.py", line 3 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
EWindows fatal exception: access violation

Current thread 0x00000fb0 (most recent call first):
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\ffi.py", line 197 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\newpassmanagers.py", line 29 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3082 in test_optnone
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 579 in _callTestMethod
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 623 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 678 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\runner.py", line 217 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\main.py", line 274 in runTests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 97 in run_tests_real
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 112 in runTests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\main.py", line 102 in __init__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 38 in __init__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__init__.py", line 50 in run_tests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__init__.py", line 56 in main
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__main__.py", line 3 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
E.Windows fatal exception: access violation

Current thread 0x00000fb0 (most recent call first):
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\ffi.py", line 197 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\newpassmanagers.py", line 29 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3048 in run_o_n
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3057 in test_run_o3
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 579 in _callTestMethod
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 623 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\case.py", line 678 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\runner.py", line 217 in run
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\main.py", line 274 in runTests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 97 in run_tests_real
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 112 in runTests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\unittest\main.py", line 102 in __init__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\customize.py", line 38 in __init__
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__init__.py", line 50 in run_tests
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__init__.py", line 56 in main
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\__main__.py", line 3 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
E....ssssssssssssssss.................s...................................................................................................................................................................................................................................................................
======================================================================
ERROR: test_instcombine (llvmlite.tests.test_binding.TestNewModulePassManager.test_instcombine)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3072, in test_instcombine
    mpm.run(mod, pb)
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\newpassmanagers.py", line 29, in run
    ffi.lib.LLVMPY_RunNewModulePassManager(self, module, pb)
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\ffi.py", line 197, in __call__
    return self._cfn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: access violation reading 0x000000000000011F

======================================================================
ERROR: test_optnone (llvmlite.tests.test_binding.TestNewModulePassManager.test_optnone)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3082, in test_optnone
    mpm.run(mod, pb)
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\newpassmanagers.py", line 29, in run
    ffi.lib.LLVMPY_RunNewModulePassManager(self, module, pb)
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\ffi.py", line 197, in __call__
    return self._cfn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: access violation reading 0x0000000000000120

======================================================================
ERROR: test_run_o3 (llvmlite.tests.test_binding.TestNewModulePassManager.test_run_o3)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3057, in test_run_o3
    orig_asm, optimized_asm = self.run_o_n(3)
                              ^^^^^^^^^^^^^^^
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\tests\test_binding.py", line 3048, in run_o_n
    mpm.run(mod, pb)
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\newpassmanagers.py", line 29, in run
    ffi.lib.LLVMPY_RunNewModulePassManager(self, module, pb)
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\ffi.py", line 197, in __call__
    return self._cfn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: access violation reading 0x0000000000000122

[yashssh]

What is the target_triple for the system?

[esc]

What is the target_triple for the system?

Can you tell me the commands needed to discover that. I looked at:

https://llvmlite.readthedocs.io/en/latest/user-guide/binding/target-information.html

but then I only got:

$ python simple.py
Traceback (most recent call last):
  File "C:\Users\ci\simple.py", line 2, in <module>
    target = binding.Target.from_default_triple()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\targets.py", line 231, in from_default_triple
    return cls.from_triple(triple)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ci\64\miniconda3\envs\testenv_7c169bb9\Lib\site-packages\llvmlite\binding\targets.py", line 242, in from_triple
    raise RuntimeError(str(outerr))
RuntimeError: Unable to find target for this triple (no targets are registered)
(testenv_7c169bb9)

[esc]

So I used:

https://llvmlite.readthedocs.io/en/latest/user-guide/binding/target-information.html#llvmlite.binding.get_default_triple

And this gave:

x86_64-pc-windows-msvc

[yashssh]

Thank you that's what I was looking for! Although all 3 reported tests pass for me with this triple, confirming what Graham suggested to me locally that it's probably a false alarm.

FYI, this is how I tested it:

diff --git a/llvmlite/tests/test_binding.py b/llvmlite/tests/test_binding.py
index 59e653c..3536ad5 100644
--- a/llvmlite/tests/test_binding.py
+++ b/llvmlite/tests/test_binding.py
@@ -653,6 +653,7 @@ class BaseTest(TestCase):
     def setUp(self):
         llvm.initialize()
         llvm.initialize_native_target()
+        llvm.initialize_all_targets()
         llvm.initialize_native_asmprinter()
         gc.collect()
         self.old_garbage = gc.garbage[:]
@@ -3005,6 +3006,13 @@ class NewPassManagerMixin(object):
         pb = llvm.create_pass_builder(tm, pto)
         return pb
 
+    def pb_windows(self, speed_level=0, size_level=0):
+        target = llvm.Target.from_triple("x86_64-pc-windows-msvc")
+        tm = target.create_target_machine()
+        pto = llvm.create_pipeline_tuning_options(speed_level, size_level)
+        pb = llvm.create_pass_builder(tm, pto)
+        return pb
+
 
 class TestPassBuilder(BaseTest, NewPassManagerMixin):
 
@@ -3043,7 +3051,7 @@ class TestNewModulePassManager(BaseTest, NewPassManagerMixin):
     def run_o_n(self, level):
         mod = self.module()
         orig_asm = str(mod)
-        pb = self.pb(speed_level=level, size_level=0)
+        pb = self.pb_windows(speed_level=level, size_level=0)
         mpm = pb.getModulePassManager()
         mpm.run(mod, pb)
         optimized_asm = str(mod)
@@ -3064,7 +3072,7 @@ class TestNewModulePassManager(BaseTest, NewPassManagerMixin):
         self.assertIn("%.4", optimized_asm)
 
     def test_instcombine(self):
-        pb = self.pb()
+        pb = self.pb_windows()
         mpm = self.pm()
         mpm.add_instruction_combine_pass()
         mod = self.module(asm_sum4)
@@ -3075,7 +3083,7 @@ class TestNewModulePassManager(BaseTest, NewPassManagerMixin):
         self.assertNotIn("%.3", optimized_asm)
 
     def test_optnone(self):
-        pb = self.pb(speed_level=3, size_level=0)
+        pb = self.pb_windows(speed_level=3, size_level=0)
         orig_asm = str(asm_alloca_optnone.replace("optnone ", ""))

[esc]

I can also no longer reproduce this and your sentiment matches mine @yashssh @gmarkall @stuartarchibald.

[gmarkall]

It reproduces again. The last couple of llvmlite builds from the buildfarm will reproduce it, e.g.:

• win-64/llvmlite-0.44.0dev0-py312h9a6739d_139.tar.bz2
• win-64/llvmlite-0.44.0dev0-py311h9a6739d_137.tar.bz2

However, when I added the PDB to it (#1089), builds that came from that do not reproduce the issue, e.g.:

• win-64/llvmlite-0.44.0dev0-py311h9a6739d_141.tar.bz2
• win-64/llvmlite-0.44.0dev0-py312h9a6739d_141.tar.bz2
• win-64/llvmlite-0.44.0dev0-py310h9a6739d_141.tar.bz2

When I edited the binding init to dump IR after each pass, with:

set_option("llvmlite", "-print-after-all")

in llvmlite/binding/__init__.py, I got:

test_instcombine (llvmlite.tests.test_binding.TestNewFunctionPassManager.test_instcombine) ... Assertion failed: !NodePtr->isKnownSentinel(), file C:\Users\ci\64\miniconda3\envs\numba-ci\conda-bld\llvmdev_1723575894751\work\llvm\include\llvm/ADT/ilist_iterator.h, line 138
Fatal Python error: Aborted

Current thread 0x00009e7c (most recent call first):
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\binding\ffi.py", line 197 in __call__
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\binding\newpassmanagers.py", line 91 in run
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\tests\test_binding.py", line 3176 in test_instcombine
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\case.py", line 579 in _callTestMethod
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\case.py", line 623 in run
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\case.py", line 678 in __call__
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\suite.py", line 122 in run
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\runner.py", line 217 in run
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\main.py", line 274 in runTests
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\tests\customize.py", line 97 in run_tests_real
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\tests\customize.py", line 112 in runTests
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\unittest\main.py", line 102 in __init__
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\tests\customize.py", line 38 in __init__
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\tests\__init__.py", line 50 in run_tests
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\tests\__init__.py", line 56 in main
  File "C:\Users\gmarkall\AppData\Local\miniforge3\envs\anaconda-llvmlite-py311-dev\Lib\site-packages\llvmlite\tests\__main__.py", line 3 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main

rather than the access violation. At some point there is some invalid IR - perhaps a block is missing a terminator, e.g. as in https://stackoverflow.com/questions/43255774/llvm-what-does-nodeptr-isknownsentinel-function-operator-means.

[gmarkall]

Noting that further to the previous comment, setting the -print-after-all option seems to make the crash 100% reliable when running the test suite.

[gmarkall]

Also - the access violation occurs during the execution of one of the passes when the ModulePassManager is running, at https://github.com/llvm/llvm-project/blob/release/15.x/llvm/include/llvm/IR/PassManager.h#L522 (called from LLVMPY_RunNewModulePassManager())