macOS wheel: libquantized_ops_aot_lib.dylib never loads, so PT2E models fail later with "Missing out variants"

[john-rocky]

Summary

On the macOS arm64 wheel, libquantized_ops_aot_lib.dylib cannot be loaded, so the quantized_decomposed out-variants are never registered. Nothing reports it — the load sits in a bare except: that logs at INFO — and the first sign is to_executorch() failing on a model with nothing obviously wrong with it.

executorch 1.4.0, torch 2.13.0, Python 3.12, macOS arm64, installed from the pip wheel. I have not checked the Linux wheels or a source build.

What you see

RuntimeError: Missing out variants: {'quantized_decomposed::dequantize_per_channel',
'quantized_decomposed::choose_qparams', 'quantized_decomposed::dequantize_per_tensor'}

It happens for any PT2E-quantized graph that leaves a q/dq node outside the delegate. Grounding DINO tiny is a good example of how little it takes: dynamic int8 quantises 350 of its 392 aten.linear nodes, XnnpackPartitioner takes 71.5% of the graph, and exactly three quantized ops end up on the portable path —

quantized_decomposed.dequantize_per_channel.default : 1
quantized_decomposed.choose_qparams.tensor          : 1
quantized_decomposed.dequantize_per_tensor.tensor   : 1

— the same three the error names. One linear out of 392 not taken by the delegate is enough to stop the export. Models that delegate everything never see this, which is why it can sit unnoticed.

Cause

$ python -c "import ctypes; ctypes.CDLL('.../executorch/kernels/quantized/libquantized_ops_aot_lib.dylib')"
OSError: dlopen(...libquantized_ops_aot_lib.dylib, 0x0006):
  Library not loaded: @rpath/_portable_lib.cpython-312-darwin.so
  Referenced from: .../executorch/kernels/quantized/libquantized_ops_aot_lib.dylib

otool -L confirms the reference. The @rpath does not resolve from executorch/kernels/quantized/, and executorch/kernels/quantized/__init__.py wraps torch.ops.load_library in

except:
    import logging

    logging.info("libquantized_ops_aot_lib is not loaded")

so the failure is invisible at the point where it happens.

Workaround

Put _portable_lib in the process first; the dylib then finds it by install name.

import torch
from executorch.extension.pybindings import portable_lib  # noqa: F401
from pathlib import Path
import executorch.kernels.quantized as q

torch.ops.load_library(str(next(Path(q.__file__).parent.glob("**/*quantized_ops_aot_lib.*"))))

After that torch.ops.quantized_decomposed carries choose_qparams, dequantize_per_channel and dequantize_per_tensor, and the same export completes — the Grounding DINO run above goes from the error to a 254 MB .pte with nothing else changed.

Suggestions

1. Give the dylib an rpath that reaches extension/pybindings, so it loads on its own.
2. Whatever the packaging does, do not swallow the exception. A warning naming the dylib would have turned a confusing Missing out variants into a one-line fix.

Happy to send a PR for either.