SVE Backend

[jeremylt]

@MoMo569377793 we were working on an SVE backend but it appears you got much further. Would you be interested in working with us to upstream that effort in your fork to libCEED's main branch?

[MoMo569377793]

Hi @jeremylt, thanks for reaching out. I’d be very glad to work with you on upstreaming an SVE backend.

To be upfront: what is currently in my fork is an experimental prototype, partly AI-assisted, and not something I would want to push as-is. I’d like to rewrite and review it carefully into a version that can be maintained long-term, and I’d prefer to align with the design you already have in mind rather than impose my current layout.

For context, my current research focus is moving toward SME-related measurement and optimization work, so my time on the SVE effort may be a little uneven over the next while. But SVE is still important to me as a baseline and fallback path, and I’d be happy to proceed step by step, starting with a small draft/RFC PR that matches libCEED’s standards.

I’d appreciate any guidance on how you would like the work split, what tests you expect, and whether you already have a preferred backend structure or branch to build from. Looking forward to working with you all.

[jeremylt]

Sorry for the delay - In the middle of an international move and libCEED is a side project for me now

I think there are a key few pieces

1. The backend itself

The backend itself is largely the contents of backends/sve and backends/sme built to match backends/avx as well as the files backends/ceed-backend-list-sve.h and backends/weak/ceed-sve-weak.c. That should largely align with what you hard, if I recall correctly? A potential wrinkle here is the fact that sometimes a CeedScalar is a double, and sometimes a float, determined at compile time. It would be very nice to keep that capacity. (That capacity is why the AVX backend looks a bit strange perhaps, with all of those macros.)

2. The Makefile changes

The Makefile should detect if the compiler supports SVE and SME and automatically compile the contents of backends/sve and backends/sme if so. The check would be similar to

libCEED/Makefile

Line 532 in d05a68c

AVX_FLAG := $(if $(filter clang,$(CC_VENDOR)),+avx,-mavx)

, but I am not sure how that would work for SVE and SME. If you can help with writing that one (two? Dunno if SVE support implies SME support, but I think SME support definitely implies SVE support) line, I can help with the rest of the Makefile changes if that would be helpful.

3. CI testing

This is the trickiest piece, but I think I know the path on this. However, it will be lots of trial and error. The existing test suite should be adequate to verify if the new backends work as intended, but we need to hook them into CI so we get testing on pushes and PRs. If you have a PR in place that has the tests passing for /cpu/self/[sve,sme]/[serial,blocked] working, then I can help with this piece.

A note on AI - LLM usage is fine to help you do the work. Just as a heads up, our policy is that the developer is ultimately the one responsible for their own code and you need to declare any LLM usage for transparency and to improve our ability to review the code.

I'm jet lagged and thinking in 2 languages right now, so let me know if I can be clearer about any of this :) Given LineShine's recent performance, I'm excited to see if we can get libCEED running on systems that support SVE/SME (and compare to LIBXSMM on such systems).

[MoMo569377793]

Sorry for the delay - In the middle of an international move and libCEED is a side project for me now

I think there are a key few pieces

1. The backend itself

The backend itself is largely the contents of backends/sve and backends/sme built to match backends/avx as well as the files backends/ceed-backend-list-sve.h and backends/weak/ceed-sve-weak.c. That should largely align with what you hard, if I recall correctly? A potential wrinkle here is the fact that sometimes a CeedScalar is a double, and sometimes a float, determined at compile time. It would be very nice to keep that capacity. (That capacity is why the AVX backend looks a bit strange perhaps, with all of those macros.)

2. The Makefile changes

The Makefile should detect if the compiler supports SVE and SME and automatically compile the contents of backends/sve and backends/sme if so. The check would be similar to

libCEED/Makefile

Line 532 in d05a68c

AVX_FLAG := $(if $(filter clang,$(CC_VENDOR)),+avx,-mavx)
, but I am not sure how that would work for SVE and SME. If you can help with writing that one (two? Dunno if SVE support implies SME support, but I think SME support definitely implies SVE support) line, I can help with the rest of the Makefile changes if that would be helpful.
3. CI testing

This is the trickiest piece, but I think I know the path on this. However, it will be lots of trial and error. The existing test suite should be adequate to verify if the new backends work as intended, but we need to hook them into CI so we get testing on pushes and PRs. If you have a PR in place that has the tests passing for /cpu/self/[sve,sme]/[serial,blocked] working, then I can help with this piece.

A note on AI - LLM usage is fine to help you do the work. Just as a heads up, our policy is that the developer is ultimately the one responsible for their own code and you need to declare any LLM usage for transparency and to improve our ability to review the code.

I'm jet lagged and thinking in 2 languages right now, so let me know if I can be clearer about any of this :) Given LineShine's recent performance, I'm excited to see if we can get libCEED running on systems that support SVE/SME (and compare to LIBXSMM on such systems).

No worries at all, and thanks for the detailed breakdown.

This plan works for me. I propose starting with SVE only, then treating SME as a follow-up after I can validate it on real SME hardware. That should let us settle the backend layout, Makefile detection, and test path on the available path first. I’ll keep the compile-time CeedScalar float/double support.

On AI/DCO: the code currently in my fork was LLM-assisted prototyping, so I will not submit it as-is. For upstream I’ll review/rewrite the SVE backend into code I can take responsibility for, sign off via DCO, and disclose LLM usage in the PR.

A couple of alignment questions before I start the WIP PR:

1. I noticed jeremy/sve currently uses split ceed-sve-tensor-f32.c / -f64.c, thin serial/blocked delegators, and registration through the main backend list/weak files, while your note mentions ceed-backend-list-sve.h and weak/ceed-sve-weak.c. Which layout should I target? Should I base on jeremy/sve, or wait for the no-weak direction?
2. I can take the SVE Makefile detection work. My inclination is to keep SVE and SME as separate feature checks, since SVE-only systems exist and SME should not gate SVE. For the first PR I can focus on the SVE path and leave SME detection/hooks for the follow-up unless you prefer otherwise. I can mirror the AVX-style verbose-output filter or use small try-compile probes; let me know which style you prefer.

I’ll aim first for a WIP/RFC PR with /cpu/self/sve/{serial,blocked} passing the relevant libCEED tests. Once that is in place, it would be ideal if you could lead the CI wiring, and I can iterate on whatever changes are needed from the CI/test results. SME can follow once the SVE path is settled and I have hardware validation.

[jeremylt]

The branch jeremy/sve was pretty out of date. I updated it to match our current weak backend registration (aside, that was an interesting research dive to fix building/linking issues on Apple devices). Hopefully the contents of that branch are less confusing now.

I don't personally have a strong preference between handling the tensor contraction object in two files (ceed-sve-tensor-f32.c and -f64.c) or a single one (ceed-sve-tensor.c). I know that's one area where jeremy/sve still differs from the AVX backend on main.

RE verbose output filter vs try-compile - I don't personally have a preference. I generally prefer similarity between backends so maintainers have fewer differences to understand, but robustness is the first priority here. I have in the jeremy/sve branch my first attempt, but I don't know if its accurate as we never got very far along on that work before I had to switch tasks. @jedbrown may have more strong preferences than I for this

Starting with SVE only and then doing SME as a follow-up makes good sense to me.

Heads up - I'm starting at a new research center tomorrow, so my comms on this may be mostly on the weekends going forward

Thanks!