[Bazel Migration] Master Tracking Issue (Make parity)

[napetrov]

Overview

This is the master tracking issue for the Make → Bazel migration in oneDAL. The goal is to achieve full feature parity with the existing Make build system while leveraging Bazel's advantages for hermetic builds, caching, and scalability.

Migration Goals

1. Full Make Parity: All build targets, configurations, and platforms supported
2. No Regressions: Performance, binary size, and API compatibility maintained
3. Hermetic Builds: Reproducible builds with explicit dependencies
4. Better CI/CD: Faster incremental builds, better caching, parallel execution
5. Maintainability: Clearer dependency graph, easier to extend

Current Status

Completed

• ✅ PR feat: bazel 9.0 migration #3496: Bazel 9.0 migration (merged)
• ✅ PR fix: correct MKL strip_prefixes for conda packages in MODULE.bazel #3518: MKL strip_prefixes fix for conda packages in MODULE.bazel (merged)

In Progress (Open PRs)

• ✅ PR bazel: hide MKL symbols in shared libraries #3519: MKL symbol visibility — hide MKL symbols in shared libraries (merged)
• 🔄 PR bazel: add release structure improvements (versioning, SONAME, vars.sh, pkg-config) #3513: Release structure (versioning, SONAME, vars.sh, pkg-config) (open)
• 🔄 PR bazel: fix MKL embedding — static lib must not contain MKL symbols #3532: MKL static lib embedding fix — libonedal_core.a no longer contains MKL (open, in review)
• ✅ PR bazel: fix symbol visibility parity with Make build #3534: Symbol visibility parity (-fvisibility=hidden, __DAAL_IMPLEMENTATION, __ONEDAL_ENABLE_EXPORT__) (merged)
• 🔄 PR bazel: release target auto-compiles all ISAs and DPC++ by default #3537: ISA coverage via cfg transition on //:release target (open)
• 🔄 PR bazel: add debug/sanitizer configs and expand build documentation #3538: Debug/sanitizer configs + build documentation (dev/bazel/README.md) (open)
• 🔄 Platform verification (Windows, macOS, ARM)

Critical Blockers

BLOCKER 1: MKL Linkage Strategy Enforcement ✅ Resolved

Problem: Bazel build needed to:

• Embed MKL in dynamic libs and hide symbols (--exclude-libs)
• Keep static libs (.a) MKL-free (user provides at link time)

Solution (PR #3519 + PR #3532):

• Dynamic lib: alwayslink=True on MKL + --exclude-libs hides symbols → 0 exported MKL symbols
• Static lib: new mkl_embed lib_tag separates MKL from static link path → no MKL in .a

Result:

Artifact	Before	After
libonedal_core.a	~750 MB (MKL merged ❌)	~271 MB (no MKL ✅)
libonedal_core.so	MKL symbols exported ❌	0 exported MKL symbols ✅

---

BLOCKER 2: Static Artifact Size Bloat

Problem: Bazel-built static libraries are 2× larger than Make equivalents.

Example (libonedal_core.a):

• Make: ~50 MB
• Bazel: ~100 MB

Root Cause: MKL static libs pulled in without symbol pruning. Make build uses partial linking + LTO to strip unused symbols.

Impact:

• Larger distribution packages
• Slower link times for downstream users
• Higher storage/bandwidth costs

Solution Required:

• Implement LTO in Bazel build (test -flto)
• Consider linker garbage collection (-Wl,--gc-sections)
• Benchmark size vs Make baseline

Status: Deferred to Phase 1 (accept bloat for now, optimize later)

---

BLOCKER 3: ICX/DPC++ Toolchain Consolidation

Problem: Make build uses different compilers for different targets:

• icx for CPU-only code
• icpx (DPC++) for SYCL code

Bazel currently uses single toolchain. Need to verify:

• SYCL device compilation works correctly
• CPU code doesn't get SYCL overhead
• Multi-ISA dispatch compatibility

Impact: Potential performance regression or SYCL build failures

Solution Required:

• Test DPC++ build on onedal-build server
• Verify AOT (Ahead-of-Time) compilation for GPU targets
• Benchmark vs Make baseline

Status: Not yet tested on Bazel build

---

BLOCKER 4: Windows MSVC Symbol Hiding

Problem: Windows doesn't support --exclude-libs. Make build uses:

• DEF files to control exports
• Import libraries for MKL TBB threading
• Different linker flags (/WHOLEARCHIVE, /DEF)

Bazel Linux build uses --exclude-libs which doesn't work on Windows.

Impact: MKL symbols leak on Windows, breaking ABI compatibility

Solution Required:

• Generate DEF files listing only oneDAL exports
• Use __declspec(dllexport) attributes (preferred)
• Test MSVC toolchain integration

Status: Design phase, Windows builds not yet tested

---

Secondary Issues (Post-Blocker)

Release Layout Parity

Make build produces specific directory structure:

__release_lnx/
  daal/latest/
    lib/
    include/
    env/vars.sh

Bazel currently outputs flat structure. Need to match for compatibility with existing installers/scripts.

---

Multi-ISA Verification

Make build supports runtime CPU dispatch (SSE2, AVX2, AVX512). Need to verify:

• Bazel preserves all ISA variants
• Dispatch logic works correctly
• No regressions in multi-ISA performance

---

RISC-V/ARM + OpenBLAS Fallback

Make build has experimental RISC-V support with OpenBLAS (when MKL unavailable). Bazel migration should preserve this path.

---

Build Time Benchmarking

Need to measure:

• Clean build time (Make vs Bazel)
• Incremental rebuild time
• Cache hit rates

Target: Bazel should be faster on incremental builds.

---

Library Versioning & Packaging

Make build generates:

• pkg-config files
• Environment setup scripts (vars.sh, vars.bat)
• Versioned symlinks (libonedal.so.1, libonedal.so.1.2.3)

Bazel migration must preserve these for downstream compatibility.

---

ARM Platform Support

Make build supports ARM64 (Linux, macOS). Need to test Bazel cross-compilation and native builds.

---

---

BLOCKER 5: Build Options Parity (debug symbols, sanitizers, optimization levels, custom flags)

Raised by: @david-cortes-intel

Problem: Make build supports a range of build-time options that must be available in Bazel:

• Debug symbols: -g / -g0 control (debug vs release builds)
• Sanitizers: AddressSanitizer (-fsanitize=address), UBSan, ThreadSanitizer — used in CI and developer testing
• Optimization levels: -O0 / -O2 / -O3 / custom per-target
• Custom $CXXFLAGS: Make allows injecting arbitrary compiler flags via environment; Bazel needs an equivalent escape hatch

Impact: Without parity here, developers cannot reproduce Make debug/sanitizer builds in Bazel. This is a blocker for fully replacing Make in developer workflows.

Solution Required:

• Audit all Make build options (--mode=debug, --with-sanitizer, etc.)
• Map each to Bazel equivalents (--compilation_mode=dbg, --features=asan, custom copts)
• Expose user-facing flags via config_setting or .bazelrc configs
• Document the mapping in Bazel build docs

Status: Implemented in PR #3538 (Pending Review)

---

BLOCKER 6: Bazel Build Documentation

Raised by: @david-cortes-intel

Problem: There is no comprehensive documentation covering:

• How to build oneDAL with Bazel (basic usage)
• Available build configurations and flags
• Platform-specific notes (Linux, Windows, macOS)
• Migration guide: Make flag → Bazel equivalent
• How to integrate the Bazel build in downstream projects

Impact: Without documentation, contributors and downstream consumers cannot adopt the Bazel build system even if it is technically complete. This is a blocker for deprecating Make.

Solution Required:

• Create docs/bazel-build.md (or equivalent) covering:
  • Prerequisites and setup
  • Common build commands
  • Configuration flags reference
  • CI integration guide
  • Troubleshooting

Status: Implemented in PR #3538 (Pending Review)

---

---

Per-Algorithm Build Selection

Equivalent to: Make CORE.ALGORITHMS.CUSTOM=pca kmeans svm ...

Problem: Make supports building the library with only a specific subset of
algorithms (CORE.ALGORITHMS.CUSTOM), producing a smaller .so/.a. This is
useful for distribution packages targeting specific workloads.

In Bazel today:

• Building/testing individual algorithms works natively:
  bazel test //cpp/oneapi/dal/algo/svm:tests ✅
• But //:release always includes all algorithms — no filtering ❌

Solution Required:

• Add --algorithms flag (Bazel string_list_flag) to config.tpl.BUILD
• Filter srcs in daal.bzl / dal.bzl using select() based on the flag
• //:release respects --algorithms to produce a trimmed library

Status: Deferred to Phase 1

Testing Strategy

1. Unit Testing: Existing tests must pass on Bazel builds
2. Binary Compatibility: Symbol exports match Make builds
3. Performance Parity: Benchmarks show no regression
4. Platform Matrix: Linux, Windows, macOS × x86_64, ARM64
5. Integration Testing: Downstream consumers (scikit-learn-intelex, daal4py)

---

Open PRs

PR	Title	Status

| #3513 | Release structure (versioning, SONAME, vars.sh, pkg-config) | 🔄 Open |
| #3532 | MKL static lib embedding fix — no MKL in .a | 🔄 Open, in review |

| #3537 | ISA coverage via cfg transition on //:release | 🔄 Open |
| #3538 | Debug/sanitizer configs + build documentation | 🔄 Open |

Merged PRs

PR	Title
#3496	Bazel 9.0 migration ✅
#3518	MKL strip_prefixes fix for conda (MODULE.bazel) ✅
#3519	MKL symbol visibility — hide MKL symbols in shared libs ✅
#3534	Symbol visibility parity (-fvisibility=hidden) ✅

[david-cortes-intel]

@napetrov What about build options like adding or not adding debugging symbols, linkage to sanitizers, optimization levels, custom $CXXFLAGS, and so on?

Also full documentation about how to use the bazel builds should be another blocker point.