ADIOS2 is an adaptable, scalable, and unified framework for scientific data I/O. It transports self-describing variables and attributes across files, networks, and memory using a single API, and is used in production on the world's largest supercomputers.
- Unified I/O API: same interface for files, wide-area networks, and in-memory staging; switch transport without changing application code
- BP format: self-describing binary-packed format (
.bp) with rapid metadata extraction and built-in compression support - MPI-native: scales from laptop to petabyte workloads on the largest HPC systems; serial use also supported
- Streaming and step-based: asynchronous, grouped variable transport with an explicit steps abstraction for in-situ and in-transit workflows
- Multiple language bindings: C++17, Python, C99, Fortran 90, and Matlab
- Compression operators: ZFP, SZ3, MGARD, Blosc2, PNG, and others
- Encryption operators: symmetric (AES-256-GCM) and asymmetric encryption available for data at rest
- In-situ/in-transit engines: SST and DataMan engines for code coupling over RDMA, MPI, and TCP without touching files
- DAOS engine: writes and reads datasets through the DAOS data plane (native object API) directly, with no dfuse or POSIX interception required
- Plugin support: extend ADIOS2 with custom engines and operators via shared-library plugins at runtime
- Security-focused: committed to OpenSSF best practices; see the OpenSSF Scorecard and Best Practices badge
- Widely available: packaged for major Linux distributions, Conda, Spack, Homebrew, and vcpkg; validated on U.S. DoE supercomputers including Frontier, Summit, and Perlmutter
- HDF5 VOL: existing HDF5 applications can use ADIOS2 as a backend transparently via the HDF5 Virtual Object Layer
import adios2
import numpy as np
data = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
# Write to a BP file
with adios2.Stream("data.bp", "w") as f:
f.write("data", data)
# Read it back
with adios2.Stream("data.bp", "r") as f:
for _ in f.steps():
result = f.read("data")
print(result) # [1. 2. 3. 4. 5.]Python (PyPI):
pip install adios2From source: See the install documentation and scripts/runconf/runconf.sh for a CMake configuration example. Once installed, refer to Linking ADIOS2.
| Platform | Package |
|---|---|
| Summary | |
| Conda |  |
| Spack |  |
| Homebrew |  |
| Ubuntu 24.04 |  |
| Ubuntu 26.04 |  |
| Debian Unstable |  |
| Arch |  |
| OpenSUSE TumbleWeed |  |
| Nix unstable |  |
| vcpkg |  |
| Dockerhub |  |
ADIOS2 has a wide spectrum of optional features, from a small, dependency-free core to specialized backends that pull in heavy or uncommon third-party libraries. It is also a research project, so those backends come and go, and the sites that rely on them usually build from source for each environment (or via Spack) rather than from a package manager. Therefore we recommend a lean build, not trying to satisfy all possible optional dependencies.
Core (no or trivial dependencies):
- BP3 / BP4 / BP5 file engines (
.bpformat), no external dependencies - SST streaming engine (TCP by default)
- C and C++ bindings
Common options (widely packaged):
MPI, recommended but not requiredPython,FortranbindingsHDF5(interop and the HDF5 VOL)- Compression:
ZFP,SZ3,BZip2,Blosc2,PNG Campaign(requires SQLite3 and ZLIB, both lightweight and commonly available)CURL(HTTPS remote data access)
Specialized (of interest only to specific workflows):
- Additional compression / refactoring:
MGARD,SZ(for existing data only),LibPressio,BigWhoop,ProDM - GPU:
CUDA,Kokkos - Encryption:
Sodium(libsodium; backs both the EncryptionOperator and encrypted Campaign metadata) - Remote / storage backends:
XRootD,OpenSSL(TLS for HTTPS remote access),AWSSDK(S3),DAOS - In-situ visualization:
Catalyst(ParaView)
Legacy engines and transports (unmaintained, off by default):
DataMan,SSC,MHS,DataSpaces,IME
These older staging and coupling engines, and the DDN IME transport, are no longer actively
maintained and are candidates for future removal. They are disabled by default and must be enabled explicitly (for
example -DADIOS2_USE_MHS=ON) if needed. ZeroMQ is required only by the DataMan engine and
is likewise off by default, so a default build pulls in no ZeroMQ dependency.
The SST engine reaches full performance through RDMA data planes that
depend on fabric libraries: libfabric (InfiniBand/verbs and
HPE-Cray Slingshot/CXI; detected automatically when SST is enabled),
UCX (ADIOS2_USE_UCX), and Mercury
(ADIOS2_USE_Mercury). These matter on systems with an RDMA
interconnect, where an HPC-targeted package should install one or more
of libfabric, UCX and Mercury before building so SST picks them up. On
machines without an RDMA network they provide no benefit may cause
trouble (SST will fallback to TCP and or MPI dataplanes if RDMA is not
directly available.) General-purpose packages can safely leave them
out.
A default configuration is already lean: the legacy engines above are off, and every other optional feature is enabled only when its dependency is already present. Enabling the common options is a matter of turning them on explicitly, for example:
cmake -DADIOS2_USE_HDF5=ON -DADIOS2_USE_Python=ON -DADIOS2_USE_MPI=ON ..Questions about whether a given feature is worth packaging are welcome on GitHub Discussions.
- Documentation: adios2.readthedocs.io
- Latest release: v2.12.1
- All releases: github.com/ornladios/ADIOS2/releases
- Citing: if you find ADIOS2 useful, please cite our SoftwareX paper
ADIOS2 is an open source project: questions, discussion, and contributions are welcome.
- Mailing list: adios-ecp@kitware.com
- Github Discussions: https://github.com/ornladios/ADIOS2/discussions
- Bug reports: open an issue
- Contributing: see the Contributor's Guide
ADIOS2 is developed as a multi-institutional collaboration between:
- Oak Ridge National Laboratory
- Kitware Inc.
- Lawrence Berkeley National Laboratory
- Georgia Institute of Technology
- Rutgers University
ADIOS2 is licensed under the Apache License v2.0. See the accompanying Copyright.txt for more details.