You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[](https://github.com/dragonflyoss/image-service/actions/workflows/release.yml)
The nydus project implements a content-addressable filesystem on top of a RAFS format that improves the current OCI image specification, in terms of container launching speed, image space, and network bandwidth efficiency, as well as data integrity.
22
+
Nydus implements a content-addressable file system on the RAFS format, which enhances the current OCI image specification by improving container launch speed, image space and network bandwidth efficiency, and data integrity.
16
23
17
-
The following benchmarking result shows the performance improvement compared with the OCI image for the container cold startup elapsed time on containerd. As the OCI image size increases, the container startup time of using Nydus image remains very short.
24
+
The following Benchmarking results demonstrate that Nydus images significantly outperform OCI images in terms of container cold startup elapsed time on Containerd, particularly as the OCI image size increases.
18
25
19
26
20
27
21
-
Nydus' key features include:
28
+
## Principles
29
+
30
+
***Provide Fast, Secure And Easy Access to Data Distribution***
22
31
23
-
- Container images can be downloaded on demand in chunks for lazy pulling to boost container startup
24
-
- Chunk-based content-addressable data de-duplication to minimize storage, transmission and memory footprints
25
-
- Merged filesystem tree in order to remove all intermediate layers as an option
26
-
- in-kernel EROFS or FUSE filesystem together with overlayfs to provide full POSIX compatibility
27
-
- E2E image data integrity check. So security issues like "Supply Chain Attach" can be avoided and detected at runtime
28
-
- Compatible with the OCI artifacts spec and distribution spec, so nydus image can be stored in a regular container registry
29
-
- Native [eStargz](https://github.com/containerd/stargz-snapshotter) image support with remote snapshotter plugin `nydus-snapshotter` for containerd runtime.
30
-
- Various container image storage backends are supported. For example, Registry, NAS, Aliyun/OSS, S3.
31
-
- Integrated with CNCF incubating project Dragonfly to distribute container images in P2P fashion and mitigate the pressure on container registries
32
-
- Capable to prefetch data block before user IO hits the block thus to reduce read latency
33
-
- Record files access pattern during runtime gathering access trace/log, by which user abnormal behaviors are easily caught
34
-
- Access trace based prefetch table
35
-
- User I/O amplification to reduce the amount of small requests to storage backend.
-**Low Cost**: Written in memory-safed language `Rust`, numerous optimizations help improve memory, CPU, and network consumption.
34
+
-**Flexible**: Supports container runtimes such as [runC](https://github.com/opencontainers/runc) and [Kata](https://github.com/kata-containers), and provides [Confidential Containers](https://github.com/confidential-containers) and vulnerability scanning capabilities
35
+
-**Security**: End to end data integrity check, Supply Chain Attack can be detected and avoided at runtime.
36
36
37
-
Currently Nydus includes following tools:
37
+
## Key features
38
+
39
+
-**On-demand Load**: Container images/packages are downloaded on-demand in chunk unit to boost startup.
40
+
-**Chunk Deduplication**: Chunk level data de-duplication cross-layer or cross-image to reduce storage, transport, and memory cost.
41
+
-**Compatible with Ecosystem**: Storage backend support with Registry, OSS, NAS, Shared Disk, and [P2P service](https://d7y.io/). Compatible with the [OCI images](https://github.com/dragonflyoss/image-service/blob/master/docs/nydus-zran.md), and provide native [eStargz images](https://github.com/containerd/stargz-snapshotter) support.
42
+
-**Data Analyzability**: Record accesses, data layout optimization, prefetch, IO amplification, abnormal behavior detection.
43
+
-**POSIX Compatibility**: In-Kernel EROFS or FUSE filesystems together with overlayfs provide full POSIX compatibility
44
+
-**I/O optimization**: Use merged filesystem tree, data prefetching and User I/O amplification to reduce read latency and improve user I/O performance.
@@ -47,30 +57,25 @@ Currently Nydus includes following tools:
47
57
|[nydus-overlayfs](https://github.com/dragonflyoss/image-service/tree/master/contrib/nydus-overlayfs)|`Containerd` mount helper to invoke overlayfs mount with tweaking mount options a bit. So nydus prerequisites can be passed to vm-based runtime |
48
58
|[nydus-backend-proxy](./contrib/nydus-backend-proxy/README.md)| A simple HTTP server to serve local directory as a blob backend for nydusd |
49
59
50
-
Currently Nydus is supporting the following platforms in container ecosystem:
| Storage | Registry/OSS/S3/NAS | Support for OCI-compatible distribution implementations such as Docker Hub, Harbor, Github GHCR, Aliyun ACR, NAS, and Aliyun OSS-like object storage service | ✅ |
55
65
| Storage/Build |[Harbor](https://github.com/goharbor/acceleration-service)| Provides a general service for Harbor to support acceleration image conversion based on kinds of accelerator like Nydus and eStargz etc | ✅ |
56
66
| Distribution |[Dragonfly](https://github.com/dragonflyoss/Dragonfly2)| Improve the runtime performance of Nydus image even further with the Dragonfly P2P data distribution system | ✅ |
57
67
| Build |[Buildkit](https://github.com/moby/buildkit/blob/master/docs/nydus.md)| Provides the ability to build and export Nydus images directly from Dockerfile | ✅ |
68
+
| Build/Runtime |[Nerdctl](https://github.com/containerd/nerdctl/blob/master/docs/nydus.md)| The containerd client to build or run (requires nydus snapshotter) Nydus image | ✅ |
69
+
| Runtime |[Docker / Moby](https://github.com/dragonflyoss/image-service/blob/master/docs/docker-env-setup.md)| Run Nydus image in Docker container with containerd and nydus-snapshotter | ✅ |
58
70
| Runtime |[Kubernetes](https://github.com/containerd/nydus-snapshotter/blob/main/docs/run_nydus_in_kubernetes.md)| Run Nydus image using CRI interface | ✅ |
59
71
| Runtime |[Containerd](https://github.com/containerd/nydus-snapshotter)| Nydus Snapshotter, a containerd remote plugin to run Nydus image | ✅ |
60
72
| Runtime |[CRI-O / Podman](https://github.com/containers/nydus-storage-plugin)| Run Nydus image with CRI-O or Podman | 🚧 |
61
-
| Runtime |[Docker / Moby](https://github.com/dragonflyoss/image-service/blob/master/docs/docker-env-setup.md)| Run Nydus image in Docker container with containerd and nydus-snapshotter | ✅ |
62
-
| Build/Runtime |[Nerdctl](https://github.com/containerd/nerdctl/blob/master/docs/nydus.md)| The containerd client to build or run (requires nydus snapshotter) Nydus image | ✅ |
63
73
| Runtime |[KataContainers](https://github.com/kata-containers/kata-containers/blob/main/docs/design/kata-nydus-design.md)| Run Nydus image in KataContainers as a native solution | ✅ |
64
74
| Runtime |[EROFS](https://www.kernel.org/doc/html/latest/filesystems/erofs.html)| Run Nydus image directly in-kernel EROFS for even greater performance improvement | ✅ |
65
75
66
-
To try nydus image service:
67
-
68
-
1. Convert an original OCI image to nydus image and store it somewhere like Docker/Registry, NAS, Aliyun/OSS or S3. This can be directly done by `nydusify`. Normal users don't have to get involved with `nydus-image`.
69
-
2. Get `nydus-snapshotter`(`containerd-nydus-grpc`) installed locally and configured properly. Or install `nydus-docker-graphdriver` plugin.
70
-
3. Operate container in legacy approaches. For example, `docker`, `nerdctl`, `crictl` and `ctr`.
71
-
72
-
## Build Binary
76
+
## Build
73
77
78
+
### Build Binary
74
79
```shell
75
80
# build debug binary
76
81
make
@@ -80,88 +85,80 @@ make release
80
85
make docker-static
81
86
```
82
87
83
-
##Quick Start with Kubernetes and Containerd
88
+
### Build Nydus Image
84
89
85
-
For more details on how to lazily start a container with `nydus-snapshotter` and nydus image on Kubernetes nodes or locally use `nerdctl` rather than CRI, please refer to [Nydus Setup](./docs/containerd-env-setup.md)
90
+
Convert OCIv1 image to Nydus image: [Nydusify](./docs/nydusify.md), [Acceld](https://github.com/goharbor/acceleration-service)or [Nerdctl](https://github.com/containerd/nerdctl/blob/master/docs/nydus.md#build-nydus-image-using-nerdctl-image-convert).
86
91
87
-
## Build Nydus Image
92
+
Build Nydus image from Dockerfile directly: [Buildkit](https://github.com/moby/buildkit/blob/master/docs/nydus.md).
88
93
89
-
Build Nydus image from directory source: [Nydus Image Builder](./docs/nydus-image.md).
94
+
Build Nydus layer from various sources: [Nydus Image Builder](./docs/nydus-image.md).
90
95
91
-
Convert OCIv1 image to Nydus image: [Nydusify](./docs/nydusify.md), [Acceld](https://github.com/goharbor/acceleration-service) or [Nerdctl](https://github.com/containerd/nerdctl/blob/master/docs/nydus.md#build-nydus-image-using-nerdctl-image-convert).
96
+
#### Image prefetch optimization
97
+
To further reduce container startup time, a nydus image with a prefetch list can be built using the NRI plugin (containerd >=1.7): [Container Image Optimizer](https://github.com/containerd/nydus-snapshotter/blob/main/docs/optimize_nydus_image.md)
92
98
93
-
Optionally, a containerd(>=1.7) NRI plugin - [container image optimizer](https://github.com/containerd/nydus-snapshotter/blob/main/docs/optimize_nydus_image.md) - can be leveraged to build an optimized nydus image which will reduce the container startup time further.
99
+
## Run
100
+
### Quick Start
94
101
95
-
## Nydus Snapshotter
102
+
For more details on how to lazily start a container with `nydus-snapshotter` and nydus image on Kubernetes nodes or locally use `nerdctl` rather than CRI, please refer to [Nydus Setup](./docs/containerd-env-setup.md)
103
+
104
+
### Run Nydus Snapshotter
96
105
97
106
Nydus-snapshotter is a non-core sub-project of containerd.
98
107
99
108
Check out its code and tutorial from [Nydus-snapshotter repository](https://github.com/containerd/nydus-snapshotter).
100
109
It works as a `containerd` remote snapshotter to help setup container rootfs with nydus images, which handles nydus image format when necessary. When running without nydus images, it is identical to the containerd's builtin overlayfs snapshotter.
101
110
102
-
## Run Nydusd Daemon
111
+
###Run Nydusd Daemon
103
112
104
113
Normally, users do not need to start `nydusd` by hand. It is started by `nydus-snapshotter` when a container rootfs is prepared.
105
114
106
115
Run Nydusd Daemon to serve Nydus image: [Nydusd](./docs/nydusd.md).
107
116
108
-
## Run Nydus with in-kernel EROFS filesystem
117
+
###Run Nydus with in-kernel EROFS filesystem
109
118
110
119
In-kernel EROFS has been fully compatible with RAFS v6 image format since Linux 5.16. In other words, uncompressed RAFS v6 images can be mounted over block devices since then.
111
120
112
121
Since [Linux 5.19](https://lwn.net/Articles/896140), EROFS has added a new file-based caching (fscache) backend. In this way, compressed RAFS v6 images can be mounted directly with fscache subsystem, even such images are partially available. `estargz` can be converted on the fly and mounted in this way too.
113
122
114
123
Guide to running Nydus with fscache: [Nydus-fscache](./docs/nydus-fscache.md)
115
124
116
-
## Run Nydus with Dragonfly P2P system
125
+
###Run Nydus with Dragonfly P2P system
117
126
118
127
Nydus is deeply integrated with [Dragonfly](https://d7y.io/) P2P system, which can greatly reduce the network latency and the single point of network pressure for registry server, testing in the production environment shows that using Dragonfly can reduce network latency by more than 80%, to understand the performance test data and how to configure Nydus to use Dragonfly, please refer to the [doc](https://d7y.io/docs/setup/integration/nydus).
119
128
120
-
##Accelerate OCI image directly with Nydus
129
+
### Run OCI image directly with Nydus
121
130
122
131
Nydus is able to generate a tiny artifact called a `nydus zran` from an existing OCI image in the short time. This artifact can be used to accelerate the container boot time without the need for a full image conversion. For more information, please see the [documentation](./docs/nydus-zran.md).
123
132
124
-
## Build Images via Harbor
125
-
126
-
Nydus cooperates with Harbor community to develop [acceleration-service](https://github.com/goharbor/acceleration-service) which provides a general service for Harbor to support image acceleration based on kinds of accelerators like Nydus, eStargz, etc.
127
-
128
-
## Run with Docker
133
+
### Run with Docker(Moby)
129
134
130
-
A **experimental** plugin helps to start Docker container from nydus image. For more particular instructions, please refer to [Docker Nydus Graph Driver](https://github.com/nydusaccelerator/docker-nydus-graphdriver)
135
+
Nydus provides a variety of methods to support running on docker(Moby), please refer to [Nydus Setup for Docker(Moby) Environment](./docs/docker-env-setup.md)
131
136
132
-
## Run with macOS
137
+
###Run with macOS
133
138
134
-
Nydus can also run with macfuse(a.k.a osxfuse).For more details please read [nydus with macOS](./docs/nydus_with_macos.md).
139
+
Nydus can also run with macfuse(a.k.a osxfuse).For more details please read [nydus with macOS](./docs/nydus_with_macos.md).
135
140
136
-
## Run eStargz image (with lazy pulling)
141
+
###Run eStargz image (with lazy pulling)
137
142
138
143
The containerd remote snapshotter plugin [nydus-snapshotter](https://github.com/containerd/nydus-snapshotter) can be used to run nydus images, or to run [eStargz](https://github.com/containerd/stargz-snapshotter) images directly by appending `--enable-stargz` command line option.
139
144
140
145
In the future, `zstd::chunked` can work in this way as well.
141
146
142
-
##Reuse Nydus Services
147
+
### Run Nydus Service
143
148
144
149
Using the key features of nydus as native in your project without preparing and invoking `nydusd` deliberately, [nydus-service](./service/README.md) helps to reuse the core services of nyuds.
145
150
146
-
147
151
## Documentation
148
152
149
-
Browse the documentation to learn more. Here are some topics you may be interested in:
150
-
151
-
-[A Nydus Tutorial for Beginners](./docs/tutorial.md)
152
-
-[Nydus Design Doc](./docs/nydus-design.md)
153
-
- Our talk on Open Infra Summit 2020: [Toward Next Generation Container Image](https://drive.google.com/file/d/1LRfLUkNxShxxWU7SKjc_50U0N9ZnGIdV/view)
154
-
-[EROFS, What Are We Doing Now For Containers?](https://static.sched.com/hosted_files/kccncosschn21/fd/EROFS_What_Are_We_Doing_Now_For_Containers.pdf)
155
-
-[The Evolution of the Nydus Image Acceleration](https://d7y.io/blog/2022/06/06/evolution-of-nydus/)\([Video](https://youtu.be/yr6CB1JN1xg)\)
156
-
-[Introduction to Nydus Image Service on In-kernel EROFS](https://static.sched.com/hosted_files/osseu2022/59/Introduction%20to%20Nydus%20Image%20Service%20on%20In-kernel%20EROFS.pdf)\([Video](https://youtu.be/2Uog-y2Gcus)\)
153
+
Please visit [**Wiki**](https://github.com/dragonflyoss/image-service/wiki), or [**docs**](./docs)
157
154
158
155
## Community
159
156
160
157
Nydus aims to form a **vendor-neutral opensource** image distribution solution to all communities.
161
158
Questions, bug reports, technical discussion, feature requests and contribution are always welcomed!
162
159
163
160
We're very pleased to hear your use cases any time.
164
-
Feel free to reach/join us via Slack and/or Dingtalk.
0 commit comments