Nydus image acceleration plugin migrated to Containerd

Text｜Ge Changwei (flower name: Chuan Lang)

Ant Group Technical Expert

Responsible for the development and maintenance of the container image acceleration project Nydus, focusing on container image storage, persistent storage and file systems.

This article 1344 words read 4 minutes

foreword

In January of this year, the Containerd community voted to receive Nydus-snapshotter as a sub-project of the Containerd community. This is the ant container team donating sub-projects to Containerd again after ttrpc-rust.

This move will facilitate the development collaboration between Nydus and Containerd, reduce possible incompatibilities during project iteration, and allow users to more easily use the Nydus image acceleration service.

At present, Nydus has migrated the code of Nydus-snapshotter to a new repository under the Containerd organization [1].

Introduction to Nydus

Nydus is a container image acceleration project jointly open sourced by Ant Group and Alibaba Cloud. It belongs to the CNCF Dragonfly project and is part of the image service.

Nydus is a container image acceleration service designed on the basis of the latest OCI Image-Spec. It redesigns the image format and underlying file system to speed up container startup and improve the success rate of container startup in large-scale clusters.

Nydus designed a file system optimized for mirroring - Rafs.

Nydus images can be pushed and saved in the standard container image center. The Nydus image format is fully compatible with OCI Image Spec and Distribution Spec. After successfully converting or creating an image, the Nydus image will generate a metadata file Bootstrap, several data files blob, manifest.json, and config.json.

Currently, Nydus accelerated images can be created through Nydusify, Acceld or Buildkit.

Among them, Acceld[2] is a sub-project of the Harbor open source enterprise image center that the developers of Nydus and eStargz are developing together. It provides a general accelerated image conversion service and framework. Based on Acceld, Nydus and eStargz can easily trigger accelerated mirror conversion from Harbor.

At the same time, Nydus is also developing Buildkit-related support, and in the future, it is also possible to create accelerated images directly from the Dockerfile through Buildkit.

Nydus-snapshotter is a Remote Snapshotter plugin for Containerd, which is a separate process from Containerd.

After integrating Nydus-snapshotter into Containerd, Nydus-napshotter will only download the metadata part of the Nydus image Bootstrap from the image center during the container image preparation stage, and create a new process Nydusd. Nydusd is a user-mode process that handles file system operations. Through configuration, Nydusd can be used as a Linux FUSE-based user mode file system Virtio-fs Vhost-user Backend, or even a user mode process of Linux Fscache.

Nydusd is responsible for downloading file data from the mirror center or object storage in response to file read requests, and can cache file data blocks in the host's local file system.

Nydus Features

Nydus has the following important features:

1. The block-level data deduplication between mirror layers can reduce the storage cost of the mirror center and reduce the bandwidth consumption of data transmission.

2. Whiteout files will no longer be packaged into Nydus images.

3. End-to-end data integrity check.

4. As a sub-project of the CNCF incubation project Dragonfly, Nydus can access the P2P distribution system to reduce the pressure on the mirror center.

5. Support separate storage of data and metadata. Data can be stored in NAS, Alibaba Cloud OSS or AWS S3.

6. Support file access behavior record, so that you can audit and analyze the access behavior of applications in the container. Enhanced security capabilities and optimized mirrored data layout.

In addition to the above key features, Nydus can be flexibly configured as a Linux FUSE user mode file system, a Virtio-fs daemon based on lightweight virtualization technology containers, or a user mode on-demand data download service for the Linux kernel disk file system EROFS:

1. Lightweight integration into vm-based container runtime. Now KataContainers is considering natively supporting Nydus as a container image acceleration solution.

2. Nydus and EROFS work closely together and expect to use EROFS directly as a file system for container images. The first part of the relevant changes has been merged into Linux Kernel v5.16.

Nydus deployment form

When supporting Runc, Nydus acts as a FUSE user mode filesystem process:

When supporting KataContainers, Nydus acts as a Virtio-fs daemon:

At present, EROFS is trying to combine Fscache and use the kernel file system EROFS directly as the container Rootfs:

Nydus will work closely with the Containerd community to provide better container image acceleration solutions, improve image storage and distribution efficiency, and provide safe and reliable container image services.

"refer to"

[1] nydus-snapshotter repository:GitHub - containerd/nydus-snapshotter: A containerd snapshotter with capability of on-demand read.

[2] acceld repository:https://github.com/goharbor/acceleration-service

Thirst for talent:

About the Security Container and Storage Team of the Trusted Native Technology Department of Ant Group

In Ant Group, he is mainly responsible for the company's internal container runtime and cloud native storage technology. He is the guardian of the company's data link and the gatekeeper of the runtime environment. Our team is also the founder of Kata Containers, the initiator of the image acceleration service Nydus, the maintainer of the distributed transaction service Seata, and the company's data access components ZDal/ZCache/XTS and other products.

We are believers in the spirit of open source, and practitioners to achieve a win-win situation between open source software and the company's business. We are a team that pays attention to business, the forefront of the industry, infrastructure technology, and more concerned about the growth of members. We are currently recruiting 2023 interns. If you are interested, you can refer to Ant Group's 2023 Interns Recruitment.

Email: [e-mail]

Recommended reading of the week

Congratulations to Li Zhiqiang for becoming a Layotto committer!

Community Article | MOSN Routing Framework Detailed

HAVE FUN | SOFARegistry Source code analysis

BabaSSL: supports semi-homomorphic encryption algorithm EC-ElGamal