JuiceFS data encryption principle

As a distributed file system, JuiceFS deals with massive amounts of data every day, so data security is particularly critical. Today, I will introduce JuiceFS's efforts in data encryption.

Encryption of data in transit

JuiceFS encrypts data when transmitting on the network to prevent unauthorized users from eavesdropping on network communications.

The JuiceFS client always uses HTTPS to upload data to the object storage service, except in the following cases:

• Upload to Alibaba Cloud OSS using internal endpoint
• Upload to UCloud US3 using internal endpoint

Data at rest encryption

JuiceFS supports data encryption at rest, which means encrypting data before uploading to object storage. In this case, the data stored in the object storage will be encrypted, which can effectively prevent data leakage when the object storage itself is damaged.

JuiceFS uses industry standard encryption methods (AES-GCM and RSA) in client-side encryption. Encryption and decryption are performed on the JuiceFS client. The only thing the user needs to do is to provide a private key or password when JuiceFS is mounted, and use it like a normal file system. It is completely transparent to the application.

Note: The data cached on the client side is not encrypted. However, only the root user or owner can access this data. If you want to encrypt the cached data, you can put the cache directory in an encrypted file system or block storage.

Encryption and decryption methods

M must be created for each encrypted file system. Each object saved in the object store will have its own random symmetric key S . S by AES-GCM with the symmetric key S encrypted with the global RSA key M , and the RSA key is encrypted with the password specified by the user.

The detailed process of data encryption is as follows:

• Before writing to the object storage, the data block will be compressed using LZ4 or ZStandard.
• Generate a random 256-bit symmetric key S and a random seed N for each block.
• Based on AES-GCM, each block is encrypted S and N
• Use the RSA key M to encrypt the symmetric key S to obtain the ciphertext K .
• K encrypted data, ciphertext 061c45dd1d2195 and random seed N into an object, and then write it to the object storage.

The steps of data decryption are as follows:

• Read the entire encrypted object (it may be a little larger than 4MB).
• Analyze the target data to obtain the ciphertext K , the random seed N and the encrypted data.
• Use the RSA key to decrypt K to obtain the symmetric key S .
• Based on AES-GCM, use S and N decrypt the data to get the plaintext of the data block.
• Decompress the data block.

Key management

When the encryption function is enabled, the security of the RSA key is extremely important. If the key is compromised, it may lead to data leakage. If the key is lost, all encrypted data will be lost, and can not be recovered.

When using juicefs format create a new volume, you can --encrypt-rsa-key parameter to enable static encryption, and the private key will be saved to Redis. When the private key is protected by a password, the environment variable JFS_RSA_PASSPHRASE can be used to specify the password.

Instructions:

Generate RSA key

$ openssl genrsa -out my-priv-key.pem -aes256 2048

Provide the key when formatting

$ juicefs format --encrypt-rsa-key my-priv-key.pem META-URL NAME

Note: If the private key is protected by a password, juicefs mount should use JFS_RSA_PASSPHRASE to specify the password when executing 061c45dd1d2452.

performance

The implementation of TLS, HTTPS and AES-256 in modern CPUs is very efficient. Therefore, enabling encryption has little effect on the performance of the file system. The RSA algorithm is relatively slow, especially the decryption process. It is recommended to use 2048-bit RSA keys for storage encryption. Using a 4096-bit key can have a significant impact on read performance.

Summarize

File system encryption technology can be applied to almost any database storage encryption requirement based on file system. This article starts from the principle to how to generate and use the key in actual operation, and introduces in detail the process of JuiceFS encryption and decryption of data. In the future, JuiceFS will continue to work hard to escort your data security.

Recommended reading: know almost x JuiceFS: using JuiceFS Flink container to start to accelerate

If you have any help, please pay attention to our project Juicedata/JuiceFS ! (0ᴗ0✿)