Gitee fell down and built a free distributed storage service

Hello everyone, I'm not Cai Chen~

It was really scrapped by Gitee recently. Yesterday, the warehouse was set up and cannot be made public. It must be open sourced and approved before it can be made public.

Is this being interviewed? First Blog Garden, now Gitee

The small partners in the exchange group are also complaining, and the hard-built picture bed is also useless

Some friends suggest to use the paid ones directly, such as Qiniuyun, but there are many skills that do not overwhelm you. Wouldn’t it be beautiful to build one yourself!

Today, I will come to Amway, an open source free object storage suite, MinIO , create a free storage system by myself, and implement a map bed by the way.

What is MinIO?

Minio is an open source object storage suite written based on Golang. It is based on the Apache License v2.0 open source protocol. Although it is lightweight, it has good performance. It is compatible with Amazon S3 cloud storage service interface. It can be easily combined with other applications, such as NodeJS, Redis, MySQL, etc.

1. Application scenarios

The application scenario of MinIO can be used not only as an object storage service of a private cloud, but also as a gateway layer of cloud object storage, seamlessly Amazon S3 or MicroSoft Azure .

2. Features

1. High performance: As a high-performance storage, under standard hardware conditions, its read and write rates can reach 55Gb/s and 35Gb/s respectively. And MinIO supports that an object file can be of any size, ranging from a few kb to a maximum of 5T.
2. Scalable : Different MinIO clusters can form a federation to form a global namespace and support across multiple data centers.
3. Cloud native : containerization, K8S-based orchestration, multi-tenancy support.
4. Amazon S3 Compatible : Use Amazon S3 v2/v4 API. Minio server can be accessed using Minio SDK, Minio Client, AWS SDK and AWS CLI.

5. SDK supports :

   1. GO SDK: https://github.com/minio/minio-go
   2. JavaSDK: https://github.com/minio/minio-java
   3. PythonSDK: https://github.com/minio/minio-py
6. Graphical interface : there is an operation page
7. Erasure coding support : MinIO uses erasure coding, Checksum to prevent hardware errors and silent data pollution. In the highest redundancy configuration, data can be recovered even if 1/2 of the disks are lost.

The function is very powerful. This article is just to attract more ideas. Interested friends can explore it by themselves~

Install MinIO

The installation is very simple. I use docker to install here. The steps are as follows:

1. Get the image

The execution command is as follows:

 docker pull minio/minio

2. Boot image

The execution command is as follows:

 docker run -p 9000:9000 -p 9001:9001 --name minio -d --restart=always -e "MINIO_ACCESS_KEY=admin" -e "MINIO_SECRET_KEY=admin" -v /home/data:/data -v /home/config:/root/.minio minio/minio server --console-address ":9000" --address ":9001" /data

The command is explained as follows:

• -p : 9000 is the port of the graphical interface, 9001 is the port of the API, which needs to be used when using the SDK to connect
• MINIO_ACCESS_KEY : Specifies the user name of the GUI
• MINIO_SECRET_KEY : Specifies the password for the GUI

Follow the above two steps to start successfully.

3. Graphical interface operation

After the installation is successful, directly access the address: http:/ip:9000/login , as follows:

After entering the user name and password to log in successfully, it is as follows:

There are many menus, so I won't introduce them in detail here. The author directly creates a bucket in the Buckets menu as test , as shown below:

And set the privacy rule of this bucket to public , as follows:

MinIO has been successfully installed and set up

Spring Boot integrates MinIO to upload files

Although MinIO provides a manual upload operation on the graphical interface, it can also be uploaded through the SDK. Let's introduce Spring Boot's integration of MinIO to upload files.

1. Get accessKey and secretKey

The accessKey and secretKey here are not the login name and password of the graphical interface. It is very simple to obtain and operate directly in the graphical interface, as shown in the following figure:

2. Add dependencies

Add MinIO dependencies as follows:

 <dependency>
    <groupId>io.minio</groupId>
    <artifactId>minio</artifactId>
    <version>8.2.1</version>
</dependency>

3. Add configuration

Here the author has made a simple encapsulation of the SDK, and the source code of the case will be provided. Only part of the code is listed below.

Add the MInIO related configuration in the aplication.yml configuration, as follows:

 minio:
  # 访问的url
  endpoint: http://192.168.47.148
  # API的端口
  port: 9001
  # 秘钥
  accessKey: HQGWFYLWGC6FVJ0CQFOG
  secretKey: pUGhAgQhZDxJaLmN3uz65YX7Bb3FyLdLglBvcCr1
  secure: false
  bucket-name: test # 桶名 我这是给出了一个默认桶名
  image-size: 10485760 # 我在这里设定了 图片文件的最大大小
  file-size: 1073741824 # 此处是设定了文件的最大大小

4. Create a new upload file interface

The author defines an upload file interface here, as follows:

 /**
 * @author 公众号：码猿技术专栏
 */
@RequestMapping("/minio")
@RestController
public class MinioController {

    @Autowired
    private  MinioService minioService;

    @PostMapping("/upload")
    public String uploadFile(MultipartFile file, String bucketName) {
        String fileType = FileTypeUtils.getFileType(file);
        if (fileType != null) {
            return minioService.putObject(file, bucketName, fileType);
        }
        return "不支持的文件格式。请确认格式,重新上传！！！";
    }
}

The source code has been uploaded to GitHub, pay attention to the public number: Code Ape Technology Column, reply keywords: 9535 Get it!

5. Test

The above four steps have been integrated and completed. Let’s directly call the interface to upload a picture and try it, as follows:

The URL returned by the interface is the access address of the file, which can be accessed directly by entering the browser.

The stored files can also be seen in MInIO, as shown below:

If you need to share it with others, you can also share it manually. The validity period is 7 days. Once the validity period expires, it will be invalid, as follows:

The source code has been uploaded to GitHub, pay attention to the public number: Code Ape Technology Column, reply keywords: 9535 Get it!

Summarize

Although MInIO is an open source project, it has very powerful functions. It can be used for object storage in small projects, and you can also use MinIO to build a free image bed.

The picture address can be directly returned to you, so there is no need to say more about the construction of the picture bed , try it yourself~

One last word (don't be a prostitute, please pay attention)

Every article of Chen Mou is carefully output. He has written 3 columns and organized them into PDF . The way to get it is as follows:

1. "Spring Cloud Advanced" PDF: Follow the official account: [ Code Ape Technology Column ] Reply to the keyword Spring Cloud Advanced Get it!
2. "Spring Boot Advanced" PDF: Follow the official account: [ Code Ape Technology Column ] Reply to the keyword Spring Boot Advanced Get!
3. "Mybatis Advanced" PDF: Follow the official account: [ Code Ape Technology Column ] Reply to the keyword Mybatis Advanced Get it!

In addition, Chen's "Spring Cloud Alibaba Actual Combat" video column has been completed. If you need to subscribe , click on the lower left corner to read the original text .