Know the container, let’s start talking about its history

Abstract: ? It relies on the Docker image. He packaged all the dependencies of the application, completely solved the consistency problem of the environment, redefined the software delivery method, and improved production efficiency.

This article is shared from the HUAWEI cloud community " know about containers, let's talk about from its history", author: technical torchbearer.

There are already many articles on the Internet about the history, development, and technical nature of containers. Here is to combine my own work experience and understanding, through a series of articles, to clarify this technology.

The history and development of containers

1. Past life

When it comes to containers, we have to mention LXC (Linux Container), which is the predecessor of Docker, or Docker is the user of LXC. The complete LXC capability was incorporated into the Linux mainline in 2008, so the concept of container was basically finalized in 2008, and it was not created by Docker later. There are many introductions about LXC. Generally speaking, "LXC is a container technology provided by the Linux kernel, which can provide lightweight virtualization capabilities and isolate processes and resources." But in summary, there are nothing more than two knowledge points Cgroups ( Linux Control Group) and Linux Namespace. If you figure out the two, container technology will be basically mastered.

• Cgroups: The focus is on "restrictions." Restricting the use of resources, including the use of CPU, memory, and disk, reflects the ability to manage resources.
• Namespace: The focus is on "isolation". The Linux view seen by the isolated process. To put it plainly, the container and the container should not affect each other, and the container and the host should not affect each other.

2. A difficult start in adolescence

In 2009, Cloud Foundry realized the operation of containers based on LXC, and the project was named Warden. In 2010, dotCloud was also based on LXC technology and implemented a container engine using the Go language, which is now Docker. At that time, dotCloud was still a small company, and the humble birth of Docker was not very popular, and it was very difficult to live.

3. Grow into a giant

In 2013, dotCloud company decided to open source Docker. After open source, the project suddenly became popular. In a big way, the reason for the fire is Docker's slogan "Build once, Run AnyWhere". Ha ha, is it familiar? Yes, it is the same as Java's Write Once and Run AnyWhere. For a programmer, after the program is written, it can be packaged into a mirror and can be deployed and run anywhere. The development, testing, and production environments are exactly the same, which is a big temptation. Programmers no longer have to locate various cheating problems caused by environmental differences.

The extremely popular Docker open source project directly drove dotCloud to change its name to Docker in 2013. Docker has also grown rapidly, killing CoreOS's rkt container and Google's lmctfy container, directly becoming the de facto standard for containers. It is also later that when people mentioned containers, they thought it was Docker.

To sum up, why Docker is popular depends on Docker mirroring. He packaged all the dependencies of the application, completely solved the consistency problem of the environment, redefined the software delivery method, and improved production efficiency.

4. Nibbled by the great powers

Docker has grown rapidly in the container field, and its ambitions have naturally grown. In 2014, it launched the container cloud product Swarm (similar to K8s), and wanted to expand its business. At the same time, Docker has the absolute right to speak in the open source community and is quite strong. This behavior of going its own way and leaving others with nowhere to go makes other big players in the container field very upset. In order not to let Docker dominate, they decided to do him.

In June 2015, under the "operations" of major companies such as Google and Redhat, the Linux Foundation established the OCI (Open Container Initiative) organization, aiming to develop an open industrial standard around the container format and runtime. Said the OCI standard. At the same time, Docker Company donated the Libcontainer module to the CNCF community as an implementation of the OCI standard. This is the current RunC project. To put it bluntly, there is now a standard in this area. Let's play together and not be bound by a particular project.

When it comes to Docker, you have to talk about Google's Kubernetes. As the de facto standard of the container cloud platform, it has been widely used nowadays, and it has become a standard for major manufacturers. Kubernetes natively supports Docker, which keeps Docker's market share consistently high. The picture shows the market share of container runtimes in 2019.

But in 2020, Kubernetes suddenly announced that after version 1.20, that is, after 2021, Docker will no longer be supported as the default container runtime, and dockershim will be removed from the code backbone.

As shown in the figure, K8s itself defines a standard container runtime interface CRI (Container Runtime Interface), the purpose is to be able to dock any container runtime that implements the CRI interface. In the early days, Docker was the undisputed king of container runtimes. K8s has built-in support for Docker, through dockershim to achieve the adaptation of the standard CRI interface to the Docker interface, in order to gain more users. With the maturity of the open source container runtime Containerd (which implements the CRI interface and is also donated by Docker to CNCF), K8s no longer maintains dockershim, but is only responsible for maintaining the standard CRI and unbinding with a specific container runtime. Of course, it's not that K8s doesn't support Docker, it's just a question of who maintains dockershim. As the attitude of K8s changes, it is expected that more and more developers will choose to directly connect with the open source Containerd. What changes will happen to Docker and the Docker open source project (now renamed moby) in the future? It's not good.

Speaking of this, I don't know if you have noticed that the Docker company actually donated Containerd and runC. What are these two? Simply put, runC is the implementation of the OCI standard, also called the OCI runtime, which is really responsible for operating the container. Containerd provides external interfaces, manages and controls runC. So the picture above should really look like this.

Docker company is a typical case of a small company that became popular due to an explosive project. No matter the technical level, the company’s business level, and how to fight with the big factory, whether it is good or bad, it is worth learning and Learn the story behind it.

What is a container

According to international practice, when introducing a new concept, you have to start with what everyone is familiar with. Fortunately, the concept of containers is fairly easy to understand. Drinking cups, foot-washing buckets, and fish tanks are all containers. The "container" in the container technology is also a similar concept, except that the things it installs are different. What it installs is the application software itself and the dependencies that the software needs to run. Using the fish tank analogy, the application software in the fish tank is the fish, and the dependencies of the fish tank are fish food and water. So everyone can understand the docker logo. The sea is the host, the docker is the whale, the container on the whale's back is the container, and our application is installed in the container.

When talking about containers, you must not bypass the container image. Here, the container image is simply understood as a compressed package. The compressed package contains the executable program of the application and the files that the program depends on (for example: configuration files and dynamic libraries that need to be called, etc.), and then through actual operations to see what the container is.

1. Looking at the container from the perspective of the host:

1. First, we start the container.

docker run -d --name="aimar-1-container" euleros_arm:2.0SP8SPC306 /bin/sh -c "while true; do echo aimar-1-container; sleep 1; done"

This is a standard command of Docker. It means to use the euleros_arm:2.0SP8SPC306 image (image name: version number) to create a new container named "aimar-1-container", and execute the shell command in the container: print "aimar-1-container" once every second .

parameter description:
-d: Start the container in background running mode and return the container ID.
--name: Specify a name for the container.

docker run -d --name="aimar-1-container" euleros_arm:2.0SP8SPC306 /bin/sh -c "while true; do echo aimar-1-container; sleep 1; done"
207b7c0cbd811791f7006cd56e17033eb430ec656f05b6cd172c77cf45ad093c

From the output, we see a long string of characters 207b7c0cbd811791f7006cd56e17033eb430ec656f05b6cd172c77cf45ad093c. It is the container ID, which can uniquely identify a container. Of course, when using it, you don't need to use the full id, just use the abbreviated id (the first few digits of the full id). For example, in the figure below, the container id queried by docker ps is 207b7c0cbd81

After the aimar-1-container container is successfully started, we use ps on the host to view it. At this time, you can find that the container you just started is a process with a PID of 12280.

We tried to start 2 more containers, and check on the host again, you will find that 2 more processes have been added, the PIDs are 20049 and 21097 respectively.

So, we can get a conclusion. From the perspective of the host, the container is the process.

2. Next, we enter this container.

docker exec -it 207b7c0cbd81 /bin/bash
Docker exec is also a standard command of Docker, used to enter a container. It means to enter the container whose container id is 207b7c0cbd81, and execute the /bin/bash command after entering to enable command interaction.

parameter description:
-it is actually the two parameters -i and -t, which means that after the container is started, an input/output terminal must be allocated to facilitate our interaction with the container and realize the ability to "talk" with the container.

The hostname changed from kwephispra09909 to 207b7c0cbd81, indicating that we have entered the container. In the container, we try to start a new process.

[root@207b7c0cbd81 /]# /bin/sh -c "while true; do echo aimar-1-container-embed; sleep 1; done" &

Go back to the host and check the ps again. You will find that is a process from the perspective of the host, whether it is directly starting the container or starting a new process in the container.

2. Looking at the container from the perspective of the container:

Earlier we have entered the container and started a new process. But we did not look at the process in the container. If you execute ps in the container, you will find that the result is completely different from the result of executing ps on the host. The following figure is the execution result in the container.

In the Container1 container, only the shell processes (container1 and container1-embed) that have just been started can be seen, and no other processes on the host machine, nor processes in Container2 and Container3 can be seen. These processes seem to be locked in a box, completely unaware of the outside world, and even think that the container1 we are executing is the No. 1 process (the No. 1 process is also called the init process, which is the ancestor process of all other user processes in the system). Therefore, from the perspective of the container, the container feels "I am the sky, I am the earth, welcome to my world".

But embarrassingly, on the host, they are too ordinary processes. Note that for the same process, the process ID seen in the container is different from the process ID seen on the host machine. The process IDs in the container are 1 and 1859, respectively, and the corresponding process IDs on the host are 12280 and 9775 (see the figure above).

Three, summary

Through the above experiment, the definition of the container needs to add an attributive. container is a process => the container is a process isolated from the rest of the system. At this time, it’s easier to understand when we look at the figure below. The container is a process running on the host OS (the host OS of the virtual machine container is the Guest OS). There is isolation between containers and between the container and the host, such as: process Number of isolation.

In the container and on the host, the process ID of the same process is different. For example, the PID of Container1 is 1 in the container and 12280 on the host. So what is the real PID of the process? Of course it is 12280! Then why the PID seen in the container is 1? It is Linux Namespace that causes this illusion.

Linux Namespace is the method used by the Linux kernel to isolate resources. The resources under each namespace are opaque and invisible to other namespaces.

Namespace is classified by isolated resources:

Inside and outside the aforementioned container, the process IDs are different, and it is the PID Namespace that is used. So where is this Namespace? Everything is a file on Linux. Yes, this Namespace is in the file. The Namespace information corresponding to a certain process is recorded in the proc file (/proc/process number/ns) on the host. As shown in the figure below, the number (for example: pid:[ 4026534312]) represents a Namespace.

For the three containers, Container1, Container2, and Container3, we can see that their PID Namespace is different. It means that the PIDs in their three containers are isolated from each other, that is, the three containers can have processes with the same PID number at the same time, for example, there are processes with PID=1.

In a namespace, the two processes are visible to each other, but the PID is different from the one seen on the host.

At this point, we can further refine the definition of the container. container is a process isolated from the rest of the system=》Container is a process isolated from the rest of the system using Linux Namespace.

Click to follow and learn about Huawei Cloud's fresh technology for the first time~