[Practical and dry goods] Do these 16 optimizations well, and your Linux operating system will look brand new

Hello everyone, this time I would like to talk with you about the operating system optimization plan of the cloud. To put it simply, the Linux operating systems we use are mainly streamlined and optimized based on CentOS6/7. To be more complicated, we have two systems, a custom Linux system for business use and an optimized Linux system for data center use.

In business, we use tailor-made Linux systems for the purpose of being safer, more efficient, and closer to business needs, so as to facilitate lightning deployment at all points across the country, but this system is not universal, so we will not talk about it today. it. Today, I mainly share the optimized Linux version commonly used in data centers, because this is more general and suitable for everyone to use for reference. I will share from the following aspects.

Hostname setting and permanent effect

In CentOS or RHEL, there are three defined hostnames:

• static: The "static" hostname, also known as the kernel hostname, is the hostname that the system automatically initializes from /etc/hostname at boot time.
• Transient: A "transient" hostname is a hostname that is temporarily assigned while the system is running, such as through a DHCP or mDNS server.
• Pretty: "Flexible" hostnames allow free-form (including special/whitespace characters) hostnames to be displayed to end users (eg Gemini's Computer).

A good host name allows non-operation and maintenance personnel in the computer room to understand and locate the machine at a glance. For example: use name + province + computer room name + cabinet number + serial number, the example is as follows:

 HOST="DBS-ZJ-FUD-009"
hostnamectl set-hostname --static $HOST
hostnamectl set-hostname --pretty $HOST
hostnamectl set-hostname --transient  $HOST
echo "$HOST" > /proc/sys/kernel/hostname

Customize the remote login interface

The welcome screen for logging into Linux can be controlled by /etc/issue and /etc/motd. As shown below, you can know the system version, CPU and memory model capacity, running status, application version number and network connection at a glance without logging into the system.

Character set configuration

A good character set can avoid garbled characters displayed on the terminal. The en_US.utf8 character set is recommended.

 # 查看操作系统支持的所有字符集
# locale -a

cat > /etc/locale.conf <<EOF
LANG=en_US.utf8
LC_CTYPE=en_US.utf8
EOF
localectl set-locale LANG=en_US.UTF8

General basic software installation

Because the minimal and streamlined installation based on CentOS is used, some conventional basic software will be missing after installation, so we will appropriately supplement some basic software to help quickly troubleshoot and locate problems.

 yum install -y tree ntpdate  bc nc net-tools wget lsof rsync nmon bash-completion iptables-services firewalld sysstat mtr htop bind-utils yum-utils epel-release smartmontools supervisor python-setuptools python-pip pkgconfig

Time zone and time synchronization settings

In the actual production environment, it is very important to ensure the consistency of the server time zone and time. Especially in scenarios such as distributed systems, multi-machine cluster environments, database master-slave backup, and timed tasks that rely on time synchronization, time zone and time synchronization are very useful. Once the two are inconsistent, it can easily lead to various problems.

 timedatectl set-timezone Asia/Shanghai
timedatectl set-ntp 1
timedatectl set-local-rtc 0
#timedatectl set-time "2018-08-08 18:08:08"
ntpdate -u cn.pool.ntp.org

It is strongly recommended to also write the time synchronization operation into crontab for double insurance.

 # crontab -l
0 * * * * root(ntpdate -o3 192.168.1.10 211.115.194.21 )

Disable SELinux

SELinux is Security-Enhanced Linux, mainly developed by the US National Security Agency. It is rigorous in concept, complex in structure and configuration, and strict in operation. Therefore, it can be decided whether to open it according to the situation when using it, and it can be enabled in special scenarios such as confidential and information-sensitive institutions.

 sed -r -i  '/^SELINUX=/s^=.*^=disabled^g' /etc/selinux/config
set enforce 0

add normal user and sudo

sudo is a Linux system management command, a tool that allows system administrators to let ordinary users execute some or all root commands, such as halt, reboot, su, and so on. This not only reduces the root user's login and administration time, but also improves security.

It should be noted that, try not to use root directly in the production environment. It is recommended to create a new ordinary user first and then increase the permissions.

 [root@OPS-FDI-020 ~]# useradd shaohy
[root@OPS-FDI-020 ~]# usermod -G wheel  shaohy
[root@OPS-FDI-020 ~]# sed -i '/pam_wheel/s/^#//g'  /etc/pam.d/su

Add sudo for user shaohy, everything except shutdown:

 [root@OPS-FDI-020 ~]# visudo
Cmnd_Alias SHUTDOWN = /sbin/halt, /sbin/shutdown, /sbin/poweroff, /sbin/reboot, /sbin/init
shaohy         ALL=(ALL)       ALL,!SHUTDOWN
%wheel         ALL=(ALL)       ALL,!SHUTDOWN    #修改wheel组的权限，禁止关机
Defaults logfile=/var/log/sudo.log

Configure firewall rules and iptables

Since CentOS7, firewalld is used by default to manage the netfilter subsystem. It should be noted that the underlying commands still use iptables. The difference between the two is as follows:

• firewalld can dynamically modify individual rules, dynamically manage rule sets, and allow rules to be updated without breaking existing sessions and connections. And iptables, after modifying the rules must be refreshed to take effect.
• firewalld uses zones and services instead of chain rules.
• firewalld is denied by default, and it needs to be set before it can be released. And iptables is allowed by default, and only those that need to be denied are restricted.

So when choosing, you can consider not rejecting firewalld, but try to accept it.

 #!/bin/sh
IPS="192.168.0.0/16"
firewall-cmd --zone=public --remove-service=ssh
firewall-cmd --new-zone=openssh --permanent
firewall-cmd --zone=openssh --add-port=22222/tcp --permanent
firewall-cmd --permanent --zone=public --set-target=default
for ip in  $IPS;do
        firewall-cmd --zone=openssh --add-source=$ip --permanent
done
firewall-cmd --reload
firewall-cmd --runtime-to-permanent

GPT partitions and partition mounts

There are a large number of hard disks on each CDN server of Paiyun, not to mention that the amount of 4T and 6T is relatively small, even 10T has as many as 12, so it is necessary to automatically format and partition the operation and maintenance of these hard disks operate.

The early Master Boot Record (abbreviation: MBR), also known as the master boot sector, is the first sector that must be read when the computer is turned on to access the hard disk, and cannot support hard disk booting larger than 2T. Although patched MBR can also support partitions larger than 2T, GPT has become a new trend. Compared with MBR, the GPT partition scheme has the following characteristics:

• GPT is part of the UEFI standard (UEFI is a personal computer system specification that defines the software interface between the operating system and system firmware as an alternative to BIOS).
• GPT partition list supports up to 128PB (1PB=1024TB).
• 128 partitions can be defined.
• There is no concept of primary partition, extended partition and logical partition, all partitions can be formatted.

 #!/bin/sh
DEV=`lsscsi | awk '/HGST/{print $NF}'` # 筛选所有的sata硬盘
i=1
for dev in $DEV;do
        label="/disk/sata0$i"
        echo $dev $label
        parted -m -s $dev rm 1
        parted -m -s $dev mklabel gpt
        parted -m -s $dev mkpart primary ext4 2048s 100%
        partx -a $dev
        ((i++))
        nohup mkfs.ext4 -L $label ${dev}1 >/dev/null &
done

ulimit quota setting

Because the CentOS7 / RHEL7 system uses Systemd to replace the previous SysV, the configuration of the /etc/security/limits.conf file is only applicable to the resource limit of the logged-in user through PAM authentication, and the service resource limit of systemd does not take effect.

Because of the resource limitations of the systemd service, we place the global configuration in /etc/systemd/system.conf and /etc/systemd/user.conf. Where system.conf is used for system instances and user.conf is used for user instances.

 sed -r -i -e '/DefaultLimitCORE/s^.*^DefaultLimitCORE=infinity^g' -e '/DefaultLimitNOFILE/s^.*^DefaultLimitNOFILE=100000^g' -e '/DefaultLimitNPROC/s^.*^DefaultLimitNPROC=100000^g' /etc/systemd/system.conf

Increase the limit on the number of open files, the default is to set the maximum number of processes for non-root users to 4096.

 cat > /etc/security/limits.d/20-nproc.conf <<EOF
*       soft  nproc  10240
root    soft  nproc  unlimited
EOF

sysctl.conf configuration takes effect

The sysctl.conf file has many and complex configuration parameters. It will take a long time to explain the function of each parameter in detail. Here we look at some common tuning parameters to get a feel for them. Mainly focus on network and TCP parameter optimization, swap disabling, file handle enlargement.

 net.ipv4.ip_forward=1
net.ipv4.ip_local_port_range=1000 65535

net.ipv4.tcp_slow_start_after_idle=0
net.ipv4.tcp_no_metrics_save=1
net.ipv4.tcp_rfc1337=1
net.ipv4.tcp_timestamps=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_dsack=1
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_rmem=4096 102400 16777216
net.ipv4.tcp_wmem=4096 102400 16777216
net.ipv4.tcp_mem=786432 1048576 1572864
net.ipv4.tcp_syncookies=1
net.ipv4.tcp_syn_retries=3
net.ipv4.tcp_synack_retries=5
net.ipv4.tcp_retries1=3
net.ipv4.tcp_retries2=15
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_max_syn_backlog=262144                                  
net.ipv4.tcp_max_orphans=262144
net.ipv4.tcp_tw_recycle=0
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_keepalive_time=30
net.ipv4.tcp_keepalive_intvl=10
net.ipv4.tcp_keepalive_probes=3
net.ipv4.tcp_max_tw_buckets=600000
net.ipv4.tcp_congestion_control=bbr

net.core.somaxconn=8192
net.core.rmem_default=131072
net.core.wmem_default=131072
net.core.rmem_max=33554432
net.core.wmem_max=33554432
net.core.dev_weight=512
net.core.optmem_max=262144
net.core.netdev_budget=1024
net.core.netdev_max_backlog=262144

vm.swappiness=0
vm.dirty_writeback_centisecs=9000
vm.dirty_expire_centisecs=18000
vm.dirty_background_ratio=5
vm.dirty_ratio=10
vm.overcommit_memory=1
vm.overcommit_ratio=50
vm.max_map_count=200000

fs.file-max=524288
fs.aio-max-nr=1048576

/etc/passwd security check

User's shell permission check, usually considering security issues, non-root users are not allowed to have shell permissions.

 # 过滤出有uid==0, gid==0的潜伏用户,判断有没有bash
awk -F: '($3==0||$4==0) {print $0}' /etc/passwd|grep -i bash
# 除root外的用户shell全部为nologin
sed -r -i '/^[^root]/s:/bin/bash:/sbin/nologin:g' /etc/passwd

sshd service configuration

Because almost all Linux servers are managed remotely via SSH, it's easy to attract a lot of uninvited guests trying to gain access to your server via SSH. So SSH security can not be ignored! It is strongly recommended to disable password login, and use the public key method to log in to the management server using SSH before modification.

 sed -r -i '/#Port 22/s^.*^Port 22222^g;/^PasswordAuthentication/s^yes^no^g' /etc/ssh/sshd_config

Turn off unnecessary services

Linux services (Linux services) are important to every user of Linux. Turning off unnecessary services can make Linux run more efficiently, but not all Linux services can be turned off. This is a trade-off.

 systemctl disable network  postfix irqbalance tuned rpcbind.target

If there are udp-based services, such as ntpd and dns, pay attention to the reflection attack of udp. Because the reflection attack of udp is extremely destructive, it is best to close useless related services, or choose a high-defense computer room to deploy such services.

logrotate reduces the rotation log package

When there are many server processes and the log file size grows rapidly, disk space is continuously consumed and an alarm is triggered. At this time, it is necessary to manually clean up the logs according to various dimensions on a regular basis. If it is not cleaned up in time, it will easily become an operation and maintenance accident.

You can usually use the logrotate log rolling mechanism to divide log files into multiple copies by time or size, and delete old log files to save space and facilitate organization.

 sed -r -i 's@weekly@daily@g;s@^rotate.*@rotate 7@g;s@^#compress.*@compress@g' /etc/logrotate.conf
systemctl daemon-reload; systemctl restart rsyslog

journalctl adjust journal journal

Before the emergence of Systemd, the logs of the Linux system and each application were managed separately, while Systemd unified management of all Unit startup logs. The advantage of this is that you can view all kernel and application logs with just one journalctl command.

Appropriate configuration can make the journal volume controllable, so that the capacity does not explode.

 sed -r -i -e '/Compress=/s@.*@Compress=yes@g; /SystemMaxUse=/s@.*@SystemMaxUse=4G@g; ' -e '/SystemMaxFileSize=/s@.*@SystemMaxFileSize=256M@g;' -e '/MaxRetentionSec=/s@.*@MaxRetentionSec=2week@g' /etc/systemd/journald.conf

To sum up, after completing the above optimizations, a safe and reliable operating system can be officially launched to provide services. I wish you a happy trip to Linux!

Recommended reading

The interviewer asked me whether Redis is single-threaded or multi-threaded.

Base64 encoding knowledge, exhausted