Shell script avoidance guide (1)

Hi everyone, this is Zhang Jintao.

When it comes to Shell, everyone is not too unfamiliar. We usually think that Shell is the interface between us and the system. Executing commands returns output, such as bash, zsh, etc. Sometimes someone confuses Shell and Terminal, but this has nothing to do with this article, so ignore it for the time being.

As a programmer, we may use Shell every day, and occasionally we will organize some commands together and write a Shell script to improve our work efficiency.

However, there may be deep pits hidden in the seemingly simple Shell script. Here I first give two simple and similar Shell scripts, you might as well take a look at the output of these two pieces of code:

#!/bin/bash
set -e -u
i=0
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done

answer to 1619cbbb2ba8f5 is that only a 0 is output.

#!/bin/bash
set -e -u
let i=0
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done

answer to 1619cbbb2ba965 is that there is no output, just exit.

code clearly, you can probably skip the rest of this article.

Let me first break down the main knowledge points involved in this code.

Variable declaration

There are many ways to declare variables, but their behavior varies.

We must first have a basic understanding: Bash has no type system, and all variables are strings. this reason, if you let variables perform arithmetic operations, you cannot write arithmetic operators directly like in other programming languages. This will make bash interpret it as an operation on strings, rather than operations on numbers.

Direct declaration

(MoeLove)➜  ~ foo=1+1
(MoeLove)➜  ~ echo $foo
1+1

Direct declaration is the simplest, but as mentioned earlier, direct declaration will be treated as string by default, and arithmetic operations cannot be performed during declaration.

declare

(MoeLove)➜  ~ declare foo=1+1
(MoeLove)➜  ~ echo $foo
1+1

In addition to directly declaring variables, the more commonly used method is to use declare to declare variables, but by default, the declared variables are processed as strings, and normal arithmetic operations cannot be performed.

declare integer attribute

Declare When declaring variables, you can -i parameter. When the variable is assigned, arithmetic operations will be performed.

(MoeLove)➜  ~ declare -i bar=1+1
(MoeLove)➜  ~ echo $bar
2

However, it should be noted that after adding an integer attribute, if a string is assigned to it, it will fail to parse, that is: set the value to 0:

(MoeLove)➜  ~ bar=test
(MoeLove)➜  ~ echo $bar
0

let statement

Another way, we can use the let command to declare a variable. This method allows arithmetic operations during declaration and also supports assigning other values to this variable.

(MoeLove)➜  ~ let baz=1+1
(MoeLove)➜  ~ echo $baz
2
(MoeLove)➜  ~ baz=moelove.info
(MoeLove)➜  ~ echo $baz
moelove.info

while loop

while list-1; do list-2; done

The while syntax in Bash is like this. After the while keyword is a sequence (list), which can be one or more expressions/statements.

It should be noted that when the return value of list-1 is 0, list-2 will always be executed, and the final return value of the while statement is the return value of the last execution of list-2, or, if no statement is executed , It returns 0.

Arithmetic calculation in bash

You will use this part of the content frequently. Let me introduce several commonly used methods:

Arithmetic expansion

There are 7 kinds of extensions in Bash, and arithmetic extensions are just one of them. Specifically, the value of the expression is calculated by a form $((expression)) E.g:

(MoeLove)➜  ~ echo $((3+7))
10
(MoeLove)➜  ~ x=3;y=7
(MoeLove)➜  ~ echo $((x+y))
10

expr command

expr is a command provided by the coreutils package, which can perform calculations on expressions, or compare sizes and the like.

(MoeLove)➜  ~ x=3;y=7
(MoeLove)➜  ~ expr $x + $y
10
# 比较大小
(MoeLove)➜  ~ expr 2 \< 3
1
(MoeLove)➜  ~ expr 2 \< 1
0

bc command

By definition, bc is actually a computing language that supports arbitrary precision and can be executed interactively. It is much more expr , especially it also supports floating point operations. E.g:

General floating point calculation

(MoeLove)➜  ~ echo "scale=2;7/3"|bc
2.33
(MoeLove)➜  ~ echo "7/3"|bc
2

Note: scale needs to be specified manually, it represents the number of digits after the decimal point. By default, scale is 0.

Built-in function

bc also has some built-in functions, which can facilitate us to perform some fast calculations. For example, we can use sqrt() quickly calculate the square root.

(MoeLove)➜  ~ echo "scale=2;sqrt(9)" |bc
3.00
(MoeLove)➜  ~ echo "scale=2;sqrt(6)" |bc
2.44

script

In addition, bc also supports a simple syntax, which can support declaring variables, writing loops and judgment statements, etc. For example: we can print a number within 20 that is divisible by 3:

(MoeLove)➜  ~ echo "for(i=1; i<=20; i++) {if (i % 3 == 0) i;}" |bc
3
6
9
12
15
18

bash debugging

In fact, there is no built-in debugger in the bash shell. In many cases, debugging is performed by repeating running and printing. But this method is not efficient enough.

Here is a more intuitive and convenient way to debug shell code. The following is a sample shell code.

(MoeLove)➜  ~ cat compare.sh 
#!/bin/bash
read -p "请输入任意数字: " val
real_val=66
if [ "$val" -gt "$real_val" ]
then
   echo "输入值大于等于预设值"
else
   echo "输入值比预设值小"
fi

Increase the execution authority for it, or use bash to execute:

(MoeLove)➜  ~ bash compare.sh 
请输入任意数字: 33
输入值比预设值小

Detailed mode

By adding the -v option, you can turn on the detailed mode for viewing the executed commands. Of course, we can also open this mode -v option on shebang, or adding set -v

(MoeLove)➜  ~ bash -v compare.sh
which () {  ( alias;
 eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot "$@"
}
#!/bin/bash
read -p "请输入任意数字: " val
请输入任意数字: 33
real_val=66
if [ "$val" -gt "$real_val" ]
then
   echo "输入值大于等于预设值"
else
   echo "输入值比预设值小"
fi
输入值比预设值小

Use xtrace mode

We can enter the xtrace mode by adding the -x parameter, which is used to debug the variable value in the execution phase.

(MoeLove)➜  ~ bash -x compare.sh
+ read -p '请输入任意数字: ' val
请输入任意数字: 33
+ real_val=66
+ '[' 33 -gt 66 ']'
+ echo 输入值比预设值小
输入值比预设值小

Identify undefined variables

In the following example, I deliberately mistyped a character. After executing the script, you will find that there is no error, but the result is not what we expected. This type may be the majority of hand errors, so we need to check whether there are unbound variables.

(MoeLove)➜  ~ cat add.sh 
#!/bin/bash
five=5
ten=10
total=$((five+tne))
echo $total
(MoeLove)➜  ~ bash add.sh
5
(MoeLove)➜  ~ bash -u add.sh
add.sh: line 4: tne: unbound variable

Add -u option, you can check whether the variable is undefined/bound.

Use in combination

The above are several more common ways of using, of course, you can also use them in combination. For example, the problem of undefined variables above, combined with -vu you can directly see what the specific problem code is.

(MoeLove)➜  ~ bash -vu add.sh
which () {  ( alias;
 eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot "$@"
}
#!/bin/bash
five=5
ten=10
total=$((five+tne))
add.sh: line 4: tne: unbound variable

Output debugging information to the specified file

debug.log file on a specific FD. Note that this FD needs to be BASH_XTRACEFD . In addition, I modified the PS4 . Its default value is + looks messy and there is no valid information. I passed Set PS4='$LINENO: ' make it display the line number.

Then set the need to debug the position set -x , at the end of the unknown set set +x , so debug log will only record I need part of a debug log.

(MoeLove)➜  ~ cat compare.sh 
#!/bin/bash
exec 6> debug.log 
PS4='$LINENO: ' 
BASH_XTRACEFD="6" 
read -p "请输入任意数字: " val
real_val=66
set -x
if [ "$val" -gt "$real_val" ]
then
   echo "输入值大于等于预设值"
else
   echo "输入值比预设值小"
fi
set +x

echo "End"
(MoeLove)➜  ~ bash compare.sh 
请输入任意数字: 88
输入值大于等于预设值
End
(MoeLove)➜  ~ cat debug.log 
8: '[' 88 -gt 66 ']'
10: echo $'\350\276\223\345\205\245\345\200\274\345\244\247\344\272\216\347\255\211\344\272\216\351\242\204\350\256\276\345\200\274'
14: set +x

Here introduced by setting options set way simpler, plus others such as the use of trap debug mode also recommend you to try, here is not started.

Back to the beginning of the question

Then we use the debugging method just introduced to execute the two scripts at the beginning and answer the questions.

First

(MoeLove)➜  ~ bash -xv demo1.sh
#!/bin/bash
set -e -u
+ set -e -u
i=0
+ i=0
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done
+ '[' 0 -lt 6 ']'
+ echo 0
0
+ (( i++ ))

As can be seen from the above debugging results, this script ((i++)) after 0 and then executing 0619cbbb2bba2a. why? Mainly due to the addition of the set -e option at the top of the script.

This option will exit directly when it encounters the first non-zero value. let's explain:

(MoeLove)➜  ~ i=0
(MoeLove)➜  ~ $((i++))
(MoeLove)➜  ~ echo $?
1

As you can see, ((i++)) , the return value is actually 1, so set -e is triggered, and the script exits.

the second

(MoeLove)➜  ~ bash -xv demo2.sh
#!/bin/bash
set -e -u
+ set -e -u
let i=0
+ let i=0

The main difference between the second and the first is the variable assignment. The return value of let i=0 is 1, so the exit condition of set -e We try to modify the second script and execute it again:

[tao@moelove ~]$ cat demo2-1.sh
#!/bin/bash
set -e -u
let i=1
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done

[tao@moelove ~]$ bash demo2-1.sh 
1
2
3
4
5

The let i=0 modified to let i=1 to perform successfully as expected.

Summarize

In this article, we mainly talked about variable declarations, loops, mathematical operations and debugging of the bash shell in the bash shell. Does it inspire you? Welcome to leave a message for communication.

• Note: This article only discusses Bash Shell

Welcome to subscribe to my article public account【MoeLove】