关于shell和expect和ssh

之前看到一些大神说，作为一个运维，一个系统工程师的能力的其中一个很重要的检验标准就是他能够管理多少台机器，他能够自动化到什么程度，他能够多懒!---所以我也来班门弄斧了，所以就有了这篇文章。

在现今高度发展的it社会，已经有很多的自动化管理程序了，例如Puppet，Salt，func，Capistrano .......而且还有云虚拟化OpenStack,kvm,xen.....尤其Docker更是新生代黑马，为自动化管理而生的。但存在即为合理，你有高大上，我也有土肥圆，相对于快捷，简单的管理小批量linux机器，ssh和expect是非常好用的。

Expect是什么

他是一枚程序，是基于uucp（Unix to Unix Copy Protocol）的发送/预期的序列设计而来的。

The name "Expect" comes from the idea of send/expect sequences popularized by uucp, kermit and other modem control programs. However unlike uucp, Expect is generalized so that it can be run as a user-level command with any program and task in mind. Expect can actually talk to several programs at the same time.
For example, here are some things Expect can do:

Cause your computer to dial you back, so that you can login without paying for the call.
Start a game (e.g., rogue) and if the optimal configuration doesn't appear, restart it (again and again) until it does, then hand over control to you.
Run fsck, and in response to its questions, answer "yes", "no" or give control back to you, based on predetermined criteria.
Connect to another network or BBS (e.g., MCI Mail, CompuServe) and automatically retrieve your mail so that it appears as if it was originally sent to your local system.
Carry environment variables, current directory, or any kind of information across rlogin, telnet, tip, su, chgrp, etc.

从最简单的层次来说，Expect的工作方式象一个通用化的Chat脚本工具。Chat脚本最早用于UUCP网络内，以用来实现计算机之间需要建立连接时进行特定的登录会话的自动化。

Chat脚本由一系列expect-send对组成：expect等待输出中输出特定的字符，通常是一个提示符，然后发送特定的响应。例如下面的Chat脚本实现等待标准输出出现Login:字符串，然后发送somebody作为用户名；然后等待Password:提示符，并发出响应sillyme。

所以expect的工作流程是类似聊天的流程：

A跟B说 hello
B发现A跟他说hello，然后就回复hi
然后A XXXXX
然后B 发现A 在说XXXXX，所以就回复OOOOO
.......

理解的话可以这样理解，虽然不够完整，但不失其意义。

然后既然知道了expect是怎么起作用的，那么的话就可以构思我们的自动化管理设计了，因为expect的设计原理就是为了去处理“交互式”，把“交互式”处理之后，人为的干预就少了,自然就实现自动化了。

一般的expect 使用

#!/usr/bin/expect

set timeout 5

spawn ssh 192.168.6.136 -p 1024

expect "password" {send "123passwd\n"}

expect  "Last login" {send " ifconfig |grep eth0 -A3\n"}

expect eof
exit

#!/usr/bin/expect是调用expect的写法，这个跟一般的shell 写 #!/bin/bash是不同的，这里的意义是以下内容是以什么方式运行，写expect就是expect的方式，写bash就是bash。
spawn是创建一个进程，就是使用expect的时候是要运行expect进程的，spwan就是代表需要创建这样的进程的意思，理解为create也可以，这里就是创建一个ssh 连接的进程，后面的写法跟一般ssh连接执行命令无异。
timeout 表示这个expect动作的生存时间，根据我的理解，例如设置为5秒，那么执行一次expect后就要等待5秒。
expect eof和exit是指监测到eof就会执行exit，退出程序。

高端一点点可以改为......

我们观察一般的ssh正常交互会有哪些情况，首次连接提示，成功连接后会生成knowhost，以后就不会提示了。

ssh 192.168.6.136 -p 1024
The authenticity of host '[192.168.6.136]:1024 ([192.168.6.136]:1024)' can't be established.
RSA key fingerprint is 7d:68:97:bc:f8:c1:b7:8a:a9:98:5a:03:4a:77:b9:eb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[192.168.6.136]:1024' (RSA) to the list of known hosts.
root@192.168.6.136's password:

正常连接提示：

ssh 192.168.6.136 -p 1024
root@192.168.6.136's password:

连接被拒绝，可能是ssh没开，或者端口不对，或者iptables限制：

ssh 192.168.6.136 
ssh: connect to host 192.168.6.136 port 22: Connection refused

没有连接地址：

ssh sadas 
ssh: Could not resolve hostname sadas: Name or service not known

所以可以改成这样：

#!/usr/bin/expect

set timeout 5

spawn ssh 192.168.6.136 -p 1024

expect {
    "Connection refused" exit
    "Name or service not known" exit
    "continue connecting" {send "yes\r";exp_continue}
    "password:" {send "123passwd\r";exp_continue}
    "Last login" {send " ifconfig |grep eth0 -A3\n"}
}

expect eof
exit

将所以的expect收集为一个，然后使用类似switch-case的模式，匹配哪个就触发哪个，并且需要执行下一步动作的则需要加上exp_continue，其实这里就跟普通程序里面的控制循环的continue是一样的用法的。

这是执行结果：

[root@localhost test_shell_expect]# ./test3.sh  
spawn ssh 192.168.6.136 -p 1024
root@192.168.6.136's password: 
Last login: Wed Feb 25 07:07:42 2015 from 192.168.6.127
ifconfig |grep eth0 -A3
[root@wohost ~]#  ifconfig |grep eth0 -A3
eth0    Link encap:Ethernet  HWaddr 00:0C:29:DE:E9:90  
        inet addr:192.168.6.136  Bcast:192.168.6.255  Mask:255.255.255.0
        inet6 addr: fe80::20c:29ff:fede:e990/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

再高端一点点可以这样，支持变量定义和传参功能

#!/usr/bin/expect 

set timeout 5

set pw "123passwd"
set host [lindex $argv 0]

spawn ssh $host -p 1024

expect {
    "Connection refused" exit
    "Name or service not known" exit
    "continue connecting" {send "yes\r";exp_continue}
    "password:" {send "$pw\r";exp_continue}
    "Last login" {send " ifconfig |grep eth0 -A3\n"}
} 

expect eof
exit

set就是用来变量定义的，而传参的话就是使用一个lindex $argv 0 的方式，$argv是指参数项数组，lindex将参数项数组的列表生成出来，然后 0 代表的是使用第一个值，不过这里有个小疑问我还没有完全理解，就是参考debug模式可以看到argv[0] = /usr/bin/expect argv[1] = -d argv[2] = ./test3.sh argv[3] = 192.168.6.136 ，第一个值应该argv[0] = /usr/bin/expect才对，但是程序能够获取到192.168.6.136，我暂时的理解就是他读取的是我执行命令的第一个参数，例如./test3.sh 192.168.6.136，所以第一个参数就是192.168.6.136，如此类推。

效果：

./test3.sh 192.168.6.136

spawn ssh 192.168.6.136 -p 1024
root@192.168.6.136's password: 
Last login: Wed Feb 25 07:11:17 2015 from 192.168.6.127
 ifconfig |grep eth0 -A3
[root@wohost ~]#  ifconfig |grep eth0 -A3
eth0    Link encap:Ethernet  HWaddr 00:0C:29:DE:E9:90  
        inet addr:192.168.6.136  Bcast:192.168.6.255  Mask:255.255.255.0
        inet6 addr: fe80::20c:29ff:fede:e990/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

然后配合shell做个循环就可以简单实现批量管理

[root@localhost test_shell_expect]# cat test2.sh 
#!/bin/bash

while read host
do           
    ./test1.exp $host 
done <file.txt  


[root@localhost test_shell_expect]# cat 
file.txt   log        test1.exp  test2.sh   test3.sh   
[root@localhost test_shell_expect]# cat file.txt 
192.168.6.136
192.168.6.127

大功告成。

troubleshooting

打开debug模式，使用-d，可以方便调试并且观看expect的执行过程。

!/usr/bin/expect -d

输出效果如下：

./test3.sh 192.168.6.136
expect version 5.44.1.15
argv[0] = /usr/bin/expect  argv[1] = -d  argv[2] = ./test3.sh  argv[3] = 192.168.6.136  
set argc 1
set argv0 "./test3.sh"
set argv "192.168.6.136"
executing commands from command file ./test3.sh
spawn ssh 192.168.6.136 -p 1024
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {7991}

expect: does "" (spawn_id exp4) match glob pattern "Connection refused"? no
"Name or service not known"? no
"continue connecting"? no
"password:"? no
"Last login"? no
root@192.168.6.136's password: 
expect: does "root@192.168.6.136's password: " (spawn_id exp4) match glob pattern "Connection refused"? no
"Name or service not known"? no
"continue connecting"? no
"password:"? yes
expect: set expect_out(0,string) "password:"
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) "root@192.168.6.136's password:"
send: sending "123passwd\r" to { exp4 }
expect: continuing expect

expect: does " " (spawn_id exp4) match glob pattern "Connection refused"? no
"Name or service not known"? no
"continue connecting"? no
"password:"? no
"Last login"? no


expect: does " \r\n" (spawn_id exp4) match glob pattern "Connection refused"? no
"Name or service not known"? no
"continue connecting"? no
"password:"? no
"Last login"? no
Last login: Wed Feb 25 07:14:06 2015 from 192.168.6.127

expect: does " \r\nLast login: Wed Feb 25 07:14:06 2015 from 192.168.6.127\r\r\n" (spawn_id exp4) match glob pattern "Connection refused"? no
"Name or service not known"? no
"continue connecting"? no
"password:"? no
"Last login"? yes
expect: set expect_out(0,string) "Last login"
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) " \r\nLast login"
send: sending " ifconfig |grep eth0 -A3\n" to { exp4 }
 ifconfig |grep eth0 -A3
[root@wohost ~]#  ifconfig |grep eth0 -A3
eth0  Link encap:Ethernet  HWaddr 00:0C:29:DE:E9:90  
        inet addr:192.168.6.136  Bcast:192.168.6.255  Mask:255.255.255.0
        inet6 addr: fe80::20c:29ff:fede:e990/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

科普时间

1.关于expect的-f 和--

-f 其实可加可不加，因为他只是说是从一个文件读取命令，他是一个可选项，仅在使用#！的时候要，根据我测试，其实不加也可以。

The -f flag prefaces a file from which to read commands from. The flag itself is optional as it is only useful when using the #! notation (see above), so that other arguments may be supplied on the command line. (When using Expectk, this option is specified as -file.)

By default, the command file is read into memory and executed in its entirety. It is occasionally desirable to read files one line at a time. For example, stdin is read this way. In order to force arbitrary files to be handled this way, use the -b flag. (When using Expectk, this option is specified as -buffer.)

--是用来做个限制，限制参数到此为止。也是可加可不加。

may be used to delimit the end of the options. This is useful if you want to pass an option-like argument to your script without it being interpreted by Expect. This can usefully be placed in the #! line to prevent any flag-like interpretation by Expect. For example, the following will leave the original arguments (including the script name) in the variable argv.

#!/usr/local/bin/expect --
Note that the usual getopt(3) and execve(2) conventions must be observed when adding arguments to the #! line.

2.关于send的\r和--

expect的字符处理是没有换行符之类的，所以需要额外加上，\r代表是返回字符，代表输入到此为止，需要返回，其实效果类似按回车，为什么有些地方用\r，有些地方用\n，其实也无妨，只是为了输出格式好看，而\n其实等于了\r\n了，所以会多一个空行。

Sends string to the current process. For example, the command
send "hello world\r"

sends the characters, h e l l o <blank> w o r l d <return> to the current process. (Tcl includes a printf-like command (called format) which can build arbitrarily complex strings.)
Characters are sent immediately although programs with line-buffered input will not read the characters until a return character is sent. A return character is denoted "\r".

而--是强制下一个参数改为字符串来使用，有点类似强制文本化的效果。

The -- flag forces the next argument to be interpreted as a string rather than a flag. Any string can be preceded by "--" whether or not it actually looks like a flag. This provides a reliable mechanism to specify variable strings without being tripped up by those that accidentally look like flags. (All strings starting with "-" are reserved for future options.)

参考引用：

原文链接：http://www.godblessyuan.com/2015/02/25/shell_expect_ssh/

关于shell和expect和ssh

Expect是什么

一般的expect 使用

高端一点点可以改为......

再高端一点点可以这样，支持变量定义和传参功能

troubleshooting

!/usr/bin/expect -d

科普时间

线上猛如虎

引用和评论

小程序开发-梳理登录流程-v1.0

服务器ssh安全

使用SSH命令行远程登录运行在CloudFoundry上的应用

通过 ForwardAgent ，在远程机器上，使用本机 ssh key 进行验证

Ubuntu服务器搭建Tailscale Derp节点

在远程服务器上运行本地 Shell 脚本

自主实现i3WM亮度调节