Sed and AWK

本文来自于 http://dongweiming.github.io/sed_and_awk/#/3，我只是弄完过来自己记录，然后方便以后查找自己看的。

Sed

语法

sed [options] {sed-commands} {input-file}

示例

-n表示取消默认输出,p表示打印行

$sed -n 'p' /etc/passwd

只打印第三行

$sed -n '3p' /etc/passwd

打印1，3行

$sed -n '1,3p' /etc/passwd

\l能将后面的一个字符变成小写

$sed 's/Ian/IAN/' source.txt|head -1               
101,IAN Bicking,Mozilla
$/usr/local/bin/sed 's/Ian/IA\lN/' source.txt|head -1 
101,IAn Bicking,Mozilla

\L能将后面的字符都变成小写

$/usr/local/bin/sed 's/Ian/I\LAN/' source.txt|head -1
101,Ian Bicking,Mozilla

\u能将后面的一个字符变成大写

$/usr/local/bin/sed 's/Ian/IA\un/' source.txt|head -1
101,IAN Bicking,Mozilla

\U能将后面的字都变成大写

$/usr/local/bin/sed 's/Ian/\Uian/' source.txt|head -1 
101,IAN Bicking,Mozilla

\E能打断\L或者\U改变大小写

$/usr/local/bin/sed 's/Ian/\Uia\En/' source.txt|head -1
101,IAn Bicking,Mozilla

使用以上功能:调换前2列,把名字列全部大写,公司列全部小写

$/usr/local/bin/sed 's/\([^,]*\),\([^,]*\),\(.*\).*/\U\2\E,\1,\L\3/g' source.txt
IAN BICKING,101,mozilla
HAKIM EL HATTAB,102,whim
PAUL IRISH,103,google
ADDY OSMANI,104,google
CHRIS WANSTRATH,105,github
MATTT THOMPSON,106,heroku
ASK SOLEM HOEL,107,vmware

SED可执行脚本

$cat testscript.sed
#!/usr/bin/sed -nf
/root/ p
/nobody/ p
$chmod u+x testscript.sed
$./testscript.sed /etc/passwd 
nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false
root:*:0:0:System Administrator:/var/root:/bin/sh
daemon:*:1:1:System Services:/var/root:/usr/bin/false
_cvmsroot:*:212:212:CVMS Root:/var/empty:/usr/bin/false

SED修改源文件和备份

-i会修改源文件,但是可以同时使用bak备份

$sed -ibak 's/Ian/IAN/' source.txt 

# or
sed --in-place=bak 's/Ian/IAN/' source.txt 
# 这样会存在一个文件source.txtbak

行后增加语法

sed '[address] a the-line-to-append' input-file

示例

$/usr/local/bin/sed '2 a 108,Donald Stufft, Nebula' source.txt
101,IAN Bicking,Mozilla
102,Hakim El Hattab,Whim
108,Donald Stufft, Nebula
103,Paul Irish,Google
104,Addy Osmani,Google
105,Chris Wanstrath,Github
106,Mattt Thompson,Heroku
107,Ask Solem Hoel,VMware                       

行前插入语法

sed '[address] i the-line-to-insert' input-file

示例

$/usr/local/bin/sed '2 i 108,Donald Stufft, Nebula' source.txt
101,IAN Bicking,Mozilla
108,Donald Stufft, Nebula
102,Hakim El Hattab,Whim
103,Paul Irish,Google
104,Addy Osmani,Google
105,Chris Wanstrath,Github
106,Mattt Thompson,Heroku
107,Ask Solem Hoel,VMware

修改行语法

sed '[address] c the-line-to-insert' input-file

示例

# 修改符合Paul行为...
$/usr/local/bin/sed '/Paul/ c 108,Donald Stufft, Nebula' source.txt
101,IAN Bicking,Mozilla
102,Hakim El Hattab,Whim
108,Donald Stufft, Nebula
104,Addy Osmani,Google
105,Chris Wanstrath,Github
106,Mattt Thompson,Heroku
107,Ask Solem Hoel,VMware

SED其他

# -l会显示隐藏字符比如'\t', = 可以显示行号
$sed -l = source.txt
# y或翻译你要转换的字符,这里I会转化成i，B转换成b
$sed 'y/IB/ib/' source.txt |head -1
101,iAN bicking,Mozilla

SED高级话题

保持空间和模式空间

#先看一个文件
$cat source2.txt 
Ian Bicking
Mozilla
Hakim El Hattab
Whim
Paul Irish
Google
Chris Wanstrath
Github
Mattt Thompson
Heroku                          

示例

# 把mac地址的冒号替换掉
"130531170341903612","259594",2013-05-31T09:04:25Z,"1c:b0:94:b2:85:bd"

sed "s/\([0-9a-zA-Z]\{2\}\):\([0-9a-zA-Z]\{2\}\):\([0-9a-zA-Z]\{2\}\):\([0-9a-zA-Z]\{2\}\):\([0-9a-zA-Z]\{2\}\):\([0-9a-zA-Z]\{2\}\)/\1\2\3\4\5\6/"

sed "s/\(.*\):\(.*\):\(.*\):\(.*\):\(.*\):\(.*\)/\1\2\3\4\5\6/"

sed -e "s/T\(.*\):\(.*\):\(.*\)Z/\1#\2#\3/" -e "s/://g" -e "s/#/:/g"

注：其他 AWk 部分还是看原作者的 PPT 吧，更加有感觉。