Five postures for executing commands with os/exec in Go language

Five postures for executing commands with os/exec in Go language

For the complete series of tutorials, please see: http://golang.iswbm.com

---

The library used to execute commands in Golang is os/exec . The exec.Command function returns a Cmd object. According to different needs, the command execution can be divided into three situations

1. Only execute commands, do not get results
2. Execute the command and get the result (no distinction between stdout and stderr)
3. Execute the command and get the result (distinguish between stdout and stderr)

The first type: only execute the command without obtaining the result #

Calling the Run function of the Cmd object directly, only success and failure are returned, and no output result can be obtained.

package main

import (
    "log"
    "os/exec"
)

func main() {
    cmd := exec.Command("ls", "-l", "/var/log/")
    err := cmd.Run()
    if err != nil {
        log.Fatalf("cmd.Run() failed with %s\n", err)
    }
}

The second type: execute the command and get the result #

Sometimes we execute a command just to get the output result, at this time you can call the CombinedOutput function of Cmd.

package main

import (
"fmt"
"log"
"os/exec"
)

func main() {
    cmd := exec.Command("ls", "-l", "/var/log/")
    out, err := cmd.CombinedOutput()
    if err != nil {
        fmt.Printf("combined out:\n%s\n", string(out))
        log.Fatalf("cmd.Run() failed with %s\n", err)
    }
    fmt.Printf("combined out:\n%s\n", string(out))
}

The CombinedOutput function only returns out, and does not distinguish between stdout and stderr. If you want to distinguish between them, you can directly look at the third method.

$ go run demo.go 
combined out:
total 11540876
-rw-r--r--  2 root       root         4096 Oct 29  2018 yum.log
drwx------  2 root       root           94 Nov  6 05:56 audit
-rw-r--r--  1 root       root    185249234 Nov 28  2019 message
-rw-r--r--  2 root       root        16374 Aug 28 10:13 boot.log

But before that, I found a small problem: Sometimes, shell commands can be executed, not code exec can also be executed.

For example, I only want to view the /var/log/ with the log suffix in the directory 061d460fe5dd7a? Students who are a little bit Linux-based will use this command

$ ls -l /var/log/*.log
total 11540
-rw-r--r--  2 root       root         4096 Oct 29  2018 /var/log/yum.log
-rw-r--r--  2 root       root        16374 Aug 28 10:13 /var/log/boot.log

exec.Command according to this writing method

package main

import (
"fmt"
"log"
"os/exec"
)

func main() {
    cmd := exec.Command("ls", "-l", "/var/log/*.log")
    out, err := cmd.CombinedOutput()
    if err != nil {
        fmt.Printf("combined out:\n%s\n", string(out))
        log.Fatalf("cmd.Run() failed with %s\n", err)
    }
    fmt.Printf("combined out:\n%s\n", string(out))
}

what's the situation? It didn't work, and an error was reported.

$ go run demo.go 
combined out:
ls: cannot access /var/log/*.log: No such file or directory

2020/11/11 19:46:00 cmd.Run() failed with exit status 2
exit status 1

Why did it report an error? There is no problem with the shell

In fact, it is very simple, the original ls -l /var/log/*.log is not equivalent to the following code.

exec.Command("ls", "-l", "/var/log/*.log")

The Shell command corresponding to the above code should be as follows. If you write like this, ls will treat the content in the parameter as a specific file name, and ignore the wildcard *

$ ls -l "/var/log/*.log"
ls: cannot access /var/log/*.log: No such file or directory

The third type: execute commands and distinguish between stdout and stderr #

The above writing method cannot distinguish between standard output and standard error, as long as it is replaced by the following writing method, it can be realized.

package main

import (
    "bytes"
    "fmt"
    "log"
    "os/exec"
)

func main() {
    cmd := exec.Command("ls", "-l", "/var/log/*.log")
    var stdout, stderr bytes.Buffer
    cmd.Stdout = &stdout  // 标准输出
    cmd.Stderr = &stderr  // 标准错误
    err := cmd.Run()
    outStr, errStr := string(stdout.Bytes()), string(stderr.Bytes())
    fmt.Printf("out:\n%s\nerr:\n%s\n", outStr, errStr)
    if err != nil {
        log.Fatalf("cmd.Run() failed with %s\n", err)
    }
}

The output is as follows, you can see that the previous error content is classified into standard error

$ go run demo.go 
out:

err:
ls: cannot access /var/log/*.log: No such file or directory

2020/11/11 19:59:31 cmd.Run() failed with exit status 2
exit status 1

Fourth: Combination of multiple commands, please use pipeline #

Use the execution output result of the previous command as the parameter of the next command. | can be used in Shell.

For example, the following command counts the number of ERROR logs in the message log.

$ grep ERROR /var/log/messages | wc -l
19

Similarly, there is a similar implementation in Golang.

package main
import (
    "os"
    "os/exec"
)
func main() {
    c1 := exec.Command("grep", "ERROR", "/var/log/messages")
    c2 := exec.Command("wc", "-l")
    c2.Stdin, _ = c1.StdoutPipe()
    c2.Stdout = os.Stdout
    _ = c2.Start()
    _ = c1.Run()
    _ = c2.Wait()
}

The output is as follows

$ go run demo.go 
19

Fifth: Set the command-level environment variable #

The environment variables set using the Setenv function of the os library affect the life cycle of the entire process.

package main
import (
    "fmt"
    "log"
    "os"
    "os/exec"
)
func main() {
    os.Setenv("NAME", "wangbm")
    cmd := exec.Command("echo", os.ExpandEnv("$NAME"))
    out, err := cmd.CombinedOutput()
    if err != nil {
        log.Fatalf("cmd.Run() failed with %s\n", err)
    }
    fmt.Printf("%s", out)
}

As long as this process was, NAME value of this variable will be the wangbm , no matter how many times you execute the command

$ go run demo.go 
wangbm

If you want to narrow the scope of environmental variables to the command level, there are ways.

In order to facilitate verification, I created a new sh script with the following content

$ cat /home/wangbm/demo.sh
echo $NAME
$ bash /home/wangbm/demo.sh   # 由于全局环境变量中没有 NAME，所以无输出

In addition, the code in demo.go is as follows

package main
import (
    "fmt"
    "os"
    "os/exec"
)


func ChangeYourCmdEnvironment(cmd * exec.Cmd) error {
    env := os.Environ()
    cmdEnv := []string{}

    for _, e := range env {
        cmdEnv = append(cmdEnv, e)
    }
    cmdEnv = append(cmdEnv, "NAME=wangbm")
    cmd.Env = cmdEnv

    return nil
}

func main() {
    cmd1 := exec.Command("bash", "/home/wangbm/demo.sh")
  ChangeYourCmdEnvironment(cmd1) // 添加环境变量到 cmd1 命令: NAME=wangbm
    out1, _ := cmd1.CombinedOutput()
    fmt.Printf("output: %s", out1)

    cmd2 := exec.Command("bash", "/home/wangbm/demo.sh")
    out2, _ := cmd2.CombinedOutput()
    fmt.Printf("output: %s", out2)
}

After execution, you can see that the second executed command does not output the variable value of NAME.

 go run demo.go 
output: wangbm
output: