Potential risks of defer Close() in Golang

As a Gopher, we can easily form a programming convention: whenever there is an object x io.Closer interface, after getting the object and checking for errors, we will immediately use defer x.Close() to ensure that the x object is closed when the function returns. Two examples of idiomatic writing are given below.

• HTTP request

resp, err := http.Get("https://golang.google.cn/")
if err != nil {
    return err
}
defer resp.Body.Close()
// The following code: handle resp

• Access file

f, err := os.Open("/home/golangshare/gopher.txt")
if err != nil {
    return err
}
defer f.Close()
// The following code: handle f

There is a problem

In fact, this way of writing has potential problems. defer x.Close() will ignore its return value, but when executing x.Close() , we cannot guarantee that x will be shut down normally. What if it returns an error? This way of writing makes it possible for the program to have errors that are very difficult to troubleshoot.

So, what error will the Close() In POSIX operating systems, such as Linux or maxOS, the Close() function that closes the file finally calls the system method close() . We can man close manual to see what error close()

ERRORS
     The close() system call will fail if:

     [EBADF]            fildes is not a valid, active file descriptor.

     [EINTR]            Its execution was interrupted by a signal.

     [EIO]              A previously-uncommitted write(2) encountered an
                        input/output error.

Error EBADF means invalid file descriptor fd, which has nothing to do with the situation in this article; EINTR refers to Unix signal interruption; then the possible error in this article is EIO .

EIO error means uncommitted read , What is wrong?

EIO error means close() method write() the read of the file 0619b9462af7f7 was submitted.

The figure above is a classic computer memory hierarchy. In this hierarchy, from top to bottom, the access speed of the device is getting slower and slower and the capacity is getting larger and larger. The main idea of the memory hierarchy is that the upper layer of memory serves as the cache of the lower layer of memory.

CPU access to registers is very fast, in contrast, access to RAM will be very slow, and access to disk or network means that it is wasted time. If every write() call submits data to disk synchronously, then the overall performance of the system will be extremely reduced, and our computer will not work like this. When we call write() , the data is not immediately written to the target carrier. Each layer of the computer memory is caching data. At the right time, the data is flushed to the next layer of carrier, which will write the synchronization, The slow, blocking synchronization turns into a fast, asynchronous process.

In this way, the EIO error is indeed an error we need to beware of. This means that if we try to save the data to the disk and the defer x.Close() data to the disk when 0619b9462af849 is executed, then we should get the error message ( disk, then the data is not there. If the persistence is successful, it may be lost. For example, if there is a power outage, this part of the data will disappear forever, and we will not know ). But according to the conventional writing above, our program got the error nil

solution

We will discuss several feasible transformation schemes according to the situation of closing files

• The first solution is not to use defer

func solution01() error {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return err
    }

    if _, err = io.WriteString(f, "hello gopher"); err != nil {
        f.Close()
        return err
    }

    return f.Close()
}

This way of writing requires us to f.Close() to close when the execution of io.WriteString f.Close() needs to be added in every place where an error occurs. If there are f write operations for 0619b9462af8e4, there is a risk of missing the file to be closed.

• The second solution is to handle by naming the return value err and closure

func solution02() (err error) {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return
    }

    defer func() {
        closeErr := f.Close()
        if err == nil {
            err = closeErr
        }
    }()

    _, err = io.WriteString(f, "hello gopher")
    return
}

This solution solves the risk of forgetting to close the file in solution 1. If there are more if err !=nil , this mode can effectively reduce the number of lines of code.

• The third option is to call f.Close() once before the final return statement of the function

func solution03() error {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return err
    }
    defer f.Close()

    if _, err := io.WriteString(f, "hello gopher"); err != nil {
        return err
    }

    if err := f.Close(); err != nil {
        return err
    }
    return nil
}

This solution can in io.WriteString when an error occurs due defer f.Close() exist to get close call. Also in io.WriteString when no error occurred, but the cache is not flushed to disk, get err := f.Close() mistakes, and because defer f.Close() does not return an error, so do not worry about twice Close() call will cover the error.

• The last solution is to execute f.Sync() when the function returns

func solution04() error {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return err
    }
    defer f.Close()

    if _, err = io.WriteString(f, "hello world"); err != nil {
        return err
    }

    return f.Sync()
}

Because calling close() is the last opportunity to get the operating system to return an error, but when we close the file, the cache is not necessarily flushed to the disk. Then, we can call f.Sync() (its internal call system function fsync ) to force the kernel to persist the cache to the disk.

// Sync commits the current contents of the file to stable storage.
// Typically, this means flushing the file system's in-memory copy
// of recently written data to disk.
func (f *File) Sync() error {
    if err := f.checkValid("sync"); err != nil {
        return err
    }
    if e := f.pfd.Fsync(); e != nil {
        return f.wrapErr("sync", e)
    }
    return nil
}

Because fsync call, this model can well avoid close appear EIO . It is foreseeable that, due to compulsory flashing, although this scheme can guarantee data security well, it will greatly reduce the execution efficiency.