Data transmission POST mind sharing, can't you solve this bug as a front-end?

background

The grape that provides technical support to everyone anytime, anywhere is here again. What happened this time is like this. Providing demo is a routine operation, but two days ago, the customer suddenly reported that the compression transmission module threw an exception. The specific situation was that the compressed content could not be decompressed after being transmitted to the server.

Since the code has not changed, the front-end related dependencies have not been upgraded, and the server-side java version has not changed, we can infer that it is an environmental problem; after further careful inspection, after repeated comparisons, it is suddenly found that the compressed content received by the server has become longer. ; Compared with the front-end request content, it is found that all \r and \n have become \r\n.

Based on the above analysis, our preliminary judgment is that the problem lies in the browser translation. In order to verify whether the conjecture is correct, Grape rolled back the chrome version to version 92, the exception disappeared, and the content received by the server was not replaced.

The problem is solved smoothly, but the content of Chrome POST data actually changes during transmission. Grape, who has always been good at big front-end technology, will never admit defeat. In order to understand the reason, let's take a look at the details of POST operations.
Control characters First we need to figure out the meaning of several control characters.

• Carriage Return (CR) and Line Feed (LF) are control characters or bytecodes used by text files to mark newlines.
• CR = Carriage Return, the carriage return symbol (\r, hexadecimal ascii code is 0x0D, decimal ascii code is 13), used to move the mouse to the beginning of the line without advancing to the next line.
• LF = Line Feed, line feed (\n, 0x0A in hexadecimal ascii, 10 in decimal).
  Immediately after CR and LF (composed of CRLF, \r\n, or 0x0D0A in hex) moves the mouse to the beginning of the next line. (The default text line break in Windows operating systems is CRLF; Linux and macOS systems use LF by default, and early mac os systems use CR line breaks.)

In code management, CRLF can be very different under different operating systems. The following is a practical demonstration for everyone in different systems:
When creating a new document in Mac Visual Code, the default is LF, while it is CRLF in Windows. You can choose to switch the content type of the line ending sequence.

Visual Code for Mac

Windows version

Faced with this situation, developers need to unify CRLF to avoid confusion in code management caused by the development of different operating systems.

Data changes transmitted by POST

After understanding the reasons why control characters appear in different systems, we need to figure out why the POST data has changed during transmission.

Let's write a simple demo to test it. First put a textarea that allows line breaks on the page, enter text with line breaks, and get the content to see that only \n is translated. Directly post data to the server through FormData, then return directly, and see that all \n become \r\n.

 var uploadData = document.getElementById("ta").value
            var formData = new FormData();
            formData.append("data", uploadData)
            fetch("http://localhost:8088/spread/getpdf", {
                    body: formData,
                    method: "POST"
                }).then(resp => resp.text())
                .then(text => {
                    console.log(JSON.stringify(text));
                    document.getElementById("result").innerHTML= JSON.stringify(text)
                })

Browser ID:

 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36

Roll back Chrome to version 92, and send and receive texts are now coded in the same way:

 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36

Digging deeper into this reason, we learn that Internet Request for Comments 2046 (RFC 2046) 4.1.1. clearly states:

 “ The canonical form of any MIME‘text’ subtype MUST always represent a
   line break as a CRLF sequence. “

Here we can see that all text types must use CRLF, and Chrome just fixed a "bug". For users, users cannot perceive the difference between CR, LF and CRLF in normal text, but when using scene transition It becomes very important to decompress the text content.

three solutions

We all know that POST is a common method of HTTP, and another method that we commonly use is GET.
The difference between GET and POST and related issues will not be repeated here. To solve the problem of data changes transmitted by POTS, the most relevant is Content-Type.

First, let's understand what Content-Type and MIME are:
Content-Type, content type, generally refers to the Content-Type that exists in the web page, which is used to define the type of network file and the encoding of the web page, and decide what form and encoding the browser will read the file in. This is often seen The result of some Asp web page clicks is the reason for downloading a file or a picture.

Commonly used Content-Types in POST are application/x-www-form-urlencoded, multipart/form-data and application/json.

1. application/x-www-form-urlencoded
After submitting the required content to the form, the content will be encoded in the way of name1=value1&name2=value2, and both key and value will be URL transcoded.
For "\n" and "\r", they will be transcoded into '%0A' and '%0D'. Through this transmission method, the browser's CRLF correction can be avoided and the above problems can be solved.
However, this transcoding will increase the length of the text. The original 1 character becomes 3, and the result is that the compressed text becomes longer.

2. Multipart/form-data
When you need to submit files to the server, you need to use this method. In the previous code, we can see that when formData is normal text, it will be corrected. In order to solve this situation, we can encapsulate the string content into Blob as a file stream to avoid correction.
In this way, the server will receive the content as a file and directly read the Stream content; for compressed text, this processing method is optimal.

 var formData = new FormData();
            formData.append("data", uploadData)
            formData.append("data1", new Blob( [uploadData]))

The image above shows the same content, delivered in a different way.

3. application/json
Json is also a popular transmission method at present. The content of json will not be changed during post transmission. If the text content is not long, it is also a good method.

 fetch("http://localhost.charlesproxy.com:8088/spread/postjson", {
                    headers: {
                        'Content-Type': 'application/json',
                    },
                    body: JSON.stringify({data: uploadData}),
                    method: "POST"
                }).then(resp => resp.text())
                .then(text => {
                    console.log(JSON.stringify(text));
                    document.getElementById("result").innerHTML= JSON.stringify(text)
                })

Summarize

As a front-end er, in addition to HTML, CSS, and Javascript, you are proficient in using Axios and other class libraries to call APIs. What is more important is to know how to debug network requests, so that you can quickly locate the problem when a problem occurs in the project.

Here is a Demo that returns a Content-Type of text/html when dynamically loading js files under the Angular framework. If you are interested, you can download and try it yourself.

Demo address: https://gcdn.grapecity.com.cn/forum.php?mod=attachment&aid=NDc5OTJ8YmU1Mjk0NDN8MTY1MDI2MTI0M3wxfDUwOTgw

Extended reading

SpreadJS pure front-end form control official website