Fancy tuning interface: hex vs base64

background

As a front-end for you, you may often boast that you are a ruthless API calling machine, adjusting the framework API, and adjusting the server-side API; but the following may shock you, at least it has crashed me for several days.

I received a request from October to synchronize the data of our platform to another platform of the group, how? Through the open API! ! !

How difficult is this open API to adjust, let me describe it roughly

• Symmetric encryption, the other party's platform will give me a pair of secret keys; it seems that all APIs will do this, after all, safety first!!!
• MD5 calculation, oh no, it is converted to base64 after MD5 calculation; this is actually normal, in order to prevent tampering of data transmission, it is also very common to request bodymd5 encryption;
• Request header summary endorsement, what the hell is the endorsement? What the hell is the summary? Yes, it's for safety, and the gameplay is very complicated, I haven't seen it before;
• Only the online environment is adjustable, but not offline, what? ? ; But the most difficult thing is that the environment is isolated, that is, it cannot be called across the computer room. The vernacular is that the offline computer room cannot be transferred to the online computer room. Impossible), what does this indicate?

It indicates that hot updates are unavailable, and local adjustments are not feasible? To change a line of code, you need to deploy it once (> 5min) in the pre-release environment (the pre-release environment belongs to the online computer room), then adjust the newly deployed interface and find that it is wrong, then change the code, deploy, and adjust again, constantly repeat! ! ! Seriously? ? ?

Maybe you want to know now, which platform provider provides this API, let me tell you quietly, forget it, you should watch it yourself: https://help.aliyun.com/document_detail/29470.html

Fancy Tuning Interface

Postman is a good tool, but sometimes postman can also be inadequate (the same goes for postwoman), such as the call in the computer room that I will talk about today.
Therefore, the moves mentioned below may not be met or used in conventional front-end development, but it is not necessary to take a look at it, just in case you make a profit.

Move 1: Dynamic parameter transfer

Usually we can't call the server API interface, usually there are only two cases:

• The server is stupid and wrote an interface that cannot be tuned at all (I don't exist here, because the legend is connected, at least he proved to me that the gateway is connected);
  
• The front end is stupid, the parameters did not follow the instructions

Therefore, by default, I follow my stupid plan, that is, the input parameters of the server are dynamically uploaded through my interface. I will change what is wrong. It may sound a little awkward. I draw a picture: a bit sloppy .

But things were not as simple as I thought. I was really stunned, because this interface is far from being as simple as being able to call it through with the correct parameters.
Because I still have to calculate the md5 correctly, set the request header correctly, and have the correct signature;

And these request headers also need to be changed dynamically, but at this time, my inspiration suddenly appeared: What is the difference between the above solution to prevent yourself from being stupid, and the interface proxy, so I suddenly had a plan 2: interface proxy

Scenario 2: Interface proxy

Proxies and front-ends may be familiar. The webpack-dev-server has this function, which is mostly used to solve the problem of cross-domain interface. So, why do I need to pass parameters dynamically? It is not easier for me to directly add a proxy, so the plan is like this:

Two lines of code are done:

// plugin.js加入插件
export const httpProxy = {
  enable: true,
  package: 'egg-http-proxy',
};

// config.js 加入配置
  config.httpProxy = {
    '/aapi': {
      target: 'http://api-gateway.test.com',
      pathRewrite: {'^/aoneapi' : ''}
    }
  };

This solution sounds very good, hot updates are available, offline debugging is available; but because there are some middleware at the bottom of our platform, when the target platform responds with 400 or 401, our platform (node-server) will intercept it. A 302 redirect is returned, which will cause me to fail to see the real error response of the target platform, but the changes to the underlying interception are great, and it will also affect the development of other colleagues.

Solution 3: Directly adjust the remote server

In view of the previous curl attempts, it is proved that the most direct interface call and the shortest link can avoid the least errors

curl -v -X GET http://test.goaway.com/checkpreload -H 'Authorization: APPCODE f7f526fd3adf2f38d46'

Because NodeJs is a scripting language, it is different from a compiled language like Java. At this time, the benefits are reflected. Considering the source code on the server, you can directly invoke the interface call through the node command, so now the link becomes like this:

I added a file (test.js) to my warehouse, pseudo code:

const rp = require('request-promise');
const { createHash, createHmac } = require('crypto');

async function handle(str = []) {
  // 省去具体实现
}

// 获取请求相关数据
const arg = process.argv.slice(ArgStart);
handle(arg);

So when I deploy the code, I can run node test.js..., like:

When I execute it this way, I can customize the response body, and I can see whether the request header meets the specification. When an error occurs, I can clearly see the error response; most importantly, when I know the error, I can edit the code directly on the server, and then Then run the command test, so time is greatly saved. All of this is achieved thanks to a scripting language called NodeJs!!!

Share a knowledge: md5

This time debugging, my stuck point can be roughly entangled in four stages:

• The computer room is isolated and the network is unavailable
• Invalid md5
• Invalid signalture
• Authentication error since of operator not configured according to ak

The card with the longest time is Invalid md5; since the SDK was officially provided at the beginning, the md5 calculation method is provided in it:

static getContentMD5(body) {
  const hash = crypto_1.createHash('md5');
  hash.update(body);
  return hash.digest('hex');
}

But the problem lies in this. The MD5 calculation is inconsistent with the official document:

It can be seen that after the md5 calculation, the base64 method is used for encoding.
The official SDK provides only the following modifications to obtain consistent results:

static getContentMD5(body) {
  const hash = crypto_1.createHash('md5');
  hash.update(body);
  return hash.digest('base64');
}

So, what is the secret of the digest method?

Digest into the ginseng

NodeJs official document

In the version of crypto:0.1.92, the input typescript is defined in this way;

type BinaryToTextEncoding = "base64" | "base64url" | "hex"

In addition to the above three parameters (in version 0.1.94, there is no base64url option), in fact, this input parameter can also be empty. If it is empty, it returns Buffer, otherwise it returns string. All input parameters in daily scenes exist, because It doesn’t make much sense for us to hold a Buffer. When doing md5 calculations for hello world, the buffer we get is: <Buffer fc 3f f9 8e 8c 6a 0d 30 87 d5 15 c0 47 3f 86 77>
Then why not use Buffer directly, because the object is not conducive to transmission!!!

base64 vs hex

It can also be understood as base64 vs base16;
So when we pass in hex, the result is: fc3ff98e8c6a0d3087d515c0473f8677, which happens to be the above buffer stringification.

When base64 is passed in, the result is: /D/5joxqDTCH1RXARz+Gdw==
Because the Buffer itself is composed of a string of hexadecimal numbers, it is easy to convert to hex, and to base64, you need to re-encode

Regarding base64 encoding: not all characters transmitted on the network are printable characters, such as binary files, pictures, etc. The emergence of Base64 is to solve this problem. It is a method of representing binary data based on 64 printable characters (AZ, az, 0-9, +, /); transcoding process: first convert the data to be converted The characters are divided into groups of three bytes, each byte occupies 8 bits, so there are 24 binary bits, and then each group of 6 binary bits is divided into 4 groups. Add two 0s in front of each group, each group changes from 6 to 8 binary bits, a total of 32 binary bits, that is, four bytes. Finally, the corresponding value is obtained according to the Base64 encoding comparison table (the case of insufficient number of bits: when the number of bits is converted from 8bit to 6bit, the incomplete 6bit is filled with 0, and the 8bit without corresponding 6bit is directly filled with =
Here is a rough transformation, the hex result is 32 characters, each character is 4 binary numbers

第一步: 转化为二进制数：1111-1010-0011.....-0111-0111(32 * 4 = 128)

第二步：转化为6个bit为一小组，然后三个一大组： 111110-100011-.....-011101-11（7 * 3 * 6 + 2）

第三步：补0，补=：00111110-00100011-....-00011101-00110000-=-=
 
第四步：对照表转字符：/-D-....-d-w-=-=

At least the beginning and the end are consistent, and the routine should be fine.
As for when to use base64 and when to use base16; for md5 calculations, it depends on the mood of the server! But for the transmission of big byte streams, such as files and pictures, base64 will be used because it can save traffic and improve transmission efficiency.

So what is base64Url? base64Url is also known as secure Base64, and the difference from base64 is only the conversion between the 63rd and 64th characters;

Since "/", "=" etc. are reserved characters or unsafe characters in the URL, if Base64 encoding is directly transmitted in the URL, the reserved characters and unsafe characters will be replaced with the form of %XX, for the backend Decoding is not convenient. If it is not replaced, it will cause URL injection vulnerabilities. So the purpose of base64Url is to solve the transmission problem of +, \, = in url.

end

Writing an article should also have a sense of ritual, and it can't be overkill. Here is the end with a sigh: it's so difficult to be a front end! ! !

Welcome to follow my front-end public : 161a5d346f2408 Front-end black hole