1
头图
Pay attention to WeChat public account: K brother crawler, QQ exchange group: 808574309, continue to share advanced crawler, JS/Android reverse engineering and other technical dry goods!

【01x00】 Introduction

The national secret is the domestic encryption algorithm recognized by the National Cryptographic Bureau. When crawling engineers do JS reverse engineering, they will encounter various encryption algorithms. Among them, RSA, AES, SHA and other algorithms are the most common. These algorithms are all foreign , it is also introduced in Brother K’s previous article: 161970a020416e "[Crawler Knowledge] Common Encryption and Decryption Algorithms for

In fact, since 2010, my country’s National Cryptography Administration has begun to publish a series of domestic encryption algorithms, including SM1, SM2, SM3, SM4, SM7, SM9, ZUC (Zu Chongzhi encryption algorithm), etc., SM represents Commercial cipher, namely commercial cipher, refers to the cipher technology used in commerce that does not involve state secrets. SM1 and SM7 are not disclosed , and the other algorithms have become ISO/IEC international standards.

Among these domestic encryption algorithms, the SM2, SM3 and SM4 are the more common ones . When crawling some gov websites, you may also encounter these algorithms, so as a crawler engineer, it is necessary to understand these algorithms. Yes, as shown in the figure below, a gov website uses SM2 and SM4 encryption algorithms:

01.png

[02x00] Algorithm overview

Algorithm nameAlgorithm categoryApplication fieldFeatures
SM1Symmetric (packet) encryption algorithmchipThe packet length and key length are both 128 bits
SM2Asymmetric (based on elliptic curve ECC) encryption algorithmdata encryptionECC elliptic curve cryptography mechanism is 256 bits, which is faster and consumes less than RSA
SM3Hash function algorithmIntegrity checkThe security and efficiency are equivalent to SHA-256, and the compression function is more complicated
SM4Symmetric (packet) encryption algorithmData encryption and LAN productsThe packet length and key length are both 128 bits, and the number of calculation rounds is large
SM7Symmetric (packet) encryption algorithmNon-contact IC cardThe packet length and key length are both 128 bits
SM9Identity Encryption Algorithm (IBE)End-to-end offline secure communicationThe encryption strength is equivalent to the RSA encryption algorithm with a 3072-bit key
ZUCSymmetric (serial) encryption algorithmMobile communication 4G networkStream cipher

[03x00] Detailed algorithm

[03x01] SM1 block encryption algorithm

SM1 is a block encryption algorithm, symmetric encryption, the block length and key length are both 128 bits, so when encrypting and decrypting a message, if the message length is too long, it needs to be grouped, if the message length is not enough, it needs to be filled. The security strength of the algorithm and the performance of related software and hardware are equivalent to AES. The algorithm is not public and only exists in the chip in the form of an IP core. When calling the algorithm, it needs to be called through the interface of the encryption chip. The algorithm has been developed using this algorithm. Series chips, smart IC cards, smart password keys, encryption cards, encryption machines and other security products are widely used in various application fields of e-government, e-commerce and national economy (including important fields such as national government affairs and police affairs). There are relatively few people who know it, and crawler engineers will not encounter this encryption algorithm.

[03x02] SM2 Elliptic Curve Public Key Encryption Algorithm

SM2 is an elliptic curve (ECC) public key encryption algorithm, asymmetric encryption, SM2 algorithm and RSA algorithm are both public key encryption algorithms, SM2 algorithm is a more advanced and secure algorithm, which is used to replace in our country’s commercial cryptosystem RSA algorithm, this type of encryption algorithm can be seen on many gov websites. Research on elliptic curve cryptography by Chinese scholars began in the 1980s and has achieved a lot of results. The SM2 elliptic curve public key cryptography algorithm has the following advantages over the RSA algorithm:

SM2RSA
safety256-bit SM2 strength has exceeded RSA-2048generally
Algorithm structureBasic Elliptic Curve (ECC)Based on a special reversible modular exponentiation operation
Computational complexityFully exponentialSub-exponential
Storage space (key length)192-256 bit2048-4096 bit
Key generation speed100 times faster than RSA algorithmslow
Decryption and encryption speedFastergenerally

[03x03] SM3 hash algorithm

SM3 is a cryptographic hash algorithm, which uses cryptographic hash function standards to replace international algorithms such as MD5/SHA-1/SHA-2. It is an improved algorithm based on SHA-256. The message packet length is 512 bits, the length of the digest value is 256 bits, which uses exclusive OR, modulo, modulo addition, shift, AND, OR, and NOT operations, which are composed of padding, iterative process, message expansion and compression functions. In the commercial cryptosystem, SM3 is mainly used for digital signature and verification, message authentication code generation and verification, and random number generation. According to the National Cryptography Administration, its security and efficiency are higher than the MD5 algorithm and the SHA-1 algorithm, and are comparable to SHA-256.

[03x04] SM4 block encryption algorithm

SM4 is a standard packet encryption algorithm for wireless local area network, symmetric encryption, used to replace international algorithms such as DES/AES, SM4 algorithm and AES algorithm have the same key length and packet length, both are 128 bits, so when encrypting and decrypting messages , If the message length is too long, it needs to be grouped, if the message length is not enough, it needs to be filled. Both the encryption algorithm and the key expansion algorithm use 32 rounds of non-linear iterative structure. The decryption algorithm has the same structure as the encryption algorithm, except that the use order of the round keys is reversed. The decryption round keys are the reverse order of the encryption round keys.

SM4DESAES
Calculate the number of rounds3216 (3DES is 16*3)10/12/14
Password componentS-box, nonlinear transformation, linear transformation, composite transformationStandard arithmetic and logical operations, replacement after replacement, without linear transformationS-box, row shift transformation, column mixing transformation, circle key plus transformation (AddRoundKey)

[03x05] SM7 block encryption algorithm

SM7 is a block encryption algorithm, symmetric encryption, the algorithm is not open, applications include identification applications (non-contact IC cards, access control cards, work permits, entry certificates, etc.), ticketing applications (large event tickets, exhibition tickets, etc.), Payment and card applications (point consumption card, campus card, enterprise card, etc.). Crawler engineers basically do not encounter such algorithms.

[03x06] SM9 logo encryption algorithm

SM9 is an identity-based encryption algorithm (Identity-Based Cryptography), asymmetric encryption, identity encryption uses the user's identity (such as WeChat ID, email address, mobile phone number, QQ number, etc.) as the public key, omitting the process of exchanging digital certificates and public keys , Which makes the security system easy to deploy and manage, and is suitable for the security of various emerging applications of Internet applications, such as cloud-based password services, email security, smart terminal protection, Internet of Things security, cloud storage security, etc. These security applications can use mobile phone numbers or email addresses as public keys to implement data encryption, identity authentication, call encryption, channel encryption, etc. In commercial cryptosystems, SM9 is mainly used for user identity authentication. According to a public report by Xinhuanet, the encryption strength of SM9 is equivalent to the RSA encryption algorithm with a 3072-bit key.

[03x07] ZUC Zu Chong's Algorithm

ZUC is a stream cipher algorithm with symmetric encryption. This confidentiality algorithm can be applied to encryption and decryption in 3GPP LTE communications. The algorithm includes Zu Chongzhi algorithm (ZUC), confidentiality algorithm (128-EEA3) and integrity algorithm (128-EIA3) ) Three parts. It has been recommended by the international organization 3GPP as a candidate algorithm for the third set of international encryption and integrity standards for 4G wireless communications.

[04x00] Programming language implementation

[04x01] Python language implementation

There is no official library in Python to implement the national secret algorithm. Only two of the more complete third-party libraries are listed here. It should be noted that the SM1 and SM7 algorithms are not public. At present, most libraries only implement SM2 and SM2. Three encryption algorithms, SM3 and SM4.

Among them, gmssl-python is an improved version of gmssl, gmssl-python newly supports SM9 algorithm, but as of this writing, gmssl-python has not released pypi, nor PR to gmssl, pip install gmssl installed with 061970a02045c0 does not support SM9 algorithm. If you want to use the SM9 algorithm, you can download the gmssl-python source code and install it manually.

Taking the SM2 algorithm of gmssl as an example, the implementation is as follows (for other algorithms and detailed usage, please refer to its official documents):

SM2 encryption (encrypt) and decryption (decrypt):

from gmssl import sm2


# 16 进制的公钥和私钥
private_key = '00B9AB0B828FF68872F21A837FC303668428DEA11DCD1B24429D0C99E24EED83D5'
public_key = 'B9C9A6E04E9C91F7BA880429273747D7EF5DDEB0BB2FF6317EB00BEF331A83081A6994B8993F3F5D6EADDDB81872266C87C018FB4162F5AF347B483E24620207'
sm2_crypt = sm2.CryptSM2(public_key=public_key, private_key=private_key)

# 待加密数据和加密后数据为 bytes 类型
data = b"this is the data to be encrypted"
enc_data = sm2_crypt.encrypt(data)
dec_data = sm2_crypt.decrypt(enc_data)

print('enc_data: ', enc_data.hex())
print('dec_data: ', dec_data)

# enc_data:  3cb96dd2e0b6c24df8e22a5da3951d061a6ee6ce99f46a446426feca83e501073288b1553ca8d91fad79054e26696a27c982492466dafb5ed06a573fb09947f2aed8dfae243b095ab88115c584bb6f0814efe2f338a00de42b244c99698e81c7913c1d82b7609557677a36681dd10b646229350ad0261b51ca5ed6030d660947

# dec_data:  b'this is the data to be encrypted'

SM2 signature (sign) and verification (verify):

from gmssl import sm2, func


# 16 进制的公钥和私钥
private_key = '00B9AB0B828FF68872F21A837FC303668428DEA11DCD1B24429D0C99E24EED83D5'
public_key = 'B9C9A6E04E9C91F7BA880429273747D7EF5DDEB0BB2FF6317EB00BEF331A83081A6994B8993F3F5D6EADDDB81872266C87C018FB4162F5AF347B483E24620207'
sm2_crypt = sm2.CryptSM2(public_key=public_key, private_key=private_key)

# 待签名数据为 bytes 类型
data = b"this is the data to be signed"
random_hex_str = func.random_hex(sm2_crypt.para_len)

#  16 进制
sign = sm2_crypt.sign(data, random_hex_str)
verify = sm2_crypt.verify(sign, data)

print('sign: ', sign)
print('verify: ', verify)

# sign:  45cfe5306b1a87cf5d0034ef6712babdd1d98547e75bcf89a17f3bcb617150a3f111ab05597601bab8c41e2b980754b74ebe9a169a59db37d549569910ae273a

# verify:  True

[04x02] JavaScript language implementation

There are relatively mature implementation libraries in JavaScript. Here, sm-crypto is recommended. Currently, SM2, SM3 and SM4 are supported. It should be noted that the result of SM2 asymmetric encryption is composed of C1, C2, and C3, where C1 is generated The calculated elliptic curve point of a random number, C2 is the cipher text data, C3 is the digest value of SM3, the initial result of the national secret standard is in the order of C1C2C3, the new standard is stored in the order of C1C3C2, sm-crypto Support setting cipherMode, which is the order of C1C2C3.

sm-crypto:https://www.npmjs.com/package/sm-crypto

Take the SM2 algorithm as an example, the implementation is as follows (for other algorithms and detailed usage, please refer to its official documents):

SM2 encryption (encrypt) and decryption (decrypt):

const sm2 = require('sm-crypto').sm2

// 1 - C1C3C2,0 - C1C2C3,默认为1
const cipherMode = 1

// 获取密钥对
let keypair = sm2.generateKeyPairHex()
let publicKey = keypair.publicKey   // 公钥
let privateKey = keypair.privateKey // 私钥

let msgString = "this is the data to be encrypted"
let encryptData = sm2.doEncrypt(msgString, publicKey, cipherMode)    // 加密结果
let decryptData = sm2.doDecrypt(encryptData, privateKey, cipherMode) // 解密结果

console.log("encryptData: ", encryptData)
console.log("decryptData: ", decryptData)

// encryptData:  ddf261103fae06d0efe20ea0fe0d82bcc170e8efd8eeae24e9559b3835993f0ed2acb8ba6782fc21941ee74ca453d77664a5cb7dbb91517e6a3b0c27db7ce587ae7af54f8df48d7fa822b7062e2af66c112aa57de94d12ba28e5ba96bf4439d299b41da4a5282d054696adc64156d248049d1eb1d0af28d76b542fe8a95d427e

// decryptData:  this is the data to be encrypted

SM2 signature (sign) and verification (verify):

const sm2 = require('sm-crypto').sm2

// 获取密钥对
let keypair = sm2.generateKeyPairHex()
let publicKey = keypair.publicKey   // 公钥
let privateKey = keypair.privateKey // 私钥

// 纯签名 + 生成椭圆曲线点
let msgString = "this is the data to be signed"
let sigValueHex = sm2.doSignature(msgString, privateKey)                    // 签名
let verifyResult = sm2.doVerifySignature(msgString, sigValueHex, publicKey) // 验签结果

console.log("sigValueHex: ", sigValueHex)
console.log("verifyResult: ", verifyResult)

// sigValueHex:  924cbb9f2b5adb554ef77129ff1e3a00b2da42017ad3ec2f806d824a77646987ba8c8c4fb94576c38bc11ae69cc98ebbb40b5d47715171ec7dcea913dfc6ccc1

// verifyResult:  true

[04x03] Other language implementation and reference materials

[05x00] Attachment: GM/T password industry standard


K哥爬虫
166 声望154 粉丝

Python网络爬虫、JS 逆向等相关技术研究与分享。