Hyperledger based on blockchain technology - from theory to practice

What is blockchain? Simply put, a blockchain is a distributed ledger, or a distributed database.

The data structure of the blockchain is a linked list. The transaction data is stored in the blocks of the linked list. The first block of the blockchain is called the genesis block. In addition to the genesis block, each block also contains the previous block. The hash pointer of the block, the value of this hash pointer is calculated based on the actual data of the previous block. The hash pointer points to the previous block, and the subsequent blocks can find the information of all the previous blocks.

The data structure of the ledger is such a linked list, so what is the meaning of distributed?

Many participants in the blockchain form a loose and autonomous P2P network. We call the participants in the blockchain network a node. Each node has a copy of the ledger, and the information of all the ledger is consistent. There is no central node. Every time a new transaction comes in, the ledgers of all nodes are updated and are eventually consistent. The way to update is not to modify the value of a block, but to save transaction records. For example, in the Bitcoin system, it does not have the concept of user asset records. Unlike ordinary databases, which use a piece of data to store assets, the value of Bitcoin user assets is obtained by concatenating all transaction records. The origin can be traced all the way up until the genesis block. The transaction data in the blockchain can be any information that needs to be recorded according to specific scenarios.

smart contract

In order to support the continuous updating of information and the management of the ledger (writing transactions, making queries, etc.), the blockchain network introduces smart contracts to achieve access and control of the ledger. Smart contracts can not only be used to package information in a blockchain network, they can also be used to automatically execute specific transaction operations defined by participants.

For example, smart contracts can specify the transportation costs in logistics, charge different fees according to the speed of logistics, and perform automatic transfers according to the arrival time of the goods. The smart contract uploaded to the blockchain network will be packaged into a certain block, so once the smart contract is written into the blockchain, it cannot be changed.

consensus mechanism

The process of synchronizing transaction information in a blockchain network ensures that transactions are updated only after they have been approved by the appropriate participants. All participants will update the same information in the same order. This process is called consensus. Consensus mechanism is one of the core of blockchain.

The first application of the blockchain, Bitcoin, uses the Proof of Work consensus mechanism. Briefly introduce the consensus mechanism of Bitcoin. For the specific details of the algorithm, you can check the white paper. After a node receives a transaction, it will verify the validity of the transaction according to the judgment criteria, and invalid transactions will be discarded. Transactions that pass validity verification will be broadcast to other nodes. Other nodes will do the same independent verification. When a valid transaction reaches all nodes in the entire network, the entire network has reached a consensus that "this transaction is valid". Each node will receive many transactions that are valid but have not yet been packaged into blocks. These transactions are assembled into Merkle Tree. The first transaction of Merkle Tree is special, called coinbase, which is created by the node itself and will be rewarded for mining. Payments are made to the miner's own address. The mining reward includes the newly created bitcoin and the total fee for all transactions included in the block. Then the node calculates a hash value that meets the difficulty. Mining is to continuously calculate the block hash value by modifying the parameters until the difficulty requirement is met, which indirectly proves that the node has paid the corresponding workload, which is the workload proof. Another article of the author 300 lines of ABAP code implements a simplest blockchain prototype uses an ABAP method CL_ABAP_MESSAGE_DIGEST=>CALCULATE_HASH_FOR_CHAR to calculate the hash value of the block.

When a node calculates a block hash that meets the difficulty, it means that the miner succeeded in mining, the node assembles the block into the local blockchain, and broadcasts the block to other nodes at the same time. After receiving the block, other nodes will verify whether the block is valid. It is possible that two nodes have mined new blocks B1 and B2 at the same time, and their previous blocks are the same block P. Some nodes may receive B1 first, while others may receive B2 first. At this time, there are two temporary forks in the blockchain. To break this situation, it depends on whether the next block is generated based on B1 or based on B2. If based on B1, the chain of B1 becomes the longest chain, other nodes including B2 will re-select the longest chain, and B2 is discarded as an orphan.

So far, we can think of the blockchain as a shared, decentralized multi-backup system that updates transaction data through smart contracts, while maintaining consistency across all nodes in the network through a consensus-based collaborative process.

Transactions here can refer to any data, such as digital currency, contracts, records, or any other information.

Types of Blockchains

• Public chain: Nodes in the network can be accessed arbitrarily, data read and write permissions in the network are not restricted, and all nodes participate in the consensus process. Digital currencies such as Bitcoin and Ethereum all belong to the public chain.
• Private chain: The nodes in the network are controlled by an organization, which exclusively has the write permission of the blockchain. The private chain is not much different from other distributed storage.
• Alliance Chain: Multiple companies or organizations access through authorization, and certain nodes participate in the consensus process. Hyperledger Fabric is a consortium chain.

What is Hyperledger Fabric?

Hyperledger Fabric is one of the Hyperledger projects initiated by the Linux Foundation. Hyperledger Fabric is an open source blockchain-based distributed ledger designed for use in enterprise environments. Hyperledger Fabric can be used in areas such as global supply chain management, financial transactions, asset accounting, human resources, insurance, health, and digital music.

The ledger subsystem (ledger) in Hyperledger Fabric includes two components: world state and transaction log. The worldview records the current state of the ledger at a specific point in time and is a key-value database. The transaction log records all transactions that produce the current value of the world state and is an update history of the world view. The underlying database of the ledger's worldview can be replaced, and you can choose to use levelDB or couchDB.

Hyperledger Fabric is the first blockchain platform to support writing smart contracts in a common language, which can be written in java, nodejs and go languages. Smart contracts in Hyperledger Fabric are also called chain codes.

The biggest difference from other public blockchain platforms is that Hyperledger Fabric is private and requires authorization to access. It has an MSP (Membership Service Provider) module that provides member management services.

CA (Certificate Authority) is responsible for authority management, membership-related certificate management (Enrollment Certificate Authority) and maintenance transaction-related certificate management (Transaction Certificate Authority) and so on.

Hyperledger Fabric provides the ability to create channels, which allow participants to create a separate ledger for transactions. This feature becomes especially important when some of the participants in the network are competitors. Because these participants do not want all transaction information - such as specific price information provided to some customers - to be open to all participants in the network. Only participants in the same channel will own the ledger in the channel, and other participants who are not in the channel cannot see the ledger.

Hyperledger Fabric uses an independent ordering node (order) to provide consensus services, which are responsible for ordering transactions and providing a globally confirmed transaction order.

Applications access Hyperledger Fabric through the SDK.

In the design of the latest version of Hyperledger Fabric, node roles are decoupled according to their functions, allowing different nodes to handle different types of workloads. From the business logic, the nodes are divided into endorsement nodes (Endorser) and commit nodes (Committer).

• Endorser peer: Responsible for checking the validity and authority of the transaction from the client (simulated transaction), signing the signature and returning the result to the client after passing the check.
• Committer peer: responsible for maintaining the ledger, checking the status of the batch transaction results in the agreed sequence, generating blocks, executing legal transactions, and writing them into the ledger. The same physical node can act as both an endorser and a committer.

Hyperledger Fabric Transaction Flow: Consensus

The consensus process is mainly divided into three stages: Proposal, Packaging and Validation.

Proposal

The application submits a transaction proposal, and then submits it to all the endorsing nodes. After receiving it, the latter uses it as an input to execute the chaincode to generate the corresponding transaction proposal response. Instead of updating the Ledger at this point, the transaction proposal response is signed and returned to the application. When the application receives the signed response, the first phase of the consensus process is complete.

Packaging

In this stage, the order node sorts and packs the transaction. The Order node receives transaction proposal responses from various applications, then sorts these transactions, and packs them into blocks after sorting.

Validation

In the last stage of the consensus process, the order node distributes the block to all nodes connected to it. These nodes will confirm that the transactions in the block are signed by the endorsing nodes, and then update the confirmed block to the ledger.

The whole process is called consensus, all nodes have agreed on the transaction content and order, and this process is controlled by the order node. Consensus is a multi-step process, and the ledger is only updated when the entire process is complete, and each node may have slightly different update times.

Building a Hyperledger Fabric platform is no easy task, requiring both hardware infrastructure investment and full development and operations management (DevOps). In addition to the platform itself, a complete solution also includes management functions such as device access, access control, and service monitoring.

SAP Cloud Platform (hereinafter referred to as SCP) provides out-of-the-box Hyperledger Fabric Service, providing developers with powerful service support:

• The intuitive and friendly visual monitoring and operation interface helps developers apply for blockchain networks on demand and create management nodes and channels without considering the underlying hardware resources.
• Simple and easy-to-use smart contract development and testing environment, which is convenient for developers to manage application code.
• Security, privacy protection, and performance optimization of related resources.

The following are the specific practical steps. This step uses the go language to develop a set of microservices. This set of microservices includes two APIs, read and write, which can write data into the hyperledger service built on the SAP cloud platform.

Simply put, the application reads and writes to and from the Hyperledger through the smart contract interface. We will develop a Hello World smart contract and deploy it on the SAP cloud platform. For the sake of simplicity, we did not develop an application, but simply consumed the Hello World smart contract directly on the API console of the SAP cloud platform to read and write to the hyperledger on the cloud platform.

Open the github repository address of the Hyperledger project Fabric:

https://github.com/hyperledger/fabric

It is found that the Fabric project is developed by Google's programming language GoLang, so we also use the Go language to develop smart contracts in this exercise.

1. Download the binary package of Go language version 1.11 from the Google website to the local, and extract it to the /usr/local directory:

sudo tar -C /usr/local -xzf /home/vagrant/Downloads/go1.11.linux-amd64.tar.gz

Configure this directory to the environment variable PATH:

2. The Fabric project has encapsulated the communication between smart contracts and Hyperledger into an interface called shim. We only need to call the shim interface directly in the smart contract code we write.

We use import to introduce the dependencies of this shim interface, and define a structure on line 14, which contains two fields of ID and Value. This structure is the data structure to be written into the hyperledger, and the ABAP consultant can regard it as a structure defined in the ABAP data dictionary.

The method Invoke defined in line 46 is the core code of the simplest smart contract. The syntax of cc *MessageStore is very similar to the C language, and a pointer variable cc of type MessageStore is defined. This pointer variable is similar to the this pointer of C++ and the me reference of ABAP. When the method is called, it points to the caller of the method.

The stub shim.ChaincodeStubInterface in parentheses after Invoke defines the input parameter (formal parameter) stub of the method, which is of type shim.ChaincodeStubInterface.

This Invoke method is not called explicitly by the application, but is called back by the Hyperledger program: when the method is called, the pointer cc and the input parameter stub have been automatically assigned corresponding values by the Fabric framework. In the context of the Invoke method running, the input parameter stub is used to determine whether the current callback scene is read or write, and then enter the corresponding branch. The branch internally calls the write and read methods developed by ourselves to interact with the hyperledger.

This approach of handling multiple scenarios through the switch case inside the same callback function should be familiar to ABAP and Java developers. For example, the following figure is an example of implementing Java dynamic proxy through InvocationHandler. The logical structure of the invoke method is very similar to the structure of the smart contract code in this article.

For the writing of various static agents and dynamic agents in ABAP and Java, please refer to my blog:

Various Proxy Design Pattern implementation variants in Java, ABAP and JavaScript

3. Build the developed smart contract source file into an executable file. This step ensures that all potential errors are detected and fixed in the local development environment before deploying the smart contract to the SAP cloud platform.

4. Log in to the SAP Cloud Platform and click on the Hyperledger Fabric hyperlink in the Service Marketplace:

Create a new Service instance:

During the creation process, you need to fill in the channel ID and key.

Blockchains are divided into public chains, private chains and consortium chains, while Hyperledger belongs to consortium chains. In the alliance chain, there is a special module called MSP (Membership Service Provider), which provides member management services, and only authorized users can access the blockchain network. Here I created a channel on the SAP cloud platform in advance and authenticated it, so directly enter a legal channel ID and key here. For the steps of creating a Hyperledger channel on the SAP cloud platform and authorizing member access, please refer to this SAP help document .

After the Service instance is created, click the Create Service Key button to create the key. The purpose is to generate the clientId and clientSecret for OAuth authentication, which is convenient for subsequent API calls.

Click on the Referencing Apps panel of the Service instance and click the button Open Dashboard:

Click Deploy Chaincode, select the locally built zip package, upload and deploy. This button is the same logic as the SAP cloud platform CloudFoundry environment to deploy the local application.

After successful deployment, click the Test Chaincode hyperlink to enter the API console.

The console integrates the Swagger framework. Before calling the post request to perform the write operation of the hyperledger and the get request to perform the read operation, you must first click the Authorize button for authentication:

Enter the clientID and clientSecret generated after creating the Service Key in step 4 for authentication:

After successful authentication, you can call post and get requests in the Swagger console.

First send a post request, the request payload is a simple json object, the id is i042416, and the value is Hello World:

The post request is successfully executed on the Hyperledger on the SAP cloud platform, and the 200 response code is returned:

Then execute the get request and enter the data id just written: i042416:

The get request can successfully read the data previously written to the ledger through the post request:

Log in to the Hyperledger console of the SAP cloud platform, and you can see that the data written through post has been added to the block at the end of the blockchain. Click on the block to view the data details:

Details of each Hyperledger read and write can also be seen in the API Calls and Logs panels of the Hyperledger console.

Summarize

The Hyperledger service of the SAP cloud platform has successfully helped companies wishing to use this blockchain technology to avoid the investment of hardware infrastructure, while shielding most of the underlying details of Hyperledger platform management. Through the console provided by the SAP cloud platform, management functions such as device access, access control, and service monitoring for Hyperledger can be realized. At the same time, once the smart contract written in Go language is deployed to the SAP cloud platform, the generated Restful API can be easily consumed by other programming languages. Data written to the Hyperledger blockchain by calling these APIs can no longer be tampered with. Using the Hyperledger service of the SAP cloud platform, application developers can focus on writing application logic without spending too much energy on the Hyperledger architecture itself.