Deploy RocketMq source code with Idea (4.9.4)

Deploy RocketMq source code with Idea (4.9.4)

1. Introduction

Note-based, Idea deploys RocketMq's simplified process.

https://github.com/apache/rocketmq

2. Tips

2.1 IDEA version

Idea version for personal use.

2.2 RocketMq source code version

Before downloading the source code, please check your own java version, the minimum requirement is JDK1.8 or above.

The personally pulled version is 4.9.4 , because some codes may be improved in future versions due to timeliness, so pay attention to version issues.

 java version "1.8.0_341"
Java(TM) SE Runtime Environment (build 1.8.0_341-b10)
Java HotSpot(TM) 64-Bit Server VM (build 25.341-b10, mixed mode)

Pull the source code, this step does not need too much introduction, just click on the upper right corner of the home page.

Here is the project from the official fork

The wall of github is getting thicker and thicker now, and pulling the code often fails. There is no way but to build another layer. After fork it with gitee, the double-layer nesting doll belongs to it.

Finally, I finally successfully pulled the code to the local (it's not easy).

3. Operation steps

3.1 Download source code

Personal fork version (4.9.9): https://gitee.com/lazyTimes/rocketmq-adong.git

Official gituhb (latest version): https://github.com/apache/rocketmq.git

Source directory structure :

• broker : broker module (broke start process)
• client : message client, including message producers and message consumers related classes
• common : common package
• dev : developer information (not source code)
• distribution : Deployment instance folder (non-source)
• example : RocketMQ example code
• filter : basic classes related to message filtering
• filtersrv : Message filtering server implementation related classes (Filter startup process)
• logappender : log implementation related classes
• namesrv : NameServer implements related classes (NameServer starts the process)
• openmessageing : an open standard for messaging
• remoting : Remote communication module, based on Netty
• srcutil : service utility class
• store : message store implementation related classes
• style : checkstyle related implementation
• test : test related classes
• tools : tool class, monitoring command related implementation class

3.2 Check Maven configuration

Don't rush to adjust the configuration after pulling it down. We first check whether the Maven configuration in idea has been reset. If it is reset, change it back. In addition, it is best to check the currently used project compilation version. IDEA often uses the built-in JDK The version compiles the project, and I personally waste a lot of time because of this problem.

In summary, pulling down the item does three things:

• Compile and check for errors
• Check JDK version
• Check the Maven configuration, and the JDK packaged version used by Maven (you can see it in settings).
• Double check to make sure it's correct.

3.3 Setting up NameServ

The first is to set up the naming service, which is one of the necessary services for the Producer to send messages.

There are two core steps in this step:

• Startup parameter setting ROCKETMQ_HOME , if not set, directly to the root path of the project, but it is recommended to set a separate directory for subsequent observation.
• Set logback_namesrv.xml , its path must be the same as ROCKETMQ_HOME , otherwise NameServ will report an error at startup.

Set ROCKETMQ Run the home directory, the path set by Mac system and Window is different, I personally use Windows as a case, so the set path is: E:\adongstack\project\selfUp\rocketmq-adong

1. Then we enter the NameSer project and directly run the org.apache.rocketmq.namesrv.NamesrvStartup#main() method, and no accident will give a prompt:

Please set the ROCKETMQ_HOME variable in your environment to match the location of the RocketMQ installation

The log is very concise and clear. It requires setting ROCKETMQ_HOME , but where should it be set and how should it be set?

2. After running the org.apache.rocketmq.namesrv.NamesrvStartup#main() method once, Idea will keep the historical startup records. At this time, you can configure the startup parameters:

3. Click the startup parameter Env....., and then configure the aforementioned ROCKETMQ_HOME in it. Here, I set an independent path. It is recommended that readers use a separate empty file directory when trying to facilitate later search.

4. Note that the button position of the new version of IDEA has changed. After clicking the "+" sign, fill in the parameters.

5. After confirmation, the following effects will be found. Then proceed to run Main() method.

But unfortunately, it still reports an error, the log prompt needs logback_namesrv.xml , which is actually the logback log configuration file. We can find this file from the source code E:\adongstack\project\selfUp\rocketmq-adong\distribution\conf\logback_namesrv.xml (personal path as an example) configuration.

6. Because we don't know where to put the file, we try to put the project's logback_namesrv.xml configuration to the following location to see what the effect is.

The configuration of logback will not be introduced here. You can consult the official documentation or check the information on the Internet to understand the meaning of the configuration.

However, this configuration method is wrong . In fact, we have to move the configuration to the directory where ROCKETMQ_HOME is located. Log errors will guide us to correct improvements.

7. Finally try to execute the org.apache.rocketmq.namesrv.NamesrvStartup#main() method, success! The print content is as follows.

 The Name Server boot success. serializeType=JSON

3.4 Set up Broker

The Broker is responsible for persisting queue messages and finding routing information from NameServ. After the Producer finds the Broker with the help of NameServ, it will push the message. After the Broker successfully receives the message, it notifies the Producer that the message was sent successfully.

This step is more critical to set up, carefully observe the configuration.

There are two main points to consider when setting up Broker. The first point is broker.conf configuration, and the second point is to configure the xml log file of Broker's logback.

Next, we set up the Broker. The startup entry of the Broker is src/main/java/org/apache/rocketmq/broker/BrokerStartup.java . When we first came up, we did not rush to change the parameters of the startup environment, but first changed the key configuration of the Broker, mainly setting the address of the NameServer, the address of the Broker Name and other related properties , these depend on the configuration file broker.conf .

First of all, we have to check the IP address of our local loopback network card (that is, the IP of the network card you are using to access the Internet), for example, my IP is 192.168.0.6 .

After we have the IP, we can first backup a distribution/conf/broker.conf file, and then modify the configuration inside:

You can also directly copy a copy distribution/conf/broker.conf configure it to ROCKETMQ_HOME and then make changes. This is just a matter of personal habits, so that you can immediately see your own defined configuration directly in the project in the future.

broker.conf Official default configuration:

 brokerClusterName = DefaultCluster  
brokerName = broker-a  
brokerId = 0  
deleteWhen = 04  
fileReservedTime = 48  
brokerRole = ASYNC_MASTER  
flushDiskType = ASYNC_FLUSH

broker.conf The modified configuration, pay attention to the new content, the personal name is broker-back.conf :

These configurations can be found in the corresponding parameter properties from the configuration file of the source code.

 # 使用如下配置文件  
brokerClusterName = DefaultCluster  
brokerName = broker-a  
brokerId = 0  
deleteWhen = 04  
fileReservedTime = 48  
brokerRole = ASYNC_MASTER  
flushDiskType = ASYNC_FLUSH  
storePathRootDir=E:\\adongstack\\project\\selfUp\\rocketmq-adong\\tmp\\run-dev\\store  
storePathCommitLog=E:\\adongstack\\project\\selfUp\\rocketmq-adong\\tmp\\run-dev\\store\\commitlog  
# 重点内容
namesrvAddr=127.0.0.1:9876  
brokerIP1=192.168.0.6  
brokerIP2=192.168.0.6  
autoCreateTopicEnable=true

Why store and store\commitlog, one is Broker's persistent warehouse address, and the other is the commit log storage address. It is recommended to separate them here. RocketMq's commitlog uses the method of appending writing.

Note that some tutorials on the Internet may have more configuration files than this file. In fact, they are just more detailed configurations. These simple configurations are enough for us to debug the simplest source code.

After the modification is completed, we also copy distribution/conf/logback_brokerxml and broker.conf to ROCKETMQ_HOME . After some operation and configuration, the final result is as follows:

Here, put all the configurations into a folder for easy management. The root path of the file to which the screenshot belongs is E:\adongstack\project\selfUp\rocketmq-adong\tmp\run-dev

The anatomical path is as follows:

 # `E:\adongstack\project\selfUp\rocketmq-adong\tmp\run-dev`//.....
- store
    - commitlog
- broker.conf
- logback_broker.xml
- logback_namesrv.xml

Finally, we configure the startup parameters of cc as follows:

-c 启动参数 corresponds to the conf configuration file.

 -c E:\\adongstack\\project\\selfUp\\rocketmq-adong\\tmp\\run-dev\\broker.conf

Finally, run the main method of BrokerStartup again. The following content appears to indicate that the configuration is successful:

 The broker[broker-a, 192.168.0.103:10911] boot success. serializeType=JSON and name server is 127.0.0.1:9876

3.5 Running the test

First, we start the configuration of NameServ and Broker configured in the previous two sections:

Broker starts printing:

 The broker[broker-a, 127.0.0.1:10911] boot success. serializeType=JSON and name server is 127.0.0.1:9876

NameServ starts printing:

 The Name Server boot success. serializeType=JSON

The next step is to use the Producer for testing. The method used here is org.apache.rocketmq.example.quickstart.Producer#main() . Note that this case cannot be used directly, because we use the debug mode and need to release a line of commented code .

Because all the above configurations use the officially recommended default configuration, you only need to let go of this comment. The default connection to the NameServ port is 9876.

// Uncomment the following line while debugging, namesrvAddr should be set to your local address
// Uncomment the following lines when debugging, namesrvAddr should be set to your local address, the first time readers see this should be commented
producer.setNamesrvAddr(DEFAULT_NAMESRVADDR);

The final print result of the Producer is as follows:

 SendResult [sendStatus=SEND_OK, msgId=7F000001858818B4AAC28569CA6C0000, offsetMsgId=7F00000100002A9F00000000000000EF, messageQueue=MessageQueue [topic=TopicTest, brokerName=broker-a, queueId=0], queueOffset=1]
21:45:02.846 [NettyClientSelector_1] INFO RocketmqRemoting - closeChannel: close the connection to remote address[127.0.0.1:9876] result: true
21:45:02.848 [NettyClientSelector_1] INFO RocketmqRemoting - closeChannel: close the connection to remote address[127.0.0.1:10911] result: true

summary

The overall process of IDEA deploying source code is relatively simple, but the more troublesome is the configuration of some running parameters. In addition, the connection of github is very unstable now. It is recommended to use some domestic code management websites for synchronization, such as gitee and coding.

It should be noted that the annotation must be released when testing, otherwise there will always be an error exception that the Producer cannot connect.