Goodbye RocketMQ! A new generation of message middleware, with visual management, full document thieves!

Recently, the popular message middleware Pulsar, I wanted to learn, and found that many articles on the Internet introduce performance and compare Kafka, and there are few practical articles! So I practiced a wave of official documents and wrote this article. It is estimated that it is the first Pulsar practical article in China. I hope it will be helpful to everyone!

SpringBoot actual combat e-commerce project mall (40k+star) address: https://github.com/macrozheng/mall

Introduction to Pulsar

Pulsar is a server-to-server messaging middleware with advantages of multi-tenancy and high performance. Pulsar was originally developed by Yahoo and is currently managed by the Apache Software Foundation. Pulsar uses the publish-subscribe design model. Producer publishes messages to Topic, Consumer subscribes to Topic, and processes messages in Topic.

Pulsar has the following characteristics:

• A single instance of Pulsar natively supports clusters.
• Very low release delay and end-to-end delay.
• Can be seamlessly expanded to more than one million topics.
• Simple and easy-to-use client API supports Java, Go, Python and C++.
• Support multiple Topic subscription modes (exclusive subscription, shared subscription, failover subscription).
• The persistent message storage mechanism provided by Apache BookKeeper ensures message delivery.

Pulsar installation

Using Docker to install Pulsar is the easiest, this time we use Docker to install.

• First download the Docker image of Pulsar;

docker pull apachepulsar/pulsar:2.7.1

• After the download is complete, run the Pulsar container, the http protocol access uses the 8080 port, and the pulsar protocol (Java, Python, etc. client) access uses the 6650 port.

docker run --name pulsar \
-p 6650:6650 \
-p 8080:8080 \
--mount source=pulsardata,target=/pulsar/data \
--mount source=pulsarconf,target=/pulsar/conf \
-d apachepulsar/pulsar:2.7.1 \
bin/pulsar standalone

Pulsar visualization

Pulsar Manager is an official visualization tool that can visually manage multiple Pulsars. Although there are not many functions, it is basically enough and supports Docker deployment.

• Download the Docker image of pulsar-manager

docker pull apachepulsar/pulsar-manager:v0.2.0

• After the download is complete, run the pulsar-manager container, and you can access the Web page 9527

docker run -it --name pulsar-manager\
    -p 9527:9527 -p 7750:7750 \
    -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
    -d apachepulsar/pulsar-manager:v0.2.0

• After the operation is successful, we cannot access at the beginning, we need to create an administrator account, here the created account is admin:apachepulsar :

CSRF_TOKEN=$(curl http://localhost:7750/pulsar-manager/csrf-token)
curl \
    -H "X-XSRF-TOKEN: $CSRF_TOKEN" \
    -H "Cookie: XSRF-TOKEN=$CSRF_TOKEN;" \
    -H 'Content-Type: application/json' \
    -X PUT http://localhost:7750/pulsar-manager/users/superuser \
    -d '{"name": "admin", "password": "apachepulsar", "description": "test", "email": "[e-mail]"}'

• After the creation is successful, log in through the login page and visit the address: http://192.168.5.78:9527

• After successful login, we need to configure an environment first, that is, configure the Pulsar service that needs to be managed. The configured Service URL is: http://192.168.5.78:8080

• You can view the tenant list;

• Can view Topic list and manage Topic;

• You can also view the detailed information of Topic.

Pulsar combined with SpringBoot use

It is also very simple to use Pulsar in combination with SpringBoot. We can use Pulsar's official Java SDK or a third-party SpringBoot Starter. Using Starter here is very simple!

• First add Pulsar related dependencies pom.xml

<!--SpringBoot整合Pulsar-->
<dependency>
    <groupId>io.github.majusko</groupId>
    <artifactId>pulsar-java-spring-boot-starter</artifactId>
    <version>1.0.4</version>
</dependency>

• Then application.yml added Pulsar in Service URL configuration;

pulsar:
  service-url: pulsar://192.168.5.78:6650

• Then add Pulsar's Java configuration, declare two topics, and determine the type of message sent;

/**
 * Pulsar配置类
 * Created by macro on 2021/5/21.
 */
@Configuration
public class PulsarConfig {
    @Bean
    public ProducerFactory producerFactory() {
        return new ProducerFactory()
                .addProducer("bootTopic", MessageDto.class)
                .addProducer("stringTopic", String.class);
    }
}

• Create a Pulsar producer and send messages to Topic. Here you can find that Pulsar supports sending message objects directly;

/**
 * Pulsar消息生产者
 * Created by macro on 2021/5/19.
 */
@Component
public class PulsarProducer {
    @Autowired
    private PulsarTemplate<MessageDto> template;

    public void send(MessageDto message){
        try {
            template.send("bootTopic",message);
        } catch (PulsarClientException e) {
            e.printStackTrace();
        }
    }
}

• Create a Pulsar consumer, get and consume messages from Topic, and you can also directly get the message object;

/**
 * Pulsar消息消费者
 * Created by macro on 2021/5/19.
 */
@Slf4j
@Component
public class PulsarRealConsumer {

    @PulsarConsumer(topic="bootTopic", clazz= MessageDto.class)
    public void consume(MessageDto message) {
        log.info("PulsarRealConsumer consume id:{},content:{}",message.getId(),message.getContent());
    }

}

• Add a test interface and call the producer to send a message;

/**
 * Pulsar功能测试
 * Created by macro on 2021/5/19.
 */
@Api(tags = "PulsarController", description = "Pulsar功能测试")
@Controller
@RequestMapping("/pulsar")
public class PulsarController {

    @Autowired
    private PulsarProducer pulsarProducer;

    @ApiOperation("发送消息")
    @RequestMapping(value = "/sendMessage", method = RequestMethod.POST)
    @ResponseBody
    public CommonResult sendMessage(@RequestBody MessageDto message) {
        pulsarProducer.send(message);
        return CommonResult.success(null);
    }
}

• Call the interface in Swagger for testing;

• After the call is successful, the console will enter the following information, indicating that the message has been successfully received and consumed.

2021-05-21 16:25:07.756  INFO 11472 --- [al-listener-3-1] c.m.m.tiny.component.PulsarRealConsumer  : PulsarRealConsumer consume id:1,content:SpringBoot Message!

to sum up

Last time I wrote an article "Eagle, a graphical tool for Kafka, which must be recommended to you!" " introduces the basic use of Kafka, here is a comparison with Pulsar. Pulsar's support for Docker is undoubtedly better, and the official documentation is also more complete. Comparing the graphical tools Pulsar Manager and Kafka Eagle , Pulsar's graphical tools feel a bit rudimentary. Since Yahoo, Tencent, 360 and other major Internet companies are currently using Pulsar, the performance and stability of Pulsar should be very good!

Reference

Pulsar's official documentation is very complete, the style is also good, basically follow the documentation to get started.

• Pulsar official document: https://pulsar.apache.org/docs/en/standalone-docker/
• SpringBoot Starter official document: https://github.com/majusko/pulsar-java-spring-boot-starter

Project source code address

https://github.com/macrozheng/mall-learning/tree/master/mall-tiny-pulsar

This article GitHub https://github.com/macrozheng/mall-learning has been included, welcome to Star!