Kakfa -- consumer initiation process

When the consumer starts, it mainly initializes several core components.

The deserializer, mainly the message sent by the producer is serialized to the broker, so it needs to be deserialized for processing.

NetworkClient has been mentioned in the producer before and is used for network communication.

ConsumerCoordinator is the coordinator. If the same group consumes, it needs to be coordinated. Unlike rocketmq, rocketmq is coordinated by consumers ( RocketMQ -- written before message pulling ), and kafka is coordinated through broker. In the same way, consumers need to send heartbeats to brokers, so that brokers know which message fees need to be consumed.

Fetcher is mainly used to pull messages.

After the Consumer is initialized, it starts to pull messages, but before pulling messages, the Coordinator needs to be determined.

When there is no Coordinator, it is not as good as consumer1, which will randomly find the address of the broker, such as broker1.

Call the interface of broker1 to request related data.

After broker1 receives the request, it will hash the groupid and take the modulo according to the topic data of __consumer_offsets. The number of __consumer_offsets is 50.

Suppose the hash value is 10, and the partition of __consumer_offsets is 10 on broker2, then this broker2 is the Coordinator server.

After consumer1 knows that broker2 is the Coordinator server, it starts to register with broker2 and send heartbeats.

Similarly, since consumer2 has the same groupid, the calculation result is also broker2, and finally registers with broker2 and sends a heartbeat.

broker2 will save the relevant consumer information and select the leader consumer.

Whoever registers first is the leader consumer.

After the registration is successful, if it is a leader consumer, a partition scheme will be formulated and sent to broker2, and broker2 will then send the partition scheme to other consumers.