Kafka User-guide Summary - Basic Usage

official site http://kafka.apache.org
basic beginning step: download -> start zookeeper -> start Kafka

Basic Usage

use kafka-topics.sh script to create test topic, using 1 replica, 1 partition, then we can use list command to see the topic

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
bin/kafka-topics.sh --list --zookeeper localhost:2181

run the producer and the consumer and send message to Kafka

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
blablabla
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
blablabla

start both producer and the consumer, what we type in producer will appear in consumer

Cluster Deploying

create two new config files

cp config/server.properties config/server-1.properties
cp config/server.properties config/server-2.properties

and configure as following

config/server-1.properties:

broker.id=1
listeners=PLAINTEXT://:9093
log.dir=/tmp/kafka-logs-1

config/server-2.properties:

broker.id=2
listeners=PLAINTEXT://:9094
log.dir=/tmp/kafka-logs-2

then start these two nodes

bin/kafka-server-start.sh config/server-1.properties &
...
bin/kafka-server-start.sh config/server-2.properties &
...

create a topic with 3 replications (according to the new config)

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic

to see the status and the workload, use --describe

bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic