主要内容来自
https://www.tutorialspoint.com/apache_kafka/apache_kafka_quick_guide.htm
Step 1: 安装并启动 zookeeper
A) 到 http://www-us.apache.org/dist/zookeeper/zookeeper-3.4.10/ 下载 zookeeper-3.4.10.tar.gz 到/opt/.
B) tar -zxf zookeeper-3.4.10.tar.gz
C) 修改 conf/zoo.cfg 使其包含如下设置
tickTime=2000
dataDir=/path/to/zookeeper/data
clientPort=2181
initLimit=5
syncLimit=2
D) 启动zookeeper服务器
bin/zkServer.sh start
E) (在另一个新窗口）启动zookeeper客户端
bin/zkCli.sh
Step 2：安装并启动kafka
A) 到 https://kafka.apache.org/downloads/ 下载 kafka_2.11-0.11.0.1.tgz
B) 到/opt/运行 tar -zxf kafka_2.11-0.11.0.1.tgz
C) 在 /opt/kafka_2.11-0.11.0.1/中 运行
bin/kafka-server-start.sh config/server.properties
Step 3: SingleNode_SingleBroker 模式
A) Create a topic: (创建一个topic Hello-Kafka) 在/opt/kafka_2.11-0.11.0.1/中 运行 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor1 --partitions1 --topic Hello-Kafka
B) 在/opt/kafka_2.11-0.11.0.1/中 运行 bin/kafka-topics.sh --list --zookeeper localhost:2181 查看 topic list
C) (在另一个窗口中启动producer 产生messages): 在/opt/kafka_2.11-0.11.0.1/中 运行 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Hello-Kafka
D) (在另一个窗口中启动consumer来消费messages): 在/opt/kafka_2.11-0.11.0.1/中 运行
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic Hello-Kafka --from-beginning
E) 运行了C) and D)，then whatever messages you input in the terminal of C) will appear in the terminal of D).
Step 4: SingleNode_MultipleBrokers模式
A) 在kafka/config/中复制文件server.properties为server-one.properties和server-two.properties。在server-one.properties中包含如下内容
port.id=1
listeners=PLAINTEXT://:9093
log.dirs=/tmp/kafka-logs-1
在server-two.properties中包含如下内容
port.id=2
listeners=PLAINTEXT://:9094
log.dirs=/tmp/kafka-logs-2
B) 然后可以启动三个brokers:
Broker1
bin/kafka-server-start.sh config/server.properties
Broker2
bin/kafka-server-start.sh config/server-one.properties
Broker3
bin/kafka-server-start.sh config/server-two.properties
C) 创建一个topic: 另开一个窗口，到/opt/kafka_2.11-0.11.0.1/中运行
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic Multibrokerapplication
D) The 'Describe' command is used to check which broker is listening on the current created topic as shown below −
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic Multibrokerapplication
运行的结果
Topic:Multibrokerapplication PartitionCount:1 ReplicationFactor:3 Configs:
Topic: Multibrokerapplication Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1
From the above output, we can conclude that first line gives a
summary of all the partitions, showing topic name, partition count and
the replication factor that we have chosen already. In the second line,
each node will be the leader for a randomly selected portion of the
partitions.
In our case, we see that our third broker (with broker.id 2) is the leader. Then Replicas:2,0,1 means that all the brokers replicate the topic finallyIsris the set ofin-syncreplicas. Well, this is the subset of replicas that are currently alive and caught up by the leader.
E) 启动一个producer产生message:
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Multibrokerapplication
F) 启动一个Consumer接收消费message:
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic Multibrokerapplication --from-beginning
关于topic的基本操作
A) 修改topic (Modifying a Topic)
bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic Hello-kafka --partitions 2
这条命令修改Hello-kafka的分区为2。
B) 删除一个topic
bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic Multibrokerapplication
Kafka-Python API
https://github.com/dpkp/kafka-python
用pip install kafka-python
安装kafka-python后， 启动zookeeper 和kafka server and client以后,可以测试 github的 kafka-python 中的例子。
Another python API:
https://github.com/confluentinc/confluent-kafka-python
But I have not tested the latter one.
