Apache Kafka
I‘m good at dealing with logs, don’t just consider me as a queue.
event streaming platform
To publish (write) and subscribe to (read) streams of events, including continuous import/export of your data from other systems.
发布/订阅
To store streams of events durably and reliably for as long as you want.
可持久化
To process streams of events as they occur or retrospectively.
关键概念
An event records the fact that “something happened” in the world or in your business.
一个事件，用户点击了这个按钮浏览了那个网页
Events are organized and durably stored in topics.
相当于mysql中的table
Topics are partitioned, meaning a topic is spread over a number of “buckets” located on different Kafka brokers.
相当于水平分表
Broker
节点而已
A computer, instance, or container running the Kafka process
Manage partitions
Handle write and read requests
Manage replication of partitions
Intentionally very simple
Replication
高可用/副本
copies of data for fault tolerance
One lead partition and N-1 followers
In general, writes and reads happen to the leader
An invisible process to most developers
Tunable in the Producer
尝鲜
下载kafka并解压
wget https://downloads.apache.org/kafka/2.7.0/kafka_2.13-2.7.0.tgz
tar -xzf kafka_2.13-2.7.0.tgzcd kafka_2.13-2.7.0复制
启动zookeeper和kafka
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties复制
创建TOPIC来保存EVENTS
bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092// 显示使用信息
bin/kafka-topics.sh --describe --topic quickstart-events --bootstrap-server localhost:9092复制
写
bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092复制
读
bin/kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server localhost:9092复制
With Golang
librdkafka
brew install librdkafka
apt install librdkafka-dev
yum install librdkafka-devel复制
confluent-kafka-go
https://github.com/confluentinc/confluent-kafka-go
kafka-go
https://github.com/segmentio/kafka-go
That’s it ~
Have Fun ᕕ( ᐛ )ᕗ
