运行环境前置要求

​ Kafka是使用Scala语言编写而成,Scala运行在Java虚拟机上,并兼容现有的Java程序,因此要部署Kafka需要先安装JDK环境,建议安装Java8+(Java8、Java11、Java17、Java21都可以,这里使用Java17)。

你也可以使用jenv管理多版本jdk,具体操作参考我之前的文章:

Kafka安装

前往Kafka官网下载最新3.7.0版本

image-20240614120611869

解压缩:tar -zxvf kafka_2.13-3.7.0.tgz

进入bin目录:cd kafka_2.13-3.7.0/bin/

启动Kafka

Apache Kafka可以使用ZooKeeper或者KRaft启动,但只能使用其中一种方式,不能同时使用。

KRaft:Apache Kafka的内置共识机制,用于取代Apache ZooKeeper。

  1. 生成Cluster UUID:./kafka-storage.sh random-uuid
  2. ./kafka-storage.sh format -t <你的uuid> -c …/config/kraft/server.properties
  3. 启动Kafka:./kafka-server-start.sh …/config/kraft/server.properties &
  1. 启动ZooKeeper:./zookeeper-server-start.sh …/config/zookeeper.properties &
  2. 启动Kafka:./kafka-server-start.sh …/config/server.properties &
  3. 关闭Kafka:./kafka-server-stop.sh …/config/server.properties
  4. 关闭ZooKeeper:./zookeeper-server-stop.sh …/config/zookeeper.properties

KRaft

  1. 生成Cluster UUID

查看帮助文档:./kafka-storage.sh -h

image-20240614152449786

生成Cluster UUID:./kafka-storage.sh random-uuid

image-20240614152755154

  1. 格式化Kafka日志目录

查看format帮助文档:./kafka-storage.sh format -h

image-20240614153040302

格式化Kafka日志目录:./kafka-storage.sh format -t <你的uuid> -c …/config/kraft/server.properties

image-20240614153247325

  1. 启动Kafka

启动Kafka:./kafka-server-start.sh …/config/kraft/server.properties &

image-20240614153512549

查看进程

image-20240614153604694

Zookeeper服务器

当然你也可以自己去下载一个ZooKeeper,而不是使用Kafka自带的。

ZooKeeper官网

image-20240614143021301

下载最新版

image-20240614143107649

使用推荐地址下载

image-20240614143153136

解压缩:tar -zxvf apache-zookeeper-3.9.2-bin.tar.gz

配置ZooKeeper:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
cd apache-zookeeper-3.9.2-bin/conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg # 查看配置文件 不需要修改
====================#配置文件内容#==========================
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpHost=0.0.0.0
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true
==========================================================

启动ZooKeeper

1
2
3
cd ../bin
./zkServer.sh start
ps -ef | grep zookeeper #查看是否启动

image-20240614144306060

1
netstat -nlpt #查看占用端口

image-20240614144551300

发现占用了三个端口

它占用了8080端口

修改Zookeeper占用8080端口

1
2
3
4
5
6
7
8
9
10
11
12
cd ../conf
vim zoo.cfg

==================#在最后添加如下内容#================
# admin.serverPort 默认占8080端口
admin.serverPort=9089
===================================================

#重启zookeeper
cd ../bin
./zkServer.sh stop
./zkServer.sh start

image-20240614145539025

image-20240614145639403

发现端口已经修改成功。

Docker安装Kafka

如果不知道怎么安装docker的小朋友,请看我之前的教程:

  1. 拉取kafka镜像:docker pull apache/kafka:3.7.0
  2. 启动kafka容器:docker run -p 9092:9092 apache/kafka:3.7.0