Windows11下安装Kafka超级详细

Windows11下安装Kafka超级详细

2024-03-30 16:59:08.0|分类: kafka|浏览量: 548

摘要: Kafka是一种高吞吐量的分布式发布订阅消息系统，它可以处理消费者在网站中的所有动作流数据

Kafka 是一种流行的分布式消息队列系统。


它具有以下特点：


  1. 高吞吐量：能够处理大量的消息。

  2. 分布式：可扩展，支持在多个服务器上部署。

  3. 可靠性：保证消息的可靠传递。

  4. 高性能：提供了高效的消息处理能力。


Kafka 的常见应用场景包括：


  1. 日志收集：收集各种系统和应用的日志。

  2. 数据管道：在不同系统之间传输数据。

  3. 实时数据处理：支持实时处理和分析数据。


在使用 Kafka 时，需要注意以下几点：


  1. 配置：合理配置 Kafka 的各种参数，以满足应用需求。

  2. 数据格式：选择合适的数据格式，以便高效地处理消息。

  3. 容错处理：考虑容错机制，以确保系统的可靠性。


1、先安装Zookeeper

2、下载安装包kafka_2.13-3.7.0.tgz

下载地址：https://mirrors.aliyun.com/apache/kafka/

3、复制kafka_2.13-3.7.0.tgz到C:\me\studyhome\kafka

4、解压安装包kafka_2.13-3.7.0.tgz

5、进入到配置文件C:\me\studyhome\kafka\kafka_2.13-3.7.0\config

6、修改配置文件server.properties

log.dirs=C:\\me\\studyhome\\kafka\\kafka_2.13-3.7.0\\kafka-logs

zookeeper.connect=localhost:2181

7、进入安装目录C:\me\studyhome\kafka\kafka_2.13-3.7.0

.\bin\windows\kafka-server-start.bat .\config\server.properties

报错：输入行太长。 命令语法不正确。


原因：目录太深了：C:\me\studyhome\kafka\kafka_2.13-3.7.0


解决办法将安装目录移动到C判断，安装目录缩短：C:\kafka3.7.0


8、进入安装目录C:\kafka3.7.0，执行启动命令：


C:\kafka3.7.0>.\bin\windows\kafka-server-start.bat .\config\server.properties

[2024-03-30 16:56:05,657] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)

[2024-03-30 16:56:06,099] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)

[2024-03-30 16:56:06,203] INFO starting (kafka.server.KafkaServer)

[2024-03-30 16:56:06,204] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)

[2024-03-30 16:56:06,223] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)

[2024-03-30 16:56:06,230] INFO Client environment:zookeeper.version=3.8.3-6ad6d364c7c0bcf0de452d54ebefa3058098ab56, built on 2023-10-05 10:34 UTC (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,230] INFO Client environment:host.name=LAPTOP-78C0012V (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,230] INFO Client environment:java.version=1.8.0_231 (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,230] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,230] INFO Client environment:java.home=C:\Program Files\Java\jdk1.8.0_231\jre (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,230] INFO Client environment:java.class.path=.;C:\Program Files\Java\jdk1.8.0_231\lib\dt.jar;C:\Program Files\Java\jdk1.8.0_231\lib\tools.jar;C:\kafka3.7.0\libs\activation-1.1.1.jar;C:\kafka3.7.0\libs\aopalliance-repackaged-2.6.1.jar;C:\kafka3.7.0\libs\argparse4j-0.7.0.jar;C:\kafka3.7.0\libs\audience-annotations-0.12.0.jar;C:\kafka3.7.0\libs\caffeine-2...ns-2.2.8.jar;C:\kafka3.7.0\libs\trogdor-3.7.0.jar;C:\kafka3.7.0\libs\zookeeper-3.8.3.jar;C:\kafka3.7.0\libs\zookeeper-jute-3.8.3.jar;C:\kafka3.7.0\libs\zstd-jni-1.5.5-6.jar (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,232] INFO Client environment:java.library.path=C:\Program Files\Java\jdk1.8.0_231\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\iCLS\;C:\Program Files\Intel\Intel(R) Management Engine Components\iCLS\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Java\jdk1.8.0_231\bin;C:\Program Files\Java\jdk1.8.0_231\jre\bin;C:\me\workhome\maven\apache-maven-3.6.2\bin;C:\me\workhome\mysql-5.7.26-winx64\bin;C:\Program Files\TortoiseSVN\bin;C:\Users\conca\AppData\Roaming\npm;C:\Program Files\nodejs;C:\Program Files\nodejs\node-global;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\PuTTY\;C:\Program Files (x86)\Tencent\微信web开发者工具\dll;C:\Program Files\nodejs\;C:\Program Files\微信web开发者工具\dll;C:\me\studyhome\flink\flink-1.12.7\bin;C:\Users\conca\AppData\Local\Programs\Python\Python311\Scripts\;C:\Users\conca\AppData\Local\Programs\Python\Python311\;C:\Users\conca\AppData\Local\Microsoft\WindowsApps;C:\Users\conca\AppData\Roaming\npm;C:\Users\conca\AppData\Local\Programs\EmEditor;. (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,233] INFO Client environment:java.io.tmpdir=C:\Users\conca\AppData\Local\Temp\ (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,234] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,235] INFO Client environment:os.name=Windows 10 (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,236] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,237] INFO Client environment:os.version=10.0 (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,237] INFO Client environment:user.name=conca (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,245] INFO Client environment:user.home=C:\Users\conca (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,246] INFO Client environment:user.dir=C:\kafka3.7.0 (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,247] INFO Client environment:os.memory.free=1003MB (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,248] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,248] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,253] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@765d7657 (org.apache.zookeeper.ZooKeeper)

[2024-03-30 16:56:06,263] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)

[2024-03-30 16:56:06,272] INFO zookeeper.request.timeout value is 0. feature enabled=false (org.apache.zookeeper.ClientCnxn)

[2024-03-30 16:56:06,275] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)

[2024-03-30 16:56:06,278] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)

[2024-03-30 16:56:06,279] INFO Socket connection established, initiating session, client: /127.0.0.1:50661, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)

[2024-03-30 16:56:06,295] INFO Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x10003ffe1230001, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)

[2024-03-30 16:56:06,299] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)

[2024-03-30 16:56:06,599] INFO Cluster ID = cMVm3bq8Rn2zW-3NKX406A (kafka.server.KafkaServer)

[2024-03-30 16:56:06,657] INFO KafkaConfig values:

        advertised.listeners = null

        alter.config.policy.class.name = null

        alter.log.dirs.replication.quota.window.num = 11

       ....

        zookeeper.ssl.truststore.password = null

        zookeeper.ssl.truststore.type = null

 (kafka.server.KafkaConfig)

[2024-03-30 16:56:06,720] INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)

[2024-03-30 16:56:06,722] INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)

[2024-03-30 16:56:06,722] INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)

[2024-03-30 16:56:06,729] INFO [ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)

[2024-03-30 16:56:06,744] INFO [KafkaServer id=0] Rewriting C:\kafka3.7.0\kafka-logs\meta.properties (kafka.server.KafkaServer)

[2024-03-30 16:56:06,795] INFO Loading logs from log dirs ArraySeq(C:\kafka3.7.0\kafka-logs) (kafka.log.LogManager)

[2024-03-30 16:56:06,802] INFO No logs found to be loaded in C:\kafka3.7.0\kafka-logs (kafka.log.LogManager)

[2024-03-30 16:56:06,814] INFO Loaded 0 logs in 19ms (kafka.log.LogManager)

[2024-03-30 16:56:06,818] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)

[2024-03-30 16:56:06,820] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)

[2024-03-30 16:56:06,905] INFO [kafka-log-cleaner-thread-0]: Starting (kafka.log.LogCleaner$CleanerThread)

[2024-03-30 16:56:06,925] INFO [feature-zk-node-event-process-thread]: Starting (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)

[2024-03-30 16:56:06,948] INFO [MetadataCache brokerId=0] Updated cache from existing None to latest Features(version=3.7-IV4, finalizedFeatures={}, finalizedFeaturesEpoch=0). (kafka.server.metadata.ZkMetadataCache)

[2024-03-30 16:56:06,980] INFO [zk-broker-0-to-controller-forwarding-channel-manager]: Starting (kafka.server.NodeToControllerRequestThread)

[2024-03-30 16:56:07,411] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)

[2024-03-30 16:56:07,437] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(PLAINTEXT) (kafka.network.SocketServer)

[2024-03-30 16:56:07,445] INFO [zk-broker-0-to-controller-alter-partition-channel-manager]: Starting (kafka.server.NodeToControllerRequestThread)

[2024-03-30 16:56:07,493] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,495] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,495] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,499] INFO [ExpirationReaper-0-ElectLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,500] INFO [ExpirationReaper-0-RemoteFetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,520] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)

[2024-03-30 16:56:07,521] INFO [AddPartitionsToTxnSenderThread-0]: Starting (kafka.server.AddPartitionsToTxnManager)

[2024-03-30 16:56:07,587] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient)

[2024-03-30 16:56:07,610] INFO Stat of the created znode at /brokers/ids/0 is: 137858,137858,1711788967604,1711788967604,1,0,0,72061991566639105,214,0,137858

 (kafka.zk.KafkaZkClient)

[2024-03-30 16:56:07,611] INFO Registered broker 0 at path /brokers/ids/0 with addresses: PLAINTEXT://LAPTOP-78C0012V:9092, czxid (broker epoch): 137858 (kafka.zk.KafkaZkClient)

[2024-03-30 16:56:07,682] INFO [ExpirationReaper-0-topic]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,709] INFO [ExpirationReaper-0-Heartbeat]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,710] INFO [ExpirationReaper-0-Rebalance]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,733] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.group.GroupCoordinator)

[2024-03-30 16:56:07,740] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.group.GroupCoordinator)

[2024-03-30 16:56:07,769] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)

[2024-03-30 16:56:07,773] INFO [TxnMarkerSenderThread-0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)

[2024-03-30 16:56:07,773] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)

[2024-03-30 16:56:07,851] INFO [ExpirationReaper-0-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2024-03-30 16:56:07,895] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)

[2024-03-30 16:56:07,906] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Enabling request processing. (kafka.network.SocketServer)

[2024-03-30 16:56:07,910] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.DataPlaneAcceptor)

[2024-03-30 16:56:07,921] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)

[2024-03-30 16:56:07,922] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)

[2024-03-30 16:56:07,926] INFO Kafka startTimeMs: 1711788967916 (org.apache.kafka.common.utils.AppInfoParser)

[2024-03-30 16:56:07,929] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

[2024-03-30 16:56:08,391] INFO [zk-broker-0-to-controller-forwarding-channel-manager]: Recorded new controller, from now on will use node LAPTOP-78C0012V:9092 (id: 0 rack: null) (kafka.server.NodeToControllerRequestThread)

[2024-03-30 16:56:08,420] INFO [zk-broker-0-to-controller-alter-partition-channel-manager]: Recorded new controller, from now on will use node LAPTOP-78C0012V:9092 (id: 0 rack: null) (kafka.server.NodeToControllerRequestThread)

一键分享文章

分类列表

版权所有 cookqq 感谢访问 支持开源 京ICP备15030920号
CopyRight 2015-2018 cookqq.com All Right Reserved.