[Hadoop] Zookeeper伪分布式集群部署

1. Zookeeper集群概念

ZK集群：3个节点，一个leader，2个follower

2. 伪分布式集群部署

伪分布式：1台机器上面搭建多个ZK节点

2.1 解压安装zk

解压zookeeper安装包并拷贝成三份，也就是三个zk节点

  1. [hadoop@hadoop000 app]$ cp -r zookeeper-3.4.5-cdh5.7.0 zk001
  2. [hadoop@hadoop000 app]$ cp -r zookeeper-3.4.5-cdh5.7.0 zk002
  3. [hadoop@hadoop000 app]$ cp -r zookeeper-3.4.5-cdh5.7.0 zk003

2.2 修改zk配置文件

分别修改三个zk节点的配置文件

  1. # zk001
  2. [hadoop@hadoop000 app]$ vi zk001/conf/zoo.cfg
  3. dataDir=/home/hadoop/tmp/zookeeper001
  4. server.1=localhost:2888:3888
  5. server.2=localhost:2889:3889
  6. server.3=localhost:2890:3890
  7. clientPort=2181
  8. # zk002
  9. [hadoop@hadoop000 app]$ vi zk002/conf/zoo.cfg
  10. dataDir=/home/hadoop/tmp/zookeeper002
  11. server.1=localhost:2888:3888
  12. server.2=localhost:2889:3889
  13. server.3=localhost:2890:3890
  14. clientPort=2182
  15. # zk003
  16. [hadoop@hadoop000 app]$ vi zk003/conf/zoo.cfg
  17. dataDir=/home/hadoop/tmp/zookeeper003
  18. server.1=localhost:2888:3888
  19. server.2=localhost:2889:3889
  20. server.3=localhost:2890:3890
  21. clientPort=2183
  22. # 其中，2888是服务器与zk leader(master)通信的端口，3888为leader挂掉时，重新选举leader的端口。伪分布式部署时3个zk节点都在同一台机器上，为了端口不发生冲突，所以各节点的端口号依次+1

为每个zk节点创建dataDir文件夹

  1. [hadoop@hadoop000 app]$ mkdir ~/tmp/zookeeper001
  2. [hadoop@hadoop000 app]$ mkdir ~/tmp/zookeeper002
  3. [hadoop@hadoop000 app]$ mkdir ~/tmp/zookeeper003

为每个zk节点创建myid

  1. [hadoop@hadoop000 app]$ echo 1 > ~/tmp/zookeeper001/myid
  2. [hadoop@hadoop000 app]$ echo 2 > ~/tmp/zookeeper002/myid
  3. [hadoop@hadoop000 app]$ echo 3 > ~/tmp/zookeeper003/myid

2.3 启动3个zk节点

启动3个zk节点，每个节点会产生一个QuorumPeerMain进程

  1. # 启动zk001
  2. [hadoop@hadoop000 app]$ zk001/bin/zkServer.sh start
  3. [hadoop@hadoop000 app]$ jps
  4. 13919 QuorumPeerMain
  5. # 启动zk002
  6. [hadoop@hadoop000 app]$ zk002/bin/zkServer.sh start
  7. [hadoop@hadoop000 app]$ jps
  8. 13969 QuorumPeerMain
  9. 13919 QuorumPeerMain
  10. # 启动zk003
  11. [hadoop@hadoop000 app]$ zk003/bin/zkServer.sh start
  12. [hadoop@hadoop000 app]$ jps
  13. 13969 QuorumPeerMain
  14. 14036 QuorumPeerMain
  15. 13919 QuorumPeerMain
  16. [hadoop@hadoop000 app]$

2.4 查看节点状态

查看每个节点状态，可以看到本次启动后，zk002是leader，zk001和zk003是follower

  1. [hadoop@hadoop000 app]$ zk001/bin/zkServer.sh status
  2. JMX enabled by default
  3. Using config: /home/hadoop/app/zk001/bin/../conf/zoo.cfg
  4. Mode: follower
  5. [hadoop@hadoop000 app]$ zk002/bin/zkServer.sh status
  6. JMX enabled by default
  7. Using config: /home/hadoop/app/zk002/bin/../conf/zoo.cfg
  8. Mode: leader
  9. [hadoop@hadoop000 app]$ zk003/bin/zkServer.sh status
  10. JMX enabled by default
  11. Using config: /home/hadoop/app/zk003/bin/../conf/zoo.cfg
  12. Mode: follower
  13. [hadoop@hadoop000 app]$

2.5 测试集群数据

从节点zk001启动一个zk client，连接到zk server 2182端口（zk002)。并且创建一个节点，写入数据。

  1. [hadoop@hadoop000 app]$ zk001/bin/zkCli.sh -server localhost:2182
  2. [zk: localhost:2182(CONNECTED) 0] ls /
  3. [zookeeper]
  4. [zk: localhost:2182(CONNECTED) 1] create /test hellozk002
  5. Created /test
  6. [zk: localhost:2182(CONNECTED) 2] get /test
  7. hellozk002
  8. cZxid = 0x100000002
  9. ctime = Thu Aug 16 19:53:03 CST 2018
  10. mZxid = 0x100000002
  11. mtime = Thu Aug 16 19:53:03 CST 2018
  12. pZxid = 0x100000002
  13. cversion = 0
  14. dataVersion = 0
  15. aclVersion = 0
  16. ephemeralOwner = 0x0
  17. dataLength = 10
  18. numChildren = 0
  19. [zk: localhost:2182(CONNECTED) 3]

去节点zk002启动一个zk client，连接zk server 2183端口（zk003），可以查看到刚才创建的test节点和数据。

  1. [hadoop@hadoop000 app]$ zk002/bin/zkCli.sh -server localhost:2183
  2. [zk: localhost:2183(CONNECTED) 0] ls /
  3. [zookeeper, test]
  4. [zk: localhost:2183(CONNECTED) 1] get /test
  5. hellozk002
  6. cZxid = 0x100000002
  7. ctime = Thu Aug 16 19:53:03 CST 2018
  8. mZxid = 0x100000002
  9. mtime = Thu Aug 16 19:53:03 CST 2018
  10. pZxid = 0x100000002
  11. cversion = 0
  12. dataVersion = 0
  13. aclVersion = 0
  14. ephemeralOwner = 0x0
  15. dataLength = 10
  16. numChildren = 0
  17. [zk: localhost:2183(CONNECTED) 2]

2.6 测试节点挂掉情况

将leader节点zk002进程停止，查看集群中各节点选举情况

  1. [hadoop@hadoop000 app]$ kill -9 13969
  2. # 此时zk003变成了leader
  3. [hadoop@hadoop000 app]$ zk002/bin/zkServer.sh status
  4. JMX enabled by default
  5. Using config: /home/hadoop/app/zk002/bin/../conf/zoo.cfg
  6. Error contacting service. It is probably not running.
  7. [hadoop@hadoop000 app]$ zk003/bin/zkServer.sh status
  8. JMX enabled by default
  9. Using config: /home/hadoop/app/zk003/bin/../conf/zoo.cfg
  10. Mode: leader
  11. [hadoop@hadoop000 app]$

集群挂掉后数据还是可以查到，集群内部容错性是可以保证的。

 

3. 分布式集群部署

真正的分布式集群部署，过程基本一致，只是不同节点上的ZK路径完全一致，配置文件要使用同样的端口号。

  1. [hadoop@hadoop000 app]$ vi zookeeper-3.4.5-cdh5.7.0/conf/zoo.cfg
  2. dataDir=/home/hadoop/tmp/zookeeper
  3. server.1=hadoop000:2888:3888
  4. server.2=hadoop001:2888:3888
  5. server.3=hadoop002:2888:3888
  6. clientPort=2181

每个节点上的myid还是按照上面伪分布式部署的方式，myid号每个节点不同。其他的步骤与伪分布式一模一样。

赞 (0)

相关推荐