Cluster Env Setup 1. remote ssh access using generated ssh key a. ssh-keygen -t rsa b. go to ~/.ssh/ and check for id_rsa and id_rsa.pub c. ssh-copy-id -i ~/.ssh/id_rsa.pub <targetserver> d. more ~/.ssh/authorized_keys 2. download the kafka and zookeeper to each nodes 3. for each purpose, use clush to push the kafka to all nodes. clush -g nodeGroup -c <path of kafka> Install zookeeper in Cluster 4. setup zookeeper: a. rename zoo_sample to zoo.cfg b. add the following to zoo.cfg for cluster setup. server.1 = <host1>:2888:3888 server.2 = <host2>:2888:3888 server.3 = <host3>:2888:3888 2888: is used between leader and follower. 3888: is used for leader election. 5. sync the zoo.cfg to all clusters node 6. assign the different ID to different node. 1. create unique id under /tmp/zookeeper 7. start zookeeper on each node 8. test if the all 3 zookeeper are synced. 1. login zookeeper conosle:
