Problem: This service will provide short aliases redirecting to long URLs. Step 1: Requirement Analysis Understand the the basic core features: 1. create short url from long url. 2. get the long url from the short url. Nice to have feature: 3. will url get expired in certain time? 4. could user define their customized short url? here is some questions need to clarify: 1. How long we need keep the url? (it will have impact on storage, it is very import to understand to how long will the data be if such data will be stored in local storage). 2. Do we allow N : 1 or only 1: 1 mapping? (have impact about algorithm and data storage. Step 2: Estimation Of Resource Usage common resources: data storage || web services: QPS Let's the estimation right now: Assume DAU is about 500M, Create: and one user will create new one item every 5 days. so the total creation per Second will be a. yearly new record: 500M/5 * 365 ~ 50G, new records a. monthly storage: 500M/5 * 100 * 30 = 100M *
Cluster Env Setup 1. remote ssh access using generated ssh key a. ssh-keygen -t rsa b. go to ~/.ssh/ and check for id_rsa and id_rsa.pub c. ssh-copy-id -i ~/.ssh/id_rsa.pub <targetserver> d. more ~/.ssh/authorized_keys 2. download the kafka and zookeeper to each nodes 3. for each purpose, use clush to push the kafka to all nodes. clush -g nodeGroup -c <path of kafka> Install zookeeper in Cluster 4. setup zookeeper: a. rename zoo_sample to zoo.cfg b. add the following to zoo.cfg for cluster setup. server.1 = <host1>:2888:3888 server.2 = <host2>:2888:3888 server.3 = <host3>:2888:3888 2888: is used between leader and follower. 3888: is used for leader election. 5. sync the zoo.cfg to all clusters node 6. assign the different ID to different node. 1. create unique id under /tmp/zookeeper 7. start zookeeper on each node 8. test if the all 3 zookeeper are synced. 1. login zookeeper conosle: