Apache Storm Integration With Apache Kafka
Installing Apache Storm The prerequisite for storm to work on the machine. a. Download and installation commands for ZeroMQ 2.1.7: Run the following commands on terminals
|
1 2 3 4 5 6 7 8 9 10 11 12 |
wget http://download.zeromq.org/zeromq-2.1.7.tar.gz tar -xzf zeromq-2.1.7.tar.gz cd zeromq-2.1.7 ./configure make sudo make install |
b. Download and installation commands for JZMQ:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
git clone https://github.com/nathanmarz/jzmq.git 2. cd jzmq 3. ./autogen.sh 4. ./configure 5. make 6. sudo make install Note: if its failing during build you need to run below commands to install required libraries sudo yum install libuuid* sudo yum install uuid-* sudo yum install gcc-* sudo yum install git sudo yum install libtool* |
2. Download latest storm from http://storm.apache.org/downloads.html
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
In my case “apache-storm-0.10.0.zip” $unzip apache-storm-0.10.0.zip $vi conf/storm.yaml Need to add below code.(If file not there then look for storm.default.yaml and save as storm.yaml ########### These MUST be filled in for a storm configuration storm.zookeeper.servers: - "localhost" // your ip address storm.zookeeper.port: 2181 nimbus.host: "localhost" // your ip address nimbus.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true" ui.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true" supervisor.childopts: "-Djava.net.preferIPv4Stack=true" worker.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true" nimbus.thrift.port: 8627 ui.port: 8772 storm.local.dir: "/home/storm/storm-0.8.1/data" // your data dir path java.library.path: "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/" //Java home path supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703 |
Second start Storm Cluster by starting master and worker nodes. Start master node i.e. nimbus. To start master i.e. nimbus go to the ‘bin’ directory of the […]
