JStorm部署

机器规划

集群部署

准备安装包

wget https://github.com/alibaba/jstorm/releases/download/2.2.1/jstorm-2.2.1.zip unzip jstorm-2.2.1.zip mv jstorm-2.2.1 /opt/jstorm 

环境变量

在~/.bash_profile 文件中或/etc/profile 文件中添加如下配置

vi ~/.bash_profile vi /etc/profile ··· export JSTORM_HOME=/opt/jstorm export PATH=$PATH:$JSTORM_HOME/bin ··· 

配置

编辑$JSTORM_HOME/conf/storm.yaml 配置文件

vim $JSTORM_HOME/conf/storm.yaml 

配置storm.zookeeper.servers为zookeeper地址

# 端口默认2181 storm.zookeeper.servers: - "172.16.10.220" 

配置storm.zookeeper.root为jstorm在zookeeper的节点名称

 storm.zookeeper.root: "/jstorm" 

去掉nimbus.host的注释，配置nimbus节点

 nimbus.host: "172.16.10.210" 

配置storm.local.dir，表示jstorm的临时数据存放目录

 storm.local.dir: "%JSTORM_HOME%/data" 

去掉supervisor.slots.ports前面的注释，设置supervisor节点执行worker使用的端口列表

 supervisor.slots.ports: - 6800 - 6801 - 6802 - 6803 

完整配置

 These MUST be filled in for a storm configuration storm.zookeeper.servers: - "172.16.10.220" storm.zookeeper.root: "/jstorm" # cluster.name: "default" #nimbus.host/nimbus.host.start.supervisor is being used by $JSTORM_HOME/bin/start.sh #it only support IP, please don't set hostname # For example nimbus.host: "172.16.10.210" #nimbus.host: "localhost" #nimbus.host.start.supervisor: false # %JSTORM_HOME% is the jstorm home directory storm.local.dir: "%JSTORM_HOME%/data" # please set absolute path, default path is JSTORM_HOME/logs # jstorm.log.dir: "absolute path" # java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib" # if supervisor.slots.ports is null, # the port list will be generated by cpu cores and system memory size # for example, # there are cpu_num = system_physical_cpu_num/supervisor.slots.port.cpu.weight # there are mem_num = system_physical_memory_size/(worker.memory.size * supervisor.slots.port.mem.weight) # The final port number is min(cpu_num, mem_num) # supervisor.slots.ports.base: 6800 # supervisor.slots.port.cpu.weight: 1.2 # supervisor.slots.port.mem.weight: 0.7 # supervisor.slots.ports: null supervisor.slots.ports: - 6800 - 6801 - 6802 - 6803 # Default disable user-define classloader # If there are jar conflict between jstorm and application, # please enable it # topology.enable.classloader: false # ... 

部署nimbus

运行

切换路径

cd $JSTORM_HOME/bin 

直接运行nimbus

jstorm nimbus 

后台运行

nohup jstorm nimbus >> nimbus.txt 2>&1 & 

HA

jstorm支持多个nimbus做HA，在另一台机器上执行同样的命令即可，根据zookeeper的原理，后执行的就自动standby了，充当slave，部署了jstorm-ui后，可以在jstorm-ui上可以看到。

部署supervisor

切换路径

cd $JSTORM_HOME/bin 

直接运行nimbus

jstorm supervisor 

后台运行

nohup jstorm supervisor >> supervisor.txt 2>&1 & 

部署storm-ui

下载的jstorm包解压后在根目录下有jstorm-ui-2.2.1.war，这是一个标准的war包，放在tomcat中执行即可

配置

因为storm-ui也需要storm.yaml的配置，默认使用~/.jstorm/storm.yaml

mkdir ~/.jstorm cp -f $JSTORM_HOME/conf/storm.yaml ~/.jstorm 

开机启动

nimbus

编写脚本 nimbus-start.sh

#!/bin/bash #chkconfig: 112 63 37 nohup $JSTORM_HOME/bin/jstorm nimbus >> $JSTORM_HOME/bin/nimbus.txt 2>&1 & 

将脚本移动到/etc/rc.d/init.d目录下

mv nimbus-start.sh /etc/rc.d/init.d 

赋予脚本可执行权限

chmod +x /etc/rc.d/init.d/nimbus-start.sh 

添加脚本到开机自动启动项目中

cd /etc/rc.d/init.d chkconfig --add nimbus-start.sh chkconfig nimbus-start.sh on 

supervisor

编写脚本 supervisor-start.sh

#!/bin/bash #chkconfig: 112 63 37 nohup $JSTORM_HOME/bin/jstorm supervisor >> $JSTORM_HOME/bin/supervisor 2>&1 & 

将脚本移动到/etc/rc.d/init.d目录下

mv supervisor-start.sh /etc/rc.d/init.d 

赋予脚本可执行权限

chmod +x /etc/rc.d/init.d/supervisor-start.sh 

添加脚本到开机自动启动项目中

cd /etc/rc.d/init.d chkconfig --add supervisor-start.sh chkconfig supervisor-start.sh on 

jstorm-ui

编写脚本 jstorm-ui-start.sh

#!/bin/bash #chkconfig: 112 63 37 ${TOMCAT_HOME}/bin/startup.sh 

将脚本移动到/etc/rc.d/init.d目录下

mv jstorm-ui-start.sh /etc/rc.d/init.d 

赋予脚本可执行权限

chmod +x /etc/rc.d/init.d/jstorm-ui-start.sh 

添加脚本到开机自动启动项目中

cd /etc/rc.d/init.d chkconfig --add jstorm-ui-start.sh chkconfig jstorm-ui-start.sh on 

提交topology

提交topology的命令格式

jstorm jar springboot-jstorm-cal.jar com.*.*.CalBootstrap 

杀死topology

jstorm kill topologyname

例如:

jstorm kill topologyname