storm自身不支持开机自启和集群整体启动和关闭的功能,当集群包含很多节点时管理起来会很麻烦。本文介绍了批量启动和停止集群所有节点的脚本以及设置storm开机自启。
一、设置节点间无密码访问
为了实现批量启动和停止,需要提前配置好各节点间的无密码访问,具体方法详见我以前的帖子或者问度娘,这里不再介绍。
二、(主节点)切换当前路径到storm的bin目录下,并创建以下脚本和文件。
cd /software/storm/apache-storm-0.9.2-incubating/bin
touch start-supervisor.sh
touch start-all.sh
touch stop-supervisor.sh
touch stop-all.sh
touch supervisor-hosts
赋予以上脚本可执行权限
chmod +x *.sh
三、脚本编写
1、start-supervisor.sh
#!/bin/bash
. /etc/profile
# storm的bin目录
bin=/software/storm/apache-storm-0.9.2-incubating/bin
supervisors=$bin/supervisor-hosts
storm nimbus >/dev/null 2>&1 &
storm ui >/dev/null 2>&1 &
cat $supervisors | while read supervisor
do
echo $supervisor
ssh $supervisor $bin/start-supervisor.sh &
done
2、start-supervisor.sh
#!/bin/bash
. /etc/profile
storm supervisor >/dev/null 2>&1 &
3、stop-all.sh
#!/bin/bash
. /etc/profile
# storm的bin目录
bin=/software/storm/apache-storm-0.9.2-incubating/bin
supervisors=$bin/supervisor-hosts
kill -9 `ps -ef|grep daemon.nimbus| awk '{print $2}'`
kill -9 `ps -ef|grep ui.core| awk '{print $2}'`
cat $supervisors | while read supervisor
do
echo $supervisor
ssh $supervisor $bin/stop-supervisor.sh &
done
4、stop-supervisor.sh
#!/bin/bash
. /etc/profile
kill -9 `ps -ef|grep daemon.supervisor| awk '{print $2}'`
# 这里我直接清理了storm的工作路径和log文件,根据自身需要来设置
rm -rf /software/storm/workdir/*
rm -rf /software/storm/apache-storm-0.9.2-incubating/logs/*
5、supervisor-hosts
文件中写入所有节点的主机名或者ip,格式如下:
storm1
storm2
storm3
四、脚本的使用
脚本需要在主节点上使用。为了便于使用,请确保环境变量中已经加入storm的bin目录。
将上文编辑好的start-supervisor和stop-supervisor脚本复制到所有节点相同路径下。
scp *-supervisor.sh storm2:/software/storm/apache-storm-0.9.2-incubating/bin
scp *-supervisor.sh storm3:/software/storm/apache-storm-0.9.2-incubating/bin
- 启动集群中所有节点supervisor进程,并在主节点上启动nimbus和ui进程
start-all.sh
- 停止集群中所有节点supervisor进程,并停止nimbus和ui进程
stop-all.sh
五、设置开机自启
要实现storm开机自启,比较简单的方法就是开机自动运行下之前的start-all脚本,设置方法如下
vi /etc/rc.d/rc.local
添加执行脚本
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
sh /software/storm/apache-storm-0.9.2-incubating/bin/start-all.sh