日志服务部署
<h2>环境准备</h2>
<h4>1 jdk 安装 中间件安装 centos7</h4>
<h3>2 项目插件安装</h3>
<h4><strong>Elasticsearch单点安装</strong></h4>
<ol>
<li>上传es安装包([elasticsearch-5.4.3.tar.gz](<a href="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/7e51300e7415f40aa91316d06e676463">http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/7e51300e7415f40aa91316d06e676463</a> "[elasticsearch-5.4.3.tar.gz"))，并解压至指定目录(文档以/risen/soft/es为例)</li>
<li>解压ik分词器([elasticsearch-analysis-ik-5.4.3.zip](<a href="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/fdee34ad1acf45d159c3a079f5bb1e42">http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/fdee34ad1acf45d159c3a079f5bb1e42</a> "[elasticsearch-analysis-ik-5.4.3.zip"))至名为ik的文件夹，并移动至es家目录下的plugins
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/74aba2953c2add05bab6ab7358a5d706" alt="" /></li>
<li>将新的jna.jar([jna.jar](<a href="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/1b4efb7d0c8efee7678340260bac3f9c">http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/1b4efb7d0c8efee7678340260bac3f9c</a> "[jna.jar"))替换掉家目录下lib中的jna4.4.0.jar
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/2a6d3404efe5f0b7ee4c9b800b747baf" alt="" /></li>
<li>编辑elasticsearch.yml文件 (路径：/risen/soft/es/elasticsearch-5.4.3/config)
配置集群名称
cluster.name: RisenES
配置节点名称
node.name: node-1
配置数据目录
path.data: /risen/soft/es/data
配置日志目录
path.logs: /risen/soft/es/logs
配置ip地址
network.host: 192.168.133.148
配置http访问端口
http.port: 9200
是否支持跨域
http.cors.enabled: true
<em>表示支持所有域名
http.cors.allow-origin: "</em>"
(解决银河麒麟seccomp unavailable错误
bootstrap.memory_lock: false
bootstrap.system_call_filter: false)
设置允许script运行
script.engine.groovy.inline.aggs: on
script.engine.groovy.inline.search: on</li>
<li>编辑limits.conf配置文件
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/2076669b5dd8b86dd77e11074b1d2e3d" alt="" />
保存退出重新登录用户后生效
临时生效：ulimit -n 65536</li>
<li>修改配置文件sysctl.conf
vi /etc/sysctl.conf添加下面配置
vm.max_map_count=655360
保存退出，执行sysctl -p立即生效</li>
<li>添加bigdata用户，并将es相关目录赋权给bigdata
添加用户
useradd bigdata
修改权限
chown -R bigdata.bigdata /risen/soft/es/</li>
<li>配置elasticsearch，安装ingest-attachment([ingest-attachment-5.4.3.zip](<a href="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/dc471afb010986f2edaf6327042c3f64">http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/dc471afb010986f2edaf6327042c3f64</a> "[ingest-attachment-5.4.3.zip"))，并启动
在bin目录启动文件elasticsearch内添加JAVA_HOME变量(如已有JAVA_HOME变量，则跳过)
vim /risen/soft/es/elasticsearch-5.4.3/bin/elasticsearch
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/c64148a50e90d32aaf09be23a96eec49" alt="" />
切换bigdata用户
su - bigdata
进入bin目录执行
./elasticsearch-plugin install file:///risen/soft/es/ingest-attachment-5.4.3.zip
安装ingest-attachment，回显如下图
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/2c760c92d22a8aa4b30d53b67c78042e" alt="" />
启动es
./elasticsearch</li>
<li>注册es为服务
编辑es服务文件
vim /lib/systemd/system/elasticsearch.service
添加如下内容(注意elasticsearch目录及启动命令参数)：
[Unit]
Description=elasticsearch service
After=network.target
[Service]
Type=idle
User=bigdata
LimitNOFILE=65536
ExecStart=/risen/soft/es/elasticsearch-5.4.3/bin/elasticsearch
ExecReload=/bin/kill -1 $MAINPID
ExecStop=/bin/kill -9 $MAINPID
PrivateTmp=true
[Install]
WantedBy=multi-user.target
调试es服务
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/266dd40259a91902b332afe209953bfa" alt="" />
停止es服务
systemctl stop elasticsearch
启动es服务
systemctl start elasticsearch
查看es服务状态
systemctl status elasticsearch
重启es服务
systemctl restart elasticsearch
设置开机自启
systemctl enable elasticsearch</li>
</ol>
<h4><strong>安装zookeeper</strong></h4>
<p>上传zookeeper安装包，并解压至指定目录(文档以/risen/soft/zookeeper为例)
修改zookeeper中conf目录下的zoo_sample.cfg为zoo.cfg
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/65934af4e9a38df3203aac981a273cf4" alt="" />
修改zoo.cfg文件，修改dataDir,默认端口2181，如有需要修改即可
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/0d60f4d2fe9b3246cdc9bf4a9ef525b3" alt="" />
进入zookeeper的bin目录，启动zookeeper
启动命令:
./zkServer.sh start
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/f7d0cb3b547ad675dc5131f0f2165420" alt="" />
查看命令:
./zkServer.sh status
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/daa7f36f0c3a42087077bf62c14b8608" alt="" />
图片中为启动成功，模式为单点
注册zookeeper为服务
编辑zookeeper服务文件
vim /lib/systemd/system/zookeeper.service
[Unit]
Description=zookeeper service
After=network.target
[Service]
Type=forking
ExecStart=/risen/soft/zookeeper/zookeeper-3.4.13/bin/zkServer.sh start
ExecReload=/risen/soft/zookeeper/zookeeper-3.4.13/bin/zkServer.sh restart
ExecStop=/risen/soft/zookeeper/zookeeper-3.4.13/bin/zkServer.sh stop
PrivateTmp=true
[Install]
WantedBy=multi-user.target
调试zookeeper服务
停止zookeeper服务
systemctl stop zookeeper
启动zookeeper服务
systemctl start zookeeper
查看zookeeper服务状态
systemctl status zookeeper
重启zookeeper服务
systemctl restart zookeeper
设置开机自启
systemctl enable zookeeper
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/952dbfd8655453a05165cc67654c1de7" alt="" /></p>
<h4><strong>安装kafka</strong></h4>
<p>上传或下载安装包至指定目录(文档以/risen/soft/kafka为例)
有安装包则上传安装包([kafka_2.11-2.2.0.tgz](<a href="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/6396e2925da911381dc8075e6186d557">http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/6396e2925da911381dc8075e6186d557</a> "[kafka_2.11-2.2.0.tgz"))，没有则可以使用如下命令直接下载：
wget <a href="http://apache.opencas.org/kafka/0.9.0.1/kafka_2.11-0.9.0.1.tgz">http://apache.opencas.org/kafka/0.9.0.1/kafka_2.11-0.9.0.1.tgz</a>
移动至指定目录下，解压
tar -zxvf kafka_2.11-0.9.0.1.tgz
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/c00488f6293e137216a447486933aac7" alt="" />
修改server.properties (/risen/soft/kafka/kafka_2.11-2.2.0/config)
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/16ec91366fa54cd557453c62fc57dd13" alt="" />
修改kafka监听端口，即使用端口
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/b8cc0e7e32fbb8453133caad0487c64b" alt="" />
修改日志文件夹(修改前创建该文件夹)
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/72702ce2ee3ff00477f2e313d8df0286" alt="" />
配置zookeeper连接
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/4425126151181a33adc74880701a0c28" alt="" />
启动kafka
进入bin目录，执行启动命令
./kafka-server-start.sh -daemon ../config/server.properties
执行后，使用ps -ef|grep kafka查看是否有kafka进程
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/932413739de2fe5f9ddaa6bf8a587e64" alt="" />
验证是否正常使用</p>
<pre><code class="language-java">创建topic
sh kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic test
验证toipic是否创建成功
sh kafka-topics.sh --list --zookeeper 127.0.0.1:2181</code></pre>
<p><img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/43a2f5e5c070072ea0ab5e3f197b3c26" alt="" />
注册kafka服务
vim /lib/systemd/system/kafka.service
添加如下内容(注意kafka目录)：
[Unit]
Description=kafka service
After=zookeeper.target
[Service]
Type=forking
ExecStart=/risen/soft/kafka/kafka_2.11-2.2.0/bin/kafka-server-start.sh -daemon /risen/soft/kafka/kafka_2.11-2.2.0/config/server.properties
ExecReload=/bin/kill -1 $MAINPID
ExecStop=/bin/kill -9 $MAINPID
PrivateTmp=true
KillMode=process
[Install]
WantedBy=multi-user.target
调试kafka服务
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/5dee64d23a894d3d88276e3da0da31f9" alt="" />
停止kafka服务
systemctl stop kafka
启动kafka服务
systemctl start kafka
查看kafka服务状态
systemctl status kafka
重启kafka服务
systemctl restart kafka
重新查看kafka服务状态
systemctl status kafka
设置开机自启
systemctl enable kafka
至此，安装完毕</p>
<h2>3 服务部署</h2>
<p>将项目war包上传至中间件tomcat的webapps路径下
修改数据库连接信息 （路径：log/WEB-INF/classes/com/risen/base/config/connection.properties）
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/956e1c2c3e600faee31c120cc10b2548" alt="" />
es连接信息 (路径：log/WEB-INF/classes/com/risen/base/config/connection.properties)
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/a10426fb8cd150e2952d2a3eb34ef631" alt="" />
启动运行
访问项目地址http://IP:PORT/log，确保项目正常运行即可
<img src="http://60.191.64.5:16100/server/index.php?s=/api/attachment/visitFile/sign/6561b493b431ca1897a823a2500e9ff7" alt="" />
日志采集</p>