Rootop 服务器运维与web架构

elk日志分析安装及简单配置

服务器信息:
ip     :   192.168.1.50
系统:centos7.3 x64
elk官网:https://www.elastic.co/downloads
elk版本:5.4.0

elk 3个组件全部安装到一台机器测试

环境配置:
1、修改主机名及hosts映射

[root@elk ~]# hostname elk
[root@elk ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.50 elk
[root@elk ~]# cat /etc/hostname
elk

2、安装jdk1.8
过程略…

[root@elk ~]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

3、iptables停掉,firewalld停掉(安装完成后手动开放相应端口)。

安装elasticsearch:

[root@localhost ELK]# rpm -ivh elasticsearch-5.4.0.rpm
warning: elasticsearch-5.4.0.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
Preparing... ################################# [100%]
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Updating / installing...
 1:elasticsearch-0:5.4.0-1 ################################# [100%]
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

修改elasticsearch配置文件:

[root@elk ~]# cat /etc/elasticsearch/elasticsearch.yml | grep -v "#"
node.name: elk # 主机名
path.data: /home/elk/data # 数据目录
path.logs: /home/elk/logs # 日志目录
network.host: 192.168.1.50 # 监听地址
http.port: 9200 # 监听端口
discovery.zen.ping.unicast.hosts: ["elk"] # 手动发现节点
http.cors.enabled: true # 允许跨域
http.cors.allow-origin: "*" # 允许跨域

# 创建目录

[root@elk ~]# mkdir -p /home/elk/data
[root@elk ~]# mkdir -p /home/elk/logs

# 写入权限,否则/var/log/message里会提示拒绝访问
[root@elk ~]# chmod -R 777 /home/elk

# 文件打开数
[root@elk ~]# cat /etc/security/limits.conf | grep -v "#" | grep -v "^$"
* soft nofile 65535
* hard nofile 65535
# java命令软连接,否则启动es的时候日志报找不到java
[root@localhost ~]# ln -s /usr/local/jdk/bin/java /usr/local/bin/java
# 开机启动
[root@elk ~]# systemctl enable elasticsearch.service

# 启动elasticsearch
[root@elk ~]# systemctl start elasticsearch.service

# 查看elasticsearch端口状态
[root@elk ~]# netstat -tnlp | grep -E "9200|9300"
tcp6 0 0 192.168.1.50:9200 :::* LISTEN 5419/java
tcp6 0 0 192.168.1.50:9300 :::* LISTEN 5419/java


安装elasticsearch图形插件:
先安装nodejs

[root@elk ~]# yum install -y epel-*
[root@elk ~]# yum install -y nodejs

# 安装node.js构建工具
[root@elk ~]# npm install -g grunt

# 克隆elasticsearch-head
[root@elk ~]# cd /usr/local/
[root@elk local]# git clone git://github.com/mobz/elasticsearch-head.git
[root@elk local]# cd elasticsearch-head/
[root@elk elasticsearch-head]# npm install phantomjs-prebuilt@2.1.13 --ignore-scripts

# 修改 _site/app.js 里的地址
[root@elk elasticsearch-head]# vi _site/app.js

this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";
改为
this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://192.168.1.50:9200";

# 启动 elasticsearch-head
[root@elk elasticsearch-head]# grunt server &
# 监听在9100端口

安装logstash:

[root@elk ELK]# ln -s /usr/local/jdk/bin/java /usr/bin/java
[root@elk ELK]# rpm -ivh logstash-5.4.0.rpm

# 创建配置文件目录
[root@elk ~]# mkdir /usr/share/logstash/conf
[root@elk ~]# cd /usr/share/logstash/conf

[root@elk conf]# cat test.conf
input {
 file {
 type => "nginx_log"
 path => "/var/log/nginx/access.log"
 }
}
output {
 elasticsearch {
 hosts => "192.168.1.50"
 index => "nginx-access-%{+YYYY.MM.dd}"
 }
 stdout {
 codec => rubydebug
 }
}

# 启动logstash
[root@elk conf]# /usr/share/logstash/bin/logstash -f /usr/share/logstash/conf/test.conf &

# yum 安装nginx测试,启动nginx,访问以下80端口,产生访问日志,去es里查看

安装kibana:

[root@elk ELK]# rpm -ivh kibana-5.4.0-x86_64.rpm
修改配置文件:
[root@elk ~]# cat /etc/kibana/kibana.yml | grep -v "#" | grep -v "^$"
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://elk:9200"

# 启动kibana ,端口5601
[root@elk ELK]# systemctl start kibana

浏览器访问

PS:
项目环境的jdk可能不会与elk的jdk版本一直,就会存在两个版本jdk
logstash可以修改java版本,通过在/usr/share/logstash/bin/logstash头部添加:
#!/bin/sh
JAVACMD=/usr/local/jdk1.8.0_131/bin/java

在/etc/logstash/startup.options 配置中发现可以修改jdk,但是始终没有生效。还是选择上面方法。
# After changing anything here, you need to re-run $LS_HOME/bin/system-install
# as root to push the changes to the init script.
################################################################################

# Override Java location
JAVACMD=/usr/local/jdk1.8.0_131/bin/java

最新的5.4版本要求jdk1.8,如果低于1.8,组件无法启动。

原创文章,转载请注明。本文链接地址: https://www.rootop.org/pages/3781.html

作者:Venus

服务器运维与性能优化

评论已关闭。