[elk]install

ouyida3ouyida3 117 9月15日 发布

logstash -e 'input { stdin { } } output { stdout {codec => rubydebug} }'
原来要等一段时间!

logstash -e 'input { stdin { } } output { elasticsearch { hosts => ["localhost:9200"] } stdout { codec => rubydebug }}'
成功
[webapp@kafka-2 elasticsearch-5.4.3]$ index.sh
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open logstash-2020.09.14 rGfyivbkRmuoHUMh3sNWig 5 1 2 0 14.7kb 14.7kb

日志
/data/webapp/log/logstash-7.6.0/logs
tail -f logstash-plain.log
2020-09-14T09:52:21,715[logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

curl localhost:9600
{"host":"kafka-2","version":"7.6.0","http_address":"127.0.0.1:9600","id":"32755ce8-329d-409f-ba86-df6b7ba150ca","name":"kafka-2","ephemeral_id":"55fa6270-42e9-431a-a4f5-927805122efc","status":"green","snapshot":false,"pipeline":{"workers":16,"batch_size":125,"batch_delay":50},"build_date":"2020-02-06T01:45:20+00:00","build_sha":"f33a9321111930d326b304a980946c8a8d89a445","build_snapshot":false}

elasticsearch-5.4.3
logstash-7.6.0

input {
file {

path => "/data/webapp/logs/interface-st-sms/interface-st-sms.log"
type => "sms"
start_position => "beginning"

}
}
output {
elasticsearch {

hosts => ["localhost:9200"]
index => "sms-%{+YYYY.MM.dd}"

}
}

https://www.orchome.com/486 type随便

logstash -f ../config/logstash.conf
启动要一分多钟

成功:
[webapp@kafka-2 elasticsearch-5.4.3]$ index.sh
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open sms-2020.09.14 gu6wTNUFSnC3LHipYVbFtQ 5 1 154981 0 84.4mb 84.4mb

curl -XGET http://localhost:9200/sms-2020.09.14/mappings/sms?pretty

docker search elasticsearch-head

localhost:~ danni$ docker run -p 9100:9100 mobz/elasticsearch-head:5
Unable to find image 'mobz/elasticsearch-head:5' locally
5: Pulling from mobz/elasticsearch-head

Started connect web server on http://localhost:9100

curl -XGET 'http://localhost:9200/_mapping?pretty=true'
https://blog.csdn.net/sinat_3... 获取所有type

连不上head,设了代理。
docker logs 6e01a5789df4 没报错日志

https://blog.csdn.net/wolfcod...
2、设置连接elasticsearch的地址
head插件默认是连接本机的elasticsearch的,如果你的elasticsearch和head插件是安装在同一台主机上,那么就不需要修改配置,如果不是安装在同一台主机的,就必须修改配置了,配置文件在head插件安装目录的_site目录下,文件名为app.js。
vi app.js #编辑app.js文件
把this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";这行配置中的localhost改成你elasticsearch服务所在IP地址(如果安装在同一台主机就不需要修改),具体看以下配置文件截图:

3、elasticsearch配置允许跨域访问
cd /usr/local/elasticsearch/config #进入elasticsearch存放配置文件的目录
vi elasticsearch.ym #编辑elasticsearch配置文件
在该配置文件中最末尾添加两个属性:http.cors.enabled: true和http.cors.allow-origin: "*" 使head插件可以访问elasticsearch,具体看以下配置文件截图:

elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"
成功了

after

es停了,logstash会不断报错

启动
bin/kibana
比较久。

设置代理
http://localhost:5601/
Kibana server is not ready yet

License information could not be obtained from Elasticsearch due to [illegal_argument_exception] No endpoint or operation is available at [_xpack] :: {"path":"/_xpack","statusCode":400,"response":"{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"No endpoint or operation is available at [_xpack]"}],"type":"illegal_argument_exception","reason":"No endpoint or operation is available at [_xpack]"},"status":400}"} error

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://localhost:9200"
kibana.index: ".kibana"

FATAL Error: [config validation of [elasticsearch].url]: definition for this key is missing
应该是:
elasticsearch.hosts: ["http://localhost:9200"]

https://blog.csdn.net/weixin_... 安装xpack

  1. 安装kibana首先要求就是版本号要与elasticsearch的版本号相符
  2. 这是以为elasticsearch下的x-pack没有装好,只有kibana那边的x-pack是OK的,这个时候看一下是否是在elasticsearch启动的情况下安装的x-pack,注意,这里需要停止elasticsearch再安装
  3. 更改kibana里的kibana.yml,把“server.host: localhost”改成“server.host: 0.0.0.0”

https://www.jianshu.com/p/f1b... 改密码changeme
./elasticsearch-plugin install file:///data/webapp/log/x-pack-5.4.3.zip
等很久
Continue with installation? [y/N]Exception in thread "main" java.lang.RuntimeException: installation aborted by user

好像不停es安装xpack也行。
ERROR: plugin directory [/data/webapp/log/elasticsearch-5.4.3/plugins/x-pack] already exists; if you need to update the plugin, uninstall it first using command 'remove x-pack'

http://localhost:9200/
要输入密码了!
elastic/changeme

xpack就是用来安全认证的

新错误
License information could not be obtained from Elasticsearch due to [security_exception] missing authentication token for REST request [/_xpack], with { header={ WWW-Authenticate="Basic realm="security" charset="UTF-8"" } } :: {"path":"/_xpack","statusCode":401,"response":"{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/_xpack]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/_xpack]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}","wwwAuthenticateDirective":"Basic realm="security" charset="UTF-8""} error
解决:
elasticsearch.username: "elastic"
elasticsearch.password: "changeme"
https://www.jianshu.com/p/f1b...
https://discuss.elastic.co/t/...

log [07:17:48.864] error This version of Kibana (v7.9.1) is incompatible with the following Elasticsearch nodes in your cluster: v5.4.3 @ 127.0.0.1:9200 (127.0.0.1)
版本不对!

https://www.elastic.co/cn/dow...

output {
elasticsearch {

user => elastic
password => changeme
hosts => ["localhost:9200"]
index => "sms-%{+YYYY.MM.dd}"

}
}
加用户名密码:
https://blog.csdn.net/QiaoRui...
https://www.elastic.co/guide/... 官网也是不需要引号
成功

logstash kill不掉。要kill -9

es head增加用户密码登录
http.cors.allow-headers: Authorization
/?auth_user=elastic&auth_password=changeme
http://localhost:9100/?auth_user=elastic&auth_password=changeme
成功
https://blog.csdn.net/vah101/...

ps | grep elastic | awk 'NR==1{print $1}' | xargs kill
ps -ef |grep elastic|grep 8480 |awk '{print $2}'|xargs kill -9

ps -ef |grep elastic |grep -v grep|grep -v x-pack|awk '{print $2}'|xargs kill
ps -ef |grep node |grep -v grep|awk '{print $2}'|xargs kill

5.4.3的时候确实叫url,不叫host

The URL of the Elasticsearch instance to use for all your queries.

elasticsearch.url: "http://localhost:9200"

localhost:5601
成功了

用户名密码就是elastic/changeme
首先要management - create

停止kibana
https://blog.csdn.net/gong_ya...
ps -ef|grep node

默认是 logstash-*
_type:"doc"

Management - Index Patterns - +图片
sms-* - create

message:"条数据"

https://blog.csdn.net/u010509...
之前用logstash做日志采集,但是发现logstash很占用机器资源导致机器运行有点慢。
查询资料表明logstash使用Java编写,插件是使用jruby编写,对机器的资源要求会比较高,网上有一篇关于其性能测试的报告。
之前做过和filebeat的测试对比。在采集日志方面,对CPU,内存上都要比前者高很多。那么果断使用filebeat作为替代方案

评论
载入中...