elasticsearch6.0安装xpack并配置ldap认证

elasticsearch概念解释参考： https://segmentfault.com/a/11...
elasticsearch安装可参考：https://segmentfault.com/a/11...

安装xpack扩展

下载xpack插件包： https://artifacts.elastic.co/...

通过elasticsearch-plugin命令安装xpack；

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/elasticsearch-plugin install file:///home/elasticsearch/software/x-pack-6.0.0.zip --batch
-> Downloading file:///home/elasticsearch/software/x-pack-6.0.0.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \\.\pipe\* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
* java.util.PropertyPermission sun.nio.ch.bugLevel write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@        WARNING: plugin forks a native controller        @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.
Elasticsearch keystore is required by plugin [x-pack], creating...
-> Installed x-pack
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

配置java相关权限上述操作提示即为需添加下述配置
添加下述配置至$JAVA_HOME/jre/lib/security/java.policy文件
permission java.lang.RuntimePermission
"accessClassInPackage.com.sun.activation.registries"; permission
java.lang.RuntimePermission "getClassLoader"; permission
java.lang.RuntimePermission "setContextClassLoader"; permission
java.lang.RuntimePermission "setFactory"; permission
java.security.SecurityPermission "createPolicy.JavaPolicy"; permission
java.security.SecurityPermission "getPolicy"; permission
java.security.SecurityPermission "putProviderProperty.BC"; permission
java.security.SecurityPermission "setPolicy"; permission
java.util.PropertyPermission "*","read,write"; permission
java.util.PropertyPermission "sun.nio.ch.bugLevel","write"; permission
javax.net.ssl.SSLPermission "setHostnameVerifier";

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ vim ~/software/jdk1.8.0_121/jre/lib/security/java.policy

图片描述

通过ES_HOME/bin/x-pack/certgen生成ssl证书

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/x-pack/certgen
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL in the Elastic stack. Depending on the command
line option specified, you may be prompted for the following:

* The path to the output file
    * The output file is a zip file containing the signed certificates and
      private keys for each instance. If a Certificate Authority was generated,
      the certificate and private key will also be included in the output file.
* Information about each instance
    * An instance is any piece of the Elastic Stack that requires a SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.
* Certificate Authority private key password
    * The password may be left empty if desired.

Let's get started...

Please enter the desired output file [certificate-bundle.zip]: 
Enter instance name: elasticsearch
Enter name for directories and files [elasticsearch]: 
Enter IP Addresses for instance (comma-separated if more than one) []: 127.0.0.1,10.59.30.96,10.59.30.97
Enter DNS names for instance (comma-separated if more than one) []: elasticsearch,elasticsearch-1,elasticsearch-2
Would you like to specify another instance? Press 'y' to continue entering instance information: 
Certificates written to /home/elasticsearch/software/elasticsearch-6.0.0/certificate-bundle.zip

This file should be properly secured as it contains the private keys for all
instances and the certificate authority.

After unzipping the file, there will be a directory for each instance containing
the certificate and private key. Copy the certificate, key, and CA certificate
to the configuration directory of the Elastic product that they will be used for
and follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

图片描述

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ls
bin  certificate-bundle.zip  config  data  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.textile
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ unzip certificate-bundle.zip -d config
Archive:  certificate-bundle.zip
   creating: config/ca/
  inflating: config/ca/ca.crt        
  inflating: config/ca/ca.key        
   creating: config/elasticsearch/
  inflating: config/elasticsearch/elasticsearch.crt  
  inflating: config/elasticsearch/elasticsearch.key
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

配置下述参数至ES_HOME/config/elasticsearch.yml开启ssl支持
xpack.ssl.key: elasticsearch/elasticsearch.key
xpack.ssl.certificate: elasticsearch/elasticsearch.crt
xpack.ssl.certificate_authorities: ca/ca.crt
xpack.security.transport.ssl.enabled: true

图片描述

自定义内置账户(elastic、kibana、logstash_system)密码
账户elastic为elasticsearch超级管理员，拥有所有权限
账户kibana用于kibana组件获取相关信息用于web展示
账户logstash_system用于logstash服务获取elasticsearch的监控数据
注意：此步骤需先启动elasticsearch服务

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/x-pack/setup-passwords interactive
Initiating the setup of reserved user elastic,kibana,logstash_system passwords.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [elastic]
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

验证内置账户访问

若不提供用户名密码则返回401

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "missing authentication token for REST request [/_cat/indices?pretty]",
        "header" : {
          "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "missing authentication token for REST request [/_cat/indices?pretty]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}

提供相应用户信息后可访问，若用户权限不足则返回403
使用logstash_system用户访问

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u logstash_system:logstash_system
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [indices:monitor/stats] is unauthorized for user [logstash_system]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [indices:monitor/stats] is unauthorized for user [logstash_system]"
  },
  "status" : 403
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

使用kibana用户访问

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:kibana
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1    
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0    
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1    
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

添加自定义角色

添加角色接口为 POST /_xpack/security/role/<rolename>
下述示例为添加超级管理员角色的方法

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role/admin?pretty' -d '{
>   "run_as": [ "elastic" ],
>   "cluster": [ "all" ],
>   "indices": [
>     {
>       "names": [ "*" ],
>       "privileges": [ "all" ]
>     }
>   ]
> }'
{
  "role" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role/admin?pretty'
{
  "admin" : {
    "cluster" : [
      "all"
    ],
    "indices" : [
      {
        "names" : [
          "*"
        ],
        "privileges" : [
          "all"
        ]
      }
    ],
    "run_as" : [
      "elastic"
    ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

添加自定义账户

添加用户接口为 POST /_xpack/security/user/<username>
下述为添加rocshen账户并添加至admin角色操作方法

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/rocshen?pretty' -d '{
>   "password" : "123456",
>   "full_name" : "Roc Shen",
>   "roles" : ["admin"],
>   "email" : "rocshen@rocshen.com"
> }'
{
  "user" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/rocshen?pretty'
{
  "rocshen" : {
    "username" : "rocshen",
    "roles" : [
      "admin"
    ],
    "full_name" : "Roc Shen",
    "email" : "rocshen@rocshen.com",
    "metadata" : { },
    "enabled" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u rocshen:123456 'http://10.59.30.96:9200/_cat/indices?pretty'
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1 4883 88   2.5mb   2.5mb
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    0  0  24.2kb  24.2kb
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0                        
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1  630  0 703.3kb 703.3kb
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    5  0  33.3kb  33.3kb
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    1  0   6.5kb   6.5kb
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

修改账户密码

修改密码需使用超级管理员权限即elastic账户，接口为POST _xpack/security/user/<username>/_password
curl参数含义如下
-XPOST 使用post方法传递参数
-H 指定http协议的header信息
-u 指定用于认证的用户信息用户名与密码使用冒号分隔
-d 指定具体要传递的参数信息

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/kibana/_password?pretty' -d '{"password": "123456"}'
{ }

密码修改后使用老密码访问则返回401，使用更新后的密码则正常

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:kibana
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "failed to authenticate user [kibana]",
        "header" : {
          "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "failed to authenticate user [kibana]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:123456
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1    
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0    
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1    
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

配置ldap帐号认证

ldap服务安装可参考：https://segmentfault.com/a/11...
添加下述ldap相关述配置 bind_dn为ldap的管理DN bind_password为管理dn的密码
user_search.base_dn为linux系统账户信息导入ldap的信息
user_search.attribute为账户在ldap中的标识信息
group_search.base_dn为linux系统组信息导入ldap的信息

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ vim config/elasticsearch.yml 

......

network.host: 10.59.30.96
bootstrap.system_call_filter: false

xpack.ssl.key: elasticsearch/elasticsearch.key
xpack.ssl.certificate: elasticsearch/elasticsearch.crt
xpack.ssl.certificate_authorities: ca/ca.crt
xpack.security.transport.ssl.enabled: true

xpack:
  security:
    authc:
      realms:
        ldap1:
          type: ldap
          order: 0
          url: "ldap://10.59.30.95"
          bind_dn: "cn=Manager, dc=rocshen, dc=com"
          bind_password: 123456
          user_search:
            base_dn: "ou=People,dc=rocshen,dc=com"
            attribute: uid
          group_search:
            base_dn: "ou=Group,dc=rocshen,dc=com"
          unmapped_groups_as_roles: false

图片描述

配置AD域帐号认证

添加下ldap相关述配置至elasticsearch.yml，此处为接着上述LDAP配置添加，如果只需配置AD认证请将ldap相关配置删除即可；
domain_name为AD域的域名
url为AD域的地址
bind_dnw为随意的域账户名称（格式为user@domain）
bind_password为上述账户的密码

xpack:
  security:
    authc:
      realms:
        ldap1:
          type: ldap
          order: 0
          url: "ldap://10.59.30.94"
          bind_dn: "cn=Manager, dc=rocshen, dc=com"
          bind_password: 123456
          user_search:
            base_dn: "ou=People,dc=rocshen,dc=com"
            attribute: uid
          group_search:
            base_dn: "ou=Group,dc=rocshen,dc=com"
          unmapped_groups_as_roles: false
        active_directory:
          type: active_directory
          order: 1
          domain_name: rocshen.com
          url: ldap://ad.rocshen.com
          bind_dn: rocshen@rocshen.com
          bind_password: AD.123456

图片描述

重启elasticsearch服务并使用ldap域账户user01登录

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ killall java
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/elasticsearch -d
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user01:user01 'http://10.59.30.96:9200/_cat?pretty'
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

使用AD域账户rocshen登录

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl http://10.59.30.96:9200/_cat?pretty -u rocshen:AD.123456
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

为域账户信息映射角色

接口为：POST /_xpack/security/role_mapping/<name>
下述为映射user1*账户为管理员角色的操作步骤

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role_mapping/ldap_user_admin?pretty' -d '{
>   "roles": [ "admin" ],
>   "enabled": true,
>   "rules": {
>     "any": [
>       {
>         "field": {
>           "username": "/user1*/"
>         }
>       }
>     ]
>   }
> }'
{
  "role_mapping" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role_mapping/ldap_user_admin?pretty'
{
  "ldap_user_admin" : {
    "enabled" : true,
    "roles" : [
      "admin"
    ],
    "rules" : {
      "any" : [
        {
          "field" : {
            "username" : "/user1*/"
          }
        }
      ]
    },
    "metadata" : { }
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

验证域账户权限，使用user01无权访问indices接口，使用user11可以访问；

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user01:user01 'http://10.59.30.96:9200/_cat/indices?pretty'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [cluster:monitor/state] is unauthorized for user [user01]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [cluster:monitor/state] is unauthorized for user [user01]"
  },
  "status" : 403
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user11:user11 'http://10.59.30.96:9200/_cat/indices?pretty'
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1 6178 44  5.9mb  5.9mb
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    0  0 11.7kb 11.7kb
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0                      
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1  777  0  1.1mb  1.1mb
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    5  0 40.2kb 40.2kb
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    1  0 12.8kb 12.8kb
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

常见报错

No subject alternative names matching IP address

[2018-01-10T19:19:35,483][WARN ][o.e.x.s.t.n.SecurityNetty4Transport] [fzP4t-4] exception caught on transport layer [[id: 0x5d97fe48, L:/0:0:0:0:0:0:0:1:49121 ! R:/0:0:0:0:0:0:0:1:9300]], closing connection
    io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
......
Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address 0:0:0:0:0:0:0:1 found

解决方案为一种是关闭IPv6地址，另一种是修改ES_HOME/config/elasticsearch.yml中的network.host值为本机eth0的IP

参考文档

官方安装步骤：https://www.elastic.co/guide/...
配置内置账户密码：
https://www.elastic.co/guide/...
修改账户密码：
https://www.elastic.co/guide/...
用户相关操作：
https://www.elastic.co/guide/...
使用LDAP认证： https://www.elastic.co/guide/...
用户角色映射： https://www.elastic.co/guide/...

elasticsearch6.0安装xpack并配置ldap认证

安装xpack扩展

验证内置账户访问

添加自定义角色

添加自定义账户

修改账户密码

配置ldap帐号认证

配置AD域帐号认证

为域账户信息映射角色

常见报错

参考文档

已注销

引用和评论

ECS实例搭建阿里云RDS实例的从库

ElasticSearch源码本地调试指南

一夜之间，7.0万 Star，全部清零！

Elasticsearch 8.x 重要变化（qbit）

KylinV10 ARM 版本安装 Ambari + HDP + ElasticSerach 记录

Elasticsearch Serverless 高性价比智能日志分析关键技术解读

Elasticsearch 查看节点状态信息（qbit）