Helm3-安装带有ik分词的ElasticSearch

前言

上一篇写了Helm3安装ElasticSearch和Kibana，但是发现没有安装ik中文分词，于是在此基本上操作如何安装带有ik分词的elasticsearch分享给大家。

操作

在操作之前我要感谢smokelee的文章：k8s无脑系列(十一)-安装ElasticSearch到集群并设置中文分词，我的这篇文章也是基于此文章的。

操作步骤：
1、下载ik分词包
2、制作带有ik分词的docker镜像
3、修改bitnami/elasticsearch的values.yaml文件，并使用刚刚制作的docker镜像
4、安装elasticsearch和kibana
5、在kibana中使用develop tools验证ik是否安装成功

1、下载ik和pinyin分词包

下面是包地址，大家可以选择合适的版本下载即可。

https://github.com/medcl/elasticsearch-analysis-ik/releases
https://github.com/medcl/elasticsearch-analysis-pinyin/releases

2、制作带有ik和pinyin分词的docker镜像

下载完成之后，将文件夹按照下图显示排放

然后，我们来编写Dockerfile

FROM docker.io/bitnami/elasticsearch:8.2.2-debian-11-r0
COPY ik /opt/bitnami/elasticsearch/plugins/ik

注意，这里我们使用的是bitnami/elasticsearch的docker镜像，版本号为8.2.2-debian-11-r0，大家可以参考这个地址选用合适的版本，还有另外一种办法找到这个版本，就是查看elasticsearch中的values.yaml文件，如下所示：

接着，我们在docker目录下按顺序执行下面命令来制作含有ik和pingyin分词的docker镜像。

docker login --username=xxx registry.cn-hangzhou.aliyuncs.com
docker build -t registry.cn-hangzhou.aliyuncs.com/com-seaurl/seaurl-elasticsearch:x.x.x  .
docker push registry.cn-hangzhou.aliyuncs.com/com-seaurl/seaurl-elasticsearch:x.x.x

上面的地址我是使用阿里云的，大家按照规范设置成自己的镜像中心地址就好，这样我们就完成了docker镜像的制作。

3、修改bitnami/elasticsearch的values.yaml文件，并使用刚刚制作的docker镜像

按照上篇文章中的讲解，我们来配置下values.yaml文件

global:
  storageClass: "alicloud-cnfs-nas"
  elasticsearch:
    service:
      name: elasticsearch
      ports:
        restAPI: 9200
  kibanaEnabled: false
image:
  registry: registry.cn-hangzhou.aliyuncs.com
  repository: com-seaurl/seaurl-elasticsearch
  # 这里的x.x.x就是你刚才制作docker镜像的版本号：x.x.x
  tag: x.x.x
sysctlImage:
  enabled: false
plugins: analysis-icu
service:
  type: NodePort
master:
  replicaCount: 1
data:
  replicaCount: 1
coordinating:
  replicaCount: 1
ingest:
  replicaCount: 1

4、安装elasticsearch和kibana

执行下面命令安装elasticsearch，

➜  ~ kubectl create ns elasticsearch
namespace/elasticsearch created

➜  ~ helm install -f test-values.yaml test-elasticsearch bitnami/elasticsearch --namespace elasticsearch

输出：

elasticsearch
NAME: test-elasticsearch
LAST DEPLOYED: Mon Jul 11 21:44:14 2022
NAMESPACE: elasticsearch
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: elasticsearch
CHART VERSION: 18.2.9
APP VERSION: 8.2.2

-------------------------------------------------------------------------------
 WARNING

    Elasticsearch requires some changes in the kernel of the host machine to
    work as expected. If those values are not set in the underlying operating
    system, the ES containers fail to boot with ERROR messages.

    To check whether the host machine meets the requirements, run the command
    below:

      kubectl logs --namespace elasticsearch $(kubectl get --namespace elasticsearch \
        pods -l app=elasticsearch,role=master -o jsonpath='{.items[0].metadata.name}') \
        elasticsearch

    You can adapt the Kernel parameters on you cluster as described in the
    official documentation:

      https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster

    As an alternative, you can specify "sysctlImage.enabled=true" to use a
    privileged initContainer to change those settings in the Kernel:

      helm upgrade --namespace elasticsearch test-elasticsearch bitnami/elasticsearch --set sysctlImage.enabled=true

    Note that this requires the ability to run privileged containers, which is likely not
    the case on many secure clusters. To cover this use case, you can also set some parameters
    in the config file to customize the default settings:

      https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-store.html
      https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-virtual-memory.html

    For that, you can place the desired parameters by using the "config" block present in the values.yaml

** Please be patient while the chart is being deployed **

  Elasticsearch can be accessed within the cluster on port 9200 at test-elasticsearch.elasticsearch.svc.cluster.local

  To access from outside the cluster execute the following commands:

    export NODE_PORT=$(kubectl get --namespace elasticsearch -o jsonpath="{.spec.ports[0].nodePort}" services test-elasticsearch)
    export NODE_IP=$(kubectl get nodes --namespace elasticsearch -o jsonpath="{.items[0].status.addresses[0].address}")
    curl http://$NODE_IP:$NODE_PORT/

然后，我们通过上面命令获取ip和port

# 获取port
kubectl get --namespace elasticsearch -o jsonpath="{.spec.ports[0].nodePort}" services test-elasticsearch

# 获取ip
kubectl get nodes --namespace elasticsearch -o jsonpath="{.items[0].status.addresses[0].address}"

获取elasticsearch的ip和port之后，我们来安装kibana，执行下面命令：

helm install -f test-values.yaml test-kibana bitnami/kibana --namespace elasticsearch    

输出：

NAME: test-kibana
LAST DEPLOYED: Mon Jul 11 21:52:08 2022
NAMESPACE: elasticsearch
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: kibana
CHART VERSION: 10.1.9
APP VERSION: 8.2.2

** Please be patient while the chart is being deployed **

1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace elasticsearch -o jsonpath="{.spec.ports[0].nodePort}" services test-kibana)
  export NODE_IP=$(kubectl get nodes --namespace elasticsearch -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

WARNING: Kibana is externally accessible from the cluster but the dashboard does not contain authentication mechanisms. Make sure you follow the authentication guidelines in your Elastic stack.
+info https://www.elastic.co/guide/en/elasticsearch/reference/current/setting-up-authentication.html

然后，我们通过上面命令获取ip和port

# 获取port
kubectl get --namespace elasticsearch -o jsonpath="{.spec.ports[0].nodePort}" services test-kibana

# 获取ip
kubectl get nodes --namespace elasticsearch -o jsonpath="{.items[0].status.addresses[0].address}"

5、在kibana中使用develop tools验证ik是否安装成功

安装好elasticsearch和kibana之后，我们尝试在Chrome里面打开Dev Tools，验证ik分词包是否安装成功，于是我们输入下面代码并执行：

POST _analyze
{
  "analyzer": "ik_smart",
  "text": "我喜欢你"
}

成功！！！

2023-5-24 问题1：ik分词和elasticsearch版本不一致

我在使用最新版本8.7.0安装ik分词时使用下面的dockerfile代码：

Dockerfile

FROM docker.io/bitnami/elasticsearch:latest
COPY ik /opt/bitnami/elasticsearch/plugins/ik

执行下面命令制作docker镜像：

docker login --username=xxx registry.cn-hangzhou.aliyuncs.com
docker build -t registry.cn-hangzhou.aliyuncs.com/com-seaurl/seaurl-elasticsearch:x.x.x  .
docker push registry.cn-hangzhou.aliyuncs.com/com-seaurl/seaurl-elasticsearch:x.x.x

然后编写k8s文件

global:
  storageClass: alibabacloud-cnfs-nas
  elasticsearch:
    service:
      name: elasticsearch
      ports:
        restAPI: 9200
  kibanaEnabled: false
image:
  registry: registry.cn-hangzhou.aliyuncs.com
  repository: xxx/xxx
  tag: x.x.x
sysctlImage:
  enabled: false
plugins: analysis-icu
service:
  type: NodePort
master:
  replicaCount: 1
data:
  replicaCount: 1
coordinating:
  replicaCount: 1
ingest:
  replicaCount: 1

接着执行helm命令：

helm install -f test-values.yaml elasticsearch bitnami/elasticsearch --namespace elasticsearch

查看安装情况：

➜  Helm k get all -n elasticsearch
NAME                               READY   STATUS             RESTARTS        AGE
pod/elasticsearch-coordinating-0   0/1     CrashLoopBackOff   6 (2m50s ago)   8m37s
pod/elasticsearch-data-0           0/1     CrashLoopBackOff   6 (2m37s ago)   8m37s
pod/elasticsearch-ingest-0         0/1     CrashLoopBackOff   6 (2m45s ago)   8m37s
pod/elasticsearch-master-0         0/1     CrashLoopBackOff   6 (2m30s ago)   8m37s

NAME                                    TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)                         AGE
service/elasticsearch                   NodePort    192.168.127.102   <none>        9200:30482/TCP,9300:30313/TCP   8m38s
service/elasticsearch-coordinating-hl   ClusterIP   None              <none>        9200/TCP,9300/TCP               8m38s
service/elasticsearch-data-hl           ClusterIP   None              <none>        9200/TCP,9300/TCP               8m38s
service/elasticsearch-ingest-hl         ClusterIP   None              <none>        9200/TCP,9300/TCP               8m38s
service/elasticsearch-master-hl         ClusterIP   None              <none>        9200/TCP,9300/TCP               8m38s

NAME                                          READY   AGE
statefulset.apps/elasticsearch-coordinating   0/1     8m38s
statefulset.apps/elasticsearch-data           0/1     8m38s
statefulset.apps/elasticsearch-ingest         0/1     8m38s
statefulset.apps/elasticsearch-master         0/1     8m38s

NAME                                                                                             AGE
containernetworkfilesystem.storage.alibabacloud.com/cnfs-nas-cc845fc16cd884ceb9497c74e10e532d4   177d

发现一直报错，然后去阿里云控制台，发现报下面的错：

0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling. 

提了工单，问了阿里技术人员，他们的解释是：

您拿着这个分词镜像docker 启动一下测试下，能不能运行起来呢，目前是镜像里面代码跑不起来
先用docker调试运行没问题之后，再在ACK集群里面跑

然后我按照他的提示，把制作的分词docker镜像在本地运行起来

docker run -d --name=es-ik -p 9200:9200 -p 9300:9300 f58508bc62ce

然后发现容器启动失败

➜  Helm docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

然后查看容器的日志，输出如下：

➜  Helm docker logs 25f4767af358
elasticsearch 02:58:10.75
elasticsearch 02:58:10.75 Welcome to the Bitnami elasticsearch container
elasticsearch 02:58:10.75 Subscribe to project updates by watching https://github.com/bitnami/containers
elasticsearch 02:58:10.75 Submit issues and feature requests at https://github.com/bitnami/containers/issues
elasticsearch 02:58:10.76
elasticsearch 02:58:10.76 INFO  ==> ** Starting Elasticsearch setup **
warning: ignoring JAVA_HOME=/opt/bitnami/java; using ES_JAVA_HOME
elasticsearch 02:58:11.28 WARN  ==> Setting ELASTICSEARCH_IS_DEDICATED_NODE is disabled.
elasticsearch 02:58:11.28 WARN  ==> ELASTICSEARCH_NODE_ROLES and ELASTICSEARCH_NODE_TYPE will be ignored and Elasticsearch will asume all different roles.
elasticsearch 02:58:11.28 INFO  ==> Configuring/Initializing Elasticsearch...
elasticsearch 02:58:11.31 INFO  ==> Setting default configuration
elasticsearch 02:58:11.33 INFO  ==> Configuring Elasticsearch cluster settings...
warning: ignoring JAVA_HOME=/opt/bitnami/java; using ES_JAVA_HOME
elasticsearch 02:58:12.07 INFO  ==> ** Elasticsearch setup finished! **

elasticsearch 02:58:12.08 INFO  ==> ** Starting Elasticsearch **
warning: ignoring JAVA_HOME=/opt/bitnami/java; using ES_JAVA_HOME
[2023-05-24T02:58:13,741][INFO ][o.e.n.Node               ] [25f4767af358] version[8.7.1], pid[353], build[tar/f229ed3f893a515d590d0f39b05f68913e2d9b53/2023-04-27T04:33:42.127815583Z], OS[Linux/5.15.49-linuxkit/aarch64], JVM[BellSoft/OpenJDK 64-Bit Server VM/17.0.7/17.0.7+7-LTS]
[2023-05-24T02:58:13,742][INFO ][o.e.n.Node               ] [25f4767af358] JVM home [/opt/bitnami/java], using bundled JDK [false]
[2023-05-24T02:58:13,742][INFO ][o.e.n.Node               ] [25f4767af358] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-12279741605701294257, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms1024m, -Xmx1024m, -XX:MaxDirectMemorySize=536870912, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/opt/bitnami/elasticsearch/lib, --add-modules=jdk.net, -Djdk.module.main=org.elasticsearch.server]
[2023-05-24T02:58:14,457][INFO ][c.a.c.i.j.JacksonVersion ] [25f4767af358] Package versions: jackson-core=2.13.4, jackson-databind=2.13.4-2, jackson-dataformat-xml=2.13.4, jackson-datatype-jsr310=2.13.4, azure-core=1.34.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot
[2023-05-24T02:58:14,993][ERROR][o.e.b.Elasticsearch      ] [25f4767af358] fatal exception while booting Elasticsearch java.lang.IllegalArgumentException: Plugin [analysis-ik] was built for Elasticsearch version 8.7.0 but version 8.7.1 is running
    at org.elasticsearch.server@8.7.1/org.elasticsearch.plugins.PluginsUtils.verifyCompatibility(PluginsUtils.java:108)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:424)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:292)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:161)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.plugins.PluginsService.lambda$getPluginsServiceCtor$15(PluginsService.java:668)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.node.Node.<init>(Node.java:417)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.node.Node.<init>(Node.java:324)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216)
    at org.elasticsearch.server@8.7.1/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67)

ERROR: Elasticsearch did not exit normally - check the logs at /opt/bitnami/elasticsearch/logs/elasticsearch.log

ERROR: Elasticsearch exited unexpectedly

上面的错误信息已经明确了就是版本不一致导致的：

Plugin [analysis-ik] was built for Elasticsearch version 8.7.0 but version 8.7.1 is running

bitnami/elasticsearch的版本是8.7.1，所以我们把ik8.7.1分词代码里面的代码换成这种然后重新制作镜像发布并执行就ok了。

FROM docker.io/bitnami/elasticsearch:latest
COPY ik /opt/bitnami/elasticsearch/plugins/ik

2023-5-24 问题1：ik分词打包的镜像CPU版本不一致

先把问题抛出来：

/opt/bitnami/ scripts/elasticsearch/entrypoint. sh: exec format error

分析了半天，原来跟打包docker镜像的电脑CPU有关，如果你的CPU是intel的，那么你docker打包的镜像就是amd64(x86)，如果你的CPU是arm的，那么你用docker打包的镜像就是arm架构，而我的macbook就是m1max的arm架构的芯片所以打包之后是arm所以才会报上面的错，然后我在家里的macmini intel i5打包之后就没问题了，所以还是跟你打包所属的CPU环境有关，切记！

总结

Dockerfile里面的8.2.2-debian-11-r0这个版本号不要写错了，我当时就是因为这个错纠结了一个星期，大家不要犯我一样的错！

FROM docker.io/bitnami/elasticsearch:8.2.2-debian-11-r0

引用

k8s无脑系列(十一)-安装ElasticSearch到集群并设置中文分词