在第一篇文章中，我们已经可以通过 docker 安装 elasticsearch 和 kibana 了。那么这次就直接进入实战演练。
我们会先准备数据，针对不同常见应用场景，然后分别通过 Query DSL 和 Spring Data JPA 来实现。
Query DSL：ElasticSearch提供了一个可以执行的JSON风格的DSL(domain-specific language 领域特定语言)，这个被称为Query DSL。
1. 准备
1.1. 索引数据准备
下面就是通过 Query DSL 维护了一个名为 operation_log 的索引，用于记录系统中各个模块的操作日志。
1. 创建索引
PUT /operation_log2. 维护mapping结构
PUT /operation_log/_mapping
{
"properties": {
"ip": {
"type": "keyword"
},
"trace_id": {
"type": "keyword"
},
"operation_time": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
},
"module": {
"type": "keyword"
},
"action_code": {
"type": "keyword"
},
"location": {
"type": "text",
"analyzer": "ik_max_word",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"object_id": {
"type": "keyword"
},
"object_name": {
"type": "text",
"analyzer": "ik_max_word",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"operator_id": {
"type": "keyword"
},
"operator_name": {
"type": "keyword"
},
"operator_dept_id": {
"type": "keyword"
},
"operator_dept_name": {
"type": "text",
"analyzer": "ik_max_word",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"changes": {
"type": "nested",
"properties": {
"field_name": {
"type": "keyword"
},
"old_value": {
"type": "keyword"
},
"new_value": {
"type": "keyword"
}
}
}
}
}3. 新建文档
下面一个个文档逐个的新增：
POST /operation_log/_doc
{
"ip": "10.1.11.1",
"trace_id": "670021ff9a2dc6b7",
"operation_time": "2022-05-02 09:31:18",
"module": "企业组织",
"action_code": "UPDATE",
"location": "企业组织->员工管理->身份管理",
"object_id": "xxxxx-1",
"object_name": "成德善",
"operator_id": "operator_id-1",
"operator_name": "张三",
"operator_dept_id": "operator_dept_id-1",
"operator_dept_name": "研发中心-后端一部",
"changes": [
{
"field_name": "手机号码",
"old_value": "13055660000",
"new_value": "13055770001"
},
{
"field_name": "姓名",
"old_value": "成德善",
"new_value": "成秀妍"
}
]
}
// 同样的调用方式，再插入下面6个文档
// data-2
{
"ip": "22.1.11.0",
"trace_id": "990821e89a2dc653",
"operation_time": "2022-09-05 11:31:10",
"module": "资源中心",
"action_code": "UPDATE",
"location": "资源中心->文件管理->文件权限",
"object_id": "fffff-1",
"object_name": "《2022员工绩效打分细则》",
"operator_id": "operator_id-2",
"operator_name": "李四",
"operator_dept_id": "operator_dept_id-2",
"operator_dept_name": "人力资源部",
"changes": [
{
"field_name": "查看权限",
"old_value": "仅李四可查看",
"new_value": "全员可查看"
},
{
"field_name": "编辑权限",
"old_value": "仅李四可查看",
"new_value": "人力资源部可查看"
}
]
}
// data-3
{
"ip": "22.1.11.0",
"trace_id": "780821e89b2dc653",
"operation_time": "2022-10-02 12:31:10",
"module": "资源中心",
"action_code": "DELETE",
"location": "资源中心->文件管理",
"object_id": "fffff-1",
"object_name": "《2022员工绩效打分细则》",
"operator_id": "operator_id-3",
"operator_name": "王五",
"operator_dept_id": "operator_dept_id-2",
"operator_dept_name": "人力资源部",
"changes": []
}
// data-4
{
"ip": "10.1.11.1",
"trace_id": "670021e89a2dc7b6",
"operation_time": "2022-05-03 09:35:10",
"module": "企业组织",
"action_code": "ADD",
"location": "企业组织->员工管理->身份管理",
"object_id": "xxxxx-2",
"object_name": "成宝拉",
"operator_id": "operator_id-1",
"operator_name": "张三",
"operator_dept_id": "operator_dept_id-1",
"operator_dept_name": "研发中心-后端一部",
"changes": [
{
"field_name": "姓名",
"new_value": "成宝拉"
},
{
"field_name": "性别",
"new_value": "女"
},
{
"field_name": "手机号码",
"new_value": "13055770002"
},
{
"field_name": "邮箱",
"new_value": "baola@qq.com"
}
]
}
// data-5
{
"ip": "10.1.11.5",
"trace_id": "670021e89a2dc655",
"operation_time": "2022-05-05 10:35:12",
"module": "企业组织",
"action_code": "DELETE",
"location": "企业组织->员工管理->身份管理",
"object_id": "xxxxx-1",
"object_name": "成德善",
"operator_id": "operator_id-2",
"operator_name": "李四",
"operator_dept_id": "operator_dept_id-2",
"operator_dept_name": "人力资源部",
"changes": []
}
// data-6
{
"ip": "10.0.0.0",
"trace_id": "670021ff9a28ei6",
"operation_time": "2022-10-02 09:31:00",
"module": "资源中心",
"action_code": "DELETE",
"location": "资源中心->文件管理",
"object_id": "fffff-a",
"object_name": "《有空字符串的文档》",
"operator_id": "operator_id-a",
"operator_dept_id": "",
"operator_dept_name": "",
"operator_name": "路人A",
"changes": []
}
// data-7
{
"ip": "10.0.0.0",
"trace_id": "670021ff9a28768",
"operation_time": "2022-10-02 09:32:00",
"module": "资源中心",
"action_code": "DELETE",
"location": "资源中心->文件管理",
"object_id": "fffff-b",
"object_name": "《有NULL的文档》",
"operator_id": "operator_id-b",
"operator_name": "路人B",
"changes": []
}也可被替换为下列的 bulk 方式批量新增，具体 bulk 的语法后续后文会讲：
POST /operation_log/_bulk
{"create":{"_index":"operation_log"}}
{"ip":"10.1.11.1","trace_id":"670021ff9a2dc6b7","operation_time":"2022-05-02 09:31:18","module":"企业组织","action_code":"UPDATE","location":"企业组织->员工管理->身份管理","object_id":"xxxxx-1","object_name":"成德善","operator_id":"operator_id-1","operator_name":"张三","operator_dept_id":"operator_dept_id-1","operator_dept_name":"研发中心-后端一部","changes":[{"field_name":"手机号码","old_value":"13055660000","new_value":"13055770001"},{"field_name":"姓名","old_value":"成德善","new_value":"成秀妍"}]}
{"create":{"_index":"operation_log"}}
{"ip":"22.1.11.0","trace_id":"990821e89a2dc653","operation_time":"2022-09-05 11:31:10","module":"资源中心","action_code":"UPDATE","location":"资源中心->文件管理->文件权限","object_id":"fffff-1","object_name":"《2022员工绩效打分细则》","operator_id":"operator_id-2","operator_name":"李四","operator_dept_id":"operator_dept_id-2","operator_dept_name":"人力资源部","changes":[{"field_name":"查看权限","old_value":"仅李四可查看","new_value":"全员可查看"},{"field_name":"编辑权限","old_value":"仅李四可查看","new_value":"人力资源部可查看"}]}
{"create":{"_index":"operation_log"}}
{"ip":"22.1.11.0","trace_id":"780821e89b2dc653","operation_time":"2022-10-02 12:31:10","module":"资源中心","action_code":"DELETE","location":"资源中心->文件管理","object_id":"fffff-1","object_name":"《2022员工绩效打分细则》","operator_id":"operator_id-3","operator_name":"王五","operator_dept_id":"operator_dept_id-2","operator_dept_name":"人力资源部","changes":[]}
{"create":{"_index":"operation_log"}}
{"ip":"10.1.11.1","trace_id":"670021e89a2dc7b6","operation_time":"2022-05-03 09:35:10","module":"企业组织","action_code":"ADD","location":"企业组织->员工管理->身份管理","object_id":"xxxxx-2","object_name":"成宝拉","operator_id":"operator_id-1","operator_name":"张三","operator_dept_id":"operator_dept_id-1","operator_dept_name":"研发中心-后端一部","changes":[{"field_name":"姓名","new_value":"成宝拉"},{"field_name":"性别","new_value":"女"},{"field_name":"手机号码","new_value":"13055770002"},{"field_name":"邮箱","new_value":"baola@qq.com"}]}
{"create":{"_index":"operation_log"}}
{"ip":"10.1.11.5","trace_id":"670021e89a2dc655","operation_time":"2022-05-05 10:35:12","module":"企业组织","action_code":"DELETE","location":"企业组织->员工管理->身份管理","object_id":"xxxxx-1","object_name":"成德善","operator_id":"operator_id-2","operator_name":"李四","operator_dept_id":"operator_dept_id-2","operator_dept_name":"人力资源部","changes":[]}
{"create":{"_index":"operation_log"}}
{"ip":"10.0.0.0","trace_id":"670021ff9a28ei6","operation_time":"2022-10-02 09:31:00","module":"资源中心","action_code":"DELETE","location":"资源中心->文件管理","object_id":"fffff-a","object_name":"《有空字符串的文档》","operator_id":"operator_id-a","operator_dept_id":"","operator_dept_name":"","operator_name":"路人A","changes":[]}
{"create":{"_index":"operation_log"}}
{"ip":"10.0.0.0","trace_id":"670021ff9a28768","operation_time":"2022-10-02 09:32:00","module":"资源中心","action_code":"DELETE","location":"资源中心->文件管理","object_id":"fffff-b","object_name":"《有NULL的文档》","operator_id":"operator_id-b","operator_name":"路人B","changes":[]}
1.2. spring 项目准备
1. pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>引入了 spring-boot-starter-data-elasticsearch，我们 spring-parent 版本是 2.7.4 的，即这里对应的 starter 版本也是 2.7.4。对应 spring-data-elasticsearch 版本是 4.4.3。
spring data 官网 里有推荐 spring-data-elasticsearch 版本和 elasticsearch 版本的对应关系，建议按照推荐同步版本，本例中 elasticsearch 版本就是 7.17.6。
然后下文中 spring 的代码，最好的教材还是去看 spring data 官网 。
2. application
spring:
elasticsearch:
uris: http://localhost:9200
jackson:
default-property-inclusion: non_null3. EO
索引对应的类需要加上 @Document，字段需要加上 @Field。
OperationLog.java
@Data
@Document(indexName = "operation_log")
public class OperationLog {
@Id
private String id;
@Field(type = FieldType.Keyword)
private String ip;
@Field(value = "trace_id", type = FieldType.Keyword)
private String traceId;
// format={} 不能少
@Field(value = "operation_time", type = FieldType.Date, format = {}, pattern = "yyyy-MM-dd HH:mm:ss")
@JsonFormat(pattern = "yyyy.MM.dd HH:mm:ss", timezone = "GMT+8")
private LocalDateTime operationTime;
@Field(type = FieldType.Keyword)
private String module;
@Field(value = "action_code", type = FieldType.Keyword)
private String actionCode;
@Field(type = FieldType.Text, analyzer = "ik_max_word")
private String location;
@Field(value = "object_id", type = FieldType.Keyword)
private String objectId;
@Field(value = "object_name", type = FieldType.Text, analyzer = "ik_max_word")
private String objectName;
@Field(value = "operator_id", type = FieldType.Keyword)
private String operatorId;
@Field(value = "operator_name", type = FieldType.Keyword)
private String operatorName;
@Field(value = "operator_dept_id", type = FieldType.Keyword)
private String operatorDeptId;
@Field(value = "operator_dept_name", type = FieldType.Text, analyzer = "ik_max_word")
private String operatorDeptName;
@Field(type = FieldType.Nested)
private List<OperationLogChange> changes;
}OperationLogChange.java
@Data
public class OperationLogChange {
@Field(value = "field_name", type = FieldType.Keyword)
private String fieldName;
@Field(value = "old_value", type = FieldType.Keyword)
private String oldValue;
@Field(value = "new_value", type = FieldType.Keyword)
private String newValue;
}2. 查询
我个人不太喜欢 通过继承 ElasticsearchRepository 来实现 Dao层方法，主要是使用局限性太大，不灵活。官方文档也不太推荐这种，而是比较推崇调用 ElasticsearchRestTemplate 方法。
在官方查询的章节中，有介绍过3种方法：
- CriteriaQuery：标准的查询方式，简单的查询还行，但针对一些复杂的查询就有些捉襟见肘了。
- NativeSearchQuery：原生的查询方式，基本上和 Query DSL 里面的语法逻辑很相似，所以不担心搞不定复杂的查询。
- StringQuery：直接支持执行 Query DSL 字符串。
我个人是推荐 NativeSearchQuery，如果哪天真的面对搞不定的查询，可以偶尔尝试一下 StringQuery。所以，下文中所有 spring 的例子，都是基于 NativeSearchQuery的。
2.1. match_all
1. DSL
GET /operation_log/_search
{
"query": {
"match_all": {}
}
}2. spring
@AllArgsConstructor
@RestController
@RequestMapping("/dql")
public class DqlController {
private final ElasticsearchRestTemplate esRestTemplate;
@GetMapping("")
public List<OperationLog> findAll() {
Query query = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchAllQuery())
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());
}
}2.2. match(term)
1. DSL
GET /operation_log/_search
{
"query": {
"match": {
"module": "资源中心"
}
}
}2. spring
Query query =new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchQuery("module", "资源中心"))
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());2.3. nested、object
嵌套查询，包含nested、object两种类型。es字段默认类型是object，但数据其实是扁平化存储的。常见的用法是nested，基于父子嵌套文档。可了解nested和object的区别，下面例子按照nested来。
1. DSL
GET operation_log/_search
{
"query": {
"nested": {
"path": "changes",
"query": {
"term": {
"changes.field_name": "姓名"
}
}
}
}
}2. spring
Query query = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.nestedQuery("changes",
QueryBuilders.termQuery("changes.field_name", "姓名"),
ScoreMode.None))
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());2.4. bool(and) - 1
1. DSL
GET operation_log/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"action_code": "UPDATE"
}
},
{
"nested": {
"path": "changes",
"query": {
"term": {
"changes.field_name": "姓名"
}
}
}
}
]
}
}
}2. spring
Query query = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.boolQuery()
.must(QueryBuilders.termQuery("action_code", "UPDATE"))
.must(QueryBuilders.nestedQuery("changes",
QueryBuilders.termQuery("changes.field_name", "姓名"), ScoreMode.None)))
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());2.5. bool(and) - 2
1. DSL
GET operation_log/_search
{
"query": {
"bool": {
"must_not": [
{
"term": {
"action_code": "UPDATE"
}
}
],
"must": [
{
"nested": {
"path": "changes",
"query": {
"term": {
"changes.field_name": "姓名"
}
}
}
}
]
}
}
}2. spring
Query query = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.boolQuery()
.mustNot(QueryBuilders.termQuery("action_code", "UPDATE"))
.must(QueryBuilders.nestedQuery("changes",
QueryBuilders.termQuery("changes.field_name", "姓名"), ScoreMode.None)))
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());2.6. bool(or)、exist
1. DSL
GET /operation_log/_search
{
"query": {
"bool": {
"should": [
{
"term": {
"operator_dept_name.keyword": ""
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "operator_dept_name"
}
}
]
}
}
]
}
}
}2. spring
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()
.should(QueryBuilders.termQuery("operator_dept_name.keyword", ""))
.should(QueryBuilders.boolQuery()
.mustNot(QueryBuilders.existsQuery("operator_dept_id")));
Query query = new NativeSearchQuery(boolQueryBuilder);
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());2.7. _source、sort
如果只想查询索引中某几个字段，就可以用到 _source，其中包含两个属性：
includes：查询结果包含某些字段。excludes：查询结果屏蔽某些字段。
当二者同时出现时，优先级上：excludes > includes。
当只有_source中 includes 时，可以忽略 includes 不写，直接 "_source":[field,...]
sort 可用于排序。
1. DSL
GET /operation_log/_search
{
"query": {
"match": {
"location": "文件"
}
},
"_source": {
"includes": [
"module",
"location",
"operator_name",
"operation_time",
"changes.field_name"
],
"excludes": [
"module"
]
},"sort": [
{
"operation_time": {
"order": "asc"
}
}
]
}
// 也等同于
{
"query": {
"match": {
"location": "文件"
}
},
"_source": [
"location",
"operator_name",
"operation_time",
"changes.field_name"
],
"sort": [
{
"operation_time": {
"order": "asc"
}
}
]
}2. spring
SourceFilter sourceFilter = new FetchSourceFilterBuilder()
.withIncludes("module", "location", "operator_name", "operation_time", "changes.field_name")
.withExcludes("module")
.build();
Query query = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchQuery("location", "文件"))
.withSourceFilter(sourceFilter)
.withSort( Sort.by(Sort.Direction.ASC, "operation_time"))
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());2.8. highlight
这里主要介绍一下highlight里的标签
pre_tags、post_tags:这两个标签定义了分割出的结果以什么tag包围起来，和我们前端的<></>效果差不多fields：定义要高亮搜索的属性，name代表名称要高亮，keyWords代表关键词要高亮1. DSL
GET /operation_log/_search { "query": { "match": { "location": "文件" } }, "highlight": { "fields": { "location": {} }, "pre_tags": "<span style='color:red'>", "post_tags": "</span>" } }2. spring
String matchField = "location";
HighlightBuilder highlightBuilder = new HighlightBuilder()
.field(matchField)
.preTags("<span style='color:red'>")
.postTags("</span>");
Query query = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchQuery(matchField, "文件"))
.withHighlightBuilder(highlightBuilder)
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(hit -> {
OperationLog operationLog = hit.getContent();
operationLog.setLocation(hit.getHighlightField(matchField).get(0));
return operationLog;
})
.collect(Collectors.toList());2.9. pageable
1. DSL
GET /operation_log/_search
{
"query": {
"match_all": {}
},
"from": 0,
"size": 5,
"sort": [
{
"operation_time": {
"order": "desc"
}
}
]
}
2. spring
Query query = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchAllQuery())
.withPageable(PageRequest.of(0, 5, Sort.by(Sort.Direction.DESC,"operation_time")))
.build();
return esRestTemplate.search(query, OperationLog.class).stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());引用：
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。