niewj

niewj 查看完整档案

北京编辑山西农业大学  |  计算机应用 编辑大厂  |  Java程序员 编辑 www.niewj.com 编辑
编辑

遇见超乎想象的自己!

个人动态

niewj 回答了问题 · 1月13日

Java:ConcurrentHashMap 的 initTable 方法末尾的 n - (n >>> 2) 是什么意思?

n>>>2就是n无符号右移2位, 相当于最后面的小数点左移了2位, 相当于除以了2的2次方;
n>>>5 n缩小了 2的5次方倍:
就是说:
sc = n - n/2^2

    public static void main(String[] args) {
        int sc = 0;
        int n = 16;

        System.out.println("16>>>2 --> " + (16>>>2));

        sc = n - (n>>>2);
        System.out.println("n=16, sc = n - (n>>>2); sc = " + sc);
    }
    
16>>>2 --> 4
n=16, sc = n - (n>>>2); sc = 12

左移整个数往左边走,右面补0, 补一位, 就扩大2倍, 补n位就扩大2的n次方倍; 右移正相反~

关注 3 回答 2

niewj 发布了文章 · 2020-12-17

Spring中使用JSR303请求约束判空

1. 适用场景

有时候我们在表单里提交一系列参数, 到后台封装成一个对象, 要对对象的属性做各种字段值的约束; 这时候, 当然可以if-else一个一个的判断, 有更简洁的做法, 就是使用 JSR303+spring的validation:

2. 使用方法步骤(分3步)

    1. 实体类加字段约束注解
    1. Controller类中@Valid标注启用(@Validated也兼容@Valid)
    1. BindingResult获取报错信息

    2.1 实体类加字段约束注解

如我们要收集前端表单的字段数据到Person实体中:

对需要约束的字段进行注解标注;

示例:

2.1.1 Person.java

package com.niewj.demo.model;

import lombok.Data;
import org.hibernate.validator.constraints.Length;
import org.hibernate.validator.constraints.URL;

import javax.validation.constraints.*;
import java.util.List;

@Data
public class Person {
    @Length(min = 4, max = 10)
    @NotNull(message = "name不能为空")
    private String name;

    @Min(0)
    @Max(40)
    @NotNull(message = "age不能为空")
    private Integer age;

    @NotBlank
    @NotBlank
    @URL(message = "logo必须是URL格式")
    private String logo;

    @NotEmpty(message = "hobbies不能为空")
    private List<String> hobbies;

    @Email
    @NotNull(message = "email不能为空!")
    private String mail;
}

@NotNull 约束字段不可为空;

@NotEmpty 约束集合/map等不能为空不能为0元素

@Email 约束是 email格式

@URL 约束必须是url的格式

message属性可以修改默认错误说明

2.2 Controller类中@Valid标注启用(@Validated兼容@Valid)

2.2.1 HelloController.java

package com.niewj.demo.controller;

import com.google.gson.Gson;
import com.niewj.demo.common.Result;
import com.niewj.demo.model.Person;
import com.niewj.demo.service.TestService1;
import org.springframework.stereotype.Controller;
import org.springframework.validation.BindingResult;
import org.springframework.web.bind.annotation.*;

import javax.annotation.Resource;
import javax.validation.Valid;
import java.util.HashMap;
import java.util.Map;

/**
 * test
 *
 * @author niewj
 * @since 2020/11/16 15:22
 */
@Controller
public class HelloController {

    /**
     * @param person
     * @param bindingResult
     * @return
     */
    @PostMapping("/test")
    @ResponseBody
    public Result doSth(@Valid @RequestBody Person person, BindingResult bindingResult) {
        Map<String, String> map = new HashMap<>();
        if (bindingResult.hasErrors()) {
            bindingResult.getFieldErrors().stream().forEach(fe-> map.put(fe.getField(), fe.getDefaultMessage()));
            System.out.println(bindingResult.getFieldError().getDefaultMessage());
            return Result.withErrorParamData(map);
        }

        return Result.withSuccessData(person);
    }
}

@RequestBody可以将body中的请求流信息(Request.getInputStream)通过HttpMessageConverter自动转换为目标java类型:

  • 如果前端 Content-Type为application/json, 就使用 JSON消息转换器 帮你转为JSON对象;
  • 如果前端 Content-Type为application/xml, 就使用 XML消息转换器 帮你转为xml;
  • 如果前端 Content-Type为text/plain, 就是用 String消息转换器 帮你转成 String; (只有类型为String时才可以转换)

2.2.2 Result.java:

package com.niewj.demo.common;

import lombok.Data;

import java.io.Serializable;
import java.util.Map;

/**
 * 返回结果模板封装
 *
 * @author niewj
 * @since 2020/12/17 18:05
 */
@Data
public class Result<T> implements Serializable {
    private int code;
    private String msg;
    private T data;

    public Result(int code, String msg, T data) {
        this(code, msg);
        this.data = data;
    }

    public Result(int code, String msg) {
        this.msg = msg;
        this.code = code;
    }

    public static <T> Result<T> withData(ResponseEnum responseCode, T data) {
        Result<T> re = new Result(responseCode.getCode(), responseCode.getMsg());
        re.data = data;
        return re;
    }

    public static Result<String> withSuccessNoData() {
        Result re = new Result(ResponseEnum.SUCCESS.getCode(), ResponseEnum.SUCCESS.getMsg());
        re.data = "";
        return re;
    }

    public static <T> Result<T> withSuccessData(T data) {
        Result re = new Result(ResponseEnum.SUCCESS.getCode(), ResponseEnum.SUCCESS.getMsg());
        re.data = data;
        return re;
    }

    public static Result<Map<String, String>> withErrorParamData(Map<String, String> data) {
        Result re = new Result(ResponseEnum.BAD_REQUEST.getCode(), ResponseEnum.BAD_REQUEST.getMsg());
        re.data = data;
        return re;
    }
}

2.2.3 ResponseEnum.java

package com.niewj.demo.common;

/**
 * 通用响应码
 */
public enum ResponseEnum {

    SUCCESS(200, "成功"),
    BAD_REQUEST(400, "请求参数不合规");

    /**
     * 错误码
     */
    private Integer code;
    /**
     * 错误信息
     */
    private String msg;

    ResponseEnum(Integer code, String msg) {
        this.code = code;
        this.msg = msg;
    }

    public String getMsg() {
        return msg;
    }

    public Integer getCode() {
        return code;
    }
}

2.3 BindingResult获取报错信息

  1. BindingResult用来收集约束字段的错误信息, 可通过 bindingResult.hasErrors()true来过滤错误信息;
  2. bindingResult.getFieldErrors() 会返回 List<FieldError>;
  3. FieldError 得到字段名: getField(); 得到错误信息: getDefaultMessage()

3. 调用测试:

http://localhost:8888/test

3.1 请求用例1: 字段为空

header: Content-Type: application/json

{
    "name": "1"
}

响应:

{
    "code": 400,
    "msg": "请求参数不合规",
    "data": {
        "mail": "email不能为空!",
        "hobbies": "hobbies不能为空",
        "name": "长度需要在4和10之间",
        "logo": "不能为空",
        "age": "age不能为空"
    }
}

3.2 请求用例2: list无元素/email/url格式不对

header: Content-Type: application/json

{
    "name": "1234",
    "hobbies": [],
    "mail": "niewj",
    "logo": "niewj.com"
}

响应:

{
    "code": 400,
    "msg": "请求参数不合规",
    "data": {
        "mail": "不是一个合法的电子邮件地址",
        "hobbies": "hobbies不能为空",
        "logo": "logo必须是URL格式",
        "age": "age不能为空"
    }
}

3.3 请求用例3: 完整字段信息

header: Content-Type: application/json

{
    "name": "1234",
    "hobbies": ["running"],
    "mail": "hi@niewj.com",
    "logo": "http://niewj.com",
    "age": 40
}

响应:

{
    "code": 200,
    "msg": "成功",
    "data": {
        "name": "1234",
        "age": 40,
        "logo": "http://niewj.com",
        "hobbies": [
            "running"
        ],
        "mail": "hi@niewj.com"
    }
}
查看原文

赞 0 收藏 0 评论 0

niewj 发布了文章 · 2020-11-26

elasticsearch笔记-006-文档API-CRUD-单文档更新操作

[toc]

1. 单文档 Update API

update API 根据提供的script更新文档; 从索引中获取document, 执行脚本, 并对新的结果进行索引.

如何确保 get 和 reindex 期间没有发生更新? --> 使用版本控制;

这种操作仍然意味着对文档进行完整的重新索引, 它只是移除了一些网络往返损耗,并减少了get和索引之间版本冲突的机会.

1. script更新: 数值计算

示例: 新建一个document到 test 索引, 然后修改 counter=counter+4; 然后为tags新增一个 blue的颜色:

#1. 新建一个document到 test 索引
PUT test/_doc/1
{
    "counter" : 1,
    "tags" : ["red"]
}

将 "counter"的值加上给定的参数:

POST test/_update/1
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    }
}

查看当前: GET test/_source/1

{
    "counter" : 5,
    "tags" : [
      "red"
    ]
}

同理: counter=counter*2:

POST test/_update/1
{
  "script": {
    "source": "ctx._source.counter *= params.val",
    "params": {
      "val": 2
    }
  }
}

2. script更新: 数组(集合)增加/删除

2.1 增加一个 yellow:

POST test/_update/1
{
  "script": {
    "source": "ctx._source.tags.add(params.clr)",
    "params": {"clr": "yellow"}
  }
}

output:

{
    "counter" : 10,
    "tags" : [
      "red",
      "yellow"
    ]
}

2.2 删除(注意是索引)

实际是ArrayList的方法!

POST test/_update/1
{
  "script": {
    "source": "ctx._source.tags.remove(params.val)",
    "params": {
      "val": 0
    }
  }
}
{
    "counter" : 10,
    "tags" : [ "yellow" ]
}

2.3 clear()清空

同理, 可以调用 ArrayList的其他方法: 比如 :

POST test/_update/1
{
  "script": {
    "source": "ctx._source.tags.clear()"
  }
}

2.4 如何删除匹配的内容

如果现在 tags = ["yello", "red", "blue"], 要删除 "red", 根据内容而非索引:

POST test/_update/1
{
  "script": {
    "source": "if (ctx._source.tags.contains(params.val)) {ctx._source.tags.remove(ctx._source.tags.indexOf(params.val));}",
    "params": {
      "val": "red"
    }
  }
}

GET test/_source/1 output:

{
    "counter" : 10,
    "tags" : [
      "yellow",
      "blue"
    ]
}

注意: script.source 里的";"可以不要的;

2.5 script更新: ctx的其他内置变量

除了 _source 之外,ctx映射还提供了以下变量: _index_type_id_version_routing_now(当前时间戳)。

2.6 script更新: 增加/删除字段entry

我们给 test 索引中的 文档 1 新增一个字段: "addr": "beijing":

# update 增加字段entry
POST test/_update/1
{
  "script": {
    "source": "ctx._source.addr='beijing'"
    }
}
# 再增加一条
POST test/_update/1
{
  "script": {
    "source": "ctx._source.location='yizhuang'"
    }
}

GET test/_source/1

{
  "counter" : 10,
  "tags" : [
    "yellow"
  ],
  "addr" : "beijing",
  "location" : "yizhuang"
}

删除:

# update 删除字段
POST test/_update/1
{
  "script": {
    "source": "ctx._source.remove('location')"
  }
}

GET test/_source/1 :

{
  "counter" : 10,
  "tags" : [
    "yellow"
  ],
  "addr" : "beijing"
}

3. doc新增字段:

除了使用 script 外, 也可以使用 doc 来新增字段; 如果二者皆有, 会报错!

POST test/_update/1
{
    "doc" : {
        "loc" : "yz"
    }
}
{
  "counter" : 10,
  "tags" : [
    "yellow"
  ],
  "addr" : "beijing",
  "loc" : "yz"
}

如果指定了doc,那么它的值将与现有的_source合并; 如果 doc执行了多次, 数据并没有变化, \_version和 \_seq\_no都不会更新! "result" : "noop"

4. upsert 更新+插入

4.1 script+upsert方式

如果文档还不存在,那么upsert元素的内容将作为新文档插入。如果该文档存在,则执行脚本:

DELETE test/_doc/2 我们先执行此操作删除id=2的document; 然后执行下面 upsert 操作:

POST test/_update/2
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    },
    "upsert" : {
        "counter" : 2,
        "name": "ifnotexists"
    }
}

因为此刻没有 id=2的文档, 所以默认+=操作时是不会有的; 这样的话, upsert中的内容就会作为文档的内容保存;

这是 重复上面的 upsert 操作一次!! 再查结果: GET test/_source/2

{
  "counter" : 6,
  "name" : "ifnotexists"
}

可以看到, counter就是 2+4了;

4.2 doc方式的upsert

doc方式也可以实现upsert: 如下: 如果单纯是在上面有id=2的document的基础上执行下面:

POST test/_update/2
{
  "doc": {
    "gender": "male"
  },
  "doc_as_upsert": true
}

只是增加了一个 gender的字段!

{
  "counter" : 6,
  "name" : "ifnotexists",
  "gender" : "male"
}

现在删掉document: DELETE test/_doc/2 , 就会表现为 upsert的功能了, 此时 GET test/_source/2:

{"gender" : "male"}

5. update的其他参数

timeout/wait_for_active_shards/version, 以及 if_seq_no + if_primary_term

5.1 if_seq_no+if_primary_term

POST test/_update/2?if_seq_no=31&if_primary_term=1
{
  "script": {
    "source": "ctx._source.counter=params.val",
    "params": {
      "val": 3
    }
  }
}

如果 _seq_no_primary_term匹配, 才执行更新, 否则报错!

2. TODO: Update By Query API

查看原文

赞 0 收藏 0 评论 0

niewj 发布了文章 · 2020-11-26

elasticsearch笔记-005-文档API-CRUD-单文档删除操作

[toc]

1. 单文档Delete API

1. 使用id删除document

DELETE /twitter/_doc/1
{
  "_index" : "twitter",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}

2. document 指定版本的删除

索引的每个document 都有版本。在删除文档时,可以指定版本,确保删除的文档实际上已经被删除,且在此期间没有发生更改; 对文档执行的每个写操作(包括删除)都会导致其版本增加;删除后的文档的版本号在短时间内仍然可用,以便控制并发操作, 被删除文档的版本保持可用的时间长度由 index.gc_deletes这个设置(setting)决定: 默认是 60 seconds;

3. routing的索引的删除

如果建索引时, 使用了 routing, 那么 删除索引时, 也需要指定 routing; DELETE /twitter/_doc/1?routing=kimchy, 没有正确路由的情况下发出删除将导致文档不被删除。当按需要设置_routing映射并且没有指定路由值时,delete API将抛出一个RoutingMissingException并拒绝请求

4. 删除时检查活跃分片数量

发出删除操作时, 可以设置wait_for_active_shards参数,要求在开始处理删除请求之前,检查活动的切分副本的最小数量;

5. 删除时的 timeout

在执行删除操作时,分配给执行删除操作的主shard可能不可用! 原因可能是主shard当前正在从存储中恢复或正在进行重新定位; 默认情况下,delete操作将等主shard可用的时长为1分钟,然后失败并响应一个错误。这个 timeout参数可以在删除时指定: DELETE /twitter/_doc/1?timeout=5m

2. TODO: Delete By Query API

查看原文

赞 0 收藏 0 评论 0

niewj 发布了文章 · 2020-11-26

elasticsearch笔记-004-文档API-CRUD-单文档查操作

[toc]

单文档 get API

1. Get基本查询

GET weibo/_doc/1

output:

{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "user" : "niewj",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
  }
}

2. head查询文档是否存在

GET weibo/_doc/2 结果:

{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "2",
  "found" : false
}

可以看到id=2的文档不存在:

HEAD weibo/_doc/2 结果: 404 - Not Found

文档id=1是存在的: GET weibo/_doc/1 结果:

{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "user" : "niewj",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
  }
}

HEAD查询: HEAD weibo/_doc/1, 结果: 200 - OK

HEAD: 查不到: 404; 查得到: 200;

3. get查询索引的实时性

如果文档已经更新,但还没有刷新,那么get API将发出一个刷新调用,使文档可见; 默认情况下,get API就是实时的,不受索引刷新速率的影响.(当然也可以禁用realtime GET,可将realtime参数设为false。)

4. \ _source的禁用

默认情况下,get操作返回source内容,除非禁用_source字段:

GET weibo/_doc/1:

{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "user" : "niewj",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
  }
}

禁用: GET weibo/_doc/1?_source=false:

{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true
}

5. \_source字段的过滤(excludes/includes)

如果您只要source中的一个或两个字段,可以使用_source_include_source_exclude参数来包含或过滤:

  • _source_includes 只想查询 \_source节点中的 user 和 message 字段:
GET weibo/_doc/1?_source_includes=user,message
{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "message" : "trying out Elasticsearch",
    "user" : "niewj"
  }
}
  • _source 上述的包含, 可以简写为: _source
GET weibo/_doc/1?_source=user,message
  • source_excludes 想排除 \_source节点中的 message 字段, 其余的全展示:
GET weibo/_doc/1?_source_excludes=message
{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "post_date" : "2009-11-15T14:12:12",
    "user" : "niewj"
  }
}

6. 直接使用\_source获取数据

使用/{index}/\_source/{id} 只获取文档的\_source字段,而不包含任何其他附加内容。例如:

GET weibo/_source/1
{
  "user" : "niewj",
  "post_date" : "2009-11-15T14:12:12",
  "message" : "trying out Elasticsearch"
}

可以看到, 只有纯数据返回; 也可以使用上面的 _source的过滤字段:

GET weibo/_source/1?_source=user
{
  "user" : "niewj"
}

7. 索引时的stored_fields参数

默认情况下,字段值被索引以使其可搜索,但不存储它们(的原始值); 这意味着可以查到字段,但不能检索其原始字段值。举例:

# 设置mapping中counter字段只索引不存储;tags字段索引且存储;
PUT twitter
{
   "mappings": {
       "properties": {
          "counter": {
             "type": "integer",
             "store": false
          },
          "tags": {
             "type": "keyword",
             "store": true
          }
       }
   }
}

索引一条记录:

# 索引一条文档, id=1
PUT twitter/_doc/1
{
    "counter" : 1,
    "tags" : ["red"]
}

查询 GET twitter/_doc/1

{
  "_index" : "twitter",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "counter" : 1,
    "tags" : [
      "red"
    ]
  }
}

都可以查到; 使用 stored_fields 参数查询 : GET twitter/_doc/1?stored_fields=tags,counter:

{
  "_index" : "twitter",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "fields" : {
    "tags" : [
      "red"
    ]
  }
}

可以看到, 没了 counter 字段; 因为mapping里设置了 store: false

8. 索引查询性能相关参数之: refresh

refresh设为true,以在get之前刷新相关分片,使其可搜索。在将其设置为true之前,应该仔细考虑并确认: 因为这可能导致系统负载过重(降低索引速度)。

查看原文

赞 0 收藏 0 评论 0

niewj 发布了文章 · 2020-11-26

elasticsearch笔记-003-文档API-CRUD-单文档索引操作

[toc]

单文档Index API

1. 索引一个文档

PUT /weibo/_doc/1
{
  "user": "niewj",
  "post_date": "2020-11-20T17:00:00",
  "message": "trying out Elasticsearch"
}

output:

{
  "_index" : "weibo",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}
  1. 索引如果不存在, 会自动创建一个索引;
  2. 如果不存在dynamic mapping, index时会创建一个;
  3. 如果需要, 新的字段和对象会自动加入到mapping 定义中;

2. 可选项op_type或_create(put_if_absent)

put默认是覆盖的; 但是如果使用一个op_type选项, 可以改变这个逻辑:

PUT /customer/_doc/5?op_type=create
{
  "name": "niewj5"
}

如果没有id=5的doc了, 索引成功; 如果已有, 就会失败;

另一种写法:

PUT /customer/_create/5
{
  "name": "niewj5"
}

output:

{
  "error": {
    "root_cause": [
      {
        "type": "version_conflict_engine_exception",
        "reason": "[5]: version conflict, document already exists (current version [7])",
        "index_uuid": "NjlJEQ5nS5e-PZQd_n2_Rw",
        "shard": "0",
        "index": "customer"
      }
    ],
    "type": "version_conflict_engine_exception",
    "reason": "[5]: version conflict, document already exists (current version [7])",
    "index_uuid": "NjlJEQ5nS5e-PZQd_n2_Rw",
    "shard": "0",
    "index": "customer"
  },
  "status": 409
}

3. 不指定id时的post和put

# 不指定索引id时, 自动生成
POST twitter/_doc/
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

不指定id的post, 每次都生成新的随机id, output: "FPOw9HUB7InhghdESz3n"

{
  "_index" : "twitter",
  "_type" : "_doc",
  "_id" : "FPOw9HUB7InhghdESz3n",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 3,
  "_primary_term" : 1
}

但是不指定id的put会报错:

# 下面的操作会报错405: 不带ID的put=报错405
PUT twitter/_doc/
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

output:

{
  "error": "Incorrect HTTP method for uri [/twitter/_doc/?pretty] and method [PUT], allowed: [POST]",
  "status": 405
}

4. 索引的乐观并发控制

if_seq_noif_primary_term索引操作可以是通过有条件才触发的, 通过附带这两个参数的赋值;比如

PUT products/_doc/1567?if_seq_no=362&if_primary_term=2

这个操作就会在版本序号对应的条件下才实施索引动作的执行;

参见: Optimistic concurrency control

5. 索引分片的路由

默认, 索引分片的路由, 是通过文档 id 的 hash 来确定文档路由到哪个分片的; 但是可以通过制定参数 routing=xxx来显式控制; 例如: 下面的文档, 会按照提供的参数:“kimchy”来路由分片

POST twitter/_doc?routing=kimchy
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

注意: 如果在mapping中定义了 _routing 且声明为 required , 此时如果索引时没有制定 routing值, 将会发生失败!

索引操作将会被定位到路由到的主分片, 并在主分片的节点上执行; (如果需要), 主分片执行完后, 将会分发到相应的副本(replicas).

6. 关于索引写操作的参数

wait_for_active_shards=1 默认索引的写操作只等待主分片, 只要它处于活动状态后就可写; 这个是可以修改的: 可以指定一个具体的数字N, 这样, 就会等待N个分片处于活跃时才进行写索引的操作; 如果没有N个, 就一直等到所需的副本数启动(或一直等到超时).

这个默认值可以通过设置 index.write.wait_for_active_shards 在索引设置中动态覆盖。

7. 索引操作与版本数

internal

只有给定版本与存储文档的版本相同时,才索引文档

externalexternal_gt

只有在给定版本严格高于存储文档的版本(或没有现有文档)时,才索引文档: 给定版本将作为新版本与新文档存储在一起。
查看原文

赞 0 收藏 0 评论 0

niewj 发布了文章 · 2020-11-20

elasticsearch笔记-002-API通用可选参数

[toc]

1. API通用可选参数

1.1. 美观、格式化结果数据

  • ?pretty=true 格式化为json
  • ?format=yaml 结果展示为 yml
  • ?human=false 关闭 1h/1kb等人性化转换

1.2 响应过滤:filter_path

1.2.1 响应过滤用法

原来请求:

GET /bank/_doc/1

原响应:

{
  "_index" : "bank",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 1001,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "account_number" : 1,
    "balance" : 39225,
    "firstname" : "Amber",
    "lastname" : "Duke",
    "age" : 32,
    "gender" : "M",
    "address" : "880 Holmes Lane",
    "employer" : "Pyrami",
    "email" : "amberduke@pyrami.com",
    "city" : "Brogan",
    "state" : "IL"
  }
}

加上响应过滤:采用逗号分隔的过滤器列表

GET /bank/_doc/1?filter_path=_version,_source.firstname,_source.age,_source.balance

加上后的响应:

{
  "_version" : 3,
  "_source" : {
    "balance" : 39225,
    "firstname" : "Amber",
    "age" : 32
  }
}

1.2.2 响应过滤支持通配符 "*"

请求:

# 响应过滤
GET /bank/_doc/1?filter_path=_version,_source.*name,_source.age,_source.*e

响应:

{
  "_version" : 3,
  "_source" : {
    "balance" : 39225,
    "firstname" : "Amber",
    "lastname" : "Duke",
    "age" : 32,
    "state" : "IL"
  }
}

原响应参考上边 1.2.1所列;

1.2.3 响应过滤:排除字段 "-"

请求:

GET /bank/_doc/1?filter_path=-_type,-_source

过滤后响应:

{
  "_index" : "bank",
  "_id" : "1",
  "_version" : 3,
  "_seq_no" : 1001,
  "_primary_term" : 1,
  "found" : true
}

可见, _type字段和_source节点,都被排除掉了;

1.2.4 响应过滤:*-混用时,先排除,再过滤

例1:

GET /bank/_doc/1?filter_path=_version,_source.*name,_source.age,_source.*e,-_source

结果:

{
  "_version" : 3
}

例2:

GET /bank/_doc/1?filter_path=_version,_source.*name,_source.age,_source.*e,-_source.lastname

结果:

{
  "_version" : 3,
  "_source" : {
    "balance" : 39225,
    "firstname" : "Amber",
    "age" : 32,
    "state" : "IL"
  }
}

1.3 时间单位

2d for 2 days

dDays
hHours
mMinutes
sSeconds
msMilliseconds
microsMicroseconds
nanosNanoseconds

 1.4 字节单位

bBytes
kbKilobytes
mbMegabytes
gbGigabytes
tbTerabytes
pbPetabytes

1.5 打印错误堆栈踪迹:error_trace=true

看下面3个输出

GET /library/_search?size=1&filter_path=hits.hits._source
{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "title" : "Book #1",
          "rating" : 200.1
        }
      }
    ]
  }
}

这个是正确时的输出.

我们传个错误参数 size=Y

GET /library/_search?size=Y&filter_path=hits.hits._source

输出:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Failed to parse int parameter [size] with value [Y]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Failed to parse int parameter [size] with value [Y]",
    "caused_by": {
      "type": "number_format_exception",
      "reason": "For input string: \"Y\""
    }
  },
  "status": 400
}

下面再看个完整堆栈信息的方式: error_trace

GET /library/_search?size=Y&filter_path=hits.hits._source&error_trace=true

outpu:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Failed to parse int parameter [size] with value [Y]",
        "stack_trace": "[Failed to parse int parameter [size] with value [Y]]; nested: IllegalArgumentException[Failed to parse int parameter [size] with value [Y]]; nested: NumberFormatException.........."
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Failed to parse int parameter [size] with value [Y]",
    "caused_by": {
      "type": "number_format_exception",
      "reason": "For input string: \"Y\"",
      "stack_trace": "java.lang.NumberFormatException: For input string: \"Y\"\n\tat java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:68)................"
  },
  "status": 400
}

........省去了很多输出信息!

查看原文

赞 0 收藏 0 评论 0

niewj 发布了文章 · 2020-11-18

zookeeper笔记之-001-watch

zookeeper官网文档摘抄


1. ZooKeeper Watches->ZooKeeper中的所有读取操作:

(All of the read operations in ZooKeeper):

getData(), getChildren(), and exists() - have the option of setting a watch as a side effect.

getData(), getChildren(), and exists() - 可以有选择的设置一个watch作为一个副作用

Here is ZooKeeper's definition of a watch: a watch event is one-time trigger, sent to the client that set the watch, which occurs when the data for which the watch was set changes.

以下是ZooKeeper对 watch 的定义:watch事件是一次性触发的,发送给设置watch的客户端,当watch所设置的数据发生变化时,就会发生watch事件。

1.2 在这个watch的定义中,有三个关键点需要考虑:

There are three key points to consider in this definition of a watch:

1. One-time trigger(一次性触发)

One watch event will be sent to the client when the data has changed.

当数据发生变化时,将发送一个watch事件给客户端。

For example, if a client does a getData("/znode1", true) and later the data for /znode1 is changed or deleted, the client will get a watch event for /znode1.

例如,如果客户端执行了getData("/znode1", true),之后/znode1的数据被更改或删除,客户端将获得/znode1的一个监视事件。

If /znode1 changes again, no watch event will be sent unless the client has done another read that sets a new watch.

如果/znode1再次更改,将不会发送watch事件,除非客户端已经完成了另一个读取来设置新的watch。

2. Sent to the client (发送给client):

This implies that an event is on the way to the client, but may not reach the client before the successful return code to the change operation reaches the client that initiated the change.

这意味着一个事件正在到达客户端,但是在变更操作成功返回的代码到达发起变更的客户端之前可能无法到达客户端。

Watches are sent asynchronously to watchers.

watch被异步发送给观察者。

ZooKeeper provides an ordering guarantee: a client will never see a change for which it has set a watch until it first sees the watch event.

ZooKeeper提供了一个有序保证:客户在第一次看到watch事件之前,不会看到它已经设置了watch的变化。

Network delays or other factors may cause different clients to see watches and return codes from updates at different times.

网络延迟或其他因素可能导致不同的客户端在不同的时间查看监视和从更新返回代码。

The key point is that everything seen by the different clients will have a consistent order.

关键的一点是,不同客户看到的所有东西都有一个一致的顺序。

3. The data for which the watch was set (为watch设置的数据):

This refers to the different ways a node can change.

这指的是node可以更改的不同方式。

It helps to think of ZooKeeper as maintaining two lists of watches: data watches and child watches.

我们可以将ZooKeeper看作维护两个watch列表:数据watch和child watch。

getData() and exists() set data watches. getChildren() sets child watches.

getData()和exists()设置数据监视。getChildren()设置child watch。

Alternatively, it may help to think of watches being set according to the kind of data returned.

另一种方法是根据返回的数据类型来设置watch。

getData() and exists() return information about the data of the node, whereas getChildren() returns a list of children.

getData()和 exists()返回关于节点数据的信息,而getChildren()返回子节点列表。

Thus, setData() will trigger data watches for the znode being set (assuming the set is successful).

因此,setData()将触发正在设置的znode的数据监视(假设设置成功)。

A successful create() will trigger a data watch for the znode being created and a child watch for the parent znode.

成功的create()将触发正在创建的znode的数据监视和父znode的子监视。

A successful delete() will trigger both a data watch and a child watch (since there can be no more children) for a znode being deleted as well as a child watch for the parent znode.

成功的delete()将同时触发删除znode的数据监视和子监视(因为不可能有更多的子监视),以及父znode的子监视。

2. watches的管理

Watches are maintained locally at the ZooKeeper server to which the client is connected.

watches 在客户机连接的ZooKeeper服务器上本地维护。

This allows watches to be lightweight to set, maintain, and dispatch.

这使得 watches 在设置、维护和分派方面都是轻量级的。

When a client connects to a new server, the watch will be triggered for any session events.

当客户端连接到新服务器时,watch会被任何会话事件触发

Watches will not be received while disconnected from a server.

当从服务器断开连接时,将不会收到 Watches。

When a client reconnects, any previously registered watches will be reregistered and triggered if needed.

当客户端重新连接时,任何之前注册的 watches 都将重新注册并在需要时触发。

In general this all occurs transparently.

一般来说,这一切都是透明的。

There is one case where a watch may be missed: a watch for the existence of a znode not yet created will be missed if the znode is created and deleted while disconnected.

有一种情况可能会漏掉一个 watch : watch 因为znode还没有创建, 此时断开连接,就会漏掉。

3. Watches的语义

We can set watches with the three calls that read the state of ZooKeeper: exists, getData, and getChildren.

我们可以用读取ZooKeeper状态的三个调用来设置watch:exists、getData和getChildren

The following list details the events that a watch can trigger and the calls that enable them:

下面的列表详细说明了watch可以触发的事件,以及启用它们的调用:

Created event: Enabled with a call to exists.

通过调用exists启用

Deleted event: Enabled with a call to exists, getData, and getChildren.

通过调用exists、getData和getChildren启用。

Changed event: Enabled with a call to exists and getData.

通过调用exists和getData启用

Child event: Enabled with a call to getChildren.

通过调用获取子节点启用

4. 持久、递归的watch: Persistent, Recursive Watches

New in 3.6.0: There is now a variation on the standard watch described above whereby you can set a watch that does not get removed when triggered.

现在有一个变化的标准watch上面描述,你可以设置一个watch,触发时不删除。

Additionally, these watches trigger the event types NodeCreated, NodeDeleted, and NodeDataChanged and, optionally, recursively for all znodes starting at the znode that the watch is registered for.

此外,这些监视会触发NodeCreated、NodeDeleted和NodeDataChanged这些事件类型,而且对于从监视所注册的znode开始的所有znodes,还可以递归地触发这些事件类型

Note that NodeChildrenChanged events are not triggered for persistent recursive watches as it would be redundant.

注意,对于持久递归监视,不会触发NodeChildrenChanged事件,因为它是冗余的。

Persistent watches are set using the method addWatch().

使用addWatch()方法设置持久监视。

The triggering semantics and guarantees (other than one-time triggering) are the same as standard watches.

触发语义和保证(除了一次性触发)与标准watch相同

The only exception regarding events is that recursive persistent watchers never trigger child changed events as they are redundant.

关于事件的唯一例外是,递归持久监视器不会触发子更改事件,因为它们是冗余的

Persistent watches are removed using removeWatches() with watcher type WatcherType.Any.

使用removeWatches()和观察者类型WatcherType.Any来删除持久的监视。

5. 移除watches:(Remove Watches):

We can remove the watches registered on a znode with a call to removeWatches.

我们可以通过调用 removeWatches 来删除在znode上注册的watch。

Also, a ZooKeeper client can remove watches locally even if there is no server connection by setting the local flag to true.

另外,通过将local标志设置为true,即使没有服务器连接,ZooKeeper客户端也可以在本地删除watch。

The following list details the events which will be triggered after the successful watch removal.

以下列表详细说明了成功移除watch后将触发的事件。
  • Child Remove event(子移除事件):
    Watcher which was added with a call to getChildren. 监视器,它是通过调用getChildren添加的。
  • Data Remove event(数据删除事件):
    Watcher which was added with a call to exists or getData.通过调用exists或getData添加的监视
  • Persistent Remove event(持久删除事件):
    Watcher which was added with a call to add a persistent watch.监视器,它是通过调用添加一个持久的watch而添加的。

6. 关于Watcher需要记住的事情(Things to Remember about Watches)

  • Standard watches are one time triggers; if you get a watch event and you want to get notified of future changes, you must set another watch
标准watch为一次性触发器;如果您获得了一个watch事件,并且希望获得关于未来更改的通知,则必须设置另一个watch
  • Because standard watches are one time triggers and there is latency between getting the event and sending a new request to get a watch you cannot reliably see every change that happens to a node in ZooKeeper. Be prepared to handle the case where the znode changes multiple times between getting the event and setting the watch again. (You may not care, but at least realize it may happen.)
因为标准watch是一次性触发器,在获取事件和发送新请求来获取watch之间存在延迟,你不能可靠地看到ZooKeeper中某个节点发生的每个变化。
准备好处理在获取事件和再次设置watch之间znode多次更改的情况。
(你可能不在乎,但至少要意识到这可能会发生。)
  • A watch object, or function/context pair, will only be triggered once for a given notification. For example, if the same watch object is registered for an exists and a getData call for the same file and that file is then deleted, the watch object would only be invoked once with the deletion notification for the file.
一个watch对象,或函数/上下文对,将只触发一次,为一个给定的通知。例如,如果同一个watch对象注册了exists和同一个文件的getData调用,然后该文件被删除,那么这个watch对象只会被调用一次,并带有该文件的删除通知。
  • When you disconnect from a server (for example, when the server fails), you will not get any watches until the connection is reestablished. For this reason session events are sent to all outstanding watch handlers. Use session events to go into a safe mode: you will not be receiving events while disconnected, so your process should act conservatively in that mode.
当您断开与服务器的连接时(例如,当服务器失败时),在重新建立连接之前,您将不会获得任何watch。因此,会话事件被发送到所有未完成的监视处理程序。使用会话事件进入安全模式:断开连接时将不会接收事件,因此流程在该模式下应谨慎行事。
查看原文

赞 0 收藏 0 评论 0

niewj 回答了问题 · 2020-11-17

跨项目读取Spring配置文件的一个问题?

检查看看打包引入进去了否

关注 2 回答 1

niewj 回答了问题 · 2020-11-16

IDEA 在jsp文件中插入<% for(){ %>html语句<% } %>,这样是错的吗?

空指针, 是不是没取到你后台放到request中的数据

关注 2 回答 1

认证与成就

  • 获得 61 次点赞
  • 获得 3 枚徽章 获得 0 枚金徽章, 获得 0 枚银徽章, 获得 3 枚铜徽章

擅长技能
编辑

开源项目 & 著作
编辑

注册于 2016-08-07
个人主页被 2.8k 人浏览