Practice 002-elasticsearch's batch operation API-bulk, mget, msearch

头像
丰木
    阅读 13 分钟
    中文

    [toc]

    1. Data preparation

    Now there are the following indexes: books, my_index, users

    1.books index content:

    _index _type _id ▲_score bookId bookName author
    books _doc 1 1 1 Thinking in Java Bruce Eckel
    books _doc 2 1 2 General History of China Lu Simian
    books _doc 3 1 3 General History of China Lu Simian
    books _doc NCvOY4ABRxSL2QPxNNHf 1 4 General History of China 4 Lu Simian
    books _doc NSvOY4ABRxSL2QPxPdEy 1 4 General History of China 4 Lu Simian
    books _doc NivOY4ABRxSL2QPxQdEo 1 4 General History of China 4 Lu Simian
    books _doc NyvOY4ABRxSL2QPxRNHC 1 4 General History of China 4 Lu Simian
    books _doc OCvOY4ABRxSL2QPxSdHA 1 4 General History of China 4 Lu Simian
    books _doc OSvOY4ABRxSL2QPxTNHb 1 4 General History of China 4 Lu Simian
    books _doc OivOY4ABRxSL2QPxYNGs 1 4 Lu Xun's Prose Lu Xun

    2. my_index index content:

    _index _type _id _score firstName lastName
    my_index _doc 1 1 nie wj
    my_index _doc 2 1

    3.users index content

    _index _type _id _score name age gender birth
    users _doc 1 1 niewj 36 male 1985-01-01
    users _doc 2 1 nie 26 female 1995-02-02
    users _doc 3 1 lifubo 33 male
    users _doc 4 1 weibinbin 32 male

    2. API test demo of bulk, mget, msearch

    1. Bulk executes index/create/update/delete of multiple indexes

    1.1 Test execution requirements

    1. Gender in users is changed to "male";
    2. Add "Zhang Daqian" to my_index and use index; use create for the two records of "Li Taibai, Du Fu"
    3. Delete the record with bookId=4 in books

    1.2 Execute the script

     POST _bulk
{"delete":{"_index":"books","_id":"NSvOY4ABRxSL2QPxPdEy"}}
{"delete":{"_index":"books","_id":"NivOY4ABRxSL2QPxQdEo"}}
{"delete":{"_index":"books","_id":"NyvOY4ABRxSL2QPxRNHC"}}
{"delete":{"_index":"books","_id":"OCvOY4ABRxSL2QPxSdHA"}}
{"delete":{"_index":"books","_id":"OSvOY4ABRxSL2QPxTNHb"}}
{"delete":{"_index":"books","_id":"OivOY4ABRxSL2QPxYNGs"}}
{"delete":{"_index":"books","_id":"NCvOY4ABRxSL2QPxNNHf"}}
{"index":{"_index":"my_index","_id":"3"}}
{"firstName":"张","lastName":"大千"}
{"create":{"_index":"my_index","_id":"4"}}
{"firstName":"李","lastName":"太白"}
{"create":{"_index":"my_index","_id":"5"}}
{"firstName":"杜","lastName":"甫"}
{"update":{"_index":"users","_id":"2"}}
{"doc":{"gender":"male"}}

    1.3 Note that delete cannot be specified directly: bookId: 4

    1.4 Execution results:

     {
  "took" : 68,
  "errors" : false,
  "items" : [
    {
      "delete" : {
        "_index" : "books",
        "_type" : "_doc",
        "_id" : "NSvOY4ABRxSL2QPxPdEy",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 12,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "books",
        "_type" : "_doc",
        "_id" : "NivOY4ABRxSL2QPxQdEo",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 13,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "books",
        "_type" : "_doc",
        "_id" : "NyvOY4ABRxSL2QPxRNHC",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 14,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "books",
        "_type" : "_doc",
        "_id" : "OCvOY4ABRxSL2QPxSdHA",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 15,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "books",
        "_type" : "_doc",
        "_id" : "OSvOY4ABRxSL2QPxTNHb",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 16,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "books",
        "_type" : "_doc",
        "_id" : "OivOY4ABRxSL2QPxYNGs",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 17,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "books",
        "_type" : "_doc",
        "_id" : "NCvOY4ABRxSL2QPxNNHf",
        "_version" : 3,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 18,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "index" : {
        "_index" : "my_index",
        "_type" : "_doc",
        "_id" : "3",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 3,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "create" : {
        "_index" : "my_index",
        "_type" : "_doc",
        "_id" : "4",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 4,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "create" : {
        "_index" : "my_index",
        "_type" : "_doc",
        "_id" : "5",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 5,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "update" : {
        "_index" : "users",
        "_type" : "_doc",
        "_id" : "2",
        "_version" : 10,
        "result" : "updated",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 15,
        "_primary_term" : 4,
        "status" : 200
      }
    }
  ]
}

    1.5 Execution result: users

    _index _type _id _score firstName lastName
    my_index _doc 1 1 nie wj
    my_index _doc 2 1
    my_index _doc 3 1 open Dachi
    my_index _doc 4 1 plum Taibai
    my_index _doc 5 1 Du just

    1.6 Execution result: my_index

    _index _type _id _score name age gender birth
    users _doc 1 1 niewj 36 male 1985-01-01
    users _doc 3 1 lifubo 33 male
    users _doc 4 1 weibinbin 32 male
    users _doc 2 1 nie 26 male 1995-02-02

    1.7 Execution result: books

    _index _type _id ▲_score bookId bookName author
    books _doc 1 1 1 Thinking in Java Bruce Eckel
    books _doc 2 1 2 General History of China Lu Simian
    books _doc 3 1 3 General History of China Lu Simian

    2. mget: get documents in multiple indexes

    2.1 mget script

     GET _mget
{"docs":[
  {"_index":"users","_id":"1"},
  {"_index":"books","_id":"1"},
  {"_index":"my_index","_id":"1"}
 ]
}

    2.2 mget results:

     {
  "docs" : [
    {
      "_index" : "users",
      "_type" : "_doc",
      "_id" : "1",
      "_version" : 1,
      "_seq_no" : 12,
      "_primary_term" : 4,
      "found" : true,
      "_source" : {
        "name" : "niewj",
        "age" : 36,
        "gender" : "male",
        "birth" : "1985-01-01"
      }
    },
    {
      "_index" : "books",
      "_type" : "_doc",
      "_id" : "1",
      "_version" : 2,
      "_seq_no" : 1,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "bookId" : "1",
        "bookName" : "Thinking in Java",
        "author" : "Bruce Eckel"
      }
    },
    {
      "_index" : "my_index",
      "_type" : "_doc",
      "_id" : "1",
      "_version" : 1,
      "_seq_no" : 0,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "firstName" : "nie",
        "lastName" : "wj"
      }
    }
  ]
}

    3.msearch batch query multiple indexes

    3.1 Query multiple conditions for multiple indexes (reduce the number of IOs)

     POST _msearch
{"index":"movies"}
{"query":{"match_all":{}},"from":0,"size":3}
{"index":"users"}
{"query":{"match_phrase":{"name.keyword":"nie"}}}
{"index":"books"}
{"query":{"match":{"bookName":"中国通史"}}}

    3.2 Query results

     {
  "took" : 0,
  "responses" : [
    {
      "took" : 0,
      "timed_out" : false,
      "_shards" : {
        "total" : 1,
        "successful" : 1,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : {
          "value" : 9743,
          "relation" : "eq"
        },
        "max_score" : 1.0,
        "hits" : [
          {
            "_index" : "movies",
            "_type" : "_doc",
            "_id" : "982",
            "_score" : 1.0,
            "_source" : {
              "id" : "982",
              "genre" : [
                "Drama"
              ],
              "title" : "Picnic",
              "@version" : "1",
              "year" : 1955
            }
          },
          {
            "_index" : "movies",
            "_type" : "_doc",
            "_id" : "984",
            "_score" : 1.0,
            "_source" : {
              "id" : "984",
              "genre" : [
                "Comedy",
                "Drama"
              ],
              "title" : "Pompatus of Love, The",
              "@version" : "1",
              "year" : 1996
            }
          },
          {
            "_index" : "movies",
            "_type" : "_doc",
            "_id" : "986",
            "_score" : 1.0,
            "_source" : {
              "id" : "986",
              "genre" : [
                "Adventure",
                "Children"
              ],
              "title" : "Fly Away Home",
              "@version" : "1",
              "year" : 1996
            }
          }
        ]
      },
      "status" : 200
    },
    {
      "took" : 0,
      "timed_out" : false,
      "_shards" : {
        "total" : 1,
        "successful" : 1,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : {
          "value" : 1,
          "relation" : "eq"
        },
        "max_score" : 0.87546873,
        "hits" : [
          {
            "_index" : "users",
            "_type" : "_doc",
            "_id" : "2",
            "_score" : 0.87546873,
            "_source" : {
              "name" : "nie",
              "age" : 26,
              "gender" : "male",
              "birth" : "1995-02-02"
            }
          }
        ]
      },
      "status" : 200
    },
    {
      "took" : 0,
      "timed_out" : false,
      "_shards" : {
        "total" : 1,
        "successful" : 1,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : {
          "value" : 2,
          "relation" : "eq"
        },
        "max_score" : 1.8126037,
        "hits" : [
          {
            "_index" : "books",
            "_type" : "_doc",
            "_id" : "2",
            "_score" : 1.8126037,
            "_source" : {
              "bookId" : "2",
              "bookName" : "中国通史",
              "author" : "吕思勉"
            }
          },
          {
            "_index" : "books",
            "_type" : "_doc",
            "_id" : "3",
            "_score" : 1.8126037,
            "_source" : {
              "bookId" : "3",
              "bookName" : "中国通史",
              "author" : "吕思勉"
            }
          }
        ]
      },
      "status" : 200
    }
  ]
}
    Each indexed query returned its own results
    elasticsearch
    丰木
    322 声望19 粉丝

    遇见超乎想象的自己!

    « 上一篇
    实践001-elasticsearch的index、create、update
    下一篇 »
    实践003-elasticsearch之analyzer

    引用和评论

    推荐阅读
    头像
    npm安装vue.js的2.x和3.x版本的vuex/vue-router/vue-cli安装命令

    丰木阅读 470

    头像
    一夜之间，7.0万 Star，全部清零！

    CodeSheep阅读 1.5k

    头像
    ElasticSearch源码本地调试指南

    Foghost阅读 1.1k

    头像
    Elasticsearch 8.x 重要变化（qbit）

    qbit阅读 819

    头像
    超值选择：阿里云Elasticsearch Serverless在企业数据检索与分析中的高性能与灵活性

    阿里云大数据AI阅读 443

    头像
    试试 Elasticsearch 的 unsigned_long（qbit）

    qbit阅读 432

    头像
    阿里云 Elastic Enterprise 正式上线！

    阿里云大数据AI阅读 432

    0 条评论
    得票最新