Starting on September 8, Amazon Elasticsearch Service (referred to as Amazon ES) has been renamed to Amazon OpenSearch Service in all Amazon Cloud Technology regions. For more information, see the Amazon OpenSearch Service page.
Amazon OpenSearch Servcie's feature enhancements
The following functions are provided by Amazon OpenSearch Servcie. The open source Elasticsearch does not currently have these functions.
security management: provides encryption, identity verification, authorization and audit functions. Including integration with Active Directory, LDAP, SAML, Kerberos, JSON Web token, etc. OpenSearch also provides fine-grained role-based access control to indexes, documents, and fields.
- Safety management:
https://github.com/opensearch-project/security
SQL query syntax: provides familiar SQL query syntax. Use aggregation, group by, and where clauses to investigate your data. Reading data in the form of JSON documents or CSV tables allows you to flexibly adopt the most suitable format.
- SQL query syntax:
https://github.com/opensearch-project/sql
statistical reports: plans, exports and shares reports from the control panel, saved searches, alerts and visualizations.
- Statistical report:
https://github.com/opensearch-project/dashboards-reports
anomaly detection: uses the machine learning anomaly detection function based on Random Forest (RCF) algorithm to automatically detect anomalies when the data is extracted. conjunction with alarm 161cea370d9a2c, it can monitor data in near real-time and automatically send alarm notifications.
- abnormal detection:
https://github.com/opensearch-project/anomaly-detection - Random Forest (RCF) algorithm:
https://github.com/aws/random-cut-forest-by-aws - alarm:
https://github.com/opensearch-project/alerting
index management: define custom policies to automatically perform routine index management tasks such as scrolling and deletion, they are applied to the index and index modes, as well as conversion .
- Index management:
https://github.com/opensearch-project/anomaly-detection - Conversion:
https://opensearch.org/docs/im-plugin/index-transforms/index/
Performance Analyzer and RCA framework: queries a large number of cluster performance indicators and aggregations. Use PerfTop command line interface (CLI) to quickly display and analyze these indicators. Use the Root Cause Analysis (RCA) framework to investigate performance and reliability issues in the cluster.
- Performance Analyzer and RCA framework:
https://github.com/opensearch-project/performance-analyzer
asynchronous search: runs complex queries without worrying about the timeout of asynchronous search queries running in the background. Track the progress of the query and retrieve the available partial results.
- Asynchronous search:
https://github.com/opensearch-project/asynchronous-search
Trace analysis: extracts and visualizes OpenTelemetry data for distributed applications. Visualize the flow of events between these applications to identify performance issues.
- Trace analysis:
https://github.com/opensearch-project/trace-analytics
alert: automatically monitors data and sends alert notifications to stakeholders. Through the intuitive interface and powerful API, you can easily set up, manage and monitor alarms. Use OpenSearch's complete query language and scripting capabilities to create very specific alert conditions.
k-NN search: uses machine learning to run the nearest neighbor search algorithm on billions of documents with thousands of dimensions, just as easy as ordinary OpenSearch queries. Use aggregation and filtering clauses to further optimize the similarity search operation. K-NN similarity search provides support for use cases such as product suggestion, fraud detection, image and video search, and related document search.
- k-NN search:
https://github.com/opensearch-project/k-NN
vertical bar separated processing language: provides familiar query syntax to query data, which contains a set of commands separated by vertical bars (|).
- Processing languages separated by vertical bars:
https://github.com/opensearch-project/piped-processing-language
control panel notebook: combines control panel, visualization, text, etc., to provide context and detailed instructions when analyzing data.
- Control panel notebook:
https://github.com/opensearch-project/dashboards-notebooks
OpenSearch 1.0 provides three new features, they are: Transforms (transformation) Data Streams (data stream) and Notebooks does not support the existing version of Amazon Opensearch Service. these functions.
- Transforms
https://opensearch.org/docs/im-plugin/index-transforms/index/ - Data Streams
https://opensearch.org/docs/opensearch/data-streams/ - Notebooks
https://opensearch.org/docs/dashboards/notebooks/
release history
In 2015, our launched Amazon Elasticsearch Service (referred to as: Amazon ES), which is a fully managed service that allows you to easily perform interactive log analysis, real-time application monitoring, website search, etc.
- Launched Amazon Elasticsearch Service:
https://aws.amazon.com/blogs/aws/new-amazon-elasticsearch-service/
Amazon ES has always been a popular log analysis service because of its ability to extract large amounts of log data. In addition, with UltraWarm and cold storage layer , you can reduce the cost to one-tenth of traditional hot storage on Amazon ES. With Amazon ES integrating with Logstash, Amazon Kinesis Data Firehose, Amazon CloudWatch Logs, and Amazon IoT , you can choose data extraction tools that meet the requirements of the scenario.
- UltraWarm：
https://aws.amazon.com/blogs/aws/general-availability-of-ultrawarm-for-amazon-elasticsearch-service/ - Cold storage layer
https://aws.amazon.com/blogs/big-data/introducing-cold-storage-for-amazon-elasticsearch-service/ - Logstash
https://aws.amazon.com/ko/elasticsearch-service/the-elk-stack/logstash - Amazon Kinesis Data Firehose
https://aws.amazon.com/kinesis/data-firehose/ - Amazon CloudWatch Logs
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html - Amazon IoT
https://aws.amazon.com/iot
There are many reasons why developers accept open source software. One of the most important reasons is to be able to use the software in the desired way and in the desired location freely. On January 21, 2021, Elastic NV announced that they will change their software licensing strategy. After Elasticsearch version 7.10.2 and Kibana 7.10.2, they will not release new versions of Elasticsearch and Kibana under the relaxed Apache license version 2.0 (Alv2). In contrast, Elastic NV releases Elasticsearch and Kibana under the Elastic license, and the source code is provided under the Elastic license or Server Side Public License (SSPL). These licenses are not open source and do not provide users with the same freedom.
For this reason, we decided to create and maintain the OpenSearch project . It is an open source project community, based on the last Alv2 version of Elasticsearch and Kibana. We recently released version 1.0 , long-term stability and will contribute the OpenSearch project .
- OpenSearch：
https://aws.amazon.com/blogs/opensource/introducing-opensearch/ - Community-driven open source projects:
https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/ - Version 1.0 is released:
https://www.opensearch.org/blog/updates/2021/07/opensearch-general-availability-announcement/ - OpenSearch project:
http://opensearch.org/
OpenSearch provides a highly scalable system. Through the integrated visualization tool OpenSearch control panel , you can quickly access and respond to large amounts of data, allowing users to easily browse their data. The OpenSearch and OpenSearch control panels originally came from Elasticsearch7.10.2 and Kibana7.10.2. Like Elasticsearch and Apache Solr , OpenSearch also implements a search engine Apache Lucene library
- control panel:
https://github.com/opensearch-project/OpenSearch-Dashboards - Apache Solr
https://solr.apache.org/ - Apache Lucene
https://lucene.apache.org/
launched Amazon OpenSearch Service
We renamed Amazon Elasticsearch Service to Amazon OpenSearch Service because the service now supports OpenSearch 1.0. Although the name has changed, we will continue to provide the same experience without any negative impact on clusters that are ongoing, under development, or commercially used.
- Amazon OpenSearch Service：
https://aws.amazon.com/opensearch-service/
Amazon OpenSearch Service provides a series of open source engines for deployment and operation, including currently available 19 versions of Alv2 Elasticsearch (including 7.10 and earlier versions ) and OpenSearch 1.0. We will continue to support and maintain the Alv2 Elasticsearch version, including providing security and bug fixes. At the same time, we will provide brand new features and functions through the OpenSearch and OpenSearch control panels. The Amazon OpenSearch Service API will be backward compatible with existing service APIs, so you don’t need to update your current client code or applications. We will make OpenSearch compatible with open source .
- 7.10 and earlier versions:
https://aws.amazon.com/about-aws/whats-new/2021/04/amazon-elasticsearch-service-announces-support-for-elasticsearch-version-7-10/ - The OpenSearch client is compatible with open source:
https://aws.amazon.com/blogs/opensource/keeping-clients-of-opensearch-and-elasticsearch-compatible-with-open-source/
To get started, select Create a domain Amazon Cloud Management Console In Step 1: Select the deployment type, select OpenSearch 1.0 (the latest version).
- Amazon Cloud Technology Management Console:
https://console.aws.amazon.com/
If you want to deploy a new cluster and access the latest features and enhancements, we recommend OpenSearch 1.0. OpenSearch 1.0 is compatible with the open source Elasticsearch 7.10 API and most clients.
upgrade to OpenSearch 1.0
Amazon OpenSearch Service provides a seamless in-place upgrade path from existing Elasticsearch 6.x and 7.x managed clusters to OpenSearch.
Upgrade Elasticsearch and Service Software Upgrade in the Amazon OpenSearch Service Developer Guide.
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。