Elasticsearch教程之四：AWS Opensearch

AWS Elasticsearch Opensearch
教程
发布日期:   2021-11-14

AWS Opensearch源于Elasticsearch，因此其基本概念和使用方法和Elasticsearch基本一样。具体请参考前面的文章。关于两者区别请参考官网以及Elastic官网的说明。下面具体介绍一下如何使用AWS Opensearch。

Elasticsearch教程
Elasticsearch教程

Elasticsearch系列教程

创建Opensearch实例

首先需要的就是创建域(domain)。Domain其实就是Opensearch集群。在domain中包含相应的实例类型，实例数，存储资源等。

在创建域的时候输入/选择：

  • Domain name: lcoding
  • Deployment type: Development and testing
  • Version: 1.0 (latest)
  • Enable compatibility mode (勾选)
  • Instance type: t2.small.search
  • Dedicated master nodes: 由于是测试环境，无需勾选
  • Network: Public access

创建过程大约需要几分钟左右。在创建完毕后，可以看到两个地址：

  • OpenSearch Dashboards URL: 这是Kibina的URL
  • Domain endpoint: 这是在程序中集成时需要使用的

但在默认情况下是不能访问的：

json
{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet with an explicit deny"}

其原因也不难理解，默认的访问策略是这样的：

json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:eu-west-1:xxxxxxxx:domain/testindex/*"
    }
  ]
}

需要更改这个访问策略，比如允许指定IP进行访问：

json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:eu-west-1:xxxxxxxx:domain/*",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": [
            "xx.xx.xx.xx/32",
            "yy.yy.yy.yy/32"
          ] 
        }
      }
    }
  ]
}

这样就没有问题了。

OpenSearch Dashboards:

Elasticsearch教程
Elasticsearch教程

Devtool:

Elasticsearch教程
Elasticsearch教程

通过命令行测试

bash
curl https://xxxxxx.eu-west-1.es.amazonaws.com/

其返回值类似于这样：

json
{                                                           
  "name" : "xxxx",              
  "cluster_name" : "xxxx:YOUR_DOMAIN",         
  "cluster_uuid" : "xxxx",                
  "version" : {                                             
    "number" : "7.1.1",                                     
    "build_flavor" : "oss",                                 
    "build_type" : "tar",                                   
    "build_hash" : "b22a3b7",                               
    "build_date" : "2021-10-20T12:26:19.802441Z",           
    "build_snapshot" : false,                               
    "lucene_version" : "8.0.0",                             
    "minimum_wire_compatibility_version" : "6.8.0",         
    "minimum_index_compatibility_version" : "6.0.0-beta1"   
  },                                                        
  "tagline" : "You Know, for Search"                        
}

常用命令：

索引列表

bash
curl https://xxxxxx.eu-west-1.es.amazonaws.com/_cat/indices

系统状态

bash
curl https://xxxxxx.eu-west-1.es.amazonaws.com/_cat/health?v

显示节点信息

bash
curl https://xxxxxx.eu-west-1.es.amazonaws.com/_cat/nodes?v

创建索引

bash
curl -XPUT -H "Content-Type: application/json" https://xxxxxx.eu-west-1.es.amazonaws.com/YOUR_INDEX_NAME/external/1?pretty -d '{ "name": "OSS test"}'

推荐工具

强烈推荐一个浏览器插件：ElasticVue，在其中可以方便的创建索引，查询数据等：

Elasticsearch教程
Elasticsearch教程

安装OpenSearch Docker镜像

安装

官网：https://hub.docker.com/r/opensearchproject/opensearch

bash
docker pull opensearchproject/opensearch:latest
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:latest

测试

bash
curl -XGET https://localhost:9200 -u admin:admin --insecure
curl -XGET https://localhost:9200/_cat/nodes?v -u admin:admin --insecure
curl -XGET https://localhost:9200/_cat/plugins?v -u admin:admin --insecure

输出：

json
{
  "name" : "7bd7e6a758c2",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "lTjKmPFJQiePJjQlcRKy_w",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.1.0",
    "build_type" : "tar",
    "build_hash" : "15e9f137622d878b79103df8f82d78d782b686a1",
    "build_date" : "2021-10-04T21:29:03.079792Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

停止OpenSearch服务

首先找到container-id:

bash
docker ps

然后运行：

bash
docker stop <container-id>

解决使用React访问OpenSearch时的跨域问题(CORS)

首先运行如下命令将opensearch.yml从docker中复制到host：

bash
docker cp cc7fd6e140de:/usr/share/opensearch/config/opensearch.yml .

然后添加如下内容：

yaml
http.cors.enabled : true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: X-Requested-With,X-Auth-Token,Content-Type,Content-Length
http.cors.allow-credentials: true

再将文件复制回去：

bash
docker cp ./opensearch.yml cc7fd6e140de:/usr/share/opensearch/config/opensearch.yml

重新启动docker容器：

bash
docker restart cc7fd6e140de

AWS Elasticsearch / OpenSearch的安全性

以下以OpenSearch为例，ES完全相同。

OpenSearch(OS)本身并没有Authn/Authz层。因此需要额外的配置来实现安全性。一种通常的做法是添加一个访问OS的Proxy，然后在Proxy中实现访问控制。

OpenSearch的安全性包括：

  • Access
  • Authentication
  • Authorization
  • Audit
文章作者: 逻思
文章链接: https://www.lcoding.club/elasticsearch-tutorial-4-aws-opensearch/
版权声明: 本博客所有文章除特別声明外，均采用 CC BY-NC-ND 4.0 许可协议。转载请注明来源 逻思 !
AWS Elasticsearch Opensearch
赏