diff --git a/README.md b/README.md index 8c5ac8b..b4e2d3b 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ 5. 单字符串多字段查询:Dis Max Query 6. 单字符串多字段查询:Multi Match 7. 多语言及中文分词与检索 -8. Space Jam,一次全文搜索的实例 +8. Space Jam,一次全文搜索的实例 9. 使用 Search Template 和 Index Alias 查询 10. 综合排序:Function Score Query 优化算分 11. Term & Phrase Suggester @@ -58,9 +58,9 @@ ### 第 7 章:数据建模 1. 对象及 Nested 对象 2. 文档的父子关系 -3. Update By Query & Reindex API -4. Ingest Pipeline & Painless Script -5. Elasticsearch 数据建模最佳实践 +3. Elasticsearch 数据建模最佳实践 +4. Update By Query & Reindex API +5. Ingest Pipeline & Painless Script 6. 第二部分总结回顾 ## 第三部分:管理 Elasticsearch 集群 ### 第 8 章:保护你的数据 @@ -95,7 +95,7 @@ 1. 使用 Index Pattern 配置数据 2. 使用 Kibana Discover 探索数据 3. 基本可视化组件介绍 -4. Visual Builder 介绍 +4. Visual Builder 介绍 5. 构建 Dashboard ## 第 14 章:探索 X-Pack 套件 1. 用 Monitoring 和 Alerting 监控 Elasticsearch 集群 @@ -117,3 +117,6 @@ ### 备战:Elastic 认证 1. Elastic 认证介绍 2. 考点梳理 + +# ELK 相关下载资源 +1. ELK 7.x 推荐官网直接下载,如网速低,可使用以下链接 - 百度网盘下载(https://pan.baidu.com/s/1CRT3W4wEESglCBDnslk2AA) diff --git "a/part-1/2.1-Elasticsearch\347\232\204\345\256\211\350\243\205\344\270\216\347\256\200\345\215\225\351\205\215\347\275\256/README.md" "b/part-1/2.1-Elasticsearch\347\232\204\345\256\211\350\243\205\344\270\216\347\256\200\345\215\225\351\205\215\347\275\256/README.md" index a75dbef..98a45a0 100644 --- "a/part-1/2.1-Elasticsearch\347\232\204\345\256\211\350\243\205\344\270\216\347\256\200\345\215\225\351\205\215\347\275\256/README.md" +++ "b/part-1/2.1-Elasticsearch\347\232\204\345\256\211\350\243\205\344\270\216\347\256\200\345\215\225\351\205\215\347\275\256/README.md" @@ -1,31 +1,34 @@ # Elasticsearch 的安装与简单配置 ## 课程Demo ``` -//启动单节点 +#启动单节点 bin/elasticsearch -E node.name=node0 -E cluster.name=geektime -E path.data=node0_data -//安装插件 + +#安装插件 bin/elasticsearch-plugin install analysis-icu -//查看插件 +#查看插件 bin/elasticsearch-plugin list -//查看安装的插件 +#查看安装的插件 GET http://localhost:9200/_cat/plugins?v -//start multi-nodes Cluster +#start multi-nodes Cluster bin/elasticsearch -E node.name=node0 -E cluster.name=geektime -E path.data=node0_data bin/elasticsearch -E node.name=node1 -E cluster.name=geektime -E path.data=node1_data bin/elasticsearch -E node.name=node2 -E cluster.name=geektime -E path.data=node2_data bin/elasticsearch -E node.name=node3 -E cluster.name=geektime -E path.data=node3_data -//查看集群 +#查看集群 GET http://localhost:9200 -//查看nodes +#查看nodes GET _cat/nodes GET _cluster/health ``` ## 相关阅读 -- https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html +- 安装指南 https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch. +- Elastic Support Matrix(OS / JDK ) https://www.elastic.co/cn/support/matrix +- Elasticsearch 的一些重要配置 https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html - https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html - https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html - Elasticsearch on Kuvernetes https://www.elastic.co/cn/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond diff --git "a/part-1/2.2-Kibana\347\232\204\345\256\211\350\243\205\344\270\216\347\225\214\351\235\242\345\277\253\351\200\237\346\265\217\350\247\210/README.md" "b/part-1/2.2-Kibana\347\232\204\345\256\211\350\243\205\344\270\216\347\225\214\351\235\242\345\277\253\351\200\237\346\265\217\350\247\210/README.md" index dc62b90..fa139ce 100644 --- "a/part-1/2.2-Kibana\347\232\204\345\256\211\350\243\205\344\270\216\347\225\214\351\235\242\345\277\253\351\200\237\346\265\217\350\247\210/README.md" +++ "b/part-1/2.2-Kibana\347\232\204\345\256\211\350\243\205\344\270\216\347\225\214\351\235\242\345\277\253\351\200\237\346\265\217\350\247\210/README.md" @@ -1,10 +1,10 @@ # Kibana 的安装与界面快速浏览 ## 课程Demo ``` -//启动kibana +#启动kibana bin/kibana -//查看插件 +#查看插件 bin/kibana-plugin list ``` diff --git "a/part-1/2.3-\345\234\250Docker\345\256\271\345\231\250\344\270\255\350\277\220\350\241\214Elasticsearch,Kibana\345\222\214Cerebro/README.md" "b/part-1/2.3-\345\234\250Docker\345\256\271\345\231\250\344\270\255\350\277\220\350\241\214Elasticsearch,Kibana\345\222\214Cerebro/README.md" index 3a7f5ac..b66d0fb 100644 --- "a/part-1/2.3-\345\234\250Docker\345\256\271\345\231\250\344\270\255\350\277\220\350\241\214Elasticsearch,Kibana\345\222\214Cerebro/README.md" +++ "b/part-1/2.3-\345\234\250Docker\345\256\271\345\231\250\344\270\255\350\277\220\350\241\214Elasticsearch,Kibana\345\222\214Cerebro/README.md" @@ -4,25 +4,25 @@ 进入 7.x-docker-2-es-instance目录 ``` -//启动 +#启动 docker-compose up -//停止容器 +#停止容器 docker-compose down -//停止容器并且移除数据 +#停止容器并且移除数据 docker-compose down -v -//一些docker 命令 +#一些docker 命令 docker ps docker stop Name/ContainerId docker start Name/ContainerId -删除单个容器 +#删除单个容器 $docker rm Name/ID -f, –force=false; -l, –link=false Remove the specified link and not the underlying container; -v, –volumes=false Remove the volumes associated to the container -删除所有容器 +#删除所有容器 $docker rm `docker ps -a -q` 停止、启动、杀死、重启一个容器 $docker stop Name/ID @@ -39,3 +39,4 @@ $docker restart name/ID - 如何设置 Docker 网络 - https://www.elastic.co/cn/blog/docker-networking - Cerebro 源码 https://github.com/lmenezes/cerebro - 一个开源的 ELK(Elasticsearch + Logstash + Kibana) docker-compose 配置 https://github.com/deviantony/docker-elk +- Install Elasticsearch with Docker https://www.elastic.co/guide/en/elasticsearch/reference/7.2/docker.html diff --git "a/part-1/2.3-\345\234\250Docker\345\256\271\345\231\250\344\270\255\350\277\220\350\241\214Elasticsearch,Kibana\345\222\214Cerebro/docker-es-7.2/docker-compose.yaml" "b/part-1/2.3-\345\234\250Docker\345\256\271\345\231\250\344\270\255\350\277\220\350\241\214Elasticsearch,Kibana\345\222\214Cerebro/docker-es-7.2/docker-compose.yaml" new file mode 100644 index 0000000..aa5196e --- /dev/null +++ "b/part-1/2.3-\345\234\250Docker\345\256\271\345\231\250\344\270\255\350\277\220\350\241\214Elasticsearch,Kibana\345\222\214Cerebro/docker-es-7.2/docker-compose.yaml" @@ -0,0 +1,74 @@ +version: '2.2' +services: + cerebro: + image: lmenezes/cerebro:0.8.3 + container_name: cerebro + ports: + - "9000:9000" + command: + - -Dhosts.0.host=http://elasticsearch:9200 + networks: + - es72net + kibana: + image: docker.elastic.co/kibana/kibana:7.2.0 + container_name: kibana72 + environment: + #- I18N_LOCALE=zh-CN + - XPACK_GRAPH_ENABLED=true + - TIMELION_ENABLED=true + - XPACK_MONITORING_COLLECTION_ENABLED="true" + ports: + - "5601:5601" + networks: + - es72net + elasticsearch: + image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0 + container_name: es72_01 + environment: + - cluster.name=geektime + - node.name=es72_01 + - bootstrap.memory_lock=true + - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + - discovery.seed_hosts=es72_01 + - network.publish_host=elasticsearch + - cluster.initial_master_nodes=es72_01,es72_02 + ulimits: + memlock: + soft: -1 + hard: -1 + volumes: + - es72data1:/usr/share/elasticsearch/data + ports: + - 9200:9200 + networks: + - es72net + elasticsearch2: + image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0 + container_name: es72_02 + environment: + - cluster.name=geektime + - node.name=es72_02 + - bootstrap.memory_lock=true + - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + - discovery.seed_hosts=es72_01 + - network.publish_host=elasticsearch + - cluster.initial_master_nodes=es72_01,es72_02 + ulimits: + memlock: + soft: -1 + hard: -1 + volumes: + - es72data2:/usr/share/elasticsearch/data + networks: + - es72net + + +volumes: + es72data1: + driver: local + es72data2: + driver: local + +networks: + es72net: + driver: bridge diff --git "a/part-1/2.4-Logstash\345\256\211\350\243\205\344\270\216\345\257\274\345\205\245\346\225\260\346\215\256/README.md" "b/part-1/2.4-Logstash\345\256\211\350\243\205\344\270\216\345\257\274\345\205\245\346\225\260\346\215\256/README.md" index dccdc1c..6adce2d 100644 --- "a/part-1/2.4-Logstash\345\256\211\350\243\205\344\270\216\345\257\274\345\205\245\346\225\260\346\215\256/README.md" +++ "b/part-1/2.4-Logstash\345\256\211\350\243\205\344\270\216\345\257\274\345\205\245\346\225\260\346\215\256/README.md" @@ -7,9 +7,9 @@ - movielens/logstash6.conf //logstash 6.x 配置文件 ``` -//下载与ES相同版本号的logstash,(7.1.0),并解压到相应目录 -//修改movielens目录下的logstash.conf文件 -//path修改为,你实际的movies.csv路径 +#下载与ES相同版本号的logstash,(7.1.0),并解压到相应目录 +#修改movielens目录下的logstash.conf文件 +#path修改为,你实际的movies.csv路径 input { file { path => "YOUR_FULL_PATH_OF_movies.csv" @@ -18,7 +18,7 @@ input { } } -//启动Elasticsearch实例,然后启动 logstash,并制定配置文件导入数据 +#启动Elasticsearch实例,然后启动 logstash,并制定配置文件导入数据 bin/logstash -f /YOUR_PATH_of_logstash.conf ``` diff --git "a/part-1/3.1-\345\237\272\346\234\254\346\246\202\345\277\2651\347\264\242\345\274\225\346\226\207\346\241\243\345\222\214RESTAPI/README.md" "b/part-1/3.1-\345\237\272\346\234\254\346\246\202\345\277\2651\347\264\242\345\274\225\346\226\207\346\241\243\345\222\214RESTAPI/README.md" index 77fb73a..b6c8297 100644 --- "a/part-1/3.1-\345\237\272\346\234\254\346\246\202\345\277\2651\347\264\242\345\274\225\346\226\207\346\241\243\345\222\214RESTAPI/README.md" +++ "b/part-1/3.1-\345\237\272\346\234\254\346\246\202\345\277\2651\347\264\242\345\274\225\346\226\207\346\241\243\345\222\214RESTAPI/README.md" @@ -5,31 +5,31 @@ Index 相关 API ``` -//查看索引相关信息 +#查看索引相关信息 GET kibana_sample_data_ecommerce -//查看索引的文档总数 +#查看索引的文档总数 GET kibana_sample_data_ecommerce/_count -//查看前10条文档,了解文档格式 +#查看前10条文档,了解文档格式 POST kibana_sample_data_ecommerce/_search { } -//_cat indices API -//查看indices +#_cat indices API +#查看indices GET /_cat/indices/kibana*?v&s=index -//查看状态为绿的索引 +#查看状态为绿的索引 GET /_cat/indices?v&health=green -//按照文档个数排序 +#按照文档个数排序 GET /_cat/indices?v&s=docs.count:desc -//查看具体的字段 +#查看具体的字段 GET /_cat/indices/kibana*?pri&v&h=health,index,pri,rep,docs.count,mt -//How much memory is used per index? +#How much memory is used per index? GET /_cat/indices?v&h=i,tm&s=tm:desc ``` diff --git "a/part-1/3.10-DynamicMapping\345\222\214\345\270\270\350\247\201\345\255\227\346\256\265\347\261\273\345\236\213 /README.md" "b/part-1/3.10-DynamicMapping\345\222\214\345\270\270\350\247\201\345\255\227\346\256\265\347\261\273\345\236\213/README.md" similarity index 78% rename from "part-1/3.10-DynamicMapping\345\222\214\345\270\270\350\247\201\345\255\227\346\256\265\347\261\273\345\236\213 /README.md" rename to "part-1/3.10-DynamicMapping\345\222\214\345\270\270\350\247\201\345\255\227\346\256\265\347\261\273\345\236\213/README.md" index 643f7a4..0fa62a5 100644 --- "a/part-1/3.10-DynamicMapping\345\222\214\345\270\270\350\247\201\345\255\227\346\256\265\347\261\273\345\236\213 /README.md" +++ "b/part-1/3.10-DynamicMapping\345\222\214\345\270\270\350\247\201\345\255\227\346\256\265\347\261\273\345\236\213/README.md" @@ -21,7 +21,7 @@ strict -//写入文档,查看 Mapping +#写入文档,查看 Mapping PUT mapping_test/_doc/1 { "firstName":"Chan", @@ -29,14 +29,14 @@ PUT mapping_test/_doc/1 "loginDate":"2018-07-24T10:29:48.103Z" } -//查看 Mapping文件 +#查看 Mapping文件 GET mapping_test/_mapping -//Delete index +#Delete index DELETE mapping_test -//dynamic mapping,推断字段的类型 +#dynamic mapping,推断字段的类型 PUT mapping_test/_doc/1 { "uid" : "123", @@ -46,17 +46,17 @@ PUT mapping_test/_doc/1 "heigh":180 } -//查看 Dynamic +#查看 Dynamic GET mapping_test/_mapping -//默认Mapping支持dynamic,写入的文档中加入新的字段 +#默认Mapping支持dynamic,写入的文档中加入新的字段 PUT dynamic_mapping_test/_doc/1 { "newField":"someValue" } -//该字段可以被搜索,数据也在_source中出现 +#该字段可以被搜索,数据也在_source中出现 POST dynamic_mapping_test/_search { "query":{ @@ -67,20 +67,20 @@ POST dynamic_mapping_test/_search } -//修改为dynamic false +#修改为dynamic false PUT dynamic_mapping_test/_mapping { "dynamic": false } -//新增 anotherField +#新增 anotherField PUT dynamic_mapping_test/_doc/10 { "anotherField":"someValue" } -//该字段不可以被搜索,应为dynamic已经被设置为false +#该字段不可以被搜索,应为dynamic已经被设置为false POST dynamic_mapping_test/_search { "query":{ @@ -92,7 +92,7 @@ POST dynamic_mapping_test/_search get dynamic_mapping_test/_doc/10 -//修改为strict +#修改为strict PUT dynamic_mapping_test/_mapping { "dynamic": "strict" @@ -100,7 +100,7 @@ PUT dynamic_mapping_test/_mapping -//写入数据出错,HTTP Code 400 +#写入数据出错,HTTP Code 400 PUT dynamic_mapping_test/_doc/12 { "lastField":"value" diff --git "a/part-1/3.11-\346\230\276\345\274\217Mapping\350\256\276\347\275\256\344\270\216\345\270\270\350\247\201\345\217\202\346\225\260\344\273\213\347\273\215/README.md" "b/part-1/3.11-\346\230\276\345\274\217Mapping\350\256\276\347\275\256\344\270\216\345\270\270\350\247\201\345\217\202\346\225\260\344\273\213\347\273\215/README.md" index 492d255..a5f466c 100644 --- "a/part-1/3.11-\346\230\276\345\274\217Mapping\350\256\276\347\275\256\344\270\216\345\270\270\350\247\201\345\217\202\346\225\260\344\273\213\347\273\215/README.md" +++ "b/part-1/3.11-\346\230\276\345\274\217Mapping\350\256\276\347\275\256\344\270\216\345\270\270\350\247\201\345\217\202\346\225\260\344\273\213\347\273\215/README.md" @@ -2,7 +2,7 @@ ## 课程Demos ``` -//设置 index 为 false +#设置 index 为 false DELETE users PUT users { @@ -41,7 +41,7 @@ POST /users/_search -//设定Null_value +#设定Null_value DELETE users PUT users @@ -90,7 +90,7 @@ GET users/_search -//设置 Copy to +#设置 Copy to DELETE users PUT users { @@ -128,7 +128,7 @@ POST users/_search } -//数组类型 +#数组类型 PUT users/_doc/1 { "name":"onebird", diff --git "a/part-1/3.12-\345\244\232\345\255\227\346\256\265\347\211\271\346\200\247\345\217\212Mapping\344\270\255\351\205\215\347\275\256\350\207\252\345\256\232\344\271\211Analyzer/README.md" "b/part-1/3.12-\345\244\232\345\255\227\346\256\265\347\211\271\346\200\247\345\217\212Mapping\344\270\255\351\205\215\347\275\256\350\207\252\345\256\232\344\271\211Analyzer/README.md" index bf639af..070b004 100644 --- "a/part-1/3.12-\345\244\232\345\255\227\346\256\265\347\211\271\346\200\247\345\217\212Mapping\344\270\255\351\205\215\347\275\256\350\207\252\345\256\232\344\271\211Analyzer/README.md" +++ "b/part-1/3.12-\345\244\232\345\255\227\346\256\265\347\211\271\346\200\247\345\217\212Mapping\344\270\255\351\205\215\347\275\256\350\207\252\345\256\232\344\271\211Analyzer/README.md" @@ -22,7 +22,7 @@ POST _analyze -//使用char filter进行替换 +#使用char filter进行替换 POST _analyze { "tokenizer": "standard", diff --git "a/part-1/3.13-IndexTemplate\345\222\214DynamicTemplate/README.md" "b/part-1/3.13-IndexTemplate\345\222\214DynamicTemplate/README.md" index 8cfc327..bf7955f 100644 --- "a/part-1/3.13-IndexTemplate\345\222\214DynamicTemplate/README.md" +++ "b/part-1/3.13-IndexTemplate\345\222\214DynamicTemplate/README.md" @@ -2,7 +2,7 @@ ## 课程Demo ``` -//数字字符串被映射成text,日期字符串被映射成日期 +#数字字符串被映射成text,日期字符串被映射成日期 PUT ttemplate/_doc/1 { "someNumber":"1", @@ -11,7 +11,7 @@ PUT ttemplate/_doc/1 GET ttemplate/_mapping -//Create a default template +#Create a default template PUT _template/template_default { "index_patterns": ["*"], @@ -38,12 +38,12 @@ PUT /_template/template_test } } -//查看template信息 +#查看template信息 GET /_template/template_default GET /_template/temp* -//写入新的数据,index以test开头 +#写入新的数据,index以test开头 PUT testtemplate/_doc/1 { "someNumber":"1", @@ -71,7 +71,7 @@ DELETE /_template/template_test -//Dynaminc Mapping 根据类型和字段名 +#Dynaminc Mapping 根据类型和字段名 DELETE my_index PUT my_index/_doc/1 @@ -109,7 +109,7 @@ PUT my_index DELETE my_index -//结合路径 +#结合路径 PUT my_index { "mappings": { diff --git "a/part-1/3.14-Elasticsearch\350\201\232\345\220\210\345\210\206\346\236\220\347\256\200\344\273\213/README.md" "b/part-1/3.14-Elasticsearch\350\201\232\345\220\210\345\210\206\346\236\220\347\256\200\344\273\213/README.md" index 6b95682..0209c6d 100644 --- "a/part-1/3.14-Elasticsearch\350\201\232\345\220\210\345\210\206\346\236\220\347\256\200\344\273\213/README.md" +++ "b/part-1/3.14-Elasticsearch\350\201\232\345\220\210\345\210\206\346\236\220\347\256\200\344\273\213/README.md" @@ -2,7 +2,7 @@ ## 课程Demo - 需要通过Kibana导入Sample Data的飞机航班数据。具体参考“2.2节-Kibana的安装与界面快速浏览” ``` -//按照目的地进行分桶统计 +#按照目的地进行分桶统计 GET kibana_sample_data_flights/_search { "size": 0, @@ -17,7 +17,7 @@ GET kibana_sample_data_flights/_search -//查看航班目的地的统计信息,增加平均,最高最低价格 +#查看航班目的地的统计信息,增加平均,最高最低价格 GET kibana_sample_data_flights/_search { "size": 0, @@ -49,7 +49,7 @@ GET kibana_sample_data_flights/_search -//价格统计信息+天气信息 +#价格统计信息+天气信息 GET kibana_sample_data_flights/_search { "size": 0, diff --git "a/part-1/3.2-\345\237\272\346\234\254\346\246\202\345\277\2652\350\212\202\347\202\271,\351\233\206\347\276\244,\345\210\206\347\211\207\345\217\212\345\211\257\346\234\254/README.md" "b/part-1/3.2-\345\237\272\346\234\254\346\246\202\345\277\2652\350\212\202\347\202\271,\351\233\206\347\276\244,\345\210\206\347\211\207\345\217\212\345\211\257\346\234\254/README.md" index 503e3da..3e2947a 100644 --- "a/part-1/3.2-\345\237\272\346\234\254\346\246\202\345\277\2652\350\212\202\347\202\271,\351\233\206\347\276\244,\345\210\206\347\211\207\345\217\212\345\211\257\346\234\254/README.md" +++ "b/part-1/3.2-\345\237\272\346\234\254\346\246\202\345\277\2652\350\212\202\347\202\271,\351\233\206\347\276\244,\345\210\206\347\211\207\345\217\212\345\211\257\346\234\254/README.md" @@ -18,7 +18,7 @@ GET /_cluster/health/kibana_sample_data_flights?level=shards The cluster state API allows access to metadata representing the state of the whole cluster. This includes information such as GET /_cluster/state -//cluster get settings +#cluster get settings GET /_cluster/settings GET /_cluster/settings?include_defaults=true diff --git "a/part-1/3.3-\346\226\207\346\241\243\347\232\204\345\237\272\346\234\254CRUD\344\270\216\346\211\271\351\207\217\346\223\215\344\275\234/README.md" "b/part-1/3.3-\346\226\207\346\241\243\347\232\204\345\237\272\346\234\254CRUD\344\270\216\346\211\271\351\207\217\346\223\215\344\275\234/README.md" index 9be6987..8aded4c 100644 --- "a/part-1/3.3-\346\226\207\346\241\243\347\232\204\345\237\272\346\234\254CRUD\344\270\216\346\211\271\351\207\217\346\223\215\344\275\234/README.md" +++ "b/part-1/3.3-\346\226\207\346\241\243\347\232\204\345\237\272\346\234\254CRUD\344\270\216\346\211\271\351\207\217\346\223\215\344\275\234/README.md" @@ -1,8 +1,8 @@ # 文档的基本 CRUD 与批量操作 ## 课程Demo ``` -//############Create Document############ -//create document. 自动生成 _id +############Create Document############ +#create document. 自动生成 _id POST users/_doc { "user" : "Mike", @@ -10,7 +10,7 @@ POST users/_doc "message" : "trying out Kibana" } -//create document. 指定Id。如果id已经存在,报错 +#create document. 指定Id。如果id已经存在,报错 PUT users/_doc/1?op_type=create { "user" : "Jack", @@ -18,7 +18,7 @@ PUT users/_doc/1?op_type=create "message" : "trying out Elasticsearch" } -//create document. 指定 ID 如果已经存在,就报错 +#create document. 指定 ID 如果已经存在,就报错 PUT users/_create/1 { "user" : "Jack", @@ -26,13 +26,13 @@ PUT users/_create/1 "message" : "trying out Elasticsearch" } -//### Get Document by ID -//Get the document by ID +### Get Document by ID +#Get the document by ID GET users/_doc/1 -//### Index & Update -//Update 指定 ID (先删除,在写入) +### Index & Update +#Update 指定 ID (先删除,在写入) GET users/_doc/1 PUT users/_doc/1 @@ -42,8 +42,8 @@ PUT users/_doc/1 } -//GET users/_doc/1 -// 在原文档上增加字段 +#GET users/_doc/1 +#在原文档上增加字段 POST users/_update/1/ { "doc":{ @@ -54,15 +54,15 @@ POST users/_update/1/ -//### Delete by Id -//删除文档 +### Delete by Id +# 删除文档 DELETE users/_doc/1 -//### Bulk 操作 -//执行两次,查看每次的结果 +### Bulk 操作 +#执行两次,查看每次的结果 -//执行第1次 +#执行第1次 POST _bulk { "index" : { "_index" : "test", "_id" : "1" } } { "field1" : "value1" } @@ -73,7 +73,7 @@ POST _bulk { "doc" : {"field2" : "value2"} } -//执行第2次 +#执行第2次 POST _bulk { "index" : { "_index" : "test", "_id" : "1" } } { "field1" : "value1" } @@ -83,7 +83,7 @@ POST _bulk { "update" : {"_id" : "1", "_index" : "test"} } { "doc" : {"field2" : "value2"} } -//### mget 操作 +### mget 操作 GET /_mget { "docs" : [ @@ -99,7 +99,7 @@ GET /_mget } -//URI中指定index +#URI中指定index GET /test/_mget { "docs" : [ @@ -139,7 +139,7 @@ GET /_mget ] } -//### msearch 操作 +### msearch 操作 POST kibana_sample_data_ecommerce/_msearch {} {"query" : {"match_all" : {}},"size":1} @@ -147,8 +147,8 @@ POST kibana_sample_data_ecommerce/_msearch {"query" : {"match_all" : {}},"size":2} -//### 清除测试数据 -//清除数据 +### 清除测试数据 +#清除数据 DELETE users DELETE test DELETE test2 diff --git "a/part-1/3.5-\351\200\232\350\277\207\345\210\206\346\236\220\345\231\250\350\277\233\350\241\214\345\210\206\350\257\215/README.md" "b/part-1/3.5-\351\200\232\350\277\207\345\210\206\346\236\220\345\231\250\350\277\233\350\241\214\345\210\206\350\257\215/README.md" index 05681ca..f3dec7e 100644 --- "a/part-1/3.5-\351\200\232\350\277\207\345\210\206\346\236\220\345\231\250\350\277\233\350\241\214\345\210\206\350\257\215/README.md" +++ "b/part-1/3.5-\351\200\232\350\277\207\345\210\206\346\236\220\345\231\250\350\277\233\350\241\214\345\210\206\350\257\215/README.md" @@ -2,23 +2,23 @@ ## 课程Demo ``` -//Simple Analyzer – 按照非字母切分(符号被过滤),小写处理 -//Stop Analyzer – 小写处理,停用词过滤(the,a,is) -//Whitespace Analyzer – 按照空格切分,不转小写 -//Keyword Analyzer – 不分词,直接将输入当作输出 -//Patter Analyzer – 正则表达式,默认 \W+ (非字符分隔) -//Language – 提供了30多种常见语言的分词器 -//2 running Quick brown-foxes leap over lazy dogs in the summer evening - -//查看不同的analyzer的效果 -//standard +#Simple Analyzer – 按照非字母切分(符号被过滤),小写处理 +#Stop Analyzer – 小写处理,停用词过滤(the,a,is) +#Whitespace Analyzer – 按照空格切分,不转小写 +#Keyword Analyzer – 不分词,直接将输入当作输出 +#Patter Analyzer – 正则表达式,默认 \W+ (非字符分隔) +#Language – 提供了30多种常见语言的分词器 +#2 running Quick brown-foxes leap over lazy dogs in the summer evening + +#查看不同的analyzer的效果 +#standard GET _analyze { "analyzer": "standard", "text": "2 running Quick brown-foxes leap over lazy dogs in the summer evening." } -//simpe +#simpe GET _analyze { "analyzer": "simple", @@ -33,14 +33,14 @@ GET _analyze } -//stop +#stop GET _analyze { "analyzer": "whitespace", "text": "2 running Quick brown-foxes leap over lazy dogs in the summer evening." } -//keyword +#keyword GET _analyze { "analyzer": "keyword", @@ -54,7 +54,7 @@ GET _analyze } -//english +#english GET _analyze { "analyzer": "english", diff --git "a/part-1/3.6-SearchAPI\346\246\202\350\247\210/README.md" "b/part-1/3.6-SearchAPI\346\246\202\350\247\210/README.md" index fe6d4f7..1175943 100644 --- "a/part-1/3.6-SearchAPI\346\246\202\350\247\210/README.md" +++ "b/part-1/3.6-SearchAPI\346\246\202\350\247\210/README.md" @@ -5,13 +5,13 @@ 具体参考“2.2节-Kibana的安装与界面快速浏览”一节教程 ``` -//URI Query +#URI Query GET kibana_sample_data_ecommerce/_search?q=customer_first_name:Eddie GET kibana*/_search?q=customer_first_name:Eddie GET /_all/_search?q=customer_first_name:Eddie -//REQUEST Body +#REQUEST Body POST kibana_sample_data_ecommerce/_search { "profile": true, diff --git "a/part-1/3.7-URISearch\350\257\246\350\247\243/README.md" "b/part-1/3.7-URISearch\350\257\246\350\247\243/README.md" index 7e9fc2b..889fa9f 100644 --- "a/part-1/3.7-URISearch\350\257\246\350\247\243/README.md" +++ "b/part-1/3.7-URISearch\350\257\246\350\247\243/README.md" @@ -2,91 +2,87 @@ ## 课程Demo ``` -//基本查询 +#基本查询 GET /movies/_search?q=2012&df=title&sort=year:desc&from=0&size=10&timeout=1s -//带profile +#带profile GET /movies/_search?q=2012&df=title { "profile":"true" } -//泛查询,正对_all,所有字段 +#泛查询,正对_all,所有字段 GET /movies/_search?q=2012 { "profile":"true" } -//指定字段 +#指定字段 GET /movies/_search?q=title:2012&sort=year:desc&from=0&size=10&timeout=1s { "profile":"true" } -// 查找美丽心灵, Mind为泛查询 +# 查找美丽心灵, Mind为泛查询 GET /movies/_search?q=title:Beautiful Mind { "profile":"true" } -// 泛查询 +# 泛查询 GET /movies/_search?q=title:2012 { "profile":"true" } -//使用引号,Phrase查询 +#使用引号,Phrase查询 GET /movies/_search?q=title:"Beautiful Mind" { "profile":"true" } -//分组,Bool查询 +#分组,Bool查询 GET /movies/_search?q=title:(Beautiful Mind) { "profile":"true" } -//布尔操作符 - -// 查找美丽心灵 +#布尔操作符 +# 查找美丽心灵 GET /movies/_search?q=title:(Beautiful AND Mind) { "profile":"true" } -// 查找美丽心灵 +# 查找美丽心灵 GET /movies/_search?q=title:(Beautiful NOT Mind) { "profile":"true" } -// 查找美丽心灵 +# 查找美丽心灵 GET /movies/_search?q=title:(Beautiful %2BMind) { "profile":"true" } -//范围查询 ,区间写法 / 数学写法 -GET /movies/_search?q=title:beautiful&year:[1980 & 2018} +#范围查询 ,区间写法 +GET /movies/_search?q=title:beautiful AND year:[2002 TO 2018%7D { "profile":"true" } -//通配符查询 +#通配符查询 GET /movies/_search?q=title:b* { "profile":"true" } -//正则表达式 - - //模糊匹配&近似度匹配 GET /movies/_search?q=title:beautifl~1 { diff --git "a/part-1/3.8-RequestBody\344\270\216QueryDSL\347\256\200\344\273\213/README.md" "b/part-1/3.8-RequestBody\344\270\216QueryDSL\347\256\200\344\273\213/README.md" index 5f8ca2a..8820d15 100644 --- "a/part-1/3.8-RequestBody\344\270\216QueryDSL\347\256\200\344\273\213/README.md" +++ "b/part-1/3.8-RequestBody\344\270\216QueryDSL\347\256\200\344\273\213/README.md" @@ -4,8 +4,8 @@ - 需导入Movie测试数据,具体参考“2.4-Logstash安装与导入数据” ``` -//ignore_unavailable=true,可以忽略尝试访问不存在的索引“404_idx”导致的报错 -//查询movies分页 +#ignore_unavailable=true,可以忽略尝试访问不存在的索引“404_idx”导致的报错 +#查询movies分页 POST /movies,404_idx/_search?ignore_unavailable=true { "profile": true, @@ -24,7 +24,7 @@ POST /kibana_sample_data_ecommerce/_search } -//对日期排序 +#对日期排序 POST kibana_sample_data_ecommerce/_search { "sort":[{"order_date":"desc"}], @@ -34,7 +34,7 @@ POST kibana_sample_data_ecommerce/_search } -//source filtering +#source filtering POST kibana_sample_data_ecommerce/_search { "_source":["order_date"], @@ -44,7 +44,7 @@ POST kibana_sample_data_ecommerce/_search } -//脚本字段 +#脚本字段 GET kibana_sample_data_ecommerce/_search { "script_fields": { diff --git "a/part-1/3.9-QueryString&SimpleQueryString\346\237\245\350\257\242/README.md" "b/part-1/3.9-QueryString&SimpleQueryString\346\237\245\350\257\242/README.md" index e6ea8f7..daf20e0 100644 --- "a/part-1/3.9-QueryString&SimpleQueryString\346\237\245\350\257\242/README.md" +++ "b/part-1/3.9-QueryString&SimpleQueryString\346\237\245\350\257\242/README.md" @@ -39,7 +39,7 @@ POST users/_search } -//Simple Query 默认的operator是 Or +#Simple Query 默认的operator是 Or POST users/_search { "query": { @@ -75,7 +75,7 @@ GET /movies/_search } -// 多fields +# 多fields GET /movies/_search { "profile": true, diff --git a/part-1/FAQ.md b/part-1/FAQ.md new file mode 100644 index 0000000..e69de29 diff --git "a/part-2/4.1-\345\237\272\344\272\216\350\257\215\351\241\271\345\222\214\345\237\272\344\272\216\345\205\250\346\226\207\347\232\204\346\220\234\347\264\242/README.md" "b/part-2/4.1-\345\237\272\344\272\216\350\257\215\351\241\271\345\222\214\345\237\272\344\272\216\345\205\250\346\226\207\347\232\204\346\220\234\347\264\242/README.md" new file mode 100644 index 0000000..24d2724 --- /dev/null +++ "b/part-2/4.1-\345\237\272\344\272\216\350\257\215\351\241\271\345\222\214\345\237\272\344\272\216\345\205\250\346\226\207\347\232\204\346\220\234\347\264\242/README.md" @@ -0,0 +1,134 @@ +# 基于词项和基于全文的搜索 + +## 课程demo + +``` +DELETE products +PUT products +{ + "settings": { + "number_of_shards": 1 + } +} + + +POST /products/_bulk +{ "index": { "_id": 1 }} +{ "productID" : "XHDK-A-1293-#fJ3","desc":"iPhone" } +{ "index": { "_id": 2 }} +{ "productID" : "KDKE-B-9947-#kL5","desc":"iPad" } +{ "index": { "_id": 3 }} +{ "productID" : "JODL-X-1937-#pV7","desc":"MBP" } + +GET /products + +POST /products/_search +{ + "query": { + "term": { + "desc": { + //"value": "iPhone" + "value":"iphone" + } + } + } +} + +POST /products/_search +{ + "query": { + "term": { + "desc.keyword": { + //"value": "iPhone" + //"value":"iphone" + } + } + } +} + + +POST /products/_search +{ + "query": { + "term": { + "productID": { + "value": "XHDK-A-1293-#fJ3" + } + } + } +} + +POST /products/_search +{ + //"explain": true, + "query": { + "term": { + "productID.keyword": { + "value": "XHDK-A-1293-#fJ3" + } + } + } +} + + + + +POST /products/_search +{ + "explain": true, + "query": { + "constant_score": { + "filter": { + "term": { + "productID.keyword": "XHDK-A-1293-#fJ3" + } + } + + } + } +} + + +#设置 position_increment_gap +DELETE groups +PUT groups +{ + "mappings": { + "properties": { + "names":{ + "type": "text", + "position_increment_gap": 0 + } + } + } +} + +GET groups/_mapping + +POST groups/_doc +{ + "names": [ "John Water", "Water Smith"] +} + +POST groups/_search +{ + "query": { + "match_phrase": { + "names": { + "query": "Water Water", + "slop": 100 + } + } + } +} + + +POST groups/_search +{ + "query": { + "match_phrase": { + "names": "Water Smith" + } + } +} +``` diff --git "a/part-2/4.10-\347\273\274\345\220\210\346\216\222\345\272\217\357\274\232Function Score Query \344\274\230\345\214\226\347\256\227\345\210\206/README.md" "b/part-2/4.10-\347\273\274\345\220\210\346\216\222\345\272\217\357\274\232Function Score Query \344\274\230\345\214\226\347\256\227\345\210\206/README.md" new file mode 100644 index 0000000..c15f3c4 --- /dev/null +++ "b/part-2/4.10-\347\273\274\345\220\210\346\216\222\345\272\217\357\274\232Function Score Query \344\274\230\345\214\226\347\256\227\345\210\206/README.md" @@ -0,0 +1,114 @@ +# 综合排序:Function Score Query 优化算分 +## 课程Demo +``` +DELETE blogs +PUT /blogs/_doc/1 +{ + "title": "About popularity", + "content": "In this post we will talk about...", + "votes": 0 +} + +PUT /blogs/_doc/2 +{ + "title": "About popularity", + "content": "In this post we will talk about...", + "votes": 100 +} + +PUT /blogs/_doc/3 +{ + "title": "About popularity", + "content": "In this post we will talk about...", + "votes": 1000000 +} + + +POST /blogs/_search +{ + "query": { + "function_score": { + "query": { + "multi_match": { + "query": "popularity", + "fields": [ "title", "content" ] + } + }, + "field_value_factor": { + "field": "votes" + } + } + } +} + +POST /blogs/_search +{ + "query": { + "function_score": { + "query": { + "multi_match": { + "query": "popularity", + "fields": [ "title", "content" ] + } + }, + "field_value_factor": { + "field": "votes", + "modifier": "log1p" + } + } + } +} + + +POST /blogs/_search +{ + "query": { + "function_score": { + "query": { + "multi_match": { + "query": "popularity", + "fields": [ "title", "content" ] + } + }, + "field_value_factor": { + "field": "votes", + "modifier": "log1p" , + "factor": 0.1 + } + } + } +} + + +POST /blogs/_search +{ + "query": { + "function_score": { + "query": { + "multi_match": { + "query": "popularity", + "fields": [ "title", "content" ] + } + }, + "field_value_factor": { + "field": "votes", + "modifier": "log1p" , + "factor": 0.1 + }, + "boost_mode": "sum", + "max_boost": 3 + } + } +} + +POST /blogs/_search +{ + "query": { + "function_score": { + "random_score": { + "seed": 911119 + } + } + } +} +``` diff --git a/part-2/4.11-Term&PhraseSuggester/README.md b/part-2/4.11-Term&PhraseSuggester/README.md new file mode 100644 index 0000000..9e8c8de --- /dev/null +++ b/part-2/4.11-Term&PhraseSuggester/README.md @@ -0,0 +1,154 @@ +# Term & Phrase Suggester +## 课程Demo +``` +DELETE articles +PUT articles +{ + "mappings": { + "properties": { + "title_completion":{ + "type": "completion" + } + } + } +} + +POST articles/_bulk +{ "index" : { } } +{ "title_completion": "lucene is very cool"} +{ "index" : { } } +{ "title_completion": "Elasticsearch builds on top of lucene"} +{ "index" : { } } +{ "title_completion": "Elasticsearch rocks"} +{ "index" : { } } +{ "title_completion": "elastic is the company behind ELK stack"} +{ "index" : { } } +{ "title_completion": "Elk stack rocks"} +{ "index" : {} } + + +POST articles/_search?pretty +{ + "size": 0, + "suggest": { + "article-suggester": { + "prefix": "elk ", + "completion": { + "field": "title_completion" + } + } + } +} + +DELETE articles + +POST articles/_bulk +{ "index" : { } } +{ "body": "lucene is very cool"} +{ "index" : { } } +{ "body": "Elasticsearch builds on top of lucene"} +{ "index" : { } } +{ "body": "Elasticsearch rocks"} +{ "index" : { } } +{ "body": "elastic is the company behind ELK stack"} +{ "index" : { } } +{ "body": "Elk stack rocks"} +{ "index" : {} } +{ "body": "elasticsearch is rock solid"} + + +POST _analyze +{ + "analyzer": "standard", + "text": ["Elk stack rocks rock"] +} + +POST /articles/_search +{ + "size": 1, + "query": { + "match": { + "body": "lucen rock" + } + }, + "suggest": { + "term-suggestion": { + "text": "lucen rock", + "term": { + "suggest_mode": "missing", + "field": "body" + } + } + } +} + + +POST /articles/_search +{ + + "suggest": { + "term-suggestion": { + "text": "lucen rock", + "term": { + "suggest_mode": "popular", + "field": "body" + } + } + } +} + + +POST /articles/_search +{ + + "suggest": { + "term-suggestion": { + "text": "lucen rock", + "term": { + "suggest_mode": "always", + "field": "body", + } + } + } +} + + +POST /articles/_search +{ + + "suggest": { + "term-suggestion": { + "text": "lucen hocks", + "term": { + "suggest_mode": "always", + "field": "body", + "prefix_length":0, + "sort": "frequency" + } + } + } +} + + +POST /articles/_search +{ + "suggest": { + "my-suggestion": { + "text": "lucne and elasticsear rock hello world ", + "phrase": { + "field": "body", + "max_errors":2, + "confidence":0, + "direct_generator":[{ + "field":"body", + "suggest_mode":"always" + }], + "highlight": { + "pre_tag": "", + "post_tag": "" + } + } + } + } +} +``` diff --git "a/part-2/4.12-\350\207\252\345\212\250\350\241\245\345\205\250\344\270\216\345\237\272\344\272\216\344\270\212\344\270\213\346\226\207\347\232\204\346\217\220\347\244\272/README.md" "b/part-2/4.12-\350\207\252\345\212\250\350\241\245\345\205\250\344\270\216\345\237\272\344\272\216\344\270\212\344\270\213\346\226\207\347\232\204\346\217\220\347\244\272/README.md" new file mode 100644 index 0000000..114d80f --- /dev/null +++ "b/part-2/4.12-\350\207\252\345\212\250\350\241\245\345\205\250\344\270\216\345\237\272\344\272\216\344\270\212\344\270\213\346\226\207\347\232\204\346\217\220\347\244\272/README.md" @@ -0,0 +1,96 @@ +# 自动补全与基于上下文的提示 +## 课程demo +``` +DELETE articles +PUT articles +{ + "mappings": { + "properties": { + "title_completion":{ + "type": "completion" + } + } + } +} + +POST articles/_bulk +{ "index" : { } } +{ "title_completion": "lucene is very cool"} +{ "index" : { } } +{ "title_completion": "Elasticsearch builds on top of lucene"} +{ "index" : { } } +{ "title_completion": "Elasticsearch rocks"} +{ "index" : { } } +{ "title_completion": "elastic is the company behind ELK stack"} +{ "index" : { } } +{ "title_completion": "Elk stack rocks"} +{ "index" : {} } + + +POST articles/_search?pretty +{ + "size": 0, + "suggest": { + "article-suggester": { + "prefix": "elk ", + "completion": { + "field": "title_completion" + } + } + } +} + + +DELETE comments +PUT comments +PUT comments/_mapping +{ + "properties": { + "comment_autocomplete":{ + "type": "completion", + "contexts":[{ + "type":"category", + "name":"comment_category" + }] + } + } +} + +POST comments/_doc +{ + "comment":"I love the star war movies", + "comment_autocomplete":{ + "input":["star wars"], + "contexts":{ + "comment_category":"movies" + } + } +} + +POST comments/_doc +{ + "comment":"Where can I find a Starbucks", + "comment_autocomplete":{ + "input":["starbucks"], + "contexts":{ + "comment_category":"coffee" + } + } +} + + +POST comments/_search +{ + "suggest": { + "MY_SUGGESTION": { + "prefix": "sta", + "completion":{ + "field":"comment_autocomplete", + "contexts":{ + "comment_category":"coffee" + } + } + } + } +} +``` diff --git "a/part-2/4.13-\350\267\250\351\233\206\347\276\244\346\220\234\347\264\242/README.md" "b/part-2/4.13-\350\267\250\351\233\206\347\276\244\346\220\234\347\264\242/README.md" new file mode 100644 index 0000000..7a4700a --- /dev/null +++ "b/part-2/4.13-\350\267\250\351\233\206\347\276\244\346\220\234\347\264\242/README.md" @@ -0,0 +1,79 @@ +# 跨集群搜索 +## 课程demo +``` +//启动3个集群 + +bin/elasticsearch -E node.name=cluster0node -E cluster.name=cluster0 -E path.data=cluster0_data -E discovery.type=single-node -E http.port=9200 -E transport.port=9300 +bin/elasticsearch -E node.name=cluster1node -E cluster.name=cluster1 -E path.data=cluster1_data -E discovery.type=single-node -E http.port=9201 -E transport.port=9301 +bin/elasticsearch -E node.name=cluster2node -E cluster.name=cluster2 -E path.data=cluster2_data -E discovery.type=single-node -E http.port=9202 -E transport.port=9302 + + +//在每个集群上设置动态的设置 +PUT _cluster/settings +{ + "persistent": { + "cluster": { + "remote": { + "cluster0": { + "seeds": [ + "127.0.0.1:9300" + ], + "transport.ping_schedule": "30s" + }, + "cluster1": { + "seeds": [ + "127.0.0.1:9301" + ], + "transport.compress": true, + "skip_unavailable": true + }, + "cluster2": { + "seeds": [ + "127.0.0.1:9302" + ] + } + } + } + } +} + +#cURL +curl -XPUT "http://localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d' +{"persistent":{"cluster":{"remote":{"cluster0":{"seeds":["127.0.0.1:9300"],"transport.ping_schedule":"30s"},"cluster1":{"seeds":["127.0.0.1:9301"],"transport.compress":true,"skip_unavailable":true},"cluster2":{"seeds":["127.0.0.1:9302"]}}}}}' + +curl -XPUT "http://localhost:9201/_cluster/settings" -H 'Content-Type: application/json' -d' +{"persistent":{"cluster":{"remote":{"cluster0":{"seeds":["127.0.0.1:9300"],"transport.ping_schedule":"30s"},"cluster1":{"seeds":["127.0.0.1:9301"],"transport.compress":true,"skip_unavailable":true},"cluster2":{"seeds":["127.0.0.1:9302"]}}}}}' + +curl -XPUT "http://localhost:9202/_cluster/settings" -H 'Content-Type: application/json' -d' +{"persistent":{"cluster":{"remote":{"cluster0":{"seeds":["127.0.0.1:9300"],"transport.ping_schedule":"30s"},"cluster1":{"seeds":["127.0.0.1:9301"],"transport.compress":true,"skip_unavailable":true},"cluster2":{"seeds":["127.0.0.1:9302"]}}}}}' + + +#创建测试数据 +curl -XPOST "http://localhost:9200/users/_doc" -H 'Content-Type: application/json' -d' +{"name":"user1","age":10}' + +curl -XPOST "http://localhost:9201/users/_doc" -H 'Content-Type: application/json' -d' +{"name":"user2","age":20}' + +curl -XPOST "http://localhost:9202/users/_doc" -H 'Content-Type: application/json' -d' +{"name":"user3","age":30}' + + +#查询 +GET /users,cluster1:users,cluster2:users/_search +{ + "query": { + "range": { + "age": { + "gte": 20, + "lte": 40 + } + } + } +} + +``` + +# 相关阅读 + +- 在Kibana中使用Cross data search https://kelonsoftware.com/cross-cluster-search-kibana/ diff --git "a/part-2/4.2-\347\273\223\346\236\204\345\214\226\346\220\234\347\264\242/README.md" "b/part-2/4.2-\347\273\223\346\236\204\345\214\226\346\220\234\347\264\242/README.md" new file mode 100644 index 0000000..1d11dd0 --- /dev/null +++ "b/part-2/4.2-\347\273\223\346\236\204\345\214\226\346\220\234\347\264\242/README.md" @@ -0,0 +1,306 @@ +# 结构化搜索 + +## 课程demo +``` + +#结构花搜索,精确匹配 +DELETE products +POST /products/_bulk +{ "index": { "_id": 1 }} +{ "price" : 10,"avaliable":true,"date":"2018-01-01", "productID" : "XHDK-A-1293-#fJ3" } +{ "index": { "_id": 2 }} +{ "price" : 20,"avaliable":true,"date":"2019-01-01", "productID" : "KDKE-B-9947-#kL5" } +{ "index": { "_id": 3 }} +{ "price" : 30,"avaliable":true, "productID" : "JODL-X-1937-#pV7" } +{ "index": { "_id": 4 }} +{ "price" : 30,"avaliable":false, "productID" : "QQPX-R-3956-#aD8" } + +GET products/_mapping + + + +#对布尔值 match 查询,有算分 +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "term": { + "avaliable": true + } + } +} + + + +#对布尔值,通过constant score 转成 filtering,没有算分 +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "constant_score": { + "filter": { + "term": { + "avaliable": true + } + } + } + } +} + + +#数字类型 Term +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "term": { + "price": 30 + } + } +} + +#数字类型 terms +POST products/_search +{ + "query": { + "constant_score": { + "filter": { + "terms": { + "price": [ + "20", + "30" + ] + } + } + } + } +} + +#数字 Range 查询 +GET products/_search +{ + "query" : { + "constant_score" : { + "filter" : { + "range" : { + "price" : { + "gte" : 20, + "lte" : 30 + } + } + } + } + } +} + + +# 日期 range +POST products/_search +{ + "query" : { + "constant_score" : { + "filter" : { + "range" : { + "date" : { + "gte" : "now-1y" + } + } + } + } + } +} + + + +#Null 数值 +POST products/_search +{ + "query": { + "constant_score": { + "filter": { + "exists": { + "field": "date" + } + } + } + } +} + +#处理多值字段 +POST /movies/_bulk +{ "index": { "_id": 1 }} +{ "title" : "Father of the Bridge Part II","year":1995, "genre":"Comedy"} +{ "index": { "_id": 2 }} +{ "title" : "Dave","year":1993,"genre":["Comedy","Romance"] } + + +#处理多值字段,term 查询是包含,而不是等于 +POST movies/_search +{ + "query": { + "constant_score": { + "filter": { + "term": { + "genre.keyword": "Comedy" + } + } + } + } +} + + +#字符类型 terms +POST products/_search +{ + "query": { + "constant_score": { + "filter": { + "terms": { + "productID.keyword": [ + "QQPX-R-3956-#aD8", + "JODL-X-1937-#pV7" + ] + } + } + } + } +} + + + +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "match": { + "price": 30 + } + } +} + + +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "term": { + "date": "2019-01-01" + } + } +} + +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "match": { + "date": "2019-01-01" + } + } +} + + + + +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "constant_score": { + "filter": { + "term": { + "productID.keyword": "XHDK-A-1293-#fJ3" + } + } + } + } +} + +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "term": { + "productID.keyword": "XHDK-A-1293-#fJ3" + } + } +} + +#对布尔数值 +POST products/_search +{ + "query": { + "constant_score": { + "filter": { + "term": { + "avaliable": "false" + } + } + } + } +} + +POST products/_search +{ + "query": { + "term": { + "avaliable": { + "value": "false" + } + } + } +} + +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "term": { + "price": { + "value": "20" + } + } + } +} + +POST products/_search +{ + "profile": "true", + "explain": true, + "query": { + "match": { + "price": "20" + } + } + } +} + + +POST products/_search +{ + "query": { + "constant_score": { + "filter": { + "bool": { + "must_not": { + "exists": { + "field": "date" + } + } + } + } + } + } +} + + +``` diff --git "a/part-2/4.3-\346\220\234\347\264\242\347\232\204\347\233\270\345\205\263\346\200\247\347\256\227\345\210\206/README.md" "b/part-2/4.3-\346\220\234\347\264\242\347\232\204\347\233\270\345\205\263\346\200\247\347\256\227\345\210\206/README.md" new file mode 100644 index 0000000..037604a --- /dev/null +++ "b/part-2/4.3-\346\220\234\347\264\242\347\232\204\347\233\270\345\205\263\346\200\247\347\256\227\345\210\206/README.md" @@ -0,0 +1,84 @@ +# 搜索的相关性算分 + +## 课程demo +``` + + +PUT testscore +{ + "settings": { + "number_of_shards": 1 + }, + "mappings": { + "properties": { + "content": { + "type": "text" + } + } + } +} + + +PUT testscore/_bulk +{ "index": { "_id": 1 }} +{ "content":"we use Elasticsearch to power the search" } +{ "index": { "_id": 2 }} +{ "content":"we like elasticsearch" } +{ "index": { "_id": 3 }} +{ "content":"The scoring of documents is caculated by the scoring formula" } +{ "index": { "_id": 4 }} +{ "content":"you know, for search" } + + + +POST /testscore/_search +{ + //"explain": true, + "query": { + "match": { + "content":"you" + //"content": "elasticsearch" + //"content":"the" + //"content": "the elasticsearch" + } + } +} + +POST testscore/_search +{ + "query": { + "boosting" : { + "positive" : { + "term" : { + "content" : "elasticsearch" + } + }, + "negative" : { + "term" : { + "content" : "like" + } + }, + "negative_boost" : 0.2 + } + } +} + + +POST tmdb/_search +{ + "_source": ["title","overview"], + "query": { + "more_like_this": { + "fields": [ + "title^10","overview" + ], + "like": [{"_id":"14191"}], + "min_term_freq": 1, + "max_query_terms": 12 + } + } +} + + + +``` diff --git "a/part-2/4.4-Query&Filtering\345\256\236\347\216\260\345\244\232\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242/README.md" "b/part-2/4.4-Query&Filtering\345\256\236\347\216\260\345\244\232\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242/README.md" new file mode 100644 index 0000000..2ebf323 --- /dev/null +++ "b/part-2/4.4-Query&Filtering\345\256\236\347\216\260\345\244\232\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242/README.md" @@ -0,0 +1,302 @@ +# Query & Filtering 与多字符串多字段查询 + +## 课程demo +``` +POST /products/_bulk +{ "index": { "_id": 1 }} +{ "price" : 10,"avaliable":true,"date":"2018-01-01", "productID" : "XHDK-A-1293-#fJ3" } +{ "index": { "_id": 2 }} +{ "price" : 20,"avaliable":true,"date":"2019-01-01", "productID" : "KDKE-B-9947-#kL5" } +{ "index": { "_id": 3 }} +{ "price" : 30,"avaliable":true, "productID" : "JODL-X-1937-#pV7" } +{ "index": { "_id": 4 }} +{ "price" : 30,"avaliable":false, "productID" : "QQPX-R-3956-#aD8" } + + + +#基本语法 +POST /products/_search +{ + "query": { + "bool" : { + "must" : { + "term" : { "price" : "30" } + }, + "filter": { + "term" : { "avaliable" : "true" } + }, + "must_not" : { + "range" : { + "price" : { "lte" : 10 } + } + }, + "should" : [ + { "term" : { "productID.keyword" : "JODL-X-1937-#pV7" } }, + { "term" : { "productID.keyword" : "XHDK-A-1293-#fJ3" } } + ], + "minimum_should_match" :1 + } + } +} + +#改变数据模型,增加字段。解决数组包含而不是精确匹配的问题 +POST /newmovies/_bulk +{ "index": { "_id": 1 }} +{ "title" : "Father of the Bridge Part II","year":1995, "genre":"Comedy","genre_count":1 } +{ "index": { "_id": 2 }} +{ "title" : "Dave","year":1993,"genre":["Comedy","Romance"],"genre_count":2 } + +#must,有算分 +POST /newmovies/_search +{ + "query": { + "bool": { + "must": [ + {"term": {"genre.keyword": {"value": "Comedy"}}}, + {"term": {"genre_count": {"value": 1}}} + + ] + } + } +} + +#Filter。不参与算分,结果的score是0 +POST /newmovies/_search +{ + "query": { + "bool": { + "filter": [ + {"term": {"genre.keyword": {"value": "Comedy"}}}, + {"term": {"genre_count": {"value": 1}}} + ] + + } + } +} + + +#Filtering Context +POST _search +{ + "query": { + "bool" : { + + "filter": { + "term" : { "avaliable" : "true" } + }, + "must_not" : { + "range" : { + "price" : { "lte" : 10 } + } + } + } + } +} + + +#Query Context +POST /products/_bulk +{ "index": { "_id": 1 }} +{ "price" : 10,"avaliable":true,"date":"2018-01-01", "productID" : "XHDK-A-1293-#fJ3" } +{ "index": { "_id": 2 }} +{ "price" : 20,"avaliable":true,"date":"2019-01-01", "productID" : "KDKE-B-9947-#kL5" } +{ "index": { "_id": 3 }} +{ "price" : 30,"avaliable":true, "productID" : "JODL-X-1937-#pV7" } +{ "index": { "_id": 4 }} +{ "price" : 30,"avaliable":false, "productID" : "QQPX-R-3956-#aD8" } + + +POST /products/_search +{ + "query": { + "bool": { + "should": [ + { + "term": { + "productID.keyword": { + "value": "JODL-X-1937-#pV7"}} + }, + {"term": {"avaliable": {"value": true}} + } + ] + } + } +} + + +#嵌套,实现了 should not 逻辑 +POST /products/_search +{ + "query": { + "bool": { + "must": { + "term": { + "price": "30" + } + }, + "should": [ + { + "bool": { + "must_not": { + "term": { + "avaliable": "false" + } + } + } + } + ], + "minimum_should_match": 1 + } + } +} + + +#Controll the Precision +POST _search +{ + "query": { + "bool" : { + "must" : { + "term" : { "price" : "30" } + }, + "filter": { + "term" : { "avaliable" : "true" } + }, + "must_not" : { + "range" : { + "price" : { "lte" : 10 } + } + }, + "should" : [ + { "term" : { "productID.keyword" : "JODL-X-1937-#pV7" } }, + { "term" : { "productID.keyword" : "XHDK-A-1293-#fJ3" } } + ], + "minimum_should_match" :2 + } + } +} + + + +POST /animals/_search +{ + "query": { + "bool": { + "should": [ + { "term": { "text": "brown" }}, + { "term": { "text": "red" }}, + { "term": { "text": "quick" }}, + { "term": { "text": "dog" }} + ] + } + } +} + +POST /animals/_search +{ + "query": { + "bool": { + "should": [ + { "term": { "text": "quick" }}, + { "term": { "text": "dog" }}, + { + "bool":{ + "should":[ + { "term": { "text": "brown" }}, + { "term": { "text": "brown" }}, + ] + } + + } + ] + } + } +} + + +DELETE blogs +POST /blogs/_bulk +{ "index": { "_id": 1 }} +{"title":"Apple iPad", "content":"Apple iPad,Apple iPad" } +{ "index": { "_id": 2 }} +{"title":"Apple iPad,Apple iPad", "content":"Apple iPad" } + + +POST blogs/_search +{ + "query": { + "bool": { + "should": [ + {"match": { + "title": { + "query": "apple,ipad", + "boost": 1.1 + } + }}, + + {"match": { + "content": { + "query": "apple,ipad", + "boost": + } + }} + ] + } + } +} + +DELETE news +POST /news/_bulk +{ "index": { "_id": 1 }} +{ "content":"Apple Mac" } +{ "index": { "_id": 2 }} +{ "content":"Apple iPad" } +{ "index": { "_id": 3 }} +{ "content":"Apple employee like Apple Pie and Apple Juice" } + + +POST news/_search +{ + "query": { + "bool": { + "must": { + "match":{"content":"apple"} + } + } + } +} + +POST news/_search +{ + "query": { + "bool": { + "must": { + "match":{"content":"apple"} + }, + "must_not": { + "match":{"content":"pie"} + } + } + } +} + +POST news/_search +{ + "query": { + "boosting": { + "positive": { + "match": { + "content": "apple" + } + }, + "negative": { + "match": { + "content": "pie" + } + }, + "negative_boost": 0.5 + } + } +} + +``` diff --git "a/part-2/4.5-\345\215\225\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242-DisMaxQuery/README.md" "b/part-2/4.5-\345\215\225\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242-DisMaxQuery/README.md" new file mode 100644 index 0000000..6a4bfeb --- /dev/null +++ "b/part-2/4.5-\345\215\225\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242-DisMaxQuery/README.md" @@ -0,0 +1,56 @@ +# 单字符串多字段查询:Dis Max Query +## 课程demo +``` + +PUT /blogs/_doc/1 +{ + "title": "Quick brown rabbits", + "body": "Brown rabbits are commonly seen." +} + +PUT /blogs/_doc/2 +{ + "title": "Keeping pets healthy", + "body": "My quick brown fox eats rabbits on a regular basis." +} + +POST /blogs/_search +{ + "query": { + "bool": { + "should": [ + { "match": { "title": "Brown fox" }}, + { "match": { "body": "Brown fox" }} + ] + } + } +} + +POST blogs/_search +{ + "query": { + "dis_max": { + "queries": [ + { "match": { "title": "Quick pets" }}, + { "match": { "body": "Quick pets" }} + ] + } + } +} + + +POST blogs/_search +{ + "query": { + "dis_max": { + "queries": [ + { "match": { "title": "Quick pets" }}, + { "match": { "body": "Quick pets" }} + ], + "tie_breaker": 0.2 + } + } +} + + +``` diff --git "a/part-2/4.6-\345\215\225\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242-Multi-Match/README.md" "b/part-2/4.6-\345\215\225\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242-Multi-Match/README.md" new file mode 100644 index 0000000..f489bde --- /dev/null +++ "b/part-2/4.6-\345\215\225\345\255\227\347\254\246\344\270\262\345\244\232\345\255\227\346\256\265\346\237\245\350\257\242-Multi-Match/README.md" @@ -0,0 +1,143 @@ +# 单字符串多字段查询:Multi Match +## 课程demo +``` +POST blogs/_search +{ + "query": { + "dis_max": { + "queries": [ + { "match": { "title": "Quick pets" }}, + { "match": { "body": "Quick pets" }} + ], + "tie_breaker": 0.2 + } + } +} + +POST blogs/_search +{ + "query": { + "multi_match": { + "type": "best_fields", + "query": "Quick pets", + "fields": ["title","body"], + "tie_breaker": 0.2, + "minimum_should_match": "20%" + } + } +} + + + +POST books/_search +{ + "multi_match": { + "query": "Quick brown fox", + "fields": "*_title" + } +} + + +POST books/_search +{ + "multi_match": { + "query": "Quick brown fox", + "fields": [ "*_title", "chapter_title^2" ] + } +} + + + +DELETE /titles +PUT /titles +{ + "settings": { "number_of_shards": 1 }, + "mappings": { + "my_type": { + "properties": { + "title": { + "type": "string", + "analyzer": "english", + "fields": { + "std": { + "type": "string", + "analyzer": "standard" + } + } + } + } + } + } +} + +PUT /titles +{ + "mappings": { + "properties": { + "title": { + "type": "text", + "analyzer": "english" + } + } + } +} + +POST titles/_bulk +{ "index": { "_id": 1 }} +{ "title": "My dog barks" } +{ "index": { "_id": 2 }} +{ "title": "I see a lot of barking dogs on the road " } + + +GET titles/_search +{ + "query": { + "match": { + "title": "barking dogs" + } + } +} + +DELETE /titles +PUT /titles +{ + "mappings": { + "properties": { + "title": { + "type": "text", + "analyzer": "english", + "fields": {"std": {"type": "text","analyzer": "standard"}} + } + } + } +} + +POST titles/_bulk +{ "index": { "_id": 1 }} +{ "title": "My dog barks" } +{ "index": { "_id": 2 }} +{ "title": "I see a lot of barking dogs on the road " } + +GET /titles/_search +{ + "query": { + "multi_match": { + "query": "barking dogs", + "type": "most_fields", + "fields": [ "title", "title.std" ] + } + } +} + +GET /titles/_search +{ + "query": { + "multi_match": { + "query": "barking dogs", + "type": "most_fields", + "fields": [ "title^10", "title.std" ] + } + } +} + +``` diff --git "a/part-2/4.7-\345\244\232\350\257\255\350\250\200\345\217\212\344\270\255\346\226\207\345\210\206\350\257\215\344\270\216\346\243\200\347\264\242/README.md" "b/part-2/4.7-\345\244\232\350\257\255\350\250\200\345\217\212\344\270\255\346\226\207\345\210\206\350\257\215\344\270\216\346\243\200\347\264\242/README.md" new file mode 100644 index 0000000..039121b --- /dev/null +++ "b/part-2/4.7-\345\244\232\350\257\255\350\250\200\345\217\212\344\270\255\346\226\207\345\210\206\350\257\215\344\270\216\346\243\200\347\264\242/README.md" @@ -0,0 +1,168 @@ +# 多语言及中文分词与检索 +## 课程demo + +- 来到杨过曾经生活过的地方,小龙女动情地说:“我也想过过过儿过过的生活。” + +- 你也想犯范范玮琪犯过的错吗 +- 校长说衣服上除了校徽别别别的 +- 这几天天天天气不好 +- 我背有点驼,麻麻说“你的背得背背背背佳 + +``` +#stop word + +DELETE my_index +PUT /my_index/_doc/1 +{ "title": "I'm happy for this fox" } + +PUT /my_index/_doc/2 +{ "title": "I'm not happy about my fox problem" } + + +POST my_index/_search +{ + "query": { + "match": { + "title": "not happy fox" + } + } +} + + +#虽然通过使用 english (英语)分析器,使得匹配规则更加宽松,我们也因此提高了召回率,但却降低了精准匹配文档的能力。为了获得两方面的优势,我们可以使用multifields(多字段)对 title 字段建立两次索引: 一次使用 `english`(英语)分析器,另一次使用 `standard`(标准)分析器: + +DELETE my_index + +PUT /my_index +{ + "mappings": { + "blog": { + "properties": { + "title": { + "type": "string", + "analyzer": "english" + } + } + } + } +} + +PUT /my_index +{ + "mappings": { + "blog": { + "properties": { + "title": { + "type": "string", + "fields": { + "english": { + "type": "string", + "analyzer": "english" + } + } + } + } + } + } +} + + +PUT /my_index/blog/1 +{ "title": "I'm happy for this fox" } + +PUT /my_index/blog/2 +{ "title": "I'm not happy about my fox problem" } + +GET /_search +{ + "query": { + "multi_match": { + "type": "most_fields", + "query": "not happy foxes", + "fields": [ "title", "title.english" ] + } + } +} + + +#安装插件 +./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.1.0/elasticsearch-analysis-ik-7.1.0.zip +#安装插件 +bin/elasticsearch install https://github.com/KennFalcon/elasticsearch-analysis-hanlp/releases/download/v7.1.0/elasticsearch-analysis-hanlp-7.1.0.zip + + + + +#ik_max_word +#ik_smart +#hanlp: hanlp默认分词 +#hanlp_standard: 标准分词 +#hanlp_index: 索引分词 +#hanlp_nlp: NLP分词 +#hanlp_n_short: N-最短路分词 +#hanlp_dijkstra: 最短路分词 +#hanlp_crf: CRF分词(在hanlp 1.6.6已开始废弃) +#hanlp_speed: 极速词典分词 + +POST _analyze +{ + "analyzer": "hanlp_standard", + "text": ["剑桥分析公司多位高管对卧底记者说,他们确保了唐纳德·特朗普在总统大选中获胜"] + +} + +#Pinyin +PUT /artists/ +{ + "settings" : { + "analysis" : { + "analyzer" : { + "user_name_analyzer" : { + "tokenizer" : "whitespace", + "filter" : "pinyin_first_letter_and_full_pinyin_filter" + } + }, + "filter" : { + "pinyin_first_letter_and_full_pinyin_filter" : { + "type" : "pinyin", + "keep_first_letter" : true, + "keep_full_pinyin" : false, + "keep_none_chinese" : true, + "keep_original" : false, + "limit_first_letter_length" : 16, + "lowercase" : true, + "trim_whitespace" : true, + "keep_none_chinese_in_first_letter" : true + } + } + } + } +} + + +GET /artists/_analyze +{ + "text": ["刘德华 张学友 郭富城 黎明 四大天王"], + "analyzer": "user_name_analyzer" +} + + +``` + +## 相关资源 +- Elasticsearch IK分词插件 https://github.com/medcl/elasticsearch-analysis-ik/releases +- Elasticsearch hanlp 分词插件 https://github.com/KennFalcon/elasticsearch-analysis-hanlp + +- 分词算法综述 https://zhuanlan.zhihu.com/p/50444885 + +### 一些分词工具,供参考: +- 中科院计算所NLPIR http://ictclas.nlpir.org/nlpir/ +- ansj分词器 https://github.com/NLPchina/ansj_seg +- 哈工大的LTP https://github.com/HIT-SCIR/ltp +- 清华大学THULAC https://github.com/thunlp/THULAC +- 斯坦福分词器 https://nlp.stanford.edu/software/segmenter.shtml +- Hanlp分词器 https://github.com/hankcs/HanLP +- 结巴分词 https://github.com/yanyiwu/cppjieba +- KCWS分词器(字嵌入+Bi-LSTM+CRF) https://github.com/koth/kcws +- ZPar https://github.com/frcchang/zpar/releases +- IKAnalyzer https://github.com/wks/ik-analyzer diff --git "a/part-2/4.8-SpaceJam\344\270\200\344\270\252\345\205\250\346\226\207\346\220\234\347\264\242\347\232\204\345\256\236\344\276\213/README.md" "b/part-2/4.8-SpaceJam\344\270\200\344\270\252\345\205\250\346\226\207\346\220\234\347\264\242\347\232\204\345\256\236\344\276\213/README.md" new file mode 100644 index 0000000..01cae09 --- /dev/null +++ "b/part-2/4.8-SpaceJam\344\270\200\344\270\252\345\205\250\346\226\207\346\220\234\347\264\242\347\232\204\345\256\236\344\276\213/README.md" @@ -0,0 +1,29 @@ +# Space Jam,一次全文搜索的实例 +## 课程demo +``` +POST tmdb/_search +{ +"_source": ["title","overview"], + "query": { + "match_all": {} + } +} + +POST tmdb/_search +{ + "_source": ["title","overview"], + "query": { + "multi_match": { + "query": "basketball with cartoon aliens", + "fields": ["title","overview"] + } + }, + "highlight" : { + "fields" : { + "overview" : { "pre_tags" : ["\\033[0;32;40m"], "post_tags" : ["\\033[0m"] }, + "title" : { "pre_tags" : ["\\033[0;32;40m"], "post_tags" : ["\\033[0m"] } + + } + } +} +``` diff --git "a/part-2/4.9-\344\275\277\347\224\250SearchTemplate\345\222\214IndexAlias\350\277\233\350\241\214\346\237\245\350\257\242/README.md" "b/part-2/4.9-\344\275\277\347\224\250SearchTemplate\345\222\214IndexAlias\350\277\233\350\241\214\346\237\245\350\257\242/README.md" new file mode 100644 index 0000000..14f41cb --- /dev/null +++ "b/part-2/4.9-\344\275\277\347\224\250SearchTemplate\345\222\214IndexAlias\350\277\233\350\241\214\346\237\245\350\257\242/README.md" @@ -0,0 +1,36 @@ +# 使用 Search Template 和 Index Alias 查询 +## 课程Demo + +``` +POST _scripts/tmdb +{ + "script": { + "lang": "mustache", + "source": { + "_source": [ + "title","overview" + ], + "size": 20, + "query": { + "multi_match": { + "query": "{{q}}", + "fields": ["title","overview"] + } + } + } + } +} +DELETE _scripts/tmdb + +GET _scripts/tmdb + +POST tmdb/_search/template +{ + "id":"tmdb", + "params": { + "q": "basketball with cartoon aliens" + } +} + + +``` diff --git "a/part-2/5.1-\351\233\206\347\276\244\345\210\206\345\270\203\345\274\217\346\250\241\345\236\213\345\217\212\351\200\211\344\270\273\344\270\216\350\204\221\350\243\202\351\227\256\351\242\230/README.md" "b/part-2/5.1-\351\233\206\347\276\244\345\210\206\345\270\203\345\274\217\346\250\241\345\236\213\345\217\212\351\200\211\344\270\273\344\270\216\350\204\221\350\243\202\351\227\256\351\242\230/README.md" new file mode 100644 index 0000000..8acc697 --- /dev/null +++ "b/part-2/5.1-\351\233\206\347\276\244\345\210\206\345\270\203\345\274\217\346\250\241\345\236\213\345\217\212\351\200\211\344\270\273\344\270\216\350\204\221\350\243\202\351\227\256\351\242\230/README.md" @@ -0,0 +1,9 @@ +# 集群分布式模型及选主与脑裂问题 + +## 课程Demo +``` +bin/elasticsearch -E node.name=node1 -E cluster.name=geektime -E path.data=node1_data +bin/elasticsearch -E node.name=node2 -E cluster.name=geektime -E path.data=node2_data +bin/elasticsearch -E node.name=node3 -E cluster.name=geektime -E path.data=node3_data + +``` diff --git "a/part-2/5.2-\345\210\206\347\211\207\344\270\216\351\233\206\347\276\244\347\232\204\346\225\205\351\232\234\350\275\254\347\247\273/README.md" "b/part-2/5.2-\345\210\206\347\211\207\344\270\216\351\233\206\347\276\244\347\232\204\346\225\205\351\232\234\350\275\254\347\247\273/README.md" new file mode 100644 index 0000000..3bc48fb --- /dev/null +++ "b/part-2/5.2-\345\210\206\347\211\207\344\270\216\351\233\206\347\276\244\347\232\204\346\225\205\351\232\234\350\275\254\347\247\273/README.md" @@ -0,0 +1 @@ +# 分片与集群的故障转移 diff --git "a/part-2/5.3-\346\226\207\346\241\243\345\210\206\345\270\203\345\274\217\345\255\230\345\202\250/README.md" "b/part-2/5.3-\346\226\207\346\241\243\345\210\206\345\270\203\345\274\217\345\255\230\345\202\250/README.md" new file mode 100644 index 0000000..9773217 --- /dev/null +++ "b/part-2/5.3-\346\226\207\346\241\243\345\210\206\345\270\203\345\274\217\345\255\230\345\202\250/README.md" @@ -0,0 +1 @@ +# 文档分布式存储 diff --git "a/part-2/5.4-\345\210\206\347\211\207\345\217\212\345\205\266\347\224\237\345\221\275\345\221\250\346\234\237/README.md" "b/part-2/5.4-\345\210\206\347\211\207\345\217\212\345\205\266\347\224\237\345\221\275\345\221\250\346\234\237/README.md" new file mode 100644 index 0000000..e9509f6 --- /dev/null +++ "b/part-2/5.4-\345\210\206\347\211\207\345\217\212\345\205\266\347\224\237\345\221\275\345\221\250\346\234\237/README.md" @@ -0,0 +1 @@ +# 分片及其生命周期 diff --git "a/part-2/5.5-\345\211\226\346\236\220\345\210\206\345\270\203\345\274\217\346\237\245\350\257\242\345\217\212\347\233\270\345\205\263\346\200\247\350\257\204\345\210\206/README.md" "b/part-2/5.5-\345\211\226\346\236\220\345\210\206\345\270\203\345\274\217\346\237\245\350\257\242\345\217\212\347\233\270\345\205\263\346\200\247\350\257\204\345\210\206/README.md" new file mode 100644 index 0000000..2ad3bc9 --- /dev/null +++ "b/part-2/5.5-\345\211\226\346\236\220\345\210\206\345\270\203\345\274\217\346\237\245\350\257\242\345\217\212\347\233\270\345\205\263\346\200\247\350\257\204\345\210\206/README.md" @@ -0,0 +1,63 @@ +# 剖析分布式查询及相关性评分 +## 课程demo +``` +DELETE message +PUT message +{ + "settings": { + "number_of_shards": 20 + } +} + +GET message + +POST message/_doc?routing=1 +{ + "content":"good" +} + +POST message/_doc?routing=2 +{ + "content":"good morning" +} + +POST message/_doc?routing=3 +{ + "content":"good morning everyone" +} + +POST message/_search +{ + "explain": true, + "query": { + "match_all": {} + } +} + + +POST message/_search +{ + "explain": true, + "query": { + "term": { + "content": { + "value": "good" + } + } + } +} + + +POST message/_search?search_type=dfs_query_then_fetch +{ + + "query": { + "term": { + "content": { + "value": "good" + } + } + } +} + +``` diff --git "a/part-2/5.6-\346\216\222\345\272\217\345\217\212DocValues&Fielddata/README.md" "b/part-2/5.6-\346\216\222\345\272\217\345\217\212DocValues&Fielddata/README.md" new file mode 100644 index 0000000..23055c3 --- /dev/null +++ "b/part-2/5.6-\346\216\222\345\272\217\345\217\212DocValues&Fielddata/README.md" @@ -0,0 +1,123 @@ +# 排序及Doc Values & Fielddata +## 课程demo +``` +#单字段排序 +POST /kibana_sample_data_ecommerce/_search +{ + "size": 5, + "query": { + "match_all": { + + } + }, + "sort": [ + {"order_date": {"order": "desc"}} + ] +} + +#多字段排序 +POST /kibana_sample_data_ecommerce/_search +{ + "size": 5, + "query": { + "match_all": { + + } + }, + "sort": [ + {"order_date": {"order": "desc"}}, + {"_doc":{"order": "asc"}}, + {"_score":{ "order": "desc"}} + ] +} + +GET kibana_sample_data_ecommerce/_mapping + +#对 text 字段进行排序。默认会报错,需打开fielddata +POST /kibana_sample_data_ecommerce/_search +{ + "size": 5, + "query": { + "match_all": { + + } + }, + "sort": [ + {"customer_full_name": {"order": "desc"}} + ] +} + +#打开 text的 fielddata +PUT kibana_sample_data_ecommerce/_mapping +{ + "properties": { + "customer_full_name" : { + "type" : "text", + "fielddata": true, + "fields" : { + "keyword" : { + "type" : "keyword", + "ignore_above" : 256 + } + } + } + } +} + +#关闭 keyword的 doc values +PUT test_keyword +PUT test_keyword/_mapping +{ + "properties": { + "user_name":{ + "type": "keyword", + "doc_values":false + } + } +} + +DELETE test_keyword + +PUT test_text +PUT test_text/_mapping +{ + "properties": { + "intro":{ + "type": "text", + "doc_values":true + } + } +} + +DELETE test_text + + +DELETE temp_users +PUT temp_users +PUT temp_users/_mapping +{ + "properties": { + "name":{"type": "text","fielddata": true}, + "desc":{"type": "text","fielddata": true} + } +} + +Post temp_users/_doc +{"name":"Jack","desc":"Jack is a good boy!","age":10} + +#打开fielddata 后,查看 docvalue_fields数据 +POST temp_users/_search +{ + "docvalue_fields": [ + "name","desc" + ] +} + +#查看整型字段的docvalues +POST temp_users/_search +{ + "docvalue_fields": [ + "age" + ] +} +``` diff --git "a/part-2/5.7-\345\210\206\351\241\265\344\270\216\351\201\215\345\216\206-FromSize&SearchAfter&ScrollAPI/README.md" "b/part-2/5.7-\345\210\206\351\241\265\344\270\216\351\201\215\345\216\206-FromSize&SearchAfter&ScrollAPI/README.md" new file mode 100644 index 0000000..de47d77 --- /dev/null +++ "b/part-2/5.7-\345\210\206\351\241\265\344\270\216\351\201\215\345\216\206-FromSize&SearchAfter&ScrollAPI/README.md" @@ -0,0 +1,96 @@ +# 分页与遍历 - From, Size, Search_after & Scroll API +## 课程Demo +``` + +POST tmdb/_search +{ + "from": 10000, + "size": 1, + "query": { + "match_all": { + + } + } +} + +#Scroll API +DELETE users + +POST users/_doc +{"name":"user1","age":10} + +POST users/_doc +{"name":"user2","age":11} + + +POST users/_doc +{"name":"user2","age":12} + +POST users/_doc +{"name":"user2","age":13} + +POST users/_count + +POST users/_search +{ + "size": 1, + "query": { + "match_all": {} + }, + "sort": [ + {"age": "desc"} , + {"_id": "asc"} + ] +} + +POST users/_search +{ + "size": 1, + "query": { + "match_all": {} + }, + "search_after": + [ + 10, + "ZQ0vYGsBrR8X3IP75QqX"], + "sort": [ + {"age": "desc"} , + {"_id": "asc"} + ] +} + + +#Scroll API +DELETE users +POST users/_doc +{"name":"user1","age":10} + +POST users/_doc +{"name":"user2","age":20} + +POST users/_doc +{"name":"user3","age":30} + +POST users/_doc +{"name":"user4","age":40} + +POST /users/_search?scroll=5m +{ + "size": 1, + "query": { + "match_all" : { + } + } +} + + +POST users/_doc +{"name":"user5","age":50} +POST /_search/scroll +{ + "scroll" : "1m", + "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAWAWbWdoQXR2d3ZUd2kzSThwVTh4bVE0QQ==" +} + + +``` diff --git "a/part-2/5.8\345\244\204\347\220\206\345\271\266\345\217\221\350\257\273\345\206\231/README.md" "b/part-2/5.8\345\244\204\347\220\206\345\271\266\345\217\221\350\257\273\345\206\231/README.md" new file mode 100644 index 0000000..bc23beb --- /dev/null +++ "b/part-2/5.8\345\244\204\347\220\206\345\271\266\345\217\221\350\257\273\345\206\231/README.md" @@ -0,0 +1,34 @@ +# 处理并发读写操作 +## 课程DEMO +``` + +DELETE products +PUT products + +PUT products/_doc/1 +{ + "title":"iphone", + "count":100 +} + + + +GET products/_doc/1 + +PUT products/_doc/1?if_seq_no=1&if_primary_term=1 +{ + "title":"iphone", + "count":100 +} + + + +PUT products/_doc/1?version=30000&version_type=external +{ + "title":"iphone", + "count":100 +} + + + +```