真实的国产乱ⅩXXX66竹夫人,五月香六月婷婷激情综合,亚洲日本VA一区二区三区,亚洲精品一区二区三区麻豆

成都創(chuàng)新互聯(lián)網(wǎng)站制作重慶分公司

EFK教程(3)-ElasticSearch多實(shí)例部署

EFK教程(3) - ElasticSearch多實(shí)例部署

公司主營(yíng)業(yè)務(wù)：網(wǎng)站設(shè)計(jì)、網(wǎng)站制作、移動(dòng)網(wǎng)站開(kāi)發(fā)等業(yè)務(wù)。幫助企業(yè)客戶真正實(shí)現(xiàn)互聯(lián)網(wǎng)宣傳，提高企業(yè)的競(jìng)爭(zhēng)能力。創(chuàng)新互聯(lián)是一支青春激揚(yáng)、勤奮敬業(yè)、活力青春激揚(yáng)、勤奮敬業(yè)、活力澎湃、和諧高效的團(tuán)隊(duì)。公司秉承以“開(kāi)放、自由、嚴(yán)謹(jǐn)、自律”為核心的企業(yè)文化，感謝他們對(duì)我們的高要求，感謝他們從不同領(lǐng)域給我們帶來(lái)的挑戰(zhàn)，讓我們激情的團(tuán)隊(duì)有機(jī)會(huì)用頭腦與智慧不斷的給客戶帶來(lái)驚喜。創(chuàng)新互聯(lián)推出南豐免費(fèi)做網(wǎng)站回饋大家。

基于ElasticSearch多實(shí)例架構(gòu)，實(shí)現(xiàn)資源合理分配、冷熱數(shù)據(jù)分離

作者：“發(fā)顛的小狼”，歡迎轉(zhuǎn)載與投稿

目錄

? 用途
? 架構(gòu)
? 192.168.1.51 elasticsearch-data部署雙實(shí)例
? 192.168.1.52 elasticsearch-data部署雙實(shí)例
? 192.168.1.53 elasticsearch-data部署雙實(shí)例
? 測(cè)試

用途

前情提要：

? 在第一篇《EFK教程 - 快速入門(mén)指南》中，闡述了EFK的安裝部署，其中ES的架構(gòu)為三節(jié)點(diǎn)，即master、ingest、data角色同時(shí)部署在三臺(tái)服務(wù)器上。
? 在第二篇《EFK教程 - ElasticSearch高性能高可用架構(gòu)》中，闡述了EFK的data/ingest/master角色的用途及分別部署三節(jié)點(diǎn)，在實(shí)現(xiàn)性能最大化的同時(shí)保障高可用

前兩篇文章，ES集群中只存在一個(gè)實(shí)例，而在本文中，將在一個(gè)集群中部署多個(gè)ES實(shí)例，來(lái)實(shí)現(xiàn)資源合理分配。例如data服務(wù)器存在SSD與SAS硬盤(pán)，可以將熱數(shù)據(jù)存放到SSD，而冷數(shù)據(jù)存放到SAS，實(shí)現(xiàn)數(shù)據(jù)冷熱分離。

在本文中，將為data服務(wù)器創(chuàng)建2個(gè)實(shí)例，分別基于SSD和基于SAS硬盤(pán)，將nginx的9月份索引放在SAS盤(pán)上，其它的全放在SSD盤(pán)上

架構(gòu)

架構(gòu)圖

EFK教程(3) - ElasticSearch多實(shí)例部署

服務(wù)器配置

EFK教程(3) - ElasticSearch多實(shí)例部署

192.168.1.51 elasticsearch-data部署雙實(shí)例

索引遷移（此步不能忽略）：將192.168.1.51上的索引放到其它2臺(tái)data節(jié)點(diǎn)上

curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index.routing.allocation.include._ip": "192.168.1.52,192.168.1.53"
}'

確認(rèn)當(dāng)前索引存儲(chǔ)位置：確認(rèn)所有索引不在192.168.1.51節(jié)點(diǎn)上

curl "http://192.168.1.31:9200/_cat/shards?h=n"

EFK教程(3) - ElasticSearch多實(shí)例部署

停掉192.168.1.51的進(jìn)程，修改目錄結(jié)構(gòu)及配置：請(qǐng)自行按SSD和SAS硬盤(pán)掛載好數(shù)據(jù)盤(pán)

EFK教程(3) - ElasticSearch多實(shí)例部署

# 安裝包下載和部署請(qǐng)參考第一篇《EFK教程 - 快速入門(mén)指南》
cd /opt/software/
tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz
mv /opt/elasticsearch /opt/elasticsearch-SAS
mv elasticsearch-7.3.2 /opt/
mv /opt/elasticsearch-7.3.2 /opt/elasticsearch-SSD
chown elasticsearch.elasticsearch /opt/elasticsearch-* -R
rm -rf /data/SAS/*
chown elasticsearch.elasticsearch /data/* -R
mkdir -p /opt/logs/elasticsearch-SAS
mkdir -p /opt/logs/elasticsearch-SSD
chown elasticsearch.elasticsearch /opt/logs/* -R

SAS實(shí)例/opt/elasticsearch-SAS/config/elasticsearch.yml配置

cluster.name: my-application
node.name: 192.168.1.51-SAS
path.data: /data/SAS
path.logs: /opt/logs/elasticsearch-SAS
network.host: 192.168.1.51

http.port: 9200
transport.port: 9300
# discovery.seed_hosts和cluster.initial_master_nodes 一定要帶上端口號(hào)，不然會(huì)走h(yuǎn)ttp.port和transport.port端口
discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
http.cors.enabled: true
http.cors.allow-origin: "*"

node.master: false
node.ingest: false
node.data: true

# 本機(jī)只允行啟2個(gè)實(shí)例
node.max_local_storage_nodes: 2

SSD實(shí)例/opt/elasticsearch-SSD/config/elasticsearch.yml配置

cluster.name: my-application
node.name: 192.168.1.51-SSD
path.data: /data/SSD
path.logs: /opt/logs/elasticsearch-SSD
network.host: 192.168.1.51

http.port: 9201
transport.port: 9301
# discovery.seed_hosts和cluster.initial_master_nodes 一定要帶上端口號(hào)，不然會(huì)走h(yuǎn)ttp.port和transport.port端口
discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
http.cors.enabled: true
http.cors.allow-origin: "*"

node.master: false
node.ingest: false
node.data: true

# 本機(jī)只允行啟2個(gè)實(shí)例
node.max_local_storage_nodes: 2

SAS實(shí)例和SSD實(shí)例啟動(dòng)方式

sudo -u elasticsearch /opt/elasticsearch-SAS/bin/elasticsearch
sudo -u elasticsearch /opt/elasticsearch-SSD/bin/elasticsearch

確認(rèn)SAS和SSD已啟2實(shí)例

curl "http://192.168.1.31:9200/_cat/nodes?v"

EFK教程(3) - ElasticSearch多實(shí)例部署

192.168.1.52 elasticsearch-data部署雙實(shí)例

索引遷移（此步不能忽略）：將192.168.1.52上的索引放到其它2臺(tái)data節(jié)點(diǎn)上

curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index.routing.allocation.include._ip": "192.168.1.51,192.168.1.53"
}'

確認(rèn)當(dāng)前索引存儲(chǔ)位置： 確認(rèn)所有索引不在192.168.1.52節(jié)點(diǎn)上

curl "http://192.168.1.31:9200/_cat/shards?h=n"

EFK教程(3) - ElasticSearch多實(shí)例部署

EFK教程(3) - ElasticSearch多實(shí)例部署

停掉192.168.1.52的進(jìn)程，修改目錄結(jié)構(gòu)及配置：請(qǐng)自行按SSD和SAS硬盤(pán)掛載好數(shù)據(jù)盤(pán)

EFK教程(3) - ElasticSearch多實(shí)例部署

# 安裝包下載和部署請(qǐng)參考第一篇《EFK教程 - 快速入門(mén)指南》
cd /opt/software/
tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz
mv /opt/elasticsearch /opt/elasticsearch-SAS
mv elasticsearch-7.3.2 /opt/
mv /opt/elasticsearch-7.3.2 /opt/elasticsearch-SSD
chown elasticsearch.elasticsearch /opt/elasticsearch-* -R
rm -rf /data/SAS/*
chown elasticsearch.elasticsearch /data/* -R
mkdir -p /opt/logs/elasticsearch-SAS
mkdir -p /opt/logs/elasticsearch-SSD
chown elasticsearch.elasticsearch /opt/logs/* -R

SAS實(shí)例/opt/elasticsearch-SAS/config/elasticsearch.yml配置

cluster.name: my-application
node.name: 192.168.1.52-SAS
path.data: /data/SAS
path.logs: /opt/logs/elasticsearch-SAS
network.host: 192.168.1.52

http.port: 9200
transport.port: 9300
# discovery.seed_hosts和cluster.initial_master_nodes 一定要帶上端口號(hào)，不然會(huì)走h(yuǎn)ttp.port和transport.port端口
discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
http.cors.enabled: true
http.cors.allow-origin: "*"

node.master: false
node.ingest: false
node.data: true

# 本機(jī)只允行啟2個(gè)實(shí)例
node.max_local_storage_nodes: 2

SSD實(shí)例/opt/elasticsearch-SSD/config/elasticsearch.yml配置

cluster.name: my-application
node.name: 192.168.1.52-SSD
path.data: /data/SSD
path.logs: /opt/logs/elasticsearch-SSD
network.host: 192.168.1.52

http.port: 9201
transport.port: 9301
# discovery.seed_hosts和cluster.initial_master_nodes 一定要帶上端口號(hào)，不然會(huì)走h(yuǎn)ttp.port和transport.port端口
discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
http.cors.enabled: true
http.cors.allow-origin: "*"

node.master: false
node.ingest: false
node.data: true

# 本機(jī)只允行啟2個(gè)實(shí)例
node.max_local_storage_nodes: 2

SAS實(shí)例和SSD實(shí)例啟動(dòng)方式

sudo -u elasticsearch /opt/elasticsearch-SAS/bin/elasticsearch
sudo -u elasticsearch /opt/elasticsearch-SSD/bin/elasticsearch

確認(rèn)SAS和SSD已啟2實(shí)例

curl "http://192.168.1.31:9200/_cat/nodes?v"

EFK教程(3) - ElasticSearch多實(shí)例部署

192.168.1.53 elasticsearch-data部署雙實(shí)例

索引遷移（此步不能忽略）：一定要做這步，將192.168.1.53上的索引放到其它2臺(tái)data節(jié)點(diǎn)上

curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index.routing.allocation.include._ip": "192.168.1.51,192.168.1.52"
}'

確認(rèn)當(dāng)前索引存儲(chǔ)位置：確認(rèn)所有索引不在192.168.1.52節(jié)點(diǎn)上

curl "http://192.168.1.31:9200/_cat/shards?h=n"

EFK教程(3) - ElasticSearch多實(shí)例部署

EFK教程(3) - ElasticSearch多實(shí)例部署

停掉192.168.1.53的進(jìn)程，修改目錄結(jié)構(gòu)及配置：請(qǐng)自行按SSD和SAS硬盤(pán)掛載好數(shù)據(jù)盤(pán)

EFK教程(3) - ElasticSearch多實(shí)例部署

# 安裝包下載和部署請(qǐng)參考第一篇《EFK教程 - 快速入門(mén)指南》
cd /opt/software/
tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz
mv /opt/elasticsearch /opt/elasticsearch-SAS
mv elasticsearch-7.3.2 /opt/
mv /opt/elasticsearch-7.3.2 /opt/elasticsearch-SSD
chown elasticsearch.elasticsearch /opt/elasticsearch-* -R
rm -rf /data/SAS/*
chown elasticsearch.elasticsearch /data/* -R
mkdir -p /opt/logs/elasticsearch-SAS
mkdir -p /opt/logs/elasticsearch-SSD
chown elasticsearch.elasticsearch /opt/logs/* -R

SAS實(shí)例/opt/elasticsearch-SAS/config/elasticsearch.yml配置

cluster.name: my-application
node.name: 192.168.1.53-SAS
path.data: /data/SAS
path.logs: /opt/logs/elasticsearch-SAS
network.host: 192.168.1.53

http.port: 9200
transport.port: 9300
# discovery.seed_hosts和cluster.initial_master_nodes 一定要帶上端口號(hào)，不然會(huì)走h(yuǎn)ttp.port和transport.port端口
discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
http.cors.enabled: true
http.cors.allow-origin: "*"

node.master: false
node.ingest: false
node.data: true

# 本機(jī)只允行啟2個(gè)實(shí)例
node.max_local_storage_nodes: 2

SSD實(shí)例/opt/elasticsearch-SSD/config/elasticsearch.yml配置

cluster.name: my-application
node.name: 192.168.1.53-SSD
path.data: /data/SSD
path.logs: /opt/logs/elasticsearch-SSD
network.host: 192.168.1.53

http.port: 9201
transport.port: 9301
# discovery.seed_hosts和cluster.initial_master_nodes 一定要帶上端口號(hào)，不然會(huì)走h(yuǎn)ttp.port和transport.port端口
discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]
http.cors.enabled: true
http.cors.allow-origin: "*"

node.master: false
node.ingest: false
node.data: true

# 本機(jī)只允行啟2個(gè)實(shí)例
node.max_local_storage_nodes: 2

SAS實(shí)例和SSD實(shí)例啟動(dòng)方式

sudo -u elasticsearch /opt/elasticsearch-SAS/bin/elasticsearch
sudo -u elasticsearch /opt/elasticsearch-SSD/bin/elasticsearch

確認(rèn)SAS和SSD已啟2實(shí)例

curl "http://192.168.1.31:9200/_cat/nodes?v"

EFK教程(3) - ElasticSearch多實(shí)例部署

測(cè)試

將所有索引移到SSD硬盤(pán)上

# 下面的參數(shù)會(huì)在后面的文章講解，此處照抄即可
curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index.routing.allocation.include._host_ip": "",
  "index.routing.allocation.include._host": "",
  "index.routing.allocation.include._name": "",
  "index.routing.allocation.include._ip": "",
  "index.routing.allocation.require._name": "*-SSD"
}'

確認(rèn)所有索引全在SSD硬盤(pán)上

curl "http://192.168.1.31:9200/_cat/shards?h=n"

EFK教程(3) - ElasticSearch多實(shí)例部署

將nginx9月份的日志索引遷移到SAS硬盤(pán)上

curl -X PUT "192.168.1.31:9200/nginx_*_2019.09/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index.routing.allocation.require._name": "*-SAS"
}'

確認(rèn)nginx9月份的日志索引遷移到SAS硬盤(pán)上

curl "http://192.168.1.31:9200/_cat/shards"

EFK教程(3) - ElasticSearch多實(shí)例部署


名稱欄目：EFK教程(3)-ElasticSearch多實(shí)例部署
網(wǎng)站路徑：http://weahome.cn/article/jcphid.html

其他資訊

在線咨詢

微信咨詢

電話咨詢

028-86922220（工作日）

18980820575（7×24）

提交需求

返回頂部