mongdb分片原理
分片,是指將數(shù)據(jù)拆分,將其分散到不同的機(jī)器上,分片類似于raid0,副本類似于raid1
MongoDB的副本集與我們常見的主從有所不同，主從在主機(jī)宕機(jī)后所有服務(wù)將停止

站在用戶的角度思考問題，與客戶深入溝通，找到泉州網(wǎng)站設(shè)計(jì)與泉州網(wǎng)站推廣的解決方案，憑借多年的經(jīng)驗(yàn)，讓設(shè)計(jì)與互聯(lián)網(wǎng)技術(shù)結(jié)合，創(chuàng)造個(gè)性化、用戶體驗(yàn)好的作品，建站類型包括：網(wǎng)站制作、網(wǎng)站建設(shè)、企業(yè)官網(wǎng)、英文網(wǎng)站、手機(jī)端網(wǎng)站、網(wǎng)站推廣、域名注冊(cè)、虛擬主機(jī)、企業(yè)郵箱。業(yè)務(wù)覆蓋泉州地區(qū)。

分片集群主要由三種組件組成:mongos,config server,shard
1) mongos (路由進(jìn)程, 應(yīng)用程序接入 mongos 再查詢到具體分片)
數(shù)據(jù)庫集群請(qǐng)求的入口,所有的請(qǐng)求都通過 mongos 進(jìn)行協(xié)調(diào),不需要在應(yīng)用程序添加一個(gè)路由選擇器,mongos 自己就是一個(gè)請(qǐng)求分發(fā)中心,它負(fù)責(zé)把對(duì)應(yīng)的數(shù)據(jù)請(qǐng)求
請(qǐng)求轉(zhuǎn)發(fā)到對(duì)應(yīng)的 shard 服務(wù)器上。在生產(chǎn)環(huán)境通常有多個(gè) mongos 作為請(qǐng)求的入口,防止其中一個(gè)掛掉所有的 mongodb 請(qǐng)求都沒有辦法操作。
2) config server (路由表服務(wù)。 每一臺(tái)都具有全部 chunk 的路由信息)
顧名思義為配置服務(wù)器,存儲(chǔ)所有數(shù)據(jù)庫元信息(路由、分片)的配置。mongos 本身沒有物理存儲(chǔ)分片服務(wù)器和數(shù)據(jù)路由信息,只是緩存在內(nèi)存里,配置服務(wù)器則實(shí)際存儲(chǔ)
這些數(shù)據(jù)。mongos 第一次啟動(dòng)或者關(guān)掉重啟就會(huì)從 config server 加載配置信息,以后如果配置服務(wù)器信息變化會(huì)通知到所有的 mongos 更新自己的狀態(tài),這樣
mongos 就能繼續(xù)準(zhǔn)確路由。在生產(chǎn)環(huán)境通常有多個(gè) config server 配置服務(wù)器,因?yàn)樗鎯?chǔ)了分片路由的元數(shù)據(jù),這個(gè)可不能丟失!就算掛掉其中一臺(tái),只要還有存貨,
mongodb 集群就不會(huì)掛掉。
3) shard (為數(shù)據(jù)存儲(chǔ)分片。 每一片都可以是復(fù)制集(replica set))
這就是傳說中的分片了。如圖所示,一臺(tái)機(jī)器的一個(gè)數(shù)據(jù)表 Collection1 存儲(chǔ)了 1T 數(shù)據(jù),壓力太大了!在分給 4 個(gè)機(jī)器后, 每個(gè)機(jī)器都是 256G,則分?jǐn)偭思性谝慌_(tái)
機(jī)器的壓力。事實(shí)上,上圖4個(gè)分片如果沒有副本集(replica set)是個(gè)不完整架構(gòu),假設(shè)其中的一個(gè)分片掛掉那四 分之一的數(shù)據(jù)就丟失了,所以在高可用性的分片架構(gòu)還
需要對(duì)于每一個(gè)分片構(gòu)建 replica set 副本集保 證分片的可靠性。生產(chǎn)環(huán)境通常是 2 個(gè)副本 + 1 個(gè)仲裁。

廢話不多說

1.從github上拉取配置文件
git clone git@github.com:herrywen-nanj/mongodb.git

2.啟動(dòng)順序?yàn)閏onfigserver --> mongos --> shared

3.刪除dbPath下的內(nèi)容
rm -rf configserver/dbPath

4.根據(jù)對(duì)應(yīng)配置文件啟動(dòng)mongdb進(jìn)程，進(jìn)入mongdb中配置副本集
configserver啟動(dòng):
mongod -f mongdb.conf
configserver2啟動(dòng):
mongod -f mongdb.conf
配置副本集:

# 進(jìn)入configserver
mongo --port 27018
# 初始化
>>>rs.initiate()
# 添加副本節(jié)點(diǎn)
>>>rs.add("worker2:27019")
# 查看副本集狀態(tài)
>>>rs.status
返回內(nèi)容：

MongoDB Enterprise config-rs:PRIMARY> rs.status()
{
        "set" : "config-rs",       # 副本集已經(jīng)配置成功
        "date" : ISODate("2019-11-23T04:56:35.588Z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "syncingTo" : "",
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "configsvr" : true,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1574484982, 1),
                        "t" : NumberLong(1)
                },
                "lastCommittedWallTime" : ISODate("2019-11-23T04:56:22.464Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1574484982, 1),
                        "t" : NumberLong(1)
                },
                "readConcernMajorityWallTime" : ISODate("2019-11-23T04:56:22.464Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1574484982, 1),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1574484982, 1),
                        "t" : NumberLong(1)
                },
                "lastAppliedWallTime" : ISODate("2019-11-23T04:56:22.464Z"),
                "lastDurableWallTime" : ISODate("2019-11-23T04:56:22.464Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1574484952, 30),
        "lastStableCheckpointTimestamp" : Timestamp(1574484952, 30),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2019-11-23T04:55:51.134Z"),
                "termAtElection" : NumberLong(1),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1574484951, 1),
                        "t" : NumberLong(-1)
                },
                "numVotesNeeded" : 1,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "newTermStartDate" : ISODate("2019-11-23T04:55:52.141Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2019-11-23T04:55:52.266Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "worker2:27018",
                        "ip" : "192.168.255.134",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 722,
                        "optime" : {
                                "ts" : Timestamp(1574484982, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2019-11-23T04:56:22Z"),
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "could not find member to sync from",
                        "electionTime" : Timestamp(1574484951, 2),
                        "electionDate" : ISODate("2019-11-23T04:55:51Z"),
                        "configVersion" : 2,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "worker2:27019",
                        "ip" : "192.168.255.134",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 13,
                        "optime" : {
                                "ts" : Timestamp(1574484982, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1574484982, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2019-11-23T04:56:22Z"),
                        "optimeDurableDate" : ISODate("2019-11-23T04:56:22Z"),
                        "lastHeartbeat" : ISODate("2019-11-23T04:56:34.705Z"),
                        "lastHeartbeatRecv" : ISODate("2019-11-23T04:56:35.176Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 2
                }
        ],
        "ok" : 1,
        "$gleStats" : {
                "lastOpTime" : {
                        "ts" : Timestamp(1574484982, 1),
                        "t" : NumberLong(1)
                },
                "electionId" : ObjectId("7fffffff0000000000000001")
        },
        "lastCommittedOpTime" : Timestamp(1574484982, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1574484982, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1574484982, 1)
}

5.刪除dbPath目錄并啟動(dòng)mongs路由進(jìn)程

cd luyou
mongos -f mongdb.conf
lsof -i:40000

6.分別進(jìn)入分片節(jié)點(diǎn)目錄，啟動(dòng)進(jìn)程，完成副本集群添加操作

cd shared && rm -rf dbpath && mongod -f mongdb.conf
lsof -i:27020 && lsof -i:27021 && lsof -i:27022

mongo --port 27020
>>>rs.initiate()
>>>rs.add("worker2:27021")
>>>rs.add("worker2:27022")
>>>rs.status()

7.在路由節(jié)點(diǎn)添加分片集群并新增一個(gè)分片
sh.addShard("shard-rs/worker2:27020,worker2:27021,worker2:27022")

新增分片：

cd shard4
sh.addShard("worker2:27023")

查看返回狀態(tài)：

MongoDB Enterprise mongos> sh.status()
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5dd8bbd8bbb4a8ac81b4b0b6")
  }
  shards:
        {  "_id" : "shard-rs",  "host" : "shard-rs/worker2:27020,worker2:27021,worker2:27022",  "state" : 1 }
        {  "_id" : "shard0001",  "host" : "worker2:27023",  "state" : 1 }
  active mongoses:
        "4.2.1" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                No recent migrations
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }

8.片鍵操作
a.為herrywen這個(gè)數(shù)據(jù)庫開啟分片功能

sh.enableSharding("herrywen")
查看返回狀態(tài)：

MongoDB Enterprise mongos> sh.status()
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5dd8bbd8bbb4a8ac81b4b0b6")
  }
  shards:
        {  "_id" : "shard-rs",  "host" : "shard-rs/worker2:27020,worker2:27021,worker2:27022",  "state" : 1 }
  active mongoses:
        "4.2.1" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                No recent migrations
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
        {  "_id" : "herrywen",  "primary" : "shard-rs",  "partitioned" : true,  "version" : {  "uuid" : UUID("56cf9d23-2f3a-4b53-8b5d-512f1f9e00c6"),  "lastMod" : 1 } }

b. 開啟特定集合功能，并指定id為片鍵
sh.shardCollection("herrywen.collections_1",{"_id":1})
查看返回結(jié)果

MongoDB Enterprise mongos> sh.status();
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5dd8bbd8bbb4a8ac81b4b0b6")
  }
  shards:
        {  "_id" : "shard-rs",  "host" : "shard-rs/worker2:27020,worker2:27021,worker2:27022",  "state" : 1 }
        {  "_id" : "shard0001",  "host" : "worker2:27023",  "state" : 1 }
  active mongoses:
        "4.2.1" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Collections with active migrations:
                herrywen.collections_1 started at Sat Nov 23 2019 14:15:13 GMT+0800 (CST)
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                2 : Success
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard-rs        1
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard-rs Timestamp(1, 0)
        {  "_id" : "herrywen",  "primary" : "shard-rs",  "partitioned" : true,  "version" : {  "uuid" : UUID("56cf9d23-2f3a-4b53-8b5d-512f1f9e00c6"),  "lastMod" : 1 } }
                herrywen.collections_1
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard-rs        3
                                shard0001       4
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : 2 } on : shard0001 Timestamp(2, 0)
                        { "_id" : 2 } -->> { "_id" : 28340 } on : shard-rs Timestamp(3, 1)
                        { "_id" : 28340 } -->> { "_id" : 42509 } on : shard-rs Timestamp(2, 2)
                        { "_id" : 42509 } -->> { "_id" : 61031 } on : shard-rs Timestamp(2, 3)
                        { "_id" : 61031 } -->> { "_id" : 75200 } on : shard0001 Timestamp(3, 2)
                        { "_id" : 75200 } -->> { "_id" : 94169 } on : shard0001 Timestamp(3, 3)
                        { "_id" : 94169 } -->> { "_id" : { "$maxKey" : 1 } } on : shard0001 Timestamp(3, 4)

9.測試
a. # 修改chunk塊大小為1M,默認(rèn)64M

use config;
db.settings.find()
db.settings.save({_id:"chunksize",value:1})
MongoDB Enterprise mongos> db.settings.find()
{ "_id" : "chunksize", "value" : 1 }

b.查看當(dāng)前集合中的數(shù)據(jù)量

MongoDB Enterprise mongos> use herrywen;
switched to db herrywen
MongoDB Enterprise mongos> db.collections_1.count()
0

c.為了能看到分片效果，往herrywen.collections_1集合中寫入10000條數(shù)據(jù)

mongo --port 40000
MongoDB Enterprise mongos> for(var i=1;i<=100000;i++){
...                        db.collections_1.insert({"_id":i,"name":"copy"+i});
... }

打開另一個(gè)終端連接，查看是否在寫入：

[root@worker2 shard3]# mongo --port 40000
MongoDB shell version v4.2.1
connecting to: mongodb://127.0.0.1:40000/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("1101172d-02b7-4881-bc3a-1a360390db39") }
MongoDB server version: 4.2.1
Server has startup warnings:
2019-11-23T13:04:19.245+0800 I  CONTROL  [main]
2019-11-23T13:04:19.245+0800 I  CONTROL  [main] ** WARNING: Access control is not enabled for the database.
2019-11-23T13:04:19.245+0800 I  CONTROL  [main] **          Read and write access to data and configuration is unrestricted.
2019-11-23T13:04:19.245+0800 I  CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.
2019-11-23T13:04:19.246+0800 I  CONTROL  [main]
MongoDB Enterprise mongos> use herrywen;
switched to db herrywen
MongoDB Enterprise mongos> db.collections_1.count();
41561
MongoDB Enterprise mongos> db.collections_1.count();
42971
MongoDB Enterprise mongos> db.collections_1.count();
43516
MongoDB Enterprise mongos> db.collections_1.count();
43776
MongoDB Enterprise mongos> db.collections_1.count();
44055
MongoDB Enterprise mongos> db.collections_1.count();
44291
MongoDB Enterprise mongos> db.collections_1.count();
44541
MongoDB Enterprise mongos> db.collections_1.count();
44775
MongoDB Enterprise mongos> db.collections_1.count();
45012
MongoDB Enterprise mongos> db.collections_1.count();
45257
MongoDB Enterprise mongos> db.collections_1.count();
45470

d.查看到數(shù)據(jù)也寫到其他分片中去了

MongoDB Enterprise mongos> sh.status();
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5dd8bbd8bbb4a8ac81b4b0b6")
  }
  shards:
        {  "_id" : "shard-rs",  "host" : "shard-rs/worker2:27020,worker2:27021,worker2:27022",  "state" : 1 }
        {  "_id" : "shard0001",  "host" : "worker2:27023",  "state" : 1 }
  active mongoses:
        "4.2.1" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Collections with active migrations:
                herrywen.collections_1 started at Sat Nov 23 2019 14:15:13 GMT+0800 (CST)
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                2 : Success
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard-rs        1
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard-rs Timestamp(1, 0)
        {  "_id" : "herrywen",  "primary" : "shard-rs",  "partitioned" : true,  "version" : {  "uuid" : UUID("56cf9d23-2f3a-4b53-8b5d-512f1f9e00c6"),  "lastMod" : 1 } }
                herrywen.collections_1
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard-rs        3
                                shard0001       4
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : 2 } on : shard0001 Timestamp(2, 0)
                        { "_id" : 2 } -->> { "_id" : 28340 } on : shard-rs Timestamp(3, 1)
                        { "_id" : 28340 } -->> { "_id" : 42509 } on : shard-rs Timestamp(2, 2)
                        { "_id" : 42509 } -->> { "_id" : 61031 } on : shard-rs Timestamp(2, 3)
                        { "_id" : 61031 } -->> { "_id" : 75200 } on : shard0001 Timestamp(3, 2)
                        { "_id" : 75200 } -->> { "_id" : 94169 } on : shard0001 Timestamp(3, 3)
                        { "_id" : 94169 } -->> { "_id" : { "$maxKey" : 1 } } on : shard0001 Timestamp(3, 4)

10.程序代碼內(nèi)無需太大更改，直接按照連接普通的mongo數(shù)據(jù)庫那樣，將數(shù)據(jù)庫連接接入接口40000

另外有需要云服務(wù)器可以了解下創(chuàng)新互聯(lián)cdcxhl.cn，海內(nèi)外云服務(wù)器15元起步，三天無理由+7*72小時(shí)售后在線，公司持有idc許可證，提供“云服務(wù)器、裸金屬服務(wù)器、高防服務(wù)器、香港服務(wù)器、美國服務(wù)器、虛擬主機(jī)、免備案服務(wù)器”等云主機(jī)租用服務(wù)以及企業(yè)上云的綜合解決方案，具有“安全穩(wěn)定、簡單易用、服務(wù)可用性高、性價(jià)比高”等特點(diǎn)與優(yōu)勢，專為企業(yè)上云打造定制，能夠滿足用戶豐富、多元化的應(yīng)用場景需求。

標(biāo)題名稱：mongdb分片原理以及分片副本集群搭建-創(chuàng)新互聯(lián)
標(biāo)題鏈接：http://weahome.cn/article/ghiih.html