簡書地址:
http://www.jianshu.com/p/1f4f9c07ce0b
之所以把MySQL.gtid_executed表的作用和Previous gtid Event的改變放到一起進行描述是因為它們后面文章探討的基礎(chǔ)。這部分使用到了我自己使用C語言寫的原生binlog解析工具infobin。
百度云盤下載如下:
http://pan.baidu.com/s/1jHIWUN0
在成都做網(wǎng)站、成都網(wǎng)站設(shè)計過程中,需要針對客戶的行業(yè)特點、產(chǎn)品特性、目標(biāo)受眾和市場情況進行定位分析,以確定網(wǎng)站的風(fēng)格、色彩、版式、交互等方面的設(shè)計方向。成都創(chuàng)新互聯(lián)公司還需要根據(jù)客戶的需求進行功能模塊的開發(fā)和設(shè)計,包括內(nèi)容管理、前臺展示、用戶權(quán)限管理、數(shù)據(jù)統(tǒng)計和安全保護等功能。
為什么要先描述什么是Gtid event呢?因為后面會用到,實際上在中其核心元素就是一個形如:
31704d8a-da74-11e7-b6bf-52540a7d243:100009
的一個Gtid處于整個事物event中的開始,用于描述這個事物的Gtid是多少,當(dāng)然在5.7中為了支持MTS其中還封裝了last_commit/sequence_number。那么使用infobin工具查看一個insert單條語句完整事物的event包括如下:
>Gtid Event:Pos:234(0Xea) N_pos:299(0X12b) Time:1513135186 Event_size:65(bytes) Gtid:31704d8a-da74-11e7-b6bf-52540a7d243:100009 last_committed=0 sequence_number=1 -->Query Event:Pos:299(0X12b) N_Pos:371(0X173) Time:1513135186 Event_size:72(bytes) Exe_time:0 Use_db:test Statment(35b-trun):BEGIN /*!Trx begin!*/ Gno:100009 ---->Map Event:Pos371(0X173) N_pos:415(0X19f) Time:1513135186 Event_size:44(bytes) TABLE_ID:108 DB_NAME:test TABLE_NAME:a Gno:100009 ------>Insert Event:Pos:415(0X19f) N_pos:455(0X1c7) Time:1513135186 Event_size:40(bytes) Dml on table: test.a table_id:108 Gno:100009 >Xid Event:Pos:455(0X1c7) N_Pos:486(0X1e6) Time:1513135186 Event_size:31(bytes) COMMIT; /*!Trx end*/ Gno:100009
當(dāng)然也可以使用mysqlbinlog進行分析,只是格式稍微不那么友好。
這一部分是重點中的重點,也是我以前一直疑惑的,請大家細(xì)細(xì)品讀。
官方文檔這樣描述gtid_executed表
Beginning with MySQL 5.7.5, GTIDs are stored in a table named gtid_executed, in the mysql database. A row in this table contains, for each GTID or set of GTIDs that it represents, the UUID of the originating server, and the starting and ending transaction IDs of the set; for a row referencing only a single GTID, these last two values are the same.
也就是說gtid_executed表是Gtid持久化的一個工具,如前文所描述Gtid_state中的get_executed_gtids/get_lost_gtids/get_gtids_only_in_table/get_previous_gtids_logged這些數(shù)據(jù)都是存儲在內(nèi)存中的,那么在數(shù)據(jù)庫重啟后需要進行初始化,那么這需要讀取Gtid持久化的介質(zhì),我們可以發(fā)現(xiàn)gtid_executed是一個innodb表建表語句如下,并且我們可以手動更改它,但是千萬不要這么干:
Table: gtid_executed Create Table: CREATE TABLE `gtid_executed` ( `source_uuid` char(36) NOT NULL COMMENT 'uuid of the source where the transaction was originally executed.', `interval_start` bigint(20) NOT NULL COMMENT 'First number of interval.', `interval_end` bigint(20) NOT NULL COMMENT 'Last number of interval.', PRIMARY KEY (`source_uuid`,`interval_start`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 STATS_PERSISTENT=0
那么在5.7.5以前沒有g(shù)tid_executed表不是也沒有問題嗎?其實除了gtid_executed表以外我們還有一個Gtid持久化的介質(zhì)那就是binlog中的Gtid event。所以總結(jié)一下Gtid持久化介質(zhì):
那么既然有了binlog的gtid event進行持久化那么為什么還需要gtid_executed表呢?這實際上就是5.7.5過后的一個優(yōu)化,我們可以反過來思考在5.6中如果使用了Gtid做從庫,從庫如果不開啟binlog并且同時設(shè)置log_slave_updates=ture那么從庫的執(zhí)行過的Gtid事物是沒有辦法持久化的。我們來一段5.6官方文檔對于搭建Gtid從庫的其中一步:
Step 3: Restart both servers with GTIDs enabled. To enable binary logging with global transaction identifiers, each server must be started with GTID mode, binary logging, slave update logging enabled, and with statements that are unsafe for GTID-based replication disabled. In addition, you should prevent unwanted or accidental updates from being performed on either server by starting both in read-only mode. This means that both servers must be started with (at least) the options shown in the following invocation of mysqld_safe: shell> mysqld_safe --gtid_mode=ON --log-bin --log-slave-updates --enforce-gtid-consistency &
開啟binlog同時設(shè)置設(shè)置log_slave_updates=ture必然造成一個問題,實際上從庫很多時候我們是不需要做級聯(lián)slave,設(shè)置log_slave_updates=ture會造成需要額外的空間和性能開銷。自然這種情況下我們需要另外的一種Gtid持久化介質(zhì),而并不是binlog中的Gtid event。為了解決這個問題,5.7中g(shù)tid_executed表應(yīng)運而生了。然而gtid_executed表是否需要實時更新呢?顯然在slave端不開啟binlog或者開啟binlog不設(shè)置log_slave_updates=ture的情況下它需要實時更新,因為I/O thread執(zhí)行過得Gtid是必須持久化的,而在主庫上因為有binlog的Gtid event的存在他是不需要實時更新的,這樣不同的對待方式也能夠減輕負(fù)擔(dān)提高性能。
同時在官方文檔上也有相關(guān)描述它分為是否開始binlog進行描述,但是其描述并不是最詳細(xì)的。所以這部分在后面我會進行詳細(xì)描述。
Previous gtid Event是包含在每一個binlog的開頭用于描述所有以前binlog所包含的全部Gtid的一個集合(包括已經(jīng)刪除的binlog)如:
da267088-9c22-11e7-ab56-5254008768e3:1-32
在5.6中如果不開啟Gtid,那么binlog是不會包含這個Previous gtid Event的,但是在5.7中不開啟Gtid也會包含這個Previous gtid Event,實際這一點的改變其意義也是非常巨大,簡單的說他為快速掃描binlog(binlog_gtid_simple_recovery=ture)獲得正確Gtid集合提供了基礎(chǔ),否則將會掃描大量的binlog,從而浪費I/O性能,這是5.6中一個非常嚴(yán)重的問題,在5.7的官方文檔這樣描述:
When binlog_gtid_simple_recovery=TRUE, which is the default in MySQL 5.7.7 and later, the server iterates only the oldest and the newest binary log files and the values of gtid_purged and gtid_executed are computed based only on Previous_gtids_log_event or Gtid_log_event found in these files. This ensures only two binary log files are iterated during server restart or when binary logs are being purged
當(dāng)然這部分也會在后面進行詳細(xì)的描述,這里只是簡單提一下。那么我們通過mysqlbinlog 和infobin工具分別確認(rèn)這一點。
mysqlbinlog:
*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #171211 16:20:10 server id 20155 end_log_pos 120 CRC32 0x12617db7 Start: binlog v 4, server v 5.6.26-74.0-log created 171211 16:20:10 # Warning: this binlog is either in use or was not closed properly. # at 120 #171211 16:20:14 server id 20155 end_log_pos 192 CRC32 0x696752cb Query thread_id=30
infobin:
------------Detail now-------------- >Format description log Event:Pos:4(0X4) N_pos:120(0X78) Time:1512980410 Event_size:116(bytes) -->Query Event:Pos:120(0X78) N_Pos:192(0Xc0) Time:1512980414 Event_size:72(bytes) Exe_time:0 Use_db:test Statment(35b-trun):BEGIN /*!Trx begin!*/ Gno:0 ---->Map Event:Pos192(0Xc0) N_pos:241(0Xf1) Time:1512980414 Event_size:49(bytes) TABLE_ID:91 DB_NAME:test TABLE_NAME:testpo Gno:0 ------>Insert Event:Pos:241(0Xf1) N_pos:281(0X119) Time:1512980414 Event_size:40(bytes) Dml on table: test.testpo table_id:91 Gno:0 >Xid Event:Pos:281(0X119) N_Pos:312(0X138) Time:1512980414 Event_size:31(bytes) COMMIT; /*!Trx end*/ Gno:0
我們并沒有發(fā)現(xiàn)Previous gtid Event,也就是5.6如果不開啟Gtid則不包含Previous gtid Event。
mysqlbinlog:
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #171211 16:26:49 server id 1 end_log_pos 123 CRC32 0xf9a36298 Start: binlog v 4, server v 5.7.14-7-debug-log created 171211 16:26:49 # Warning: this binlog is either in use or was not closed properly. # at 123 #171211 16:26:49 server id 1 end_log_pos 194 CRC32 0x5865633f **Previous-GTIDs** # da267088-9c22-11e7-ab56-5254008768e3:1-32 # at 194
infobin:
------------Detail now-------------- >Format description log Event:Pos:4(0X4) N_pos:123(0X7b) Time:1512980809 Event_size:119(bytes) >Previous gtid Event:Pos:123(0X7b) N_pos:194(0Xc2) Time:1512980809 Event_size:71(bytes) >Anonymous gtid Event:Pos:194(0Xc2) N_pos:259(0X103) Time:1512980814 Event_size:65(bytes) Gtid:Anonymous(Gno=0) last_committed=0 sequence_number=1 -->Query Event:Pos:259(0X103) N_Pos:331(0X14b) Time:1512980814 Event_size:72(bytes) Exe_time:0 Use_db:test Statment(35b-trun):BEGIN /*!Trx begin!*/ Gno:0 ---->Map Event:Pos331(0X14b) N_pos:380(0X17c) Time:1512980814 Event_size:49(bytes) TABLE_ID:154 DB_NAME:test TABLE_NAME:testpo Gno:0 ------>Insert Event:Pos:380(0X17c) N_pos:420(0X1a4) Time:1512980814 Event_size:40(bytes) Dml on table: test.testpo table_id:154 Gno:0 >Xid Event:Pos:420(0X1a4) N_Pos:451(0X1c3) Time:1512980814 Event_size:31(bytes) COMMIT; /*!Trx end*/ Gno:0
我們清晰的看到這里包含了Previous gtid Event,當(dāng)然我們還發(fā)現(xiàn)了Anonymous gtid Event這也是5.7中變化,5.7中即使不開始Gtid每個事物也包含也一個Anonymous gtid Event,雖然沒有Gtid但是它任然包含了 last_committed/sequence_number。
學(xué)習(xí)完本節(jié)至少能夠?qū)W習(xí)到:
作者微信: