本次性能調(diào)優(yōu)項(xiàng)目中由于涉及的環(huán)節(jié)較多，最好能夠?qū)⑸森h(huán)境中的所有內(nèi)容進(jìn)行監(jiān)控，同時(shí)考慮最低開(kāi)銷，這樣就從應(yīng)用服務(wù)器和數(shù)據(jù)庫(kù)服務(wù)器兩個(gè)服務(wù)器進(jìn)行，以nmon作為監(jiān)控基礎(chǔ)數(shù)據(jù)，同時(shí)監(jiān)控JVM和數(shù)據(jù)庫(kù)告警和快照。
所有監(jiān)控的內(nèi)容都是手段，只有從海量的監(jiān)控日志中得到規(guī)律性、有意義的數(shù)據(jù)才是性能優(yōu)化的基礎(chǔ)。有了數(shù)據(jù)就是對(duì)數(shù)據(jù)的分析，本文將首先介紹需要獲取的數(shù)據(jù)，內(nèi)容也將是我從項(xiàng)目獲取的經(jīng)驗(yàn)。
基礎(chǔ)環(huán)境：
紅安ssl適用于網(wǎng)站、小程序/APP、API接口等需要進(jìn)行數(shù)據(jù)傳輸應(yīng)用場(chǎng)景，ssl證書(shū)未來(lái)市場(chǎng)廣闊！成為成都創(chuàng)新互聯(lián)的ssl證書(shū)銷售渠道，可以享受市場(chǎng)價(jià)格4-6折優(yōu)惠！如果有意向歡迎電話聯(lián)系或者加微信：18980820575（備注：SSL證書(shū)合作）期待與您的合作！
兩臺(tái)數(shù)據(jù)庫(kù)服務(wù)器，做的數(shù)據(jù)庫(kù)集群。
項(xiàng)目中主要使用tongweb（老系統(tǒng)版本很低），監(jiān)控內(nèi)容類似如下：
重慶分公司
重慶分公司
...
"2018-01-11T02:25:55.663+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnCreated","10",
"2018-01-11T02:25:55.663+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnAcquired","111292",
"2018-01-11T02:25:55.663+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnNotSuccessfullyMatched","0",
"2018-01-11T02:26:25.670+0800","com.tongtech.tongweb:type=jvm,category=monitor,server=server","UpTime","222520621",
"2018-01-11T02:26:25.670+0800","com.tongtech.tongweb:type=jvm,category=monitor,server=server","HeapSize","2143485952",
"2018-01-11T02:26:25.671+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnUsed","0",
"2018-01-11T02:26:25.671+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnSuccessfullyMatched","0",
"2018-01-11T02:26:25.671+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","WaitQueueLength","0",
"2018-01-11T02:26:25.671+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnDestroyed","0",
"2018-01-11T02:26:25.671+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","ConnRequestWaitTime","4",
"2018-01-11T02:26:25.672+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnFailedValidation","0",
"2018-01-11T02:26:25.672+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnReleased","111292",
"2018-01-11T02:26:25.672+0800","com.tongtech.tongweb:name=***,type=jdbc-connection-pool,category=monitor,server=server","NumConnFree","10",
...tongweb的監(jiān)控?cái)?shù)據(jù)獲取連接池狀態(tài)等信息，我們的方法是通過(guò)Excel宏的方式將日志內(nèi)轉(zhuǎn)換成可讀數(shù)據(jù)，并進(jìn)行圖形分析。具體內(nèi)容將單獨(dú)說(shuō)明。
JVM線程監(jiān)控說(shuō)明
通過(guò)對(duì)tongweb的JVM監(jiān)控，可初步判定性能高峰時(shí)間點(diǎn)、連接池是否滿，同時(shí)進(jìn)一步判定連接高峰期的性能瓶頸是否出現(xiàn)在應(yīng)用上，這對(duì)今后的性能分析尤為重要，可將主要性能問(wèn)題歸類，減少不必要的工作。
在Internet RFC標(biāo)準(zhǔn)中，Netstat的定義是： Netstat是在內(nèi)核中訪問(wèn)網(wǎng)絡(luò)連接狀態(tài)及其相關(guān)信息的程序，它能提供TCP連接，TCP和UDP監(jiān)聽(tīng)，進(jìn)程內(nèi)存管理的相關(guān)報(bào)告。
以下是在項(xiàng)目中獲取的日志摘取
...
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:139 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:427 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:427 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:58862 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:2544 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:631 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:445 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:669 0.0.0.0:* LISTEN
...作為本次性能優(yōu)化主要的分析手段，nmon起著尤為重要的作用，以下是wiki的解釋，有時(shí)間可以了解
nmon collects the following operating system statistics:
CPU and CPU threads Utilisation
CPU frequency for servers or virtual machines that can alter their clock rate
GPU stats including utilisation, MHz and temperatures
Physical and Virtual Memory use
Disk read & write and transfers
Disk Groups decided by the user
Swap and Paging
Network read & write and transfers
Local File-systems
Network File-system (NFS)
Top Processes by CPU use, Memory size and I/O rates
Kernel stats including Run Queue, context-switch, fork, Load Average & Uptime
Large and Huge memory pages
Virtual Machine stats (depending on the hardware) - useful for Linux running KVM to host virtual machines
Resources in the Server and virtual machine
總結(jié)其實(shí)nmon更像是系統(tǒng)性能開(kāi)銷的快照，結(jié)合對(duì)nmon的分析工具可以很清楚的掌握系統(tǒng)的各項(xiàng)指標(biāo)。
下載分析工具
了解數(shù)據(jù)庫(kù)的告警日志也是掌握當(dāng)前性能的關(guān)鍵環(huán)節(jié)。
日志如下，如出現(xiàn)error可以針對(duì)具體情況進(jìn)行分析解決。
2018-01-11-00.36.36.090562+480 I13363168A459 LEVEL: Error
PID : 2228842 TID : 142490 PROC : db2sysc
INSTANCE: db2 NODE : 000 DB : TRADE
EDUID : 142490 EDUNAME: db2agent (**) 0
FUNCTION: DB2 UDB, Query Gateway, sqlqg_fedstp_hook, probe:40
MESSAGE : Unexpected error returned from outer RC=
DATA #1 : Hexdump, 4 bytes
0x07000007053F28D0 : 8126 0012 .&..
數(shù)據(jù)庫(kù)日志快照將作為主要分析依據(jù)，在快照中可以分析數(shù)據(jù)庫(kù)時(shí)間的開(kāi)銷情況，如下：
...
Number of automatic storage paths = 1
Automatic storage path = /db2data
Node number = 0
State = In Use
File system ID = 9223372079804448776
Storage path free space (bytes) = 69730709504
File system used space (bytes) = 139648946176
File system total space (bytes) = 209379655680
...
本文只是列出了分析的方法，具體操作有時(shí)間我會(huì)慢慢總結(jié)。
工具的利用固然重要，但是性能調(diào)優(yōu)并不是僅僅如此，必須步步為營(yíng)做好長(zhǎng)期作戰(zhàn)的準(zhǔn)備。
