這篇文章將為大家詳細(xì)講解有關(guān)Uber jvm profiler如何使用，小編覺(jué)得挺實(shí)用的，因此分享給大家做個(gè)參考，希望大家閱讀完這篇文章后可以有所收獲。
專(zhuān)注于為中小企業(yè)提供網(wǎng)站設(shè)計(jì)制作、網(wǎng)站建設(shè)服務(wù),電腦端+手機(jī)端+微信端的三站合一,更高效的管理,為中小企業(yè)霍邱免費(fèi)做網(wǎng)站提供優(yōu)質(zhì)的服務(wù)。我們立足成都，凝聚了一批互聯(lián)網(wǎng)行業(yè)人才，有力地推動(dòng)了上千多家企業(yè)的穩(wěn)健成長(zhǎng)，幫助中小企業(yè)通過(guò)網(wǎng)站建設(shè)實(shí)現(xiàn)規(guī)模擴(kuò)充和轉(zhuǎn)變。
uber jvm profiler是用于在分布式監(jiān)控收集jvm 相關(guān)指標(biāo)，如:cpu/memory/io/gc信息等
確保安裝了maven和JDK>=8前提下，直接mvn clean package
說(shuō)明
直接以java agent的部署就可以使用
使用
java -javaagent:jvm-profiler-1.0.0.jar=reporter=com.uber.profiling.reporters.KafkaOutputReporter,brokerList='kafka1:9092',topicPrefix=demo_,tag=tag-demo,metricInterval=5000,sampleInterval=0 -cp target/jvm-profiler-1.0.0.jar
選項(xiàng)解釋
| 參數(shù) | 說(shuō)明 |
|---|---|
| reporter | reporter類(lèi)別, 此處直接默認(rèn)為com.uber.profiling.reporters.KafkaOutputReporter就可以 |
| brokerList | 如reporter為com.uber.profiling.reporters.KafkaOutputReporter,則brokerList為kafka列表,以逗號(hào)分隔 |
| topicPrefix | 如reporter為com.uber.profiling.reporters.KafkaOutputReporter,則topicPrefix為kafka topic的前綴 |
| tag | key為tag的metric，會(huì)輸出到reporter中 |
| metricInterval | metric report的頻率，根據(jù)實(shí)際情況設(shè)置，單位為ms |
| sampleInterval | jvm堆棧metrics report的頻率，根據(jù)實(shí)際情況設(shè)置，單位為ms |
結(jié)果展示
重慶分公司
重慶分公司
"nonHeapMemoryTotalUsed": 11890584.0,
"bufferPools": [
{
"totalCapacity": 0,
"name": "direct",
"count": 0,
"memoryUsed": 0
},
{
"totalCapacity": 0,
"name": "mapped",
"count": 0,
"memoryUsed": 0
}
],
"heapMemoryTotalUsed": 24330736.0,
"epochMillis": 1515627003374,
"nonHeapMemoryCommitted": 13565952.0,
"heapMemoryCommitted": 257425408.0,
"memoryPools": [
{
"peakUsageMax": 251658240,
"usageMax": 251658240,
"peakUsageUsed": 1194496,
"name": "Code Cache",
"peakUsageCommitted": 2555904,
"usageUsed": 1173504,
"type": "Non-heap memory",
"usageCommitted": 2555904
},
{
"peakUsageMax": -1,
"usageMax": -1,
"peakUsageUsed": 9622920,
"name": "Metaspace",
"peakUsageCommitted": 9830400,
"usageUsed": 9622920,
"type": "Non-heap memory",
"usageCommitted": 9830400
},
{
"peakUsageMax": 1073741824,
"usageMax": 1073741824,
"peakUsageUsed": 1094160,
"name": "Compressed Class Space",
"peakUsageCommitted": 1179648,
"usageUsed": 1094160,
"type": "Non-heap memory",
"usageCommitted": 1179648
},
{
"peakUsageMax": 1409286144,
"usageMax": 1409286144,
"peakUsageUsed": 24330736,
"name": "PS Eden Space",
"peakUsageCommitted": 67108864,
"usageUsed": 24330736,
"type": "Heap memory",
"usageCommitted": 67108864
},
{
"peakUsageMax": 11010048,
"usageMax": 11010048,
"peakUsageUsed": 0,
"name": "PS Survivor Space",
"peakUsageCommitted": 11010048,
"usageUsed": 0,
"type": "Heap memory",
"usageCommitted": 11010048
},
{
"peakUsageMax": 2863661056,
"usageMax": 2863661056,
"peakUsageUsed": 0,
"name": "PS Old Gen",
"peakUsageCommitted": 179306496,
"usageUsed": 0,
"type": "Heap memory",
"usageCommitted": 179306496
}
],
"processCpuLoad": 0.0008024004394748531,
"systemCpuLoad": 0.23138430784607697,
"processCpuTime": 496918000,
"appId": null,
"name": "24103@machine01",
"host": "machine01",
"processUuid": "3c2ec835-749d-45ea-a7ec-e4b9fe17c23a",
"tag": "mytag",
"gc": [
{
"collectionTime": 0,
"name": "PS Scavenge",
"collectionCount": 0
},
{
"collectionTime": 0,
"name": "PS MarkSweep",
"collectionCount": 0
}
]
}說(shuō)明
和java應(yīng)用不同，需要把jvm-profiler.jar分發(fā)到各個(gè)節(jié)點(diǎn)上
使用
--jars hdfs:///public/libs/jvm-profiler-1.0.0.jar --conf spark.driver.extraJavaOptions=-javaagent:jvm-profiler-1.0.0.jar=reporter=com.uber.profiling.reporters.KafkaOutputReporter,brokerList='kafka1:9092',topicPrefix=demo_,tag=tag-demo,metricInterval=5000,sampleInterval=0 --conf spark.executor.extraJavaOptions=-javaagent:jvm-profiler-1.0.0.jar=reporter=com.uber.profiling.reporters.KafkaOutputReporter,brokerList='kafka1:9092',topicPrefix=demo_,tag=tag-demo,metricInterval=5000,sampleInterval=0
選項(xiàng)解釋
| 參數(shù) | 說(shuō)明 |
|---|---|
| reporter | reporter類(lèi)別, 此處直接默認(rèn)為com.uber.profiling.reporters.KafkaOutputReporter就可以 |
| brokerList | 如reporter為com.uber.profiling.reporters.KafkaOutputReporter,則brokerList為kafka列表,以逗號(hào)分隔 |
| topicPrefix | 如reporter為com.uber.profiling.reporters.KafkaOutputReporter,則topicPrefix為kafka topic的前綴 |
| tag | key為tag的metric，會(huì)輸出到reporter中 |
| metricInterval | metric report的頻率，根據(jù)實(shí)際情況設(shè)置，單位為ms |
| sampleInterval | jvm堆棧metrics report的頻率，根據(jù)實(shí)際情況設(shè)置，單位為ms |
結(jié)果展示
"nonHeapMemoryTotalUsed": 11890584.0,
"bufferPools": [
{
"totalCapacity": 0,
"name": "direct",
"count": 0,
"memoryUsed": 0
},
{
"totalCapacity": 0,
"name": "mapped",
"count": 0,
"memoryUsed": 0
}
],
"heapMemoryTotalUsed": 24330736.0,
"epochMillis": 1515627003374,
"nonHeapMemoryCommitted": 13565952.0,
"heapMemoryCommitted": 257425408.0,
"memoryPools": [
{
"peakUsageMax": 251658240,
"usageMax": 251658240,
"peakUsageUsed": 1194496,
"name": "Code Cache",
"peakUsageCommitted": 2555904,
"usageUsed": 1173504,
"type": "Non-heap memory",
"usageCommitted": 2555904
},
{
"peakUsageMax": -1,
"usageMax": -1,
"peakUsageUsed": 9622920,
"name": "Metaspace",
"peakUsageCommitted": 9830400,
"usageUsed": 9622920,
"type": "Non-heap memory",
"usageCommitted": 9830400
},
{
"peakUsageMax": 1073741824,
"usageMax": 1073741824,
"peakUsageUsed": 1094160,
"name": "Compressed Class Space",
"peakUsageCommitted": 1179648,
"usageUsed": 1094160,
"type": "Non-heap memory",
"usageCommitted": 1179648
},
{
"peakUsageMax": 1409286144,
"usageMax": 1409286144,
"peakUsageUsed": 24330736,
"name": "PS Eden Space",
"peakUsageCommitted": 67108864,
"usageUsed": 24330736,
"type": "Heap memory",
"usageCommitted": 67108864
},
{
"peakUsageMax": 11010048,
"usageMax": 11010048,
"peakUsageUsed": 0,
"name": "PS Survivor Space",
"peakUsageCommitted": 11010048,
"usageUsed": 0,
"type": "Heap memory",
"usageCommitted": 11010048
},
{
"peakUsageMax": 2863661056,
"usageMax": 2863661056,
"peakUsageUsed": 0,
"name": "PS Old Gen",
"peakUsageCommitted": 179306496,
"usageUsed": 0,
"type": "Heap memory",
"usageCommitted": 179306496
}
],
"processCpuLoad": 0.0008024004394748531,
"systemCpuLoad": 0.23138430784607697,
"processCpuTime": 496918000,
"appId": null,
"name": "24103@machine01",
"host": "machine01",
"processUuid": "3c2ec835-749d-45ea-a7ec-e4b9fe17c23a",
"tag": "mytag",
"gc": [
{
"collectionTime": 0,
"name": "PS Scavenge",
"collectionCount": 0
},
{
"collectionTime": 0,
"name": "PS MarkSweep",
"collectionCount": 0
}
]
}已有的reporter
| reporter | 說(shuō)明 |
|---|---|
| ConsoleOutputReporter | 默認(rèn)的repoter，一般用于調(diào)試 |
| FileOutputReporter | 基于文件的reporter,分布式環(huán)境下不適用，得設(shè)置outputDir |
| KafkaOutputReporter | 基于kafka的reporter，正式環(huán)境用的多，得設(shè)置brokerList，topicPrefix |
| GraphiteOutputReporter | 基于Graphite的reporter,需設(shè)置graphite.host等配置 |
| redisOutputReporter | 基于redis的reporter，構(gòu)建命令 mvn -P redis clean package |
| InfluxDBOutputReporter | 基于InfluxDB的reporter，構(gòu)建命令mvn -P influxdb clean package，需設(shè)置influxdb.host等配置 |
建議在生產(chǎn)環(huán)境下使用KafkaOutputReporter，操作靈活性高，可以結(jié)合clickhouse grafana進(jìn)行指標(biāo)展示
源碼分析
該jvm-profiler整體是基于java agent實(shí)現(xiàn),項(xiàng)目pom文件 指定了MANIFEST.MF中的Premain-Class項(xiàng)和Agent-Class為com.uber.profiling.Agent 具體的實(shí)現(xiàn)類(lèi)為AgentImpl
就具體的AgentImpl類(lèi)的run方法來(lái)進(jìn)行分析
public void run(Arguments arguments, Instrumentation instrumentation, CollectionobjectsToCloseOnShutdown) { if (arguments.isNoop()) { logger.info("Agent noop is true, do not run anything"); return; } Reporter reporter = arguments.getReporter(); String processUuid = UUID.randomUUID().toString(); String appId = null; String appIdVariable = arguments.getAppIdVariable(); if (appIdVariable != null && !appIdVariable.isEmpty()) { appId = System.getenv(appIdVariable); } if (appId == null || appId.isEmpty()) { appId = SparkUtils.probeAppId(arguments.getAppIdRegex()); } if (!arguments.getDurationProfiling().isEmpty() || !arguments.getArgumentProfiling().isEmpty()) { instrumentation.addTransformer(new JavaAgentFileTransformer(arguments.getDurationProfiling(), arguments.getArgumentProfiling())); } List profilers = createProfilers(reporter, arguments, processUuid, appId); ProfilerGroup profilerGroup = startProfilers(profilers); Thread shutdownHook = new Thread(new ShutdownHookRunner(profilerGroup.getPeriodicProfilers(), Arrays.asList(reporter), objectsToCloseOnShutdown)); Runtime.getRuntime().addShutdownHook(shutdownHook); }
arguments.getReporter() 獲取reporter，如果沒(méi)有設(shè)置則設(shè)置為reporterConstructor,否則設(shè)置為指定的reporter
String appId ,設(shè)置appId，首先從配置中查找，如果沒(méi)有設(shè)置，再?gòu)膃nv中查找，對(duì)于spark應(yīng)用則取spark.app.id的值
List
1.其中CpuAndMemoryProfiler，ThreadInfoProfiler，ProcessInfoProfiler是從JMX中讀取數(shù)據(jù)，ProcessInfoProfiler還會(huì)從 /pro讀取數(shù)據(jù)；
2.如果設(shè)置了durationProfiling，argumentProfiling，sampleInterval，ioProfiling，則會(huì)增加對(duì)應(yīng)的MethodDurationProfiler(輸出方法調(diào)用花費(fèi)的時(shí)間)，MethodArgumentProfiler(輸出方法參數(shù)的值)，StacktraceReporterProfiler，IOProfiler；
3.MethodArgumentProfiler和MethodDurationProfiler利用javassist第三方字節(jié)碼編譯工具來(lái)改寫(xiě)對(duì)應(yīng)的類(lèi)，具體實(shí)現(xiàn)參照J(rèn)avaAgentFileTransformer
4.StacktraceReporterProfiler從JMX中讀取數(shù)據(jù)
5.IOProfiler則是讀取本地機(jī)器上的/pro文件對(duì)應(yīng)的目錄的數(shù)據(jù)
ProfilerGroup profilerGroup = startProfilers(profilers) 開(kāi)始進(jìn)行profiler的定時(shí)report
其中還會(huì)區(qū)分oneTimeProfilers和periodicProfilers，ProcessInfoProfiler就屬于oneTimeProfilers，因?yàn)閜rocess的信息，在運(yùn)行期間是不會(huì)變的，不需要周期行的reporter
至此，整個(gè)流程結(jié)束
關(guān)于“Uber jvm profiler如何使用”這篇文章就分享到這里了，希望以上內(nèi)容可以對(duì)大家有一定的幫助，使各位可以學(xué)到更多知識(shí)，如果覺(jué)得文章不錯(cuò)，請(qǐng)把它分享出去讓更多的人看到。
