producer是線程安全的，多線程分享共用一個producer比用多個 producer整體上要快

網(wǎng)站建設(shè)公司,為您提供網(wǎng)站建設(shè),網(wǎng)站制作,網(wǎng)頁設(shè)計及定制網(wǎng)站建設(shè)服務(wù),專注于企業(yè)網(wǎng)站制作,高端網(wǎng)頁制作,對自上料攪拌車等多個行業(yè)擁有豐富的網(wǎng)站建設(shè)經(jīng)驗的網(wǎng)站建設(shè)公司。專業(yè)網(wǎng)站設(shè)計,網(wǎng)站優(yōu)化推廣哪家好,專業(yè)seo優(yōu)化優(yōu)化,H5建站,響應(yīng)式網(wǎng)站。

如果你想了解大數(shù)據(jù)的學(xué)習(xí)路線，想學(xué)習(xí)大數(shù)據(jù)知識以及需要免費的學(xué)習(xí)資料可以加群：784789432.歡迎你的加入。每天下午三點開直播分享基礎(chǔ)知識，晚上20:00都會開直播給大家分享大數(shù)據(jù)項目實戰(zhàn)。

可以命令行查看某consumer group所有consumer的offset，和落后的多少，也就是說可以查看Kafka中數(shù)據(jù)的堆積情況，以下來自官方文檔

Sometimes it's useful to see the position of your consumers. We have a tool that will show the position of all consumers in a consumer group as well as how far behind the end of the log they are. To run this tool on a consumer group named my-group consuming a topic named my-topic would look like this:

bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group

Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).

TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
my-topic 0 2 4 2 consumer-1-029af89c-873c-4751-a720-cefd41a669d6 /127.0.0.1 consumer-1
my-topic 1 2 3 1 consumer-1-029af89c-873c-4751-a720-cefd41a669d6 /127.0.0.1 consumer-1
my-topic 2 2 3 1 consumer-2-42c1abd4-e3b2-425d-a8bb-e1ea49b29bb2 /127.0.0.1 consumer-2

This tool also works with ZooKeeper-based consumers:

bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group my-group

Note: This will only show information about consumers that use ZooKeeper (not those using the Java consumer API).

TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID
my-topic 0 2 4 2 my-group_consumer-1
my-topic 1 2 3 1 my-group_consumer-1
my-topic 2 2 3 1 my-group_consumer-2

retention

Kafka保留消費后的數(shù)據(jù)，但也不是永遠(yuǎn)保留，默認(rèn)7天后，會自動刪除數(shù)據(jù)。當(dāng)然我們可以設(shè)置保留（retention）幾天，由broker config （也就是server.properties）中的log.retention.ms、log.retention.minutes、log.rentention.hours設(shè)置，優(yōu)先級遞增。默認(rèn)log.rentention.hours=168。

還有一種retention的設(shè)置方法，就是log.retention.bytes，也是server.properties中設(shè)置，定義了一個partition存儲的最大大小。兩種方法一種滿足就會刪除。

kafka刪除是按照segment來刪，一次只能整個刪掉一個或多個segment。

另外也可以設(shè)置每個topic的retention情況，具體看官方文檔http://kafka.apache.org/documentation/

See http://kafka.apache.org/documentation/#brokerconfigs for the full list of log.retention./log.roll./log.segment.* configs

auto.offset.reset

What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted):
earliest: automatically reset the offset to the earliest offset
latest: automatically reset the offset to the latest offset
none: throw exception to the consumer if no previous offset is found for the consumer's group
anything else: throw exception to the consumer.

default: latest

每一個topic記錄offset，offset屬于不同的group，一個group只有一個offset（每個partition）