如下所示:
創(chuàng)新互聯(lián)建站是專業(yè)的紅安網(wǎng)站建設(shè)公司,紅安接單;提供網(wǎng)站設(shè)計制作、網(wǎng)站設(shè)計,網(wǎng)頁設(shè)計,網(wǎng)站設(shè)計,建網(wǎng)站,PHP網(wǎng)站建設(shè)等專業(yè)做網(wǎng)站服務(wù);采用PHP框架,可快速的進(jìn)行紅安網(wǎng)站開發(fā)網(wǎng)頁制作和功能擴(kuò)展;專業(yè)做搜索引擎喜愛的網(wǎng)站,專業(yè)的做網(wǎng)站團(tuán)隊(duì),希望更多企業(yè)前來合作!""" 提取文檔數(shù)超過10000的數(shù)據(jù) 按照某個字段的值具有唯一性進(jìn)行升序, 按照@timestamp進(jìn)行降序, 第一次查詢,先將10000條數(shù)據(jù)取出, 取出最后一個時間戳, 在第二次查詢中,設(shè)定@timestamp小于將第一次得到的最后一個時間戳, 同時設(shè)定某個字段的值具有唯一性進(jìn)行升序, 按照@timestamp進(jìn)行降序, """ from elasticsearch import Elasticsearch import os write_path = "E:\\公司\\案例數(shù)據(jù)采集\\olt告警案例分析\\10000_data.txt" es = Elasticsearch(hosts="", timeout=1500) write_file = open(write_path, "a+") def _first_query(): index_ = "gather-010" _source = ["TWICE_BOOK_TIME", "@timestamp"] try: rs = es.search(index=index_, body={ "size": 10000, "query": { "match_all": {} }, "sort": [ { "@timestamp": { "order": "desc" } }, { "TASK_RECEIVE_ID.keyword": { "order": "asc" } } ], "_source": _source }) return rs except: raise Exception("{0} search error".format(index_)) def _get_first_data(first_rs): i = 0 if first_rs: for hit in first_rs['hits']['hits']: IptvAccount = hit['_source']['TWICE_BOOK_TIME'] timestamp = hit['_source']['@timestamp'] if IptvAccount is None: IptvAccount = "" write_file.write(IptvAccount + "," + timestamp + "\n") i += 1 if i == 10000: return timestamp def _second_query(timestamp): index_ = "gather-010" _source = ["TWICE_BOOK_TIME", "@timestamp"] try: rs = es.search(index=index_, body={ "size": 10000, "query": { "bool": { "filter": { "range": { "@timestamp": { "lt": timestamp } } } } }, "sort": [ { "@timestamp": { "order": "desc" } }, { "TASK_RECEIVE_ID.keyword": { "order": "asc" } } ], "_source": _source }) return rs except: raise Exception("{0} search error".format(index_)) if __name__ == "__main__": first_rs = _first_query() first_timestamp = _get_first_data(first_rs) print(first_timestamp) while True: second_rs = _second_query(first_timestamp) first_timestamp = _get_first_data(second_rs) if first_timestamp is None: break print(first_timestamp)