redis如何設(shè)置過期時間?相信很多新手小白還沒學(xué)會這個技能,通過這篇文章的總結(jié),希望你能學(xué)會這個技能。以下資料是實現(xiàn)的步驟。
創(chuàng)新互聯(lián)是一家專業(yè)提供高密企業(yè)網(wǎng)站建設(shè),專注與成都網(wǎng)站建設(shè)、成都做網(wǎng)站、H5建站、小程序制作等業(yè)務(wù)。10年已為高密眾多企業(yè)、政府機構(gòu)等服務(wù)。創(chuàng)新互聯(lián)專業(yè)網(wǎng)站設(shè)計公司優(yōu)惠進行中。
1、Redis中key的過期時間
通過EXPIRE key seconds命令來設(shè)置數(shù)據(jù)的過期時間。返回1表明設(shè)置成功,返回0表明key不存在或者不能成功設(shè)置過期時間。在key上設(shè)置了過期時間后key將在指定的秒數(shù)后被自動刪除。被指定了過期時間的key在Redis中被稱為是不穩(wěn)定的。
當(dāng)key被DEL命令刪除或者被SET、GETSET命令重置后與之關(guān)聯(lián)的過期時間會被清除
127.0.0.1:6379> setex s 20 1 OK 127.0.0.1:6379> ttl s (integer) 17 127.0.0.1:6379> setex s 200 1 OK 127.0.0.1:6379> ttl s (integer) 195 127.0.0.1:6379> setrange s 3 100 (integer) 6 127.0.0.1:6379> ttl s (integer) 152 127.0.0.1:6379> get s "1\x00\x00100" 127.0.0.1:6379> ttl s (integer) 108 127.0.0.1:6379> getset s 200 "1\x00\x00100" 127.0.0.1:6379> get s "200" 127.0.0.1:6379> ttl s (integer) -1
使用PERSIST可以清除過期時間
127.0.0.1:6379> setex s 100 test OK 127.0.0.1:6379> get s "test" 127.0.0.1:6379> ttl s (integer) 94 127.0.0.1:6379> type s string 127.0.0.1:6379> strlen s (integer) 4 127.0.0.1:6379> persist s (integer) 1 127.0.0.1:6379> ttl s (integer) -1 127.0.0.1:6379> get s "test"
使用rename只是改了key值
127.0.0.1:6379> expire s 200 (integer) 1 127.0.0.1:6379> ttl s (integer) 198 127.0.0.1:6379> rename s ss OK 127.0.0.1:6379> ttl ss (integer) 187 127.0.0.1:6379> type ss string 127.0.0.1:6379> get ss "test"
說明:Redis2.6以后expire精度可以控制在0到1毫秒內(nèi),key的過期信息以絕對Unix時間戳的形式存儲(Redis2.6之后以毫秒級別的精度存儲),所以在多服務(wù)器同步的時候,一定要同步各個服務(wù)器的時間
2、Redis過期鍵刪除策略
Redis key過期的方式有三種:
(1)、被動刪除:當(dāng)讀/寫一個已經(jīng)過期的key時,會觸發(fā)惰性刪除策略,直接刪除掉這個過期key
(2)、主動刪除:由于惰性刪除策略無法保證冷數(shù)據(jù)被及時刪掉,所以Redis會定期主動淘汰一批已過期的key
(3)、當(dāng)前已用內(nèi)存超過maxmemory限定時,觸發(fā)主動清理策略
被動刪除
只有key被操作時(如GET),REDIS才會被動檢查該key是否過期,如果過期則刪除之并且返回NIL。
1、這種刪除策略對CPU是友好的,刪除操作只有在不得不的情況下才會進行,不會其他的expire key上浪費無謂的CPU時間。
2、但是這種策略對內(nèi)存不友好,一個key已經(jīng)過期,但是在它被操作之前不會被刪除,仍然占據(jù)內(nèi)存空間。如果有大量的過期鍵存在但是又很少被訪問到,那會造成大量的內(nèi)存空間浪費。expireIfNeeded(redisDb *db, robj *key)函數(shù)位于src/db.c。
/*----------------------------------------------------------------------------- * Expires API *----------------------------------------------------------------------------*/ int removeExpire(redisDb *db, robj *key) { /* An expire may only be removed if there is a corresponding entry in the * main dict. Otherwise, the key will never be freed. */ redisAssertWithInfo(NULL,key,dictFind(db->dict,key->ptr) != NULL); return dictDelete(db->expires,key->ptr) == DICT_OK; } void setExpire(redisDb *db, robj *key, long long when) { dictEntry *kde, *de; /* Reuse the sds from the main dict in the expire dict */ kde = dictFind(db->dict,key->ptr); redisAssertWithInfo(NULL,key,kde != NULL); de = dictReplaceRaw(db->expires,dictGetKey(kde)); dictSetSignedIntegerVal(de,when); } /* Return the expire time of the specified key, or -1 if no expire * is associated with this key (i.e. the key is non volatile) */ long long getExpire(redisDb *db, robj *key) { dictEntry *de; /* No expire? return ASAP */ if (dictSize(db->expires) == 0 || (de = dictFind(db->expires,key->ptr)) == NULL) return -1; /* The entry was found in the expire dict, this means it should also * be present in the main dict (safety check). */ redisAssertWithInfo(NULL,key,dictFind(db->dict,key->ptr) != NULL); return dictGetSignedIntegerVal(de); } /* Propagate expires into slaves and the AOF file. * When a key expires in the master, a DEL operation for this key is sent * to all the slaves and the AOF file if enabled. * * This way the key expiry is centralized in one place, and since both * AOF and the master->slave link guarantee operation ordering, everything * will be consistent even if we allow write operations against expiring * keys. */ void propagateExpire(redisDb *db, robj *key) { robj *argv[2]; argv[0] = shared.del; argv[1] = key; incrRefCount(argv[0]); incrRefCount(argv[1]); if (server.aof_state != REDIS_AOF_OFF) feedAppendOnlyFile(server.delCommand,db->id,argv,2); replicationFeedSlaves(server.slaves,db->id,argv,2); decrRefCount(argv[0]); decrRefCount(argv[1]); } int expireIfNeeded(redisDb *db, robj *key) { mstime_t when = getExpire(db,key); mstime_t now; if (when < 0) return 0; /* No expire for this key */ /* Don't expire anything while loading. It will be done later. */ if (server.loading) return 0; /* If we are in the context of a Lua script, we claim that time is * blocked to when the Lua script started. This way a key can expire * only the first time it is accessed and not in the middle of the * script execution, making propagation to slaves / AOF consistent. * See issue #1525 on Github for more information. */ now = server.lua_caller ? server.lua_time_start : mstime(); /* If we are running in the context of a slave, return ASAP: * the slave key expiration is controlled by the master that will * send us synthesized DEL operations for expired keys. * * Still we try to return the right information to the caller, * that is, 0 if we think the key should be still valid, 1 if * we think the key is expired at this time. */ if (server.masterhost != NULL) return now > when; /* Return when this key has not expired */ if (now <= when) return 0; /* Delete the key */ server.stat_expiredkeys++; propagateExpire(db,key); notifyKeyspaceEvent(REDIS_NOTIFY_EXPIRED, "expired",key,db->id); return dbDelete(db,key); } /*----------------------------------------------------------------------------- * Expires Commands *----------------------------------------------------------------------------*/ /* This is the generic command implementation for EXPIRE, PEXPIRE, EXPIREAT * and PEXPIREAT. Because the commad second argument may be relative or absolute * the "basetime" argument is used to signal what the base time is (either 0 * for *AT variants of the command, or the current time for relative expires). * * unit is either UNIT_SECONDS or UNIT_MILLISECONDS, and is only used for * the argv[2] parameter. The basetime is always specified in milliseconds. */ void expireGenericCommand(redisClient *c, long long basetime, int unit) { robj *key = c->argv[1], *param = c->argv[2]; long long when; /* unix time in milliseconds when the key will expire. */ if (getLongLongFromObjectOrReply(c, param, &when, NULL) != REDIS_OK) return; if (unit == UNIT_SECONDS) when *= 1000; when += basetime; /* No key, return zero. */ if (lookupKeyRead(c->db,key) == NULL) { addReply(c,shared.czero); return; } /* EXPIRE with negative TTL, or EXPIREAT with a timestamp into the past * should never be executed as a DEL when load the AOF or in the context * of a slave instance. * * Instead we take the other branch of the IF statement setting an expire * (possibly in the past) and wait for an explicit DEL from the master. */ if (when <= mstime() && !server.loading && !server.masterhost) { robj *aux; redisAssertWithInfo(c,key,dbDelete(c->db,key)); server.dirty++; /* Replicate/AOF this as an explicit DEL. */ aux = createStringObject("DEL",3); rewriteClientCommandVector(c,2,aux,key); decrRefCount(aux); signalModifiedKey(c->db,key); notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"del",key,c->db->id); addReply(c, shared.cone); return; } else { setExpire(c->db,key,when); addReply(c,shared.cone); signalModifiedKey(c->db,key); notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"expire",key,c->db->id); server.dirty++; return; } } void expireCommand(redisClient *c) { expireGenericCommand(c,mstime(),UNIT_SECONDS); } void expireatCommand(redisClient *c) { expireGenericCommand(c,0,UNIT_SECONDS); } void pexpireCommand(redisClient *c) { expireGenericCommand(c,mstime(),UNIT_MILLISECONDS); } void pexpireatCommand(redisClient *c) { expireGenericCommand(c,0,UNIT_MILLISECONDS); } void ttlGenericCommand(redisClient *c, int output_ms) { long long expire, ttl = -1; /* If the key does not exist at all, return -2 */ if (lookupKeyRead(c->db,c->argv[1]) == NULL) { addReplyLongLong(c,-2); return; } /* The key exists. Return -1 if it has no expire, or the actual * TTL value otherwise. */ expire = getExpire(c->db,c->argv[1]); if (expire != -1) { ttl = expire-mstime(); if (ttl < 0) ttl = 0; } if (ttl == -1) { addReplyLongLong(c,-1); } else { addReplyLongLong(c,output_ms ? ttl : ((ttl+500)/1000)); } } void ttlCommand(redisClient *c) { ttlGenericCommand(c, 0); } void pttlCommand(redisClient *c) { ttlGenericCommand(c, 1); } void persistCommand(redisClient *c) { dictEntry *de; de = dictFind(c->db->dict,c->argv[1]->ptr); if (de == NULL) { addReply(c,shared.czero); } else { if (removeExpire(c->db,c->argv[1])) { addReply(c,shared.cone); server.dirty++; } else { addReply(c,shared.czero); } } }
但僅是這樣是不夠的,因為可能存在一些key永遠(yuǎn)不會被再次訪問到,這些設(shè)置了過期時間的key也是需要在過期后被刪除的,我們甚至可以將這種情況看作是一種內(nèi)存泄露----無用的垃圾數(shù)據(jù)占用了大量的內(nèi)存,而服務(wù)器卻不會自己去釋放它們,這對于運行狀態(tài)非常依賴于內(nèi)存的Redis服務(wù)器來說,肯定不是一個好消息
主動刪除
先說一下時間事件,對于持續(xù)運行的服務(wù)器來說, 服務(wù)器需要定期對自身的資源和狀態(tài)進行必要的檢查和整理, 從而讓服務(wù)器維持在一個健康穩(wěn)定的狀態(tài), 這類操作被統(tǒng)稱為常規(guī)操作(cron job)
在 Redis 中, 常規(guī)操作由 redis.c/serverCron 實現(xiàn), 它主要執(zhí)行以下操作:
更新服務(wù)器的各類統(tǒng)計信息,比如時間、內(nèi)存占用、數(shù)據(jù)庫占用情況等。
清理數(shù)據(jù)庫中的過期鍵值對。
對不合理的數(shù)據(jù)庫進行大小調(diào)整。
關(guān)閉和清理連接失效的客戶端。
嘗試進行 AOF 或 RDB 持久化操作。
如果服務(wù)器是主節(jié)點的話,對附屬節(jié)點進行定期同步。
如果處于集群模式的話,對集群進行定期同步和連接測試。
Redis 將 serverCron 作為時間事件來運行, 從而確保它每隔一段時間就會自動運行一次, 又因為 serverCron 需要在 Redis 服務(wù)器運行期間一直定期運行, 所以它是一個循環(huán)時間事件: serverCron 會一直定期執(zhí)行,直到服務(wù)器關(guān)閉為止。
在 Redis 2.6 版本中, 程序規(guī)定 serverCron 每秒運行 10 次, 平均每 100 毫秒運行一次。 從 Redis 2.8 開始, 用戶可以通過修改 hz選項來調(diào)整 serverCron 的每秒執(zhí)行次數(shù)。
也叫定時刪除,這里的“定期”指的是Redis定期觸發(fā)的清理策略,由位于src/redis.c的activeExpireCycle(void)函數(shù)來完成。
serverCron是由redis的事件框架驅(qū)動的定位任務(wù),這個定時任務(wù)中會調(diào)用activeExpireCycle函數(shù),針對每個db在限制的時間REDIS_EXPIRELOOKUPS_TIME_LIMIT內(nèi)遲可能多的刪除過期key,之所以要限制時間是為了防止過長時間 的阻塞影響redis的正常運行。這種主動刪除策略彌補了被動刪除策略在內(nèi)存上的不友好。
因此,Redis會周期性的隨機測試一批設(shè)置了過期時間的key并進行處理。測試到的已過期的key將被刪除。典型的方式為,Redis每秒做10次如下的步驟:
(1)隨機測試100個設(shè)置了過期時間的key
(2)刪除所有發(fā)現(xiàn)的已過期的key
(3)若刪除的key超過25個則重復(fù)步驟1
這是一個基于概率的簡單算法,基本的假設(shè)是抽出的樣本能夠代表整個key空間,redis持續(xù)清理過期的數(shù)據(jù)直至將要過期的key的百分比降到了25%以下。這也意味著在任何給定的時刻已經(jīng)過期但仍占據(jù)著內(nèi)存空間的key的量最多為每秒的寫操作量除以4.
Redis-3.0.0中的默認(rèn)值是10,代表每秒鐘調(diào)用10次后臺任務(wù)。
除了主動淘汰的頻率外,Redis對每次淘汰任務(wù)執(zhí)行的最大時長也有一個限定,這樣保證了每次主動淘汰不會過多阻塞應(yīng)用請求,以下是這個限定計算公式:
#define ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC 25 /* CPU max % for keys collection */ ... timelimit = 1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server.hz/100;
hz調(diào)大將會提高Redis主動淘汰的頻率,如果你的Redis存儲中包含很多冷數(shù)據(jù)占用內(nèi)存過大的話,可以考慮將這個值調(diào)大,但Redis作者建議這個值不要超過100。我們實際線上將這個值調(diào)大到100,觀察到CPU會增加2%左右,但對冷數(shù)據(jù)的內(nèi)存釋放速度確實有明顯的提高(通過觀察keyspace個數(shù)和used_memory大?。?/p>
可以看出timelimit和server.hz是一個倒數(shù)的關(guān)系,也就是說hz配置越大,timelimit就越小。換句話說是每秒鐘期望的主動淘汰頻率越高,則每次淘汰最長占用時間就越短。這里每秒鐘的最長淘汰占用時間是固定的250ms(1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/100),而淘汰頻率和每次淘汰的最長時間是通過hz參數(shù)控制的。
從以上的分析看,當(dāng)redis中的過期key比率沒有超過25%之前,提高h(yuǎn)z可以明顯提高掃描key的最小個數(shù)。假設(shè)hz為10,則一秒內(nèi)最少掃描200個key(一秒調(diào)用10次*每次最少隨機取出20個key),如果hz改為100,則一秒內(nèi)最少掃描2000個key;另一方面,如果過期key比率超過25%,則掃描key的個數(shù)無上限,但是cpu時間每秒鐘最多占用250ms。
當(dāng)REDIS運行在主從模式時,只有主結(jié)點才會執(zhí)行上述這兩種過期刪除策略,然后把刪除操作”del key”同步到從結(jié)點。
maxmemory
當(dāng)前已用內(nèi)存超過maxmemory限定時,觸發(fā)主動清理策略:
volatile-lru:只對設(shè)置了過期時間的key進行LRU(默認(rèn)值)
allkeys-lru : 刪除lru算法的key
volatile-random:隨機刪除即將過期key
allkeys-random:隨機刪除
volatile-ttl : 刪除即將過期的
noeviction : 永不過期,返回錯誤當(dāng)mem_used內(nèi)存已經(jīng)超過maxmemory的設(shè)定,對于所有的讀寫請求,都會觸發(fā)redis.c/freeMemoryIfNeeded(void)函數(shù)以清理超出的內(nèi)存。注意這個清理過程是阻塞的,直到清理出足夠的內(nèi)存空間。所以如果在達(dá)到maxmemory并且調(diào)用方還在不斷寫入的情況下,可能會反復(fù)觸發(fā)主動清理策略,導(dǎo)致請求會有一定的延遲。
當(dāng)mem_used內(nèi)存已經(jīng)超過maxmemory的設(shè)定,對于所有的讀寫請求,都會觸發(fā)redis.c/freeMemoryIfNeeded(void)函數(shù)以清理超出的內(nèi)存。注意這個清理過程是阻塞的,直到清理出足夠的內(nèi)存空間。所以如果在達(dá)到maxmemory并且調(diào)用方還在不斷寫入的情況下,可能會反復(fù)觸發(fā)主動清理策略,導(dǎo)致請求會有一定的延遲。
清理時會根據(jù)用戶配置的maxmemory-policy來做適當(dāng)?shù)那謇恚ㄒ话闶荓RU或TTL),這里的LRU或TTL策略并不是針對redis的所有key,而是以配置文件中的maxmemory-samples個key作為樣本池進行抽樣清理。
maxmemory-samples在redis-3.0.0中的默認(rèn)配置為5,如果增加,會提高LRU或TTL的精準(zhǔn)度,redis作者測試的結(jié)果是當(dāng)這個配置為10時已經(jīng)非常接近全量LRU的精準(zhǔn)度了,并且增加maxmemory-samples會導(dǎo)致在主動清理時消耗更多的CPU時間,建議:
(1)盡量不要觸發(fā)maxmemory,最好在mem_used內(nèi)存占用達(dá)到maxmemory的一定比例后,需要考慮調(diào)大hz以加快淘汰,或者進行集群擴容。
(2)如果能夠控制住內(nèi)存,則可以不用修改maxmemory-samples配置;如果Redis本身就作為LRU cache服務(wù)(這種服務(wù)一般長時間處于maxmemory狀態(tài),由Redis自動做LRU淘汰),可以適當(dāng)調(diào)大maxmemory-samples。
以下是上文中提到的配置參數(shù)的說明
# Redis calls an internal function to perform many background tasks, like # closing connections of clients in timeout, purging expired keys that are # never requested, and so forth. # # Not all tasks are performed with the same frequency, but Redis checks for # tasks to perform according to the specified "hz" value. # # By default "hz" is set to 10. Raising the value will use more CPU when # Redis is idle, but at the same time will make Redis more responsive when # there are many keys expiring at the same time, and timeouts may be # handled with more precision. # # The range is between 1 and 500, however a value over 100 is usually not # a good idea. Most users should use the default of 10 and raise this up to # 100 only in environments where very low latency is required. hz 10 # MAXMEMORY POLICY: how Redis will select what to remove when maxmemory # is reached. You can select among five behaviors: # # volatile-lru -> remove the key with an expire set using an LRU algorithm # allkeys-lru -> remove any key according to the LRU algorithm # volatile-random -> remove a random key with an expire set # allkeys-random -> remove a random key, any key # volatile-ttl -> remove the key with the nearest expire time (minor TTL) # noeviction -> don't expire at all, just return an error on write operations # # Note: with any of the above policies, Redis will return an error on write # operations, when there are no suitable keys for eviction. # # At the date of writing these commands are: set setnx setex append # incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd # sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby # zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby # getset mset msetnx exec sort # # The default is: # maxmemory-policy noeviction # LRU and minimal TTL algorithms are not precise algorithms but approximated # algorithms (in order to save memory), so you can tune it for speed or # accuracy. For default Redis will check five keys and pick the one that was # used less recently, you can change the sample size using the following # configuration directive. # # The default of 5 produces good enough results. 10 Approximates very closely # true LRU but costs a bit more CPU. 3 is very fast but not very accurate. # maxmemory-samples 5
Replication link和AOF文件中的過期處理
為了獲得正確的行為而不至于導(dǎo)致一致性問題,當(dāng)一個key過期時DEL操作將被記錄在AOF文件并傳遞到所有相關(guān)的slave。也即過期刪除操作統(tǒng)一在master實例中進行并向下傳遞,而不是各salve各自掌控。
這樣一來便不會出現(xiàn)數(shù)據(jù)不一致的情形。當(dāng)slave連接到master后并不能立即清理已過期的key(需要等待由master傳遞過來的DEL操作),slave仍需對數(shù)據(jù)集中的過期狀態(tài)進行管理維護以便于在slave被提升為master會能像master一樣獨立的進行過期處理。
看完這篇文章,你們學(xué)會redis設(shè)置過期時間的方法了嗎?如果還想學(xué)到更多技能或想了解更多相關(guān)內(nèi)容,歡迎關(guān)注創(chuàng)新互聯(lián)行業(yè)資訊頻道,感謝各位的閱讀。