当redis被用作缓存时,有时我们希望了解key的大小分布,或者想知道哪些key占的空间比较大。本文提供了几种方法。
这是redis-cli自带的一个命令。对整个redis进行扫描,寻找较大的key。例:
redis-cli -h b.redis -p 1959 --bigkeys
输出:
# Scanning the entire keyspace to find biggest keys as well as # average sizes per key type. You can use -i 0.1 to sleep 0.1 sec # per 100 SCAN commands (not usually needed). [00.00%] Biggest hash found so far 's_9329222' with 3 fields [00.00%] Biggest string found so far 'url_http://mini.eastday.com/mobile/170722090206890.html?qid=sgllq&ch=east_sogou_push&pushid=13' with 8 bytes [00.00%] Biggest string found so far 'foo' with 40 bytes [00.00%] Biggest hash found so far 's_9329084' with 4 fields [00.23%] Biggest zset found so far 'region_hot_菏泽地' with 625 members [00.23%] Biggest zset found so far 'region_hot_葫芦岛' with 914 members [00.47%] Biggest string found so far 'top_notice_list' with 135193 bytes [00.73%] Biggest zset found so far 'region_hot_自贡' with 2092 members [01.90%] Biggest hash found so far 'uno_facet_2018-12-20' with 59 fields [11.87%] Biggest zset found so far 'region_hot_上海' with 2233 members [27.05%] Biggest set found so far 'blacklist_set_key' with 31832 members [73.87%] Biggest string found so far 'PUSH_NEWS' with 3104237 bytes [86.18%] Biggest zset found so far 'region_hot_北京' with 2688 members -------- summary ------- Sampled 4263 keys in the keyspace! Total key length in bytes is 174847 (avg len 41.02) Biggest string found 'PUSH_NEWS' has 3104237 bytes Biggest set found 'blacklist_set_key' has 31832 members Biggest hash found 'uno_facet_2018-12-20' has 59 fields Biggest zset found 'region_hot_北京' has 2688 members 1616 strings with 3771161 bytes (37.91% of keys, avg size 2333.64) 0 lists with 0 items (00.00% of keys, avg size 0.00) 1 sets with 31832 members (00.02% of keys, avg size 31832.00) 2353 hashs with 7792 fields (55.20% of keys, avg size 3.31) 293 zsets with 333670 members (06.87% of keys, avg size 1138.81)
说明:
更多关于bigkeys的说明可以参考这里。
redis的命令,可以查看某个key序列化后的长度。
例:
连接上redis后执行如下命令 b.redis:1959> hmset myhash k1 v1 k2 v2 k3 v3 OK b.redis:1959> debug object myhash Value at:0x7f005c6920a0 refcount:1 encoding:ziplist serializedlength:36 lru:3341677 lru_seconds_idle:2
关于输出的项的说明:
b.redis:1959> set str1 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa OK b.redis:1959> set str2 abcdefghijklmnopqrstuvwxyz1234 OK b.redis:1959> debug object str1 Value at:0x7f007c035b80 refcount:1 encoding:embstr serializedlength:12 lru:3342615 lru_seconds_idle:13 b.redis:1959> debug object str2 Value at:0x7f00654df400 refcount:1 encoding:embstr serializedlength:31 lru:3342622 lru_seconds_idle:7
两个字串的实际长度都是30, 但str1的serializedlength为12, str2的为31。
这是一个redis rdb file的分析工具,可以根据rdb file生成内存报告。
需要python2.4以上版本和pip。
pip install rdbtools
首先我们需要有一份rdb文件,如果你在配置中开启了rdb,那么redis会自动生成rdb文件。如果没有,可以手动执行bgsave。如果是线上机器,执行时要考虑机器负载等问题。拿到rdb文件后,我们就可以生成内存报告了。命令如下:
rdb -c memory file
例:
rdb -c memory /tmp/dump.rdb database,type,key,size_in_bytes,encoding,num_elements,len_largest_element,expiry 0,hash,data:index_flow_yingshi,10492,hashtable,1,8992,2019-01-14T08:20:10.236000 0,hash,data:index_movie,22068,hashtable,7,2896,2019-01-14T07:29:19.685000 0,string,block:index_module_novel,8296,string,7694,7694,2019-01-13T00:27:46.128000 0,string,block:index_bottom_baike_aikan,8296,string,7632,7632,2019-01-14T02:27:11.850000 0,string,block:index_bottom_tools,5224,string,4549,4549,2019-01-13T01:02:09.171000 0,string,block:index_module_travel,7272,string,6408,6408,2019-01-13T00:43:39.478000 ...
输出了db,数据类型,key, 大小, 编码等多列信息。至于分析数据,你可以用shell,也可以保存成csv用excel排序,或者干脆存到db里,想怎么排怎么排。
如果只要知道最大的N个key, 可以使用-l选项。例:
[@sjs_73_171 ~]$ rdb -c memory -l 3 /tmp/dump.rdb database,type,key,size_in_bytes,encoding,num_elements,len_largest_element,expiry 0,hash,city_tong,724236,hashtable,3113,216,2019-01-14T01:10:59.407000 0,hash,iplocsearch,406292,hashtable,383,180190,2019-01-30T05:37:56.082000 0,hash,weather_tong3,583844,hashtable,319,1658,2019-01-07T10:22:33.742000
如果我们只需要查询单个key所使用的内存可以不必依赖rdb file, 使用redis-memory-for-key命令即可。
例:
[@sjs_73_171 WEB-INF]$ redis-memory-for-key -s b.redis -p 1959 myhash Key myhash Bytes 83 Type hash Encoding ziplist Number of Elements 3 Length of Largest Element 2 [@sjs_73_171 WEB-INF]$ redis-memory-for-key -s b.redis -p 1959 str1 Key str1 Bytes 80 Type string [@sjs_73_171 WEB-INF]$ redis-memory-for-key -s b.redis -p 1959 str2 Key str2 Bytes 80 Type string
rdb --help
也可以查看git上的帮助文档。