Redis メモリ使用率解析

Nid: 1201

目次

  1. redis-rdb-tools インストール
  2. データセットを保存
  3. メモリスナップショットを CSV に出力
  4. SQLite にインポート
  5. 解析
Back to top

1 . redis-rdb-tools インストール

$ pip install rdbtools
Back to top

2 . データセットを保存

バックグラウンドでデータセットを保存

$ redis-cli bgsave
Background saving started

タイムスタンプを変換して、保存終了を確認

$ date --date "@`redis-cli lastsave`" +"%Y/%m/%d %H:%M"
2019/01/01 01:11
Back to top

3 . メモリスナップショットを CSV に出力

$ cd ~/work
$ sudo sh -c "cp /var/lib/redis/dump.rdb dump.rdb && chmod go+r dump.rdb"
$ rdb -c memory dump.rdb >| memory.csv

$ head memory.csv
database,type,key,size_in_bytes,encoding,num_elements,len_largest_element,expiry
0,hash,888bd4bde1c4af001869b510e173b5fd:cache_entity_file:744,876,hashtable,6,306,2020-01-03T14:58:16.997000
0,hash,6fa02f748b75462a31c34c99bcfb144d:cache_views_data:mvproulette:block_2:results:a56481832527ad364be41407c34fc2ec,764,hashtable,6,69,2019-01-09T15:51:33.654000
0,hash,6fa02f748b75462a31c34c99bcfb144d:cache_entity_node:41325,4140,hashtable,6,3513,2020-01-04T11:54:56.740000
0,hash,de61249a4782a16c8c13425fd4880176:cache_entity_file:604082,940,hashtable,6,323,2020-01-07T11:34:23.143000
0,hash,fbe5b7db2c5bd7e242d12ff5bb999d63:cache:i18n:string:obj:field:field_sentencejp:field_sentencevovab2:label,1196,hashtable,6,453,2020-01-08T03:31:30.184000
0,hash,d7625c7487795319358b1f81c680b9c7:cache_menu:links:user-menu:page:node/58924:ja:1:1,764,hashtable,6,109,2020-01-03T12:46:52.807000
0,hash,octaviadata.com:cache_field:field:node:44421,2100,hashtable,6,1378,2020-01-08T01:25:14.285000
0,hash,d7625c7487795319358b1f81c680b9c7:cache_entity_node:50380,7724,hashtable,6,6681,2020-01-04T05:58:22.717000
0,hash,d7625c7487795319358b1f81c680b9c7:cache_entity_node:52237,7724,hashtable,6,6253,2020-01-04T03:02:10.187000
Back to top

4 . SQLite にインポート

$ sqlite3 memory.db
sqlite> create table memory(database int,type varchar(128),key varchar(128),size_in_bytes int,encoding varchar(128),num_elements int,len_largest_element varchar(128));
sqlite>.mode csv memory
sqlite>.import memory.csv memory
Back to top

5 . 解析

$ sqlite3 memory.db

メモリ使用率降順10件

sqlite> select * from memory order by size_in_bytes desc limit 10;
database|type|key|size_in_bytes|encoding|num_elements|len_largest_element
0|hash|a3506d02be75633484735650f99fec5d:cache:htmlpurifier:4.10.0|4114918a13a428a8482a8a449792a5a8747582b5|1|29356|hashtable
0|hash|2844ac7154fd748893aac76a4ac36222:cache:htmlpurifier:4.10.0|4114918a13a428a8482a8a449792a5a8747582b5|1|29356|hashtable
0|hash|a3506d02be75633484735650f99fec5d:cache:htmlpurifier:4.10.0|25cabe5ef0d88d070afb523631881c821da8f768|1|1324|hashtable
0|hash|2844ac7154fd748893aac76a4ac36222:cache:htmlpurifier:4.10.0|09075b5f207f804d595ee0c49bd6fbc917280664|1|98988|hashtable
0|hash|a3506d02be75633484735650f99fec5d:cache:htmlpurifier:4.10.0|09075b5f207f804d595ee0c49bd6fbc917280664|1|98988|hashtable
0|hash|2844ac7154fd748893aac76a4ac36222:cache:htmlpurifier:4.10.0|0375dea7a79a8c4cdc99638fdb9a88e179499acd|1|1324|hashtable
0|hash|ce189b084b9c1f25848cb66f69760e01:cache_browscap:Mozilla/5.0 (Windows NT 6.2;en-US) AppleWebKit/537.32.36 (KHTML| live Gecko) Chrome/59.0.3005.80 Safari/537.32|1324|hashtable|6
0|hash|ce189b084b9c1f25848cb66f69760e01:cache_browscap:Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML| like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)|1996|hashtable|6
0|hash|ce189b084b9c1f25848cb66f69760e01:cache_browscap:Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML| like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)|1612|hashtable|6

特定キーのメモリ使用量

sqlite> select sum(size_in_bytes) from memory where key like '6bcbce1f17ecd23f67742a858b3239f5%';
229691930

キーからサイト名を確認

$ grep 6bcbce1f17ecd23f67742a858b3239f5 memory.csv | grep -Eo '(http|https)://[^/"]+' | sort | uniq
http://octaviadata.com
https://octaviadata.com

終了

sqlite> .quit

特定キーの削除

$ redis-cli --scan --pattern 6bcbce1f17ecd23f67742a858b3239f5* | xargs redis-cli del
Back to top
Ref Link: