RDB:RedisDB每隔一定的时间生成一个redis数据库时点快照。
调用生成快照的方式:
# Save the DB to disk. # # save <seconds> <changes> #second秒内数据改变changes次就进行RDB持久化 # Redis will save the DB if both the given number of seconds and the given # number of write operations against the DB occurred. # # Snapshotting can be completely disabled with a single empty string argument # as in following example: # # save "" 此操作禁止快照生成。 # # Unless specified otherwise, by default Redis will save the DB: #除非特别声明,否则机会使用默认的Redis快照生成条件 # * After 3600 seconds (an hour) if at least 1 key changed # 经过一个小时,数据库有一个或多个key变化时,就进行RDB生成快照 # * After 300 seconds (5 minutes) if at least 100 keys changed # 经过五分钟,数据库至少有100个key发送变化,就进行RDB生成快照 # * After 60 seconds if at least 10000 keys changed # 经过一分钟,如果数据库key的变化数量大于10000,就进行RDB快照生成 # You can set these explicitly by uncommenting the three following lines. #默认值 # save 3600 1 # save 300 100 # save 60 10000 # By default Redis will stop accepting writes if RDB snapshots are enabled # (at least one save point) and the latest background save failed. # This will make the user aware (in a hard way) that data is not persisting # on disk properly, otherwise chances are that no one will notice and some # disaster will happen. # # If the background saving process will start working again Redis will # as in following example: # The filename where to dump the DB dbfilename dump.rdb #生成的RDB文件名 # Note that you must specify a directory here, not a file name. dir /var/lib/redis/6379 #生成的RDB文件存放目录
dump.rdb 文件内容:
前置知识:
系统调用fork函数:复刻(英语:fork,又译作派生、分支)是UNIX或类UNIX中的分叉函数,fork函数将运行着的程序分成2个(几乎)完全一样的进程,每个进程都启动一个从代码的同一位置开始执行的线程。这两个进程中的线程继续执行,就像是两个用户同时启动了该应用程序的两个副本,父子进程对数据的修改对于对方都是不可见的,数据隔离性。
fork系统调用用于创建一个新进程,称为子进程,它与进程(称为系统调用fork的进程)同时运行,此进程称为父进程。创建新的子进程后,两个进程将执行fork()系统调用之后的下一条指令。子进程使用相同的pc(程序计数器),相同的CPU寄存器,在父进程中使用的相同打开文件。
**CopyOnWrite机制:**简称COW写时复制。在数据写入时,会首先创建一个元数据的副本,通过对副本进行修改,然后改变元数据的引用指向副本。
当Redis数据库达到RDB的条件时,首先父进程会fork出一个子进程用于专门进行RDB时点快照的生成。创建的子进程会复制父进程中所有数据的引用,而不是复制父进程的所有数据,所以创建进程速度快。当客户端需要修改数据时,访问父进程,利用系统内核级的CopyOnWrite方式,改变父进程当前数据的引用。而此时子进程察觉不到数据的变化,仍然是那一时刻的数据。子进程生成快照完成,自动销毁进程。
优点:
缺点:
AOF生成备份的方式是记录客户端所有的修改操作,追加到appendonly.aof文件内。
首先默认是没有开启AOF的,需要在配置文件中,开启设置。AOF的所有配置都在配置文件APPEND ONLY MODE下。
AOF和RDB可以被同时开启,如果AOF开启了,redis启动就会加载AOF,因为AOF中保存的数据更加完整。
# The Append Only File is an alternative persistence mode that provides # much better durability. For instance using the default data fsync policy # (see later in the config file) Redis can lose just one second of writes in a # dramatic event like a server power outage, or a single write if something # wrong with the Redis process itself happens, but the operating system is # still running correctly. # # AOF and RDB persistence can be enabled at the same time without problems. # If the AOF is enabled on startup Redis will load the AOF, that is the file # with the better durability guarantees. # # Please check https://redis.io/topics/persistence for more information. appendonly yes
AOF生成文件的路径:文件名为appendonly.aof,路径复用了RDB的路径位置。
# The name of the append only file (default: "appendonly.aof") appendfilename "appendonly.aof"
**在AOF存入磁盘过程中会存在从redis进程—>内核中缓冲区---->磁盘的过程。在缓冲区满的时候,内核会自动将缓冲区的数据flush到磁盘中,但是如果缓冲区没有满,缓冲区中的数据就有可能会因故障而数据丢失。所以AOF提供了三种flush缓冲区的方式:no,always,everysec.
# no: don't fsync, just let the OS flush the data when it wants. Faster. # always: fsync after every write to the append only log. Slow, Safest. # everysec: fsync only one time every second. Compromise. # # The default is "everysec", as that's usually the right compromise between # speed and data safety. It's up to you to understand if you can relax this to # "no" that will let the operating system flush the output buffer when # it wants, for better performances (but if you can live with the idea of # some data loss consider the default persistence mode that's snapshotting), # or on the contrary, use "always" that's very slow but a bit safer than # everysec. # # More details please check the following article: # http://antirez.com/post/redis-persistence-demystified.html # # If unsure, use "everysec". # appendfsync always appendfsync everysec # appendfsync no
AOF会记录每次客户端的修改操作,然后追加到AOF文件中。这就面临着这个文件会越来愈大,并且在记录的所有操作里有可能会存在一些可以合并的操作(比如对同一个key的多此创建与删除,多此自增操作等)。为了防止AOF文件一直变大,可以设置AOF文件中指令重写的最小size,默认值为64。如果AOF文件达到64mb,就会自动触发文件的重写,合并重复的指令,减少文件大小。但是如果文件64mb并且无法减少,就会面临文件一直重写的问题。所以设置了文件增长比率,默认值为100.64mb只在第一次重写时进行比较,后续重写会根据当前文件大小是否大于上一次文件大小的一倍(100%)。比如第二次重写文件就需要达到64+64大小才会触发。
# Automatic rewrite of the append only file. # Redis is able to automatically rewrite the log file implicitly calling # BGREWRITEAOF when the AOF log size grows by the specified percentage. # # This is how it works: Redis remembers the size of the AOF file after the # latest rewrite (if no rewrite has happened since the restart, the size of # the AOF at startup is used). # # This base size is compared to the current size. If the current size is # bigger than the specified percentage, the rewrite is triggered. Also # you need to specify a minimal size for the AOF file to be rewritten, this # is useful to avoid rewriting the AOF file even if the percentage increase # is reached but it is still pretty small. # # Specify a percentage of zero in order to disable the automatic AOF # rewrite feature. auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb
在redis4.0之前,AOF的重写只会合并一些操作,减少文件大小。
在redis4.0之后,AOF的重写会生成一个复合文件,即RDB+AOF的合体。前提开启了如下配置:
# When rewriting the AOF file, Redis is able to use an RDB preamble in the # AOF file for faster rewrites and recoveries. When this option is turned # on the rewritten AOF file is composed of two different stanzas: # # [RDB file][AOF tail] # # When loading, Redis recognizes that the AOF file starts with the "REDIS" # string and loads the prefixed RDB file, then continues loading the AOF # tail. aof-use-rdb-preamble yes
执行如下操作:
AOF文件
执行BGREWRITEAOF执行,4.0之后重写AOF
redis4.0之前,只会进行合并指令
优点:
持久化的速度快,因为每次都只是追加,rdb每次都全量持久化 数据相对更可靠,丢失少,因可以配置每秒持久化、每个命令执行完就持久化
缺点:
灾难性恢复的时候过慢,因为aof每次都只追加原命令,导致aof文件过大,但是后面会rewrite,但是相对于rdb也是慢的。 会对主进程对外提供请求的效率造成影响,接收请求、处理请求、写aof文件这三步是串行原子执行的。而非异步多线程执行的。Redis单线程!