The Other Backup: dump
Mark L. Nassal
I am continually amazed at how many UNIX users and admins are unaware of the dump command. Ask them what they use for backups, and they will often say tar. Using tar for system backups is like using emacs to publish a magazine; great tool, wrong job. tar works great for packaging software distributions or archiving a single directory tree. But using it to backup and restore entire systems, pushes it beyond reasonable limits. Versions of dump have been around since Berkeley v4 and AT&T UNIX v6. The tool is stable, relatively easy to use, and available for virtually all flavors of UNIX and UNIX-like operating systems. Sun Solaris calls their version ufsdump, while BSD, SunOs, and Linux call it dump. All of the various versions are essentially the same with a few minor nuances. We will discuss the Solaris and Linux versions in this article. dump Basics dump is a backup and restore utility comprised of two components, dump and restore (or ufsdump and ufsrestore on Sun Solaris). The commands can be issued from the prompt or from scripts. Through the use of command switches, dump can determine which files need to be backed up and mark them as backed up when completed. It can also backup multiple file systems to a single tape. The ability to span volumes is very important for large file systems or meta devices. If the data cannot be stored on a single tape the user is prompted to insert another. This process is repeated until all the data has been transferred. The capacity of a tape can be specified by the user supplying as command options, the maximum capacity of the tape in bytes, the size of the tape (in feet) along with the recording density, or the total block count. If none of the parameters are entered, dump records onto the tape until it reaches the physical end-of-tape (EOT). If the tape capacity is not supplied with the dump command, results can be unpredictable. The end-of-tape signal is not always the end of the tape, but the maximum that dump thinks will fit on the tape. dump will base the tape capacity on default settings, which may or may not be correct. It is always a good idea to specify the capacity with one of the methods above. This will prevent premature volume change requests. Backup schedules are regulated by dump levels. The levels are numeric values from zero through nine. By itself, each value is meaningless. But when several are combined they help to determine which files and directories are copied to tape. dump will back up all files or directories that have been created or changed since the last dump of a lower level. Specifying a dump level allows operators to perform incremental backups, transferring small amounts of data each day or week to tape, and copying the entire specified path once a month. (More on specifying dump levels later.) One of the nicest features of dump is the restore process. When a restore is initiated in interactive mode using the command:
restore if /dev/rmt/0
the contents of the tape can be reviewed much like a disk. The operator can cd through directories and ls directory contents as if the tape were a hard drive. Files can be selectively tagged for restore, or entire directories can be so marked. The ability to see the tape contents and choose which files are to be restored is convenient. dump also provides the ability to perform remote backups. Data from several remote sources can be stored on a central tape system. By using the f switch, local devices can be replaced with remote devices or files, as in the commands:
dump 0vf host_name:/dev/rmt/0 /usr/local
or
dump 0vf host_name:/var/tmp/my_backup
Command Switches The numerous command switches available with dump (Linux) and ufsdump (Solaris) provide more flexibility than standard UNIX backup utilities. Before doing backups, you need to understand the command switches. What follows are descriptions of the more commonly used command switches for dump/restore and for ufsdump/ufsrestore. dump Command Switches Fortunately, the switches available to dump are straightforward. The dump command is issued as follows:
/sbin/dump [0123456789BbhfusTdWn[arguments xxx xxx]] filesystem/directory
0 through 9 - Determines the dump level. Zero represents a full backup. The entire file system or directory is archived regardless of the last dump date. dump levels 1 through 9 are used for incremental backups. Only new, or files changed since the last backup are copied during an incremental. Normally, your first and monthly backups are full (level 0), while weekly and daily are incremental (levels 1-9). dump defaults to level 9 unless otherwise instructed. B - This option is followed by a numeric value specifying the number of dump records per volume, that is, the maximum number of Kbytes that can be stored on each tape. This flag provides an easy way to set the maximum storage capacity of a tape. When the indicated value has been met the operator is prompted to insert the next tape. The byte value is entered in the argument section of the command, as in the following for a 2-Gb tape (a standard DDS1 90 4-mm tape).
/sbin/dump 0ufB /dev/nst0 2048000 /usr/local
This option overrides length and density calculations supplied by the d and s switches. b - Block size is represented in Kbytes and refers to the bytes per dump record. You normally won't need to change this from the default. The default block size is 512 Kb.
/sbin/dump 0ufb /dev/nst0 256 /usr/local
f - This option points to the backup destination. The destination can be a tape device (/dev/nst0), a hard drive (/dev/hda1), standard output (-), a normal file (/var/tmp/my_backup), or a remote device (hostname:device or username@hostsname:device).
/sbin/dump 0uf /dev/nst0 /usr/local
d - Tape density is set by assigning a numeric value to the d option. The density option is used to determine the maximum storage of a particular tape. It can be used in combination with length or by itself.
/sbin/dump 0ufd /dev/nst0 1600 /usr/local
n - The n option is used to send a message using wall to all the operators in the group operator. When tapes need to be changed or the backup has failed, dump will send a message to the operators. s - The size of a tape is specified by this option. Used in combination with a particular density (d), dump will determine the total amount of tape needed, and prompt operators for a tape change. The default density for Linux dump is 2300 feet.
/sbin/dump 0ufs /dev/nst0 2300 /usr/local
u - dump can write to a table that tracks when and which files have been successfully copied. The information is stored in /etc/dumpdates. The table is text based and can be read with more or view. The table format is as follows: fs name, dump level, date and time.
/dev/sda1 0 Sat Jul 5 12:02:59 1997 /dev/sda3 0 Sat Jul 5 12:04:51 1997 /dev/sda2 0 Sat Jul 5 12:08:20 1997 /dev/sdb1 0 Sat Jul 5 12:30:42 1997
W - To receive an on-screen report listing current dump dates for archived file systems use the W option with dump. dump will print the list and exit immediately.
Last dump(s) done (Dump '>' file systems): > /dev/sda1 ( /usr) Last dump: Level 0, Date Sat Jul 5 12:02 > /dev/sda2 (/export) Last dump: Level 0, Date Sat Jul 5 12:08 > /dev/sda3 ( /home) Last dump: Level 0, Date Sat Jul 5 12:04 > /dev/sdb1 ( /opt) Last dump: Level 0, Date Sat Jul 5 12:30
The information used to create this report is extracted from /etc/dumpdates. w - To view a report of file systems that need to be dumped, run dump with the w option.
Dump these file systems: /dev/sda1 ( /usr) Last dump: Level 0, Date Sat Jul 5 12:02 /dev/sda2 (/export) Last dump: Level 0, Date Sat Jul 5 12:08 /dev/sda3 ( /home) Last dump: Level 0, Date Sat Jul 5 12:04 /dev/sdb1 ( /opt) Last dump: Level 0, Date Sat Jul 5 12:30
Restore Command Switches The restore command is issued as follows:
/sbin/restore [rCRxtibDfhmNsTvy] device/file
Here are descriptions of the interactive commands. f - This option points to a file or device to restore. This switch has the same characteristics as in dump. i - Starts restore in the interactive mode. Interactive mode allows the operator to select, delete, ls, and cd through archived file systems. The command line:
[root@walleye /]# /sbin/restore if /dev/nst0
returns the prompt
restore >
From the restore prompt you can display a list of interactive commands by typing help or ?.
restore > ? Available commands are: ls [arg] - list directory cd arg - change directory pwd - print current directory add [arg] - add `arg' to list of files to be extracted delete [arg] - delete `arg' from list of files to be extracted extract - extract requested files setmodes - set modes of requested directories quit - immediately exit program what - list dump header information verbose - toggle verbose flag (useful with ``ls'') help or `?' - print this list If no `arg' is supplied, the current directory is used restore >
cd - Change to directory ls - List files pwd - List present working directory. quit - Exit restore verbose - Provides extended information to listings help or ? - Lists the available commands add - Select files or directories to restore. Use ls to list files first.
restore > ls ./local: Acrobat3/ bin/ etc/ include/ lib/ sbin/ src/ andrew/ doc/ games/ info/ man/ share/ restore > add bin restore ></code_font>
delete - Remove selected files or directories from the restore list restore > delete bin restore >
extract - Once the restore list is build, the operator uses the command extract to perform the restore. The operator will be asked which volume is to be mounted. For a restore of minimal files, start with the last tape; for full restore, start with the first tape. ufsdump Command Switches Sun's ufsdump is a bit more substantial than the general dump command. Since Sun has geared its OS more and more toward enterprise server applications, you would expect the backup apps to follow.
ump [0123456789abcdDflnosStuvwW[arguments xxx xxx]] filesystem/directory
Here are the more common switches. 0 through 9 - Determines the dump level. Zero represents a full backup. The entire file system or directory is copied regardless of the last dump date. dump levels one through nine are used for incremental backups. Only new, or files changed since the last backup are copied during an incremental. Typically, first and monthly backups are full, while weekly and daily are incremental. ufsdump defaults to level 9 unless otherwise instructed.
/usr/sbin/ufsdump 0f /dev/rmt/0 /usr/local
b - Blocking factor default is 20 blocks per write for low density tapes of 6250 BPI or less. For tapes with a density of 6250 BPI or greater, the blocking factor defaults to 64. Cartridge tapes will default to 126 with 512-Kb blocks.
/usr/sbin/ufsdump 0fb /dev/rmt/0 126 /usr/local
c - To automatically set blocking factor to 126 and density to 1000BPI, use the c (cartridge) option.
/usr/sbin/ufsdump 0cufb /dev/rmt/0 126 /usr/local usr/sbin/ufsd
d - Tape density is set by assigning a numeric value to the d option. The density option is used to determine the maximum storage of a particular tape. It can be used in combination with length or by itself.
/usr/sbin/ufsdump 0cufd /dev/rmt/0 1000 /usr/local
D - Create backup on floppy disk. f - This option points to the backup destination. The destination can be a tape device (/dev/rmt/0), a hard drive (/dev/dsk/c0t2d0s0), standard output (-), a normal file (/var/tmp/my_backup), or a remote device (hostname:device or username@hostsname:device). /usr/sbin/ufsdump 0cuf /dev/rmt/0 /dev/dsk/c0t3d0s0
l - If you are using an autoloader, the l option can be added to the command line. It causes ufsdump to wait two minutes after a tape is filled to notify the operator. If another tape autoloads within the two minutes, no message is sent. n - The n option is used to send a message using wall to all the operators in the group operator. When tapes need to be changed or the backup has failed, ufsdump will send a message to the operators. o - After the backup completes or the end-of-media is reached, the drive can be taken offline. This usually includes rewinding the tape and sending the eject signal. s - The length of a tape is determined by this option. Used in combination with a particular density, ufsdump will determine the total amount of tape needed and prompt operators for tape change.
/usr/sbin/ufsdump 0cufs /dev/rmt/0 2300 /dev/dsk/c0t3d0s0
u - ufsdump can write to a table that tracks when and which files have been successfully copied. The information is stored in /etc/dumpdates. The table is text based and can be read with more or view. The format is as follows: fs name, dump level, date, and time.
/dev/rdsk/c0t0d0s6 0 Sat Jul 5 12:02:59 1997 /dev/rdsk/c0t0d0s6 0 Sat Jul 5 12:04:51 1997 /dev/rdsk/c0t0d0s6 0 Sat Jul 5 12:08:20 1997 /dev/rdsk/c0t0d0s6 0 Sat Jul 5 12:30:42 1997
v - Verify backup. If the verification fails the operator will be prompted for a new tape. W - To receive an on-screen report listing current dump dates for archived filesystems, use the W option with ufsdump. ufsdump will print the list and exit immediately.
Last dump(s) done (Dump '>' file systems): > /dev/rdsk/c0t0d0s6 ( /usr) Last dump: Level 0, Date Sat Jul 5 12:02 > /dev/rdsk/c0t11d0s1 (/export) Last dump: Level 0, Date Sat Jul 5 12:08 > /dev/rdsk/c0t0d0s7 ( /home) Last dump: Level 0, Date Sat Jul 5 12:04 > /dev/rdsk/c0t2d0s7 ( /opt) Last dump: Level 0, Date Sat Jul 5 12:30
The information used to create this report is extracted from /etc/dumpdates. w< - To view a report of file systems that need to be dumped run ufsdump with the w option.
Dump these file systems: /dev/rdsk/c0t0d0s6 ( /usr) Last dump: Level 0, Date Sat Jul 5 12:02 /dev/rdsk/c0t11d0s1 (/export) Last dump: Level 0, Date Sat Jul 5 12:08 /dev/rdsk/c0t0d0s7 ( /home) Last dump: Level 0, Date Sat Jul 5 12:04 /dev/rdsk/c0t2d0s7 ( /opt) Last dump: Level 0, Date Sat Jul 5 12:30
ufsrestore Command Switches The ufsrestore command is issued as follows: /sbin/ufsrestore [irRtxabcdfhmsvy] device/file
Here are the descriptions of the interactive switches in ufsrestore. f - Points to the file or device to restore. This switch has the same characteristics as in ufsdump. i - Starts ufsrestore in the interactive mode. Interactive mode allows the operator to select, delete, ls, and cd through archive file systems. The command line:
[root@walleye /]# /usr/sbin/ufsrestore if /dev/rmt/0
returns the prompt
ufsrestore >/
From this prompt you can display a list of interactive commands by typing help or ?.
ufsrestore > ? Available commands are: ls [arg] - list directory cd arg - change directory pwd - print current directory add [arg] - add `arg' to list of files to be extracted delete [arg] - delete `arg' from list of files to be extracted extract - extract requested files setmodes - set modes of requested directories quit - immediately exit program what - list dump header information verbose - toggle verbose flag (useful with ``ls'') help or `?' - print this list If no `arg' is supplied, the current directory is used ufsrestore >
These commands work the same in ufsrestore as they do in restore. Backup Strategies Numerous backup strategies have been promoted over the years, but the good ones are based on the same concepts.
Maintain a balance between possible data loss and the time required to backup the systems. Applications such as dump consume system resources as well as human resources. Use good judgment and communicate with your users.
It is always better to err on the side of conservatism. The best way to determine how much tape is needed, is to base it on the file system. Use the maximum capacity of the file system not the current data stored on it. Eventually the directory will fill with data and exceed the tape. These situations often go unnoticed until a restore is incomplete. By that time it is too late.
The main reason for offsite storage is disaster recovery. In case of fire or theft, it is important to have a set of backups that can get your company up and running again.
Even when backups are performed during off hours check that other scheduled programs are not modifying files during the backup.
Sun's ufsdump contains the -v option, which forces a verification of the backup after completion. Since it is part of the execution command, human interaction isn't required unless there is a problem. So ufsdump can be run from cron. Unfortunately Linux dump does not have this switch. To verify a backup with Linux, restore must be run with the -C option. This flag will cause restore to compare the contents of the original file system to the data on the tape. Another option is to run restore or ufsrestore in interactive mode. You can then cd and ls through the directories comparing them to the original file system. This can be very time consuming for large dumps, but it is better than ending up with a bad tape. By adhering to these concepts you can create a well-balanced dump sequence that meets the needs of your network. It is important to make sure that data integrity is maintained, yet not so complicated that operators have trouble performing the routine. Most environments do well with a moderate dump schedule. This schema consists of a monthly dump 0, a weekly dump 5, and a daily dump 9. This means that once a month everything on the system is copied. Everything that has been changed or created since the weekly is stored each day, and everything after the monthly is archived each week. Using this pattern assures that there is overlap between backups. The monthly should be stored at an offsite facility. Remember that dump levels are existential symbols. The data stored by a dump level 0 is the same as a dump level 9. The difference occurs when a relationship is created between levels. Each consecutive dump level stores all information created or changed since the previous dump level of a lower number. For example, if you perform a level 1 on Monday, a 3 on Tuesday, a 4 on Wednesday, and a 2 on Thursday, the Thursday backup will contain data from Tuesday through Thursday. Every location has specific needs and limitations. It will take some fine tuning and creativity to design the perfect system for your network but, it is worth it. Local Backup Tips Now that you have a better understanding of the dump/ufsdump commands and backup concepts, I will provide some examples of their use, along with explanations of some of the interesting options. For purposes of instruction, the tape in these examples will be a 4 mm DDS1 (90 M). The maximum uncompressed storage capacity is 2 Gb. The file system to be copied is /usr, which has a maximum capacity of 500 Mb. The backup device is the first tape drive on the system (st0 for Linux, rmt/0 for Solaris). The /etc/dumpdates file will be updated, and the operator notified of any problems. Solaris
/usr/sbin/ufsdump 0cnuvfsd /dev/rmt/0 \ 295 61000 /usr
The v option has been added so that ufsdump will verify the tape after creation. To calculate the capacity of the tape the s and d (feet and BPI) options are added. To copy an entire file system, I would have used the drive id instead of directory name, as in:
/usr/sbin/ufsdump 0cnuvfsd /dev/rmt/0 295 61000 /dev/dsk/c0t0d0s6
Linux
/sbin/dump/ 0nufB /dev/st0 2048000 /usr
Without the i, d, or s option, Linux dump will request tape changes too early. By setting the dump records (B), dump will not ask for a tape change until 2 Gb of data has been copied or end-of-media is signaled. I have found the B option to be more dependable than using d and s. (dump never seems to calculate the size properly.) It should also be noted that the Linux version of dump is a in-work project. This means there are still a few problems. Single dump files spanning multiple tapes may not work properly. It has been reported that files are sometimes left out. To avoid problems, make sure to do a test run before committing to a schema. Again, if the source was an entire file system, you could replace /usr with the drive id, as in:
/sbin/dump/ 0nufB /dev/st0 2048000 /dev/sda2
Multiple dumps can be stored on a single tape by changing the destination device to no-rewind (/dev/rmt/On). If the standard device is used (st0, rst0, rmt/0), the tape will rewind after each dump and overwrite the previous. Here are some examples. Solaris
/usr/sbin/ufsdump 0cnuvfsd /dev/rmt/0n 295 61000 /usr /usr/sbin/ufsdump 0cnuvfsd /dev/rmt/0n 295 61000 /home
Linux
/sbin/dump/ 0nufB /dev/nst0 2048000 /usr /sbin/dump/ 0nufB /dev/nst0 2048000 /home
If the backup is too large to fit on one tape, dump will prompt the operator for a new tape. Remember, if the dump is being done from cron, there will not be anyone to change tapes. In this case you must make sure the backup will fit on one tape or use a stacking system. If you added the n option, dump will notify all the users in the operator group. wall will be used so you may want to leave a telnet session running on the backup server. This way if there were any problems with the backup you will see it in the morning. Another way to check the status of auto backups is to check the /etc/dumpdates file each morning. All of the file systems that were completed will be listed. Remote Backups dump and ufsdump can be used to centralized backups. Data from multiple servers can be sent to a single backup server using remote connections. You must first make sure that root can rlogin to the remote servers. For security reasons most systems are set for root login at console only. To enable root on Solaris edit the file /etc/deafult/login. You will see a line similar to CONSOLE=/dev/console. Comment this line out to allow root to login remotely. For Linux, edit the file /etc/securetty, and add several ttyp's to the tty list. For example,
tty1 tty2 tty3 ttyp1 ttyp2 ttyp3
Root will be able to login on each ttyp that you add. Next you will need a .rhosts file in the root home of the backup server. dump will use rsh for remote backups and requires a .rhosts for this purpose. Enter each of the servers from which backup scripts will be logging in.
walleye speedy jasmine
After you have created the file, change the mod to read only by owner.
chmod 400 .rhosts
This will prevent others from changing the information, plus many UNIX OS's require the mod for rsh and rlogin to work. The command for remote backup is essentially the same as local. The difference is that the target device is prefaced by the server name. Solaris
/usr/sbin/ufsdump 0cnuvfsd thore:/dev/rmt/0n 295 61000 /usr
Linux
/sbin/dump/ 0nufB thore:/dev/nst0 2048000 /usr
The commands above will dump the files from /usr to the tape device on thore (backup server). We could easily replace the tape device with a drive or file, as on local backups. Scripts Even if you do not plan to perform backups from cron, it is a good idea to create scripts for your backups. They save time and help to maintain consistency. Since dump and ufsdump interact with the console and logs successful backups, the scripts do not have to be very deep. If the operation is done through cron, you will just need to create a sequential list of dumps. But if you plan to work from a console (which I suggest), the script in Listing 1 can be modified for your site. Simply modify the devices, tape length, and paths for each dump level. I have repeated the dump list for each level to allow more flexibility. This way you can use different devices and tapes for full backups and daily backups. After you are done editing the script change the mode to 755, so it can be executed. I have also set the script to log the /etc/dumpdates info to a continuous log file (/var/log/backup.log). dump and ufsdump only keep the most recent dumps in /etc/dumpdates. To provide a backup history I have the file appended to /var/log/backup.log at the end of each backup. The script was written for Linux dump, but it can easily be converted to work on Solaris. Change dump to ufsdump, edit the devices, and paths. You may also want to switch B for d and s, and add the v (verify) option. Restoring Files Restoring files with dump and ufsdump is pretty easy. Start by switching to the directory or file system you plan to restore, then run restore/ufsrestore in interactive mode.
/sbin/restore if /dev/nst0 /usr/sbin/ufsrestore if /dev/rmt/0n
Select the files or directories using the add command, and extract to copy the files. If there are multiple dumps on the tape, you will need to move to the desired dump. To move to a new position on the tape, use the mt command:
mt -f /dev/nst0 fsf 2
Replace the numeral 2 with the dump position you want to restore. The first dump is 0, second is 1 and so on. Make sure you use the no-rewind device for mt and restore, or the tape will reposition at the beginning. Here is a sample restore of my home directory using the script in Listing 1. Start by switching to the restore directory(/home/markn). Next, use the mt command to position the tape at the second dump.
[root@jasmine temp]# mt -f /dev/nst0 fsf 1
Now, restore needs to be run in interactive mode.
[root@jasmine temp]# /sbin/restore if /dev/nst0 restore >
The ls command is used to view the directories.
restore > ls .: beth/ gopher/ lost+found/ samba/ ftp/ httpd/ markn/ restore >
Now tag the directory markn with the command add. restore > add markn
The extract command is used to restore the files.
restore > extract/
Restore then prompts for the last tape in the backup set. If there is only one, enter 1.
You have not read any tapes yet. Unless you know which volume your file(s) are on you should start with the last volume and work towards the first. Specify next volume #: 1
The files will then be restored to the current directory. When the restore is complete, dump will ask if you want to set owner/mode for '.'; answer no. That's it. The restore is now completed. The process is the same for dump and ufsdump. Where to Get It If your OS did not ship with dump (which is very unusual), checkout ftp://ftp/redhat.com and ftp://sunsite.unc.edu. Most major distributions of UNIX and UNIX-like operating systems come with one form or another of dump. For more specific information on the flavor installed on your system, refer to the man pages.
About the AuthorMark Nassal is a senior UNIX administrator at Harte Hanks Data Technologies in Billerica, MA. They operate a predominantly UNIX network with NT 3.5 and 4.0 as the standard desktop OS. He has been in the Computer/Telecommunications industry for nine years, and much of his work has centered around UNIX/Microsoft integration.
|