home › Working with photos › What is the difference between an iso image and a dd image? Creating a hard disk image using the dd utility in Unix. What does dd mean?

What is the difference between an iso image and a dd image? Creating a hard disk image using the dd utility in Unix. What does dd mean?

The other day I decided to create an image of my working, bootable flash drives with different operating systems. How to create these multi-boot flash drives, I already in one of his articles. An old and time-tested program will help us in creating images. dd. As far as I know, the dd utility should be installed on the system by default. To create an image of your flash drive, run in the Terminal next command:

sudo dd if=/dev/sdc of=image.iso

/dev/sdc- this is your flash drive,

image.iso is an image with a name image and expansion .iso, which will appear in your home folder.

To find out how your flash drive is designated in the system, you can, for example, run the Disks utility or the GParted program and look in them, or if through the Terminal, the command will help:

sudo fdisk -l

If you want to see the process of creating a flash drive, then there are several ways. I won’t tell you the first way, because I didn’t like it, but alternative solution this problem looks much better. To do this, you will need to install an improved version of dd, which is called dcfldd.

sudo apt-get install dcfldd

The staff of the DoD Computer Forensics Laboratory (DCFL) made some changes to the dd command, thereby improving it and using it in their research work. As a result, the dcfldd command was born, which ensures that the copied data is hashed at certain intervals to authenticate it. Moreover, dcfldd is much faster than dd. Now, if you want to see the progress of copying or creating an image, you need to run the command:

sudo dcfldd if=/dev/sdc of=image.iso

Now that the image is ready, you can create a new one, bootable USB flash drive. Let's insert a new, empty one instead of a flash drive with systems. I think that it will be detected by the system in the same way as the first one - sdc, but it’s better to double-check. Now the command will be like this:

sudo dd if=image.iso of=/dev/sdc

Well, if you have an empty flash drive that is the same size as the media with the data you need, then you can simply copy the entire contents of the first flash drive directly to the second, bypassing the creation of an image on HDD. In this case the command will be as follows:

sudo dd if=/dev/sdb of=/dev/sdc

Using this scheme, you can copy and create images not only of flash drives, but also hard drives entirely, or their sections, CD/DVD disks, etc. But more on that in the next article.

The choice of flash drives is up to you. You can write down boot image to regular flash drives, from famous brands: Transcend, Kingston, Apacer, Silicon Power and other manufacturers, or you can differentiate yourself a little and choose jewelry flash drives with rhinestones, diamonds and other decorations. Although this, of course, will be the choice of the fair half of humanity. Although the work, the recorded image, appearance The device has absolutely no effect.

Quite often, system administrators need to copy various binary data. For example, sometimes you may need to make a backup hard drive, create an empty file filled with zeroes to organize swap space or another virtual file system.

To solve all these problems, the dd linux utility is used, which simply copies data from one place to another at the binary level. It can copy a CD/DVD disc, a section on a disc, or even an entire hard drive. In this article we will look at what the linux dd command is, its main options and parameters, and how to use it.

First you need to understand how the dd command works and what it does. In fact, this is an analogue of the utility only for block data. The utility simply transfers one block of data of the specified size from one place to another. Since everything is considered a file in Linux, including devices, you can transfer devices to files and vice versa.

Using various utility options, you can influence the block size, and this, in turn, already affects the speed of the program. Next we will look at the main options of the utility and its capabilities.

dd command

The syntax of the utility is quite unusual, but at the same time very simple, once you remember it and get used to it:

$dd if= copy_source of= destination options

Using the if parameter, you need to specify the source from which the blocks will be copied, this could be a device, for example, /dev/sda or a file - disk.img. Next, using the of parameter, you need to specify the destination device or file. Other parameters have the same syntax as if and of.

Now let's look at the additional options:

bs- indicates how many bytes to read and write at a time;
CBS- how many bytes need to be written at a time;
count- copy the specified number of blocks, the size of one block is indicated in the bs parameter;
conv- apply filters to the data stream;
ibs- read the specified number of bytes at a time;
obs- write the specified number of bytes at a time;
seek- skip the specified number of bytes at the beginning of the reading device;
skip- skip the specified number of bytes at the beginning of the output device;
status- indicates how detailed the conclusion should be;
iflag, oflag- allows you to set additional operation flags for the input and output device, the main ones: nocache, nofollow.

These were all the basic options you might need. Now let's move closer to practice and look at several examples of how to use the dd linux utility.

How to use dd?

Regular users use the dd command most often to create images DVDs or CD. For example, to save a disk image to a file, you can use the following command:

sudo dd if=/dev/sr0 of=~/CD.iso bs=2048 conv=noerror

The noerror filter allows you to disable response to errors. Next, you can create an image of the hard drive or partition on it and save this image to disk. Just be careful not to save to the same hard drive or partition, so as not to cause recursion:

dd if=/dev/sda of=~/disk.img

A file named disk1.img will be created in your home folder, which in the future can be deployed and restored to the damaged system. To write an image to a hard drive or partition, just swap the device addresses:

dd if=~/disk.img of=/dev/sda

A very important and useful option is bs. It allows you to greatly influence the speed of the utility. This parameter allows you to set the size of one block when transferring data. Here you need to set digital value with one of these format modifiers:

With- one character;
b- 512 bytes;
kB- 1000 bytes;
K- 1024 bytes;
M.B.- 1000 kilobytes;
M- 1024 kilobytes;
G.B.- 1000 megabytes;
G- 1024 megabytes.

The dd linux command uses just such a system, it is complex, but there is no escape from it. It will have to be understood and remembered. For example, 2b is 1 kilobyte, and 1k is also 1 kilobyte, 1M is 1 megabyte. By default, the utility uses a block size of 512 bytes. For example, to speed up disk copying, you can take blocks of 5 megabytes in size. To do this, use the following command:

dd if=/dev/sda of=~/disk.img bs=5M

The next parameter is count. Using it you can specify how many blocks need to be copied. For example, we can create a 512 megabyte file by filling it with zeros from /dev/zero or random numbers from /dev/random:

sudo dd if=/dev/zero of=file.img bs=1M count=512

Please note that this parameter does not indicate the size in megabytes, but only the number of blocks. Therefore, if you specify a block size of 1b, then you only need to take two blocks to create a 1KB file. This option can also be used to backup the MBR partition table. To do this, copy the first 512 bytes of the hard drive to a file:

sudo dd if=/dev/sda of=mbr.img bs=1b count=1

To restore, use the usual command to deploy the image to disk.

If the disk image is too large, you can redirect all output to the non-standard output stream of the gzip utility:

dd if =/dev/sda2 | bzip2 disk.img.bz2

You can also use the dd linux utility to copy files, although this is not its intended purpose:

dd if=/home/sergiy/test.txt of=/home/sergiy/test1.txt

As you know, the linux dd command writes data to disk directly in binary form, which means that zeros and ones are written. They override what was previously placed on the recording device. Therefore, to erase a disk, you can simply fill it with zeros from /dev/zero.

sudo dd if=/dev/zero of=/dev/sdb

Using dd this way results in the entire disk being completely erased.

conclusions

In this article we looked at how to use dd linux, what this utility can be used for and how useful it can be. It's an almost indispensable tool. system administrator, because it can be used to make backup copies of an entire system. And now you know how. If you have any questions, ask in the comments!

Team dd is designed to use the utility of the same name, designed for low-level copying and conversion of data. Its name stands for “data duplicator” or “data duplicator”. This utility is used primarily for burning installation disk images of Linux distributions to flash drives and creating images optical media, however, the range of its functions is not limited to the listed operations. For example, dd can be used to simply copy files or change the case of text strings. In general, the utility in question is to some extent unique, because it involves the use of its own format for passing parameters.

The standard command syntax is as follows:

$dd if=<имя исходного файла>of=<имя целевого файла>[options]

It is easy to notice that the recording format is used to pass parameters to the utility <имя параметра>=<значение параметра> . The utility can read source data from standard input and output the resulting data using standard output if parameters are not used if And of, but in the vast majority of cases these parameters are necessary to specify file names with the corresponding data. The utility reads and writes data in blocks, and the block size can be changed using the parameter bs(Blocks of 512 KB are used by default). There are separate parameters for setting the sizes of readable and writable blocks, namely, ibs And obs. The number of blocks read can be limited using the parameter count. The parameter can be used to skip a specified number of blocks in the source file. skip, target file - parameter seek. The parameter can be used to specify read and write flags separated by commas iflag

append- activation of the mode of appending data to the target file.
direct- data processing mode bypassing the file system cache (increases speed).
dsync- data recording mode with synchronization (increases reliability).
sync- data and metadata recording mode with synchronization (increases reliability).
fullblock- reading only complete blocks.
nonblock- activation of non-blocking I/O mode (increases speed).
noatime- disabling the mechanism for updating file system element timestamps (increases speed).
nofollow- refusal to follow symbolic links.

Finally, the parameter can be used to specify comma-separated conversion flags conv. The most commonly used flags are:

lcase- converting string characters in ASCII encoding to lowercase.
ucase- converting string characters in ASCII encoding to uppercase.
nocreat- display an error message if the target file is missing.
excl- display an error message if the target file exists.
notrunc- refusal to trim the target file.
swab- changing the locations of every two bytes from the source file.
noerror- continuation of work even if errors occur.
fdatasync- activation of the mode of writing data to the target file before completing the utility.
fsync- activation of the mode of writing data and metadata to the target file before completing the utility.

Examples of using

Backing up disk drive data

Let's assume we are using a hard drive represented by a device file /dev/sda, and we need to create a sector-by-sector backup of all the data located on it, saving it in a file in the partition of the removable USB drive represented by the device file /dev/sdb1 and mounted in the directory /mnt/sdb1. These backup files are usually called dumps or disk images. Our disk image file will be named backup.img. This is the command with which you can create it:

# dd if=/dev/sda of=/mnt/sdb1/backup.img

In this command, using the parameter if the path to the source file is specified, and using the parameter of- to the target.

Restoring data from a backup

To recover data from the created backup copy you should boot the system from installation disk distribution and execute the reverse command.

# dd if=/mnt/sdb1/backup1.img of=/dev/sda

Warning: executing this command will overwrite the entire contents of the specified hard drive, so you should treat such commands with special attention.

Hard drive cloning

Before cloning hard disk, you must make sure that you have a hard disk of the same capacity as the original one. The same operation can be performed in the case of flash drives with a USB interface of similar sizes. Let's assume that the source flash drive is represented by a device file /dev/sdb, and the target - the device file /dev/sdc. In this case, you can clone the source drive using the following command:

# dd if=/dev/sdb of=/dev/sdc

Even if the target drive has a larger capacity, you will only have access to the size of the source flash drive stored at the file system level.

Transferring a disk image file to another computer

To transfer a disk image file over a network to another computer named target the following command can be used:

# dd if=/dev/sdb | ssh root@target "(cat >backup.img)"

Compressing a Disk Image File

To make the backup disk partition take up less space, you can compress it using a compressor such as bzip2 :

# dd if=/dev/sdb | bzip2 backup.img.bz2

Creating an ISO optical disc image

To create an image of an optical disk CD, DVD or BD, simply read its contents block by block and save this contents in a file:

# dd if=/dev/sr0 of=image.iso bs=2048

Saving a file from damaged media or creating an image of such media

If your favorite movie or music track is no longer readable due to media corruption, you can try to copy it using the utility dd, ignoring bad blocks:

# dd if=movie.avi of=/home/alex/movie.avi conv=noerror,sync

You can also create an image file of the damaged media and try to extract files from it:

# dd if=/dev/sdb of=/home/alex/movie.iso bs=2048 conv=noerror,sync

Burning an installation disk image to a USB flash drive

For installation Linux distribution from a USB flash drive, you must write an ISO installation disk image to this flash drive. A similar command can be used for this purpose:

# dd if=/home/alex/Fedora-Workstation-Live-x86_64-26_Alpha-1.7.iso of=/dev/sdc

It is important to remember that even if there are partitions on the flash drive, you should not specify the path to the device file of one of the partitions, but the path to the device file of the drive itself, in our case this is /dev/sdc.

Hard drive content analysis

Utility dd is an excellent tool for exploring file systems. To analyze the contents of a hard drive with data output from individual blocks, in our case, a block 1001 on the partition represented by the device file /dev/sdc1, just use the following command:

# dd if=/dev/sdc1 count=1 skip=1000

In order to see the first 40 bytes of your hard drive in hexadecimal notation, use the command:

# dd if=/dev/sda bs=1 count=40 | hexdump -C

In this case, using the parameter bs sets the disk block size.

Testing disk drive performance

To test the performance of a disk drive represented, for example, by a device file /dev/sda When reading blocks of different sizes, a similar command can be used:

1000000+0 records in
1000000+0 records out

# dd if=/dev/sda of=/dev/null bs=4096 count=1000000
1000000+0 records in
1000000+0 records out
4096000000 bytes (4.1 GB) copied, 29.8747 s, 137 MB/s

Thanks to the file system caching mechanism, you may encounter a mystical speedup of read operations that should not be surprising:

# dd if=/dev/sda of=/dev/null bs=512 count=1000000
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 4.25186 s, 120 MB/s

# dd if=/dev/sda of=/dev/null bs=512 count=1000000
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 0.417317 s, 1.2 GB/s

To obtain correct test results, it is recommended to bypass the file system caching mechanism altogether by using I/O mode without caching:

# dd if=/dev/sda of=/dev/null bs=512 count=100000 iflag=direct
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 5.01053 s, 10.2 MB/s

Copying files

Yes, utility dd can even be used for regular file copying. Of course, for this purpose it is better to use a utility specially designed for this purpose, namely, cp. In any case, you can copy the file using a similar command:

$ dd if=/home/alex/test.txt /home/alex/test_copy.txt

As is known, “computer users are divided into those who make backups and those who will do them”. In this article we will look at various ways Reserve copy(backup) of the entire system and, accordingly, restoration from a backup copy.

It’s worth noting right away that all operations should not be performed “live”, i.e. not on a running system, but from a liveCD or installed on a neighboring partition/flash drive/usb-hdd of the system. In cases where downtime of a few minutes is critical for the system, it is possible to copy the system from under itself, but you need to take into account some additional conditions, which are not yet discussed in this article

Further in the text, for actions performed on behalf of the superuser, will be used sudo command, which is standard for Ubuntu. On other systems it is possible to gain superuser privileges via su , some liveCD systems run in superuser mode by default

tar

One of the most popular ways to create a simple backup is to archive data using tar. The advantages of this method are the possibility of incremental backup (adding files to an existing archive, deleting or changing them), the ability to extract from the archive separate files, as well as the presence of tar in almost any Linux system.

Creating an archive

First, create mount points for the root partition and for the partition on which you are going to create a backup, for example like this

Mount both partitions. For greater reliability, you can mount the root partition in read-only mode to eliminate the possibility of accidental data changes

Sudo mount /dev/sdXY /mnt/root -o ro sudo mount /dev/sdXY /mnt/backup

(Instead of "sdXY" use your values for required sections. you can determine them using sudo fdisk -l or sudo blkid)

If you use separate partitions for /boot, /usr, /home, etc. and want to include their contents in the backup, mount them in the appropriate folders

Sudo mount /dev/sdXY /mnt/root/usr -o ro sudo mount /dev/sdXY /mnt/root/home -o ro

If necessary, create a folder on the backup partition in which you want to place the archive, for example

Sudo mkdir -p /mnt/backup/ubuntu/root

Now you can start creating the archive. To create a gzip-compressed archive, run

Sudo tar -cvzpf -C /mnt/root /mnt/backup/ubuntu-sda1.tar.gz .

(The -p switch enables saving owners and permissions for files)

For bzip2 compression use

Sudo tar -cvjpf /mnt/backup/ubuntu-sda1.tar.bz2 /mnt/root

For lzma compression

Sudo tar --lzma -cvpf /mnt/backup/ubuntu-sda1.tar.lzma /mnt/root

Similarly for lzo compression - switch --lzop instead of --lzma

Different compression algorithms produce different archive sizes and also differ in performance

Once the process is complete, unmount all mounted partitions

Sudo umount /mnt/root(/boot,/var,/home,) /mnt/backup

Restoring from an archive

Create mount points for the root partition and the partition where your archive is stored

Sudo mkdir /mnt/(root,backup)

Mount the partition with the backup archive

Sudo mount /dev/sdXY /mnt/backup -o ro

Format the root partition to the same (or another) file system. If you use separate partitions for /usr, /boot, etc. and archived them, format them too

(if you are restoring the system to new hard disk, partition it using fdisk/gparted and format the partitions)

Some file systems support setting the UUID when formatting. This makes it possible to create a file system with the same UUID as the old one, which will avoid the need to edit fstab.

For ext2/3/4, the UUID is set using the -U switch, and you can simplify the task even further with a command like

Sudo mkfs.ext4 -L "label" -U "$(sudo blkid -o value -s UUID /dev/sda1)" /dev/sda1

If you used archiving when creating the image file, first unpack it using the same archiver, for example

Bzip2 -dv /media/backup/sda5.dd.bz

Now you can mount the image

Sudo mount /media/backup/sda5.dd -o loop /mnt

(With the loop option, the mount program will automatically “pick up” the image file to a free loop device, and then mount the file system)

Now you can work with the contents of the image as with a regular file system, all your changes will be written to the image. When finished, mount the image as a regular file system

Sudo umount /mnt

dd - copy the entire hard drive

In this case, we will use dd again, only this time we will save the entire contents of the hard drive - with the partition table, the partitions themselves and all the data. Advantage this method the fact that you can save all systems installed on this hard drive in one step without having to backup each partition separately. In addition, with such a backup, all data related to the bootloader will be saved - thus, after restoring from the backup, you will not need additional manipulations, you can immediately boot from this hard drive.

Creating an image

In general, the procedure is similar to that described above for backing up individual partitions. In this case, the advice about clearing free space with “zeros” also applies - if you have free time, do this with all partitions.

Before starting the operation, make sure that none of the partitions on this hard drive are mounted. This can be done by running the mount command without parameters.

Select the partition on which you are going to place the clip file. Of course, this must be a partition from another hard drive. Also make sure that there is enough free space on this partition (for example, using the df utility) - the amount of free space should correspond to the volume of the copied hard drive (when compressed, the image will be smaller, but this depends on the type of data stored).

Mount a backup partition

Sudo mount /dev/sdXY /mnt

Now you can start

Sudo dd if=/dev/sdX bs=1M conv=noerror,sync | lzma -cv > /mnt/hdd.dd.lzma

(here “sdX” is a disk, not a partition! for copying without compression, the command is similar to the one above for backing up a partition)

Depending on the size of the hard drive and the performance of the computer, the procedure may take a long time (up to several hours). When finished, mount the backup partition

Sudo umount /mnt

Recovery from image

Attention! This method involves a complete rollback to the state at the time the archive was created with the replacement of all data!

Before starting work, make sure the power supply is reliable. Connect the AC adapter if you have a laptop, and also use a UPS or stabilizer if possible. High write rates increase the risk of disk damage in the event of a power failure

Make sure that no partition of the disk being restored is in use. Mount a backup partition

Sudo mount /dev/sdXY /mnt

You can start the procedure

Bzip2 -dc /mnt/hdd.dd.bz | sudo dd of=/dev/sdX bs=1M conv=sync,noerror

Or for an uncompressed image

Sudo dd if=/mnt/hdd.dd.bz of=/dev/sdX bs=1M conv=sync,noerror

When finished, mount the backup partition

Sudo umount /mnt

If you want to extract the image to another hard drive, it must be at least as large as the original one. If the new disk is larger, you can expand partitions or create new section on free space using parted/fdisk/gparted/etc

Don't use both hard drives(“duplicate” and “original”) at the same time! When both drives are connected, the system will have two partitions for each UUID, which will lead to operational problems or inability to boot

Mounting the image

By analogy with the partition image, you can work with the hard disk image as with a regular hard drive. In this case, the procedure becomes somewhat more complicated, since the image contains several sections.

If the image is compressed, unpack it. Now “pick up” the image to the loop device

Sudo losetup -fv /media/backup/sda.dd

(With the -f switch, the program will automatically find a free loop device, otherwise you must explicitly specify it)

losetup will display the name of the device used - if you are not working with other image files (iso, encrypted containers, etc.), it will most likely be /dev/loop0

Now we have a device that is a hard drive for the system, but we do not have access to its partitions. The kpartx program will help you get to the partitions (you may need to install the package of the same name)

Sudo kpartx -av /dev/loop0

(Key -a - add partitions for a given device; -v - informative output)

The program will display the names of the created devices for the disk partitions: loop0p1 for the first partition, loop0p2 for the second, similar to the partitions of a regular disk. The device files will be located in the /dev/mapper folder

Now you can work with partitions and FS on them. For example, mount the former sda5 and write files to it

Sudo mount /dev/mapper/loop0p5 /mnt

When finished, unmount the partition

Sudo umount /mnt

Remove partition devices using kpartx

Sudo kpartx -dv /dev/loop0

and release the loop device

Sudo losetup -v -d /dev/loop0

All! The changes are recorded, and your image becomes a regular file again

cp

Here we will look at backup using the cp utility, i.e. using simple copying. Actually, this is not the most optimal method, and it is more suitable for copying the system to another hard drive / partition / computer, rather than for creating a backup copy.

On the other hand, this method has a number of advantages:

Universality - you will find cp in any Linux system

Low resource requirements (due to lack of compression and simplicity of the mechanism)

Ease of further work with the backup (adding/changing/deleting files, extracting the necessary data, etc.)

Making a copy

Create mount points for the root and backup partitions

Sudo mkdir /mnt/(root,backup)

Mount both partitions

Sudo mount /dev/sdXY -o ro /mnt/root sudo mount /dev/sdXY /mnt/backup

Mount partitions for /usr, /boot, etc., if any

Sudo mount /dev/sdXY -o ro /mnt/root/home

Create a folder for your backup on the backup partition

Sudo mkdir /mnt/backup/ubuntu

We can start

Sudo cp -av /mnt/root/* /mnt/backup/ubuntu

(the -a switch enables copying links “as is”, saving all possible file attributes and recursive mode. -v switch - displaying information about what is happening)

Once the process is complete, unmount all partitions

In the future, you can archive your data in any convenient way.

Restoring from a copy

Attention! This method involves a complete rollback to the state at the time the archive was created, replacing all data!

Create mount points for partitions

Sudo mkdir /mnt/(root,backup)

Mount a backup partition

Sudo mount /dev/sdXY -o ro /mnt/backup

Format the root partition and /usr, /boot, etc. partitions, if any. (For formatting partitions while preserving the UUID, see the section about)

Sudo mkfs.reiserfs -l "root" /dev/sdXY sudo mkfs.ext2 -L "boot" /dev/sdXY sudo mkfs.ext4 -L "home" /dev/sdXY

Mount the newly created file systems

The copying process is similar, only in the opposite direction.

Sudo cp /mnt/backup/ubuntu/* -av /mnt/root

Once the copy is complete, edit fstab to correct the partition UUIDs

Unmount the partitions

Sudo umount /mnt/backup /mnt/root/(usr,home,)

squashfs

sudo mkfs.reiserfs -l "root" /dev/sdXY sudo mkfs.ext2 -L "boot" /dev/sdXY sudo mkfs.ext4 -L "home" /dev/sdXY

Mount the newly created file systems

Sudo mount /dev/sdXY /mnt/root sudo mount /dev/sdXY /mnt/root/usr sudo mount /dev/sdXY /mnt/root/var

We're ready to start! To unpack the image, use the unsquashfs utility

Sudo unsquashfs -d /mnt/root -f /mnt/backup/ubuntu-root.sqfs

(The -d switch specifies the path for unpacking, with the -f switch the program will use existing folders instead of trying to create new ones)

Just like when creating an image, you will see a progress bar and lots of other useful information.

When finished, edit fstab, replacing the partitions' UUIDs with new ones (if you formatted the partitions with the same UUIDs, skip this step)

Sudo nano /mnt/root/etc/fstab

Save the file and unmount all partitions

Sudo umount /mnt/backup /mnt/root(/usr,/var,)

Mounting the image

squashfs is mounted like any other image - via a loop device. Kernel support for squashfs is included in many distributions, including Ubuntu, so you can simply use the mount command with the loop option

Sudo mount /media/backup/ubuntu-root.sqfs -o ro,loop /mnt

(The ro option is not required, since writing nothing there will not work anyway)

Now you can copy any necessary files. Adding something this way will not work; to do this you will need to use mksquashfs again

When finished, mount the image as a regular file system

Sudo umount /mnt

rsync

Like cp, rsync works on files rather than block devices. The thing about rsync is that it doesn't copy files that are already at the destination. By default, it checks the size and modification time of files, but you can also check the hash (usually this is done when increased security is needed).

Easy to use

The rsync syntax is similar to cp:

Rsync -a /mnt/root /mnt/backup

The -a parameter is often sufficient; it provides what is most needed: recursive copying of directories, saving information about the owner and group, etc. To display detailed information The -v switch is used for copying, be careful with it, you may miss an error message in the data stream. The -x switch ensures that rsync does not go beyond the specified filesystem.

The rsync documentation describes a lot of options. For example, there are those that allow you to copy over SSH, or delete a file from the destination if it was deleted in the source directory.

Smart copying reduces system downtime. We run rsync directly on a running system, the data in which is constantly changing, rsync copies the data, say, within a few hours. Then we switch the system to read-only, run rsync again, now it copies only those files that have changed over these few hours. In a few minutes we have a complete copy of the original file system. Downtime was reduced by an order of magnitude compared to offline copying. And in some cases, one online copy will be enough without converting the system to read-only.

Saving previous copies

Strictly speaking, rsync is not a backup tool - it is a synchronization tool. This is important when creating regular copies, because if any important file was deleted in the source working directory, rsync will delete it in the backup copy as well. To improve data security, it is advisable to save old backup copies. However, simply storing multiple copies will require a lot of hard drive space. If copies have many identical files, then this leads to unnecessary redundancy. This problem can be solved by using hard links.

The point is that in modern file systems(including Ext4), addressing a file is done in two stages: the file name indicates a unique file number (index descriptor or i-node), and the data itself is associated with this number. Any file name is, in fact, a hard link to this number. Consequently, a file (data set) can have several names and be in different directories, and this eliminates redundancy in case of need to duplicate files (after all, a hard link takes up little memory). The data itself is not deleted until the last hard link is requested to be deleted.

A significant limitation is that hard links are only possible within the same file system.

Synchronizing directory contents for the current backup with the source directory:

Rsync \ --archive \ --delete --delete-excluded \ # deleting files that do not exist in the source and excluded files from the backup--progress\ # display information about the progress of the transfer"/home/user/Files/" \ # directory source"/backup/latest/" \ # directory for current backup--exclude = "/Public/" # exclude unnecessary directories

In the “/backup/latest/” directory, a copy of all necessary files and directories from the source will be created and everything unnecessary will be removed.

Creating another current backup without redundancy:

cp\--archive\ # save all Additional information about files--link\ # use hard links for files - eliminate redundancy"/backup/latest/" \ # source is the current backup obtained above "/backup/$(date +%Y-%m-%d_%H-%M-%S) /" # destination - directory with date in name for convenience (see man date)

The next time you create a backup, rsync will delete files in the “ /backup/latest/ ” directory that were deleted/excluded/changed in the source directory (changed files are first deleted and then written a new version). However, only the file names (the same hard links) will be deleted; the files themselves (data) will be saved, since hard links were created to them in a neighboring directory with the “cp” command.

Other tools

There are many applications for creating backups in Linux. You can search for "backup" in the Ubuntu App Center to find backup programs available in Ubuntu.

For a corporate environment and simply for fairly large-scale and critical backup tasks, we can recommend understanding one of the most popular and powerful systems backup for Linux called Bacula

By the way, you can also find Russian-language manuals on the Internet.

Parted Magic

Parted Magic is another great one, but paid a distribution kit containing a whole collection of tools for backing up and restoring information, working with disks and partitions, as well as recovering lost data. It supports many file systems, LVM2 and RAID (both hardware and software) and contains tools such as fsarchiver, GParted, the aforementioned Clonezilla, and everything that is required for the methods described in this article. In addition, the distribution includes a web browser and some other additional software. The distribution is translated into several languages, including Russian, and has a full-fledged graphical interface.

LParted

LParted is a full-featured LiveCD designed primarily for working with hard disk partitions (HDDs), permanently deleting or restoring data, and testing hardware. LiveCD based on Lubuntu Linux. LParted is a functional analogue of Parted Magic.

I would like to add here about SystemRescueCD and others

A little more about saving data

For important data, you can make a mirror partition on two disks. To do this, it is not at all necessary to have a RAID controller and disks of the same size - you can, for example, assemble a mirror from an 80 GB old drive and an 80 GB partition on a new one. Mirroring can be implemented using LVM or software RAID. However, this method is useless if, for example, a voltage of ~220V hits the +5V bus or a meteorite falls on system unit computer.

IT geeks who have their own server at home can expand the idea of mirroring and use DRBD. The same RAID-1, but hard disks are situated in different computers, which increases reliability.

A modern convenient solution is to back up data to the “clouds”, for example, with using Ubuntu One, Dropbox, http://www.adrive.com/ and others.

Neither mirroring nor replication on Ubuntu One will save you from accidentally pressing Delete, so in any case, make “classic” backups. And one day, all your hard work and efforts will be rewarded.

Popular in the category: