Skip to main content

How To Resume Partially Transferred Files Over SSH Using Rsync

What is Rsync?

For those wondering, Rsync is a fast, versatile file copying utility that can be used to copy and transfer files or folders to and from remote and local systems. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Just like SCP, rsync will also copy files over SSH. In case you wanted to download or transfer large size files and folders over SSH, I recommend you to use rsync utility as described below.

Resume Partially Transferred Files Using Rsync

Please that the rsync utility should be installed on both sides (i.e. remote and local systems) in order to resume partially transferred files. Rsync is available in the official repositories of most Linux distributions, so you can install it using the distribution’s default package manager.

On Arch Linux and its variants, install Rsync using command:

$ sudo pacman -S rsync

On Debian, Ubuntu, Linux Mint:

$ sudo apt-get install rsync

On Fedora:

$ sudo dnf install rsync

On RHEL, CentOS:

$ sudo yum install rsync

On SUSE/openSUSE:

$ sudo zypper install rsync

Now let us see an example. I am going to copy Ubuntu 18.04 ISO from my local system to remote system with command:

$ scp ubuntu.iso sk@192.168.225.22:/home/sk/

Here,

  • sk is my remote system’s username
  • 192.168.225.22 is the IP address of the remote machine.
  • /home/sk/ is the remote destination.

Now, I terminated the copy process forcibly by pressing CTRL+C.

Sample output:

sk@192.168.43.2's password: 
ubuntu-16.04-desktop-amd64.iso 26% 372MB 26.2MB/s 00:39 ETA^c

Copy file using scp command on Linux

As you see in the above output, I terminated the copy process when it reached 33%.

If I re-run the above command, it will simply overwrite the existing file. In other words, the copy process will not resume where I left it off.


In order to resume the copy process to copy the partially transferred ISO file, use rsynccommand as shown below.

$ rsync -P -rsh=ssh ubuntu.iso sk@192.168.225.22:/home/sk/

Sample output:

sk@192.168.225.22's password: 
sending incremental file list
ubuntu.iso
834.00M 100% 14.81MB/s 0:00:56 (xfr#1, to-chk=0/1)

Resume Partially Transferred Files Using Rsync Over SSH On Linux

See? Now, the copying process is resumed where we left it off earlier.

You can also use “–partial” instead of parameter “-P” like below.


$ rsync --partial -rsh=ssh ubuntu.iso sk@192.168.225.22:/home/sk/

Here, the parameter “–partial” or “-P” tells the rsync command to keep the partial downloaded file and resume the process.

Check if the file is really copied or not to the remote system using command:

$ ssh sk@192.168.225.22 ls -lh
sk@192.168.225.22's password: 
total 835M
-rw-rw-r-- 1 sk sk 834M Jul 26 2019 ubuntu.iso

Run commands remote systems over SSH

As you see in the above output, the file Ubuntu.iso is copied to the remote system and its size is 835MB.

Comments

Post a Comment

Popular posts from this blog

4 Ways to Communicate Across Browser Tabs in Realtime

1. Local Storage Events You might have already used LocalStorage, which is accessible across Tabs within the same application origin. But do you know that it also supports events? You can use this feature to communicate across Browser Tabs, where other Tabs will receive the event once the storage is updated. For example, let’s say in one Tab, we execute the following JavaScript code. window.localStorage.setItem("loggedIn", "true"); The other Tabs which listen to the event will receive it, as shown below. window.addEventListener('storage', (event) => { if (event.storageArea != localStorage) return; if (event.key === 'loggedIn') { // Do something with event.newValue } }); 2. Broadcast Channel API The Broadcast Channel API allows communication between Tabs, Windows, Frames, Iframes, and  Web Workers . One Tab can create and post to a channel as follows. const channel = new BroadcastChannel('app-data'); channel.postMessage(data); And oth...
Post a Comment
Read more

Certbot SSL configuration in ubuntu

  Introduction Let’s Encrypt is a Certificate Authority (CA) that provides an easy way to obtain and install free  TLS/SSL certificates , thereby enabling encrypted HTTPS on web servers. It simplifies the process by providing a software client, Certbot, that attempts to automate most (if not all) of the required steps. Currently, the entire process of obtaining and installing a certificate is fully automated on both Apache and Nginx. In this tutorial, you will use Certbot to obtain a free SSL certificate for Apache on Ubuntu 18.04 and set up your certificate to renew automatically. This tutorial will use a separate Apache virtual host file instead of the default configuration file.  We recommend  creating new Apache virtual host files for each domain because it helps to avoid common mistakes and maintains the default files as a fallback configuration. Prerequisites To follow this tutorial, you will need: One Ubuntu 18.04 server set up by following this  initial ...
Post a Comment
Read more

Working with Node.js streams

  Introduction Streams are one of the major features that most Node.js applications rely on, especially when handling HTTP requests, reading/writing files, and making socket communications. Streams are very predictable since we can always expect data, error, and end events when using streams. This article will teach Node developers how to use streams to efficiently handle large amounts of data. This is a typical real-world challenge faced by Node developers when they have to deal with a large data source, and it may not be feasible to process this data all at once. This article will cover the following topics: Types of streams When to adopt Node.js streams Batching Composing streams in Node.js Transforming data with transform streams Piping streams Error handling Node.js streams Types of streams The following are four main types of streams in Node.js: Readable streams: The readable stream is responsible for reading data from a source file Writable streams: The writable stream is re...
Post a Comment
Read more