tar and ssh together to backup onto remote machine

On my laptop, I have working installs of some programs and I want to back them up as tar.gz files on a server that I can access via ssh.

The disk on the server is formatted in windows NTFS. It is that way because somebody wanted to be compatible with MS Windows, and so it is not a good idea to simply rsync the files onto that disk. I loose user permissions and ownership.  So it is better to make a big tarball and copy that over. The problem is that Program.tar.gz is too huge to fit on the laptop.

For a small program, what I'd do is make a tarball on my machine and copy, like so

First, I have to become root because some files in the program are root-read-only.

$ sudo su

# cd /usr/local

# tar ~/Program-20120928.tar.gz Program

# cd

# rsync -e ssh Program-20120928.tar.gz  myusername@mybigserver.quant.ku.edu:/home/LinuxProgAddons

That failed because my computer does not have enough disk space to hold a tarball so huge as that.

I need to take a smarter route. The idea is to run the tar program and pipe the results to the remote server, thereby avoiding the need to save the giant tarball on my laptop.

First, I tested this without compression, just uses "-" (the standard output) for the target of the tar command and then a pipe with ssh to write the result to the server

# tar cf - Program | ssh myuseraccount@mybigserver.quant.ku.edu  "cat > Program.tar"

That *should* drop an uncompressed tarball onto my user home account on mybigserver.

After that, I have to log into mybigserver and shrink it down.

$ gzip Program.tar

I wondered if I could get compression finished in one shot.

# tar czf  -  Program | ssh pauljohn32@crmda-100.quant.ku.edu  "cat > Program-20120928.tar.gz"

That seems to work, but the tarballs that were created are not exactly identical. Here are the file sizes:

-rw-rw-r-- 1 pauljohn32 pauljohn32 2233615001 Sep 28 13:14 Program-20120928.tar.gz
-rw-rw-r-- 1 pauljohn32 pauljohn32 2233615013 Sep 28 13:08 Program.tar.gz

But the difference between them is very miniscule, and so I'd probably gamble that the contents of the archives are the same.

I've tested this with 3 programs and restored them. It appears to be a workable metaphor.

About pauljohn

Paul E. Johnson is a Professor of Political Science at the University of Kansas. He is an avid Linux User, an adequate system administrator and C programmer, and humility is one of his greatest strengths.
This entry was posted in Uncategorized. Bookmark the permalink.