Jamie Fargen's Weblog

Tag: scp

Distributed Copy Util

by on Oct.29, 2012, under Work and stuff

Say you have 1 file that is 10TBytes, like a VM image, that you need to copy between say two nfs file stores (that you don’t control). Your only problem is a single host is only capable of 50MB/sec over a 1Gb/sec network connection. The good news is you have hundreds of hosts all mounting these two nfs filesystems. Maybe you can lash 10-12 hosts together to greatly the reduce the time it takes to copy the file? Then check out scp-tsunami – http://code.google.com/p/scp-tsunami.

Leave a Comment :, , , , , , , more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Blogroll

A few highly recommended websites...