[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[ale] rsync with big files (.5GB and above)
- Subject: [ale] rsync with big files (.5GB and above)
- From: jlightner at water.com (Jeff Lightner)
- Date: Mon, 30 Apr 2007 15:59:07 -0400
- In-reply-to: <[email protected]>
- References: <[email protected]>
I don't think it's a generic rsync issue. At a prior job we used to
run all day rsync scripts to transfer databases - of course that was all
done within the same VLAN.
Perhaps if you script it so it just resumes where it left off if it
drops the connection?
From: ale-bounces at ale.org [mailto:ale-bounces at ale.org] On Behalf Of Greg
To: ale at ale.org
Sent: Monday, April 30, 2007 3:42 PM
To: Atlanta Linux Enthusiasts
Subject: [ale] rsync with big files (.5GB and above)
I setup a dreamhost account like I think I mentioned before. I've
been rsync'ing my encrypted backups there for a few weeks.
As part of my implementation process I started only sending small
files (via the --max-size option of rsync).
When I was working with files below .5 GB I had the occasional
time-out, but in general it worked most of the time.
I set my max to 2 GB about a week ago. Since then I get a few
additional files copied each attempt, but I don't think I have sent
more than 2 files in any one rsync instance.
Does anyone know if this is a generic issue with rsync, or am I just
seeing more timeouts because my discrete rsync instances are running
longer and dreamhost is having issues. (I've never run more than 3 or
4 hours without a timeout).
FYI: I have --timeout set to 5 minutes in my backup script, but I've
been manually running rsync during the day with timeouts as long as 2
hours. Longer timeouts seem more reliable, but rsync still fails
after a couple of files. And I think the timeout is per i/o, not per
The Norcross Group
Forensics for the 21st Century
Ale mailing list
Ale at ale.org