[Dev] Get your backups! Hot off the press!

[Freemor]

On Mon, Jan 22, 2018 at 06:10:24PM -0500, Luke Shumaker wrote:
> On Sat, 20 Jan 2018 20:15:42 -0500,
> Freemor wrote:
> > 
> > That sounds like an excellent idea. I'd love to pull those down and pop them on
> > my NAS. 
> 
> Thanks!
> 
> >  About how big are the back-ups in question?
> 
> Right now, about 20GB from winston, and 35GB from proton; 55GB in all.
> We'll see how quickly that grows with the daily incremental backups.

  Thanks, I've pulled winston already with a similar wget command. It took
about 6 hours as it only came in at about 800 KB/s average. 

> 
> >  Preferred method of grabbing them? rsync,scp,etc.
> 
> As I said, they're accessible right now over HTTPS.
> 
>     https://winston.parabola.nu/backup/
>     https://proton.parabola.nu/backup/
> 
> Because files are immutable after they are created (until they are
> deleted), there's no need for a complex protocol like rsync, though I
> probably will add rsync at some point.

  Yes re-read and so went for the first grab. I'm new to the mailing list but
have been around long enough to know that resources for Parabola are not
limitless. Thus the asking about preferred methodologies

> 
> I pull them down with
> 
>     wget --mirror --continue --no-parent --accept=gpg https://winston.parabola.nu/backup/ https://proton.parabola.nu/backup/
> 
> >  Rate limiting preferences? So as not to overload server.
> 
> Please don't make more than 1 connection to a server at a time (don't
> try to download things from the same server in parallel).
> 
> Since the files are chunked so that no one file is more than about
> 200MB, they won't cause the large-file long-running connection issues
> that we've seen in the past with ISOs; so that's not a concern.
> 
> Since backups are taken daily, there's no reason to download more
> frequently than that, but if nothing's changed, then it's also
> downloading basically nothing, so there's no harm in it either.

I'll probably have to limit my downloads a bit as I'm doing this over my
residential ISP line. (Fiber) There is no stated cap but I still want to be a
little cautious about multi hour/multi GB downloads.

> 
> >  Other things I've forgotten to ask that you want to add.
> 
> The `wget` command I gave above will download new backups and keep the
> old ones in place; it will never delete any old backups.  When you do
> go to delete old stuff, remember that it's pointless to keep an
> incremental backup around if you don't also keep the previous full
> backup, and all the incrementals in between them.

I'll have to ponder the best balance, and I do understand about incremental
backups but thanks for the reminder. 

I'll work with the winston back-up for a bit.. Get a good routine at my end..
then I'll see about adding proton. 

> 
> I tried to cover most everything in the original email.

Yep, I'm just being cautious and not wanting to step on toes or accidentally
knock something over.

> 
> -- 
> Happy hacking,
> ~ Luke Shumaker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <https://lists.parabola.nu/pipermail/dev/attachments/20180122/37b6b46f/attachment.sig>