Difference between revisions of "Service/backup"
| (10 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
== Locations == | == Locations == | ||
| − | * /backup on | + | * /backup on [[Machine/justice]]: daily filesystem backups for [[Machine/freedom]] |
| − | * backup | + | * /backup on [[Machine/freedom]]: daily filesystem backups for [[Machine/justice]] |
| − | * /backup on sunjammer.sugarlabs.org ( | + | * /backup on [[Machine/housetree]]: daily filesystem backups for [[Machine/sunjammer]] |
| + | * /backup on sunjammer.sugarlabs.org: daily database dumps (ldap, mysql and pgsql) for [[Machine/sunjammer]] | ||
== Administrative contact == | == Administrative contact == | ||
| − | + | * Bernie Innocenti <bernie AT sugarlabs DOT org> | |
| − | bernie AT sugarlabs DOT org | ||
| − | |||
== Sysadmins == | == Sysadmins == | ||
| Line 14: | Line 13: | ||
* [[User:Bernie|Bernie Innocenti]] | * [[User:Bernie|Bernie Innocenti]] | ||
| − | |||
| − | == | + | == wizbackup == |
| − | + | * [http://codewiz.org/gitweb/wizbackup.git Source code] | |
| + | * [http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL Installation instructions] | ||
| − | + | This is the main backup script for all [[Machine|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps). | |
| − | + | The `wizbackup-driver` script runs daily on the backup servers from `/etc/cron.daily/wizbackup`. | |
| + | It reads a list of hostnames to backup from `/backup/HOSTS/<machine-group-name>` and an optional list of files exclusions from `/backup/EXCLUDES`, in the same format taken by the `--exclude` flag of rsync. | ||
| − | + | For each machine in the list, `wizbackup-driver` invokes `wizbackup` with this command-line: | |
| − | + | wizbackup <hostname>:/ <local-backup-dir> --exclude-file /backup/EXCLUDE/ALWAYS --exclude-file /backup/EXCLUDE/<hostname> | |
| + | |||
| + | Wizbackup uses rsync and ssh to connect to the remote machine. Hence, the backup server must be authorized to connect as root to all backup hosts (e.g.: by storing the identity in `/root/.ssh/authorized_keys`). Wizbackup reads the ssh private key from `/etc/wizbackup/ssh_id`). | ||
== Sunjammer == | == Sunjammer == | ||
| Line 34: | Line 36: | ||
The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups. | The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups. | ||
| − | == Restoring files from duplicity backups == | + | == Restoring files from duplicity backups (OBSOLETE) == |
| + | |||
| + | '''We no longer use Duplicity.''' | ||
To restore a backup try something like: | To restore a backup try something like: | ||
| − | duplicity restore -t 7D | + | duplicity restore -t 7D --file-to-restore <source/path/to/restore/from> scp://sugarbackup@backup.sugarlabs.org/backup/<machine.sugarlabs.org> </destination/path/to/restore/to> |
| − | |||
| − | |||
Example: | Example: | ||
| − | duplicity restore -t 7D | + | duplicity restore -t 7D --file-to-restore var/lib/mysql/wordpress scp://sugarbackup@backup.sugarlabs.org/backup/sunjammer.sugarlabs.org /home/bernie/restore/wordpress |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
[[Category:Service|zzz]] | [[Category:Service|zzz]] | ||
Revision as of 19:50, 5 April 2015
Locations
- /backup on Machine/justice: daily filesystem backups for Machine/freedom
- /backup on Machine/freedom: daily filesystem backups for Machine/justice
- /backup on Machine/housetree: daily filesystem backups for Machine/sunjammer
- /backup on sunjammer.sugarlabs.org: daily database dumps (ldap, mysql and pgsql) for Machine/sunjammer
Administrative contact
- Bernie Innocenti <bernie AT sugarlabs DOT org>
Sysadmins
For non-emergency calls, preferably send email to the administrative contact.
wizbackup
This is the main backup script for all Sugar Labs Machines. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).
The `wizbackup-driver` script runs daily on the backup servers from `/etc/cron.daily/wizbackup`. It reads a list of hostnames to backup from `/backup/HOSTS/<machine-group-name>` and an optional list of files exclusions from `/backup/EXCLUDES`, in the same format taken by the `--exclude` flag of rsync.
For each machine in the list, `wizbackup-driver` invokes `wizbackup` with this command-line:
wizbackup <hostname>:/ <local-backup-dir> --exclude-file /backup/EXCLUDE/ALWAYS --exclude-file /backup/EXCLUDE/<hostname>
Wizbackup uses rsync and ssh to connect to the remote machine. Hence, the backup server must be authorized to connect as root to all backup hosts (e.g.: by storing the identity in `/root/.ssh/authorized_keys`). Wizbackup reads the ssh private key from `/etc/wizbackup/ssh_id`).
Sunjammer
Machine/sunjammer's filesystem is being backed up on Machine/treehouse using rsync + 7 days of history with hard links.
The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups.
Restoring files from duplicity backups (OBSOLETE)
We no longer use Duplicity.
To restore a backup try something like:
duplicity restore -t 7D --file-to-restore <source/path/to/restore/from> scp://sugarbackup@backup.sugarlabs.org/backup/<machine.sugarlabs.org> </destination/path/to/restore/to>
Example:
duplicity restore -t 7D --file-to-restore var/lib/mysql/wordpress scp://sugarbackup@backup.sugarlabs.org/backup/sunjammer.sugarlabs.org /home/bernie/restore/wordpress