Service/backup - Revision history

Bernie at 20:09, 2 July 2024

2024-07-02T20:09:53Z

← Older revision		Revision as of 16:09, 2 July 2024
Line 10:		Line 10:

	* [https://codewiz.org/gitweb/wizbackup.git Source code]		* [https://codewiz.org/gitweb/wizbackup.git Source code]
	* [https://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/~~INSTALL Installation instructions~~]		* [https://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/README.md README]

	This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapshots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).		This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapshots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).

Bernie at 22:01, 1 July 2024

2024-07-01T22:01:32Z

← Older revision		Revision as of 18:01, 1 July 2024
Line 1:		Line 1:
	== Locations ==		== Locations ==

	* /backup on [[Machine/~~justice~~]]: daily filesystem backups for [[Machine/~~freedom~~]]		* /backup on [[Machine/papert]]: daily filesystem backups for [[Machine/bender]] and [[Machine/sunjammer]]
	* /backup on [[Machine/~~freedom~~]]: daily filesystem backups for ~~[[Machine/justice]]~~		* /backup on [[Machine/bender]]: daily filesystem backups for all VMs hosted on [[Machine/papert]]
	* /backup on ~~sunjammer.sugarlabs.org: daily database dumps (ldap, mysql and pgsql) for~~ [[Machine/~~sunjammer~~]]

	== Administrative contact ==		== Administrative contact ==
Line 23:		Line 22:

	Wizbackup uses rsync and ssh to connect to the remote machine. Hence, the backup server must be authorized to connect as root to all backup hosts (e.g.: by storing the identity in `/root/.ssh/authorized_keys`). Wizbackup reads the ssh private key from `/etc/wizbackup/ssh_id`).		Wizbackup uses rsync and ssh to connect to the remote machine. Hence, the backup server must be authorized to connect as root to all backup hosts (e.g.: by storing the identity in `/root/.ssh/authorized_keys`). Wizbackup reads the ssh private key from `/etc/wizbackup/ssh_id`).

	~~== Sunjammer ==~~

	~~[[Machine/sunjammer]]'s filesystem is being backed up on [[Machine/treehouse]] using rsync + 7 days of history with hard links.~~

	~~The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups.~~

	[[Category:Service\|zzz]]		[[Category:Service\|zzz]]

Srevin03: fix: typo

2021-01-05T16:54:37Z

fix: typo

← Older revision		Revision as of 12:54, 5 January 2021
Line 13:		Line 13:
	* [https://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL Installation instructions]		* [https://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL Installation instructions]

	This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem ~~snapashots~~ of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).		This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapshots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).

	The `wizbackup-driver` script runs daily on the backup servers from `/etc/cron.daily/wizbackup`.		The `wizbackup-driver` script runs daily on the backup servers from `/etc/cron.daily/wizbackup`.

Bernie at 12:49, 5 January 2021

2021-01-05T12:49:44Z

← Older revision		Revision as of 08:49, 5 January 2021
Line 3:		Line 3:
	* /backup on [[Machine/justice]]: daily filesystem backups for [[Machine/freedom]]		* /backup on [[Machine/justice]]: daily filesystem backups for [[Machine/freedom]]
	* /backup on [[Machine/freedom]]: daily filesystem backups for [[Machine/justice]]		* /backup on [[Machine/freedom]]: daily filesystem backups for [[Machine/justice]]
	* /backup on [[Machine/housetree]]: daily filesystem backups for [[Machine/sunjammer]]
	* /backup on sunjammer.sugarlabs.org: daily database dumps (ldap, mysql and pgsql) for [[Machine/sunjammer]]		* /backup on sunjammer.sugarlabs.org: daily database dumps (ldap, mysql and pgsql) for [[Machine/sunjammer]]

	== Administrative contact ==		== Administrative contact ==
	* Bernie Innocenti <bernie AT sugarlabs DOT org>
	~~== Sysadmins ==~~

	~~For non-emergency calls, preferably send email to the administrative contact.~~

	* [[User:Bernie\|Bernie Innocenti]]		* [[User:Bernie\|Bernie Innocenti]]

	== wizbackup ==		== wizbackup ==

	* [~~http~~://codewiz.org/gitweb/wizbackup.git Source code]		* [https://codewiz.org/gitweb/wizbackup.git Source code]
	* [~~http~~://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL Installation instructions]		* [https://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL Installation instructions]

	This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).		This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).

Bernie: remove references to duplicity

2021-01-05T12:47:48Z

remove references to duplicity

← Older revision		Revision as of 08:47, 5 January 2021
Line 35:		Line 35:

	The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups.		The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups.

	~~== Restoring files from duplicity backups (OBSOLETE) ==~~

	~~'''We no longer use Duplicity.'''~~

	~~To restore a backup try something like:~~

	~~duplicity restore -t 7D --file-to-restore <source/path/to/restore/from> scp://sugarbackup@backup.sugarlabs.org/backup/<machine.sugarlabs.org> </destination/path/to/restore/to>~~


	~~Example:~~

	~~duplicity restore -t 7D --file-to-restore var/lib/mysql/wordpress scp://sugarbackup@backup.sugarlabs.org/backup/sunjammer.sugarlabs.org /home/bernie/restore/wordpress~~

	[[Category:Service\|zzz]]		[[Category:Service\|zzz]]

Bernie at 23:50, 5 April 2015

2015-04-05T23:50:10Z

← Older revision		Revision as of 19:50, 5 April 2015
Line 16:		Line 16:
	== wizbackup ==		== wizbackup ==

	* C[http://codewiz.org/gitweb/wizbackup.git\|Source ~~ode~~]		* [http://codewiz.org/gitweb/wizbackup.git Source code]
	* I[http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL\|Installation ~~nstructions~~]		* [http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL Installation instructions]

	This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).		This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).

Bernie at 23:49, 5 April 2015

2015-04-05T23:49:20Z

← Older revision		Revision as of 19:49, 5 April 2015
Line 16:		Line 16:
	== wizbackup ==		== wizbackup ==

	* [http://codewiz.org/gitweb/wizbackup.git\|Source ~~code~~]		* C[http://codewiz.org/gitweb/wizbackup.git\|Source ode]
	* [http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL\|Installation ~~instructions~~]		* I[http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL\|Installation nstructions]

	This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).		This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).
Line 36:		Line 36:
	The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups.		The MySQL, PostgreSQL and LDAP databases are being backed up daily on backup.sugarlabs.org, encrypted with gpg. TODO: at the moment, we don't have a script to cleanup old backups.

	== Restoring files from duplicity backups ==		== Restoring files from duplicity backups (OBSOLETE) ==

			'''We no longer use Duplicity.'''

	To restore a backup try something like:		To restore a backup try something like:

	duplicity restore -t 7D --file-to-restore <source/path/to/restore/from> scp://sugarbackup@backup.sugarlabs.org/backup/<machine.sugarlabs.org> </destination/path/to/restore/to>		duplicity restore -t 7D --file-to-restore <source/path/to/restore/from> scp://sugarbackup@backup.sugarlabs.org/backup/<machine.sugarlabs.org> </destination/path/to/restore/to>

	* Replace <> with your values


Line 48:		Line 48:

	duplicity restore -t 7D --file-to-restore var/lib/mysql/wordpress scp://sugarbackup@backup.sugarlabs.org/backup/sunjammer.sugarlabs.org /home/bernie/restore/wordpress		duplicity restore -t 7D --file-to-restore var/lib/mysql/wordpress scp://sugarbackup@backup.sugarlabs.org/backup/sunjammer.sugarlabs.org /home/bernie/restore/wordpress

	~~== See Also ==~~

	* There are instructions on how to set up a backup here: [[Infrastructure_Team/Template_virtual_machine_lucid]]

	[[Category:Service\|zzz]]		[[Category:Service\|zzz]]

Bernie at 23:45, 5 April 2015

2015-04-05T23:45:43Z

← Older revision		Revision as of 19:45, 5 April 2015
Line 7:		Line 7:

	== Administrative contact ==		== Administrative contact ==
			* Bernie Innocenti <bernie AT sugarlabs DOT org>
	* Bernie Innocenti <bernie AT sugarlabs DOT org>

	== Sysadmins ==		== Sysadmins ==

Bernie at 23:43, 5 April 2015

2015-04-05T23:43:58Z

← Older revision		Revision as of 19:43, 5 April 2015
Line 18:		Line 18:
	== wizbackup ==		== wizbackup ==

	* [[http://codewiz.org/gitweb/wizbackup.git\|Source code]]		* [http://codewiz.org/gitweb/wizbackup.git\|Source code]
	* [[http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL\|Installation instructions]]		* [http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL\|Installation instructions]

	This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).		This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).

	The `wizbackup-driver` script runs daily on the backup servers from `/etc/cron.daily/wizbackup`.		The `wizbackup-driver` script runs daily on the backup servers from `/etc/cron.daily/wizbackup`.
	It reads a list of hostnames to backup from `/backup/HOSTS/<machine-group-name>` and an optional list of files exclusions from `/backup/EXCLUDES`.		It reads a list of hostnames to backup from `/backup/HOSTS/<machine-group-name>` and an optional list of files exclusions from `/backup/EXCLUDES`, in the same format taken by the `--exclude` flag of rsync.

			For each machine in the list, `wizbackup-driver` invokes `wizbackup` with this command-line:

			wizbackup <hostname>:/ <local-backup-dir> --exclude-file /backup/EXCLUDE/ALWAYS --exclude-file /backup/EXCLUDE/<hostname>

			Wizbackup uses rsync and ssh to connect to the remote machine. Hence, the backup server must be authorized to connect as root to all backup hosts (e.g.: by storing the identity in `/root/.ssh/authorized_keys`). Wizbackup reads the ssh private key from `/etc/wizbackup/ssh_id`).

	== Sunjammer ==		== Sunjammer ==

Bernie at 23:35, 5 April 2015

2015-04-05T23:35:43Z

← Older revision		Revision as of 19:35, 5 April 2015
Line 1:		Line 1:
	== Locations ==		== Locations ==

	* /backup on [[~~Machine/treehouse\|~~Machine/justice]]: daily filesystem backups for [[~~Machine/jita\|~~Machine/freedom]]		* /backup on [[Machine/justice]]: daily filesystem backups for [[Machine/freedom]]
	* /backup on [[~~Machine/housetree\|~~Machine/freedom]]: daily filesystem backups for [[~~Machine/sunjammer\|~~Machine/justice]]		* /backup on [[Machine/freedom]]: daily filesystem backups for [[Machine/justice]]
	* /backup on [[Machine/housetree]]: daily filesystem backups for [[Machine/sunjammer]]		* /backup on [[Machine/housetree]]: daily filesystem backups for [[Machine/sunjammer]]
	* /backup on sunjammer.sugarlabs.org: daily database dumps (ldap, mysql and pgsql) for [[Machine/sunjammer]]		* /backup on sunjammer.sugarlabs.org: daily database dumps (ldap, mysql and pgsql) for [[Machine/sunjammer]]
Line 8:		Line 8:
	== Administrative contact ==		== Administrative contact ==

	bernie AT sugarlabs DOT org		* Bernie Innocenti <bernie AT sugarlabs DOT org>

	== Sysadmins ==		== Sysadmins ==
Line 16:		Line 16:
	* [[User:Bernie\|Bernie Innocenti]]		* [[User:Bernie\|Bernie Innocenti]]

	== ~~backup.sugarlabs.org~~ ==		== wizbackup ==

	~~This is the main backup server for all~~ [[~~Machine~~\|~~Sugarlabs Machines~~]]~~. The hostname backup.sugarlabs~~.org ~~is an alias for owl.laptop.org, a machine hosted at OLPC~~.		* [[http://codewiz.org/gitweb/wizbackup.git\|Source code]]
			* [[http://codewiz.org/gitweb/wizbackup.git/blob/HEAD:/INSTALL\|Installation instructions]]

	~~Backup account:~~		This is the main backup script for all [[Machine\|Sugar Labs Machines]]. It's a minimalist script based on rsync and keeps a history of N days using hard-links. While it's very very simple, wizbackup gets the job done and has the added benefit of not requiring any tools to read or restore a backup. The advantage over similar backup solution is that backups are just plain filesystem snapashots of the source machines; the main disadvantage is poor performance with large files which change often, such as logs or databases (my solution is to exclude these and backup the data using ad-hoc methods, such as SQL dumps).
			The `wizbackup-driver` script runs daily on the backup servers from `/etc/cron.daily/wizbackup`.
	~~sugarbackup@~~backup.~~sugarlabs~~.~~org:/home/sugarbackup/~~		It reads a list of hostnames to backup from `/backup/HOSTS/<machine-group-name>` and an optional list of files exclusions from `/backup/EXCLUDES`.

	~~We use~~ the ~~script system~~-~~full~~-backup ~~to perform the~~ daily ~~backups with duplicity~~. ~~Each~~ machine backup ~~is encrypted with a different password. If you need a password, ask [[User:Bernie]]~~.

	== Sunjammer ==		== Sunjammer ==