<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Bulletproof backups for MySQL</title>
	<atom:link href="http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/feed/" rel="self" type="application/rss+xml" />
	<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/</link>
	<description>A blog about the web</description>
	<lastBuildDate>Thu, 09 Sep 2010 14:20:43 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.0.1</generator>
	<item>
		<title>By: lv</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-22313</link>
		<dc:creator>lv</dc:creator>
		<pubDate>Mon, 16 Aug 2010 00:17:05 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-22313</guid>
		<description>To avoid issues, I do a simple file test in my rsync cron script which gets deleted at the finish of the script. If the script gets run a second time it checks for the file, and aborts if found. This prevents multiple instances being run each day, which can cause issues for the initial long running backup.</description>
		<content:encoded><![CDATA[<p>To avoid issues, I do a simple file test in my rsync cron script which gets deleted at the finish of the script. If the script gets run a second time it checks for the file, and aborts if found. This prevents multiple instances being run each day, which can cause issues for the initial long running backup.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Dan</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-19952</link>
		<dc:creator>Dan</dc:creator>
		<pubDate>Sat, 10 Apr 2010 04:36:14 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-19952</guid>
		<description>Great information. My concerns are managing my backups in bulk (if possible) With 100 + accounts on one server, it get very time consuming.</description>
		<content:encoded><![CDATA[<p>Great information. My concerns are managing my backups in bulk (if possible) With 100 + accounts on one server, it get very time consuming.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Lawrence</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-19310</link>
		<dc:creator>Lawrence</dc:creator>
		<pubDate>Sat, 20 Mar 2010 13:30:55 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-19310</guid>
		<description>I do our backups in a similar manner (and I *do* get to test using backups when clients go whoops!)

How do I do it?


Step 1:

First is to backup the database(s) to the filesystem in a common folder.

MySQL daily backups to a folder under /home/dbbackup for all DB&#039;s using an easy to install backup shell script (automysqlbackup)

http://sourceforge.net/projects/automysqlbackup/

apt-get install automysqlbackup on debian based systems.

automysqlbackup gets run daily, and it automatically handles daily, weekly, monthly folders + rotation (no need to reinvent the wheel).


Step 2:

rsync my entire home folder system (which is where I store all my user specific data) to another server.

This also runs daily (usually in our early 2am - 4am period).


Step 3: Optional

I also do a weekly backup of the complete backup folder to a different server so I have some &quot;history&quot;.

To make it even safer, I also do a monthly offsite of the backups.


Our specific setup is that 
I usually have a central backup server that I send data to (stuck on a spare drive on a less used web server).

Secondary and Tertiary backup backups are run from that.

Caveats - if your database is in iffy condition  - eg latin1 encoding, with UTF-8 data, or similar, or crashed tables, the backup will not be 100%.  automysqlbackup can be configured to email you on issues.  encoding related ones will not be seen until a restore is attempted, but other issues will be.  
I have successfully restored even with &quot;corrupt&quot; encoding data, whether it was latin1 -&gt; utf8 or utf8 double encoding.  Does need some handholding though if that needs to happen.

Having different snapshots of your data from different time periods (eg, daily, weekly, monthly) is important (and thats why I do it!).   
The number of times clients have come back with &quot;oh I deleted something 2 weeks ago&quot;, and I&#039;ve been able to restore it has meant that they&#039;re happy.

That said, it does mean you need to use a whack of drives for backup purposes.

In our case we only have 6 servers, and web + mysql takes roughly 600GB (still fairly easy to stick onto a single drive though).  They are scattered around different / regions also, so it can take a few days to complete an initial backup if I deploy a new backup server.  
To avoid issues, I do a simple file test in my rsync cron script which gets deleted at the finish of the script.  If the script gets run a second time it checks for the file, and aborts if found.  This prevents multiple instances being run each day, which can cause issues for the initial long running backup.</description>
		<content:encoded><![CDATA[<p>I do our backups in a similar manner (and I *do* get to test using backups when clients go whoops!)</p>
<p>How do I do it?</p>
<p>Step 1:</p>
<p>First is to backup the database(s) to the filesystem in a common folder.</p>
<p>MySQL daily backups to a folder under /home/dbbackup for all DB&#8217;s using an easy to install backup shell script (automysqlbackup)</p>
<p><a href="http://sourceforge.net/projects/automysqlbackup/" rel="nofollow">http://sourceforge.net/projects/automysqlbackup/</a></p>
<p>apt-get install automysqlbackup on debian based systems.</p>
<p>automysqlbackup gets run daily, and it automatically handles daily, weekly, monthly folders + rotation (no need to reinvent the wheel).</p>
<p>Step 2:</p>
<p>rsync my entire home folder system (which is where I store all my user specific data) to another server.</p>
<p>This also runs daily (usually in our early 2am &#8211; 4am period).</p>
<p>Step 3: Optional</p>
<p>I also do a weekly backup of the complete backup folder to a different server so I have some &#8220;history&#8221;.</p>
<p>To make it even safer, I also do a monthly offsite of the backups.</p>
<p>Our specific setup is that<br />
I usually have a central backup server that I send data to (stuck on a spare drive on a less used web server).</p>
<p>Secondary and Tertiary backup backups are run from that.</p>
<p>Caveats &#8211; if your database is in iffy condition  &#8211; eg latin1 encoding, with UTF-8 data, or similar, or crashed tables, the backup will not be 100%.  automysqlbackup can be configured to email you on issues.  encoding related ones will not be seen until a restore is attempted, but other issues will be.<br />
I have successfully restored even with &#8220;corrupt&#8221; encoding data, whether it was latin1 -&gt; utf8 or utf8 double encoding.  Does need some handholding though if that needs to happen.</p>
<p>Having different snapshots of your data from different time periods (eg, daily, weekly, monthly) is important (and thats why I do it!).<br />
The number of times clients have come back with &#8220;oh I deleted something 2 weeks ago&#8221;, and I&#8217;ve been able to restore it has meant that they&#8217;re happy.</p>
<p>That said, it does mean you need to use a whack of drives for backup purposes.</p>
<p>In our case we only have 6 servers, and web + mysql takes roughly 600GB (still fairly easy to stick onto a single drive though).  They are scattered around different / regions also, so it can take a few days to complete an initial backup if I deploy a new backup server.<br />
To avoid issues, I do a simple file test in my rsync cron script which gets deleted at the finish of the script.  If the script gets run a second time it checks for the file, and aborts if found.  This prevents multiple instances being run each day, which can cause issues for the initial long running backup.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Bruce</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-18900</link>
		<dc:creator>Bruce</dc:creator>
		<pubDate>Mon, 01 Mar 2010 17:10:16 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-18900</guid>
		<description>Excellent article. I have been using MySQL for quite sometime and have learned the importance of having regular good backups the hard way!</description>
		<content:encoded><![CDATA[<p>Excellent article. I have been using MySQL for quite sometime and have learned the importance of having regular good backups the hard way!</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: chris lea</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-18399</link>
		<dc:creator>chris lea</dc:creator>
		<pubDate>Tue, 09 Feb 2010 08:29:23 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-18399</guid>
		<description>Yup, I actually have stat in the script I use. I put this in here so people could see the awk snippet that people use all the time since it&#039;s really useful. As for using passwordless keys, that assumes you can key the remote machine, which isn&#039;t always the case. With the IBackup service I was detailing here, that&#039;s not an option for example. Though it does work perfectly well of course.

Thanks for pointing this out!</description>
		<content:encoded><![CDATA[<p>Yup, I actually have stat in the script I use. I put this in here so people could see the awk snippet that people use all the time since it&#8217;s really useful. As for using passwordless keys, that assumes you can key the remote machine, which isn&#8217;t always the case. With the IBackup service I was detailing here, that&#8217;s not an option for example. Though it does work perfectly well of course.</p>
<p>Thanks for pointing this out!</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: chris lea</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-18397</link>
		<dc:creator>chris lea</dc:creator>
		<pubDate>Tue, 09 Feb 2010 08:21:27 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-18397</guid>
		<description>Hi Greg:

So the short answer is that just using a VPS somewhere else works just fine. iBackup has a few extra advantages.

1) All they do is backups. They probably don&#039;t have the sorts of issues that normal ISPs have to deal with (DDOS attacks, bad neighbor effects, etc). So their uptime is just great.

2) They are really super good about backing up whatever you have with them with their own backup systems. So your data is even *more* backed up. And more is better when it comes to backups IMO.

But honestly, sending backups off to another server in another data center is just fine too if it&#039;s cost sensitive.</description>
		<content:encoded><![CDATA[<p>Hi Greg:</p>
<p>So the short answer is that just using a VPS somewhere else works just fine. iBackup has a few extra advantages.</p>
<p>1) All they do is backups. They probably don&#8217;t have the sorts of issues that normal ISPs have to deal with (DDOS attacks, bad neighbor effects, etc). So their uptime is just great.</p>
<p>2) They are really super good about backing up whatever you have with them with their own backup systems. So your data is even *more* backed up. And more is better when it comes to backups IMO.</p>
<p>But honestly, sending backups off to another server in another data center is just fine too if it&#8217;s cost sensitive.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: chris lea</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-18396</link>
		<dc:creator>chris lea</dc:creator>
		<pubDate>Tue, 09 Feb 2010 08:12:36 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-18396</guid>
		<description>Yup, snapshotting is an excellent option. I didn&#039;t go into it here for two reasons:

1) A lot of people these days are using VPS servers where you don&#039;t have the ability to do snapshots. You can on some, you can&#039;t on others.

2) Even if you&#039;re just running bare metal boxes, you may not have a filesystem that supports snapshotting. Most people just run with defaults and that usually means ext3 these days (no snapshots).

But yes, it&#039;s a great and easy option if it&#039;s available. I was just shooting for something that would always work. Thanks for pointing it out!</description>
		<content:encoded><![CDATA[<p>Yup, snapshotting is an excellent option. I didn&#8217;t go into it here for two reasons:</p>
<p>1) A lot of people these days are using VPS servers where you don&#8217;t have the ability to do snapshots. You can on some, you can&#8217;t on others.</p>
<p>2) Even if you&#8217;re just running bare metal boxes, you may not have a filesystem that supports snapshotting. Most people just run with defaults and that usually means ext3 these days (no snapshots).</p>
<p>But yes, it&#8217;s a great and easy option if it&#8217;s available. I was just shooting for something that would always work. Thanks for pointing it out!</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: snetts</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-18395</link>
		<dc:creator>snetts</dc:creator>
		<pubDate>Tue, 09 Feb 2010 08:08:59 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-18395</guid>
		<description>Nice article on data backup. Making backups is a very important aspect in every IT pros career. Good work.</description>
		<content:encoded><![CDATA[<p>Nice article on data backup. Making backups is a very important aspect in every IT pros career. Good work.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Dharmesh Shah</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-18368</link>
		<dc:creator>Dharmesh Shah</dc:creator>
		<pubDate>Sun, 07 Feb 2010 03:57:43 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-18368</guid>
		<description>We&#039;ve used this XFS snapshot model with great success (we have really big database for which MySQLDump is just not practical).

Works well within Amazon EC2 as well.</description>
		<content:encoded><![CDATA[<p>We&#8217;ve used this XFS snapshot model with great success (we have really big database for which MySQLDump is just not practical).</p>
<p>Works well within Amazon EC2 as well.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Kristian</title>
		<link>http://thinkvitamin.com/dev/bulletproof-backups-for-mysql/#comment-18367</link>
		<dc:creator>Kristian</dc:creator>
		<pubDate>Sun, 07 Feb 2010 03:27:55 +0000</pubDate>
		<guid isPermaLink="false">http://carsonified.com/?p=4365#comment-18367</guid>
		<description>Thinking too slowly :D

I think, now I know why you wrote:
STUNNEL_PORT=`/bin/grep ‘accept’ /etc/stunnel/ibackup.conf &#124; awk ‘{print $3}’`;

With $() command substitution you can use variables, so it make it more usable than backticks, for example:

#!/bin/sh
LSBIN=&#039;/bin/ls&#039;;
LSOUT=$($LSBIN -l);
echo $LSOUT;

So you can have variables for all the binaries.</description>
		<content:encoded><![CDATA[<p>Thinking too slowly :D</p>
<p>I think, now I know why you wrote:<br />
STUNNEL_PORT=`/bin/grep ‘accept’ /etc/stunnel/ibackup.conf | awk ‘{print $3}’`;</p>
<p>With $() command substitution you can use variables, so it make it more usable than backticks, for example:</p>
<p>#!/bin/sh<br />
LSBIN=&#8217;/bin/ls&#8217;;<br />
LSOUT=$($LSBIN -l);<br />
echo $LSOUT;</p>
<p>So you can have variables for all the binaries.</p>
]]></content:encoded>
	</item>
</channel>
</rss>
