Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » Warm Spare / rsync
  •  
tpyro

Messages: 14
Karma: 0
Send a private message to this user
We would like to set up a warm spare server, that can be switched on with a (nearly) current version of the mail stores in the event that the main server crashes. I understand that the functionality doesn't really exist in the current version, and that we risk corrupting data if we sync the store files directly without stopping the server first.

It doesn't seem like we have any better option though. Currently we are doing differential backups every two hours, but these take usually about 45 minutes, bogging the CPU down for the duration. I think what we will do is use rsync and risk corrupting a few mail indexes along the way -- knowing that they can be rebuilt. In case there is more severe corruption we'll still have daily backups.

Am I missing anything that can specifically go wrong with this plan? It seems like the lesser of two evils when compared with losing a day's worth of email.
  •  
Nixs

Messages: 159
Karma: 0
Send a private message to this user
robocopy XP010;


robocopy "\\server\d$\Kerio Store\Store\mail" "c:\keriobu"
/ipg:1 /z /mir /mon:2

robocopy "\\server\d$\Kerio Store\Store\lists" "c:\keriobu"
/ipg:1 /z /mir /mon:2

robocopy "\\server\d$\Kerio Store\Store\contacts" "c:\keriobu"
/ipg:1 /z /mir /mon:2


Haven't had to restore from it yet though.... The /mir will delete things I have deleted. The /mon monitors for changes so this is always running. The copy is to a volume with a ton of free space and Microsoft VSS. So in case the server files are deleted, and the deletion is replicated, I can use Prior Versions to go back to prior to the deletion.

You could also use /MAXAGE:n so /MAXAGE:1 won't copy anything more than 1 day old. That way in a failure you'd restore from your backup and then apply this on top of it. Depends on what you have in mind.



[Updated on: Tue, 13 May 2008 00:26]

  •  
My IT Indy

Messages: 1262
Karma: 40
Send a private message to this user
I've heard that you can use rsync against the store folder to a coldspare server IF the receiving mailserver is turned off. Then you can turn up the backup mailserver when disaster hits.

I haven't tried this and it's on my todo list sometime soon.

[Updated on: Tue, 13 May 2008 00:29]


-
My IT Indy
Kerio Certified Reseller and Hosted Provider
http://www.myitindy.com
  •  
tpyro

Messages: 14
Karma: 0
Send a private message to this user
Thanks for the suggestions. I should have mentioned that we are running KMS on OS X, not Windows. Anyway I think rsync / rsnapshot will be enough. We will definitely not leave the backup KMS running during a transfer.

Our bigger worry is whether the backup server will be functional. I know we might have a few partially transferred email files, and some indexes which can be rebuilt. Are there any other bases we'll need to cover? Are there any other critical files that could be inconsistent/corrupt and not easily dealt with?
  •  
Phideaux00

Messages: 5
Karma: 0
Send a private message to this user
I use rsync to do a remote backup through the Internet to an offsite server. On the offsite server, I store the data on a USB HDD, that I can pack up and take back with me. Both servers are Linux, and I only use Webmail, and smartphones, and iPhones via ActiveSync.

Like others, I'm pretty sure you could use this procedure with another install of Kerio on the backup server that you could just fire up, but have not tested it.

I've attached my backup script and header file (used to send the email notification). Granted, I could really spruce it up with variables, etc, but it was a rough draft just to see if it would work as a backup. I run this script on the kerio mailserver via a cron job every night. I had to establish an SSL relationship between the servers so that rsync would not ask for the remote server's password each time it ran.

I still do the normal kerio backups every night. This is just an extra step to get all data offsite. Like most (if not all) of you, I can't afford to lose data.

In my next post, I'll show you my output.

  •  
Phideaux00

Messages: 5
Karma: 0
Send a private message to this user
Below is my log file that the script sends me when it is done. I back up the config files, ssl certificates, and the whole data store daily.

Note that it does not take very long (IMHO). My data store is 117GB (208,000 messages) and growing. I have only 28 users. Yesterday I backed up about 800MB of data, but with compression turned on with rsync, only ~500MB actually went across the wire and it took about 1 1/2 hours. My Internet connection between the two servers is only 768K up/down. I intend to upgrade to a full T-1 soon, and in theory, this should cut the backup time in half.

Anyway, that's my story. If it helps jump start your implementation, great!

From: Postmaster<_a.t_>mailserver.com
To: Admin<_a.t_>wherever.com
Sent: Mon, 08 Sep 2008 21:38:01 -0500
Subject: Email Backup Offsite Log



Mon Sep 8 20:00:02 CDT 2008
#This is the backup of the config files
Number of files: 4
Number of files transferred: 1
Total file size: 255.73K bytes
Total transferred file size: 97.88K bytes
Literal data: 97.88K bytes
Matched data: 0 bytes
File list size: 82
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 98.03K
Total bytes received: 42

sent 98.03K bytes received 42 bytes 9.34K bytes/sec
total size is 255.73K speedup is 2.61

#This is the backup of the certificates
Number of files: 2
Number of files transferred: 0
Total file size: 1.35K bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 44
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 64
Total bytes received: 20

sent 64 bytes received 20 bytes 33.60 bytes/sec
total size is 1.35K speedup is 16.05

#This is the backup of the whole mailstore
Number of files: 208525
Number of files transferred: 2527
Total file size: 117.84G bytes
Total transferred file size: 881.99M bytes
Literal data: 856.83M bytes
Matched data: 0 bytes
File list size: 3.62M
File list generation time: 58.095 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 499.38M
Total bytes received: 55.74K

sent 499.38M bytes received 55.74K bytes 85.19K bytes/sec
total size is 117.84G speedup is 235.94

Mon Sep 8 21:37:56 CDT 2008
  •  
campodoro74

Messages: 119
Karma: 0
Send a private message to this user
Excellent, thank you! I added the -t option in rsync though to preserve the timestamp.
If you have a Synology box to where you want to backup your Kerio server; it has an Rsync server onboard; quite usefull.
  •  
ocmacdude

Messages: 17
Karma: 1
Send a private message to this user
Big question??? Are you stopping the mail server prior to executing? I'm just wondering if there is really any harm in backing up a running kerio database. What I'm really concerned about is whether there is any risk to the running DB in doing so.

Thanks....
  •  
campodoro74

Messages: 119
Karma: 0
Send a private message to this user
No, the mailserver is running. It's not a Exchange DB; no one big file needed to be backed up but 'just' seperate files. Just make sure you don't backup the queue folder (if you have Kerio running on your destination, if not it doesn't matter).
  •  
ocmacdude

Messages: 17
Karma: 1
Send a private message to this user
That's what I thought. I'm actually running Retrospect (not rsync) to the the backup. As far as I can tell there is no risk to the running database from conducting a backup while the mailserver is running. If anyone either has information to the contrary, or can further substantiate my assumption, it would be great to hear from you.
  •  
monsterfreecity

Messages: 60
Karma: 2
Send a private message to this user
On our Mac OS X server, we do a nightly SuperDuper "smart copy" (incremental clone) to an external firewire drive, then let Retrospect come through and back up the firewire drive -- we had a couple GSoDs awhile back when Retro was pointed at the live store.

Pointing Retrospect at the external has resulted in zero crashes in over two years. Perhaps the latest Retrospect client software (6.2.229) is more stable, but I'm not going to beta test for them anymore!
  •  
ocmacdude

Messages: 17
Karma: 1
Send a private message to this user

> On our Mac OS X server, we do a nightly SuperDuper "smart copy" >(incremental clone) to an external firewire drive, then let
> Retrospect come through and back up the firewire drive -- we had
> a couple GSoDs awhile back when Retro was pointed at the live > > store.

OK, I'll bite.... GSoD?
Meanwhile, thanks for the info. Now I'm a little less confident in the approach.
  •  
monsterfreecity

Messages: 60
Karma: 2
Send a private message to this user
Sorry... GSoD = Gray Screen of Death, or kernel panic.
  •  
pstacks

Messages: 6
Karma: 0
Send a private message to this user
Phideaux00,

I have a similar setup but my rsync runs every 2 hrs over lan. I could see splitting the script up for config and certs daily and the store on a more frequent schedule. I like the reporting on your rig better. Mine just tailed the log so I could see that it completed.
Previous Topic: Windows Phone 8.1 and public/shared contacts
Next Topic: Backup Recovery problem
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Mon Oct 23 13:28:20 CEST 2017

Total time taken to generate the page: 0.00577 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.