Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » duplicate mailstore
  •  
skeates

Messages: 104
Karma: 0
Send a private message to this user
Does any one know if I can have kerio deliver email to two separate locations?

I want to create a duplicate mailstrore for a client. There mailstore currently sits at about 600gb and having to unzip a full copy from the backup and then stich together missing mail is just a night mare as happend to me recently when the smart status went on the drive.

The other idea was to have an rsync run every 20min or so to keep a separate machine upto date, but the option to deliver the mail to two destinations would be a better idea I think.

Any ideas or would this be a feature request if so not something very difficult I would suspect.
  •  
My IT Indy

Messages: 1262
Karma: 40
Send a private message to this user
You have 600GB of email sitting on 1 hard drive?

Yikes.

What OS are you running your server on?

-
My IT Indy
Kerio Certified Reseller and Hosted Provider
http://www.myitindy.com
  •  
skeates

Messages: 104
Karma: 0
Send a private message to this user
Not any more.

Historically the system had a 1tb hard drive with the mailstore on it that was set up before my time.

It now sits on a MacPro with an raid card and 4 1TB drive setup as RAID 5.

Basicaly I want to create another machine exactly the same setup with a upto date live mail store so if the mailserver goes done all I need to do is reassign ip addess to be up and running again.

The system is running 10.5 server at the moment. I can run an rsync every 20min or so, but the option to have email dropped into two mailstore folders would be better.
  •  
My IT Indy

Messages: 1262
Karma: 40
Send a private message to this user
You can use rsync and keep them in sync pretty much up to the minute. Have the master KMS turned on and then sync the mailstore folder to the KMS backup and then setup rsync to run every minute or so.

-
My IT Indy
Kerio Certified Reseller and Hosted Provider
http://www.myitindy.com
  •  
stewie

Messages: 106
Karma: 0
Send a private message to this user
That would be a great & most welcome feature. I know some 3rd party offers some realtime syncing but I can't recall the product. Try searching the forum & google for "kerio sync."

rsync works very well but it takes time to run. Our store is around 250 GB & rsync takes a minimum of 3-4 minutes to run, even if run back-to-back. The cause is a lot of ".fld" files.

Our users file away most of their email so they can have a lot of folders & Kerio creates a ".fld" file in each one. And for some reason these files are always copied (even though it's highly unlikely they've actually changed).
  •  
mitev13

Messages: 18
Karma: 0
Send a private message to this user
i have a posting about this and i have been trying to google and read here in the forums to confirm that we are safe to rsync the store while kms is running.
we have about 130GB on raid1 - 400GB drives.
i assume it will take a while to restore from those zip files though if DR is needed.
i was looking for faster way (rsyncing nightly or 2-3 times a day) to a firewire clone drive attached to the KSM or another machine.
i always have LDAP backup and a recent clone of the server as well as i do KMS backups nightly to the 3rd drive on the G5 xserve and then back those up again to another machine.
ideally i would run superduper or CCClone nightly to 2 RAID1 FW drives so all i need to do is pop those out of the enclosures and put them in the sleds and load them back up and boot.
i was just wondering if there is anything to exclude (i think the queue) that may cause problems with the running KMS.
any help is appreciated.
thanks
martin
  •  
giobbi

Messages: 90
Karma: 0
Send a private message to this user
HoosierMac wrote on Tue, 04 August 2009 00:36
You can use rsync and keep them in sync pretty much up to the minute. Have the master KMS turned on and then sync the mailstore folder to the KMS backup and then setup rsync to run every minute or so.


Kerio absolutly not recomend running any backupsoftware while kerio is running. This makes any live backups impossible.

I have run rsync, on aprox. 200Gb data, live, until now. I started getting very strange quota problems on all users, full mailboxes even thou the weren't "full", missing mail in Apple Mail, did show up on web interface etc. Two users Apple Mail claims that their inboxes are 6 alt 8 GB big, on server they actually are 700Mb alt 450Mb. This i have not solved yet with help from kerio.

I would love to use rsync so if anyone can solve this... But be careful..
  •  
skeates

Messages: 104
Karma: 0
Send a private message to this user
I have since setup a duplicate server that has an rsync run every half hour. It keeps the two mail stores in sync and seems to work quite well. I don't sync any on the settings or users across at the moment. I just replicate the changes manually and that works.

So if the server is to go down I just need to assign the correct IP address the new server and at worst I am 30min out of sync with the original server.

So far I have not had to use it, but this beast the alternative which I had to deal with before. Unziping a 500gb mail store took about 8 hours including getting the server back up and running.
  •  
GlennK

Messages: 252
Karma: 3
Send a private message to this user
I've been reading this with interest and trying to decide what to do myself. My mailstore is on the boot drive (mirrored), 10.5 server and backed up by Retrospect 6 nightly. It is about 45 GB so far. The drive is 1TB and backs up to another 1TB. I also have another 1TB drive attached that I am considering using the Retrospect "duplicate" script to make a copy of the boot, store, etc (entire drive).

I have a few concerns. One of them is the alleged kerio stance not to use backup software (meanwhile the retro nightly into compressed backup file seems to be no problem), and the other is of the server booting off the wrong drive on startup, which shouldn't happen but it seems a bit too possible. Also, all that duplicating is going to thrash the drives quite a bit. Anyway, I'm wondering what the best tool to use is for the duplicate. Rsync or Retro or something else. Since I am already running Retro that seems to be the best tool although when I've run dupes in the past it seems to take forever to read the drives and replace the corresponding files.
  •  
campodoro74

Messages: 119
Karma: 0
Send a private message to this user
I use Time Machine on my Mac Mini Server running 10.6.2 to backup the system disk. The /usr/local/kerio folder is synced to an external disk on an Ubuntu server using a bash script, called by cron every 5 minutes. Store size is 91GB. At night, I let Kerio also perform it's backup to another external disk, just in case.

So if things go wrong, I can hook up the external disk with the latest rsync copy of my store folder to any server running Kerio (Linux, OSX, Windows), it will only take the time to mount the disk and start the Kerio service.

I've posted the bash script on my blog: www.campodoro.org

[Updated on: Fri, 13 November 2009 14:02]

  •  
stewie

Messages: 106
Karma: 0
Send a private message to this user
rsync works fine for us, & it's probably safe for most of the small/midsize organizations that use kerio. Here's why.

rsync compares files then copies them as directed. The copy operation itself is generally harmless. The concern are open files, like if kerio is writing a new message. How this situation is handled is determined by the OS--some allow access, others don't.

If there's no access then there is no problem, the file is skipped.

If access is granted then there's the possibility for corruption or problems with execution. Even so, when rysnc runs again it will try to copy/overwrite the partial file with the completed--& by now closed--file.

Regardless, this is the situation where potential problems arise. But it's much safer & easier to rsync a number of small individual files that aren't being modified than is it to modify a large, changing file like a database.

In our case (50 users, 275GB mail store), running lsof on kerio a few times shows that it's only accessing the log files. So in our case the logs may present an issue (but we're not too concerned about them) but will generally rsync just fine.

BUT, if you've got a lot of traffic then kerio will undoubtedly be touching a lot files & the potential danger increases.

So see for yourself. Run run lsof periodically & see what is being accessed. Then decide if rsync will work for you. If you're really concerned (& can afford to), stop kerio, rsync, then restart kerio. At the very least, most organizations should allow a 1am pausing of mail service for 5 or 10 minutes to perform a safe backup.

(Feature request: It would be great if Kerio added rsync support into mail so the two processes could coordinate to insure a safe & complete sync.)
  •  
mitev13

Messages: 18
Karma: 0
Send a private message to this user
Smile - well i am back at square one getting mixed messages.
i will for sure keep using KMS build in backup as i know that is supported.
i will NOT be using retrospect 6 or the new 8 unless they have a tool as they do for PC exchange server to be able to backup running DBs.
i have two concerns:
1) loosing delta data if my server fails at the end of the day before my KMS backup runs at midnight.
2) if disaster strikes - the time to restore the mail server and become operational again with the large data store of 130GB we have.
i can afford to take the server down for couple hours on late night weekends.
so far i am really entertaining the idea of having a firewire drive (maybe even software raid 1 two drives) with a clone of the server and just rsync the kerio store.
that is how i can just restore my latest LDAP backup and upgrade to the latest kerio KMS and OS and put the drives in the sleds and be good to go with the delta loss from the last rsync.
i think it is time to email kerio support too to find out directly what they would say.
thanks
martin
  •  
GlennK

Messages: 252
Karma: 3
Send a private message to this user
I'm not quite as worried about it getting damaged on the resulting backup as I am in the original server files getting damaged. It doesn't seem like that would happen. We've got about 100 users and maybe 10 completely different domains. The server is used pretty heavily. As far as stopping it, I avoid that at all costs. I don't see a way to schedule the server being stopped and started, or I would consider the 1am option except that it seems too dangerous that one day it would not start again. I only stop it if I can watch it stop and start. It seems that rsync would be more effective than Retrospect, although not as easy to work with.

Another thing I would like to do eventually is to have 2 Xserves, each one acting as a backup for the other. For instance, one is the Kerio server, the other is a web/file/ftp server. The web/file/ftp server has Kerio installed and ready to go with the config synced and the Kerio server has the web/file/ftp services ready to go with config synced. Making small changes like redirecting the IPs in our router and turning the services on would be the only thing to do in case of a melt-down.
  •  
stewie

Messages: 106
Karma: 0
Send a private message to this user
Mitev13, we rsync every hour. That's an acceptable compromise for our business, so our delta is 1 hr.

The other great thing about rsync is not having to recover--it's ready to go! (All we have to do is take our main server offline, change the ip & hostname of our backup server to the ip & hostname of our primary server. Done.) BTW, we rsync the mail store every hour & our mail server (/opt/kerio) every month.

GlennK, the original files should not be effected. rsync'g is no different then you trying to manually copy a file. You can test this yourself by saving a large test file & trying to copy while it's saving. See what happens.

As for scheduling, use cron (linux) or launchctl (on os x) or scheduled tasks (on win). You'll have to check on the specifics but for linux, /etc/init.d/keriomailserver stop then later /etc/init.d/keriomailserver start.

I suppose I understand your concern about restarting the process. Of course, any service can stop at any time, so restarting kerio really isn't risky. Besides, if kerio is down from, say, 1-5am, it's probably not an issue because users probably aren't accessing mail & any incoming mail will continue to be sent periodically until delivered.

Besides you can write the script to stop kerio, run rysnc, start kerio, then check that it is running. And if you use any service monitoring software, it will alert you of any problems.


Good luck!
mitev13

Messages: 18
Karma: 0
Send a private message to this user
stewie,
this makes a lot of sense and also quite assuring email that rsyns works Smile
i also have no problem stopping the KMS at 3 am to rsync the whole kerio folder in additional the the hour store rsyncs, so this will work for me as well.
for the very little money we pay for 70 licenses and support for kerio this is totally worth it for our business.
i am fully aware that a full clustering exchange or novell solution will be also much much more expensive.
thanks for your input.
martin
Previous Topic: restore backup
Next Topic: Wildcards in mail filters
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Mon Nov 20 10:44:25 CET 2017

Total time taken to generate the page: 0.00550 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.