Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » Kerio Connect performance investigation (Trying to outsmart the problem)
  •  
BudDurland

Messages: 348

Karma: 10
Send a private message to this user
Current Mail server config:

Kerio Connect 7.1.0 patch 1
Windows 2003R2, Patched
HP DL380/G6, 4gb RAM, 2x Xeon CPU
2x 72GB SAS drives, RAID 1 for OS
2x 300GB SAS drives RAID 1 for mail store

About 100 users, 800K files use about 90GB in the mail store.

For the last few days, we've been trying to track down a performance issue on some of our workstations; they would periodically grind to a halt, and users were rebooting often.

Current evidence leads us to our Mail server. Using just windows performance monitor, we can see the disk that has the Kerio mail store go to 100% utilization and stay there for several minutes. On the client side, we see OutLook (with the offline connector, KOFF) go off into never-never land, "hanging" the whole workstation.

Today, a couple volunteers who regularly had problems used nothing but web mail, and had absolutely no problems. I also have a couple of users still using the on-line connector (KOC), and they don't seem to be having any trouble. So tomorrow, I will change one of the "problem children" from KOFF to KOC and see if that helps. It's like a lab experiment; change one thing, document what you see.

From this, I have a couple questions for Kerio (or for anyone with advice to share)..

1 -- Given the size and number of file in my message store, will I see better performance switching the mail server to SuSE linux, using EXT3 or RieserFS?

2 -- Will the OutLook on-line connector continue to be developed for the foreseeable future?

3 -- is there any advantage to running Kerio in 64-bit windows, even though Kerio is a 32-bit program?

Thx in advance

Good is better than evil because it's nicer
--Mammy Yokum
  •  
Pomodoro

Messages: 8
Karma: 0
Send a private message to this user
RAID 1 is terrible to use for mail store. Have you considered to switch to RAID 0+1?

[Updated on: Wed, 22 September 2010 00:54]

  •  
elias

Messages: 114
Karma: 0
Send a private message to this user
BudDurland wrote on Tue, 21 September 2010 14:22
Current evidence leads us to our Mail server. Using just windows performance monitor, we can see the disk that has the Kerio mail store go to 100% utilization and stay there for several minutes.

You need more disks. You should also watch Average Disk Queue Length for a while in perfmon and see where it sits.

I have 50 very heavy email users and it takes 8 15K SAS drives in a RAID 10 array to get the performance I need. My store has over 3 million files totaling 180 GB though.

While you might be able to squeeze out a little more performance by moving to Linux, it won't be much. Similarly, Microsoft made some significant improvements to NTFS in Server 2008 R2 for volumes which have an enormous number of files (like the Kerio store), but that also won't get you very far. Ultimately, I bet you've just maxed out the number of IOPS your drives can physically handle.

-Elias
  •  
mrralan

Messages: 151
Karma: 3
Send a private message to this user
Pomodoro wrote on Wed, 22 September 2010 00:53
RAID 1 is terrible to use for mail store. Have you considered to switch to RAID 0+1?


Would you recommend RAID 0+1 over RAID 1+0? If so, why?
  •  
Pomodoro

Messages: 8
Karma: 0
Send a private message to this user
- RAID 0+1 has the same fault tolerance as RAID level 5. The data will survive the loss of a single disk, but at this point, all you have is a striped RAID0 disk set. It does provide high performance, with lower resilience than RAID10.

- RAID1+0 is a combination of RAID1 mirroring and data striping. This means it has very good performance, and high reliability, so its ideal for mission critical database applications. All that redundancy means that it is expensive.


I guess, you can go either way. They both provided high performance, but you are right, RAID 1+0 has higher reliability.


  •  
elias

Messages: 114
Karma: 0
Send a private message to this user
mrralan wrote on Wed, 22 September 2010 08:00
Would you recommend RAID 0+1 over RAID 1+0? If so, why?

RAID 0+1 is an obscure RAID level that is essentially not used anymore and almost no commercially available RAID controller supports it.

The difference is that RAID 0+1 is a mirror of two RAID 0 stripes. If any drive in one of the stripe sets fails, that whole half of the mirror is gone. If you then lose another drive in the other stripe, your array is dead.

RAID 1+0 (or 10), is a stripe of multiple RAID 1 mirrors. Every pair of drives is mirrored, and then each mirrored pair becomes part of a RAID 0 stripe. In this arrangement, you can potentially lose up to half of the drives in your array so long as you don't lose both drives in any mirrored pair.

So the biggest difference is that you can usually sustain multiple drive failures in a RAID 10 array without losing the array. With a 4-drive array, this doesn't really help, but when you have larger arrays with more drives, the odds of losing both drives in the same mirrored pair drops substantially.

-Elias
  •  
BudDurland

Messages: 348

Karma: 10
Send a private message to this user
elias wrote on Tue, 21 September 2010 19:10

You need more disks. You should also watch Average Disk Queue Length for a while in perfmon and see where it sits.


I figured that to also be part of the equation. I need to study the traffic patterns (read vs. write) a bit, as the type of array and the number of spindles influences the performance of the different operations


Good is better than evil because it's nicer
--Mammy Yokum
  •  
Pomodoro

Messages: 8
Karma: 0
Send a private message to this user
You are absolutely correct! Very Happy

elias wrote on Wed, 22 September 2010 09:58
mrralan wrote on Wed, 22 September 2010 08:00
Would you recommend RAID 0+1 over RAID 1+0? If so, why?

RAID 0+1 is an obscure RAID level that is essentially not used anymore and almost no commercially available RAID controller supports it.

The difference is that RAID 0+1 is a mirror of two RAID 0 stripes. If any drive in one of the stripe sets fails, that whole half of the mirror is gone. If you then lose another drive in the other stripe, your array is dead.

RAID 1+0 (or 10), is a stripe of multiple RAID 1 mirrors. Every pair of drives is mirrored, and then each mirrored pair becomes part of a RAID 0 stripe. In this arrangement, you can potentially lose up to half of the drives in your array so long as you don't lose both drives in any mirrored pair.

So the biggest difference is that you can usually sustain multiple drive failures in a RAID 10 array without losing the array. With a 4-drive array, this doesn't really help, but when you have larger arrays with more drives, the odds of losing both drives in the same mirrored pair drops substantially.

-Elias

[Updated on: Wed, 22 September 2010 19:18]

  •  
elias

Messages: 114
Karma: 0
Send a private message to this user
BudDurland wrote on Wed, 22 September 2010 10:06
elias wrote on Tue, 21 September 2010 19:10
You need more disks. You should also watch Average Disk Queue Length for a while in perfmon and see where it sits.


I figured that to also be part of the equation. I need to study the traffic patterns (read vs. write) a bit, as the type of array and the number of spindles influences the performance of the different operations

That's why disk queue length can be helpful. Watching the disk queue during the peak periods of usage can help you gauge how many drives you'll need to sustain those loads.

You didn't say what you were using for your disk controller, but I'm using a P800 with 512 MB of BBWC. I've configured it with a split of 75% read, 25% write, and I've found that with this split, that's just enough cache to handle multiple large write operations to the store. As a result, my % Disk Write time rarely gets over 5% (my reads however, still peak well over 100%). So with just 128 MB of write cache, writes are essentially a non-issue for me now and I can just add disks to achieve the read performance I need.

-Elias
  •  
stewie

Messages: 106
Karma: 0
Send a private message to this user
elias wrote on Tue, 21 September 2010 18:10
I have 50 very heavy email users and it takes 8 15K SAS drives in a RAID 10 array to get the performance I need. My store has over 3 million files totaling 180 GB though.


Wow, that sounds like a lot of iron, elias.

I also have 50 users in a mixed environment (but mostly macs) with a mail store over 400GB. A single 600GB WD Velociraptor SATA drive connected via a 6Gbps controller on a dedicated server works fine for us.

Like BudDurland, we use to run in RAID1 (with a pair of 300GB raptors) & had very good performance using Kerio 6. (We're currently just using a single drive & periodically rsync'g to a secondary server for an acceptable (to us) amount of redundancy.)

I'm only stating this because while IO should definitely be checked, our mail server is running fine with minimal hardware. And the fact that you imply that you were running fine & are only now having these issures, I'm inclined to look elsewhere. I'd do a tcpdump & dtrace (sorry, not sure about the windows equivalent) & see what's going on.

Good luck!
  •  
beatle20359

Messages: 29
Karma: 0
Send a private message to this user
I'm running 350 users on an Xserve with a Promise RAID attached for the mail store. HFS+ is certainly the weaker filesystem here and we don't break a sweat either. Our store is about 650GB with most of these users also having an iPhone or similar. Last year we moved away from Exchange 2000 adv on a 2004 HP DL370 series with a 4 disk internal RAID 5 array which peformed perfectly well even with it's paltry 2gb of ram.

I've PM'd Bud as it sounds like he has the duplicating calendar entry issue which can grind the clients to a halt and slow the server down, unless it's over specced like mine and a few others on here of course Smile , we only noticed when 3 entourage databases had to be rebuilt on the same day having started to slow down to the point of being unusable.

Anyway good luck all
Beatle

[Updated on: Thu, 23 September 2010 00:27]

  •  
GlennK

Messages: 252
Karma: 3
Send a private message to this user
RAID 1 here, working just fine, FWIW. ~150 users plenty of IMAP and iphones. Kerio 7.02.
  •  
sedell

Messages: 1168
Karma: 1
Send a private message to this user
It also depends on how your clients connect. On one end, you have the Outlook Connector without caching which is extremely hard on the server IO, and on the other, POP3 which has very minimal impact on server IO. If a majority of your users are toward the POP3 end of the scale, you might be able to get away with a SATA RAID 1.

Our users are about 50/50 KOC and KOFF. Our original SATA RAID 1 was always behind. I forget the numbers offhand, but the performance counters for things like disk queue were way way way (did I stress way?) above the recommended max. We moved to an 8 disk RAID 10 SAS setup, and the difference was huge. Before, if I logged in to the admin account via IMAP to check an archive, it took 30-40 minutes to download the headers for 1 full month. Now, it takes 2-3 minutes. The improvement on day to day work was similarly huge.

Scott
Previous Topic: Missing Message in Inbox after Synchronization (KOFF)
Next Topic: KC Spamhaus Whitelist Support possible?
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Sun Sep 24 21:27:22 CEST 2017

Total time taken to generate the page: 0.00540 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.