Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » What RAID level?
  •  
bigmountain

Messages: 116

Karma: 0
Send a private message to this user
All,

I have been trying to decide what the best RAID deployment would be on my next upgrade. Currently I am using a 4 disk RAID5 deployment on an Xserve RAID. I have been consistently seeing performance between 130-140Mb/sec. I know I can do better, so I am trying to weigh the options of:

1. Create new 6 disk RAID5 set on my other controller and migrate the data over to it.
2. Move my data to a backup drive, re-create either RAID50 or 51 utilizing both controllers and then move the data back. This would get me a total of two sets of 6 disk RAID5 either striped across the two arrays or mirrored.
3. Create a RAID10 or a RAID0+1. I was told that an Apple engineer who helped design the Xserve RAID said that a RAID50 would perform better with this hardware, so maybe the RAID10 is not the best option? While other posts in this forum favor a RAID10 deployment, what hardware and how many disks are being used in this scenario for those of you who have expressed a preference for this RAID level?

Basically, if I create any RAID set that mirrors over the two controllers would get me the best redundancy. I can lose a controller or an entire array and still be up and running. If I use any deployment that stripes across the two controllers I will gain better performance, but if either controller goes down, then I am done until I can replace the controller. In a hosting environment, I cannot afford downtime while we replace the controller. For this reason I like the idea of having either a RAID0+1 or a RAID5+1. My hard disk utilization is cut way down and this is a more expensive way to go, but I wanted everyone's feedback from those who use Kerio MailServer.

RAID0+1 would get me pretty good read/write speed across a 6 disk striped array, but there would be no failure protection in any single array. My protection would be the mirrored array. If I use a RAID5+1 scheme, performance would drop a bit, especially on the write speed, but I would have array failure protection on either array. This by far is the most reliable deployment, but may not make the most sense. I all scenarios, I would most likely want to leave my 7th bay on each controller available for a hot spare. I have 14 drives available.

Is there anyone else out there who is using an Xserve RAID for their mailstore who can share their deployment scheme as well as their read/write performance? Perhaps hearing about real time performance from other fellow Xserve RAID users will help me decide how to proceed. Thank you!

Preferred Kerio Partner and Cloud Solutions Provider - Offering both shared and dedicated Kerio Connect hosting solutions.
Visit us at http://bigmountainmail.com
  •  
winkelman

Messages: 2119
Karma: 3
Send a private message to this user
Well, I can only tell you to not look at the continues read/write performance (from the figures it looks like that must be what you're quoting). KMS will almost never read in big chunks of continues data. Almost all of it's disk activity is reading/writing pretty small files all over the drive, and all that with high concurrency. IMHO, high performance under such a load is what makes or breaks KMS performance.

So: low-latency is what you'd want more then 'dumb' high throughput. Which immediate makes your question a problem: RAID will alleviate concurrency issues (for reads), but it will not help with lowering latency. (Only the DRAM cache on your RAID controller will help latency somewhat.)

Thus: use 15.000RPM drives if possible. The higher the RPM, the lower the latency (all else being equal). Secondly: use (Serial attached or normal) SCSI, not SATA. SCSI is better at lots of concurrent activity.

I bet SSD's (Solid State Disks) would just get KMS into a whole new level of high performance, even though SSD throughput is pretty low. SSD's are just perfect for what KMS needs: low-latency and high random read/write performance. I'd really be curious what those would do for KMS (for redundancy still in RAID of course). SSD's are just now getting into sizes I could be satisfied with (128Gb).

I know I did not get into your specific question (RAID-50 or 0+1 or ...), but I hope all this still helps.
  •  
bigmountain

Messages: 116

Karma: 0
Send a private message to this user
Thanks for the feedback Winkelman. Unfortunately, I am tied to my Xserve RAID which only accepts the older ultra-ATA drives. I will only be able to compensate performance with running multiple drives which is the beauty of having up to 14 drives running over two contollers (7 drives each). I believe that the Xserve RAID is capable of up to 320mb/sec, but I believe those benchmarks were utilizing all 14 drives in one big stripe.

My biggest dilemma is that I am a smaller hosting company. I am a hour away from the data center should something happen to my hardware that I need to physically have access to it. That is also depending on whether or not I am in town, etc. I would like to have the best performance available, while utilizing a mirror over the two controllers so that if an array goes down or if a controller goes bad, my client's service does not go down and I can replace the controller when it convenient for me. I would have a hot spare running on each controller so it was a matter of a drive failing, then the hot spare would kick in automatically and the array would essentially be self-healing.

In a private business I think I would choose a senario where I would simply go for the best speed, but in the hosted scenario, I have to consider being able to drop a controller and continue operation. With that said, leaving the 7th slot available for my hot spare, I can either run with a 6 disk stripe on one controller and mirror it to the other controller (RAID0+1) or I could run with a 6 disk RAID5 on one controller and mirror it to the other controller (RAID5+1). I believe RAID0+1 will gain me better write speeds, but I think the read speed might be pretty comparable between a RAID5 and RAID0 on the Xserve RAID, with the read speed being only slightly better on the RAID0. RAID5 would give me better redundancy in the array itself, while RAID0 gives me no redundancy in the individual array.

What I am hoping to find is other Xserve RAID users who may have any of these array deployments who can share their performance so I can weigh in the redundancy vs. performance using real time stats. If I was in the position to purchase a new RAID set, perhaps a Promise RAID, then I would opt for the newer SAS drives, but I am limited to the hardware I have on hand and just trying to make the most of it.

Also, as for my server.... I am running a new Xserve 2.8 quad Xeon with 4Gb RAM. My main boot drive which Kerio is also installed is a mirrored drive and the RAID is attached via fiber cable, currently only using one controller with a 4 disk RAID5. I have 420 current users of which about 225 are active users. My server barely reaches above idle (only about 4-10%) utilization during peak hours and backups it reaches up to about 20%. My clients are claiming very fast performance for a hosted solution. I just don't want to stop there and need to plan for growth and additional redundancy. Knock on wood I have not had any hardware failures in over 3 years and I don't want to jeopardize that record... hence wanting to mirror two arrays over two controllers.

I do have one question that one of my techs came up with... if an array goes down and the Xserve RAID starts to rebuild it, I am told that the RAID will turn itself into a read-only until the array is repaired. How does KMS handle that if it cannot write to the RAID, but can only read from it? I assume that Kerio will shut down. If this is the case, then other than protecting the data itself, unless I mirror the two arrays, then I would experience a long service disruption for my clients. If mirrored, I would assume the damaged array would shut itself down until rebuilt, while both read/write would take place on the other array. Has anyone ever experienced this and can speak on behalf of the Xserve RAID and how it behaved? I don't necessarily trust what is written on paper and what the text say the RAID will do, but rather real world experience. Thanks everyone... I look forward to receiving more input!

Preferred Kerio Partner and Cloud Solutions Provider - Offering both shared and dedicated Kerio Connect hosting solutions.
Visit us at http://bigmountainmail.com
  •  
winkelman

Messages: 2119
Karma: 3
Send a private message to this user
Not being able to rebuild (fully) online seems pretty strange, I doubt that's the case. At least I've never come across RAID controllers that can't rebuilt online. Even my 6 year old normal desktop PC at home with onboard RAID-0 and 1 can rebuilt online, I would be very surprised if this wasn't the case for a business server solution.

Perhaps it is only the case with certain RAID configurations? If so: I would avoid those. It kind of negates a big part of the reason why you'd use RAID anyway. It is pretty likely you will experience failed drives (the more drives you have, the higher the probability, with 14 drives it's bound to happen 1 or more times per year). Who knows what happens to your system if it's read-only for some hours? I doubt anyone can say anything final about that, but it can't be very good.

And I agree with you: better have better redundancy (mirror over separate controllers) then higher speed. (But not if it means read-only rebuilding.) Speed is, as you say, high enough anyway and will most probably stay so even if you expand. I think your customers would not like to loose a small amount of emails (with a failed array you may have to go back to the last backup) and be down for an extended period.

Pitty you have to use PATA, that excludes any 15k RPM drives. Still, you might want to consider 10k Western Digital Raptor drives http://www.wdc.com/en/products/products.asp?driveid=189

I'll now shut up and leave it to people with actual Xserve experience. Cool

Edit: the link to those Raptors is to SATA drives, but Raptors also exist(ed?) with PATA interface...

[Updated on: Thu, 03 July 2008 10:28]

  •  
supersheep

Messages: 7
Karma: 0
Send a private message to this user
An Xserve RAID (any model) will rebuild online - I think your tech needs to take his Apple certs again! Smile

There's no reason to go into read-only mode, from a logical perspective. Also, read-only mode makes absolutely zero sense from a downtime perspective. After all, RAID is designed to keep you running despite failed/failing hardware.
  •  
My IT Indy

Messages: 1262
Karma: 40
Send a private message to this user
Your raid will rebuild just fine in read-write mode. The only things you cannot do with the XServe RAID when it's rebuilding is create Lun's or setup xSAN.

I got the opportunity to setup a Promise 16 drive raid array hooked up to an xserve and it was far above and beyond the speed of the xserve raid. IN just under 4 hours it was nearly half initialized whereas the XServe RAID takes at least 48 hours to do even 6 disks.

-
My IT Indy
Kerio Certified Reseller and Hosted Provider
http://www.myitindy.com
  •  
supersheep

Messages: 7
Karma: 0
Send a private message to this user
That's why the Promise costs a hell of a lot more than the Xserve RAID ever did! Smile

Also, it actually uses SATA drives, and you can use both controllers on a single array for performance - something the Xserve RAID could never do.

Unfortunately, Apple priced themselves out of a fairly lucrative market with small studios needing 2+ terabytes storage - the base Promise is just too expensive. We recently switched to using United Digital - they're about the same price as the Xserve RAID but not as good-looking or as straightforward to admin.

I really wish they had pushed out an upgraded Xserve RAID chassis with SATA drives - it was so damn pretty and very cost-effective. Oh well! More proof Apple doesn't give a poo about the SMB market.
  •  
My IT Indy

Messages: 1262
Karma: 40
Send a private message to this user
I've been installing Highpoint eSata cards + raidon eSata rack mounted cases with 4x1TB drives. 2.7TB of usable RAID 5 storage for under $2k. Plus, if you spend a bit more on the esata card you can get more ports and expand your storage rather easily with more eSata cases. All administration is via a webpage. The downside is that you cannot utilize the array until it's done initializing which can take all night for 4x1TB drives.

The Promise array isn't much more than a fully loaded XServe RAID was, IIRC. It just starts out higher in price.

-
My IT Indy
Kerio Certified Reseller and Hosted Provider
http://www.myitindy.com
  •  
bigmountain

Messages: 116

Karma: 0
Send a private message to this user
Hoosier,

What kind of performance are you getting out of the Raidon enclosures with SATA and what RAID level are you using? Are you mounting these to Xserves? I like the price point. It actually looks like you can get the enclosure, 2TB of storage, raid card with independent eSATA controllers, cables, etc. for about $900. I wonder what type of abuse they can handle and what their reliability rate is? How many people out there are using these boxes that can speak on behalf of their reliability?

Preferred Kerio Partner and Cloud Solutions Provider - Offering both shared and dedicated Kerio Connect hosting solutions.
Visit us at http://bigmountainmail.com
  •  
My IT Indy

Messages: 1262
Karma: 40
Send a private message to this user
I've set up some as long ago as 6 months and they haven't had any problems. I ran xbench on one this morning actually:

Disk Test 45.24
Sequential 181.11
Uncached Write 283.08 173.81 MB/sec [4K blocks]
Uncached Write 310.56 175.71 MB/sec [256K blocks]
Uncached Read 84.14 24.63 MB/sec [4K blocks]
Uncached Read 289.95 145.73 MB/sec [256K blocks]
Random 25.85
Uncached Write 7.48 0.79 MB/sec [4K blocks]
Uncached Write 62.41 19.98 MB/sec [256K blocks]
Uncached Read 509.02 3.61 MB/sec [4K blocks]
Uncached Read 317.22 58.86 MB/sec [256K blocks]

This was on a 5 disk RAID 5 array with port replication so the speeds will be a bit slower than straight SATA connections for individual drives.

-
My IT Indy
Kerio Certified Reseller and Hosted Provider
http://www.myitindy.com
  •  
marcin_rybak

Messages: 12
Karma: 0
Send a private message to this user
Think about raid 10, it's really faster in read, and has smaller surcharge for the controller
  •  
sjourney

Messages: 132
Karma: 0
Send a private message to this user
Hey HoosierMac what model of Radion are you using? Ive been looking for some cheap SATA storage for a B2D solution. This model SR7610-4S-WBS1 says only 2TB supported? Is this just old specs I'm reading? I need to use 1 TB drives.

Thanks
  •  
My IT Indy

Messages: 1262
Karma: 40
Send a private message to this user
1) I noticed the jumpers on the 1TB drives was set to Sata 1.5 and not 3.0 so my benchmarks may not be as fast as they should. I will re-test when I can.

2) The model I was using is no longer being produced, they don't have a port replication model for their rack-mounted models anymore. Here's what they replaced it with:

http://eshop.macsales.com/item/Raidon/ST76104SS2/

You need 4 eSata ports to use this enclosure which should be faster than port replication. The good news is that the enclosure is cheaper than the port-replicated model, the bad news is that you might need to spend more $$ on a card with more esata ports.

3) If you don't need a rack-mounted solution I bought this:

http://eshop.macsales.com/item/Raidon/ST66005SS2/

-
My IT Indy
Kerio Certified Reseller and Hosted Provider
http://www.myitindy.com
  •  
Nixs

Messages: 159
Karma: 0
Send a private message to this user
The Google is your friend (tm) search is;
Rackmount Drive Enclosure

Or;
http://www.google.com/products?q=Rackmount%20Drive%20Enclosu re&safe=vss&um=1&ie=UTF-8&sa=N&tab=wf

Which leads to tons of cabinets. If you don't mind more than 1U, you can find them under $500.

http://www.macmall.com/macmall/shop/detail.asp?Redir=1&d escription=iStarUSA-4U+Rackmount+Chassis+w%2F+8+Hot+Swap+Dri ve-Racks+Enclosures+%26+Arrays&dpno=7500145&store=ma cmall&source=mwbfroogle
Previous Topic: Entourage SSL Certificate
Next Topic: Server update to 6.5.2 build 6426
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Wed Sep 20 23:50:12 CEST 2017

Total time taken to generate the page: 0.00555 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.