Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » Kerio's Auto Remove Duplicates Feature
  •  
bearona

Messages: 4
Karma: 0
Send a private message to this user
Hi There,

I'm doing some testing for a web app I'm developing. This app needs the ability to send out approx 1000 emails at a time to an always changing list of clients, which I was planning on doing through the Kerio SMTP Mail Server called from a php script. Currently I have the demo version running which I'm using to send out a single email with 1000 email address bcc'd. For testing purposes I have set up a catch all user in Kerio which will forward all email to my one testuser account. It seems to send out the 1000 emails fine (to testuser0<_a.t_>..., testuser1<_a.t_>... etc), but I only receive one email in the main testuser account. I'm assuming it is seeing that all 1000 are duplicates and is filtering them out as an anti-spam measure. However, I need to make sure this server can send out 1000 emails without any problems, so I want to see all 1000 emails as received. So without creating 1000 separate user accounts, I need to shut off this auto-duplicate filtering to verify the server can send this many emails at once. I've tried looking through the docs, but don't see a way to do this, anybody know how?
  •  
bearona

Messages: 4
Karma: 0
Send a private message to this user
nobody has any ideas on how to do this? I would have thought stress testing a mail server would be a more discussed issue.
  •  
Pavel Dobry (Kerio)

Messages: 5245
Karma: 251
Send a private message to this user
First of all, there is nothing like "duplicate auto remove feature" when sending emails in most mail servers. Definitely not in Kerio MailServer,
Due to various security reasons the number of recipients for the message that can be used in single SMTP session is limited (100 is configurable limit in KMS SMTP server). If you need to send an email to more recipients, send it in several copies with 100 recipients each.
  •  
hbianchi

Messages: 121
Karma: 8
Send a private message to this user
We are using GrouMail, sendign packages of 1500 to 5000 mails (single recipient mails) through the KMS SMTP and no problem at all provided we do this once or twice a week and during nigth. So, I do not think you will have problems.
  •  
bearona

Messages: 4
Karma: 0
Send a private message to this user
hbianchi,

By Single recipient mails, do you mean a separate smtp connection for each sent email? Initially I tried doing a loop in php sending 1000 emails by doing 1 SMTP connection per email and the server crashed. That's why I went down the bcc route. With regards to the max number of recipients setting, I changed that to 1100 so there should be no problem sending 1000. It just seems strange that I only get one copy of the message to my test account instead of all 1000.
  •  
elias

Messages: 114
Karma: 0
Send a private message to this user
bearona wrote on Mon, 20 July 2009 15:38

With regards to the max number of recipients setting, I changed that to 1100 so there should be no problem sending 1000. It just seems strange that I only get one copy of the message to my test account instead of all 1000.

Most mail servers I've used do this; its pretty standard.

However, to your original post about testing KMS performance, I think you're testing the wrong thing. Delivering 1000 emails all across the internet is very different than delivering 1000 emails to local mailboxes. Performance delivering to local mailboxes is probably only limited by the speed of your disks, but the net is much different. Depending on the bandwidth you have, DNS latency, remote server performance, 400-level "mailbox full"-type defers, greylisting, etc, what you'll see locally in no way mimics the load or type of performance you'll see delivering remotely. If you're using a smarthost at your ISP, your performance might be similar to local delivery and only be limited by your bandwidth. But if you're delivering directly, there's no way to correlate performance based on local mailbox delivery rates.

That said, with modest and appropriate hardware, you shouldn't have a problem delivering 1000 emails in a batch with KMS. Just don't watch the "Message Queue" in the Admin Console with a 5 second refresh or you'll really hurt KMS's performance and the Admin tool will hang. You'll also want to experiment with the number of "delivery threads" on the SMTP Server -> Queue Options tab based on your available bandwidth. I think the default is 4 and that's pretty small. If you have the bandwidth, you can really improve delivery performance by increasing that to 16 or 32 or possibly more (I keep mine at 32).

-Elias

  •  
Nixs

Messages: 159
Karma: 0
Send a private message to this user
Isn't the reason you are seeing a single email because you are sending a single e-mail with BCC to 999 people? There are not actually 1000 e-mails at the point Kerio receives them, just one. If you want to see 1000 e-mails you need to not BCC, you need to connect, send to one e-mail address, disconnect an repeat 1000 times.

We have a message volume of about 30,000 Internet bound e-mails per day and many times that number that stay internal. We have no problems with our hardware & Kerio combination keeping up. Quantity wise, 1000 e-mails is not large. If there is a large attachment too, then that is a different story.

Was it the Kerio server that crashed, or was it the script you were running?

  •  
bearona

Messages: 4
Karma: 0
Send a private message to this user
The reason why I thought it was sending to all 1000 emails on the bcc list (even though they were the same address) was because the log listed 1000 as sent. So I just assumed that it was sending them. The more I look into it though, it doesn't appear to be sending more than the first one, and it really wouldn't make sense anyway to send 1000 duplicate emails, so I don't blame it for doing that.

As far as testing the delivery of 1000 external emails, I guess I have no way to test that, so I wanted to test with an internal account first. As far as load goes, 1000 is only the starting point here. Eventually that number may scale to 50,000 or more so I'm just trying to find a way to test how many emails I can actually send reliably and in how short a time frame. Obviously I could create a php script to time delay sending batches of 100 emails every X seconds - but I have no idea what to send those two variables at until I know the upper limit of the server.

With regards to the crash I experienced when I initiated 1000 SMTP connections in a php while loop, it actually crashed the whole server (which is running the latest version of Ubuntu linux), and a reboot was required. Running the same loop with 100 however did work just fine.

I realize now that testing what I want here just may not be possible. I think I'll just create a php script that will loop through and send emails in batches of 100 every 10 seconds or so. Then as time goes on if everything is working with no problems, I can always tighten those values up. I'll try playing with delivery threads setting as well to try to speed up the sending a bit. Thanks for the help.
  •  
elias

Messages: 114
Karma: 0
Send a private message to this user
bearona wrote on Wed, 22 July 2009 09:59

As far as load goes, 1000 is only the starting point here. Eventually that number may scale to 50,000 or more so I'm just trying to find a way to test how many emails I can actually send reliably and in how short a time frame. Obviously I could create a php script to time delay sending batches of 100 emails every X seconds - but I have no idea what to send those two variables at until I know the upper limit of the server.

Kerio is probably not the best choice for what you want to do, especially if you're already running Linux. For large amounts of automated outbound email, you should use a high performance SMTP server like Postfix instead. Postfix on really modest hardware can very easily handle your load and quite a bit more.

This isn't a knock against KMS; they're just vastly different products. KMS is really a groupware server that provides all the Exchange-like features for corporate email. Postfix is just a super efficient and highly tuned SMTP server - that's all it does and its incredibly fast.

I have this exact setup myself. I run KMS on Windows in the office for our corporate email, but for all our production web servers that send out tens of thousands of automated emails per hour to our customers, all of those get passed to a Linux box running Postfix.

-Elias
Previous Topic: "Public Folders" being invited to iCal events
Next Topic: shared calendar
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Tue Nov 21 05:43:44 CET 2017

Total time taken to generate the page: 0.00419 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.