Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » Double attachment size sitting in queue
  •  
JohnnyG

Messages: 12
Karma: 0
Send a private message to this user
I have noticed that when a user sends a message with an attachment, that it actually puts two files in the queue directory. In a test I ran sending one message to a domain that I knew would cause the message to get placed back in the queue, I found that there were two files associated with it. A .eml and a .emf file, and that both of them were 5MB in size (the size of the email and the attachment).

Can someone explain why there are two copies stored there? Also, is there a way to stop that from happening? It seems like a severe waste of space. Thanks.

  •  
Pavel Dobry (Kerio)

Messages: 5245
Karma: 251
Send a private message to this user
Because the message queue processing requires this. I don't think that 5MB is severe waste of space. The files are removed once the message is delivered to all recipients.
  •  
JohnnyG

Messages: 12
Karma: 0
Send a private message to this user
It's not for individual messages, but we are trying to ramp up our system to send out mass mailing between 1000 and 10000 in number, and that will quickly become cumbersome. Is there no way to only have it use a single file per message?
  •  
TorW

Messages: 769
Karma: 9
Send a private message to this user
I'd say "ramping up the system" for a mass mailing also includes ensuring you have adequate disk space to handle the queue in accordance with how KC (or KMS) actually handles it. I'm not sure what you mean by "cumbersome" since KC/KMS continously cleans out the queue.
  •  
JohnnyG

Messages: 12
Karma: 0
Send a private message to this user
The point is that I do not understand why it is required to have two copies of the same message. What is the purpose? Because the message queue processing requires this just doesn't seem like adequate reasoning. And I am ensuring it has proper disk space. That is why I was testing.

Discovering that attachments end up being 30% larger on the KMS server due to Base64 encoding was interesting enough, but then to find that those larger files are stored twice is even more problematic.

As far as cumbersome, I know that many mail servers have a limitation on how many SMTP threads they will allow. When we are sending out mass mailings without attachments this is normally not an too big of an issue since they go through so quickly and don't tend to fill up their limit due to the speed. However, 1000's of emails with 4MB attachments means they are going to start getting backed up in the Kerio mail queue, meaning I have to account for 260% more disk space than would have seemed logical off the bat (30% growth for encoding, then doubled). To me, that is cumbersome.
  •  
elias

Messages: 114
Karma: 0
Send a private message to this user
JohnnyG wrote on Thu, 09 December 2010 07:29
Discovering that attachments end up being 30% larger on the KMS server due to Base64 encoding was interesting enough, but then to find that those larger files are stored twice is even more problematic.

This isn't a Kerio thing. This is how email works.

Quote:
However, 1000's of emails with 4MB attachments means they are going to start getting backed up in the Kerio mail queue <snip>

I can't speak to the duplicate entries in the queue; I'll agree that seems odd. But regardless of that, Kerio isn't a good choice for what you're trying to do. Kerio does perform pretty well if you've taken into account its I/O needs, but its a groupware server, not a mass mailing server. It just isn't optimized for that and doesn't have the tuning parameters you want for fast and efficient mass mailing.

I haven't done any formal testing of outbound performance, but based on what I've seen, if you try hand off 10,000 emails to KMS in a very short period of time, it'll bring Kerio to its knees (even without the attachments). What you need is to build an outbound mail server designed for this purpose. I personally use and would recommend Postfix for fast and efficient mail delivery. You can run Postfix on your favorite Linux distribution and its usually an available package on the installation disc (although Sendmail is usually installed by default so you'll want to swap it out).

In any case, even if the duplicate queue file issue wasn't present, KMS isn't the right choice for what you're trying to do.

-Elias
  •  
TorW

Messages: 769
Karma: 9
Send a private message to this user
Base64 is a "law of nature" in SMTP, so make friends with it as soon as you can Wink As far as number of concurrent delivery threads are concerned, they are there to preserve resources. Sure, you may have a fat, 20 Mbit/s low-latency pipe to the internet, but the receiver may not be so lucky. Blasting out mails at 1000 simultaneous deliveries may sound like a mass mailers wet dream, but you'll never know if half of your recipients have the same MX. A thousand (or even a few dozen) delivery attempts of 4000-5000 bytes each is very hard to distinguish from a DoS-attack. Don't ask me how I know (well, OK, I was once at the business end of a qmail server ...). This is one of the reaons concurrency limiting exists.

A customer of ours is bulk sending text-only to around 1000 recipients once a week, and with even such modest requirements the queue sometimes take a whole day to clear. Neither our customer nor us have any control whatsoever over what the receiving end accepts, and as little as five simultaneous inbound connections from the same IP isn't unheard of. Then you have greylisting, rabid spam filters, DCC checks that times out, vacation auto-replies, full mailboxes, broken name servers, broken MTAs, broken links and whatnot.

Whitehat mass mailing is cumbersome by nature. Local (cheap) disk space is probably going to be your smallest problem. Bounce handling and list pruning is much more of a challenge Razz

Edit: you could also do well by reading this.

[Updated on: Thu, 09 December 2010 20:25]

Previous Topic: Shared Calendar Recurring Event
Next Topic: DROID X (and new Adroid 2.2) Corpotate Sync workaround (still bugs)
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Fri Sep 22 06:39:34 CEST 2017

Total time taken to generate the page: 0.00488 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.