Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » What if the index gets corrupt?
  •  
Paddy

Messages: 25
Karma: 0
Send a private message to this user
Hi,

In one of my previos threads this was mentioned;

"Something else to consider... KMS uses the file date to index mail. On top of all the headaches of importing from PSTs, all of the mail will have a file date of the date of import. This is true if you are importing from a PST, or using the Exchange mirgration tool. If the index ever gets corrupt, all of your mail will be re-indexed using the DATE OF IMPORT, not the actual message date. This has caused me many headaches, to the point where I've had to write my own program to correct the mail. Kerio has no fix, and hasn't even acknowledged the issue as a bug/flaw/problem."

How common is this?
What can be done to recover from this sittuation?

Thanks in advance.

--
Paddy
  •  
sedell

Messages: 1168
Karma: 1
Send a private message to this user
Basically, you're screwed. I'm still arguing with support over this issue. They think their method is "the most reliable". Re-indexing needs to be done "as fast as possible" and they feel parsing the message would take too long. They have told me they have no intention of fixing this problem "We are not planning to redesign this functionality in near future".

So, basically, their design will work if ALL of your mail was received by KMS. If you've imported mail from another source, and you get a corrupt index file, you're screwed. This is why I had to write a program to correct the problem. It's not 100% since the Received headers are all gone, but it's closer than the import date, that's for sure. They have no intention of fixing it, and their "work around" for the problem is to change your e-mail client to use a different header to display the date in the message lists. So now, your mail is permanently damaged because it will export/copy to other mail stores with these incorrect dates.

Their stance on this issue alone is enough to make me want to look at other products. We have over 500,000 messages sitting in KMS mail stores, and there is no way to fix it when their index files get corrupt. As for how often it happens, I can remember it happening 5 or 6 times in the last 6 months. In some cases, it was all imported mail, and the folder could be re-imported. In other cases, it was imported mail that had been intermingled with mail received by KMS, so a re-import was not an option.

Scott
  •  
sedell

Messages: 1168
Karma: 1
Send a private message to this user
Oh yeah, I almost forgot to mention... the other thing that absolutely drives me nuts about this is that they keep saying that there's nothing they can do about the Received headers getting stripped. That's why there is no good way to reindex a folder, because the information is gone. I've been testing other mail servers with Outlook connectors, and the mail RETAINS all of the headers, INCLUDING Received headers.

[Updated on: Wed, 09 November 2005 17:41]


Scott
  •  
Paddy

Messages: 25
Karma: 0
Send a private message to this user
Sedell,

Thanks for posting.

I have to say that this makes me very nervous about switching to kerio.
I'm suprised that no one else has commented on this - not even Kerio. This is a active forum so this must have hit other?
  •  
teco

Messages: 80
Karma: 0
Send a private message to this user
to sedell:

have you found another mailserver.

Deletion of the header and creating a new message date on rebuild of an damaged index can make us some problems with our german tax authority. We have strictly to prevent this issue in accordance with our tax law. There are also some other restrictions we have to take care and this is also against these restrictions.

If switching is the solution, we have to.

(to kerio: sorry folks, but our tax law is very rigid to this matter.)
  •  
Pavel Dobry (Kerio)

Messages: 5245
Karma: 251
Send a private message to this user
teco wrote on Mon, 14 November 2005 12:44

to sedell:

have you found another mailserver.

Deletion of the header and creating a new message date on rebuild of an damaged index can make us some problems with our german tax authority. We have strictly to prevent this issue in accordance with our tax law. There are also some other restrictions we have to take care and this is also against these restrictions.

If switching is the solution, we have to.

(to kerio: sorry folks, but our tax law is very rigid to this matter.)


This is the main purpose of archive folders (and mail log) - to keep an archive of all communication which comes through the mailserver. I doubt that tax law can be applied to user's mailbox where you can modify e-mail messages.
  •  
teco

Messages: 80
Karma: 0
Send a private message to this user
Quote:


This is the main purpose of archive folders (and mail log)


Archive folder:
does they have no index which can get corrupt?

Mail log:
The tax law says simply that the original receiving header must be archived/protected. Even on reindexing a folder or archive. Log files are not enough because they can not be handled via WIN-IDEA to select a email.

The relevant part of the german tax law are the GDPDU.

Our other restriction:
We have some governmental customers. They request that a message mus be archived completely without any modification. If the archive has a index - what happens if it get corrupt?????

Log files are out of interest for these customers.

Quote:

I doubt that tax law can be applied to user's mailbox where you can modify e-mail messages.


German tax law says simply: Nobody is allowed to change or delete a received message with company related content for a minimum time period of 6 years. Any archive is to proof that these messages are not deleted. If some header is changed in the user folders - this could be a illegal changing of a received mail with company related content.

There is a webmail function in german "als neu editieren". Could you make this feature to be turned on and off?

[Updated on: Mon, 14 November 2005 13:02]

  •  
Pavel Dobry (Kerio)

Messages: 5245
Karma: 251
Send a private message to this user
teco wrote on Mon, 14 November 2005 12:56


Archive folder:
does they have no index which can get corrupt?


Yes, they have message index too. I think I have to clarify what the message index contains. There are some informations important
for KMS, especially for fast message retrieval (synchronizing with groupware clients). There are also stored some message flags (read/unread message, archived message etc.). Lost of this information does not damage the e-mails in the folder.
Quote:


Mail log:
The tax law says simply that the original receiving header must be archived/protected. Even on reindexing a folder or archive. Log files are not enough because they can not be handled via WIN-IDEA to select a email.


Original headers are keep intact in archive folder (and in user's mailbox too). The e-mail headers may be lost only if you add an e-mail to the folder with some tool (import tool or drag&drop in the client) and this tool does not save entire message headers. THIS CANNOT HAPPEN ON E-MAIL SENT OR RECEIVED BY KMS.
Quote:


The relevant part of the german tax law are the GDPDU.

Our other restriction:
We have some governmental customers. They request that a message mus be archived completely without any modification. If the archive has a index - what happens if it get corrupt?????


Nothing will happen. Since all e-mails were received by KMS, they contain correct timestamps. The e-mail in the folder is always keep intact, including all message headers. The only thing that can happen is that administrator will see all e-mails in archive folder as unread.

Quote:


Log files are out of interest for these customers.

[Updated on: Mon, 14 November 2005 13:11]

  •  
teco

Messages: 80
Karma: 0
Send a private message to this user
OK, this makes me happy.

(and the return of the fax plugin much, much more, even if "absolutly unsopported"; and a free to configurate antivirus plugin for clamwin or anything else...)

Thank you for clarify some open questions.
  •  
sedell

Messages: 1168
Karma: 1
Send a private message to this user
I've been working with Pavel Stepanek from Czech Republic on this now. He's been very helpful.

Apparently there was a problem prior to 6.0.6, a difference between file creation date and file modification date, that has been corrected. This accounts for our older mailboxes displaying the import date on the file and re-indexing incorrectly.

There also appears to be an issue with Outlook 2003 using older version PSTs (we do this often for compatability accross the company). It appears to function properly on the surface, but is giving me sometimes flaky results with dragging messages from the PST to a KMS message store.

Different versions of Outlook using it's native version PST is importing mail correctly.

The lesson here is don't mix Outlook versions and PST versions. Doesn't help if you've got mail pre-existing with the date of import as the file date though. I guess as a work around to protect against a re-index corrupting dates, mail with a file date matching import date could be brought into a PST then back to the KMS store. This should sync the file date/received date. This would be very time consuming though.

There are still no Received headers. This does appear to be a limitation of MAPI, but it's still disturbing to lose this information. And it still doesn't explain why mail going into a KMS store loses headers, but mail coming out doesn't.

Scott
  •  
peterj

Messages: 852
Karma: 1
Send a private message to this user

So no resolution then!!!
  •  
sedell

Messages: 1168
Karma: 1
Send a private message to this user
The date problem has been corrected for mail you import now. There is no resolution for older mail though. With no Received header, there's no way to correct date on already corrupt indexes.

Scott
Previous Topic: Embedding images to mail
Next Topic: Import from Exchange and changing username
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Sat Nov 18 19:01:53 CET 2017

Total time taken to generate the page: 0.00452 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.