Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » High CPU for Kerio Outlook Connector
  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
Dear Kerio,

Background information:

I have setup a Remote Desktop environment with around 15 users (on one Windows 2012 machine). All the users use Outlook with the Kerio Outlook Connector to communicate with the mailserver.

There is one email account that has a folder (with many many many sub-folders). This folder acts as a shared folder. All users are archiving the new incoming email in this folder. All users have delegate access to this email account (i done this by altering the settings via the web interface). And i added the account via the Kerio Outlook Connect as additional mailbox. This account also holds the central calendar.

The email account which holds the public folder has a size of 36.1 GB and holds 134078 items (to give you an impression of the huge size).

Problems:

The Windows 2012 Server has constantly a load of +/- 98% CPU(CPU --> Xeon E5-2620 v3 <_at_>2,4GHz). The KoffBackend.exe is consuming CPU all the time (for each user, one process around 7% all the time). And OUTLOOK.EXE is consuming CPU too (for each user, one process).

When a user is searching in his inbox (in outlook) it is very slow, he must wait approximately ~2 minutes before there is a result (this email account has a size of 7.8 GB and holds 72614 items, around 40.000 emails are in the root of the inbox). The smaller email accounts do not have problems.

When i configure the same account via ActiveSync (not via Kerio Connect) there are no problems, the search goes very fast (within 3 sec there is a result). But i can't use Active Sync to connect the public folder or a email account. And when i add a new ActiveSync account for the email account which holds the public folder it does not sync the public folders.

Plus the fact that i do not want a large OST file (what comes from the ActiveSync offline data) for each user on my Server.

Questions:

- How comes that it consumes so much CPU? Because it is syncing the public folder constantly?
- Is it true that there is no support to add a account in Outlook as Exchange?
- Is it possible to create one shared ".FDB" file for the shared folder for all users? (All the users are working on one machine).

Any help is appreciated!!
  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
!!Update!!

The ActiveSync is now syncing public folders from the account (it took a while before it started).

!!Edit!!

Also when users are creating new sub folders (in the public folder) they all get a annoying message that the public folders are changed, is there a way to turn this automatic message off?

Thanks
  •  
Pavel Dostal (Kerio)

Messages: 180
Karma: 5
Send a private message to this user
- Theoretically it is possible that constant changes in that shared folder force Kerio Outlook Connector to synchronize all the time. Hard to say if it is your case - you have quite extreme configuration. You can check if there are some errors logged al the time in Kerio Outlook Connector log.
- No, you can't connect Outlook to Kerio Connect using Exchange account
- Sorry, it is not possible . Access to single fdb file is possible from single process only

Pavel Dostal
Kerio Technologies
  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
Hi,

Thanks for the reply, i checked the log files, and i saw this coming up very very much (for multiple users):

[23/02/2016 17:00:12.385](20596){err}{scp-worker} In SCProvider\SyncRequestBatchDownloadMessages.cpp:72 (SyncRequestBatchDownloadMessages::processSyncBatchDownloadM essages)
[#248490] (4804) Exception of class HResultException: SCProvider\SyncRequestBatchDownloadMessages.cpp(64), SyncRequestBatchDownloadMessages::processSyncBatchDownloadMe ssages:
0x80004004 E_ABORT: Batch message download was not successful

Any idea?

  •  
Pavel Dostal (Kerio)

Messages: 180
Karma: 5
Send a private message to this user
Yes, if such message is logged many time, it is exactly the issue that can synchronization to demand more resources (including CPU) than it is necessary. Can you see such logs several times every minute and all the time?

Pavel Dostal
Kerio Technologies
  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
Well, for some users the log file exceeded the maximum size of 97MB, so there are incredible much log entry's.

It happens in milliseconds:

[23/02/2016 17:01:28.804](7056){err}{scp-worker} In SCProvider\SyncRequestBatchDownloadMessages.cpp:72 (SyncRequestBatchDownloadMessages::processSyncBatchDownloadMessages)
[#231167] (F823)                                 Exception of class HResultException: SCProvider\SyncRequestBatchDownloadMessages.cpp(64), SyncRequestBatchDownloadMessages::processSyncBatchDownloadMessages:
                                                 0x80004004 E_ABORT: Batch message download was not successful

[23/02/2016 17:01:28.820](7056){err}{scp-worker} In SCProvider\SyncRequestBatchDownloadMessages.cpp:72 (SyncRequestBatchDownloadMessages::processSyncBatchDownloadMessages)
[#231168] (F823)                                 Exception of class HResultException: SCProvider\SyncRequestBatchDownloadMessages.cpp(64), SyncRequestBatchDownloadMessages::processSyncBatchDownloadMessages:
                                                 0x80004004 E_ABORT: Batch message download was not successful


But on the other side i'm checking a user which have around 7% cpu all the time, but the logfile for today is quite short (see screenshot):

[03/03/2016 08:37:59.663]   ****** START ******  Common log has just started (KMS.KOFF64 8.5.3.5082)

[03/03/2016 08:37:59.663](12228){err}{updater} In UpdaterCore\UpdaterCore.cpp:329 (UpdateCore::internalDownload)
[#1] (common)                                  Can't check presence of Kerio Updater Service
                                               HRESULT: 0x00000002

[03/03/2016 10:40:24.504]   ****** START ******  "Outlook.67497A53-0ABF-47E6-8AA7-B87D8732A92C" profile log has just started (KMS.KOFF64 8.5.3.5082)(profile created at (D.M.Y H:M:S) 5.1.2016 15:19:45)

[03/03/2016 10:40:24.504](16676){err}{mapi-interface} In StoreProvider\MAPIFolderImpl.cpp:1061 (MAPIFolderImpl::GetMessageStatus)
[#1] (A92C)                                           Exception of class HResultException: StoreProvider\MAPIFolderImpl.cpp(1059), MAPIFolderImpl::GetMessageStatus:
                                                      0x8004010f MAPI_E_NOT_FOUND

[03/03/2016 11:09:54.526](16676){err}{mapi-interface} In StoreProvider\MAPIFolderImpl.cpp:1061 (MAPIFolderImpl::GetMessageStatus)
[#2] (A92C)                                           Exception of class HResultException: StoreProvider\MAPIFolderImpl.cpp(1059), MAPIFolderImpl::GetMessageStatus:
                                                     0x8004010f MAPI_E_NOT_FOUND


What do you think?

  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
I thought maybe the process is waiting for something, so i checked the waiting chain via the task manager, it says:

One or more threads of KoffBackend.exe are waiting to finish network I/O.

Is this normal?

  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
I analyzed the thread the process was waiting for (see screenshot of my previous post).

And i analyzed the memory stack of this thread and compared it with "normal" threads. The Result:

Problem Thread:

ntoskrnl.exe!KeSynchronizeExecution+0x2246
ntoskrnl.exe!KeWaitForMultipleObjects+0x135e
ntoskrnl.exe!KeWaitForMultipleObjects+0xdd9
ntoskrnl.exe!KeWaitForSingleObject+0x373
ntoskrnl.exe!KeStallWhileFrozen+0x1977
ntoskrnl.exe!KeIsAttachedProcess+0x95d
ntoskrnl.exe!KeWaitForMultipleObjects+0x152f
ntoskrnl.exe!KeWaitForMultipleObjects+0xdd9
ntoskrnl.exe!KeWaitForSingleObject+0x373
ntoskrnl.exe!NtWaitForSingleObject+0xb2
ntoskrnl.exe!setjmpex+0x34a3
ntdll.dll!ZwWaitForSingleObject+0xa
mswsock.dll!Tcpip6_WSHGetSockaddrType+0xc9f
mswsock.dll!Tcpip4_WSHOpenSocket2+0x2b28
WS2_32.dll!recv+0x197
KTLIBEAY100x64_1.0.1m.DLL!BIO_sock_should_retry+0xb1
KTLIBEAY100x64_1.0.1m.DLL!BIO_read+0xbf
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x3854
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x3ead
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x4b47
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x1da3
SCProvider.dll!createSynchronizator+0x1d691d
SCProvider.dll!createSynchronizator+0x1d4658
SCProvider.dll!createSynchronizator+0x1ace6f
SCProvider.dll!createSynchronizator+0x199baa
SCProvider.dll!createSynchronizator+0x199b11
SCProvider.dll!createSynchronizator+0x1b606b
SCProvider.dll!createSynchronizator+0x27bcf4
MSVCP100.dll!std::basic_istream<char,std::char_traits<char> >::peek+0x99
SCProvider.dll!createSynchronizator+0x11b20
SCProvider.dll!createSynchronizator+0x12630
SCProvider.dll!createSynchronizator+0x12e29
SCProvider.dll!createSynchronizator+0x46677
SCProvider.dll!createSynchronizator+0x120fa
SCProvider.dll!createSynchronizator+0x513c8
KoffThreads.dll!Thread::`default constructor closure'+0x2fd
MSVCR100.dll!endthreadex+0x43
MSVCR100.dll!endthreadex+0xdf
KERNEL32.DLL!BaseThreadInitThunk+0x22
ntdll.dll!RtlUserThreadStart+0x34


"Normal" Thread:

ntoskrnl.exe!KeSynchronizeExecution+0x2246
ntoskrnl.exe!KeWaitForMultipleObjects+0x135e
ntoskrnl.exe!KeWaitForMultipleObjects+0xdd9
ntoskrnl.exe!KeWaitForSingleObject+0x373
ntoskrnl.exe!KeStallWhileFrozen+0x1977
ntoskrnl.exe!KeIsAttachedProcess+0x95d
ntoskrnl.exe!KeWaitForMultipleObjects+0x152f
ntoskrnl.exe!KeWaitForMultipleObjects+0xdd9
ntoskrnl.exe!KeWaitForSingleObject+0x373
ntoskrnl.exe!NtWaitForSingleObject+0xb2
ntoskrnl.exe!setjmpex+0x34a3
ntdll.dll!ZwWaitForSingleObject+0xa
KERNELBASE.dll!WaitForSingleObjectEx+0x98
KoffBackend.exe+0x35319
KoffThreads.dll!Thread::`default constructor closure'+0x2fd
MSVCR100.dll!endthreadex+0x43
MSVCR100.dll!endthreadex+0xdf
KERNEL32.DLL!BaseThreadInitThunk+0x22
ntdll.dll!RtlUserThreadStart+0x34



Differences (marked the red as intrestring):

mswsock.dll!Tcpip6_WSHGetSockaddrType+0xc9f
mswsock.dll!Tcpip4_WSHOpenSocket2+0x2b28
WS2_32.dll!recv+0x197
KTLIBEAY100x64_1.0.1m.DLL!BIO_sock_should_retry+0xb1
KTLIBEAY100x64_1.0.1m.DLL!BIO_read+0xbf
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x3854
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x3ead
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x4b47
KTSSLEAY100x64_1.0.1m.DLL!SSLv3_client_method+0x1da3

SCProvider.dll!createSynchronizator+0x1d691d
SCProvider.dll!createSynchronizator+0x1d4658
SCProvider.dll!createSynchronizator+0x1ace6f
SCProvider.dll!createSynchronizator+0x199baa
SCProvider.dll!createSynchronizator+0x199b11
SCProvider.dll!createSynchronizator+0x1b606b
SCProvider.dll!createSynchronizator+0x27bcf4
MSVCP100.dll!std::basic_istream<char,std::char_traits<char > >::peek+0x99
SCProvider.dll!createSynchronizator+0x11b20
SCProvider.dll!createSynchronizator+0x12630
SCProvider.dll!createSynchronizator+0x12e29
SCProvider.dll!createSynchronizator+0x46677
SCProvider.dll!createSynchronizator+0x120fa
SCProvider.dll!createSynchronizator+0x513c8

AND for the other users it is the same (the thread it is waiting for got the same properties as the posted problem thread.

Yours sincerely,

[Updated on: Thu, 03 March 2016 14:06]

  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
The users are using OpenVPN to connect with their server.
  •  
Pavel Dostal (Kerio)

Messages: 180
Karma: 5
Send a private message to this user
It is correct, there should be one such thread in KoffBackend.

BTW there shouldn't be any CPU demanding wait in KOFF - if it takes 7% of CPU, it does something. It probably synchronizes changes from server, so we can say that constant synchronization can cause permanent CPU load even without any error (As I said, it is really possible with your configuration). I don't think we can help you here.

However, if you have some user with log full of that synchronization failures (and it continues to log them even now), it is something else. There is probably some synchronization issue we should be able to identify and (maybe) solve or suppress. Nevertheless it will be necessarry to gather specific information (probably including some more sensitive information and most likely several times), so it is out of scope of this forum. I recomment do ask Kerio Tech Support for help.

Pavel Dostal
Kerio Technologies
  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
My apologies for the extreme late reaction.

But i don't even care about the high CPU that much. But the most annoying part is that the search function in Outlook is extremely slow!

Okay, the user(s) i'm talking about got a extreme amount of e-mails in the inbox (yes... right in his inbox, no subfolders...).

He got around 20.000 --> 40.000 e-mails.

But that should NOT be a reason that a simple search takes 2/3 minutes. If i add the acocunt via Active Sync it is supa dupa Ferrari fast (takes around 2 seconds). I found out that the offline file (that Active Sync creates) is a lot larger than that Kerio creates.

I tried turning off indexing in Windows.
  •  
Pavel Dostal (Kerio)

Messages: 180
Karma: 5
Send a private message to this user
Technically it is possible to wait several minutes on search results. The speed of a search depends on several factors.
- do you search in single folder only or across the whole mailbox? By default, search in Inbox in OL2013 and newer is set to search in all folders. It is much slower and it can easily take several minutes.
- how may words are in the search term. Less is better, more than 5 words slow down the search significantly
- how many search results are there. KOFF is not able to produce more than some 30-50 results per second, so every 1000 hits makes 30 seconds of search. If you expect thousands of results, it is better to search for longer phrase to limit the number or results.

Other factors depends on the machine (terminal server in your case)
- speed and usage of the disk with KOFF database. Can be quite an issue with TS.
- available memory for database and filesystem cache.

Try to search just in Inbox for some nonsense word (to have 0 results). Repeat it again (with same or different word). How long takes the second search?

Pavel Dostal
Kerio Technologies
  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
Answer on your questions:

- I search in the current mailbox.
- It varies very much, i do not know.
- very intrestring, i'm talking about a customer who has 42.000 emails in his inbox.

I'm searching for a nonsense word (kgggrhk) in the current mailbox (which contains 42.000 items in 1 folder). It takes about 4 minutes to complete, it din't find anything...

Now i'm searching for another nonsense word (zhhhxh) in the current mailbox (which contains 42.000 items in 1 folder). It takes about 4 minutes too.

During this process (search) outlook gets to be unresponsive, i can't do anything. Even if i end outlook.exe and restart it, no result. Only if i end the kerio outlook connector process and outlook it works again.

A notable thing is that Outlook is much more slower with the outlook offline connector than it is with ActiveSync. Loading all the e-mails and so on...


I agree that it is absurd to have this large amount of e-mails in the inbox. But you know how customers can be... Fact is; with ActiveSync it is fast, with the old Microsoft Exchange solution it was fast too.

Thank you for the fast replies.
  •  
renss78

Messages: 17
Karma: 0
Send a private message to this user
If i search in the Current Mailbox for a term which occurs in several emails it takes: +5 minutes and gives me 201 items as a result.
During this search i do not see results at all, and everything gets stuck all the time (when i click the outlook screen one time)..

If i search in the Current Folder for a term which occurs in several emails it takes: 2 minute and 30 seconds and gives me 151 items as a result.
During this search i see the results grow..

During the process outlook gets to be unresponsive. When i for example change the location where i search Outlook get unresponsive too (after a huge search task). During the time (when outlook gets stuck) i analyse the waiting chain for outlook.exe and looks like it is waiting for Kerio (see screenshot 44, 47).

[Updated on: Thu, 31 March 2016 14:07]

Pavel Dostal (Kerio)

Messages: 180
Karma: 5
Send a private message to this user
Search times in KOFF will be always worse than in ActiveSync or with Exchange.

Nevertheless such times are really slow (I would expect them on some old machine with 2GB memory). 40000 messages is quite a lot, nevertheless on the client machine with enough memory and a dedicated disc you can expect less than 1 minute when searching in the current folder (although the first search would take much longer). Btw you can expect results in some 10 seconds on a machine with SSD.

I suppose it is either caused by permanent processor load, or by shared usage of disk in the Terminal Server. Most likely by both. You can try the similar test when there is no other KOFF user accessing the TS.

If you get much better results, it is really caused by the load of TS and the only solution are smaller mailboxes (or somehow use dedicated disc for every user).

However, if you get comparable results as before, try to create a new profile (and maybe disable Public folder, so there won't be another 30GB mailbox accessed from the same KOFF). It is possible that there is heavy fragmentation in database that slows down the access and the new profile can solve it.

Pavel Dostal
Kerio Technologies
Previous Topic: Authenticate users to AD from DMZ
Next Topic: CardDAV for contact folders other than "My Contacts"
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Mon Feb 27 00:34:41 CET 2017

Total time taken to generate the page: 0.14027 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.