Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » Unwanted messages in archive (Wish to excise messages in the archive and block new ones)
  •  
Lyle M

Messages: 410

Karma: 7
Send a private message to this user
Howdy,

An internal system that once emailed registered users with a daily notification through it's built-in SMTP service was unintentionally reconfigured to relay its notices via our Connect server. That system is due to be decommissioned in the coming months and is unlikely to be touched until then.

The result is thousands of emails per week in our archive folders that I would like to eliminate. I also wish to exclude additional ones from appearing (and I'm sure I could make use of such a feature quite often).

Has anyone encountered these needs and found a solution that's Connect-friendly (no index corruption, etc)?

Any ideas are appreciated.

Cheers.
  •  
Lyle M

Messages: 410

Karma: 7
Send a private message to this user
Ok, here we go...

If I execute this within each of the affected #msgs folders (excluding the currently active week):

find -X . | xargs grep -l "my text search string" | awk '{print "rm "$1}' > ReviewAndRun.sh (Credit to Martin Beckett)

I get the makings of a shell script that I can review and execute. When the shell script is run It will delete the 16,000 unwanted .eml files that I have in each of these folders. I'm sure I could refine this further, but it's a start.

The real problem is of course, how to do this without negatively impacting Connect.

I believe the files most affected by mucking with Connect-owned eml files are search.fld and index.fld. Can I do a simple 'rebuild index' from the Connect Admin site for my administrator account? If not, what are my lowest risk options to complete this task (ideally without having to stop the server)?

Thanks!
  •  
Bud Durland

Messages: 387
Karma: 42
Send a private message to this user
Why not just reconfigure the troublesome server? There is no friendly way to remove the files at the OS level. However, there may be an API over in the developer's section that you can use to remove the messages.
  •  
Lyle M

Messages: 410

Karma: 7
Send a private message to this user
Bud Durland wrote on Sun, 04 May 2014 15:59
Why not just reconfigure the troublesome server? There is no friendly way to remove the files at the OS level. However, there may be an API over in the developer's section that you can use to remove the messages.

Hi Bud,

Thanks for the feedback. I'd like very much to stop the other system from sending those emails. In the near term, that's not an option that's open to me. Even if the email barrage were disabled, I'd still very much like to remove the unwanted archive content.

I checked out the Kerio Developer Zone's code examples and 3rd-party tools, but nothing jumped out at me. I doubt I'm going to have the time to write my own code for their API - as much fun as that sounds.

I just stumbled upon a post by Tony Lawrence on his site http://aplawrence.com/Kerio/index_maintenance.html that addresses the index.fld and search.fld concern. Although I'm still running 8.1.3, his article is from 2011, so I believe my version will behave this way.

So, I think I'm just going to give this a shot on the oldest archive folder and see how it goes. Unless someone is aware of a specific risk, I'm not yet seeing the downside - other than I have to do this at all!

I'll post back with results once I get around to it.

Regards.

ps. Tony, if you ever google yourself and find this - THANK YOU!

[Updated on: Wed, 04 June 2014 14:33]

  •  
Lyle M

Messages: 410

Karma: 7
Send a private message to this user
I finally had some time to execute this on my live server.

I modified the command as follows:

find -X /PATH_TO_STORE/archive/2013-W35/#msgs | xargs grep -l "my search string" | awk '{print "rm -v "$1}' > rm-eml.sh

By using the full path in the find command, my resulting shell script also includes the full path to the file(s) being removed. I felt this was safer, and it lets me run it from wherever I want the shell file written. I also added -v to rm so I can watch the progress in the terminal. If anyone else takes this un-supported and not-recommended approach, note that you'll also have to give the rm-eml.sh file execute permissions with chmod before it will run.

My first rm-eml.sh file had 860 removes in it against a folder with over 43k files. This folder pre-dates my flooding problem; more recent folders have over 16k items to remove. I used this older folder as I wanted to start (after all my off-line testing) with a smaller sample and see how the Kerio service reacted.

After running the rm-rml.sh file, I attempted to access the related folder in the old webmail client. Initially, it showed 43335 items. I did a find for the unwanted items, which triggered some extended activity and odd results in the display. The Kerio error log displayed many instances like the following:

[04/Jun/2014 12:12:59] MessageFolder.cpp: Cannot open message file /mailserver/store/archive/2013-W35/#msgs/0000a3db.eml. The file has been probably removed.

A subsequent search in webmail showed a 'reindexing' warning message. I came back to it about 45 minutes later and the item count had gone down to 42475 (860 less than before) and searches worked as expected. So far, I call this a success.

The grep process took quite a while, so I'll restrict my remaining efforts to off hours. Our storage arrangement is currently sub-optimal, so I'm sure YMMV.
Previous Topic: Out Of Office & Redirect
Next Topic: Release history pages missing a few versions
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Sat Aug 19 09:24:36 CEST 2017

Total time taken to generate the page: 0.00464 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.