Connect. Communicate. Collaborate. Securely.

Home » Kerio User Forums » Kerio Connect » 7.2 - How can I trace received email? (Tracing received emails through Kerio Connect)
  •  
MarkK

Messages: 454
Karma: 46
Send a private message to this user
It should be possible to use the Debug log. It has been there at least since the early 6.x versions. I'm using 7.2 (.4 as of last night) on Windows.

In the Kerio Admin panel, go to LOGS, click DEBUG, then in the right side panel right click and select MESSAGES. There you will find several options to log events from Services, Message Delivery, Content Filters, Message Store, HTTP Server Modules, Auxiliary Modules, and Local Services. I would think that the Message Delivery and the Content Filters should be able to tell you what is happening to the messages.

  •  
freakinvibe

Messages: 1526
Karma: 60
Send a private message to this user
We normally see missing "Sent:" entries in the mail log in the following cases:


  • Mail has been rejected by Spam filter
  • Mail has been rejected by AV
  • Mail can't finish processing for some reason


Normally, you would see corresponding entries in the Security, Spam, Warning or Error Logs. If not, you should really use the debug log as MarkK correctly states.

Dexion AG - The Blackberry Specialists in Switzerland
https://dexionag.ch
  •  
Waco1

Messages: 89
Karma: 0
Send a private message to this user
MarkK wrote on Thu, 27 October 2011 17:09
It should be possible to use the Debug log. It has been there at least since the early 6.x versions. I'm using 7.2 (.4 as of last night) on Windows.

In the Kerio Admin panel, go to LOGS, click DEBUG, then in the right side panel right click and select MESSAGES. There you will find several options to log events from Services, Message Delivery, Content Filters, Message Store, HTTP Server Modules, Auxiliary Modules, and Local Services. I would think that the Message Delivery and the Content Filters should be able to tell you what is happening to the messages.



We're on Linux. VMS and its offshoots, like Windows, are too slow/old for my blood. I'm a rebel. I even drive a Ducati. Shocked

The only drawback appears to be some shortcomings in the Web UI, but I understand that some of that is addressed in 7.3.

Off-topic: I'd be happy to port the admin tool to the Qt toolkit, gratis, if the Kerio folks will provide the Qt license, and the current source. (Hint. Hint.) Then, the admin tool would be portable to Mac, Windows, Linux, Unix, AIX, Android, iOS, and a several dozen other platforms. Modmins: Email me if you're interested.
  •  
Waco1

Messages: 89
Karma: 0
Send a private message to this user
freakinvibe wrote on Fri, 28 October 2011 01:53
We normally see missing "Sent:" entries in the mail log in the following cases:


  • Mail has been rejected by Spam filter
  • Mail has been rejected by AV
  • Mail can't finish processing for some reason


Normally, you would see corresponding entries in the Security, Spam, Warning or Error Logs. If not, you should really use the debug log as MarkK correctly states.


Thanks, I checked ALL of these possibilities (and several others) before my OP, and the debug options aren't available (AFAIK) on any server platform except Windows.

The SMTP service appears to have never written the received email to disk, or else it wrote it and removed it within the 30 minutes it took the user to notice it wasn't received and I finished searching for it on disk.

I have no doubt that the two emails are malformed in some way. I have no way of determining how they were malformed, because Kerio appears to have dropped the data as a result. That's a shame, because the Kerio folks would have spotted it in a heartbeat and fixed it. Kerio Connect is AWESOME at interpreting garbage that's not even close to RFC-compliant.

No, the emails were not written to disk.

Linux, Unix, and MacOS have a distinct advantage: stored data can't hide from the admin, except via encryption. It isn't possible. That's one of the top reasons why the most effective antivirus/antimalware programs for Windows boot into Linux.

I would really like to see the logging code changed, so that the original RFC-822 Message-Id: header value appears in every log file that references a given message.

The current logging method provides no way to reliably track a message between the different log files; mail.log uses a queue-id, while spam.log does not. Yes, a human can figure out which spam.log entry corresponds to a mail.log entry (most of the time), but automating that process is ... difficult.

The following perl script does NOT work as a result of the logging issues I described above, but it might get you very very close if a user complains that an expected email did not arrive:
#!/usr/bin/perl

use warnings;
use strict;
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$year += 1900;
my @month_abbr = qw( Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec );
my $today = "$mday/$month_abbr[$mon]/$year";
my @line = "";
my $qid = "";
my $to = "";
my $from = "";
my $sent = 0;
my $rcv = 0;
my $matched = 0;

my $logfile = '/opt/kerio/mailserver/store/logs/mail.log';
my $spamlog = '/opt/kerio/mailserver/store/logs/spam.log';

# Find unmatched Recv/Sent pairs in Kerio mail.log
open(LOG, $logfile) || die "Can't open log";
while (<LOG>) {
  chomp;
  if (/$today/) {
    s/,//g;
    s/\<//g;
    s/\>//g;
    @line = split(' ');
    $rcv = 0;
    if ($line[2] eq "Recv:" && $line[6] eq "SMTP,") {
      $qid = $line[4];
      $from = $line[8];
      $to = $line[10];

      ## We're examining an SMTP Recv: entry
      ## Make sure there are as many Sent: as Recv entries with this QID.
      open(TLOG, $logfile) || die "Can't open log";
      $rcv = 0;
      $sent = 0;
      while (<TLOG>) {
        chomp;
        if (/$qid/) {
          @line = split(' ');
          if ($line[2] eq "Sent:") {
            $sent += 1;
          } elsif ($line[2] eq "Recv:") {
            $rcv += 1;
          }
        }
      }
      close(TLOG);
      if ($rcv != 0 && $rcv != $sent) {
        ## Recv: and Sent: entries don't match. Check the spam log.
        $matched = 0;
        open(TLOG, $spamlog) || die $!;
        while (<TLOG>) {
          chomp;
          s/~//g;
          s/\// /g;
          @line = split(' ');

          ##
          ## BROKEN: $to is the Recv's To: field
          ## spam.log has the actual Kerio user id instead.
          ## Like Pat Morita's Crane Technique: no can defense
          if ($line[12] eq $from && $line[14] eq $to) {
            $matched = 1;
            last;
          }
        }
        close(TLOG);
        ## There was a mismatch in the mail.log entries, so if there is no
        ## entry for this in spam.log, the email got lost.
        if ($matched == 0) {
          print "Lost: $qid\n      $from $to\n";
        }
      }
    }
  }
}


Corrected code 09:56 EST

[Updated on: Fri, 28 October 2011 15:56]

  •  
clan

Messages: 235
Karma: 22
Send a private message to this user
I know I am just asking the obvious, but have you checked the mail queue? If there are only Recv entires in the logs I would expect them there. Other than that you should activate the debug log, it is working fine on Linux here.
  •  
Waco1

Messages: 89
Karma: 0
Send a private message to this user
clan wrote on Fri, 28 October 2011 11:15
I know I am just asking the obvious, but have you checked the mail queue? If there are only Recv entires in the logs I would expect them there. Other than that you should activate the debug log, it is working fine on Linux here.


No, that's an excellent suggestion, but I looked in the queue when it wasn't in the SPAM log.

I have a debug log too, but there are no entries at all for the day the email was lost. There are no options for that log in the webui. Apparently, Windows has some options for the debugging log. Linux w/ 7.2 does not, unless it's buried somewhere non-obvious, or set from a command-line option. I've done a few half-hearted searches for kerio connect command-line arguments, and didn't get any useful hits.

There used to be a standalone admin GUI for Windows, but not for 7.2. Not for Linux. At least, not that I'm aware of.
  •  
Pavel Dobry (Kerio)

Messages: 5245
Karma: 251
Send a private message to this user
Waco1 wrote on Fri, 28 October 2011 15:29

Thanks, I checked ALL of these possibilities (and several others) before my OP, and the debug options aren't available (AFAIK) on any server platform except Windows.


Debug options are available on all platforms. Simply right-click in the Debug log screen.
  •  
Waco1

Messages: 89
Karma: 0
Send a private message to this user
Quote:
Debug options are available on all platforms. Simply right-click in the Debug log screen.


Wow, am I embarrassed.

It never occurred to me to right-click inside the log display, probably because "it's just another web page text area". That's probably in the documentation that I skimmed.

That's handy.

This question is answered.

Thank you, Pavel!

Please do consider changing the logging output. Parsing multiple log files, w/o any reliable message reference, is close to impossible. Try it with domain and user aliases, and you'll see why I say that.
  •  
freakinvibe

Messages: 1526
Karma: 60
Send a private message to this user
Quote:
I have a debug log too, but there are no entries at all for the day the email was lost.

I am pretty sure you have the same debug log options on any platform. The debug log would be absolutely useless if nothing was written to it.

The debug log is my most important log for troubleshooting.

Here is what I do:

Open the admin console in Firefox

Click on the debug log

On the debug log, do a right-click

In the context menu, click on "Messages"

You will get a new window where you can choose from the debug log entries you want to see. In your case, I would choose "Queue processing" to see what's going on.

Dexion AG - The Blackberry Specialists in Switzerland
https://dexionag.ch
  •  
MarkK

Messages: 454
Karma: 46
Send a private message to this user
Not to say I told you so, but "go to LOGS, click DEBUG, then in the right side panel right click".

The debug log will tell you pretty much 99% of what Connect is doing.

[Updated on: Fri, 28 October 2011 18:43]

  •  
Waco1

Messages: 89
Karma: 0
Send a private message to this user
MarkK wrote on Fri, 28 October 2011 12:38
"go to LOGS, click DEBUG, then in the right side panel right click".


C'mon, you didn't expect me to read ALL of that rambling diatribe, did you? I mean, you go on and on and on... Embarassed

<lamedefense>Hey, I was born w/o a brain. Ask mom. This is all spinal cord activity. I gotta go now. I'm discussing important things with my pet chicken and it's very clever and interesting. Look! A balloon!</lamedefense>
Previous Topic: apple mail & deleting messages
Next Topic: Off-site Backup of large Kerio install
Goto Forum:
  


Disclaimer:
Kerio discussion forums are intended for open communication between forum members and may contain information and material posted by members which may be useful in learning about Kerio products. The discussion forums are not intended to provide technical support for any specific product. Any information implied or expressed in the discussion forums is that of the posting member. Kerio is in no way responsible for the information posted in the forums, or its accuracy. Kerio employees may participate in the discussions, but their postings do not represent an offical position of the company on any issues raised or discussed. Kerio reserves the right to monitor and maintain the forums to promote free and accurate exchange of information.

Current Time: Thu Sep 21 06:53:54 CEST 2017

Total time taken to generate the page: 0.00497 seconds
.:: Contact :: Home ::.
Powered by: FUDforum 3.0.4.