| Search website: | |
|
Advanced Server Software
|
This page is out of date, please use our new website https://surgemail.com
Reporting
Three types of usage reports are available in surgemail, each provides slightly different information:
- Monthly usage reports
- Current status reports
- High use log file analysis
- Monitoring your
server
Monthly usage reports (fast to generate)
The Monthly usage reports are an efficient way of providing usage statistics on a per account, per domain or per server basis. This will typically be used for accounting purposes or for routine monitoring of the behaviour of users of the system.
A variety of statistics are recorded on a per account basis as each message is processed. These statistics can be broadly classified as relating to mail sent and received, and to mail retrieved via POP and IMAP. For local accounts both these sets of information are recorded. If surgemail is running in a surgewall configuration or has gateway rules defined, statistics are also gathered on the "gateway" accounts. These are obviously not real accounts on the system and this usage information is based on mail delivered to unique gateway email addresses in this case.
Sample report
Account usage (all domains) - matched 5 records today
account msg Rcv mb Rcv msg Sent mb sent -- p/i sess mb_read -- ~~~~~~~
wrapped
joebloggs@mydomain2.com 3 0.0 7 0.0 -- 1 0.0 -- below
marijn@mydomain.com 8 0.0 4 1.6 -- 10 0.0 -- ~~~~~~~
user1@mydomain.com 4 1.6 0 0.0 -- 1 0.0 --
user1@mydomain2.com 4 0.4 0 0.0 -- 0 0.0 --
user2@mydomain2.com 14 1.4 0 0.0 -- 0 0.0 --
Total 33 3.4 11 1.6 -- 12 0.0 --
~~~~~~~
last connected lastip cpu_sec elap_sec mb quota -- virus blk -- msgsleft mb_left
12-Oct-2004 12:25:30 127.0.0.1 0 54 0.0 -- 0 -- 3 0.0
12-Oct-2004 12:20:09 127.0.0.1 0 14 0.0 -- 0 -- 0 0.0
12-Oct-2004 12:24:34 127.0.0.1 0 22 1.6 -- 0 -- 4 1.6
never 0 0 0.4 -- 0 -- 0 0.0
never 0 0 1.4 -- 0 -- 0 0.0
never 0 90 3.4 -- 0 -- 7 1.6
~~~~~~~
The above report contains the following information:
- Number of messages received + size in Mb
- Number of messages sent + size in Mb
- POP / IMAP sessions + size of data read in Mb
- Time and IP address of last POP/IMAP connection
- Actual CPU time used (only accurate on windows) and elapsed CPU time
- Current quota used (in Mb)
- Viruses blocked on this account
- Messages and size of messages left online during last pop session
Domain usage reports provide a summary per domain with a total of all domains at the bottom. Account usage reports list each account individually with a total of all entries at the bottom. In addition the reports can be sorted on any column and limited to the top N accounts. In advanced mode wildcard exclude and limit filters can be applied similar to the main log file searching.
Available from
These reports are available to domain administrators and to server administrators using the Reports link in the web admin interface. In addition these reports are available on the command line using the following command: tellmail report report_type [output file] [days=n sort_type=user ...]
eg. Top 20 senders of mail over the last 5 days saved to output.txt:
tellmail report account_usage c:\output.txt days=5 top=20 sort_type=bytesenteg. Top 10 domains mail over the last 20 days counted by messages sent output to the console:
tellmail report domain_usage days=20 top=10 sort_type=msgsent
Valid values to pass
each of the options of this tellmail command can be found by
checking the na_reports.htm template file. You can pass the
tellmail report command the following optional parameters:
domain, days, sort_type, top, exclude, limit.
In addtion you can pass a date range over which to generate the
report. To do so use "range=DDMMYYYY-DDMMYYYY" instead of
"days=n".
Miscellaneous
The data for these reports is stored in daily usage snapshots (recYYMM/statYYMMDD.dat) with a monthly summary (recYYMM/summary.dat). When generating these reports it is most efficient to generate a report for "previous calendar month" as the monthly summary just gets loaded and appropriately filtered and displayed. In any of the n days to-date reports, each of the daily summary files gets totaled before the report gets generated. These statistics snapshot files are stored in binary format but can be dumped as text format using the command: tellmail pstat_dump filename
There are several other aspects one should be aware of as to the detailed operation of the mail sent and received logging:
- Under default installed operation "mail sent" is attributed to the account logged in using SMTP authentication. If the message is sent without SMTP authentication then the mail from address is used to determine the account sending mail. This can obviously be faked. The setting g_acctlog_authonly allows you to only record messages sent from a particular account if smtp authentication is used.
- For surgewalled accounts / gatewayed accounts the delivery of a message to a unique gatewayed email address will start the recording of mail sent and received on a particular account.
- When a message is redirected either from a particular account (eg user.cgi level forwarding or using aliases or redirection rules) this is attributed to a particular accounts as having "sent this message". This allows for the easy identification of accounts that have high volume redirections to non local addresses. This logging can be disabled using g_acctlog_noaliases.
- If you have a large number of inactive accounts in your userdatabase the recording of deliveries to these accounts can make the record files grow excessively large. If g_acctlog_sum_inactive is set, any mail delivered to an account that has never logged in yet is attributed to the not_logged_in_yet@yourdomain.com.
In addition to normal accounts usage information is also logged for the following special case accounts. Aliases as noted above, mailing lists and robots including the isspam address. eg:
Account usage (all domains) - matched 5 records today account MSG Rcv Mb Rcv MSG Sent Mb Sent -- ... alias@mydomain.com 0 0.0 11 0.0 -- ...
isspam@mydomain.com 0 0.0 12 0.0 -- ...
testlist_marijn-bounce@mydomain.com 0 0.0 4 0.0 -- ...
testlist_marijn@mydomain.com 1 0.0 0 0.0 -- ...
|c:\surgemail\some_robot.exe 4 0.0 0 0.0 -- ...
|isspam 70 0.5 0 0.0 -- ...
Total 75 0.5 27 0.1 -- ...
Current status reports (may be slow to generate)
The current status reports are generated based on actual users currently in the authentication database. If you have a slow authentication database and a large number of users the generation of this report can be rather slow.
Currently only a disk quota status report is available.
As for the monthly usage reports this report may be sorted on any column and limited to the top n entries or limited via the filters in advanced mode. Also this report is available to domain administrators.
Also as for the monthly reports this report can be generated on the command line using : tellmail report quota c:\output.txt top=20 sort=quota_pct
Account name % used bytes used byte limit user2@mydomain.com 34% 6.9mb 20.0mb user1@mydomain.com 7% 1.5mb 20.0mb marijn@mydomain.com 0% 8.0kb 20.0mb user1@mydomain2.com 0% 392kb unlimited joebloggs@mydomain2.com 0% 2.6kb unlimited user2@mydomain2.com 0% 1.3mb unlimited Total NA 10.2mb unlimited
High use log file analysis (may be very slow to generate)
This report screen lets you analyse a section of the delivery logs on the fly to trouble shoot a particular "high use" problem.
- From IP address
- From Email address
- To Email address
The report can be sorted by the number of messages:
- Sent
- Stored
- Rejected
- Failed
- Spam
- Received
You can also specify the time period, and number of lines of the report to display. For Example:
Size Stored Rejected Spam Sent Received Failed InQue To 23k 0 1 0 0 2 0 1 chrisp@netwin.co.nz 4k 3 0 0 0 3 0 0 test3@lap 3k 0 1 0 0 0 0 0 spam2@netwin.co.nz
The complete delivery
logs are processed each time for the generation of this report.
If the log files are large then this is likely to be a rather
disk io intensive process. For a typical system of 50K active
users the delivery logs could easily be 250mb per day.
Monitoring your server.
The server will monitor
itself and restart on failure. Most critical reports will be
sent to the g_manager email address, this includes:
Disk quota warnings
Server crash/restart reports.
Server load reports (if cpu or disk response
are too slow).
Additional reporting
settings/limits you can add...
g_hack_report "user@xyz.com" -- Reports of weak passwords sent to this address.
g_orbs_report "your.ip.address" -- List of ip addresses to check against known rbl listings, so you will be alerted if your server is blacklisted.
g_lowdisk_warning "1500mb" -- Alert manager on critical disk shortage.
g_pop_warning "700" -- Alert manager if this many pop/imap sessions used.
g_queue_warning "5000" -- Alert manager if outgoing mail queue exceeds this...
g_smtp_warning "500" -- Alert manager if this many smtp incoming sessions
g_user_send_warning "1000" -- Alert manager if a single user sends more than this many message in one day.
In addition you may
wish to occasionally examine these log file:
mail.err
startstop.log
login_failed.log