DMail Email Servers - Performance and Requirements

Currently our largest customers are running systems of 500,000 users and are growing rapidly, e.g. some are increasing at 60,000 users/month.

DMail has been designed to be scaleable, in 'server farm' arrangements.

We are committed to making sure that it grows with our customers needs. So far it is performing exceptionally well in terms of handling load for systems of this size.

Note: This page is provided in order to give you some pre-purchase information. It is intended to give you a feel for system requirements.

A few notes:

  • Please appreciate that many items will never be filled in and contact names may never be provided due to commercial sensitivity. NetWin is very thankful to all contributors to this page.
  • Commas are used to indicate thousands on some numbers, e.g.,
    12,346
    is 12 thousand 3 hundred and 46, not a fraction over 12 :-)
  • Real world systems and test systems are clearly marked as such.
  • By providing this information, NetWin in no way guarantees any performance quoted nor implied.
  • We are continually in the process of gathering the information for this page - if your system will be of interest to others, for being big, small, fast, unusual, etc., please let us know :-)

On this page ...


Performance

System A - real world system

General:

Running Version: 2.3j
Platform: Linux 2.0.36
Total user database size: 60-70,000 -includes forward/redirects, auto responders, POP clients and WebMail clients.
No. of servers: 1 running DMail, another 4 for WebMail serving
Authentication System: External - all users are db-based.
Virtual Domains: not available (but known to have many)
DList lists: not available (but known to have many)

Machine Details:

AMD K6-2 300, with 128M ram.

Total DMail disk useage on machine: 6.98 Gbytes

  • spool files: 6.6Gbytes
  • DMail (logs, index files, aliases, etc, etc): 380Mbytes

DSMTP: (averaged out to per day)

  • Messages received: 7,404
  • Messages delivered: 5,184
  • Messages sent on: 1,604
  • Total in: 7590 kbytes
  • Total out: 7583 kbytes

  • idle 74.5%
  • drop 8.6% (writing to drop file)
  • que 4.3% (processing of queued messages)
  • send 0.9% (processing outgoing messages)
  • lookup 7.0% (looking up usernames, i.e. checking if user local)
  • robot 0.5% (waiting on autoresponders etc.)

DPOP: (averaged out to per day)

  • peak sessions: 79
  • Rejected connections: 2,364
  • Connections processed: 40,083
  • Messages served: 29,432
  • Kilobytes served: 480,251
  • License accounts used: 45512
  • External authent channels: 5
  • idle 93.2%
  • pass 0.1% (password lookups)
  • retr 0.8% (actioning POP RETR command - retireving messages)
  • drop check 26.8% (checking drop files for messages)
  • burst 12.9% (bursting drop file to bin files)

Site:

www.mailbr.com.br

Any nice comments :-)

"I really appreciate the help you guys have given, and the willingness to work together with my suggestions."

Calculated Statistics (approximate):

Average disk usage per user: 110Kbytes (6.6Gb/60,000 users)

Netwin's Summary:

We are very pleased with these statistics, particularly the idle percentages for both servers, which show plenty of room for expansion.

System B - real world system

General:

Running Version: 2.8k
Platform: Linux 2.0.36
Total user database size: > 230,000
No. of servers: 1
Authentication System: External - all users are in MySQL.
Virtual Domains:
DList lists:
Server also runs: mysqld

Machine Details:

AMD K6-2 300, with 128M ram.

loadavg: usually stays around only 1

Total DMail disk useage on machine:

  • spool files:
  • DMail (logs, index files, aliases, etc, etc):

DPOP:

  • peak sessions: 102
  • Peak external authent channels used: 8
  • Peak DSlave processes used: 4

(all the following are averaged out to per day)

  • Rejected connections: 12,469
  • Connections processed: 11,6664
  • Messages served: 42,936
  • Kilobytes served: 3,094,606
  • License accounts used: 235,616

Site:

Any nice comments :-)

"I want to continue to praise the efficiency of your software. One of our systems handles > 230,000 users easily without skipping a beat"

Calculated Statistics (approximate):

Netwin's Summary:

System C - DSMTP Maximum Message Throughput Benchmark

The following are some results from testing to ascertain the maximum message throughput of DSMTP. The tests were designed to test the file (disk) access and queue file handling.

Test Setup:
On machine x , 10 tellsmtps send RSET-separated messages
On machine y , DSMTP version 2.8l processes them.
Machine y, is a P300, 128Mb Ram running Red hat Linux 6,

Pertinant dmail.conf settings:
tcp_max 300 (default is 200)
max_send 50 (default is 10)
max_queue 5000 (default is 1000)
hash_qfiles true (default is false)

All messages are addressed to a dummy address which machine y has been configured to gateway back to machine x which discards them without even writing a qfile or delivering them (using the bit-bucket feature).

Results:

1. when queue processing is suspended DSMTP will accept incoming messages at about,
80 per second

2. when it is offline (i.e. no incoming messages) DSMTP can send messages at about,
70 per second

3. During the stress-test, from the first received message to finishing all processing of the last one, the net rate is about,
50 per second.


Requirements

System Requirements are hard to give, they are very dependent on what else is running on your system and also your network bandwidth. So here are some 'rules of thumb' that we work with. NB: These are very rough. Figures are for either an NT or UNIX based box unless stated otherwise.

NB: We get a lot of customers saying,
"I need to set up a system for 1 Million users",
our response to this is that unless you currently have an active user database of 1 Million users then set up a 100,000 user system, because with our product you can add in more servers at a later date, i.e. your server can grow as your user database grows. We generally recommend starting with just one box for the email server. If redundancy is a worry then run a second box with a second SMTP server only, and if you are really worried then run two full email servers. See the Large Systems FAQ for more information.

  • Processor

    <1,000 users - P133 or higher
    <10,000 users - try a P166 if that's what you have, else buy whatever is currently available.
    >10,000 users - higher end Pentium, e.g. PII450.
    >200,00 users - start to think about multiple servers (but wait until you get there)

    NB: Multiple processors - you may not get the mail processing speed benefit you anticipate by running multiple processors (with any mail server - not just ours), because the message processing is not generally the 'bottle neck' in a mail server, it is commonly the disk access speed. However, they can be made to make more of a difference if you run an external authentication module, as these (you set how many with authent_number) are spawned by both DSMTP and DPOP. DPOP also spawns, DSlave processes (slave_number) for 'bursting' large mail drop files (mailboxes).

  • Ram

    1,000 users - 64MB RAM on NT, 32Mb on UNIX based platforms
    10,000 users - 128MB RAM
    50,000 users - 256MB RAM
    >100,00 users - 512MB RAM

  • Disk

    < 10,000 users - use whatever you have got
    >10,000 users - buy the best you can afford :-)

    We tend to recommend something like an ultra-wide SCSI disk for best performance, but on smaller systems the IDE disks and newer ATA-66 type disks offer comparable performance for less money. Please feel free to let us know when we get out of date :-).

    NB: In general, the seek speed is more important than the read speed.

    NB: If you are going to use a RAID system and you have a big system (e.g. over 10,000 users), make sure that you use hardware RAID rather than software RAID.

  • Disk Space per user

    You can limit this to the amount of disk space you can afford with the settings, max_msgsize and the user_quota system.

    Typically on a system where the sysadmin promotes or enforces that users remove their mail from the POP server, this works out to be very small, about 20kbytes per user. So if you work on about 40Kbytes per user that should be safe.

    Where you wish to allow users to leave mail on the POP server, they will of course need more space. A common limit for Hotmail type setups is 1MB per user, so 5 MB per user is probably considered quite generous. For inhouse systems you may want to limit it to something higher e.g. 10MB per user.

  • Non-user storage space(e.g. log files, workarea, executables)

    About 25Mbytes. Things that will increase this,
    - the number of messages in DSMTP's queue and their sizes (work directory)
    - number of mailing lists and in particular the size of file and message archives for your lists (dlist\listname directories)
    - size and number of your log files, (log directory)


NWAuth with 100,000 users

Netwin's own external authentication module, NWAuth comes free with DMail, but how does it perform on big systems???

From our testing on a small Windows NT test machine (P133, 32Mbytes ram) with a user database of 100,000 users NWAuth performed exceptionally well.

nwauth.exe (the version distributed with dm25d):
Required 15Mbytes of memory.
User lookups were still instant.
Password checks were still instant.
Adding users or changing a password were still instant.