Disk Use And FilesAn incoming message for a user is appended to that users unique 'drop file' by DSMTP. DSMTP will create the file if necessary. Then when the user connects to DPOP to read their mail, DPOP 'bursts' open that drop file and adds the messages that it finds to its own indexed 'bin' files. Then DPOP feeds the mail messages to the email client as it requests them. DPOP can be made to 'drop' its bin files back to the drop file format for compatibility with other email servers, but this can affect performance. See the list of DMail files at the bottom of this page.
Drop FilesDrop files are the link between SMTP and POP servers. DMail creates and uses the Unix 'standard' drop files as used by Sendmail etc. The drop file is a text file containing all the user's email messages one after another, including all of the message headers. Each message is separated in the file by a blank line followed by a line, starting 'From '. DMSTP inserts the From line at the start of each message (DPOP will also insert such a line, when it is asked to drop its bin files, back to the drop file format). E.g. Example drop file From pntest@161.29.2.46 Sat Dec 26 09:37:48 1998 Note: DPOP does not leave messages in the drop file format, so once a user has checked their mail box, their drop file will be left empty. See the section below on bin files for more information.
The Drop File FilenameThe name for each user's drop file (or mailbox) is based on the username that they log in with. So if bob is a valid user then his drop file is called simply, 'bob' (note that there is no DOS file extension). There is just one other option for the name of the drop file. Users on virtual domains by default have a prefix added to their drop file filename so that users mail does not get mixed up even if they happen to be stored in the same directory. The prefix is one of the things set by the vdomain line in dmail.conf The prefix is attached to the user's name with a special character which by default is an underscore, '_'. In general NO USER on your system is allowed this character in their username. This character can be set to something else with the setting, vdomain_separator As an example bob from domainx.com might end up with a drop file name of, See the section below on the drop_prefix setting for more on this. Often with virtual domains there are fancy schemes where users have to log in with strange usernames - e.g. in suffix based virtual domains they have to log in with a suffix to show that the user belongs to a particular domain (bob@domainx.com or bob/domainx.com). NOTE in DPOP these never carry through to the drop file name. There are only the two options for the name of the actual drop file. So even though bob might log in with bob@domainx.com because he belongs to the domain, domainx.com, if DPOP recognises that domainx.com is a valid virtual domain suffix then his drop file will be called simply, 'domx_bob' or just 'bob'.
drop_prefixThe setting drop_prefix specifies if the prefix for a virtual domain is put on the drop file name. The prefix allows you to tell which drop file belongs to which domain. It also allows you to place mail from different domains in the same directory. The prefix stops users with the same name from having the same drop filename, e.g dom1_bob and dom2_bob. By default it is true, as it is a good safety measure to have drop files named differently.
Bin FilesThis is the format that DPOP stores user's mail in while it still resides on the POP server. This is generally only until the user reads (retrieves) their email, with their email client. The user's bin files are created in a directory of the same base name and location as the user's drop file, but with the .bin extension added to its name. E.g. a user Lucy whose drop file is, Within that directory DPOP creates message data files, . . . The dropping of bin files back to drop files by DPOP can be initiated manually e.g. or it can be done automatically after a user disconnects from reading their mail, by
setting the dmail.conf setting,
Queue FilesAll messages that arrive at DSMTP's door get put in the work_path directory as a queue file. Then other sections of DSMTP deliver them either locally or non-locally or to robots etc. These files are in pairs - q_xxxx.itm being the message and q_xxxx.idx being an index of the envelope etc. You can open these up with a text editor. The .idx file is particularly useful as it contains information on the number of times a message has been tried etc. Note: there is one pair of queue files per message in the queue but each message may have a number of final destinations. The idx file shows a 'fwd' line for each destination address (recipient line) with information on when it was last tried and if it was successful or not. Each queue file remains until all recipient lines for that message have been dealt with. The way that DSMTP deals with these messages is complex. But basically it loads up 1000 of the recipient lines from the queue files into internal memory and then sets about delivering to each of those destinations. It does things like scanning the internal queue reordering it, moving messages on and off it, looking for destinations that are at the same domain to send down a channel it already has open, etc. The tellsmtp status command lists amongst other things the number of queue files with its 'pending' counter. It also lists those messages which are in its internal queue (see max_queue). The numbers down the left hand column equate to the queue file number. If DSMTP cannot deal with the current load then the number of queue files can back up. This is not a bad thing - it is the way SMTP servers work! When things quieten down a bit those messages will get delivered. What is a bad thing is if it backs up so much that messages never get out of the queue or over a period of days the queue continues to grow, i.e. it does not level out. Typical ballpark queue sizes are, Note things like large mailing lists can greatly affect queue sizes. Being hit by a spammer is a common reason for your queue to shoot through the roof! Using utilities like grep or find in the work_path directory you can often track down messages with the same sender or subject line. See the Spam section for details on how to stop the spam. Talk to DMail Support if you are worried about a queue problem. See also this FAQ, Can I delete queue files from the queue?.
Drop PathsThe path for your mail drop files is set for your main domain with the dmail.conf
setting, For each virtual domain the last value in the vdomain setting sets the drop path, e.g. If you have a number of synonyms for your main domain, i.e. a number of host_domain settings, then you can specify that certain domains put their mail in specific directories with alt_drop_path settings. NOTE that you cannot use alt_drop_path when using external authentication or for virtual domains. The drop path can also be extended by using directory hashing> as detailed in the next section.
HashingIn email systems with a large number of users you can easily end up with an unusably large mail directory. Worse is that performance can be severely affected by 'large' directories, particularly on UNIX based platforms. To get around this DMail offers two forms of directory hashing. You turn these on with the hash_spool setting. Simply, hash_spool 1 adds one extra level of directories named with single letters of the alphabet, spreading mail drop files across 26 directories. Hash_spool option 2 adds two levels and so 26 squared directories are created. All hashing directories are ADDED to the 'base' mail location set by the drop_path or vdomain settings. Note: DSMTP creates these directories as mail arrives to be put in them. DPOP will also create them if it needs to for any reason. Hashing is not done on virtual domain prefixes. See the reference section for more details on hash_spool
Quotas and Message Size LimitsRelevant settings: max_msgsize, drop_max, user_quota max_msgsize limits the size of any mail message that passes through DSMTP, whether for delivery to a local user or being sent out. drop_max limits the size of a user's drop file. If DSMTP is trying to deliver a message to a local user it checks to see that the user's drop file isn't going to be bigger than this limit after it has added the message it wants to add. If it is then it bounces the message and notifies the sender that the recipient was not able to receive the message. Note that when running DPOP a user's drop file is often 0 bytes in size. This is because when the user connects to check their mail, DPOP clears the drop file. DPOP sorts the messages from the drop_file into its own BIN files. So to limit the users' disk usage it is advisable to set up a quota system. Turning on the Quota System To do this, in dmail.conf you need to add the setting, In the username_inf text file you need to add a line like, DPOP maintains a line, In version 2.4i and above you can set a default quota for all users by simply giving a
default quota in kbytes in the user_quota setting, instead of the word 'true', e.g. Remember that in the _inf file the quota setting is in BYTES, but in dmail.conf the user_quota setting is in KBYTES. NB: The Web Gateway products, CWMail and DMailWeb etc., have their own quota systems which limit the amount of space a user uses on the web server for storage of read mail etc. The DMail quotas only affect space each user has in messages waiting to be read and messages 'left on the POP server'.
Path Settings. . . drop_path, alt_drop_path, work_path, dlist_path, log_path, dwatch_path
Server Farming, NFSClick here for general discussion and diagrams
Relevant setting: lock_id nnn The servers in DMail are quite happy for you to put the users' drop and bin files on a shared network disk. So you can also run multiple DSMTP and/or DPOP servers in order to share server load or for redundancy. You simply set the relevant path settings to a shared network drive, and then set a unique lock_id on each machine. Basically the servers work in the following way: Users connecting to read mail can only connect on one POP server at a time, so when a user logs in DPOP locks the user's drop file to read it, closes it and then talks to the user. If it finds the drop file already locked then it won't let the user connect. Mail can come in for the same user on all servers in the farm at the same time, so if DSMTP finds that it cannot access the user's drop file, then it retries to access it 3 times at 1 second intervals. On Windows platforms, you need to use UNC style names (e.g. \\serverA\shared_drive\dmail) to specify paths for the settings above between the server machines for the mail storage area. You may also need to use a UNC name for the authentication module, e.g. our NWAuth module requires this. You can map network drives instead of using UNC names but UNC names have the advantage that if the server reboots then the path specified with the UNC name will be accessible, whereas a mapped network drive is not accessible until someone logs on and the mapping is created. Whether you use UNC names or mapped network drives you MUST enable sharing on the folders that the servers are to share AND ensure that the correct permission for that shared resource is given - see the note below. NB: the DMail servers are spawned by the DWatch service on NT, which will be running as the user 'system', so that is the user that the servers and the authentication module will also be running as. You cannot give access permission to a mapped network drive for the 'system' account. So you must change the NT DWatch service so that it runs as a specific user (in Control Panel, Services, Startup) and then give read/write permission for that user on the shared directories. On UNIX based platforms: DMail and NFS drives DMail has been designed to work with Network File System (NFS) drives. They can be used to allow DSMTP or DPOP servers to share directories such as the mail spool area ( drop_path) and drop_path NFS can cause a lot of problems for mail servers because of its lack of file locking and delayed file updates. When you use the following setting DMail will use additional locking mechanisms to avoid problems with NFS Set in dmail.conf lock_id to a unique string for each server that shares the mail spool on the NFS drive. Please read the following list of notes if you are using or intending to use an NFS drive with DMail. Notes on DMail and NFS:
Log Fileslog_path . . .
Log Files: Logging of information and error messagesFollow this link for help on Deciphering Log Files. Various informational messages are logged during the normal operation of the DSMTP, DPOP and DList servers. These events logged include such things as server startup, creation of work files, rejection of connection request etc. These messages appear in separate log files for each application; dpop.log for DPOP, dsmtp.log for DSMTP and dlist.log for DList When the log file exceeds a preset maximum size it is renamed and a new file is created. For example dpop.log becomes dpop1.log This is referred to as log file rotation. For DPOP four old log files are kept - dpop4.log is the oldest and dpop.log is the current log file. All DPOP versions after 2.9f check for log file deletion every minute. If the log file has been deleted, it creates a new log file, and places a message at the top of the new log file stating that the old file had been deleted. The path for the log file is normally the same as the work directory but can also be set individually in the configuration file using the log_path setting. For Example: log_path \data\logs In addition, error, warning and debug messages can optionally be stored in the log
file. The level of information logged is set by the log_level setting in the configuration
file. The available settings are shown in the table below:
The default setting is info. The log_level setting may also be changed temporarily using the Tellpop or Tellsmtp log_level commands:
The logging level will revert when the system is restarted or when the configuration file is reloaded. When one of the servers dies, DWatch should notice and restart the server (see the dwatch section for more details on when it does not restart the server). When it does this, by default it will RENAME the current log file for that server, e.g. dpop.log, to a 'ded' file, e.g. d_1dpop.ded. These ded files are named as, d_xserver.ded, where x is incremented on each death (so 1 is the first death dwatch caught), and 'server' is the name of the server, i.e. DPOP, DSMTP or DList.
Statistics Files: Gathering connection statisticsDPOP and DSMTP can provide various statistical information. In particular DPOP can optionally provide a per connection log of client usage statistics for use in charging clients etc. These are the log/statistics files that are produced in addition to the standard log files.
File Permissions. . . mostly as MAIL user and group drop files will be set to 600 on Unix platforms if not already DPOP: check_gid = check_disable true = be happy with file permissions and don't change them. DSMTP: chmod_drop false = don't change drop file permissions, but still create on Unix platforms with 600 file permissions. So doesn't affect things like a forwarding alias to /dev/null/ (note forwarding to /dev/null can also be achieved by making an alias to user@nul, upon which DSMTP will make it the message disappear into the void :-) List of DMail filesThis list is not complete but files that we are commonly asked about will be added here as links to the section describing them.
|