Managing Your Server 

• Introduction
• Keys to Managing DNews
• What is stored where, spool, history, workarea...
• Control update of news groups (suck feeds)
• Control Internet Newsgroups
• Create local news groups
• News groups (General)
• Controlling Server Access
• News groups (General)
• Controlling Server Access
• Controlling Expiration of News
• Log Reports
• Reports
• Censoring News Articles - SPAM Protection
• Performance
• Scalability
• Putting News groups on the web
• Mail and Newsgroups
• Add a realtime news service
• Adding PGP to control.conf

Introduction

Apart from the site-specific information you provide during setup, DNews is set to work "straight out of the box". You can install it and virtually forget it. Despite this simplicity DNews is highly configurable and offers the news administrator an advanced and powerful range of options and features, allowing the server to be easily configured or tuned to meet specific needs. This manual page provides an overview to managing DNews and its many features and options.

Keys to Managing DNews

This section introduces DNews Manager Graphical User Interface (GUI) , tellnews commands and the six configuration files.

DADMIN Management GUI

Dadmin allows you to perform all the management functions of the older management gui as well as providing graphs to show statistics about your newsfeeds.  See https://netwinsite.com/dnews/guis.htm for more information.

Web Based Management - DMGRCGI

Dmgrcgi is a web based management gui.  You can access this manager through a web browser on port 7119 (e.g. http://your.news.server:7119).   You will need to configure an administrator's username and password before you can use dmgrcgi to manage your news server via the web.  Accessing port 7119 of your news server via your web browser will explain the entry you need to add to users.dat to set this access up.

To contact the DNews manager use a web address like this: http://your.host:7119 (this link may work, try it) A username and password will be stored in the file CONFIG/users.dat on your first connection it will tell you how to specify this username and password, basically edit users.dat by hand and add a line like this:

	dnews:yourpassword:Full:*:Mr Admin:

The manager service can be stopped on NT with the command "net stop dmgrsvc" and can be disabled from the control panel/services window.

Format of the users.dat file:

username:password:Access_Mode:Managed_Groups:Descriptive_Name:List_of_usergroups

• username = The user's name
• password = The user's password (possibly encrypted)
• access_mode = The level of web manager access, All = full control, None = a normal news user
• managed_groups = The list of groups that this web manager can control.
• descriptive_name = A text field that is not used
• list_of_usergroups = The user groups this user belongs to, e.g. 'adults' or 'child' etc...

In Version 4.7 the web manager allows partial controls, e.g. a user can be given access to run tellnews status but not do anything else.  Here are the possible values for access_mode

access_mode

• addusers = Can add users to users.dat
• config = Can edit the config files
• access = Can add/change access.conf entries
• status = Can run the status command
• feed = Can modify news feeds
• start = Can start/stop the server
• killitem = Can delete items.

TELLNEWS Commands

Tellnews commands are just typed in on the command line, or with the Windows NT and Macintosh versions from the DNews Manager interface select "commands" and then select the tellnews command.

Tellnews commands are used to control and query a running DNews news server, they are actually handled by DNews itself, the Tellnews or DNews Manager process simply creates an NNTP connection to the server and sends through the tellnews command and then listens for DNews to send back a response. Tellnews commands are used to:

Stop the server
Initiate background tasks like expires or database rebuilds
Inform the server that it should reload it's configuration files from disk
Create new local news groups etc...

A complete list of tellnews commands with explanations is provided in the tellnews section of this manual.

Configuration Files

There are six configuration (".conf") files that together provide all the information and rules your news server uses. The *conf files are simple text files and can be edited directly, or on the Windows and Macintosh versions common changes can be made via the GUI interface. The following provides a very brief introduction to the configuration files.

Finding the configuration files

DNews.conf is the main configuration file, and you can't move it from its built-in directory, which is:

Windows NT - /winnt35/system32/dnews.conf (system dir)
Unix - /etc./dnews.conf

All other configuration files will be in the config directory defined in dnews.conf

Dnews.conf

This is the main configuration file. When you first install DNews you are prompted for some site specific information which is added to dnews.conf . The file includes the name(s) of your feed site(s), the path to send mail on your system, when to run expires etc. All lines in the dnews.conf file follow the same syntax:

What is stored where, spool history workarea xover?

In dnews.conf these directories are defined:

• spool..spooln    These define the directories where bucket files are kept, each bucket file contains raw items (typically thousands in each one as a bucket file is usually 10mb or 50mb)
• workarea        This is where DNews stores the index for all news groups, and various other indexes and files
• history        Here DNews stores the history.idx and history.dat files.  These files are used to lookup message id's and convert them into group/item number pairs, it is also used to prevent duplicate articles being accepted.
• xover        This is where the xover indexes are stored, this is a list of header information for each specific group which speeds up news reader access.

Newsfeeds.conf

This file controls feeds out of DNews to other sites, it defines the sites to feed news to, what news groups to feed to each site (including the sites you receive news from), the type of feed (e.g. built-in, live, uucp or program) and the NNTP command used to send items (e.g. post or IHAVE).You can also exclude news received from one site from being sent to any specific site. For your main feed DNews will automatically set the exclude setting to prevent the system trying to send every article that comes in back to the feeder site.

The newsfeeds.conf file also controls what groups will be allowed 'into' DNews via the special "ME feed" parameter. This must be the first entry in newsfeeds.conf. and defines which groups will be accepted from any feed(s) into DNews. Also in this file you can set specific accept/reject rules to automatically identify and remove Spam articles. A complete list of settings in newsfeeds.conf with explanations is provided in the *.conf section.

Access.conf - Powerful and tricky!

The access.conf file is very powerful, it can be used to control access to your server in many different ways and as a result it is also tricky to use. The main source of confusion comes because of two features, first the file is passed twice, once when a user connects, and a second time when/if they supply a password. The second tricky bit is that the 'last' matching rule in the file is used. So the order of rules is very significant.

Access.conf allows you to do all of the following:

- Allow ip numbers/names access
- Allow users name and password access
- Allow read and post protection separately
- Allow DNews Manager/Tellnews commands
- Allow incoming IHAVE feed

All access rules in access.conf have the following Syntax:

Syntax:
host:perm:user:pass:groups [:extension settings]

Example _Extract access.conf file:
*:logoff:::* (Read - Immediately close a connection from anyone )

Expire.conf - Intelligent Expire settings

Expire.conf contains the expiration rules which together determine how the server will expire news items. DNews allows intelligent and flexible expiration rules, so more news is deleted as free disk space shrinks. Groups can be limited by ‘number of articles’ rather than age, busy news groups are trimmed and manageable. Light news groups become useful again. The result is management time is reduced and the news service is improved. A detailed description of the settings in expire.conf with explanations is provided in the *.conf section of this manual.

Control.conf - How newsgroups are created

The control.conf file defines how your server will deal with control messages.

Who really runs NEWS? Who creates news groups, and once a new group is approved how is it created on 10,000 news servers all over the world? The control.conf file is the key to this. It specifies who else in the world is allowed to create newsgroups on your news server.

Control messages are just like normal news messages, except they have a special header:

Control: newgroup rec.humor.really.funny

When DNews receives a control message it is checked against the control.conf file. The last matching rule is used to decide what to do. The possible actions include logging the message to a file, for editing and batch processing, emailing it to the news administrator, throwing it away or actioning it. The action will normally vary depending on who the control message is sent by and what news groups it relates to. PGP (Pretty Good Privacy) may be implemented to confirm the integrity and identity of the message and sender respectively. A detailed review of settings in control.conf with explanations is provided in the *.conf section of this manual.

Control Update of News Groups (Dynamic Suck Feed Option Only)

A Dynamic Sucking Feed periodically fetches the news articles in those news groups that are actively read by your users and updates its listing of available news groups. This section explains how to manage the timing and frequency of this update process.

Control Frequency of Updating Groups List

Your Suck feed will obtain a list of groups from your upstream server automatically and display these to your users to select from. As new groups are being added and deleted on an ongoing basis DNews by default updates this list daily. To change the frequency of update, amend the groups_at setting in dnews.conf.

The groups_at setting defines when the server will get the names of any new news groups and add them to its list of available news groups.

Basic Syntax: groups_at minutes hours

Examples:

groups_at 45 1 (This is the default setting and gets the names of any new group once per day at 1:45am)

If you don't have a permanent connection to the Internet you should disable the automatic update by entering an invalid time in the groups_at command, as it can take some time to update the groups over a modem.

Example: groups_at 999 999 (999 999 being an invalid minutes and hours)

You can manually get DNews to update its list using the command 'tellnews getgroups' to ensure DNews has an up to date listing.

Control Internet News Groups

USENET News Groups provide a valuable resource. However there will be many USENET News Groups that you do not need or possibly want on your server. This section explains how to control which news groups appear on your server.

Control which internet news groups are available on your server - There are a very large number of internet news groups (a list of the top level hierarchies is maintained on the web at http:/home.magmacom.com/~leisen/master_list.html). While some people think more is better, the reality of having thousands of foreign or local or inane news groups listed doesn't help your users read news.

To exclude news groups amend the "site me" setting in newsfeeds.conf

Essentially you list the group names you want or use wildcard * to select multiple groups. And list those you don't want by putting ! = don't allow or @ = don't allow this or anything cross posted to it. Using @ instead of ! is useful as it will stop some inappropriate cross posts. Frequently if you don't want particular groups you don't want to see articles posted to them even if they have been crossposted to other groups you do want.

The default setting in newsfeeds.conf is:

site me
*,!*.binaries.* ( read - accept all news groups, but don't accept news groups with that match anything.binaries.anything )

Examples:

* (Allow all news groups)

We recommend that you use something like the last option which picks up most of the standard news trees.

On Windows from DNews Manager select "Feeds Out", select "ME feed" , and amend the "groups" field.

Local News Groups

Local News Groups or Discussion Forums are today widely used by organizations for internal communications and document sharing. The efficiency with which they open communications, support collaboration and overcome geographical constraints of time and distance are making them an increasingly common-place element of organizations' communication systems. Local News Groups are often a more efficient solution than mailing lists, and often used in conjunction with mail and web technology. This section describes how to create and control your local discussion groups with ease.

Creating a Local News Group

To create a new group you use the tellnews newgroup command.

Syntax: tellnews newgroup <groupname> [y |m] [creator] [description]

Example: tellnews newgroup here.chat y myname Local group for chatting

Here is a detailed breakdown of each of the parts of the above command.

here.chat = name of group
y = Normal group (m = moderated)
myname = Just a record of who created the group (one word)
Local group for chatting = description for active.names file.

On Windows from DNews Manager select "COMMANDS" then select the "NEWGROUP" command and complete the appropriate fields.

Preventing posts to local groups being posted to the Internet

To ensure local news postings don't get transferred to the Internet, you would want to exclude all local groups from being posted to your feeder site. To do this you should change the groups line in your newsfeeds.conf file to exclude the local groups. For example to stop the groups here.chat and all the groups here.secret.anything or any articles cross posted to here.secret.anything from being sent to your feed site in newsfeeds.conf we would change:

site $nntp_feeder
groups *

to

site $nntp_feeder
groups *,!here.chat,@here.secret.*

(note: ! means "not this" while @ means "not this or anything cross posted to this")

On Windows from DNews Manager select "Feeds Out" and select $nntp_feeder and amend the groups field as above.

Preventing DNews from trying to suck your local groups from your feed site

This is not critical but it is recommended you prevent your server from trying to suck your local groups from your feed site. It is more efficient and allows you to use the Matchgroups command properly without having to undelete your local groups.

In dnews.conf modify your nntp_feeder line to exclude local groups. In this example we have assumed all local groups match "here.chat" or "here.secret.anything"

nntp_feeder myfeedsite

to

nntp_feeder myfeedsite *,!here.chat,!here.secret.*

(This is read suck all groups from my feed site but not groups with names that match here.chat or here.secret.anything)

Windows from DNews Manager select "Main Options" and select "Sucks"and amend the "These groups" field as above.

Naming Local News Groups

You can name your local groups however you like. It is useful to adopt a single convention so your local group names are unique, easily identifiable as local and identifiable with the section of the organization they will be used by.

Syntax: xxx.yyy.zzz.aaa.bbb. etc.

Examples:

netwin.usa.ho.directors
netwin.usa.ho.sales
netwin.usa.ny.sales
netwin.product.support

Creating a Local Moderated News Group

Where a group is moderated all articles posted to that group are emailed by DNews to the moderator, who then if he deems it appropriate posts them to the news group. You must tell DNews the news group name and email address of the moderator.

Let's say your company name is 'Genesis' and you want a local moderated group for announcing major company decisions - let's call the group "genesis.announce"

First create the moderated group using tellnews (or DNews manager)

tellnews newgroup genesis.announce m theboss Company Announcements

Then add one line to moderators.conf to send any messages posted to this group to the person who is going to 'moderate' this group. The new line must go above the default rule.

Example - Extract moderators.conf

genesis.announce:secretary@genesis.com (read - for articles posted to the group genesis.announce email them to the moderator at secretary@genesis.com)

Lastly you MUST have an SMTP gateway defined in your newsfeeds.conf. You should already have this as it was entered during the setup process, but check in newsfeeds.conf to make sure you do. It should look like this:

Example - Extract newsfeeds.conf:
site name_of_your_mail_server
   smtp

To add an SMTP Gateway On Windows from DNews Manager select "Feeds Out" select "Add", in the "Feed Name" field enter the name of your mail server and then select the "this is really the smtp gateway" indicator.

Now you are all set - the moderator will receive all posts via email, he must then add an Approved: header line to the message before re-posting it to the news group. This can be done with FreeAgent, see the FAQ for details.

News Groups - General

This section describes how to add and delete news groups, and how to delete news articles.

Add a News Group - Newgroup Command

Your News server will only accept articles for news groups it knows about. The Newgroup command is used to manually add a new group to the list of news groups on your server. This can be a local group or a new group from another feed site, or a group from the upstream feed. The optional y or m field is to set the group as normal (y) or moderated (m), the optional creator field is a one word name of the group's creator, the optional description field is a few words describing the group.

Syntax: tellnews newgroup <groupname> [y |m][creator][description]

Example: tellnews newgroup local.bicycle.rides y eddym Info about bike rides in our vicinity

Tellnews commands are just typed in on the command line - with the Windows from the DNews Manager interface, select "commands" and select the "newgroup" command.

Delete a News Group - Rmgroup Command

The Rmgroup command allows you to remove news groups from your server's groups list. Rmgroup Command flags the specified group(s) as deleted. This is a clean way to delete a group and if you make a mistake, you can undelete it. The expire process will delete the items in the now deleted groups in accord with your normal expire rules, so this may take several days or weeks.

Deleted groups will remain deleted unless a control message is received to recreate them or you undelete them using the undelete command.

Syntax: tellnews rmgroup <group name>

Example: tellnews rmrgroup "local.secret*"

Tellnews commands are just typed in on the command line - with the Windows and Macintosh versions from the DNews Manager interface select "commands" and select the "rmgroup" command.

Undelete a News Group - Undelete Command

This command is used to undelete a group previously flagged as deleted using the rmgroup command.

Syntax: tellnews undelete <group wildcard>

Example: tellnews undelete "local.secret*"

Tellnews commands are just typed in on the command line - with the Windows NT and Macintosh versions from the DNews Manager interface select "commands" and select the "undelete" command.

Purge "Deleted" News Groups - purgegroups command

This command allows you to purge from your system records all those groups that have been flagged "delete" in the system with the rmgroup command. You must stop and start DNews after using this command. If you use this on a sucking feed and the deleted groups exist on the site you are sucking from, then the groups will re-appear.

Syntax: tellnews purgegroups

Delete an Article - Killitem Command

This command allows you to delete an item in a news group. The item is defined either by its message ID or by a combination of its number and the group it is in. This information can be found by using a news reader (eg WinVN) that shows the article numbers. Killitem is usually used for removing inappropriate postings from private local groups. The Message-ID of a message can be found by clicking on 'show all headers' in any news reader.

Example: tellnews killitem "<234532.34@netwin.co.nz>"

Tellnews commands are just typed in on the command line - with the Windows NT and Macintosh versions from the DNews Manager interface select "commands" and select the "killitem" command.

Controlling Access to the Server and News Groups

DNews supports a full range of options to manage user access. This section describes how to control who has access to which news groups on your server and the type of access they will have.

Control third party access to your server

Access to your server is controlled by the access.conf file.

The Syntax for access rules in access.conf:

address:access_type:username:password:groups

DNews will by default try and put an appropriate rule in access.conf to prevent outsiders from accessing your server as follows:

The entry should look like this:

Example - Extract access.conf file:

"*:logoff:::!*" is read - any address log them off and don't allow access to any newsgroups. There is no user name or password requirement.

The next two lines have the same affect, and are read - allow anyone from the address 161.29.2.* (or Domain Name- netwinsite.com) to read and post articles to all groups. Both lines are put in in case your DNS is not working. Again there is no user name or password specification in this example. On NT the reverse DNS translation is disabled by default so you might want to enable it if you want access.conf to work for 'names' as well as numbers.

The order of the rules in the access.conf file is important, the server applies the last matching rule to decide if it will allow access or not.

On Windows from DNews Manager, select "Users & Feeds In" to view, select "add" to add a new access rule or select a feed, then select "modify" to modify an exiting rule or "delete" to delete a rule.

Control User Access to News Groups

DNews allows you to control access to your news groups by IP name, IP number and/or user name and password.

Unlike other news implementations, you can permit reading and posting to different combinations of newsgroups; for example, you may want a group of employees to be able to read all news groups but to only be able to post to local newsgroups.

On most operating systems you can use the Operating Systems Authorization mechanism to validate users. For instance on Windows NT you can use NT Authorization to look up users via the NT user database. See the setting $lookup$ for details. This is often the best approach if you have a lot of users and you want to use individual usernames and passwords.

You can limit the number of concurrent users from a particular domain or number of simultaneous connections from individual IP addresses within a domain.

The access.conf file controls access to your server. The syntax for all access rules in the access.conf file is as follows:

Syntax:

IP Names or Numbers:Access types:UserName:Password:Groups:Extention Settings

(read- for these IP names or numbers: provide this access:if they provide this username and this password: to these groups: and apply these extentions)

"IP Name or Number" - List IP Names or Numbers separated by ","or Use wildcard "*" to specify a group of IP Names or Addresses to apply the access rule to. e.g. 161.29.2.*

"Access Type" - Specify Type of Access to allow, Read and/or Post or Logoff. eg Read,Post
"UserName" - Specify UserName (if any)
"Password" - Specify Password (if any)
"Groups" - List the groups you want to allow access to separated by "," and list the groups you want to exclude access to using "!" in front of the group name. Use the wildcard "*" to match ranges of groups. e.g. *,!*.binaries.*,!netwin.secret.*
"Extensions" - These are optional and allow you to limit the number of concurrent users and or connections allowed.

Remember: The order of the rules in access.conf is important, as the server applies the last matching rule to determine if a user has access, so remember to put defaults at the top not the bottom:

On Windows from DNews Manager select "Users and Feeds In", select "Add" to add a new rule or select a rule and select "Modify" to modify the rule or "delete" to delete the rule.

Example 1: This example illustrates how to control access to specific groups by utilizing IP names or addresses. In this case we give all our users access to all groups except our secret local groups. And we give IP number 161.29.2.10 access to all groups including our secret local groups. Entries in access.conf:

*:logoff:::!* (This is a General default rule. Read - any address log them off and don't allow access to any news groups)
161.29.2*:Read,Post:::*,!netwin.secret*
*netwinsite.com:Read,Post:::*,!netwin.secret*

(This is a General default rule. As noted above, these two lines have the same effect. Read - for any of our users identified by matching the first part of the IP name or number allow access to all groups, in this case we have modified it by adding ,!netwin.secret* to the groups field, which the server reads - but don't allow these addresses access to groups with netwin.secretanything in the name header)

161.29.2.10:Read,Post:::* (This is a Specific rule. Read for IP 161.29.2.10 allow access to all groups including secret ones)

On Windows from DNews Manager on NT:

Step1: select "Users and Feeds In", select the general default to modify, select "Modify" to modify the rule. In the "Groups" field add ,!netwin.secret* modify the rule. Modify both the IP name and IP number defaults.

Step2: select "Users and Feeds In", select "Add" to add a new rule. In "User/Feeds" field enter 161.29.2.10, in the "Groups" field enter *, Select the "Read" and "Post" access boxes.

Example2:

This example illustrates how to restrict access by utilizing user names and passwords. In this case all our users are given access to all groups except the local secret groups. Access to all groups including secret groups is provided to users supplying the username "steve" and password "secret".

Entries In access.conf file:

*:logoff:::!* (General Default Rule. Read - any address log them off and don't allow access to any news groups)

161.29.2.*:Read.Post:steve:secret:* (Specific Access Rule. Read - for any IP that matches161.29.2.anything allow access to all news groups if the username "steve" and password "secret" are supplied)

161.29.2*:Read,Post:::*,!netwin.secret*

*netwinsite.com:Read,Post:::*,!netwin.secret* (General Default Rules. Read - for users with IP name or numbers matching allow access to all groups except local secret groups)

Important Note: There is a bug in Netscape 1-3 New Reader Software that requires an additional modification to the above. If you specify in any access rule a group of IP numbers with a user name and password, Netscape will require a user name and password to be entered by every IP number included in the specified group. Netscape reads the last rule "For every IP address matching 161.29.2.anything I must have a user name and password. If the user name and password is "steve" and "secret" then I will give access to all groups. If they don't provide a user name and password I won't provide access at all.

To allow for this behavior for Netscape, you need to add an additional access rule and provide your users with a general username and password. In this example General Username = NetWin and General Password = news

Example:

*:logoff:::!* (General Default Rule. Read - any address log them off and don't allow access to any news groups)

161.29.2.*:Read.Post:steve:secret:* (Specific Access Rule. Read - for any IP that matches161.29.2.anything allow access to all news groups if the username "steve" and password "secret" are supplied)

161.29.2*:Read,Post:netwin:news:*,!netwin.secret*

*netwinsite.com:Read,Post:netwin:news:*,!netwin.secret* (General Default Rules. Read - for users with IP name or numbers matching allow access to all groups except local secret groups)

Eample 3: In this case let's assume we want to set up access.conf to:

1. give everyone in the world read only (not post) access to all groups but not local groups.
2. give local users with the password "jelly" and username "jane" access to read and post to all news groups, but not local secret news groups.
3. give everyone with the password "secret" and username "mrtop" access to read and post to all groups including local secret groups.

Extract access.conf:

*:logoff:::!* (This line is read - log off anyone and don't give access to any groups. It is not required at all in this example and only left in here to highlight that because DNews applies the last matching rule it would ignore this line anyway as the next line *:read:::*,!local.* becomes the general default rule.)

*:read:::*,!local.* (Read - give everyone read access to everything except local groups)

161.29.2*:Read,Post:netwin:news:*,!local.*
*netwinsite.com:Read,Post:netwin:news:*,!netwin.* (General Default Rules. Read - for local users with IP name or numbers matching allow access to all groups except local secret groups. Note they need to supply the username "netwin" and password "news")

161.29.2*:Read,Post:jane:jelly:*,!local.secret* (Read - for local users with username "jane" and password "jelly" give read and post access to everything except any groups with the name local.anything.)

161.29.2*:Read,Post:mrtop:secret::* (Read - Give Local users with the username "mrsecret" and password "top" read post access to all groups)

Using Operating System Authorization Database to Control User Access:

To use your operating system authorization database to control user access in your access rules set <user> as "$lookup$" and <password> as "$lookup$" then your operating system. Authorization is used to lookup users via the operating systems database.

In access.conf you should add an access rule like this:

Example: your.ip.range.*:read,post:$lookup$:$lookup$:*

(read - for your ip range allow read and post access if user name and password are correct to all groups)

At the same time you must add to dnews.conf the line:

auth_case true

This tells DNews to be case sensitive when testing user names and passwords as NT username/passwords are case sensitive and by default DNews is not case sensitive.

Using external authorization database on unix. V4.5d

DNews provides external authentication support for Linux, FREEBSD, Solaris. See the file libauth.c - this file provides a Sample dynamic link library for Unix authorization. Use the procedure libauth.sh to build the shared library.

In dnews.conf you need to define:

auth_dll true

You need to implement a function that accepts these parameters

Inputs: ipname, ipnum, user, pass

Output: postgrps, readgrps, msg, int auth_user(char *ipname, char *ipnum, char *user, char *pass ,char *postgrps, char *readgrps, char *msg, int bfsz);

Example Skeleton code:

*/
#include
#include
#include
#include
#include
int auth_user(char *ipname, char *ipnum, char *user, char *pass ,char *postgrps, char *readgrps, char *msg, int bfsz);
#ifndef TRUE
#define FALSE 0
#define TRUE (!FALSE)
#endif

Access Extentions: Limiting the number of concurrent connections or simultaneous users

These extentions to access rules in access.conf allow you to limit the number of simultaneous connections from one IP and/or concurrent users from one domain. This can be useful to control users where you are selling a user or group of users a specific service.

The basic syntax for all access rules in access.conf is:

IP Names or Numbers:Access types:UserName:Password:Groups:Extention Settings

Extensions:

con=n (read - limit the number of concurrent connections from this IP to n)
users=m (read - limit the number of simultaneous users from this domain to m)
con=n,users=m (read - limit the number of concurrent connections from an IP address to n and limit the number of simultaneous users from this domain to m)

Example:

161.29.2*:Read,Post:::*,!netwin.secret*:con=3,users=20
(read - for the domain 161.29.2.* allow read and post access, there is no user name or password required, to all news groups except netwin.secret.anything but don't allow any single IP number in this domain to have more than three concurrent connections and don't allow more than 20 users from this domain to connect at any one time.)

The extension settings can get complex because they are not applied strictly by order like other settings. This is because in both cases the extension rule is affected by other users so the logic cannot be worked out by a single parse of the file. As a result you cannot limit the number of concurrent users to overlapping ip ranges, for example:

*:read,post:::*:users=2
*netwinsite.com:read,post:::*:users=10

Would NOT work, because the limit of '2' would apply to everyone not just netwinsite.com users.

Complex access.conf example

Here is a more complex example that would work: In this example we have two domains, 161.29.3.* and 161.29.4.*, one of which can have 50 concurrent users, the other 10, both are limited to 5 connections per reader to prevent a single reader hogging the system. We also have a username and password for the 'sex' groups. "naughty,secret", and two users have access to a private news group "netwin.private"

*:logoff:::*
161.29.*:read,post:naughty:secret:*,!netwin.private
161.29.*:read,post:joe:secret2:*,!*sex*
161.29.*:read,post:fred:secret3:*,!*sex*
161.29.3.*:read,post:::*,!*sex*,!netwin.private:users=50,con=5
161.29.4.*:read,post:::*,!*sex*:!netwin.private:users=10,con=5

Control Expiration of News

With version 5.x of DNews you can now use a cyclic database. With this system as new items arrive if the allocated space has been used then the oldest items are simply deleted and replaced with the new incoming items, in this way the system can never 'run out of space' and you always have as much news available to your users as is possible.

Example pile rules in expire.conf

	pile 1 use 95% groups *
	remember 3

The above rule is all you need, however to maximize the use of your resources you may want to tell DNews to use a certain amount of space for binaries, and also to keep important local groups for much longer, e.g.

	pile 1 use 20% groups *
	pile 2 use 60% groups *binaries*,*pictures*,*warez*
	pile 3 days 90 groups local.*,netwin.*,rec.humor.funny
	remember 3

Advantages over older expire systems:

• Disk utilization can remain near 100% all the time as buckets are deleted as space is needed
• Nightly expire processing is enormously faster
• Expire rules are much simpler
• Individual sets of news groups (e.g. local groups) can be easily backed up separately
• Groups can be set to expire based on a %age of your total disk use and/or an age in days

See this page for a detailed description of expire settings

Log

This section describes DNews log.

Control Log level - loglvl command

The loglvl command in the dnews.conf file allows you to change the level of logging as desired. There are four log levels:

Error - The log shows serious errors only.
Warn - The log shows serious errors and provides warning messages.
Info - In addition to warning and error messages the log provides useful information about what the server is doing. This is the default setting.
Debug - Used for debugging
Flush - This is the same as debug except the log file is flushed with every single line written; this is useful if the system is crashing but not providing a stack dump.

Syntax: tellnews loglvl error/warn/info/debug/flush

Example: tellnews loglvl warn

Tellnews commands are just typed in on the command line. With the Windows NT and Macintosh versions, from the DNews Manager interface select "commands" then select the "loglvl" command.

View Log Now - log command

The log command returns the last 50 lines of the dnews.log file which contains all log information.

Syntax: tellnews log

Tellnews commands are just typed in on the command line. With the Windows NT and Macintosh versions from the DNews Manager interface select "commands" and select the "log" command.

Reporting

DNews provides a variety of operating and performance information. This section describes how to obtain and read operating reports.

Email address to send daily reports - manager setting: The manager setting in dnews.conf gives the server the email address you want the daily report emailed to. DNews asks for this during the setup process.

Syntax: manager system@here.this.place
(On Windows from the DNews manager select Main Options and enter email address. Then select write changes and reload.)

The Daily Expire Report shows all the information from the tellnews status, tellnews stats_groups and tellnews stats_in reports, discussed below.

View DNews Status Report Now - Status Command

The status command returns the DNews Status Report which details the status of the system.

Syntax: tellnews status

Tellnews commands are just typed in on the command line. With the Windows NT and Macintosh versions, from the DNews Manager interface select "commands" and then select the "report" command.

Understanding DNews Status Report

The following shows a typical status report and includes explanatory comments

General Section:

200 161.29.2.10 DNews Version 4.2j, S0, posting OK

"200" - This is the NNTP response received from the server.200 means the server is ready #and posting is allowed.
"161.29.2.10" - This is the IP address of the server running DNews.
"DNEWS Version 4.2j " - This is the Version of DNews running.
"S0" - If you are running DMULTI this tells you which slave you are talking to e.g. 0,1,2,3,... # slave 0 is the DNews Master.
"posting ok" This tells you posting from your IP address is ok.

License Section:

483 Temporary license expires after 31-Aug-99
483 Please read LICENSE.TXT for registration information

This Tells you when your evaluation period will end and where to find registration instructions.

Feeds Section:

Feed[0]: me *,!alt.*,!*.binaries.*,@news.*,news.software.*
Feed[1]: 161.29.2.5 Posting Builtin Posts=0 0/s Rej=0 Req=0 Dup=0 R=100:0:0 Tot=0/s 0k/s *
Live cache misses 0/0
#This section povides a list of the feeds in newsfeeds.conf file. The newsfeeds.conf controls the feed into your server and the feeds out of your server . Each line specifies the rules currently applying to each feed.

"Feed[0]: me" - this is the feed into your server or me feed.
"*,!alt.*,!*.binaries.*,@news.*,news.software.* " - this defines which news groups will appear on your server and is read; accept articles for all groups except those with alt.anything and those with anything.binaries.anything in the Newsgroups header. Do not accept articles with news.anything in the Newsgroup header, but do accept articles with news.software.anything in the newsgroup header."
note 1: the order of the rules is important.
note2: "!" or "@" :"!" is used to prevent articles from being accepted to a specified newsgroup. If the article has been crossposted to other groups not specifically excluded it will be accepted to those groups. "@" is used to prevent an article posted to a specified group from being accepted to the specified group or any other group it may be cross posted to.)

Feed[1]: 161.29.2.5 Posting = Posting like a news reader. If Posting is not here it means it is sending using IHAVE Builtin = DNews has two mechanisms built in for sending feeds to other servers. The normal mechanism is called "builtin" and the other is called "live" for live feeds. If neither builtin or live is specified DNews simply creates a file of the list of things to be sent.

Posts=0 0/s Rej=0 Req=0 Dup=0 R=100:0:0 Tot=0/s 0k/s * This shows for the feed1 since startup; the number of articles posted, the number of articles rejected by the downstream server, the number of articles the downsteam server could not accept immediately and were requeued for sending later, and the number of duplicate messages that had already been received by the downstream server. The R=n:n:n is for debugging purposes. Tot=0/s 0k/s shows the total number of items sent per second and the total K bytes sent per second.

Live cache misses 0/0 = This shows the number of times the live feed had to read an item from disk because it was not in its local cache when it tried to send it.
If there is a queue of articles waiting then the size of the queued files is also shown.

Del= If the queued files grow larger than 10mb, they are deleted, and this figure keeps a record of the total size of deleted queue files. This allows DNews to keep operating when a site is down for days or weeks. The backlog could otherwise quickly fill up your entire disk.

Currently Connected Sessions:
Chan[0:140] o=0 in=0/s out=0/s 0 161.29.2.10 161.29.2.10 C_CMD
Chan[1:108] B o=130 in=15000/s out=1193/s 0 161.29.2.10 test.netwin.co.nz C_IHAVE

This is a listing of the currently connected sessions, and shows both their IP number and name. In this case the IP number is shown twice as reverse translation is disabled.

"B" This means the channel is blocked, DNews is waiting for the other end to take all the output
"o=130" This means the output buffer has 130 Bytes
"in=15000/s" This means the input speed has been 15KBytes/second for the last 30 seconds
"out=1193/s" This means output speed has been 1193Bytes/second for the last 30 secinds
"C_CMD" this means the channel is waiting for a normal NNTP command
"C_IHAVE" this means the channel is part way thru an IHAVE transaction

Expire Section:

Expire started at Mon Jul 21 02:00:05 1997 Done, start 44% end 44%
Processed 12482 removed 0 deleting 2372/-1/372 speed 1282.93
Expire finished at: Mon Jul 21 02:00:32 1997, Elapsed 0 hours, 0 minutes

"Processed 12482" - This is the number of history lines in your history file.
"removed 0" - This is the number of history lines removed
"deleting 2372/-1/372" - This is the number of items being deleted, total and actual.(of 2372 items deleted 2000 were cross posts so don't count as real deletes)
speed 1282.93 -This is the number of history articles processed per second
"Expire finished at: Mon Jul 21 02:00:32 1997, Elapsed 0 hours, 0 minutes" - time expire ran and time taken to complete.

Current Job:

Nothing is being sucked at present.
Group news.software.nntp completed, got 0 items

This sections reports on the status of the current background job, if any. In this case there is nothing in the sucking queue.

Statistics Section:

DB used 44.48MB, Slack 3.92MB 8.82%, DBI Misses 0/0, blocks 0=0k
History: disk 1598k memory 149k/360k, Str 74k/1572k/200k Malloc 5123k/6445k/3327k
Used 44MB=44%, Free work 26Mb, hist 26Mb, spool 26Mb, log 26Mb

This section provides overview statistics

The Data Base:

"DB Used 44.48MB" = The data base is currently using 44.48 MB
"Slack 3.92MB8.82%" = Gaps in the data base total 3.92 MB or 8.82% of the data base. Note - DNews does not bother to compress the data base further when under 5-10% by default.

History File:

"disk 1598k" - the history file is currently 1.598 Mbytes on disk.
"Used 44MB=44%" - DNews is using 44% of available space allocated to it.
"Free work 26Mb" - Free space on disk with Work Area
" hist 26Mb" - Free space on disk with history file
" spool 26Mb" - Free space on disk with spool file
"log 26Mb" - Free space on disk with log file
note: in this case the work area, history file, spool and log are on the same disk.

Groups Xover 5 Cached 1 of 25594, GID 25594, Uptime 0 day(s) 22 hour(s), XOver Cache 66.58%

"Xover 5 = 5" - Groups with more than 10 items read using Xover in the last week
"Cached 1 of 25594" - Of a total 25594 available groups 1 is currently being read
"GID 25594" - Number of groups IDs allocated
"Uptime 0 days 22 hours" - the system has been up for 22 hours.
"XOver Cache" - % of Xover records found in Cache therefore not required to be read off disk.

Status Report Now - Report command

The report command forces DNews to generate the status report and mail it to the manager address set in the dnews.conf file. Reports are normally sent automatically on a daily basis but the command is useful if you are not getting these reports as it allows you to check the log file and see why the report isn't successfully sent.

Syntax: tellnews report

Tellnews commands are just typed in on the command line. With the Windows NT and Macintosh versions, from the DNews Manager interface select "commands" and select the "report" command.

View Stats on Top 40 groups Now - Stats_groups command

The Stats_groups command reports the top 40 groups by the number of times they have been read by users and by the volume of articles received into that group. This allows you to spot groups that are gobbling up diskspace and bandwidth but are not being read by a significant number of users.

Syntax: tellnews stats_groups

Tellnews commands are just typed in on the command line. With the Windows NT and Macintosh versions, from the DNews Manager interface select "commands" and then select the "stats_groups" command.

This information is also included in the Daily Expire Report.

View Statistics on Articles Accepted since expire - stats_in command

The stats_in command provides statistics about what DNews has sucked/accepted since the last expire:

Syntax: tellnews stats_in

Example Stats_in report with explanations:

200 161.29.2.5 DNews Version 4.5b2, S0, posting OK
Sites that have sent us articles since the last expire:
Site Posts Rejects Dup Total Articles/S Kbytes/s K
news.iconz.co.nz 2875 0 0 2875 0.75 1.54 5870k
The report shows news.iconz.co.nz sent 2875 items, 5.8MB at 1.54K per second, no items were rejected,

Tellnews commands are just typed in on the command line. With the Windows NT and Macintosh versions, from the DNews Manager interface select "commands" and select the "stats_groups" command.

This information is also included in the Daily Expire Report.

Confirmation of Posts to Users - the confirm setting

You can configure DNews to send a confirmation message by email to users to confirm their post has been transmitted to the Internet successfully. This is normally set at the time you install DNews. The "confirm" setting is in dnews.conf.

Syntax: confirm *.this.domain,*.second.domain

The following example sets "*.this.domain" to match our local email addresses. i.e. *netwinsite.com.

Example - Extract dnews.conf:

confim *netwinsite.com (read - send email confirmation of post on successful transmission to users posting from anythingnetwinsite.com)

On Windows from DNews Manager select "Main Options" and in the "Confirm Postings" field enter the domains you want to confirm postings for.

Censoring News Articles - Spam Protection

A feature of DNews is the advanced set of methods for detecting and automatically removing inappropriate or nuisance articles (SPAM) from news groups on your server. You may reject articles based on: the number of cross posts, the identity of the poster, words used in article headers or anything in the content of articles. You can apply different rejection rules to different news groups. You can set the maximum number of articles per hour from any user or host and the maximum number of duplicate bodies.

For details on spam protection and the latest sample configurations see this page

In brief, adding spam_stop true to dnews.conf will allow DNews to reject messages that it determines are 'spam'.

Reject Rules Cross Posting

A common type of spam is where a single article is cross posted to many news groups. Genuine news articles should only be posted to one or a very small number of news groups relevant to the article subject.

To prevent this kind of spam, you can set the maximum number of cross posts on your server so that any article that is cross posted to more than the specified number of news groups will be deleted. The maxcross is set in the newfeeds.conf file and has the following syntax.

Syntax: maxcross n

(where n = maximum number of news groups an article may be cross posted to without being rejected)

Example: The following example shows the entries in newsfeeds.conf to exclude binary groups and articles cross posted to them and reject any article cross posted to more than 10 groups)

Example - Extract newsfeeds.conf:

site me (site me is your main news feed)
groups *,@*.binaries.* (This line tells the server which news groups to accept. "*" = accept all groups; "@*.binaries.*"= don't accept those matching anything.binaries.anything or articles cross posted to anything.binaries.anything)
maxcross 10 (This line tells the server to reject items cross posted to more than 10 groups )

(On NT in DNews Manager select "Feeds Out", select "ME Feed" (This is your feed in) and enter the maximum number of cross posts to allow in the "Max Cross" feild .)

Reject Control by Article Headers

With any feed in 'newsfeeds.conf' you can add accept/reject rules based on words or expressions used in article headers to control which news items the server accepts into news groups.

The rules have the following syntax:

Syntax: accept/reject article_header "key words in header"

The following example illustrates how you might edit newsfeeds.conf file to reduce spam getting onto your server, by adding rules on the 'me' feed, using key words in the subject, newsgroup and from article headers.

Example - Extract newsfeeds.conf:

site me
groups * (read - accept articles for all groups)
accept default ( read - accept the article unless one of the following matches)
reject subject "sex" ( read - reject the article if the word sex is in the subject header)
accept newsgroups "sex" ( read - accept the article if the word "sex" is in the news group header)
reject from "baduser@badsite" ( read - reject articles from baduser@badsite)
reject subject "make money fast" (read - reject the article if the phrase "make money fast" is in the subject header)

This example rejects any message with 'make money fast' anywhere in its subject header, and all articles from baduser@badsite, and also rejects articles that contain the word 'sex' in the subject unless they are in a news group that also contains the word 'sex'.

The list of rules is processed in order, and if the rule 'matches' then the 'result' is changed to accept or reject accordingly. So you must have a default rule first and the order of the rules is important.

Accept/Reject Control by article Content

With any feed in, in 'newsfeeds.conf' file you can edit the file to add rules to detect and reject articles by matching key words or phrases against the entire body of the article.

These rules follow the same syntax as above - but utilize the fake news article headers; 'body' meaning everything except the headers, and 'article' meaning everything including the headers.

Syntax: accept/reject body/article "keywords"

Note: The matching is NOT case sensitive. If the header CONTAINS the specified text then the rule is applied, rules are applied in order, if no rule matches then the item is accepted, unless there were any 'accept' rules in which case the item is rejected.

Example - Extract newsfeeds.conf:

site me
groups *
reject subject "sex"
reject from "baduser@badsite"
reject body "naughtyword"
reject article "make money fast"
site local.interest
groups *
reject subject "boring thread"
accept body "netwin.co.nz"
accept body "dnews"

Read - from my main feed site accept all groups, reject any article with "sex" in the subject header, reject articles from baduser@badsite; reject articles with "naughtyword" anywhere in the body; reject articles with "make money fast" in the header or body.

From my local.interest feed site reject articles with "boring thread" in the subject header unless they have "netwin.co.nz" or "dnews" in the body.

For performance reasons it is best to only use headers rather than body and article, and it is also best to minimize the number of rules. A more efficient way to implement this type of control over content is to use a file as described under the heading Serious Spam Protection below.

Accept/Reject Articles - Special - Mime Attachment etc

There is one special character allowed '\n' which can be used to match 'beginning of line' in the body or article option.

To reject items containing Mime attachments you might add something like the following line to newsfeeds.conf:

reject body "\nContent-Transfer-Encoding: x-uue"

Serious Spam Protection V4.5

The following will provide some automatic protection from spam. These settings can be added to dnews.conf, you can just set 'spam_stop true' to enable the the default settings. You can fiddle with all these settings and thus tune your system differently to other peoples which may be an advantage as 'spammers' will try and get past some of these rules.

Setting Default Description
spam_stop false Enables spam protection
spam_test false Enables spam protection but doesn't reject anything, just logs what it would do.
spam_keepn 5 Number of rejected items to store in spam_n.tmp files (for reviewing)
spam_dup_total 5000 Number of items to 'remember' in our history of 'duplicate' bodies. Uses about 30 bytes per entry.
spam_dup_hits 6 Number of duplicate bodies (not cross posted) to allow
spam_dup_cross 400 Number of duplicate bodies counting cross posts to allow
spam_from_hits 60 Number of messages from any single user per hour
spam_from_cross 1000 Number of messages counting cross posts from any single user per hour
spam_from_total 5000 Number of users to keep tabs on in our history of duplicate users, uses about 100 bytes per user.
spam_host false Use 'nntp-posting-host' instead of 'from' for the above 3 rules.

In addition to fiddling with all the above settings, you can create a file "filta.dat" in the WORKAREA as defined by dnews.conf and add lines to the file, any message containing text matching the lines in the file will be rejected.

The idea is to use things that common spammers place in the body of their messages, this allows you to stop anyone who manages to get past the automatic mechanisms above.

e.g. c:\dnews\spool\filter.dat

Example - filter.dat file:

sex
1-800-3333-4343
1-800-SEXYPHONE
www.sex.
Make money fast

We intend to create a second file "filter.net" which will be made available from these web pages, this will be a list that is updated regurlarly to stop well known spammers. DNews will use both files (filter.dat and filter.net) you should place your own settings in the filter.dat file.

DNews uses a 'super' efficient mechanism to test matches in this file so that you can have several hundred lines in the file without performance suffering, each line must be at least 3 characters long. The matching is not case sensitive and is only in the 'body' of the message.

For details on spam protection and the latest sample configurations see this page

Performance

See this page for a detailed descussion of performance options and factors.

Scalability

As the popularity of USENET has grown, many sites using old news server software have found themselves struggling to support increasing numbers of concurrent users, a larger news spool, and to keep up with a full news feed.

The DNews News Server is significantly faster at taking a news feed than any other news server software. DNews has no difficulty keeping up with a full news feed. DNEWS's efficiency and scalability allows sites to provide a high quality news service and maximize benefits from their investment in hardware by supporting more users per server with high performance. This section describes strategies for larger sites.

A single properly configured DNews server can support 1-10000 concurrent users or more, see this page for more details.

DMULTI Installation and Use

DMULTI is a mechanism for running multiple DNews processes on a single server utilizing a master/slave architecture, and allows you to get more out of your machine. With DMULTI you start 2-5 DNews processes, each one dealing with 50-400 concurrent users, this allows it to scale well and make use of multiple CPU's etc. DMULTI will start to be worthwhile when you have more than about 50 concurrent news readers on your system, and/or if you have people reading 24 hours a day and the expire process is causing performance problems. Generally it will give reasonable results if you have sufficient memory and multiple disk drives. If you have more than 200 - 500 concurrent users it is recommended that you configure your server with the DMULTI option. Because disk IO is often a bottleneck, we recommend multiple servers after a certain point (around 500-1000 concurrent users as a rough guide) for best performance, and this gives the added advantage of 100% hardware fail over for emergencies.

See this page for instructions on turning on/off dmulti

Multiple Servers

Multiple servers give better performance and a good backup mechanism, and with the price of PCs and disks, its probably better to budget for 4 cheap servers than to try and build 1 super server.

If you use more than one news server, you can configure DNews to send out an xreplic feed, this means that Item 23 in 'rec.humor' on the master DNews machine will be the same as Item 23 in 'rec.humor' on the slave system. This enables you to have 'n' news servers. This also gives you a 'backup' as any server can be removed or made the 'master' in a minimum of time.

However, another option is to simply configure several news servers and use normal feeds between them and then spread your users by simply telling them to use different news servers. This is simpler and therefore slightly more reliable. But this does prevent you from any load balancing.

Hardware:

A typical 'large' system would look like this:

Single or Dual Pentium 200
256-1024MB RAM
20-500Gig SCSI disks (or raid array)

Operating System for large servers

We don't want to get into an OS war, but basically the following operating systems (in no particular order) make good news servers with DNews. This only becomes an issue when you are trying to pack as much news and as many users as possible onto one server: Windows NT, Linux, Solaris, AIX, HPUX, FreeBSD, BSDI

For more than 1000 concurrent users we would recommend NT/Solaris, other operating systems may work as long as you can increase file handle limits which usually prevent more than 1000 concurrent connections.

With most of these systems it is important to configure them with at least 200MB swap partitions when you install the system, as NEWS uses a lot of memory and changing the swap partition can be difficult.

Putting News On the Web - DNewsWeb

DNewsWeb is a web to news gateway, it lets you merge your web pages and your news groups. By displaying usenet and local news groups on your web pages, users can read and post news directly using their favourite web browser.

DNewsWeb is provided FREE with your DNews News Server and is quick and simple to set-up.

See this page for installation instructions for DNewsWeb

Gatewaying Mailing Lists

With DNews you can setup gateways to send news messages to mailing lists, or email/mailing list messages to a news group, see this page for details of configuring this.

Taking a newsfeed direct from ClariNet

You can of course take a news feed from clarinet, configuraton instructions are available on this page.

Adding PGP Authentication to DNews

Install PGP software on your computer

PGP allows your server to check the authenticity of incoming control messages, see this page for details on installing pgp.