Products Downloads Prices Support Company
Index FAQ Configs Feeds In Feeds Out Feeds Out  

DNews Updates List

4.2m (11-June-1997)

Added 'default' keywords to accept/reject rules to allow more complex rules, here is an example using the new option, this example is to prevent the word 'sex' appearing in a group that does not have 'sex' in it's name. 

site clead_feed
	groups *
	accept default
	reject subject "sex"
	accept newsgroups "sex"
	accept newsgroups "erotic"

Added tellnews command

tellnews trimfile FILENAME

Reads FILENAME and then deletes any news group that is not in this file.

Increased length of demo license by one month

Added setting xover_moved, set this if your xover data is NOT stored on the same DISK as your main SPOOL area.

Added xover data into diskusage calculations.

Added expire status showing failure if expire fails due to a disk shortage at a critical moment.

Fixed bug that allowed groups to be created that did not match the 'ME' feed.

Changed log file format, added channel number to all log lines

Added command tellnews showusers, this shows a summary of users who have connected since startup. Note: the ngroups settings is the number of group commands recieved, not the number of unique groups read by that user.

Added -file FILENAME qualifier to drobot, this allows it to be used with eudora on win95

Fixed newgroup distribution to slaves with dmulti

Made most background commands like (getgroups, update, rebuild_index, trimgroups...) go into a queue so several can be given at once and they will show in the update queue.

Improved uucp bag handling to make it more robust in the face of faulty bag files. Added uucp_binary option to dnews.conf. Only set this if you are having problems. It will sometimes help with certain faulty uucp formats.

Fixed memory corruption caused by bug in checkgroups processing.

Added dnews.conf option, newnews_users wildcard_list, this lets you control which users can use the 'newnews' command.

Added dnews.conf settings: 

	bind_in 152.4.55.2
	bind_out 152.5.55.3

The bind_in setting controls which ip address DNews will listen on, the bind_out address controls which ip address DNews will appear to have when it is reading or sending news. (You must have multiple ip addresses on the one host for these settings to have any meaning)

Fixed signal 2 error on OS/2, caused by floating point underflow.

4.2k (23-May-1997)

Added log_file setting to DNewsWeb, this allows the log file to be specified

Fixed spool_import/convert for systems which don't place the xref header in non cross posted messages.

Added two feed options:

	site x.y.z
	onlyposts
	path_max 4

onlyposts is a switch which tells that feed not to send anything except locally posted articles.

path_max is a rule which stops items being sent to that feed if the item has more than 3 hosts in the path. Both of these are options to use instead of exclude which can be tricky to set. When using path_max remember that some posting software will add a path so allow a couple of extra hosts in the path. (e.g. 4 is a good number)

Fixed access bug related to logoff.

Fixed OS2 problem with large spool areas pausing the incoming feed.

4.2j (16-May-1997)

Fixed minor bugs in dnewsmanager

Fixed key problem with spaces in computer names.

Made license key's more friendly & reliable.

Increased line length limit on newnews command to 3K from the RFC suggested 512 characters.

Improved format of search output in DNewsWeb (xresults.tpl)

Corrected default post.tpl file in DNewsWeb

Corrected fault with xover missing some headers under certain rare conditions

Added DNews setting 'cache_onxover true' which will cache a group if a user requests xover records from it.

Added setting "suck_maxgroups 30", if more than this many groups are 'cached' then DNews will 'uncache' all groups automatically. This can be very useful on small systems.

Fixed erroneous sucking of uncached local groups from upstream.

Improved xover performance for NewsXpress (by a large factor) and changed the default setting for tcp_nodelay to 'true' on Windows NT.

Added expire option 'unread n' which will expire items that are more than n days old if they are not in the list of cached news groups. e.g. to delete items in unread groups after 2 days

...
upto 40% groups * default 5 maxitems 500 minitems 50 unread 2
upto 20% groups * default 10 maxitems 1000 minitems 100 unread 2
...

4.2i (8-May-1997)

Fixed Nothing to trim xxxx message (improves performance)

Fixed pgp for NT services. (Caused by popen bug on NT)

Added dnews.conf setting for PGPPATH

Fixed feed type prog on NT, (also caused by the popen bug on NT)

Inforced max str size limit which decreases memory use.

Modified string block sizes to decrease memory use.

"EXIT" command added to feed programs, this indicates DNews is exiting and the feed program should exit also.

Improved tellnews killitem <msgid>, it now deletes all cross posts and clears the cache correctly as well.

4.2h

Improved handling of suck_uucp, it now gives friendlier errors and if it fails it renames the failed file to 'uucp.bug', in this way faulty uucp files will not fill up your entire disk.

DNewsWeb, changed items.tpl to use preformatted text which allows threading display to be much clearer.

Added DNewsWeb variable %%spaces%% to indent threaded items.

Fixed bugs with DNewsWeb crashing on some news groups.

Allowed PGPPATH to be defined in dnews_start.sh

Fixed passing of TZ variable in dnews_start.sh

4.2g

Fixed Solaris x86, partial data appearing when under load

Fixed Solaris x86, dmulti bug, channel disconnecting under load

4.2f (25-Apr-1997)

Fixed crash bug on Solaris X86 and OS2, the system would crash shortly after an expire run.

Changed behaviour, DNews now disconnects channels with ONLY IHAVE access if the disk is full, this helps some news xmit programs to 'wait a while' before trying again.

Fixed bug in dmulti

Added 'experimental' feed options, (experimental means this feature may be modified in a future version after we've heard some feed back on it.) example: 

	site me
		groups *
    		reject subject "sex"
    		reject from "baduser@badsite"
		reject body "naughtyword"
		reject article "make money fast"
	site local.interest
		groups *
		reject subject "boring thread"
		accept body "netwin.co.nz"
		accept body "dnews"

Any header can be used including the fake headers 'body' meaning everything except the headers, and 'article' meaning everything.

The matching is NOT case sensitive. If the header CONTAINS the specified text then the rule is applied, rules are applied in order, if no rule matches then the item is accepted, unless there were any 'accept' rules in which case the item is rejected.

There is one special character allowed '\n' which can be used to match 'beginning of line' in the body or article option.

To reject items containing Mime attachments you might use something like: 

	reject body "\nContent-Transfer-Encoding: x-uue"

For performance reasons it is best to only use headers rather than body and article, and it is also best to minimize the number of rules.

4.2e (Beta)

New newsfeeds option, @group.wildcard, this is similar to the ! character as it means don't allow these groups, but unlike the normal logic of newsfeeds, if an item is cross posted to several groups and ANY ONE of them matches the @ group wild card then the entire item is rejected. The is useful to combat the recent habbit of people posting 'unsavoury' articles cross posting those articles to popular groups to increase the distribution of their articles.

Example: (newsfeeds.conf)

    site me
       groups *,@*binaries*,@*erotic*

New conversion commands for converting unix (INN or CNEWS spool areas)

      tellnews spool_import /var/spool/news

The spool_import command finds all news articles in the directory tree specified and imports them into DNews WITHOUT deleting the articles from the source directory. DNews will use the same item numbers so that newsrc files will still be valid. This command is ideal for testing DNews without actually breaking your current news server, you can also specify a sub tree if you don't have enough disk space to have two entire copies of your spool tree. e.g.

    tellnews spool_import /var/spool/news/comp/lang

A second identical command exists which will DELETE the items after importing them into DNews, this can be used to CONVERT a spool system from INN to DNews format without having any extra disk space.

     tellnews spool_convert /var/spool/news

Again you can specify a sub tree if you prefer to do it in smaller pieces just incase something goes wrong :-)

If you do not care wether DNews uses exactly the same item numbers as the INN system then you can add this setting to dnews.conf

         convert_add true

By default, the above commands will NOT send items to outgoing feeds, to change this behaviour you can add to dnews.conf

         convert_feed true

New tellnews command to do a rebuild_index_faster, this works by skipping the history lookups for each item, this is reasonably safe as long as you don't have duplicate messages in your bucket files. This may be worth using on large systems (above 10 gig)

New tellnews command rebuild_lost. This command is good for finding lost items, it does not destroy the existing index and does not stop the incoming feeds. You might use the command to complete a rebuild_index if you had to shut down DNews before it completed. You can also use it to find the missing items that resulted from the expire bug in DNews prior to version 4.2a.

Fixed crash during flush2, caused by more than 20 incoming news feed names (MCI).

Fixed fault in access processing (authinfo bug)

Fixed unimportant messages "stat problem xxxxxx" filling log file

Fixed NT-authorization problem with access.conf.

Fixed bug in dmulti on unix systems.

4.2d (Beta 14-Apr-1997)

Added dnews.conf setting to disable the suck that occurs when a group is posted to:

	POST_NOSUCK true

Added dnews.conf setting to disable sucking groups, but not getgroups, trimgroups, this is handy for keeping the active files in sync between two servers, it can be used with the xreplic setting below.

	SUCK_DISABLE TRUE

Added support for nntp extension, xreplic.

Added live feed option 'xreplic' to send a replicating feed. e.g.

	site mirror.news.server
	type live
	live_nchan 5
	groups *
	xreplic

A replicating feed is one where you wish to run a 'farm' of news servers that all look identical to the users (therefor articles must have the same item number). Your external news feeds all go into one master server, and from that you send out an 'xreplic' feed to each slave or mirror news server.

At the same time you should set 'post_only true' so that local posts will be sent back to the master server before they are assigned an 'item' number. So on the slave you have in dnews.conf

post_only true
suck_disable
nntp_feeder master.machine

And periodically run the command tellnews matchgroups on the slave system.

Fixed crash if you try and suck from a server which has no news groups. :-)

Added GID reallocation so they will not run out.

DNewsWeb: Fixed bug with fts output

Note: dmulti is still experimental, particularly on unix systems, if you choose to use it please send us info on problems you have with it.

4.2b (Beta 10-Apr-1997)

Fixed bug in expire. Symptoms: the expire runs but less and less articles get deleted until you are stuck at above 90%. To fix upgrade to 4.2b and the issue a tellnews rebuild_index command

Fixed corrupt gid's between 42000 and 47000.

DNewsWeb: fixed problem with top.tpl not being found.

4.2a (Beta release 6-Apr-1997)

Fix for solaris dmulti problem with posting large items dropping connection.

Fix for problem with rebuild_hist occasionally missing an article

Fix for bug in vms version with bucket check loop.

Major new version of DNewsWeb, click for details

Unix install/upgrade utility simplifies installation and upgrades, e.g.

	uncompress dnews41d_linux.tar.Z
	tar -xvf dnews41d_linux.tar
	cd DNews
	./dnews_setup

4.1a (Beta release 3-Apr-1997)

Added dmulti, a mechanism for running multiple DNews processes, this is appropriate if you have more than about 50 concurrent news readers on your system, or if you have people reading 24 hours a day and the expire process is causing performance problems. This new feature is experimental at this time. It will probably only give reasonable results if you have sufficient memory and multiple disk drives.

This version of dmulti does not support the sucking system in DNews, it is assumed if you have this many news readers on your system that you will be using a full news feed.

DMULTI installation:

Configure your system to startup DMULTI instead of DNEWS:

Unix: Use dmulti_start.sh, instead of dnews_start.sh
NT: addsvc -del dnewssvc
NT: addsvc -add dnewssvc dmulti.exe "DNEWS Multi Process Manager"

Configure the new slave parameters in dnews.conf

slave_restart 5 # The hour at which to restart the slaves (5a.m.)
slave_ports 7500 # TCPIP port to start DNews slaves processes on
slave_n 3	# Number of DNews processes to run 2-4
slave_feed 202.37.150.84,161.29.2.* # Ip NUMBER of this host and feeds

The slave_feed setting MUST include 'this machine' and any machine that is sending you a news feed, ONLY ip numbers or wild cards will work (ip names will not work)

Add your own machine to the list of hosts that can take an IHAVE feed, add this in access.conf, if you miss this step out local posts will fail without any error message!

your.ip.number:ihave:::*

Startup the dmulti process:

NT: net start dnewsvc (or click on start in DNews manager)
UNIX: /usr/local/dnews/dmulti_start.sh

There is a log file dmulti.log which will show you if anything goes wrong.

How dmulti works

DMULTI starts up 'slave_n' DNews processes (typically 2 or 3) the first one S0, is the master process, this is the one that accepts incoming news feeds, and this is the one your tellnews commands will go to. The others, S1-Sn, are reader processes, each one might have 30 news readers on it. Dmulti acts as a gateway, routing the incoming connections to the correct DNews process, this is why your slave_feed parameter must be set correctly, this is how dmulti knows wether the connecting session is a reader, or a feed.

You can communicate with each slave individually using a new tellnews option. The first status line shows which process s0-sn is talking to you: 

(Master process) tellnews -s 0 status
(First reader) tellnews -s 1 status

You should always ensure that all the processes have closed before re-starting dnews/dmulti

To uninstall dmulti

Remove the slave parameters from dnews.conf and then replace the original DNews service (or startup script) 

Uninstall NT: addsvc -del dnewssvc
Uninstall NT: addsvc -add dnewssvc dnews.exe "DNEWS News Server"

New dnews.conf setting

access_twice true, if set DNews will try and allow users access without doing a reverse dns lookup by first checking if their ip number appears in your access.conf file.

4.0t 28-March-1997 Release version

4.0s

4.0r (23 March 1997)

4.0q

4.0p

4.0n

4.0m

4.0k

4.0j (5-Mar-97)

Installation Note for version 4.0:

With version 4 you need to allow for the xover index, which could be upto 10% of your spool area, by default these files will be placed in the WORKAREA directory, but you can move them by defining XOVER_DIR in dnews.conf

4.0f (21-feb-97)

4.0a (5-Feb-97)

Some history on the reason for DNews 4.0

The DNews database was originally designed for efficiently sucking news items. And it was very good at this, however, for various reasonsabout 50% of DNews users take IHAVE feeds, and although this worked well enough, when running tests on Version 2.7 on very large news sites we found DNews was starting to struggle. With the projected growth of news being expected to double news rates every 1-2 years we took immediate action to improve the speed of the database.

Further investigation showed performance varied from .1 to 15 articles per second depending on type type of feed (ihave or suck) and the type of operating system (NT, Unix, VMS ...)

It became clear that any system based on storing articles in directories, one directory for each newsgroup, would not be able to handle the loads over the next 2 years.

As a result we decided to completely redesign the database. The new database would no longer use a group based directory structure. The design goals were:

DNews 4.0 is the result of this new database, it succeeds in all the areas mentioned above, and more besides, typically it can take test articles at over 30 articles/second while still 50% idle. This makes DNews significantly faster than any other news server and probably the only technology currently available that will be able to take the increasing news loads over the next 2 years.

DNews 4, Technical Notes:

Unix Upgrade Notes (with multiple drives connected using symbolic links)

In dnews.conf use the setting:

spool_free true

And define each of your drives as one of the linked drives. e.g. if you currently have a second drive mounted as /var/spool/dnews/alt/binaries, then your dnews.conf should look like this:

spool /var/spool/dnews
spool2 /var/spool/dnews/alt/binaries
diskspace 2000
diskspace2 1000

After the conversion has finished it is safe to move the directories, but you cannot do this before the conversion because DNews must be able to find the 'old' files where they 'used to be'.

3.0j (11-12-96)

3.0h (11-12-96)

3.0g (10-12-96)

3.0f (9-12-96)

3.0e (7-12-96)

3.0d

3.0a

2.7q

2.7p

2.7o (12-Nov-96)

2.7n

2.7k

2.7j

2.7i

2.7h

2.7g

2.7f

2.7e

2.7d

2.7c

2.7b

2.7a

2.6q

2.6o

2.6n

2.6m

2.6k

2.6j

2.6i

2.6h

2.6g

2.6f

2.6e

2.6d (28-Aug-1996)

2.6c (27-Aug-1996)

2.6b (26-Aug-1996) 

suck_stream true

This can triple the sucking speed or better on some systems.

2.6a (Beta release, 25-Aug-96)

Summary of major new featres since 2.5i

2.5l (20-August-1996)

2.5k

2.5j

2.5i

2.5h

2.5g

2.5f

2.5e

2.5d

2.5c

2.5b

2.5a

2.4q

2.4p

2.4o

2.4n

2.4m (I don't like the letter l)

2.4k (Beta)

2.4j (Beta 19/June/1996)

2.4i (Beta 18/June/1996)

2.4h (Beta 17/June/1996)

2.4g (Beta 16/June/1996) 

	xover,xhdr,list,article,body

2.4f (Beta 6/June/1996)

2.4e (5/June/1996)

2.4d (19-5-96)

2.4c

2.4b

2.4a 

nntp_feeder main.feeder.site *,!microsoft.*
nntp_suck2 alternate.feeder !*,alt.*,!microsoft.*
nntp_suck3 msnews.microsoft.com microsoft.*

update_at *5 * # Get main groups every 10 minutes
update_at2 20 3 # Get missing articles once a day
update_at3 40 9,12 # Get Micrsoft articles at 9:40 and 12:40

site me
groups *
site $nntp_feeder
groups *,!microsoft.*
builtin
posting
site msnews.microsoft.com
groups !*,microsoft.*
builtin
posting
...

Please don't update every 10 minutes from microsoft, as they could easily get over loaded :-)

2.3p 

replicate true
postonly true

site $NNTP_FEEDER
	groups *
builtin
posting

2.3o (12/5/96)

2.3n (5/5/96)

2.3m (3/5/96)

2.3l (19/4/96)

2.3k

2.3j

2.3i 

	nntp_slow true
	chan_nobuff true

tellnews stats_in, Shows stats in incoming feeds
tellnews stats_groups, Shows stats on group usage/size

2.3h

2.3g

2.3f

2.3e (21/Feb/96)

2.3d (11/Feb/96)

2.3c (7/Feb/96)

2.3b

2.3a

2.2g

2.2f (30/1/96)

2.2e

2.2d (20-Jan-1996)

2.2c (19-Jan-1996)

2.2a (15-Jan-1996)

2.1v (13-Jan-1996)

2.1u (12-Jan-1996)

2.1t (5-Jan-1996)

2.1s (3-Jan-1996)

2.1r

2.1q

2.1p

2.1o

2.1m (3/12/95)

2.1k

2.1j

2.1i (20/11/95)

2.1h

2.1g

2.1f (internal release mainly)

2.1e (4/11/1995)

2.1d (4/11/1995)

2.1c (2/11/1995)

2.1b (1/11/1995)

2.1a (31/10/1995)

2.0 Beta-5

2.0 Beta-4

2.0 Beta-3 (26-Oct-1995)

2.0 Beta-2 (24-Oct-1995)

2.0 Beta-1 (22-Oct-1995)

1.1g (30-Sep-1995) 

        tellnews purgelost


        tellnews setused nnn

1.1f (27-Sep-1995)

1.1e (26-Sep-1995)

1.1d (20-Sep-1995)

1.1c (14-Sep-1995)

1.1b (Internal only)

1.1a (6-Sep-1995)

1.0e (Internal version)

1.0d (Internal version)

1.0c (22-Aug-1995)

1.0b (20-Aug-1995)

1.0a (14-Aug-1995) 

        check <messageID>
        takethis <messageid>
        mode stream

0.9d (Wide BETA release 2)

0.9c (Wide BETA release)

0.9b