DNews Updates List
Fixed non thread safe time functions used on solaris.
Minor fix to filter for detecting attachment file names.
Fixed bug int fts responses resulting in zero responses for common words.
Minor changes to key handling and temp key generation
Change to ml limit error messages to include ip address in response.
Possible fix for header sucking problem with large news groups.
Fixed problem with pull feeds skipping new items.
Added setting 'access_applyip true' which makes access limits apply correctly to ip as opposed to user logins.
Fix with large groups causing looping index errors.
Fixed problem with every 2000th xover response missing.
Improved part performance and allowed multpile part directories. You should have multiple scsi drives for the parts system to give best performance, ideally 3 drives. See part.htm for more details.
Improved performance of "part_thread true" Note: we recommend you use a very very fast SCSI disk for the part completion buffer if you are trying to take a full news feed.
Fixed 'tellnews stats' output, the 'main mutex' figure is now a reliable measure of load, if it's above 95% then you probably have a load problem.
Some systems have lost or have incorrect 'moderated' flags, to correct this on your system install this versoin then if your upstream server is incorrect (and you are using a sucking feed) set:fixmod_disable true
This will require 5.6e5, currently available on our site in the beta directory,
Second to fix all groups download the latest active file from ftp://ftp.isc.org/pub/usenet/CONFIG and copy it to the dnews directory as active.fix and then type in
copy active \dnews\active.fix
tellnews matchmod active.fix
This will correct all moderated flags quickly and simply.
5.6e3 (1 Dec 2002)
Fixed memory leak introduced in 5.6e1 :-)
5.6e1 (25 November 2002)
Made numerous minor performance improvements.
Added the following settings:
Added two settings for tuning heavy use systems,small_fast 20000000
These are used to give higher priority response to interactive users, basically while a channel exceeds both these limits (20mb total downloaded, and 30K bytes per second for the last 10 seconds) the channel is given lower priority. On an unloaded system there should be no affect, but on a heavily loaded system interactive users reading a few items etc will see better response.
New settings that haven't been documented before:
hist_max 1000000000 - This setting tells dnews to trim the history file to this size, 1 gig, this prevents a failure that would otherwise occur if the history file grew too large. The bad effect of this is that the newnews command will not work for 'trimmed' entries and neither will 'messageid' based lookups.
xover_xhash true - This switches the xover handling to an alternate file format, which is more immune to faults, and doesn't require resorting etc if something goes wrong. It also deals better with backward searches (e.g. some dumb clients request xover data in reverse order, one line at a time, and this new format deals with that better) Lastly it's good if you are using a replicating feed
Fixed fault with hist_max setting which would result in the news also being trimmed in some situations instead of just the history.
Fixed serious performance fault with xover_xhash true setting, anyone using that setting should upgrade to this version.
Fixed bug with tellnews activate command when giving response to send via email, the fault would result in an incorrect error from the key robot suggesting updates had expired.
Fixed fault in thread input module, this will improve performance of any server accepting feeds from multiple sites.
Fixed serious fault in rebuild_piles which was making it 'renumber' items.
Changed to new license activation system, see http://netwinsite.com/activate.htm Be sure you have your registration number and email address before installing, you will need to activate your license and your old key will not work!!!
5.5h2 - Security fix release
Security release - highly recommended
- If you only want to use a rule file to process outgoing posts (local posts) then you can set in dnews.conf rules_local_only true, this is more efficient than using actual rules inside the rule file as it saves pre processing required by the rules.
- Fixed minor fault in path handling, the last field should never be the host name as it is reserved for the username of the posting user or a place holder, now DNews adds "!not-for-mail" in most cases as the last entry for local posts if a path is not given by the client.
- Fixed history leak that probably exists in all 5.5c* and later builds, if you are running a 5.5 version of DNews then we recommend upgrading, and if your history file is larger than expected you can shrink it with the command tellnews repair_history, however this command will stop incoming posts while it repairs the history and may take several hours, if your history size is not prohibitively large then you can just wait and it will repair over time by itself.
- Added output threads for DFeed, only use where the outgoing feed is the main feed to the destination system (low duplicate count) to enable add 'sendthread' to the newsfeeds.conf entry, e.g.site outging.feed.name groups * type DFeed sendthread
- An extra line will show up in the tellnews stats_out figures showing how much is being sent directly and how much is being left for the DFeed module to send.
- Fixed bug in history processing (introduced in 5.5d2) which caused the history.dat to grow. If you are on 5.5d2 or later then upgrade to this one, and you may need to do a rebuild_index although it should self repair unless you history file is already too large for it to process.
- Security patch (relatively minor but we do recommend upgrading when convenient both DNews and dnewsweb)
- History processing changes.
- post_cancel_owner true, allows cancel messages only from the matching sender or any authenticated users.
- New dnews.conf setting to limit tellnews connections to only local users, e.g. tellnews_pass_mask 127.0.0.1,10.0.0.* (always include 127.0.0.1 in this mask)
- Generally the setting dmgrsvc_access should also be defined as the same list of addresses.
- Other minor security enhancements
- Minor changes.
- Fixed bug with solaris thread_in true setting, it should now be reliable.
- Fixed unjam problem when all buckets are used up.
5.5c2 - 22 June 2001
- Fixed nasty bug in index sorting which could result in items going missing after expires. Anyone running dnews5.5a6 or later should upgrade and consider running a 'tellnews rebuild_index' if you think you may be missing some items.
- Made minor modifications to history processing.
- Fixed some problems sucking from news servers that give invalid responses.
- Added setting post_spawn, this can point to a program DNews will run for all incoming local posts, the post will be stored in a file, and your filter program can write the post to a new file if it does so DNews will use the modified file, the program is passed two parameters which are the full paths to the input and output files. The files should be read/written in binary mode and crlf pairs must be maintained (so don't convert the file to 'lf' terminated text on unix) Also the dot stuffing (lines starting with a dot have a dot added) must be maintained.
- Added setting widearea_host xxxx:7500 This will make DNews consult that central system to see if a user is logging into more than one system at a time. Be sure to set your tellnews.pass to the same string on both (all) systems, and define that setting even on the host 'xxxx' itself.
5.5b5 release build
- Added spawn_filter_max settings, limits the size of items sent to an external filter, which can significantly improve performance if you use an external spam filter.
- Added setting real_max which can be used to increase item size limits above 10mb
- Modified posting success email for moderated posts to correctly say success.
5.5b3 Release build
- Made threads get reused on unix which seems to avoid a bug in the posix thread library.
- Changes to mb=x limits behavior in access.conf
If limit_users FALSE, then limit applies to ip address
if limit_users true, then limit applies to entire area or logged in user
If limit_users false, then limit applies to ip address or logged in user
if limit_users true then limit always applies to entire area or logged in user
- Fixed last bug in thread_in for unix platforms :-)
- Added setting post_cancel_authorized true, which only allows cancels from authenticated users.
- Improved performance of group index caching.
- Made XOVER_EXPIRE_NORMAL true the default, this improves the quality of the xover indexes when spawn_expire true is used, and that improves performance.
5.5a7 Fourth release candidate 13/2/2001
- Fixed glitch in new users.dat code, it wasn't detecting changes to users.dat from dmulti slaves.
5.5a6 Third release candidate :-) 12/2/2001
- Fixed fault in dadmin regarding sucking feeds changing the return feed to the wrong type
- Changed default for reindex_stack to true (speeds up expires on large systems)
- Significantly improved performance of users.dat lookups.
- Fixed fault introduced in 5.5a5 with regard to auto detecting lost buckets which could rarely cause news to dissappear if spool files were not accessable on startup.
5.5a5 Second release candidate :-) 5/2/2001
- Fixed rare dadmin crash on installation
- Fixed default bucket size so it doesn't need increasing manually in most cases
- Fixed fault in tellnews undelete which would clear the 'moderated' flag on moderated groups.
- Fixed dadmin problem with setting the feed type for a sucking feed to posting
5.5a4 Release version 29/1/2001
- Release build.
- Added more valid settings to domain.conf, e.g.
org Freddies Fries Ltd
welcome Freddies Fries Server
post_addhead1 XInfo: Send problems to email@example.com"
- if (attach("*.exe,*.com,*.vbs")) reject "Message contains executable programs"
accept "Normal messages"
- Fixed bug with expire not releasing the history file on NT (would cause expire to fail with dmulti or readonly servers intermittently)
- Fixed bug with thread_in related to 'ihave' protocol, symptoms would include a slow feed or many ihave channels open and no data being sent. In general we recommend using streaming/takethis protocol as it is more efficient than ihave protocol.
- Fixed bug with thread_timeout on solaris which could cause the feed to block.
- Fixed bug in thread_in related to empty messages which could block feed.
- Made thread_timeout true the default, this can prevent memory loss in some cases
- Made thread_in true the default on platforms that support it.
- Fixed bug with nntp 'last' command not showing first item in group.
- Fixed memory leak related to usergroups.
- Fixed security fault with blank password entries in users.dat
- Fixed bugs in thread_in true
- NOTE: On FREEBSD 3.1 threading breaks expires and DFeed, use the nothreads build or a later version of freebsd.
- Added listen_extra host:port which makes DNews listen on an extra host and port (dnews not dmulti)
- Added master_address host:port which tells a DNews slave where to find it's master, it defaults to 127.0.0.1:7500 or bind_in:7500
- Fixed fault with auto getgroups on empty system.
- Fixed some faults in multi system readonly processing which could cause GID errors on slaves and groups to take a long time to appear on readonly servers.
- Reduced min disk space requirements during expiration.
- Made dmulti inter DNews connections use 127.0.0.1 instead of myname
- Made dmulti not restart DNews master if it crashes, (as this could hide serious configuration faults and make finding faults difficult or impossible)
- Fixed thread_in fault on freebsd (fault caused by stack fault in pthread library on freebsd)
- Fixed fault in PART processing (bug added in 5.4f10)
- Made spam_post_local_ok true work for all spam rules not just filter
- Added uucp_timer 800 setting to uucp feeds, this is 800 by default and flushes the uucp bag files from '.new' to '.bag' even if the other limits are not reached.
- Fix for SOLARIS lockup problem with 5.4f9, basically on solaris the threading libraries are buggy and the usleep() function intermittently hangs, Sun seem to think this is not a bug so haven't bothered to fix it!!, this version of DNews avoids that call and should work fine.
- Don't use 'thread_in true' on freebsd, currently this makes DNews die on this platform, we are still investigating to establish why and fix it. (FIXED IN 5.4G1 ABOVE)
- Fixed DFeed crash.
- Fixed some minor problems in DNewsWeb with errors producing malformed headers.
5.4f7 - Release 14 August 2000
- Fixed minor bugs, thread_in true should still be considered a beta feature.
5.4f3 - Release Candidate 17 July 2000
- Fixed getmail processing to accept 'todots' as well as 'todot'
- Fixed getmail processing of multi line headers
- Increased default multi part buffer index size.
We recommend the use of these settings for high performance incoming newsfeeds, however these settings should be considered 'beta' features. If you use them and have problems try removing them, and report problems to firstname.lastname@example.org. Also see the notes below on thread testing your build of dnews.
5.4f2 - Beta 14 July 2000
Fixed performance problem in part processing. If you use part processing options you should upgrade to this version.
5.4f1 -Beta 10 July 2000
Fixed bugs with thread_in true. If you ran the previously buggin version with thread_in true defined then you should also define thread_fix true in dnews.conf
Before using the setting 'thread_in true' on any unix system you should first test your thread library is reliable, to do this type in:
./dnews -test_threads 50
You should see the following output:
./dnews -test_threads 50
Successfully created 50 threads
(then 10 -30 seconds later)
Test 0 finished
Test 1 finished
Test 49 finished
Completed 50/50 tests
If you see something else then contact us and don't set thread_in true.
On LINUX when using THREAD_IN TRUE, you may notice multiple instances of DNews running this is ok it's the way linux handles threads, e.g.11203 ? D 15:38 /usr/lib/news/dnews/dnews -s 0 11238 ? S 0:00 /usr/lib/news/dnews/dnews -s 0 11239 ? S 199:18 /usr/lib/news/dnews/dnews -s 0
5.4e3 - Experimental beta - buggy do not use, stick with 5.4d4
Added threading code for incoming newsfeeds, enable with the dnews.conf setting:thread_in true
This seems to significantly help performance on NT, this feature may not be compiled into all DNews builds, it is currently in NT and Linux builds. If the setting is 'active & working' then tellnews status will show how many threads are running. In this build this setting is buggy we recommend you don't use it until it is recorded as fixed above in a later version.
Added option to part buffer, part_fast true , never use this setting, it will be removed in future. (It has a fundamental design flaw which makes it less than useful)
Fixed crashing bug in spam filter rules, introduced in 5.4
Fixes pop lookup bug, which would cause intermittent failures on pop lookups particularly on busy systems.
DNews 5.4d1 31/5/2000
- Fixed bugs in tomail and DNewsWeb 'tomail' option.
- Several security fixes. (No, more details are not available, however we do recommend upgrading to this version as early as is convenient)
- Fixed bugs in nocem processing.
- Added setting 'limit_users' this applies the access.conf mb/day download limits to user instead of ip addresses.
- Fixed bug in xhdr corrupt responses on busy systems with large numbers of items in a news group
- Fixed fault with nocem processing.
- Added authinfo processing after mode reader command is sent when sucking news.
- Added two settings
suck_noauth1 true # Disables sending of first authinfo...
suck_noauth2 true # Disables sending of second authinfo (after mode reader command)
DNews 5.4c1 27 - April - 2000
- Security patch for DNewsWeb
- Compression (Only NT and Linux builds see details below)
- Multipart binary file completion system: http://netwinsite.com/dnews/part.htm
- New email gatewaying system: http://netwinsite.com/dnews/tomail.htm
Fixed loop in DNews when processing parts.
If a file called spammers.dat exists in config directory that lists spammers one line per user, if authenticated user is found in this file, then the post is rejected but the user is told it was accepted.
- Fixes loop in DNewsWeb when no response from server.
Added support for incoming compressed feeds to DNews
Added support for outgoing compressed feeds to DFeed
Compressed news feeds
In theory these are compatible with NNTPrelay, however due to bugs in nntprelay we don't recommend you try and send a compressed feed to nntprelay.
Compression is generally NOT recommended!!! see below!!!
To send a compressed feed add 'compress' to a 'DFEED' outgoing feed.
Add to newsfeeds.conf
To disable incoming compressed feeds add to dnews.conf
The statistics on compression ratio's are shown on
The % of cpu used for compression is also shown in stats_out in DFeed. Currently the stats_in figure is 'transitory' rather than total.
When to use compression
Sending out a 800K/s feed will use up about 100Mhz of Pentium, so a 200mhz pentium might just about send two compressed news feeds.
Compression with binary groups will be about a 20% bandwidth saving ,with non binary groups about 65%.
So you could/should use compression if (and only if):
- You are sending 1 or two feeds over a medium speed/slow line
- And they are more text than binary in nature or best all text.
- And you have extra processor resources at the sending end
- And the line is not already compressed in hardware
- In most other cases compression is a bad idea, don't do it! :-)
Note: compression is only currently supported on the NT and Linux builds.
Multipart binary file completion system: http://netwinsite.com/dnews/part.htm
New email gatewaying system: http://netwinsite.com/dnews/tomail.htm
DNews 5.3e3, 16 Feb 2000 (release build)
Changed tcp buffering and idle loop to improve performance.
Fixed feeding problem with frist group in active.dat file.
Fixed spawn_expire bugs
Fixed bug in nt crashing when dns lookups returns bad structure. (this seems to occur occasionally on NT with service patch 5 or greater)
5.3d5 (release build)
Fixed bug with 4 digit dates in newnews and newgroups commands.
Fixed bug in xover with long xreferences headers, introduced in 5.3d
Fixed DNewsWeb web_login true, it now works much better and doesn't require
require_login true to be set.
Fixed bug with extra blank lines being inserted in messages, bug introduced
in version 5.3c2
Fixed crash when processing mal formed pgp control message.
Fixed problem with moderated posts on readonly slaves going to wrong moderator
Added to fts, index_numbers true, for example if set then q2343 will be indexed
separately from q323
Added tellnews matchsite site.name groups_wildcard. This is a relative of getgroups, it will fetch the list of groups from any accessable news server, and then add/remove groups to your server to make your server match that serer, the groups_wildcard is required, it is used to establish which groups should be added or deleted, e.g.
tellnews matchsite msnews.microsoft.com microsoft.*
tellnews matchsite news.installshield.com installsheild.*
This provides a pain free way of maintaining your list of news groups with several other systems. It does not allow you to merge two over-lapping systems, however a matchsite followed by a getgroups to the second system would more or less achieve this. Deleted groups on your server that exist on the upstream server will be undeleted by this command. To prevent this ban the groups in your ME feed.
Fixed problem with builtin feeds introduced in 5.3c1
Fixed problem with xover-caching detection
Fixed problem with more than 64 channels in dfeed.
Made input handling turn naked 'cr' characters into the correct 'crlf' characters.
5.3 c1 Release Candidate 11/Nov/1999 (5.2 is still the release build)
Summary of New features since previous released version (5.2):
- Major revision of online documentation to remove obsolete information and make information easier to find etc...
- Significantly faster performance for article and xover commands (giving approx 200% performance gain in some circumstances)
- Concurrent pull mechanism for sucking from several sites at once using multiple channels.
- Increased limits on concurrent users on some unix platforms (solaris in particular)
- Made user and concurrent limits work correctly across dmulti configuration, added daily user limits
- Made limit on items in group configurable and improved performance (was limited to 90,000 previously)
- Fixed problems with spawn_expire
- Fixed xover indexing problems which could sometimes cause poor performance
- High speed Multicast (satellite/broadcast) sending and receiving built in.
- Local spam attempts (posts) are emailed to a manager automatically (spam_noemail true) so local spammers can be removed before they succeed in causing a real problem.
- A mechanism for sorting database data to improve retrieval speed, ideal for 'reader' systems and text news groups.
- Minor bug fixes and features as listed below.
- Simple virtual domain support for hosting other ISP's users (see domain.conf below)
- Added advanced search options to DNewsWeb full text searching (newsgroup wild cards, dates etc)
5.3 beta 11
Fixed problem with auth_spawn external authentication module being sent the ip address as the third parameter intermittently, it is now sent three parameters all the time.
DNewsWeb - now deals with multi part uuencoded files (this is cool IMHO :-)
DNewsWeb - Fixed followup, it was not wrapping lines, (bug introduced in 5.2 probably)
DNewsWeb - added next and last in thread buttons.
Added dmulti wide accounting module, you can disable this with the dnews.conf setting ml_disable true, when enabled you can see the internal data with the command tellnews ml_show, This makes the settings in access.conf con=n,users=n work across dmulti, in addition there is a new limit, mb=n which limits the MB of data a use can read in one day.
Also the local host name and 127.0.0.1 are ignored so if you want to test this don't test it from the machine DNews is running on.
Added dnews.conf setting auth_spaces true which allows spaces in usernames passwords, passed to external auth module, use this at your own risk it seems a bit weird to me :-)
Added dnews.conf setting auth_readart true, if set then DNews will call an additional function as each item is read, a false return will cause the user to be disconnected. To implement setup auth_lib as described on access.htm and in the same dll include a function like this:
__declspec( dllexport ) int auth_readart(int chan, int type, char *ipname, char *ipnumber,
char *username, char *password,char *group,int article,int size,char *reason)
If return is false the use will be logged of after being told the reason 'reason'. We don't recommend using this mechanism.
Added to spam filter, config setting in dnews.conf spam_subject_hits nnn, default is 20, this combines subject and 'from' both trimmed in various ways, then rejects more than 'nnn' duplicates, this can help stop simple spammers. (spam_stop true must be set for this setting to take effect)
Dnews.conf setting, post_from_adduser true, if set then the from field in a locally posted message is replaced with "authenticated user" <email@doman>
Dnews.conf setting, post_force_date true, if true the date header on local posts is replaced by DNews, this stops users from dating items into the future to get them at the top of the page.
Enhanced full text search options for DNewsWeb, See search.tpl and results.tpl for the new features. If you wish to implement 'date' based searching then you must add to fts.conf
NOTE when adding this setting you must delete your existing database when changing this setting as it adds a field to the database:
tellnews refeed ftsfeed (optional)
5.3 beta 8
This makes the expire process trim old items from any group with more than this many items, this doesn't save disk space but does prevent 'huge' groups (like control.cancel) from impacting performance, we recommend a setting of 20000 for a typical system. The default is 90000. You can also increase the limit above 90,000 if you need to.
Added header_no_listgroup true setting, this sometimes helps header sucking expire items correctly if the upstream server doesn't support listgroup correctly, frankly if your upstream server is weird, then header sucking is going to be a bit dodgy, use at your own risk.
Fixed problem with cross posts to non existent groups on upstream site when header sucking causing floating 'dead' items to appear.
Added spam emailing to the manager for local posts turn off with spam_noemail true.
5.3 beta 6
Concurrent sucking mechanism added, this module is called the 'pull' module see pull.htm
Multicast sending and receiving udp feeds http://netwinsite.com/dnews/dudp.htm
- Fixed checkgroups processing so the group mask now applies to checkgroups messages in the normal way.
- Fixed spwan_expire bug introduced in 5.3 :-) If you are using spawn_expire true either upgrade to 5.3 or stop using that setting!!!
- Fixed xover expiration and increased to 30 days default
- Fixed spawned filters with dmulti
- Fixed rebuild_piles.gids from applying rules, unless rebuild_rules is used.
- Fixed fault with newgroup for nonexistent groups being refused
- Added auth_wild true, if set then "*" is valid in username or password fields in access.conf and means 'accept any username' or 'any password'
- auth_pophost can now take a list of pop servers and users can specify user@domain as login this allows virtual hosting of news services for other isp's without having to register users twice.
- (dnews.conf setting) post_quoted_limit 0.8, The ratio of quoted to unquoted, for example if you wanted to insist that there be at least 70% unquoted lines then you would use post_quoted_limit .7 A reasonable value is probably 0.95 personally I wouldn't use this :-)
- (also in dnews.conf) post_quoted_message You have too much quoted text in that message
- Added simple virtual domain support this is useful when you are hosting news for other ISP's and want to give their users the impression that the system is run by their own ISP. To implement create a file domain.conf in the config directory in the following format, you must have configured your system for multiple IP addresses and setup dns entries for them, then when the user connects to whichever ip address the welcome message and org header on posts will match the name they connected to.
org Freddies Fries Ltd
org Friendly ISP we host
- Added 'sort' as an option for piles, if this keyword is present then DNews will try and sort the buckets as they are closed to improve reader performance (this is sort of like defragmentation for news data) it only really works on non heavily cross posted text groups, and should never be used when sucking news or sending out feeds, the former because it's not needed and the latter because it breaks the feeding mechanisms.
PAM authenticatin support (requires a build with pam in the name, ask us if you can't find one and really need this)
Add to dnews.conf
And in /etc/pam.d/dnews put three lines:
auth required /lib/security/pam_pwdb.so shadow nullok
account required /lib/security/pam_pwdb.so
5.2b2 1 August 1999 (Release version)
Added settings to prevent excessive quoting of local posts. post_quoted_limit 0.9, this is the ratio of quoted to unquoted, for example if you wanted to insist that there be at least 70% unquoted
lines then you would use post_quoted_limit .7 A reasonable value is probably 0.95
personally I wouldn't use this :-), Also added, post_quoted_message You have too much quoted text in that message
Fixed bug in DFeed spawning in DNews on NT
Fixed unique count in showcon, and stats output.
Changed to NetWin regular expression parser to fix bug, this also improved performance.
Fixed the DNews zombies. (apparently not completely!)
Fixed memory leak in fts searching
Fixed score display on fts searching in DNewsWeb
Fixed bug with dmulti generating identical id's if two posts from different slaves in same second.
5.2a3 12 July 1999 (5.2 release candidate)
Summary of New features since previous released version (5.0):
- Expires can be run as a separate process (spawn_expire true)
- Built in Radius authentication
- Numerous Performance inhancements
- DFeed feeding module for sending out high speed news feeds
- New filter rule system for implementing 'cleanfeed like' spam protection
- Support for external spam protection scripts (like cleanfeed)
- Built in support for nocem (spam/cancelling system)
- Many minor bug fixes and improvements
(all of the above are documented in more detail below)
Fixed occasional pause in DFeed feeds.
Allowed DNewsWeb auto login from url, this is not recommended as it is not terribly secure :-)
Decreased DNews disk fragmentation and improved performance slightly. Please be aware that for best performance with DNews it's worth formatting your 'dnews' disk drives with large blocking factors 32K or 64K this will prevent fragmentation. (Note re-formatting your drives will loose all existing data!!! :-)
Added external expires, to implement this just set in dnews.conf spawn_expire true, then all going well DNews will spawn a copy of DNews to handle the expire, index resort and xover expires. Currently this causes a degradation of xover performance particularly on dmulti systems, so we recommend it not be used except for 'feeding only' systems until 5.2c is available which will fix this problem.
Fixed bug in web manager which broke passwords when user settings modified.
Fixed bug with replicate true and header_groups which was making DNews ignore the replicate true setting for non header_groups whan using both settings together.
Added built in Radius authentication, dnews.conf settings:
The secret is the shared secret that you must have defined on your radius server, the attribute is optional it is the attribute number as defined in your /etc/raddb/dictionary file which you want DNews to use as a list of 'usergroups' e.g.
Once you have defined auth_radius_host, and auth_radius_secret then DNews will check the radius host for $lookup$ entries in access.conf, e.g.
Changed spam_dup_hits and spam_dup_cross crc mechanism to ignore the last line of the message help avoid abuse from many common spam programs.
Fixed bug with moderated groups posting with readonly servers
Fixed bug with specific names in me feed, originally introduced in 5.2abeta1.
5.2aBeta4 (Beta Only, use at your own risk, report problems etc...)
Added new tellnews command tellnews cancel that cancels an article from the command line.
Syntax : tellnews cancel "group.name" article_number
tellnews cancel "<message-id>"
On NT changed operation of dmulti to make it more efficient.
Missing binaries, several people have reported missing binary parts of multiple part items, this can be cause by various things (bandwidth problems, items missing upstream, overloaded system upstream or overloaded DNews) however we have tracked down several issues with DNews related to this. See this web page for advice.
5.2aBeta1 (Beta Only, use at your own risk, report problems etc...)
Added feed type DFeed, an external feeding module as a means of offloading feeds from the main DNews server, this is also faster than live feeds so where you are sending out full feeds you should probably consider using this system.
log (debugging option creates workarea) x_y_z.log
dfeed_debug true (optional)
Make sure you have DFeed binary
Then start DNews or issue a tellnews reload, stats are visible in tellnews stats_out display.
Made feed masks build faster. (With 5.1beta8 and later news feeds are constructed as masks, this slows down reloads and startup, but speeds up processing of incoming messages)
Fixed status output for diskuse > 200mb
If set, then for moderated groups DNews will only accept posts if they are from a user who matches the moderators email address. This is intended for use with local moderated groups only.
Added creation of rule_n.tmp files are created when rules reject an item.
Modified behaviour of 'cachedall true' in the past if this was true DNews would try and suck all groups, now it removes the downloading message but still only tries to suck 'cached' groups, if you really want DNews to suck all groups do this:
tellnews cached "*" 10000
Fixed security issue.
Added version of DMail extern authentication module that talks to dnews's 'users.dat' file, http://netwinsite.com/dmail/dnauth.htm
Fixed bug in xmit/fts indexing which caused this error:
07 08:35:16: point write corrupt data 245 4748 groupmax 239
Fixed bug in drobot (mail gateway) which could result in lost lines from messages.
Fixed performance problem in chan_read, Fixed performance problem in db_read
Changed 'feed' group matching to masks, this will mean that @ matching on groups that don't exist won't work, so to restore the same behaviour you would need to create the groups then delete them but not killgroup them or run purgegroups
Added tellnews command: tellnews getnames n , to fetch names from alternate sucking feeds.
5.1aBeta7 (Beta Only, use at your own risk, report problems etc...)
Fixed fault in live feeds to significantly improve performance particularly when sending binaries. When sending a full feed with binaries you probably need to adjust some config settings described in recent.htm
Fixed out_size setting so it works with larger values. (up to 100K)
Fixed list newsgroups so it only shows groups the user has read access to unless auth_showprot true is set
Rebuilt linux libc6 version without profiling linked in, this was causing it to completely fail on some versions of linux for some reason.
Fixed pageup/down buttons in DNewsWeb when gaps in group.
5.1aBeta6 26 March 1999, (Beta Only, use at your own risk, report problems etc...)
Support for new filter rule system, this is an efficient and flexible system intended to give the ability to make complex filtering for incoming and outgoing feeds similar to cleanfeed etc without the need for using an external filter, this system includes multi line logic operations and regular expression matching, see rules.htm for details.
Support for adding external filters like 'cleanfeed', see filters.htm, this is provided so those who want to make extensive adjustments to the filtering can do so.
Support for NoCem added (an automated system for cancelling spam but more efficient than cancel messages and with pgp control over who can cancel) see nocem.htm
Added support for sendsys control messages, as DNews doesn't have a sys file it generates one from the newsfeeds.conf information.
Fixed dadmin fault where it couldn't find users with no usergroups defined.
Added to newsfeeds.conf to qualifier to specify a destination different from the 'site' name, this allows multiple sites to send to the same destination. (Site names must be unique in newsfeeds.conf), also added 'bind' option to allow specific network cards to be used. e.g.
Added to access.conf area qualifier, this allows concurrent user limits to apply to multiple lines in access.conf, note these rules apply to IP addresses, not to username/login rules.
If the workarea contains files called "upstream.bad" or "upstream.ok" then those files will replace the text that is sent to a user in the confirmation message when a news item is sent upstream. e.g. for upstream.bad
Dear user, your recent usenet post was NOT sent upstream
The status returned from the upstream site was:
Fixed fault with dmgrsvc on unix dieing when connections broken.
Added setting uucp_send_mod true, which if set allows incoming messages for moderated groups via uucp to be sent to the moderator if they are not approved. This should be used if you have a down stream site like a bbs which is sendign you stuff for moderated groups, this is unusual :-).
Added delay xxx seconds to newsfeeds.conf, this is only valid for LIVE feeds, the actual delay will often be up to twice 'xxx' seconds but will never be less than 'xxx' seconds. e.g.
Fixed command tellnews backup_groups x.y.* so it actually works and backs up the correct groups, this command can be useful to create a backup of local newsgroups. Do not use this command prior to 5.1 or bad things will happen :-)
Added support for CIDR syntax in access.conf
Fixed dadmin bug, crash if smtp partially defined on menu
Fixed dadmin shutting down services reliably during installation
Fixed log rotation if daily rotate used with dmultil
Added link to dadmin gui on start menu for 95/98 installation.
5.0g, 29 Jan 1999
Fixed cpu loop with history file greater than 1.2gig
Added usergroups administration to 5.0 NT/95 gui
Added warning on access.conf converstion to NT/95 gui
Several DNewsWeb bug fixes.
Fixed DNewsWeb 'new' article count so it isn't reset on login.
5.0f, 29 Jan 1999
Fixed DNewsWeb 'new' article count so it isn't reset on login.
Made default behaviour to spread news over all disks evenly. Can be set to not spread by using pile_separate true in dnews.conf
5.0c, 17 Jan 1999
Fixed serious bug with Header Caching which can result in all users getting the same 'body'. We strongly recommend upgrading if you are using header caching.
Fixed bug where some files are written to pile 0 incorrectly when using new piles system.
Added tellnews pile_move n m Where 'n' is the source pile and 'm' is the distination pile, e.g. tellnews pile_move 0 1 Can be used to move the stray items that have gotten into pile 0 into pile 1 where they will eventually be expired. This could be used instead of converting the database when changing to pile based expiration but only if you didn't care when existing items expired as they would all end up in pile 0 regardless of which pile they should be in.
5.0b, 7 Jan 1999
Made users.dat case insensitive unless auth_case is 'true' however, encrypted passwords in users.dat are still case sensitive.
Modification to DNews slaves so that they re-read diskuse.dat intermittently. (fixes minor dmulti problem)
Fixed bug that could cause crash if users.dat didn't exist.
Made some minor additions to output when db files can't be written.
Removed trailing tabs from xover data, this may stop Outlook express from crashing (so often :-) if you think this is a problem on your system, you may want to delete your existing *.xov files after upgrading but this will slow your system down for the next 24 hours.
Made nt_domain accept a list of domains, users can specify the domain they want as user=domain/username the first entry in the list is the default.
Fixed bug which allowed a local user to crash the server by typing in one particular unusual header which will remain nameless for obvious reasons.
5.0 (Beta 5)
Due to an imposter sending control messages you may find many of your groups are set incorrectly as moderated, if this has happenned then the group will not be accepting news items, the showmod and fixmod commands can detect this and partically correct it.
Showmod will show you all moderated groups that have recieved un-Approved posts since startup, fixmod n will set groups with more than 'n' unmoderted posts to 'unmoderated' this may result in some moderated groups becoming un moderated which is also not desirable.
The best way to correct your list of active groups is to download an active file from ftp://ftp.isc.org/pub/usenet/CONFIG and copy it to the workarea as active.fix and then type in these command:tellnews matchfile active.fix tellnews undelete "local.*" (to undelete any local or strange hierachies)
- We strongly recommend the file control.conf be modified to not automatically process newgroup messages for the main heirachies (search for isc.org and change the action from 'doit' to 'log') Or alternatively implent PGP authentication http://netwinsite.com/dnews/pgp.htm .
- Note: If you run multiple news servers, the mechanism in 4.7 which attempts to slow down spammers could detect one of your servers as a possible spammer and start slowing down posts, to fix this increase spam_npost in dnews.conf on the master system. Here's the log line to look for:06 14:01:22 19:error: spam: Detected local spammer, 33 posts from (18.104.22.168) using delay of 169 seconds (increase spam_npost to disable this check)
- The default for spam_npost has been increased to 300 to prevent it affecting systems accidentally, on most leaf systems I would recommend reducing this to 50 or possibly less to protect against spammers.
5.0 (Beta 2)
- Improved performance, particularly of handling of incoming news feeds with large items
- Fixed bugs with header sucking and cross posts to non header sucking groups.
- Added optional cyclic file system, see this page for details
- For NT added new GUI with full remote control ability.
- Security additions, these settings apply to LOCAL posts only they are intended to prevent abuse by spammers most of these features are not used by 'normal' users so this shouldn't affect most systems. If you don't know what these things are, don't worry, they are probably best left disabled. They are shown below with the default setting.
- post_supersedes false
- post_cancel true
- post_rmgroup false
- post_newgroup false
- post_sendsys false
- post_version false
- By default local cross posts to non existent groups are now bounced, to restore the old behaviour set in dnews.conf post_nongroups true
- Added extra default header on posting feeds X-Original-NNTP-Posting-Host, It can be turned off with this setting in dnews.conf post_no_original_host true
- Added dnews.conf setting auth_spawn, if defined this should point to a program (with parameters) that should be run to do user lookups, the program should respond to the two commands 'check' and 'exit' send on stdin, and respond on stdout, it should only exit when told to. Here is an example of testing such a program interactively.c:> nwuath check fred fredspassword ip.address.number +OK fred (possibly other info) check jack jackspassword ip.address.number -ERR jack unknown user exit +OK exiting
Two examples are available, one is NWAuth which uses a text file as a user database and comes with DMail, the other is LDAPAuth (for doing ldap authentication) which is a self extracting windows archive . Both come with full source.
Please note: the ip.address.number is not always sent prior to DNews 5.3 beta 11
- Fixed bug with items being lost if they arrive at a particular point during a resort_index
- Fixed bug with header sucking from password protected server.
Note: There is a problem with email to moderated news groups being bounced if the user specifies an invalid from address (like email@example.com) if the moderators mail server is set to bounce email from undefined addresses then this email never gets to him/her, and the bounce of course also fails. In dnews.conf you can define mail_bounce firstname.lastname@example.org which partly fixes this problem except that real bounces are then lost. It's a loose loose problem :-) (this is not related to 4.7 but is a general usenet problem)
4.7f 29 - Sept - 1998
- Fixed occasional crashing bug with header sucking if header_prefetch is also used and the remote end disconnects unexpectedly.
- Fixed problem with crossposts between header cached and non header cached groups
- Fixed delay / lost local posts on readonly servers.
4.7d 17 - Sept 1998
- Fixed problem with long message id's being truncated in the rejection message (this could confuse the sending system)
- Fixed rare problem with resort_index going into an endless loop after an expire.
- Fixed problem with IE and the web manager interface which resulted in connection reset errors with some versions of IE.
- Fixed bugs with header sucking and misc minor template faults with the web manager and new version of DNewsWeb
- Fixed ifgroup in DNewsWeb templates
- Added log_ignore xxx:,Refused,yyyy Allows you to remove dnews.log messages that are of no interest to you.
- Added tellnews copy_item group.name item.number destination.group
4.7 Beta 1, 1-Aug-1998, Feature List
- DNewsWeb with user authentication and subscribed news groups and new templates, authentication can be via a local pop host, or the users pop host, or via the news server, and can be required for posting.
- Header only news group sucking, with live fetch of article bodies (intended for use with binary news groups)
- Spam blocker for local spammers which limits incoming posts from a single user.
- Immediate/Live sucking to make DNews act as an invisible NNTP cache.
- Tellnews commands for copying items and threads between local groups
- Simplified GUP installation (and GUP for NT)
- Added tellnews command: getgroups site.name which will work on non sucking systems for adding news groups from your upstream site.
- Many minor bug fixes and enhancements as noted below
- NOTE: Moderators.conf needs updating, please change the entry in moderators.conf from: *:%email@example.com to *:%firstname.lastname@example.org
This allows you to configure DNews to suck certain groups in 'header only' mode, in this mode DNews only sucks the header of the items and then fetches the body of the message when the item is read, a cache is used so if another user reads the same body it is not requested again.
Option in dnews.conf to block some log lines, log_ignore slow:,Refused,...
tellnews copy_item rec.humor 2 local.funny
Active changes made more efficient by writing to active.pat change file
Added header stuff.
Added to dnews.conf
If a single ip address posts more than this many posts
in 30 minutes, then each following post will be delayed
by n^2 seconds, this effectively limits users to 30 posts
per half hour, This is TURNED ON BY DEFUALT, so to disable
it set spam_npost 1000000
This setting invisibly stops users sending in large amounts
of spam into the news system via your server. Unlike the
other spam rules, this only affects local posts, it has
no effect on incoming feeds.
New Tellnews commands
These are intended for moving threads between local news groups, the matching is case insensitive, and it is not possible to move an item back to the same group, or more than once to a destination group.
tellnews delete_thread group.name "<Message@id>"
tellnews copy_thread group.name.from group.name.to "<Message@id>"
tellnews move_thread group.name.from group.name.to "<Message@id>"
tellnews delete_subject group.name "Text from subject"
tellnews copy_subject group.name.from group.name.to "Text from subject"
tellnews move_subject group.name.from group.name.to "Text from subject"
tellnews copy_item group.name item.number new.group.name
tellnews showgroup group.name (lists the items in a group, subject, mid etc.)
tellnews getgroups any.news.server (now works for IHAVE feeds, usually :-)
Change to newsfeeds.conf, it is now possible to give a file name instead of a list of news groups, e.g.... site freds.news.server groups /usr/local/dnews/freds_groups.dat ...
Addition to newsfeeds.conf, reject header "string" is now permissable, this searches the entire header for the specified string.
Fixed bug with rejected control messages not being added to history correctly
Fixed log errors "IDX corrupt" on slaves processes, also increased speed with which new posts appear on slaves with dmulti.
Fixed core dump from causing cpu loop on unix version
Fixed dbi.idx locking problem on NT.
Increased timeout to solve problem with dnews/dmulti shutting down when a slave won't release dbi.idx
Enhanced dtail (-follow and -page)
Made DNews attempt twice to write to active.new
Made prog_args accept multiple parameters (space separated)
Corrected behaviour of ihave/takethis when faulty MID's are recieived.
history_moved true, evens out disk usage.
Fixed problem with database indexes not being flushed in time for readonly servers to find local posts consistently.
Improved behaviour when a live feed hits a full news server. (it now doesn't retry constantly)
Added support for multiple addresses in dnews.conf email settings (e.g. gup_to)
Turned on xover_write_cache by default (set to 1000), on large systems a setting of 2000 or 3000 may be worthwile.
Removed some logging junk from dmulti slaves
Added commands to clear the stats_in, stats_out statistics.tellnews in_clear tellnews out_clear
Fixed bug with tellnews reload clearing stats_out figures.
Improved dmulti database flushing so that items cannot appear on a slave until they have been flushed to disk by the master.
Adding code to make dnews.log get renamed even if some other process has the file locked.
Fixed problem with dmulti and deleted groups re-appearing occasionally when a slave didn't know the group was deleted.
Added logging options log_onefile true With dmulti makes all logs write to dnews.log (only works on unix)log_no_in true Disables log to dnews.in log_no_out true Disables log to dnews.out
Fixed problem with checkgroups processing for more than one top level tree.
Fixed minor security issue on unix (thats all the detail I'm giving, as ignorance is the next best thing to security)
Added tellnews commands to show internal/database info about an itemtellnews showinfo <articleid> tellnews showinfo group.name item.number
And to list a group (These commands are intended for debugging rather than normal use)tellnews showgroup group.name lists all the items in a group (could take some time)
And to make DNews temporarily forget an item exists, so you can suck it again.tellnews histforget "<articleid>"
Added to Dnews.conf, to turn of spam rules for certain groups:spam_allowgroups local.*,news.spam
Added to dnews.conf timezone_post NZST This is useful if your system timezone is not being picked up correctly by DNews for some reason.
Fixed tellnews copy_group so threading is preserved, but note that the copy_group command does not fully support cross posted articles, it is intended for use with local news groups only.
Fixed bug in auth_pophost
Added new feature, with these settings DNews will send automatic GUP messages to an upstream site:gup_site your.site.name your_gup_password gup_to email@example.com gup_add comp.lang.* gup_mask *,!alt.* gup_at 1 23
That would send an email at 23:01 every night to the email address firstname.lastname@example.org the message would add comp.lang.* to the list of groups to send, and only include cached groups that matched (*,!alt.*)
New tellnews command "test_gup" which generates and sends a gup email, to test gup set the gup_to address to your own email address before testing it.
Fixed fault with newnews command date processing.
Fixed sucking from MCI's new news server. Fixed reject_addhist true fault which could cause items to be rejected incorrectly.
Made xref_always true, the default, to match INN systems, this means an xref header is added even if the message is only posted to a single news group.
Replicate feeds: We've noticed there is a more efficent way to send them, instead of sending an xreplic feed, you can send normal streaming live feeds if you do the following
In dnews.conf on the slave add replicate true
In dnews.conf on the MASTER add xref_always true (that is the default in 4.6f and later)
Then just send a normal live feed from the master to the slave. You still need to use the postonly true setting on the slave to get posts back to the master, and often you need to use ignorepath true so that local posts get sent back to the slave.
In stats_in, moved site name to end of line.
Fixed bug in DNewsWeb with html items on unix.
Added expire_quanta setting, defaults to 3 seconds on dmulti systems, this speeds up expire processing.
Fixed date shown in x-trace to be local time not gmt.
Workaround for freebsd OS bug, with freebsd if DNews writes too fast to too many channels, the operating system gives the error "Out of MBUFS, Page Fault while in Kernal Mode" and reboots, not nice :-(
This is an OS fault but to prevent it we've added a setting to DNews max_out_total 200000 This limits the speed DNews writes to all channels at once it should be set to about what your network card can sustain or slightly less, so on a typical ethernet card 200000 is a good setting. This fix doesn't appear to work on all systems, but the bug also doesn't seem to exist on all systems. Let us know if you have this problem and/or if you get around it with this setting. Also if anyone finds a patch for the kernal to fix it we'd love to know about it.
Fixed xmit -mail problem with some mail servers that give extended response codes to 'helo'
Fixed occasional sending of downloading messages to moderators
Fixed suck_percent so it works if suck_stream is true
Fixed suck_percent so it works if suck_stream is true
Fixed shutdown on unix, it didn't exit cleanly on a system shutdown. (signal 15)
Fixed cache_ongroup which was letting users cache groups that they couldn't read
Bug fixed sucking groups with cachedonly true set.
Added ! (not) to feed rules, e.g. to reject all non smutty messages, add to newsfeeds.confsite me groups * accept default reject subject !"sex"
Improved handling of large bucket sizes (100mb)
DNews 4.6 beta16
Fixed bug in dposter, it would get stuck on nt. Added, to dnews.conf inpaths true, if this is set then DNews will write to the workarea inpaths.dat (it writes to inpaths.new, and once a day renames it to inpaths.dat, you should process and delete this file with a cron job)
DNews 4.6 beta15 26-Feb-1998
Fixed bug in dposter if used by itself with names that don't change it could get stuck on a bad message.
Added to dnews.conf xref_always true, which if set will add xref lines even when the item is not cross posted.
Fixed upgrade procedure for DNews 2.7-4.* conversions.
Fixed bug in spam duplicate body detection, it was not matching some bodies which were identical.
Fixed checkgroups processing, it sometimes created bogus news groups.
Added setting 'retry_441' to dnews.conf, basically some servers respond with 441 when a posting fails and retrying is 'in order' these servers are simply faulty. This setting tells DNews to retry in this case.
Fixed bug in rebuild_index, it was loosing control messages during the rebuild process.
Added to dmulti, if slave_feed is not defined, then dmulti will read the 'ihave' lines in access.conf, it will also do forward name translations. That means you can give a name like feeds.site.name:ihave:::* But you can't give "feeds*.site.name"
To enable forward name translations for DNews so it also recognizes all the feeds.site.name ip addresses add to dnews.conf access_forward true
Added xover write caching, this can help reduce fragmentation and improve teh speed of taking a news feed if you have more than about a thousand groups 'xover cached' (this shows up in the status output). To enable this add to dnews.conf xover_write_cache 2000 This means 2000 xover records are cached before any are written to disk, so that with luck DNews will be able to write more than one xover record at a time.
Added support for virtual domains (not supported in dmulti yet), to turn this on add to dnews.conf auth_virtual_ip true and auth_virtual_user true Then in access.conf add @local.ip.address to your access rules.
E.g. lets assume you have two ethernet cards 22.214.171.124 for local use and 126.96.36.199 for public news groups, then your access.conf would look like this:... *@188.8.131.52:read,post:::* *:read,post:email@example.com:password:* *@184.108.40.206:read,post:::*,!private.*
This lets fred read the private groups even when he connects to 220.127.116.11, you cannot use domain names for the local addresses you must use the number. auth_virtual_ip adds the local ip address to the ip address and auth_virtual_user adds the local ip address to user names.
Removed all 'bad' lockups from expire process
Increased max number of groups from 55000, to 400000, this is controled with the setting max_groups in dnews.conf, if you change this value, don't decrease it, also if you have more than 55K groups you can't downgrade to 4.5, We strongly recommend you don't use this feature, your users don't want to see more than 10K groups, the rest are junk, non local, etc, adding all these groups slows down news reading, but the choice is yours.
Fixed dmulti updating items on readonly machine.
Moved dmulti log file to 'logdir' to avoid conflicts with readonly servers.
Added expire option "split" if this is set and if you set in dnews.conf db_split true Then DNews will shuffle items in groups matching the 'split' lines in expire.conf into different bucket files, the side effect of this should be a significantly faster expire. (note: REMOVED DUE TO BUG, this may be added again at a later time)
Fixed bug with readonly system not finding items on spool'n' where n was not 1.
Change access.conf processing, if access_forward true is set then any name which does not contain wild cards, and has 'ihave' access is 'forward' translated to a number (or list of numbers). This improves security and also allows some forms of aliases to work correctly. We recommend turning this feature on.
To apply similar forward translations to "read,post" sites set in dnews.conf access_forward_all true
If 'tellnews reload' responds very slowly this feature could be the culprit.
Fixed problem with active.dat being locked by readers.
Added to DNewsWeb, any cgi environment variable can now be found using %%e_xxxxx%%, for example to use the authenticated users username in the 'from' address in post.tpl
From: <input name="from" size=20 value="%%E_REMOTE_USERfirstname.lastname@example.org"> (Give EMail address, e.g. email@example.com)<br>
Added to DNewsWeb a setting tellnews_pass xxxxxxxxxxx (find 'xxxx' in the file tellnews.pass in your workarea directory) If this is set then DNews will be able to apply access.conf rules to users reading news via DNewsWeb. If the user is authenticated by the web server then their username will also be matched against users in 'users.dat' and 'usergroup' access can also be applied.
Added X-Trace: header to assist with finding spammers. Fixed bug with 'checkgroups' messing up active.names
Fixed bug with web manager modifying long newsfeeds.conf entries and going into cpu loop.
Added dnews.conf entry auth_pophost your.pop.host If this is defined then users are authenticated via your pop server, usergroups can be added via the DNews user database. A cache is used so the load on the pop server should be minimal, users may occasionally get an error if their drop file is locked due to incoming mail, retrying should fix this.
Removed "Now =..." debugging message in log file.
Fixed error in log files names in 4.6, (dn2.ews.in etc...)
Fixed output of access_test (read,post were reversed)
Fixed problem with any domain being allowed access.
Fixed problem with web manager and MSIE (it would hang)
Added clearer logging of external authentication calls
Message id is now correctly recorded in post.rec
Fixed xmit so it won't email downloading messages to mail lists
Changed spam detection so that both from and lines headers can be used in combination to detect duplicate postings from an individual. This is enabled with the setting: spam_lines true
Added setting to allow posts from certain nntp-posting-hosts to bypass the spam protection spam_allowhosts 18.104.22.168,netwin.*
Added cancel_disable true
If approved_auto true is set in dnews.conf then if a local post is from a local moderator then the approved header is now added automatically (note: also see dmod, a free windows utiilty for moderating a news group, available on our web site)
Added to DNewsWeb templates:
This lets you match any CGI environment variable against a list of names and wild cards. You can thusly change the appearance of your web page depending on where people have recently been etc.
4.6beta-1 This is a beta, do not use it on a live system
Major new features:
- Web based administration tool to simplify administration
- Enhanced and simplified access.conf format which allows 'usergroup' based protection, e.g. you make users members of usergroups like 'adult' and 'child' and then protect newsgroups with these usergroups. (NT user groups are also supported now)
- SPOOL Sharing, this allows you to run a second or third server all sharing your main servers spool disks.
To contact the DNews manager use a web address like this: http://your.host:7119 (this link may work, try it) A username and password will be stored in the file CONFIG/users.dat in encrypted format and on your first connection it will tell you the password, if you forget this you can edit users.dat by hand and add a line like this)dnews:yourpassword:Full:*:Mr Admin:adults
The manager service can be stopped on NT with the command "net stop dmgrsvc" and can be disabled from the control panel/services window.
NOTE: If you use the new manager to modify your access.conf entries it will 'TRY' and convert your access entries to the new format, it may not succeed so you must check it. DNews itself is still backwards compatible so as long as you don't modify your access.conf file with the web manager you will not have any problems. We strongly encourage you to switch to the new format which is much easier to understand, basically it's the same but users are moved into a separate file "users.dat".
Configuring two readonly servers to share the spool disk of an existing DNews server.
- Stop DNews on the main server
- Install DNews on the readonly servers
- Copy your dnews.conf from your main server to your readonly servers
- Add to dnews.conf on the main server readonly_n 2
- Add to dnews.conf on the first readonly server readonly_id 1
- Add to dnews.conf on the second readonly server readonly_id 2
- On the readonly servers change myname to the appropriate ip number
- On the readonly servers change all directory specifications in dnews.conf to the correct network paths, e.g. (spool c:\dnews\spool --> spool h:\dnews\spool) This applies to all settings EXCEPT logdir which must be separate for each readonly server.
- Give the service on the readonly servers a username and password if necessary to access the network drive, this can be done in control panel/settings/services
- If you want to user the web manager then you must give the DNews manager service a username and password as well just as above.
Added a new option to access.conf entries, :add, tells DNews to add the specified groups or subtract them, this allows much simpler configurations, e.g., lets take an example where you have users defined as adults,kids and staff. Depending on which usergroups the user belongs to they should see different selections of newsgroups. For this example your config files would be:----users.dat---- bobby:aaa:None:*::kids,private fred:bbb:None:*::adults joe:ccc:None:*::adults,staff joeskid:ccc:None:*::staff -------------------- ----access.conf---- *:logoff:::* netwin.*:read,post:$lookup$:$lookup$:* *:read,post:,groups=staff::private.*:add *:read,post:,groups=adults::*sex*,*erotic*:add --------------------
With this example if 'bobby' connects he will get access to "*,!private.*,!*sex*,!*erotic*" if fred connects he will get access to "*,!private.*,*sex*,*erotic*" and if joe connects he will get access to "*,private.*,*sex*,*erotic*" and if joeskid connects he will get access to "*,private.*,!*sex*,!*erotic*"
Clear as mud? Trust me it's a big improvement. Access.conf does still allow usernames and passwords directly but this is strongly discouraged.
You can test your setup with this tellnews command:tellnews test_access domain.name.or.number username password
This will show you if a user connecting from that domain with that username and password would be allowed to connect, and if so which groups they could read/post to.
The format of users.dat is currently:
The Manager_access can currently be 'full' which allows access thru the web based management utility or 'none' which doesn't. In future we will add other options, e.g. 'killitem' to allow the use of the killitem command but nothing else so you could grant moderators appropriate access to certain groups.
Passwords defined by DNews manager willl be encrypted, non encrypted passwords can also be entered into this file by hand and will also work. This file should not be set world read on unix.
Misc changes in 4.6
Fixed DNewsWeb for some attachment types
Fixed logging to u_yymmdd.log_n log files for dmulti
Fixed log file names in dmulti for dnews*.in*
Made posts show local timezone when possible.
Fixed problem with groups not matching the wild card defined for a sucking feed being created locally.
Spam rejection is now silent for local posts, this means it lies to the user as sais 'thanks that was fine' rather than informing them that the post was rejected, this change was necessary as otherwise it was too easy for spammers to change what they were posting. The rejection is logged in the log file and shown in the tellnews status output in the 'filter' line.
Added a record of recent hosts rejected to the status output to simplify initial configuration.
On dmulti caching should not work correctly.
When using dmulti the global setting 'concurrent_limit n' in dnews.conf will limit the number of concurrent connections a user is permitted to all slaves.
External authentication dll added, see access.htm
Reduced memory fragmentation
Fixed dmulti problem with connections closing too quickly when data was still bufferred.
To add user defined headers to all local posts define any of these in dnews.confpost_addhead1 XHeaderName1: Text Goes Here post_addhead2 XHeaderName2: Text Goes Here post_addhead3 XHeaderName3: Text Goes HereAdded parameter to stats_groups, e.g. tellnews stats_groups x.y.z*
Removed hard link to /usr/local/dnews/dnews in dmulti
Fixed dmulti so posting to protected news groups works as expected
Fixed dmulti so suck_uucp doesn't kill it
Fixed authentication bug with listgroup with no groupname on protected news groups.
Fixed byte count of articles in xover data which was sometimes faulty, to correct the existing xover data delete *.xov.
Made tellnews setused happen automatically on startup.
Fixed response codes from tellnews which were removed in 4.5i
Fixed rare problem with sucking from fast server with a short list of news groups. Added speed controls in access.conf, e.g. *sprint.net:ihave:::*:speed=100000
Bug fix for rare crash caused by very unusual incoming messages.
4.5h 1 November 1997
Fixed rebuild_index so it is even more careful not to read in corrupt data.
Added tellnews command: tellnews copy_group old.name new.name This command copies the contents of the group, modifying the 'newsgroups' headers correctly and creating new message-ids. If you copy a group repeatedly to the same new name, only 'new' items will be copied. This allows you to have a group that is an 'archive' of a real group, the archive might have a longer retention period (in expire.conf)
Added a dnews.conf setting C_AUTH_SHOWPROT TRUE This gives the entire list of news groups even if the user does not 'currently' have access to the specified groups. This allows the user to then select a group and be prompted for the correct user/password. (This works well for NS4)
Modified authentication responses to attempt to trick both NS3 and NS4 into working with protected news groups, this appears to work as well as is possible (given the bugs in NS3), NS3 requires that you shut down netscape (both the news reader and the browser) before connecting to a second protected news group (because it refuses to prompt for a second user/password)
4.5g5 29th Oct 1997
Fixed period of time during expire process during when some items would not be readable.
Fixed slow database leak.
Fixed database leak on dmulti systems
Made spam rejection silent for local posts and non descriptive for ihave feeds. Use in dnews.conf spam_debug true to make the rejections noisey again while testing spam rules.
Added check for approved header with moderated groups.
Fixed live feeds.
Fixed bug which could under 'rare' conditions cause the active file to shrink. (e.g. killing DNews in the middle of it's startup)
Removed dependence on 'df' for most versions of unix
Fixed dmulti problem with restart leaving zombies
Fixed problem with live feeds occasionally sending bad headers.
Added config paramter cache_ongroup true which if set will allow Agent to cache an empty group that was previously cached.
Fixed problem with active.dat getting control characters in group names, this could cause 'IE' to lock up while downloading groups.
Fixed bug with sucking list of news groups introduced in 4.5e
bind_in now works on unix
suck_uucp now stops if disk space runs out.
Fixed program feeds on unix so they restart correctly on failure.
Added logging for live feed debugging.
Fixed bug with header caching not working
Fixed bug with sort_lastmbmade newgroup control messages only email if not already done
Fixed bug with dmulti and local posts over 100K intermittently failing.
Changed behaviour to allow Netscape 4 to work with some news groups password protected. (removed, will be re added when fixed)
Changed main loop to decrease cpu use.
Added support for a distrib.conf file, this should be the same format as distrib.pat as used by INN.
Fixed problem with tellnews newgroup causing readers to hang when updating the list of news groups IF the moderator flag was left off the command.
Changed update syntax from, updaten, to update n e.g. tellnews update 6 To queue an update from nntp_suck6
Fixed bug with , reject < 3000, reject > 3000 (reversed logic and allowed non quoted strings)
Added cache so multiple incoming streaming live feeds will not allow articles to come in twice. Added debugging for history (hist_debug true)
If (uucp_simple true) is set in dnews.conf then the rnews counts are ignored, and the messages are read in based on the 'rnews headers' alone. Also if the rnews count is '0' then this format is assumed.
RAS support fixed, it now works correctly and phone book entries can be used.
tellnews update_clear now clears pending tellnews commands as well
Fixed bug with some missing items in groups caused by cancels.
Fixed bug with diskuse drifting if 'xover_moved' is true.
On unix added two dnews.conf settings:log_system true # Log to system log daemon log_files false # (stop logging to DNews log file)
Added 'content_type' config parameter to dnewsweb.ini
Fixed slow database leak, use the command: tellnews remove_lost to remove the 'lost' messages from your database, this may free a significant amount of storage.
Fixed problem with IEX4 crashing when reading from DNews.
Patched output from list overview.fmt which should fix a problem with some versions of NewsWatcher
Added dnews.conf setting suck_percent 90 for sucking only old stuff, this makes DNews suck only 90% of items from a group.
Added sucking feeds 5-16. Sucking feeds 4-16 cannot overlap, e.g. you cannot suck rec.humor from sucking feed 6 and 8 but you can suck rec.humor from suck2 and suck3. (clear as mud)
Improved performance at taking a full feed
Improved performance of taking cancel messages by a large factor
Modified the index behaviour so that it is partially sorted every hour instead of only once a day. This improves reader performance.
Added standard password $lookup$ support for VMS, BSDI, FreeBSD, Linux, Solaris.
Added external authentication support for Linux, FreeBSD, Solaris. See the file libauth.c for details.
4.5a Spam protection (Beta release)
Experimental spam protection system. The following settings can be added to dnews.conf, you can just set 'spam_stop true' or you can fiddle with all the settings and thus tune your system differently to other peoples which may be an advantage as 'spammers' will try and get past some of these rules.
|spam_stop||false||Enables spam protection|
|spam_test||false||Enables spam protection but doesn't reject anything, just logs what it would do.|
|spam_keepn||5||Number of rejected items to store in spam_n.tmp files (for reviewing)|
|spam_dup_total||5000||Number of items to 'remember' in our history of 'duplicate' bodies. Uses about 30 bytes per entry.|
|spam_dup_hits||6||Number of duplicate bodies (not cross posted) to allow|
|spam_dup_cross||400||Number of duplicate bodies counting cross posts to allow|
|spam_from_hits||60||Number of messages from any single user per hour|
|spam_from_cross||1000||Number of messages counting cross posts from any single user per hour|
|spam_from_total||5000||Number of users to keep tabs on in our history of duplicate users, uses about 100 bytes per user.|
|spam_host||false||Use 'nntp-posting-host' instead of 'from' for the above 3 rules.|
In addition to fiddling with all the above settings, you can create a file in the WORKAREA as defined by dnews.conf and add lines to the file, any message containing text matching the lines in the file will be rejected.
The idea is to use things that common spammers place in the body of their messages, this allows you to stop anyone who manages to get past the automatic mechanisms above.
sex 1-800-3333-4343 1-800-SEXYPHONE www.sex. Make money fast
Here is a sample filter.dat file which has kindly been donated by one of our customers, this will stop about 98% of known spammers at the cost of killing about one in a thousand real news messages.
DNews uses a 'super' efficient mechanism to test matches in this file so that you can have several hundred lines in the file without performance sufferring, each line must be at least 3 characters long. The matching is not case sensitive and is only in the 'body' and 'subject' of the message.
Fixed slow database leak which required a 'tellnews remove_lost' to periodically recover lost space.
Fixed crashing bug related to 'tellnews matchgroups' command
Fixed erroneous warning about access.conf giving world access to your server.
4.4a - 5th August 1997 - Major release of version 4.
Fixed bug that could 'occasionally' show 'no such article' when the item really did exist.
Fixed fault with dnewsmanager not working if the server is run on a port other than 119
Fixed problem with dnews/cyclone/diablo, where messages containing nul characters caused the feed to stop. Also included are settings to prevent incoming feeds jaming up under any conditions.
ADded dnews.conf setting nidxbf. The default is 6000, this must be big enough to handle all new items that arrive during a resort_index. Added some timeout code to fix feeds that block due to faulty messages.
Made live_stream true the default.
Added code to fix unsynchronized live feeds.
Fixed problem with 'downloading' message not appearing if (cache_onxover true) is set.
Fixed newgroup command so it updates active.names correctly
If you use '$lookup$" and you have case sensitive passwords then you should add to dnews.conf auth_case true and you must change each line in access.conf to lower case unless you have DNews 4.3d or later.
Fixed major bug in 4.3a (which could cause DNews to die)
4.3a July 14 1997
Allowed 'postonly' to work on a dmulti system.
Replaced command tellnews trimgroups [feednumber] with tellnews matchgroups [feednumber] which will add and remove groups to match the groups on the specified feed, this is intended for use with replicating servers to keep the active files synchronized.
Changed startup to automatically check for duplicate entries in active.dat (which was the cause of trimgroups occasionally deleting groups erroneously)
Fixed file handle leak introduced in version 4.2i, symtoms include tellnews status not responding.
Improved performance of live feeds
Added 'unblocking' code to live feeds to prevent problem items from blocking an outgoing feed.
Changed default live mode to non streamed for performance reasons.
Changed default live_nchan to 2.
Changed format of .feed files, the group:num field has been extended, it now includes: group:num:bucket_id:offset:len This allows direct database reads, but ONLY if the data is read before the next expire.
Added dnews.conf settings for NT/95 ras_number, ras_username, ras_password If these are set (in dnews.conf) then DNews will auto dial, fetch news and then hang up on its own.
Added tellnews command to detect when is a good time to 'hangup' a modem. tellnews isbusy, this command returns a different error status if DNews is fetching news still.
Fixed bugs in DMULTI relating to slaves not detecting the index switch
Fixed bugs in DMULTI related to 'recent' news not being visible
Two extra settings in dnewsweb.ini:wrap_post 60 wrap_text 60
wrap_post: wraps new posts, wrap_text: wraps the text in existing messages when they are displayed.
Added debug option for dnews.conf, debug_mem true
If set DNews will check all malloc'ed data every 20 seconds for over-writes and under-writes. (only used for debugging purposes, not intended for normal use)
Added default for suck_maxold to prevent news groups with big gaps from confusing the sucking logic.
Increased timeout on failed posts from a slave to a master DNews process from 30 seconds to 5 minutes.
Fixed possible bug in history indexes. Minor improvement to history indexes, and speeded up startup.
Speeded up a startup between a rebuild_index and an expire (which used to be very very slow)
Fixed bug introduced in 4.2r :-)
Memory handling changed to reduce usage, changes include:
- Move strings between blocks.
- Free string blocks.
- list newsgroups, group names stored more efficiently
- Bug with db_dir using too much memory fixed
Corrected number of items shown in non cached groups
Minor changes for y2k problems with newnews/newgroups commands
Extended flexibility of acceptable dates.(Y2K issues)
Added dnews.conf setting reject_addhist true which if enabled, adds rejected items to the history database, this can reduce bandwidth wastage if you have more than one full feed coming in.
Added tellnews command: tellnews killone group.name itemnumber This is like killitem, but does not remove crossposts.
Added dnews.conf settings: post_rec true (logs pertinent information about each post)
Improved checkgroups so it will deal with mis formatted messages better.
Corrected posting mechanism on dmulti so that if it fails the user will be informed immediately.
Removed deletion of items that can't be accessed.
Updates file for 4.2m and older