|
|
|
|
|
|
|
| |
|
|||||
| |
|
|
|
|
|
|
| |
||||||
 |
 |
 |
 |
 |
 |
PULL Module - parallel multi channel sucking 5.3 and later
This module is called the 'pull' module, it makes DNews suck from different news servers at the same time, you can also use more than one channel to suck from each upstream site if you really want to.
To enable concurrent 'pull' sucking add this to dnews.conf
pull true
To specify additional parameters in addition to the standard ones provided by nntp_feeder,
nntp_suck2..., update_at..., use settings like this:
pull1 con=3 percent=90 maxitems=500 maxage=10 speed=50000 mb=10
disable=8-17
pull2 ...(corresponds with nntp_suck2 ...)
You must still specify nntp_suckN and update_atN for each upstream site you are pulling
from.
| Option | Description |
|---|---|
| con | Number of concurrent connections for that feed, default 1, don't get silly with this setting it will annoy your upstream site |
| percent | Specifies the amount of each group to get, e.g. 90% would get items 1-18 if the group had 20 items in it, this is good if you are pulling to 'fill in the gaps' and dont want to get recent items from an expensive site. |
| maxitems | Max items to suck from that site, e.g. if it has items 2000-3000 and maxitems is 10 DNews would suck 2990-3000 |
| maxage | If items have dates more than this many days old they will be rejected |
| speed | The max bytes to read per second from this site, you may need to set in_size to make this setting work with figures below 100000 bytes per second. |
| mb | The max MB of data to pull from this site per day, after this limit is exceeded DNews will stop sucking until the next day, this figure is reset when you restart dnews. |
| nonext | If set DNEWS will avoid using the 'next' nntp command, this is useful with servers that do not obey the NNTP rfc correctly. |
| disable | x-y Range of hours over which NOT to pull news. Useful for stopping DNews sucking during daytime. |
NOTE: Pull sucking cannot be used with header sucking.
Tellnews status will report statistics on the pulling mechanism:
Ok Dup Rej
Qued/Busy/Done MB/MAX Site
23 3
5 200/3/48 3.40/0 news
In the above example 23 = Number of items received ok, 3 = number of duplicates, 5 = number of items rejected (e.g. by newsfeeds.conf rules) Qued = Number of groups waiting to be pulled from this site, Busy = number of channels currently pulling news, Done = Number of groups successfully pulled, mb = amount of data pulled so far today, MAX = the MB setting, zero means no limit, site = the place to pull
To debug a pull feed type in this:
tellnews debug pull:
tellnews loglvl debug
This will then record in dnews.log all the nntp trafic sent and receieved with regard to
each pull site.