ripnews/trunk/ripnews/README

# $Dwarf: README,v 1.18 2005/02/05 12:00:39 ward Exp $
# $Source$

Ripnews is a bulk downloader for usenet. It's quite flexible in terms of
configuration. Some of it's features are:

- basic support for multiple servers per group
- cacheing of article headers to speed up reading of newsgroups
- newsrc file support (one newsrc file per server)
- flexible but simple configuration

Configuration:
==============

I'll just give a commented example config, it should be pretty clear,
after that I'll list the possible options.

<== cut here ==>
# Set the default NNTPSERVER to localhost
NNTPSERVER=localhost

# Set the cachedir, this is where the subject caches are stored
# without this ripnews will be much slower (but should still work)
CACHEDIR=/mnt/newspace/News/.ripnews_caches

# PID lockfile, prevents multiple ripnews processes from running at the
# same time [global keyword]
LOCKFILE=/local/newspace/News/.ripnewslock

# Set the datadir, this where a subdir for each group will be made to
# store the ripped articles
DATADIR=/mnt/newspace/News

# Set the tempdir, used to store the undecoded data. Without this ripnews
# uses a lot more memory
TEMPDIR=/mnt/newspace/News/ripnews_temp

# Set include pattern to a case insensitive "grateful.dead"
OPT_I=(?i)grateful.dead

# Set the base newsrc name. The server name will be appended.
NEWSRCNAME=/ward/src/ruby/ripnews/.newsrc

# Set the permission to create subdirs with
PERMISSION=0700

# Set the niceness of the ripnews process [global keyword]
NICE=20

# For alt.binaries.e-book and alt.binaries.e-books change from defaults...
alt.binaries.e-book| \
alt.binaries.e-books {
	# Set another include pattern
	OPT_I=(?i)(bible|dickens|shakespeare)
}

alt.binaries.e-book.flood {
	# Add to default pattern, this will not be case insensitive
	# anymore, because that's how ruby patterns work
	OPT_I+=|george.orwell
}

# For both alt.binaries.e-book, alt.binaries.e-books and
# alt.binaries.e-book.flood change some value
alt.binaries.e-book| \
alt.binaries.e-books| \
alt.binaries.e-book.flood {
	# Sets long filenames. If this is set the subject will be used
	# as a filename instead of the name specified in the encoding.
	OPT_L = true
}

# Change default server to news.tilbu1.nb.nl.home.com, since the config
# is parsed in order this will be used from her on down
NNTPSERVER=news.tilbu1.nb.nl.home.com

alt.binaries.music.classical| \
alt.binaries.sounds.lossless.classical| \
alt.binaries.sounds.mp3.classical {
	# Add news4.euro.net as a second server for
	# alt.binaries.music.classical,
	# alt.binaries.sounds.lossless.classical and
	# alt.binaries.sounds.mp3.classical
	NNTPSERVER+=|news4.euro.net
}

alt.binaries.music.classical| \
alt.binaries.sounds.lossless.classical| \
alt.binaries.sounds.mp3.classical {
	OPT_L=true
	OPT_I=(?i)( \
		verdi| \
		vivaldi| \
		mozart| \
		beethoven \
	)
}
<== cut here ==>

Supported commandline options:
------------------------------

"-I", "--include"	Set include pattern.
"-c", "--configfile"	Specify a different config file. Default
			.ripnewsrc
"-L", "--longname"	Sets long filenames.
"-C", "--combinedname"	Sets combined filenames.
"-X", "--exclude"	Set exclude pattern.
"-T", "--test"		Set test mode. Newsrc files will not be writen
			to.

Supported config options:
-------------------------

OPT_I=<pattern>			Set include pattern.
OPT_L=<bool>			Set long filenames.
OPT_C=<bool>			Sets combined filenames.
OPT_X=<pattern>			Set exclude pattern. Ripnews will read articles                                 matching this pattern but it will not attempt
                                to download them.
OPT_MR=<pattern>		Set "mark read" pattern. Ripnews will place
                                articles matching this pattern in your newsrc,
                                afterwards they will never be present in memory
                                again. Great for reducing memory usage when
                                checking a group for the first time.
OPT_T=<bool>			Set test mode. Newsrc files will not be written
				to.
TEMPDIR=<dir>			Set tempdir location.
NNTPSERVER=<server>[|server]	Set NNTPSERVER names
CACHEDIR=<dir>			Set cachedir location.
DATADIR=<dir>			Set output dir location.
NEWSRCNAME=<newsrcbase>		Specify newsrc basename. Server names
				will be appended.
PERMISSION=<perm>		Set permission bits for directory
				creation. Standard unix style, eg. 0755.
EXTENSIONS=<pattern>		Set extension include pattern.
OPT_M=<pattern>			Set EXTENSIONS just for multi part messages.
OPT_S=<pattern>			Set EXTENSIONS just for single part messages.
DELEXT=<pattern>		Set extension "mark read" pattern.
OPT_MD=<pattern>		Set DELEXT just for multi part messages.
OPT_SD=<pattern>		Set DELEXT just for single part messages.
INCLUDEFILE=<file>		Include another file, only works in main config.

Ruby patterns:
--------------

Ruby patterns are a lot like perl patterns, but there are some
differences. (?i) is the modifier to turn on case insensitivity, unlike
perl this modifier only works on the following block. Luckily you can
group multiple blocks into one by enclosing them with ()'s. So while
'OPT_I=(?i)foo|bar' would match 'foo' case insensitve and 'bar' case
sensitive 'OPT_I=(?i)(foo|bar)' will match both 'foo' and 'bar' case
insensitivly.

Caveat: if for some reason you use a | at the end of a list of patterns
(for instance: OPT_X=(?i)(foo|bar|) ) the pattern will also match an
empty string.  This can have the result that you exclude everything if
you use it with OPT_X, or include everything with OPT_I. You have been
warned.

Other features:
===============

You can make a running ripnews process reread it's configuration by
sending it a SIGHUP.

Where can I find newsservers:
=============================
freenews.maxbaud.net
www.newzbot.com
www.gj.net/~bhkraft

Known bugs:
===========

There are no known bugs at this moment. If you find any, please let me
know. As with all my software, if it breaks you get to keep _both_
pieces.

Credits:
========
- Stijn Hoop for adding yEnc support

Contact info:
=============

New problems can be reported directly to me at <ward@wouts.nl>. Patches
welcome ;)

Ward Wouts