From 3b1f9f304f2bf695d02ab151b6ea2145df5e5648 Mon Sep 17 00:00:00 2001 From: Ward Wouts Date: Sun, 5 May 2002 20:05:11 +0000 Subject: [PATCH] brand new documentation, whee --- trunk/ripnews/INSTALL | 8 +++ trunk/ripnews/README | 141 ++++++++++++++++++++++++++++++++++++++++++ trunk/ripnews/TODO | 15 +++++ 3 files changed, 164 insertions(+) create mode 100644 trunk/ripnews/INSTALL create mode 100644 trunk/ripnews/README create mode 100644 trunk/ripnews/TODO diff --git a/trunk/ripnews/INSTALL b/trunk/ripnews/INSTALL new file mode 100644 index 0000000..51f706e --- /dev/null +++ b/trunk/ripnews/INSTALL @@ -0,0 +1,8 @@ +# $Id$ +# $Source$ + +For now the easiest way to install this is just extract the tarball in +its own directory and run ./ripnews.rb from there. Before running you +should make your own .ripnewsrc configuration file which is described in +the README file. You may have to change the first line in ripnews.rb to +point to your ruby executable. diff --git a/trunk/ripnews/README b/trunk/ripnews/README new file mode 100644 index 0000000..e239314 --- /dev/null +++ b/trunk/ripnews/README @@ -0,0 +1,141 @@ +# $Id$ +# $Source$ + +Ripnews is a bulk downloader for usenet. It's quite flexible in terms of +configuration. Some of it's features are: + +- basic support for multiple servers per group +- cacheing of article headers to speed up reading of newsgroups +- newsrc file support (one newsrc file per server) +- flexible but simple configuration + +Configuration: +============== + +I'll just give a commented example config, it should be pretty clear, +after that I'll list the possible options. + +<== cut here ==> +# Set the default NNTPSERVER to localhost +NNTPSERVER=localhost + +# Set the cachedir, this is where the subject caches are stored +# without this ripnews will be much slower (but should still work) +CACHEDIR=/mnt/newspace/News/.ripnews_caches + +# Set the datadir, this where a subdir for each group will be made to +# store the ripped articles +DATADIR=/mnt/newspace/News + +# Set the tempdir, used to store the undecoed data. Without this ripnews +# uses a lot more memory +TEMPDIR=/mnt/newspace/News/ripnews_temp + +# Set include pattern to a case insensitive "bad religion" +OPT_I=(?i)bad religion + +# Set the base newsrc name. The server name will be appended. +NEWSRCNAME=/ward/src/ruby/ripnews/.newsrc + +# Set the permission to create subdirs with +PERMISSION=0700 + +# For alt.binaries.e-book.technical change from defaults... +alt.binaries.e-book.technical { + # Set another include pattern + OPT_I=(?i)reilly +} + +alt.binaries.e-book.flood { + # Add to default pattern, this will not be case insensitive + # anymore, cause that's how ruby patterns work + OPT_I+=|douglas adams +} + +# For both alt.binaries.e-book.technical and alt.binaries.e-book.flood +# change some value +alt.binaries.e-book.technical| \ +alt.binaries.e-book.flood { + # Sets long filenames. If this is set the subject will be used + # as a filename instead of the name specified in the encoding. + OPT_L = true +} + +# Change default server to news.tilbu1.nb.nl.home.com, since the config +# is parse in order this will be used from her on down +NNTPSERVER=news.tilbu1.nb.nl.home.com + +alt.binaries.sounds.mp3.heavy-metal| \ +alt.binaries.sounds.mp3.1980s { + # Add news4.euro.net as a second server for + # alt.binaries.sounds.mp3.heavy-metal and + # alt.binaries.sounds.mp3.1980s + NNTPSERVER+=|news4.euro.net +} + +alt.binaries.sounds.mp3.gothic-industrial +alt.binaries.sounds.mp3.heavy-metal| \ +alt.binaries.sounds.mp3.1980s { + OPT_L=true + OPT_I=(?i)( \ + bauhaus| \ + big black \ + ) +} +<== cut here ==> + +Supported commandline options: +------------------------------ + +"-I", "--include" Set include pattern. +"-c", "--configfile" Specify a different config file. Default + .ripnewsrc +"-L", "--longname" Sets long filenames. +"-X", "--exclude" Set exclude pattern. +"-T", "--test" Set test mode. Newsrc files will not be writen + to. + +Supported config options: +------------------------- + +OPT_I= Set include pattern. +OPT_L= Set long filenames. +OPT_X= Set include pattern +OPT_T= Set test mode. Newsrc files will not be written + to. +TEMPDIR= Set tempdir location. +NNTPSERVER=[|server] Set NNTPSERVER names +CACHEDIR= Set cachedir location. +DATADIR= Set output dir location. +NEWSRCNAME= Specify newsrc basename. Server names + will be appended. +PERMISSION= Set permission bits for directory + creation. Standard unix style, eg. 0755. + +Ruby patterns: +-------------- + +Ruby patterns are a lot like perl patterns, but there are some +differences. (?i) is the modifier to turn on case insensitivity, unlike +perl this modifier only works on the following block. Luckily you can +group multiple blocks into one by enclosing them with ()'s. So while +'OPT_I=(?i)foo|bar' would match 'foo' case insensitve and 'bar' case +sensitive 'OPT_I=(?i)(foo|bar)' will match both 'foo' and 'bar' case +insensitivly. + +Known bugs: +=========== + +There are a lot of known bugs at this time. Basically the error handling +is still almost non existant and server reconnects are not yet possible. +This can be annoying (ok, it _is_ annoying) when server1 timesout while +you're still processing server2. Well, if it breaks you get to keep +_both_ pieces. That's what you get with pre-alpha software ;) + +Contact info: +============= + +New problems can be reported directly to me at . Patches +welcome ;) + +Ward Wouts diff --git a/trunk/ripnews/TODO b/trunk/ripnews/TODO new file mode 100644 index 0000000..d7ca740 --- /dev/null +++ b/trunk/ripnews/TODO @@ -0,0 +1,15 @@ +# $Id$ +# $Source$ + +[ ] support mime encoding +[ ] support yEnc encoding +[ ] matching on encoded file extensions +[ ] documentation +[ ] code cleanup +[ ] finish intspan +[ ] profiling/speed ups +[ ] improve error handling +[ ] server reconnects +[ ] check if xhdr implemented +[ ] write man page +[ ] use prefered server order