DRE - An RSS-to-Email Aggregator
DRE is a Perl script that fetches articles from a list of RSS
feeds and sends them on as plain-text emails.
Feed URLs go in ~/.dre/feeds - one URL per line, blank lines are OK,
lines beginning with a hash (#) are ignored as comments.
The recipient of the emails goes in ~/.dre/recipient - single line,
plain email address, e.g.
Just run the script: dre
-d - print chatty debug information
-q - be quiet (don't complain about bad feeds) - for use with cron
-n - don't send any email (for repopulating the "seen" file)
-a - send email for all entries, even those previously seen
-f - download (to ~/.dre/downloads) an mp3 file listed as an enclosure
-h - show brief help text
Download the current version. The
licence is GPL.
Required Perl modules: XML::RSS, HTML::Entities, LWP::UserAgent,
LWP::Simple, DB_File, HTML::FormatText::WithLinks, Data::Dumper, Getopt::Std.
11th April 2005:
15th June 2005:
- we are more defensive about missing links in feed items
- added -q switch for quiet (don't whinge on bad feeds)
- XML errors no longer cause us to bail out
- I *think* high-bit characters are handled OK
31st August 2005:
- in the absence of a "link" attribute, use the "guid" attribute
instead (to handle the BBC's podcasting trial)
- added "-n" flag for "don't send any mail" - useful for
repopulating the "seen" file without getting spammed
- added "-a" flag for "send all mail", even resending previously
- added "-h" flag for brief help
23rd September 2005:
- use HTML::FormatText::WithLinks instead of the home-grown parser
based on HTML::PullParser. This should make the output look nicer and
reduce the number of error messages from dodgy feeds.
- added a small number of workarounds for "unusual" formatting
11th November 2005:
- there's a brief nod of the head to
better UTF-8 support
6th July 2006:
- Be more tolerant of feed items with no title.
4th August 2006:
- Use the guid rather than the link to track "seen" status.
- Optionally download an mp3 file referenced as an enclosure.
- (...using LWP::Simple to avoid sucking it into memory en route...)
23rd March 2007:
- Cope with non-permalink guids (argh!)
- Handle UTF-8 better in debug output.
The name sort-of stands for "Dave's RSS to Email convertor".
- In the absence of a guid or link, use an enclosure url if it's
present. This was prompted by the Classic FM Jane Austen series. Gosh,
there are some badly-formed RSS feeds out there.
Troubleshooting / Q&A
- DRE keeps sending me the same feed entries over and over again.
- Maybe you upgraded your DB library, e.g. by upgrading Debian Woody
to Sarge, and the "seen" file can't be read or updated:
dave@eyas:~/.dre$ file seen*
seen: Berkeley DB (Hash, version 8, native byte-order)
seen.old: Berkeley DB (Hash, version 5, native byte-order)
Fix: delete the "seen" file and run "dre -n" to repopulate the "seen"
- DRE doesn't understand Atom feeds.
- I know... it's on my to-do list so I can read Google Blog.
$Id: dre.html,v 1.13 2007-03-23 13:24:08 dave Exp $