Ruby and Mozilla?

ruby-talk
1

If you have further thoughts, let me know...

Rather than worrying about dealing with Mozilla, why don't you take a
mailer-generic approach and deal with the mail separately? The POP3
protocol is quite simple. There are two approaches you could take,
following in the footsteps of anti-virus programs that have done both of
these:

1) Write a Ruby POP3 client that analyzes message in-situ and deletes
the ones off the server that you don't like. Then, when your regular
POP3 client (Mozilla in this case) goes to suck in mail off the server,
the spam's already gone. This is the approach taken by SpamEater Pro,
for instance.

2) Write a Ruby POP3 client that sits between your mail reader and the
server. It acts as a proxy and listens on your local host's mail port.
You set your mail reader (Mozilla) to connect to localhost instead of
pop3.myisp.com. When the proxy gets a request for messages, it wakes up
and requests messages from your real server (pop3.myisp.com), then
passes through the ones it thinks aren't spam. This is the approach
taken by every anti-virus program I've seen.

One advantage of both of these approaches over your original idea is
that they're independent of the mail program. If, at some time in the
future, you drop Mozilla mail for Microsoft Outlook or whatever, your
anti-spam program will still work.

Todd.

2

Bradley, Todd wrote:

Rather than worrying about dealing with Mozilla, why don't you take a
mailer-generic approach and deal with the mail separately? The POP3
protocol is quite simple. There are two approaches you could take,
following in the footsteps of anti-virus programs that have done both of
these:

Yes, well, I wasn't really looking for a Moz-specific solution...
unless there was one that required less coding.

1) Write a Ruby POP3 client that analyzes message in-situ and deletes
the ones off the server that you don't like. Then, when your regular
POP3 client (Mozilla in this case) goes to suck in mail off the server,
the spam's already gone. This is the approach taken by SpamEater Pro,
for instance.

Yes, I thought of this, and it would be fairly easy.

Trouble is, I spend *many* hours in front of the computer each day. My
mail client is pretty much always up and it hits the server every
60 seconds.

That makes a synchronization problem if I keep that usage habit.

2) Write a Ruby POP3 client that sits between your mail reader and the
server. It acts as a proxy and listens on your local host's mail port.
You set your mail reader (Mozilla) to connect to localhost instead of
pop3.myisp.com. When the proxy gets a request for messages, it wakes up
and requests messages from your real server (pop3.myisp.com), then
passes through the ones it thinks aren't spam. This is the approach
taken by every anti-virus program I've seen.

OK, I wouldn't mind doing this... but I'm not sure how to create a
proxy. Do I actually run a real pop3 server or what? Duh?

One advantage of both of these approaches over your original idea is
that they're independent of the mail program. If, at some time in the
future, you drop Mozilla mail for Microsoft Outlook or whatever, your
anti-spam program will still work.

Yes, a good idea.

Someone mentioned Gurgitate, which looks cool... but again I don't know
how to stick it in between my mail client and my remote pop server.

Hal

3

Hal Fulton wrote:

OK, I wouldn't mind doing this... but I'm not sure how to create a
proxy. Do I actually run a real pop3 server or what? Duh?

Here's what I do: I run my own courier-imap server on a linux box. Every ten minutes, crons run fetchmail on that box to grab any email waiting at any of several external POP servers. Fetchmail calls (postfix) sendmail, which calls procmail, which invokes SpamAssassin. It sounds complex, but it's really not hard to set up. SpamAssassin does a great job of filtering most things out, and I can write ad hoc rules for either SpamAssassin or procmail as needed. Then I just point Mozilla at my IMAP server every minute.

The linux box is behind a NAT firewall that only forwards ssh (TCP/22). I can ssh into the linux box from anywhere, tunnel IMAP (TCP/143) through ssh, and check my personal email. In that case Mozilla thinks the IMAP server is localhost, but it's just the local proxy for the ssh tunnel.

Steve

4

* Hal Fulton <hal9000@hypermetrics.com> [1152 20:52]:

Bradley, Todd wrote:

Yes, well, I wasn't really looking for a Moz-specific solution...
unless there was one that required less coding.

>1) Write a Ruby POP3 client that analyzes message in-situ and deletes
>the ones off the server that you don't like. Then, when your regular
>POP3 client (Mozilla in this case) goes to suck in mail off the server,
>the spam's already gone. This is the approach taken by SpamEater Pro,
>for instance.

Yes, I thought of this, and it would be fairly easy.

Trouble is, I spend *many* hours in front of the computer each day. My
mail client is pretty much always up and it hits the server every
60 seconds.

Bet you're popular with your ISP :slight_smile:

>2) Write a Ruby POP3 client that sits between your mail reader and the
>server. It acts as a proxy and listens on your local host's mail port.
>You set your mail reader (Mozilla) to connect to localhost instead of
>pop3.myisp.com. When the proxy gets a request for messages, it wakes up
>and requests messages from your real server (pop3.myisp.com), then
>passes through the ones it thinks aren't spam. This is the approach
>taken by every anti-virus program I've seen.

OK, I wouldn't mind doing this... but I'm not sure how to create a
proxy. Do I actually run a real pop3 server or what? Duh?

I think what's being suggested is to code your own pop3 server - I'm assuming
Bradley meant localhost:110 when he said "local host's mail port" - that's
actually fairly simple (pop3 is a pretty trivial protocol, and the ruby pop3
library should do all the client side for you)).

The tricky bit is the spam filtering, which I presume is where Gurgitate comes
in - although I know precisely zip about it, I'd hope you could just feed a mail
(String) into a is_spam? method to scan it.

Personally I've made the switch to an IMAP server and never looked back; the
protocol is a lot more complicated, but lets multiple clients use the same
Sent/Drafts/Trash/Spam folders - even notifies mutt on my BSD box that I just
read a message with Thunderbird on the eMac. Add an LDAP address book and I get to
keep almost all my mail settings on the server, backed up once, and never have to try
to remember what Sent folder that mail was in...

[ Hats off to whoever wrote our Net::IMAP libs incidentally, they are a joy to use. ]

Each client doing its own Bayesian training would drive me potty,
so I've rolled together a SpamAssassin and Sieve combo which seems to work fairly well.

Combined with exims exiscan, I get to tell spammers where to go at SMTP time rather than
scanning mails after I've accepted them, which keeps the queue relatively free of the
hundreds of bounce messages I had before.

···

--
Yeah, life is hilariously cruel. - Bender
Rasputin :: Jack of All Trades - Master of Nuns

5

Hal Fulton wrote:
...

Someone mentioned Gurgitate, which looks cool... but again I don't know
how to stick it in between my mail client and my remote pop server.

I took a quick look at Gurgitate, which appears to be intended for use on a server, as a filter called by procmail or qmail. However, I don't see why it could not also by hooked into client-side POP3 proxy.

The trickier (or less entertaining) part might be implementing the POP3 protocol in the proxy so that the mail client is oblivious to where it gets its mail.

But Gurgitate looks all set up for crafty mail filtering; would be a shame to reinvent that part.

James

6

Dick Davies wrote:

I think what's being suggested is to code your own pop3 server - I'm assuming
Bradley meant localhost:110 when he said "local host's mail port" - that's
actually fairly simple (pop3 is a pretty trivial protocol, and the ruby pop3
library should do all the client side for you)).

I've played with the client stuff quite a bit, and it's not hard.

I'm surprised to hear anyone say it would be trivial to write a POP3 server,
though. How trivial??

The tricky bit is the spam filtering, which I presume is where Gurgitate comes
in - although I know precisely zip about it, I'd hope you could just feed a mail
(String) into a is_spam? method to scan it.

It seems well documented. That's the least of my worries. I'd even be willing to
write my own stuff.

[ Hats off to whoever wrote our Net::IMAP libs incidentally, they are a joy to use. ]

Quite right, I spent a few weeks working with IMAP, and it's a great lib. Isn't that
Shugo Maeda's work?

I think for the present, IMAP is overkill for me. In the future, definitely.

Hal

7

James Britt wrote:

I took a quick look at Gurgitate, which appears to be intended for use on a server, as a filter called by procmail or qmail. However, I don't see why it could not also by hooked into client-side POP3 proxy.

On my system, gurgitate-mail is called by postfix (in place of procmail), but I think it's easy to call it through fetchmail using the -m switch.
This way fetchmail will pass the mail to gurgitate-mail which will transfer it in the mailbox of choice using user's rules, and the MUA just has to read these mailboxes.

Moreover fetchmail can be run as a daemon, checking for new mail every few minutes, or on a "one shot" basis (i.e called by the user or the MUA when there's a need for new mail check).

I should say that the conversion from my old procmail rules has been much than easy, and since that day I have no problem at all...

Frederic Ros.

8

* Hal Fulton <hal9000@hypermetrics.com> [1110 22:10]:

Dick Davies wrote:
>I think what's being suggested is to code your own pop3 server - I'm
>assuming
>Bradley meant localhost:110 when he said "local host's mail port" - that's
>actually fairly simple (pop3 is a pretty trivial protocol, and the ruby
>pop3
>library should do all the client side for you)).

I've played with the client stuff quite a bit, and it's not hard.

I'm surprised to hear anyone say it would be trivial to write a POP3 server,
though. How trivial??

Ah, but we're talking about a *proxy* not a server -
what I meant by simple is that there are relatively few commands, and they're
just plain text so you can just forward to the real server.

( possibly misleading half-remembered technical stuff ahoy )

From what I remember of it, the basics of the protocol are just:

telnet some.popserver.com 110
user username
pass password

bing - you're logged in, then

list : list messages and sizes
retr n : dump message n <-- this is the only bit you need to bugger about with
top n x : show x first lines of message n
dele n : delete message n
quit

(that's almost certainly partly wrong, check the rfc to be sure).

So for your purposes, you just want to :

listen on localhost port 110,
open a socket to realpopserver.net on port 110,
send whatever strings you recieve from your client to the real server, and
send the response back to the client.

That should be knock-uppable[1] in an afternoon, then you extend it so
if the request string starts with retr you push the response from the server
through your spamfilter before passing on to the client. Once you're confident
its working, you can just 'dele' the message off the main server.

Obviously that's a simplification (security wise I wouldn't be happy with the
risk of other local users getting my authenticated session, for example)
but it's certainly doable... not by me though, happily :slight_smile:

>[ Hats off to whoever wrote our Net::IMAP libs incidentally, they are a
>joy to use. ]

Quite right, I spent a few weeks working with IMAP, and it's a great lib.
Isn't that Shugo Maeda's work?

Mr. mod_ruby? So it is.
"Respec'", as we say in the ghetto....

[1] - I invented a new word - do I get it named after me?

···

--
Rasputin :: Jack of All Trades - Master of Nuns

9

Frederick Ros wrote:

James Britt wrote:

I took a quick look at Gurgitate, which appears to be intended for use on a server, as a filter called by procmail or qmail. However, I don't see why it could not also by hooked into client-side POP3 proxy.

On my system, gurgitate-mail is called by postfix (in place of procmail), but I think it's easy to call it through fetchmail using the -m switch.
This way fetchmail will pass the mail to gurgitate-mail which will transfer it in the mailbox of choice using user's rules, and the MUA just has to read these mailboxes.

Moreover fetchmail can be run as a daemon, checking for new mail every few minutes, or on a "one shot" basis (i.e called by the user or the MUA when there's a need for new mail check).

I should say that the conversion from my old procmail rules has been much than easy, and since that day I have no problem at all...

Sweet, thanks for the info.

James