Ben Giddings wrote:
For those of us who are interested in the matter, but aren’t aware of
how things work, can someone give a quick summary of how the mirroring
is supposed to work?
Some questions I have:
- Is it a ‘fire and forget’ sort-of system – messages are sent to the
NNTP server and if that doesn’t receive error messages it is assumed to
have worked? Or does the software keep track of message IDs and somehow
check to see that something has happened with them?
Your ‘fire and forget’ description makes me think you’ve been looking 
More specifically, it fires at nntp.rb and forgets. I think nntp.rb
would raise if there was a problem, though. The NNTP package has been
around for a while (4 years). It is (unmodified) from:
http://www02.so-net.ne.jp/~greentea/ruby/nntp/
- How are messages mirrored from the newsgroup to the mailing list? Is
it just something like “Add all new messages as of this date/time to the
mailing list”, or does it keep track of message IDs in some way?
Add all new messages since the highest saved message-id.
Yes, it keeps track.
NG → ML direction seems to be flawless. [1]
- Is there any ‘retry’ mechanism at all?
If you’re referring to “outside of NNTP protocol”, there seems to no
attempt to retry and if there is some reason for a message to be
missed, there’s no fancy recovery.
- [DABlack] mentioned that if the message isn’t being copied by a
not-directly-upstream server, then there’s nothing that can be done to
trace it. It seems like there should be. If you can confirm that the
gateway is working properly with the immediate NNTP server, find out
what its neighbors are and ask the admins there to verify that the
messages were transferred correctly, then if so, plead with them to
check with their neighbors. I would suspect that if everything is
properly mirrored in all those places, then it will get really hard to fix.
The easy way to annoy admins is to ask them to do work that we
should be doing. IME, they’re most helpful when there’s
evidence they can’t ignore 
I’m confident that if the problem was further downstream than nntp.rb,
there would be losses from sources other than ours.
I don’t know much about NNTP, only the basics.
I’ll join you, there.
Is the way that our messages are added to the NNTP server the
same as the way that a standard network client does it? Are there extra
headers or anything that might make an NNTP server upset?
NNTP seems to be a sub-protocol of SMTP. I wondered if the
conversion from mail to news was leaving in headers that were,
strictly, SMTP only. But that doesn’t happen. Only selected headers
are copied, all are correct for NNTP.
BTW: You are asking all the right questions, IMO.
Also, maybe the way the mailing list is set up can be changed. I
imagine right now that the way things work is:
Emails sent to the mailing list are sent out to all members,
one of those members is the gateway program, it sends them out
to the newsgroup. The newsgroup looks for new messages and when
it finds them, it emails them to the mailing list.
Tick [or check in US].
One thing that bothers me is fixes to problems (e.g. floods) that
we’ve had in the past. There has to be a possibility that the
ruby-lang mailer has imposed restrictions on messages sent to the
“gateway ML member” and inadvertently introduced a corner-case.
Most of the serious people are on the ML, and if that’s where I
was viewing, I’d probably be wondering what the NG were all moaning
about. Mailing lists are reliable, Usenet isn’t? I think not.
This is a problem with the gateway although (as David pointed out
in a post that I just caught up on in the -talk archive because it
didn’t get through) that doesn’t mean the gateway software; rather,
it means the path from the ruby-lang mailer to the first Usenet
news server.
Maybe instead, we can have the mailing list be nothing but something
that copies messages to the newsgroup, and then have a secondary process
that takes everything from the newsgroup and sends it out to the mailing
list members. That way, instead of having two different lists that have
to be synchronized, we have only 1 (the newsgroup) that has to be
emailed out.
Any thoughts?
Yes, that’s interesting. The ML posters whose messages failed to get
through to c.l.ruby would have fairly immediate feedback that something
was wrong. Their messages wouldn’t appear.
Ben Giddings
I’m late in on this (one of my favourite topics), but my humble
investigation has been waiting for log evidence.
Logs are cool!
If rubygate[2] produced a log entry for any failed message, that
would provide a huge amount of information for me. I need to
know before I go chasing ghosts.
···
========================================================================
David Black said in [talk:101478]
My tests (fairly barebones, but anyway)
suggest that the gateway software is, in fact, passing along the
messages in question. Something is happening further downstream.
Software offers protection from accusations of inhumane treatment.
Pin the bug, then kill it.
-
Add regex to mail2news to exit unless the message is from
dblack@address.net (insert your ML email address) else continue.
-
Start rubygate on your machine.
-
Reply to a message via ruby-talk
RESULTS (0=false; 1=true; *=don’t care):
- [A] Appears in email inbox ?
- [B] Logged_as_forwarded ?
- [C] Appears on comp.lang.ruby ?
My humble ANALYSIS:
A - Assumed A1 because ML members see everything ?
I’m also assuming that your phantom posts appear
in your email inbox but not in c.l.ruby
Please say, if different.
B0:C0 Incorrect rubygate setup or mail2news oddity.
B0:C1 No, that didn’t happen :~>
B1:C0 suspect nntp.rb problem - OK, I’ll volunteer 
Please email the example input file for mail2news.
B1:C1 nntp.rb works - TU-Berlin news service oddity OR
“gateway ML member” (Dennis) is being censored
by ruby-lang mailer.
Modifications/criticism welcome. I want to see it fixed, too.
daz
[1] As long as the news service continues incrementing
and doesn’t reset to a lower number (ask PragDave;)
[2] (dunno if it was Dennis or David or the “Washington Post” newspaper
who came up with that fine name for the gateway software.)