Separating ruby-talk from comp.lang.ruby?

Hi –

···

On Tue, 1 Jun 2004, daz wrote:

David A. Black wrote:

[…] testing again… small meaningless tweak to headers…

Mirrored using mail2news.rb (the version you sent me - unmodified).

Worked as I expected, so that rules out nntp.rb & headers.

Took your message out of my inbox (from ML).
Windows box, so newlines are different.
I might gsub them and send again.

OK… so the appearance of that msg on the newgroup was because you
mirrored it? (I just want to make sure and not get my hopes up :slight_smile:

David


David A. Black
dblack@wobblini.net

I do not know if this is related or relevant, but I often
receive messages with the last ~89 characters repeated.
This is what I see verbatim:

David A. Black wrote:

[…] testing again… small meaningless tweak to headers…

Mirrored using mail2news.rb (the version you sent me - unmodified).

Worked as I expected, so that rules out nntp.rb & headers.

Took your message out of my inbox (from ML).
Windows box, so newlines are different.
I might gsub them and send again.

Cheers,

daz

dows box, so newlines are different.
I might gsub them and send again.

Cheers,

daz

Here is another recent example:

···

— daz dooby@d10.karoo.co.uk wrote:

— Shashank Date sdate@everestkc.net wrote:

Hi Kaspar,

“Kaspar Schiess” eule@space.ch wrote in message

Ruby Lexer for SciTE Code Editor 0.8 is available from
www.tua.ch/current.html (compiled with SciTE 1.61).

I got the “File Not Found” error when I tried to visit this site.
Thanks,
– shanko

I got the “File Not Found” error when I tried to visit this site.
Thanks,
– shanko


Do you Yahoo!?
Friends. Fun. Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/

“daz” dooby@d10.karoo.co.uk schrieb im Newsbeitrag
news:xdydnRPKJ5nIACbdSa8jmA@karoo.co.uk

Robert Klemme wrote:

Add all new messages since the highest saved message-id.
Yes, it keeps track.
NG → ML direction seems to be flawless. [1]

This doesn’t seem to fit Martin’s analysis: there seem to be messages
missing on both sides, don’t they?

There were problems last year which were fixed by using the more
reliable feed. I think there was a brief problem in the NG → ML
direction very recently. If folks are trying to fix things, this
can be expected. Martin’s data are too broad. The problem we’re
addressing started around Feb. this year.

Ah, I see.

Can we really be sure that message ID’s follow this pattern?

The message-id used is numeric; local to the news service. It’s nothing
to do with those strings in the mail headers.

Ah, ok. And of course we don’t have a problem if mails reach the gateway in
a differnt order than they reached the mail exploder, have we?

I’m confident that if the problem was further downstream than nntp.rb,
there would be losses from sources other than ours.

Do we know that there aren’t?

The hardcore Usenet crew are really vocal when things go wrong with their
communication network. I don’t know that there aren’t problems but I
can’t contemplate that a serious snag with propagation would go unreported
for 4 months.

Yeah, sounds reasonable. I see, you already checked all those traps I was
imagining. That’s good to see - and unfortunate at the same time, because
it makes things harder. :slight_smile:

NNTP seems to be a sub-protocol of SMTP.

As I understand NNTP […] rather different in nature.

Maybe copying is not sufficient, maybe some header contents have to be
massaged in order to fit with NNTP.

<from RFC 1036>
“The USENET News standard is more restrictive than the Internet standard,
placing additional requirements on each message and forbidding use of
certain Internet features. However, it should always be possible to use
a tool expecting an Internet message to process a news message.”
</>

Well, yes. But that’s just the message format, which seems indeed a sub
format of SMTP. But NNTP (and SMTP) is more than that.

Do you mean the selective transmission works only for mail → news and
not
mail → mail? Now I’m confused here…

Not by me, surely :wink:

Now I don’t even know who confused me. Was it me? :-)))

All ML members post to a single recipient ruby-talk@ruby-lang.org. It’s
up to the bulk mailer where it goes next. The g/way is just one member.
One scenario is that the missing messages are not getting to the gateway.
If they did, they would be forwarded to c.l.ruby like the rest.
That’s my favourite suspicion and could easily be disproved by a single
log entry for a message that failed to make the NG.

With this single piece of evidence (which shouldn’t be so difficult to
obtain …) we have instant progress rather than looking at every other
non-possibility (which I, and others, have already done).
If the log says a phantom message has been forwarded, it cuts the
upstream right out of the equation and IMHO leaves nntp.rb only.
Debugging is a lot easier when you know you’re focusing on the right area
and it’s a pointless waste of time when you strongly suspect that the
problem lies elsewhere.

Definitely! So we’ll wait and seen what we can learn from logs.

Regards

robert

Hi –

All ML members post to a single recipient ruby-talk@ruby-lang.org. It’s
up to the bulk mailer where it goes next. The g/way is just one member.
One scenario is that the missing messages are not getting to the gateway.
If they did, they would be forwarded to c.l.ruby like the rest.
That’s my favourite suspicion and could easily be disproved by a single
log entry for a message that failed to make the NG.

With this single piece of evidence (which shouldn’t be so difficult to
obtain …) we have instant progress rather than looking at every other
non-possibility (which I, and others, have already done).
If the log says a phantom message has been forwarded, it cuts the
upstream right out of the equation and IMHO leaves nntp.rb only.
Debugging is a lot easier when you know you’re focusing on the right area
and it’s a pointless waste of time when you strongly suspect that the
problem lies elsewhere.

Here’s a bit of an IRC exchange I had with Dennis in April which
establishes with certainty that my messages are being received from
ruby-talk and forwarded through the gateway (prior to vanishing). I
sent a message while we were chatting and Dennis was monitoring the
logs:

ok, i’ve replied to an existing msg (that’s where it’s been
not working)
14:58:25: mail2news | Forwarding Re: Variable names
that was the one?
yes

(Sorry I didn’t post this sooner – I was certain Dennis and I had
already checked this, but I couldn’t find anything in our email and
only just now remembered that some of our discussion had been on IRC.)

David

···

On Tue, 1 Jun 2004, daz wrote:


David A. Black
dblack@wobblini.net

David A. Black wrote:

Hi –

“D” == David A Black dblack@wobblini.net writes:

Sorry everyone, testing again… small meaningless tweak to
headers…

What is the difference : the header Sender: ?

Yes. I’m not claiming it’s an intelligent idea :slight_smile: Just trying
everything, since I have no idea what the problem is.

What, if any, are the differences in headers between messages
that get through and those that don’t?

Hal

···

On Tue, 1 Jun 2004, ts wrote:

Hi,

···

— Jeff Mitchell quixoticsycophant@yahoo.com wrote:

I do not know if this is related or relevant, but I
often
receive messages with the last ~89 characters
repeated.

Me too. I always wondered if I was the only one :slight_smile:

Cheers,
Joao


Do you Yahoo!?
Friends. Fun. Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/

“daz” schrieb:

Robert Klemme wrote:

[…] And of course we don’t have a problem if mails reach the gateway in
a differnt order than they reached the mail exploder, have we?

Definitely not. In ML to NG direction, all mails should be mirrored
unconditionally to Usenet. mail2news.rb has no checks for sequence.
A “X-rubymirror: yes” header is added so that when news2mail.rb reads it
from the news service, it will not be sent to ruby-talk ML.

<from RFC 1036>
“The USENET News standard is more restrictive than the Internet standard,
placing additional requirements on each message and forbidding use of
certain Internet features. However, it should always be possible to use
a tool expecting an Internet message to process a news message.”
</>

Well, yes. But that’s just the message format, which seems indeed a sub
format of SMTP. But NNTP (and SMTP) is more than that.

I think I’m trying to make a distinction between Usenet and NNTP.
The important factor being that Usenet servers must pass messages
without modification of headers or content to the next subscribed
server. The entry point for us is TU-Berlin’s service. The g/way
could send complete garbage out, but it should still be propagated
around the chain. Of course, few news servers will accept it for
presentation to their users, so it won’t appear. Since I can
mirror a message that TU-Berlin “can’t”, proves that the message
is valid in the judgment of my news service and any that received
it later.
If someone is imposing censorship on Usenet, I can promise you that
there will be an “International Incident” which you will be able to
read about in Der Spiegel next week ;-))

Realistically, it’s not going to be that.

Now I don’t even know who confused me. Was it me? :-)))

At the time you wrote, you were considering the new problem
in the other direction as well and trying to reconcile the two.
I’m ignoring that totally, for now, to avoid confusion.
BTW: That post (your long reply to my long reply) was one that
didn’t get to the ML. So the new problem continues. Hmm…

daz

···

From: Robert Klemme:

Hi –

David A. Black wrote:

[…] testing again… small meaningless tweak to headers…

Mirrored using mail2news.rb (the version you sent me - unmodified).

Worked as I expected, so that rules out nntp.rb & headers.

Took your message out of my inbox (from ML).
Windows box, so newlines are different.
I might gsub them and send again.

OK… so the appearance of that msg on the newgroup was because you
mirrored it? (I just want to make sure and not get my hopes up :slight_smile:

There was some evidence in the headers …

… by the same evidence, I am exposed as an outrageous liar in
stating that mail2news.rb was unmodified. I meant the logic :confused:

Relating to newlines; DOS nl (“\r\n”) are correct for
mail/news message standards. In fact, mail2news gsub!s
Unix (“\r”) to DOS before sending.

daz

···

From: David A. Black

On Tue, 1 Jun 2004, daz wrote:

Hi –

···

On Tue, 1 Jun 2004, Hal Fulton wrote:

David A. Black wrote:

Hi –

On Tue, 1 Jun 2004, ts wrote:

“D” == David A Black dblack@wobblini.net writes:

Sorry everyone, testing again… small meaningless tweak to
headers…

What is the difference : the header Sender: ?

Yes. I’m not claiming it’s an intelligent idea :slight_smile: Just trying
everything, since I have no idea what the problem is.

What, if any, are the differences in headers between messages
that get through and those that don’t?

None that I’ve been able to spot. For samples: ruby-talk 101880 got
through (thread initiated by me), while 101837 didn’t (reply to
someone on the list). The only difference I can see (other than
different timestamp and message ID and such) is that the latter has an
In-Reply-To: header.

David


David A. Black
dblack@wobblini.net

If this gets through to comp.lang.ruby, then it’s possible that the
gateway has been fixed…

···


David A. Black
dblack@wobblini.net

“daz” dooby@d10.karoo.co.uk schrieb im Newsbeitrag
news:003201c447f3$7bfb2c60$b3a1f9d5@vanna…

BTW: That post (your long reply to my long reply) was one that
didn’t get to the ML. So the new problem continues. Hmm…

Are you saying that although D&D tackled one problem we seem to have
another problem here? Is it

old problem = mails don’t get through to news
new problem = news don’t get through to mail

?

Kind regards

robert

[“David A. Black” dblack@wobblini.net, 2004-05-31 18.25 CEST]
[…]

None that I’ve been able to spot. For samples: ruby-talk 101880 got
through (thread initiated by me), while 101837 didn’t (reply to
someone on the list). The only difference I can see (other than
different timestamp and message ID and such) is that the latter has an
In-Reply-To: header.

[ruby-talk:101837] has a newline in the subject line.

···

"daz" <dooby@d10.karoo.co.uk> schrieb im Newsbeitrag
news:003201c447f3$7bfb2c60$b3a1f9d5@vanna...

BTW: That post (your long reply to my long reply) was one that
didn't get to the ML. So the new problem continues. Hmm...

Are you saying that although D&D tackled one problem we seem to have
another problem here? Is it

Well, for me, your response to "daz" <dooby@d10.karoo.co.uk> was
[ruby-talk:101969]

    Message-Id: <2i30srFifc29U1@uni-berlin.de>

Guy Decoux

[Carlos angus@quovadis.com.ar, 2004-05-31 19.57 CEST]

[ruby-talk:101837] has a newline in the subject line.

BTW, here are the latests messages with newlines in their subjects (only
first line shown). Maybe you want to check if they reached the newsgroup, to
be sure if this is or isn’t the problem.

$ tail -50000 ~/Mail/ruby-talk | ruby -00 -n -e ’
next if $_ !~ /\AFrom /
if $_ !~ /^Newsgroups: / && $_ =~ /^Subject: [^\n]+\n\s/
puts $&.strip
$_ =~ /^From: .*?$/ and puts $&
$_ =~ /^X-Mail-Count: \d+/ and puts $&
puts
end’
X-Mail-Count: 101831

X-Mail-Count: 101837

X-Mail-Count: 101840

X-Mail-Count: 101843

X-Mail-Count: 101855

X-Mail-Count: 101863

X-Mail-Count: 101869

X-Mail-Count: 101888

···

Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: Simon Strandgaard neoneye@adslhome.dk
Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: “David A. Black” dblack@wobblini.net
Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: “David A. Black” dblack@wobblini.net
Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: Simon Strandgaard neoneye@adslhome.dk
Subject: elegant way to say "try this thing, one at a time, until condition
From: David Garamond lists@zara.6.isreserved.com
Subject: Re: elegant way to say "try this thing, one at a time, until
From: Gregory Millam walker@lethalcode.net
Subject: Re: elegant way to say "try this thing, one at a time, until condition
From: David Garamond lists@zara.6.isreserved.com
Subject: Re: elegant way to say "try this thing, one at a time, until condition
From: David Garamond lists@zara.6.isreserved.com

Hi –

“daz” dooby@d10.karoo.co.uk schrieb im Newsbeitrag
news:003201c447f3$7bfb2c60$b3a1f9d5@vanna…

BTW: That post (your long reply to my long reply) was one that
didn’t get to the ML. So the new problem continues. Hmm…

Are you saying that although D&D tackled one problem we seem to have
another problem here? Is it

Well, for me, your response to “daz” dooby@d10.karoo.co.uk was
[ruby-talk:101969]

Message-Id: <2i30srFifc29U1@uni-berlin.de>

I think that was a different one. The one that seems to be missing started
like this (as cut-and-pasted from gnus):

···

On Wed, 2 Jun 2004, ts wrote:

Subject: Re: separating ruby-talk from comp.lang.ruby?
Newsgroups: comp.lang.ruby
Date: Mon, 31 May 2004 13:45:01 +0200

I’m not proficient with NNTP so I’ll just try to throw some common sense in.
Maybe we can dig up the reason in a joined efford. Read on…

David


David A. Black
dblack@wobblini.net

“ts” decoux@moulon.inra.fr schrieb im Newsbeitrag
news:200406011631.i51GVXg12114@moulon.inra.fr

“daz” dooby@d10.karoo.co.uk schrieb im Newsbeitrag
news:003201c447f3$7bfb2c60$b3a1f9d5@vanna…

BTW: That post (your long reply to my long reply) was one that
didn’t get to the ML. So the new problem continues. Hmm…

Are you saying that although D&D tackled one problem we seem to have
another problem here? Is it

Well, for me, your response to “daz” dooby@d10.karoo.co.uk was
[ruby-talk:101969]

Message-Id: <2i30srFifc29U1@uni-berlin.de>

I think he means my posting from 31.5. 13:45 that starts with “I’m not
proficient with NNTP so I’ll …”. I don’t see that on ruby-talk. Do
you?

Regards

robert

[Carlos angus@quovadis.com.ar, 2004-05-31 19.57 CEST]

[ruby-talk:101837] has a newline in the subject line.

BTW, here are the latests messages with newlines in their subjects (only
first line shown). Maybe you want to check if they reached the newsgroup, to
be sure if this is or isn’t the problem.

Well, technically it’s ‘folding’, used to ensure that headers doesn’t
hit the SMTP linelength limit. But maybe one of the gateway scripts
doesn’t like that?

I don’t know anything about news, but if it’s based on SMTP, I’d think
that it should handle it allright. On the other hand, if that’s not
the case, that could be the problem.

···

On Tue, Jun 01, 2004 at 03:21:13AM +0900, Carlos wrote:

$ tail -50000 ~/Mail/ruby-talk | ruby -00 -n -e ’
next if $_ !~ /\AFrom /
if $_ !~ /^Newsgroups: / && $_ =~ /^Subject: [^\n]+\n\s/
puts $&.strip
$_ =~ /^From: .*?$/ and puts $&
$_ =~ /^X-Mail-Count: \d+/ and puts $&
puts
end’
Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: Simon Strandgaard neoneye@adslhome.dk
X-Mail-Count: 101831

Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: “David A. Black” dblack@wobblini.net
X-Mail-Count: 101837

Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: “David A. Black” dblack@wobblini.net
X-Mail-Count: 101840

Subject: Re: [BUG?] Behavior of application changes when adding non-relevant
From: Simon Strandgaard neoneye@adslhome.dk
X-Mail-Count: 101843

Subject: elegant way to say "try this thing, one at a time, until condition
From: David Garamond lists@zara.6.isreserved.com
X-Mail-Count: 101855

Subject: Re: elegant way to say "try this thing, one at a time, until
From: Gregory Millam walker@lethalcode.net
X-Mail-Count: 101863

Subject: Re: elegant way to say "try this thing, one at a time, until condition
From: David Garamond lists@zara.6.isreserved.com
X-Mail-Count: 101869

Subject: Re: elegant way to say "try this thing, one at a time, until condition
From: David Garamond lists@zara.6.isreserved.com
X-Mail-Count: 101888


Thomas
beast@system-tnt.dk

Hi –

Carlos angus@quovadis.com.ar writes:

[Carlos angus@quovadis.com.ar, 2004-05-31 19.57 CEST]

[ruby-talk:101837] has a newline in the subject line.

BTW, here are the latests messages with newlines in their subjects (only
first line shown). Maybe you want to check if they reached the newsgroup, to
be sure if this is or isn’t the problem.

Thanks for the info, but that’s not it. It appears to be
sender-specific; there are certain senders (I’m one of them) for whom
there is 100% failure going from the mailing list to the newsgroup,
regardless of what thread our messages are in.

David

···


David A. Black
dblack@wobblini.net

I think he means my posting from 31.5. 13:45 that starts with "I'm not
proficient with NNTP so I'll ...". I don't see that on ruby-talk. Do
you?

No, I don't see it

Guy Decoux

Thomas Fini Hansen wrote:

···

On Tue, Jun 01, 2004 at 03:21:13AM +0900, Carlos wrote:

[Carlos angus@quovadis.com.ar, 2004-05-31 19.57 CEST]

[ruby-talk:101837] has a newline in the subject line.

BTW, here are the latests messages with newlines in their subjects (only
first line shown). Maybe you want to check if they reached the newsgroup, to
be sure if this is or isn’t the problem.

Well, technically it’s ‘folding’, used to ensure that headers doesn’t
hit the SMTP linelength limit. But maybe one of the gateway scripts
doesn’t like that?

I don’t know anything about news, but if it’s based on SMTP, I’d think
that it should handle it allright. On the other hand, if that’s not
the case, that could be the problem.

Guy spotted a problem in the past with “immediate”[1] folding:
http://www.ruby-talk.org/85857

Many headers fold. They’re O.K.

daz

[1] First line is newline followed by continuation line(s).