TCPSocket doesn't detect remote disconnection inmediatelly

ruby-talk
1

Hi, I'm experimenting an annoying issue when a server disconnects a TCP
connection established by a Ruby client.

Let me start with an example of what I consider the expected behavior:

a) Using Linux "telnet" command:

- First connect to google.com:

~$ telnet google.com 80

  Trying 209.85.229.106...
  Connected to google.com.
  Escape character is '^]'.

- Then write something ("hello\n"):

  hello

- So I receive a 400 response:

  HTTP/1.0 400 Bad Request
  [...]

- And a warning:

  "Connection closed by foreign host."

- telnet exits with status 1 due to the remote disconnection error.

b) Using Ruby 1.9.1 TCPSocket:

- First connect:

  > require "socket"
  true
  > socket = TCPSocket.new("google.com", 80)
  #<TCPSocket:0x00000001010288>

- Now netstat shows the connection as ESTABLISHED (ok).

- Write "hello\n" into the socket:

  > socket.puts "hello"
  nil

- I receive the same 400 response from the www server. Now netstat shows the
connection as CLOSE_WAIT (OK?).

- Write "hello\n" again (IMHO it should fail):

  > socket.puts "hello"
  nil <---- didn't raise !!!

- Doing a ngrep I see that irb has sent "hello\n" over the previous TCP
connection but this time it doesn't receive response from the www server. This
is, irb is using a closed connection!:

  T 2010/01/12 20:12:04.072705 192.168.1.10:35272 -> 209.85.229.105:80 [AP]
  hello

···

#

- But after this new "puts" now netstat shows nothing! (no connection => OK).

- Repeat again:

  > socket.puts "hello"
  Errno::EPIPE: Broken pipe
        from (irb):6:in `write'
        from (irb):6:in `puts'
        from (irb):6
        from /usr/bin/irb:12:in `<main>'

- Now it raises!

Why didn't Ruby raise in the second "puts" attemp??? the server already closed
the connection! Why does Ruby realize of it after a new retry?

Well, this issue causes one request lost (the second one) since there is no
way to know if the server accepted or dropped the request.

Is it a bug? is there a way to check the socket connection before sending
(loosing in fact) a request? (TCPSocket#closed? is not valid here as it just
returns true in case *we* have closed a socket).

Thanks a lot for any help.

--
Iñaki Baz Castillo <ibc@aliax.net>

2

Iñaki Baz Castillo wrote:

Hi, I'm experimenting an annoying issue when a server disconnects a TCP
connection established by a Ruby client.

Try writing a C program which uses the Unix socket API directly. It will
be instructive. The book "Unix Network Programming (vol 1)" by the late
Richard Stevens is an excellent authoritative tome.

···

--
Posted via http://www.ruby-forum.com/\.

3

I'll do it. But it would be great if at least I could know in advance if what
I described is the expected behavior :slight_smile:

···

El Martes, 12 de Enero de 2010, Brian Candler escribió:

Iñaki Baz Castillo wrote:
> Hi, I'm experimenting an annoying issue when a server disconnects a TCP
> connection established by a Ruby client.

Try writing a C program which uses the Unix socket API directly. It will
be instructive. The book "Unix Network Programming (vol 1)" by the late
Richard Stevens is an excellent authoritative tome.

--
Iñaki Baz Castillo <ibc@aliax.net>

4

Well, I've not tested in C yet, but I confirm that in PHP the same occurs :slight_smile:

···

El Martes, 12 de Enero de 2010, Iñaki Baz Castillo escribió:

El Martes, 12 de Enero de 2010, Brian Candler escribió:
> Iñaki Baz Castillo wrote:
> > Hi, I'm experimenting an annoying issue when a server disconnects a TCP
> > connection established by a Ruby client.
>
> Try writing a C program which uses the Unix socket API directly. It will
> be instructive. The book "Unix Network Programming (vol 1)" by the late
> Richard Stevens is an excellent authoritative tome.

I'll do it. But it would be great if at least I could know in advance if
what I described is the expected behavior :slight_smile:

--
Iñaki Baz Castillo <ibc@aliax.net>

5

Long story short, what you're seeing with Ruby is not a bug, but
rather how TCP sockets work. TCP sockets are bi-directional and when
one side "closes" the connection, they are not actually causing the
TCP session to tear down as telnet implies, but rather is informing
the other end of the connection that they have no more data to send.
The receive direction is still technically open via TCP, although the
HTTP protocol specifies no more communication is possible.

I also recommend "Unix Network Programming (vol 1)" by Stevens if you
want to understand the details of sockets and network communication.
Since network programming utilizes much of the same API constructs as
file IO, many newbie network programmers make these sorta mistakes
because they expect network sockets to behave like file descriptors
when they do not.

···

On Tue, Jan 12, 2010 at 1:21 PM, Iñaki Baz Castillo <ibc@aliax.net> wrote:

El Martes, 12 de Enero de 2010, Brian Candler escribió:

Iñaki Baz Castillo wrote:
> Hi, I'm experimenting an annoying issue when a server disconnects a TCP
> connection established by a Ruby client.

Try writing a C program which uses the Unix socket API directly. It will
be instructive. The book "Unix Network Programming (vol 1)" by the late
Richard Stevens is an excellent authoritative tome.

I'll do it. But it would be great if at least I could know in advance if what
I described is the expected behavior :slight_smile:

--
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
"carpe diem quam minimum credula postero"

6

Long story short, what you're seeing with Ruby is not a bug, but
rather how TCP sockets work. TCP sockets are bi-directional and when
one side "closes" the connection, they are not actually causing the
TCP session to tear down as telnet implies, but rather is informing
the other end of the connection that they have no more data to send.
The receive direction is still technically open via TCP, although the
HTTP protocol specifies no more communication is possible.

Understood, thanks a lot.

Then I wonder if there is some way to detect that the remote has closed its
side. When it occurs the remote sends a TCP segment with ACK+FIN flags
enabled.

Is there a way to detect it in Ruby using TCPSocket? if there some attribute
of the socket object that changes whe such ACK+FIN arrives? (however I've
tried most of the TCPSocket methods and found nothing).

The underlaying system of courses knows it as in case a new TCP segment is
sent (by the Ruby client) it's sent with big PUSH enabled, knowing that the
remote will reply with RESET flag enabled. Is the Ruby TCPSocket aware that
its message is sent with PUSH flag enabled? if so it could raise something
(optionally) so I know that I'll get no response and could reconnect and
retry. Is it possible?

Humm, I think that all this stuff is done at kernel TCP layer so the socket
user has nothing to deal with... am I right?

I also recommend "Unix Network Programming (vol 1)" by Stevens if you
want to understand the details of sockets and network communication.
Since network programming utilizes much of the same API constructs as
file IO, many newbie network programmers make these sorta mistakes
because they expect network sockets to behave like file descriptors
when they do not.

Sure. Thanks a lot.

···

El Martes, 12 de Enero de 2010, Aaron Turner escribió:

--
Iñaki Baz Castillo <ibc@aliax.net>

7

Long story short, what you're seeing with Ruby is not a bug, but
rather how TCP sockets work. TCP sockets are bi-directional and when
one side "closes" the connection, they are not actually causing the
TCP session to tear down as telnet implies, but rather is informing
the other end of the connection that they have no more data to send.
The receive direction is still technically open via TCP, although the
HTTP protocol specifies no more communication is possible.

Understood, thanks a lot.

Then I wonder if there is some way to detect that the remote has closed its
side. When it occurs the remote sends a TCP segment with ACK+FIN flags
enabled.

Is there a way to detect it in Ruby using TCPSocket? if there some attribute
of the socket object that changes whe such ACK+FIN arrives? (however I've
tried most of the TCPSocket methods and found nothing).

There are two ways of doing this:

1) The application running over TCP. In this case the HTTP 400 error
is telling you that the web server is done talking to you and no
future requests will be processed.

2) The other way is when you read, did you get an EOF error? EOF ==
server has called shutdown() on the socket.

Ruby sockets behave just like C sockets. It's just the Ruby language
wrapping them. There's nothing "special" about ruby sockets. What
this means is that the best resource isn't the ruby docs- they just
document the ruby API. To understand how sockets work, read Richard
Stevens or start doing google searches on things like "tcp detecting
close".

The underlaying system of courses knows it as in case a new TCP segment is
sent (by the Ruby client) it's sent with big PUSH enabled, knowing that the
remote will reply with RESET flag enabled. Is the Ruby TCPSocket aware that
its message is sent with PUSH flag enabled? if so it could raise something
(optionally) so I know that I'll get no response and could reconnect and
retry. Is it possible?

The PSH (Push) flag has nothing to do with it. PSH just tells the
receiving host that the TCP packet has interesting data for the
application and it should quickly "push it up" to the application for
processing.

Humm, I think that all this stuff is done at kernel TCP layer so the socket
user has nothing to deal with... am I right?

Depends on what you mean by "all this stuff". If you mean making TCP
a reliable, bi-directional stream & session based programming
construct then yes, the kernel does that for you. If you mean dealing
with knowing when you can read/write to the socket, then no, that's
your job. You the programmer have a lot of control over tcp sockets
and are expected to know how to use them properly. As I said last
time, while TCP sockets utilize a very similar API as File IO, but
they are more complicated and you need to know how to use them.

Good luck.

···

On Tue, Jan 12, 2010 at 2:37 PM, Iñaki Baz Castillo <ibc@aliax.net> wrote:

El Martes, 12 de Enero de 2010, Aaron Turner escribió:

--
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
"carpe diem quam minimum credula postero"

8

> Is there a way to detect it in Ruby using TCPSocket? if there some
> attribute of the socket object that changes whe such ACK+FIN arrives?
> (however I've tried most of the TCPSocket methods and found nothing).

There are two ways of doing this:

1) The application running over TCP. In this case the HTTP 400 error
is telling you that the web server is done talking to you and no
future requests will be processed.

I'm not talking with a HTTP server, but with a SIP server. When the SIP server
restarts its sends a TCP packet with FIN flag to me (Ruby client).

2) The other way is when you read, did you get an EOF error? EOF ==
server has called shutdown() on the socket.

I have good news for myself :slight_smile:

I've a thread reading the responses received in the socket:

  sip_response = @@socket.readline("\r\n\r\n")

I use "\r\n\r\n" (double CRLF) as a SIP response with no body ends with CRLF
CRLF (as HTTP I think).

However when the remote disconnects I didn't get the exception EOFError.

The very good new is that it's a bug in my code! In the rescue block I did a
typo so I wasn't invoking the reconnection command :slight_smile:

Really thanks for the good help received in this thread :slight_smile:

···

El Miércoles, 13 de Enero de 2010, Aaron Turner escribió:

--
Iñaki Baz Castillo <ibc@aliax.net>