# Problem with socket.recv() **URL:** \ **Category:** ruby-talk\ **Created:** [29 October 2007 19:54 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763 "2007-10-29T19:54:17Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Frank\_Preiswerk](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frank\_Preiswerk](https://rubytalk.org/u/Frank_Preiswerk)\ **Post date:** [29 October 2007 19:54 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/1 "2007-10-29T19:54:17Z")
Hi everyone, I'm currently developing a Ruby TCP client and have a problem with socket.recv(). After establishing a connection with my server, I exchange a couple of messages until the server begins to do some calculations based on the client's request that take about 10 seconds. After the server finishes, it sends back the result. Now my problem is that "data = socket.recv(size)" seems to behave non-blocking for some reason. It does not wait for the 10 seconds but rather executes immediately and "data" remains empty. When I add "sleep(10)" before, it works... Any hints would be great. Thanks in advance, Frank ---
**Author:** ![Joel\_VanderWerf1](https://avatars.discourse-cdn.com/v4/letter/j/94ad74/32.png) [@Joel\_VanderWerf1](https://rubytalk.org/u/Joel_VanderWerf1)\ **Post date:** [29 October 2007 23:21 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/2 "2007-10-29T23:21:01Z")
Frank Preiswerk wrote: > Hi everyone, > > I'm currently developing a Ruby TCP client and have a problem with > socket.recv(). > After establishing a connection with my server, I exchange a couple of > messages until the server begins to do some calculations based on the > client's request that take about 10 seconds. After the server finishes, it > sends back the result. Now my problem is that "data = socket.recv(size)" > seems to behave non-blocking for some reason. It does not wait for the 10 > seconds but rather executes immediately and "data" remains empty. When I add > "sleep(10)" before, it works... #recv gives you data as soon as there is data available, not necessarily waiting for size bytes. You may want to buffer it in your ruby code. > **···** > > -- >        vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407 ---
**Author:** ![7stud](https://avatars.discourse-cdn.com/v4/letter/7/57b2e6/32.png) [@7stud](https://rubytalk.org/u/7stud)\ **Post date:** [30 October 2007 09:32 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/3 "2007-10-30T09:32:31Z")
Frank Preiswerk wrote: > Now my problem is that "data = socket.recv(size)" > seems to behave non-blocking for some reason. It does not wait... I just wanted to add that as far as I know, you can't count on recv(1012) to read all the bytes sent when, say, only 500 bytes are sent. recv() may only read 1 byte the first time it is called or it may read 200 bytes, or it may read all 500 bytes. As a consequence, in order to get all the data you want out of the socket, you have to loop over recv() and either: 1) count the number of bytes that recv() read using String#length, and stop after a certain number of bytes, or 2) keep looping until recv encounters eof(i.e. a blank string is sent when the server closes the socket) and you have read all the data. In your case, you say you want to block until all the data is read, so you could use read() and specify a large limit that is bigger than the length of the data that will be sent, and simply write: all\_data = read(8196) #do something with all\_data My tests show read() will block and wait until it either reads all 8196 bytes or the server closes the socket. > **···** > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Roger\_Pack4](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@Roger\_Pack4](https://rubytalk.org/u/Roger_Pack4)\ **Post date:** [30 October 2007 17:09 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/4 "2007-10-30T17:09:55Z")
Make sure you flush on the sending side, as it might not send it all immediately (uses the nagle algorithm). Frank Preiswerk wrote: > **···** > > > Hi everyone, > > > > I'm currently developing a Ruby TCP client and have a problem with > > socket.recv(). > > After establishing a connection with my server, I exchange a couple of > > messages until the server begins to do some calculations based on the > > client's request that take about 10 seconds. After the server finishes, > > it > > sends back the result. Now my problem is that "data = socket.recv(size)" > > seems to behave non-blocking for some reason. It does not wait for the > > 10 > > seconds but rather executes immediately and "data" remains empty. When I > > add > > "sleep(10)" before, it works... > > > > Any hints would be great. > > > > Thanks in advance, > > Frank > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Frank\_Preiswerk](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frank\_Preiswerk](https://rubytalk.org/u/Frank_Preiswerk)\ **Post date:** [30 October 2007 07:22 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/5 "2007-10-30T07:22:48Z")
Yes, I understand that socket.recv() reads UP TO size bytes, but doesn't necessarily wait. How would you buffer the data? Maybe my server indeed sends some data when I think it doesn't, so my client receives a bit of data and thinks that's it. Buffering should do the trick here. But how? Thanks! > **···** > > On 10/30/07, Joel VanderWerf \ wrote: > > > Frank Preiswerk wrote: > > \> Hi everyone, > > \> > > \> I'm currently developing a Ruby TCP client and have a problem with > > \> socket.recv(). > > \> After establishing a connection with my server, I exchange a couple of > > \> messages until the server begins to do some calculations based on the > > \> client's request that take about 10 seconds. After the server finishes, > > it > > \> sends back the result. Now my problem is that "data = socket.recv(size)" > > \> seems to behave non-blocking for some reason. It does not wait for the > > 10 > > \> seconds but rather executes immediately and "data" remains empty. When I > > add > > \> "sleep(10)" before, it works... > > > > #recv gives you data as soon as there is data available, not necessarily > > waiting for size bytes. > > > > You may want to buffer it in your ruby code. > > > > -- > >        vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407 ---
**Author:** ![Frank\_Preiswerk](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frank\_Preiswerk](https://rubytalk.org/u/Frank_Preiswerk)\ **Post date:** [30 October 2007 21:40 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/6 "2007-10-30T21:40:33Z")
First of all thanks a lot for all your inputs! Using read instead of recv indeed leads to blocking (like I want it to), at least when sending a bigger amount of data. But only receiving say 2 bytes using socket.read(2) somehow does not work, probably because I don't flush on the sender side. Compared to socket.revc, it does not skip immediately, but blocks until it times out, although the sender has sent the 2 bytes. It receives nothing in the end. I'll have to investigate a bit more. Seems like I did not do my homework in understanding how sockets work in detail (especially on the server side which I wrote in C++/QT). Thanks for the fruitful discussion, Frank > **···** > > On 10/30/07, Roger Pack \ wrote: > > > Make sure you flush on the sending side, as it might not send it all > > immediately (uses the nagle algorithm). > > > > Frank Preiswerk wrote: > > \> Hi everyone, > > \> > > \> I'm currently developing a Ruby TCP client and have a problem with > > \> socket.recv(). > > \> After establishing a connection with my server, I exchange a couple of > > \> messages until the server begins to do some calculations based on the > > \> client's request that take about 10 seconds. After the server finishes, > > \> it > > \> sends back the result. Now my problem is that "data = socket.recv(size)" > > \> seems to behave non-blocking for some reason. It does not wait for the > > \> 10 > > \> seconds but rather executes immediately and "data" remains empty. When I > > \> add > > \> "sleep(10)" before, it works... > > \> > > \> Any hints would be great. > > \> > > \> Thanks in advance, > > \> Frank > > > > -- > > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). > > # -- > > Frank Preiswerk > Breitestrasse 50 > 4132 Muttenz > > phone: +41 61 463 80 30 > mobile: +41 76 390 60 29 ---
**Author:** ![Francis\_Cianfrocca](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Francis\_Cianfrocca](https://rubytalk.org/u/Francis_Cianfrocca)\ **Post date:** [30 October 2007 08:09 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/7 "2007-10-30T08:09:41Z")
If the server closes the connection after sending the data, you can simply use socket#read. If you need to wait for the data and then process them as they comes in, use #select in a loop until all the data have been received. > **···** > > On 10/30/07, Frank Preiswerk \ wrote: > > > Yes, I understand that socket.recv() reads UP TO size bytes, but doesn't > > necessarily wait. > > How would you buffer the data? > > > > Maybe my server indeed sends some data when I think it doesn't, so my > > client > > receives a bit of data and thinks that's it. Buffering should do the trick > > here. But how? ---
**Author:** ![Joel\_VanderWerf1](https://avatars.discourse-cdn.com/v4/letter/j/94ad74/32.png) [@Joel\_VanderWerf1](https://rubytalk.org/u/Joel_VanderWerf1)\ **Post date:** [30 October 2007 16:47 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/8 "2007-10-30T16:47:15Z")
Frank Preiswerk wrote: > Yes, I understand that socket.recv() reads UP TO size bytes, but doesn't > necessarily wait. > How would you buffer the data? Here's an idea that's a little different from what you're asking for (it assumes the data stream is chunked using 4 byte length fields, rather than knowing the length in advance), but it shows a working buffer implementation. (See also [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/177704\](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/177704%5C)) module Messageable    LEN\_LEN = 4    def recv\_message      if (data = recv(LEN\_LEN))        if data.empty?          nil        else          len = Message::Length.new(data).value # for extra safety: # if len \> MAXLEN # raise MessageLengthError, "MAXLEN exceeded: #{len} \> #{MAXLEN}" # end          msg = ""          part = nil          while (delta = len - msg.length) \> 0 and                (part = recv(delta)) and                not part.empty?            yield part if block\_given?            msg \<\< part          end          msg.empty? ? nil : msg        end      end    end end > **···** > > -- >        vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407 ---
**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\ **Post date:** [30 October 2007 08:30 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/9 "2007-10-30T08:30:04Z")
There is another option - even if the server does not close after sending: use #read with a given size limit. #read will block and return as soon data is available. Using #read without limit is a bad idea on a socket anyway because it has to read until EOS, as you indicated. DRb would be another option if both sides are written in Ruby. Kind regards robert > **···** > > 2007/10/30, Francis Cianfrocca \: > > > On 10/30/07, Frank Preiswerk \ wrote: > > \> > > \> Yes, I understand that socket.recv() reads UP TO size bytes, but doesn't > > \> necessarily wait. > > \> How would you buffer the data? > > \> > > \> Maybe my server indeed sends some data when I think it doesn't, so my > > \> client > > \> receives a bit of data and thinks that's it. Buffering should do the trick > > \> here. But how? > > > > If the server closes the connection after sending the data, you can simply > > use socket#read. If you need to wait for the data and then process them as > > they comes in, use #select in a loop until all the data have been received. ---
**Author:** ![7stud](https://avatars.discourse-cdn.com/v4/letter/7/57b2e6/32.png) [@7stud](https://rubytalk.org/u/7stud)\ **Post date:** [30 October 2007 09:09 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/10 "2007-10-30T09:09:22Z")
Robert Klemme wrote: > > If the server closes the connection after sending the data, you can simply > > use socket#read. If you need to wait for the data and then process them as > > they comes in, use #select in a loop until all the data have been received. > > There is another option - even if the server does not close after > sending: use #read with a given size limit. #read will block and > return as soon data is available. That's not what I'm seeing. What I see is: read() blocks until it either receives the limit number of bytes or eof is encountered(when the server closes the socket). In other words, read() does not return as soon as data is available if the amount of bytes read is less than the limit number of bytes. On the other hand, recv() returns whatever it reads immediately. Here is the code: #server: > **···** > > > 2007/10/30, Francis Cianfrocca \: > > -------------- > require 'socket' > > port = 3030 > server = TCPServer.new('', port) > > while(conn = server.accept) #serves forever >   conn.print 'h' >   sleep(5) > >   conn.print 'ello world' >   conn.close > end > -------------- > > #client using recv(): > -------------------- > require 'socket' > > port = 3030 > socket = TCPSocket.new('localhost', port) > > all\_data = > > while true >   partial\_data = socket.recv(1012) >   puts partial\_data > >   if partial\_data.length == 0 >     break >   end > >   all\_data \<\< partial\_data > end > socket.close > > puts all\_data.join() > > --output:-- > h > ello world >                   #blank string returned when server closed socket > hello world > ------------------------ > > #client using read(): > ----------------- > require 'socket' > > port = 3030 > socket = TCPSocket.new('localhost', port) > > all\_data = > > while partial\_data = socket.read(1012) >   puts partial\_data >   all\_data \<\< partial\_data > end > socket.close > > puts all\_data.join() > > --output: -- > hello world > hello world > ---------------- > > So the way I see it: > > 1) If the amount of data sent by the server is less than the size limit > specified in read(), read() will block until the server closes the > socket. Therefore, the server must close the socket. > > 2) If the amount of data sent by the server is more than the size limit > specified in read(), and you just write: > > all\_data = read(1012) > puts data > > then you aren't reading all the data. You just get the first 1012 bytes. > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Francis\_Cianfrocca](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Francis\_Cianfrocca](https://rubytalk.org/u/Francis_Cianfrocca)\ **Post date:** [30 October 2007 09:44 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/11 "2007-10-30T09:44:30Z")
> That's not what I'm seeing. What I see is: read() blocks until it > either receives the limit number of bytes or eof is encountered(when the > server closes the socket). In other words, read() does not return as > soon as data is available if the amount of bytes read is less than the > limit number of bytes. On the other hand, recv() returns whatever it > reads immediately. Here is the code: ...snip... So the way I see it: > 1) If the amount of data sent by the server is less than the size limit > specified in read(), read() will block until the server closes the > socket. Therefore, the server must close the socket. > > 2) If the amount of data sent by the server is more than the size limit > specified in read(), and you just write: > > all\_data = read(1012) > puts data > > then you aren't reading all the data. You just get the first 1012 bytes. We know nothing about the protocol that the OP is using (because he hasn't told us yet). It may not be realistic for the client to have an expectation about how much data the server will send. #select may turn out to be the best choice. #read with no parameters works if the server closes the connection after sending. But as a general rule, that's a bad server design (ideally, the client should close the connection, for reasons relating to the TIME\_WAIT state). > **···** > > On 10/30/07, 7stud -- \ wrote: ---
**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\ **Post date:** [30 October 2007 10:16 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/12 "2007-10-30T10:16:19Z")
Yes, I was imprecise: when using #read(limit) the method will block until either limit bytes were sent or the server closes the connection; up to limit bytes will be returned. When the server closed the connection then the next #read will return nil indicating EOS. Thanks for catching that! Kind regards robert > **···** > > 2007/10/30, 7stud -- \: > > > Robert Klemme wrote: > > \> 2007/10/30, Francis Cianfrocca \: > > \>\> > > \>\> If the server closes the connection after sending the data, you can simply > > \>\> use socket#read. If you need to wait for the data and then process them as > > \>\> they comes in, use #select in a loop until all the data have been received. > > \> > > \> There is another option - even if the server does not close after > > \> sending: use #read with a given size limit. #read will block and > > \> return as soon data is available. > > > > That's not what I'm seeing. What I see is: read() blocks until it > > either receives the limit number of bytes or eof is encountered(when the > > server closes the socket). In other words, read() does not return as > > soon as data is available if the amount of bytes read is less than the > > limit number of bytes. On the other hand, recv() returns whatever it > > reads immediately. Here is the code: ---
**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\ **Post date:** [30 October 2007 10:19 UTC](https://rubytalk.org/t/problem-with-socket-recv/41763/13 "2007-10-30T10:19:06Z")
> We know nothing about the protocol that the OP is using (because he hasn't > told us yet). It may not be realistic for the client to have an expectation > about how much data the server will send. #select may turn out to be the > best choice. Quoting from the original posting: > Now my problem is that "data = socket.recv(size)" > seems to behave non-blocking for some reason. It does not wait for the 10 > seconds but rather executes immediately and "data" remains empty. When I add > "sleep(10)" before, it works... So it seems pretty clear that he knows how many bytes to expect. In which case I'd use #read because the code will be simpler than with #select. > #read with no parameters works if the server closes the connection after > sending. But as a general rule, that's a bad server design (ideally, the > client should close the connection, for reasons relating to the TIME\_WAIT > state). Agreed: the client should establish and close a connection. Kind regards robert > **···** > > 2007/10/30, Francis Cianfrocca \: