Anyone know what's up with TCPSocket#write_noblock actually blocking when sending a packet that exceeds the buffer size of the socket in 1.8.6 on Mac.
The behaviour seems slightly counter-productive.
/C
Christoffer Lernö wrote:
Anyone know what's up with TCPSocket#write_noblock
The method hasn't been invented yet?
···
--
Posted via http://www.ruby-forum.com/\.
How long does it block? I see ruby occasionally being delayed 10ms when using write_nonblock, but not repeatably for the same packet sizes. Probably my benchmark script is wrong.
Could you supply a benchmark script that illustrates your problem?
#write_nonblock first does rb_io_check_closed() which might call down to fseek(3) then write(2) before calling rb_io_set_nonblock(). I'm not even close to being a sockets expert though, I just read the man pages.
···
On Oct 13, 2007, at 14:55 , Christoffer Lernö wrote:
Anyone know what's up with TCPSocket#write_noblock actually blocking when sending a packet that exceeds the buffer size of the socket in 1.8.6 on Mac.
The behaviour seems slightly counter-productive.
--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars
cfp:~ > ruby -r io/wait -r yaml -e' y IO.instance_methods.grep(/non/) '
···
On Oct 13, 2007, at 4:20 PM, 7stud -- wrote:
The method hasn't been invented yet?
---
- read_nonblock
- write_nonblock
but it works for me on osx
can the OP post a non-working code example?
cheers.
a @ http://codeforpeople.com/
--
it is not enough to be compassionate. you must act.
h.h. the 14th dalai lama
That's rather harsh for a one letter typo.
···
On Oct 13, 2007, at 15:20 , 7stud -- wrote:
Christoffer Lernö wrote:
Anyone know what's up with TCPSocket#write_noblock
The method hasn't been invented yet?
--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars
Thanks. Know it "should work" made me try to analyze the situation. Here is an example code:
require 'socket'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
# Using a.accept will give a socket that does not exhibit problems,
# Only TCPServer#accept_nonblock seems to create this
# odd situation.
other = a.accept_nonblock
puts "Start Write"
# Lower values, say "X" * 2000, writes all in one sweep and does
# not cause it block
other.write_nonblock("X" * 300000)
puts "End Write"
It looks like write_nonblock together with accept_nonblock causes some issues.
Now I could be using these wrong as both are new (in 1.8.6?)
/Christoffer
···
On 14 Oct 2007, at 01:23, Eric Hodel wrote:
On Oct 13, 2007, at 14:55 , Christoffer Lernö wrote:
Anyone know what's up with TCPSocket#write_noblock actually blocking when sending a packet that exceeds the buffer size of the socket in 1.8.6 on Mac.
The behaviour seems slightly counter-productive.
How long does it block? I see ruby occasionally being delayed 10ms when using write_nonblock, but not repeatably for the same packet sizes. Probably my benchmark script is wrong.
Could you supply a benchmark script that illustrates your problem?
#write_nonblock first does rb_io_check_closed() which might call down to fseek(3) then write(2) before calling rb_io_set_nonblock(). I'm not even close to being a sockets expert though, I just read the man pages.
Eric Hodel wrote:
Christoffer Lern� wrote:
Anyone know what's up with TCPSocket#write_noblock
The method hasn't been invented yet?
That's rather harsh for a one letter typo.
Harsh? In what way?
I wasn't aware there was a similarly named method. I looked in the
index of pickaxe2, and there was no write_noblock method listed, and now
that I look again there is no write_nonblock method listed either. I
also checked the standard library documentation for BasicSocket, Socket,
and TCPSocket, and there was no write_noblock method listed, and now
that I look again, there is no write_nonblock method either. In fact,
there is no method that starts with 'write'. I also searched google and
came up with 0 hits for write_noblock, which has to be a first for me.
Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.
I couldn't reproduce with your this test using:
$ ruby -v
ruby 1.8.6 (2007-09-23 patchlevel 5000) [powerpc-darwin8.10.0]
I'll to try again tomorrow though, since I'm tired.
I experimented a bit with your benchmark, and found that writing a large string would only write as much as the socket buffer would accept (81660 bytes, in my case).
Maybe you need to set the TCP_NODELAY socket option?
socket.setsockopt Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1
I once made write_nonblock raise Errno::EAGAIN, but I accidentally deleted that change. I'll play some more tomorrow when I'm not so tired.
···
On Oct 14, 2007, at 24:13 , Christoffer Lernö wrote:
On 14 Oct 2007, at 01:23, Eric Hodel wrote:
On Oct 13, 2007, at 14:55 , Christoffer Lernö wrote:
Anyone know what's up with TCPSocket#write_noblock actually blocking when sending a packet that exceeds the buffer size of the socket in 1.8.6 on Mac.
The behaviour seems slightly counter-productive.
How long does it block? I see ruby occasionally being delayed 10ms when using write_nonblock, but not repeatably for the same packet sizes. Probably my benchmark script is wrong.
Could you supply a benchmark script that illustrates your problem?
#write_nonblock first does rb_io_check_closed() which might call down to fseek(3) then write(2) before calling rb_io_set_nonblock(). I'm not even close to being a sockets expert though, I just read the man pages.
Thanks. Know it "should work" made me try to analyze the situation. Here is an example code:
[...]
It looks like write_nonblock together with accept_nonblock causes some issues.
Now I could be using these wrong as both are new (in 1.8.6?)
--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars
Eric Hodel wrote:
Christoffer Lern� wrote:
Anyone know what's up with TCPSocket#write_noblock
The method hasn't been invented yet?
That's rather harsh for a one letter typo.
Harsh? In what way?
responding that way when you didn't know
I wasn't aware there was a similarly named method. I looked in the
index of pickaxe2, and there was no write_noblock method listed, and now
that I look again there is no write_nonblock method listed either. I
also checked the standard library documentation for BasicSocket, Socket,
and TCPSocket, and there was no write_noblock method listed, and now
that I look again, there is no write_nonblock method either. In fact,
there is no method that starts with 'write'. I also searched google and
came up with 0 hits for write_noblock, which has to be a first for me.
searching != knowing
besides, google pulls 'ruby write_noblock' up on the first hit
Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.
now you are being harsh again.
cfp:~ > PAGER=cat ri IO|grep non
pos, pos=, print, printf, putc, puts, read, read_nonblock,
write_nonblock
cfp:~ > PAGER=cat ri IO
-------------------------------------------------------------- Class: IO
Class +IO+ is the basis for all input and output in Ruby. An I/O
stream may be _duplexed_ (that is, bidirectional), and so may use
more than one native operating system stream.
Many of the examples in this section use class +File+, the only
standard subclass of +IO+. The two classes are closely associated.
As used in this section, _portname_ may take any of the following
forms.
* A plain string represents a filename suitable for the
underlying operating system.
* A string starting with ``+|+'' indicates a subprocess. The
remainder of the string following the ``+|+'' is invoked as a
process with appropriate input/output channels connected to it.
* A string equal to ``+|-+'' will create another Ruby instance as
a subprocess.
Ruby will convert pathnames between different operating system
conventions if possible. For instance, on a Windows system the
filename ``+/gumby/ruby/test.rb+'' will be opened as
``+\gumby\ruby\test.rb+''. When specifying a Windows-style filename
in a Ruby string, remember to escape the backslashes:
"c:\gumby\ruby\test.rb"
Our examples here will use the Unix-style forward slashes;
+File::SEPARATOR+ can be used to get the platform-specific
separator character.
I/O ports may be opened in any one of several different modes,
which are shown in this section as _mode_. The mode may either be a
Fixnum or a String. If numeric, it should be one of the operating
system specific constants (O_RDONLY, O_WRONLY, O_RDWR, O_APPEND and
so on). See man open(2) for more information.
If the mode is given as a String, it must be one of the values
listed in the following table.
Mode | Meaning
···
On Oct 13, 2007, at 6:31 PM, 7stud -- wrote:
On Oct 13, 2007, at 15:20 , 7stud -- wrote:
-----+--------------------------------------------------------
"r" | Read-only, starts at beginning of file (default mode).
-----+--------------------------------------------------------
"r+" | Read-write, starts at beginning of file.
-----+--------------------------------------------------------
"w" | Write-only, truncates existing file
> to zero length or creates a new file for writing.
-----+--------------------------------------------------------
"w+" | Read-write, truncates existing file to zero length
> or creates a new file for reading and writing.
-----+--------------------------------------------------------
"a" | Write-only, starts at end of file if file exists,
> otherwise creates a new file for writing.
-----+--------------------------------------------------------
"a+" | Read-write, starts at end of file if file exists,
> otherwise creates a new file for reading and
> writing.
-----+--------------------------------------------------------
"b" | (DOS/Windows only) Binary file mode (may appear with
> any of the key letters listed above).
The global constant ARGF (also accessible as $<) provides an
IO-like stream which allows access to all files mentioned on the
command line (or STDIN if no files are mentioned). ARGF provides
the methods +#path+ and +#filename+ to access the name of the file
currently being read.
------------------------------------------------------------------------
Includes:
---------
Enumerable(all?, any?, collect, detect, each_cons, each_slice,
each_with_index, entries, enum_cons, enum_slice, enum_with_index,
find, find_all, grep, group_by, include?, include_any?, index_by,
inject, injecting, map, max, member?, min, partition, reject,
select, sort, sort_by, sum, to_a, to_json, to_set, zip),
File::Constants()
Constants:
----------
SEEK_CUR: INT2FIX(SEEK_CUR)
SEEK_END: INT2FIX(SEEK_END)
SEEK_SET: INT2FIX(SEEK_SET)
Class methods:
--------------
for_fd, foreach, new, open, pipe, popen, read, readlines, select,
sysopen
Instance methods:
-----------------
<<, binmode, block_scanf, close, close_read, close_write, closed?,
each, each_byte, each_line, eof, eof?, fcntl, fileno, flush, fsync,
getc, gets, inspect, ioctl, isatty, lineno, lineno=, open, pid,
pos, pos=, print, printf, putc, puts, read, read_nonblock,
readbytes, readchar, readline, readlines, readpartial, reopen,
rewind, scanf, seek, soak_up_spaces, stat, sync, sync=, sysread,
sysseek, syswrite, tell, to_i, to_io, tty?, ungetc, write,
write_nonblock
i know ruby-core is *always* looking for people to contribute for docs. you certainly get what you pay for with bloody open source eh?
a @ http://codeforpeople.com/
--
it is not enough to be compassionate. you must act.
h.h. the 14th dalai lama
Could this be due to me running
ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?
The funny thing was that only the socket returned by accept_nonblock had this behaviour, that is
require 'socket'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
other = a.accept
puts "Start Write"
other.write_nonblock("X" * 300000)
puts "End Write"
Works.
require 'socket'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
other = a.accept_nonblock
puts "Start Write"
other.write_nonblock("X" * 300000)
puts "End Write"
Doesn't work.
What's so special about a socket returned by accept_nonblock?
/Christoffer
···
On 14 Oct 2007, at 09:58, Eric Hodel wrote:
On Oct 14, 2007, at 24:13 , Christoffer Lernö wrote:
On 14 Oct 2007, at 01:23, Eric Hodel wrote:
On Oct 13, 2007, at 14:55 , Christoffer Lernö wrote:
Anyone know what's up with TCPSocket#write_noblock actually blocking when sending a packet that exceeds the buffer size of the socket in 1.8.6 on Mac.
The behaviour seems slightly counter-productive.
How long does it block? I see ruby occasionally being delayed 10ms when using write_nonblock, but not repeatably for the same packet sizes. Probably my benchmark script is wrong.
Could you supply a benchmark script that illustrates your problem?
#write_nonblock first does rb_io_check_closed() which might call down to fseek(3) then write(2) before calling rb_io_set_nonblock(). I'm not even close to being a sockets expert though, I just read the man pages.
Thanks. Know it "should work" made me try to analyze the situation. Here is an example code:
[...]
It looks like write_nonblock together with accept_nonblock causes some issues.
Now I could be using these wrong as both are new (in 1.8.6?)
I couldn't reproduce with your this test using:
$ ruby -v
ruby 1.8.6 (2007-09-23 patchlevel 5000) [powerpc-darwin8.10.0]I'll to try again tomorrow though, since I'm tired.
I experimented a bit with your benchmark, and found that writing a large string would only write as much as the socket buffer would accept (81660 bytes, in my case).
Maybe you need to set the TCP_NODELAY socket option?
socket.setsockopt Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1
I once made write_nonblock raise Errno::EAGAIN, but I accidentally deleted that change. I'll play some more tomorrow when I'm not so tired.
Eric Hodel wrote:
Christoffer Lern� wrote:
Anyone know what's up with TCPSocket#write_noblock
The method hasn't been invented yet?
That's rather harsh for a one letter typo.
Harsh? In what way?
responding that way when you didn't know
Exactly.
Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.now you are being harsh again.
cfp:~ > PAGER=cat ri IO|grep non
cfp:~ > PAGER=cat ri IO
Also,
$ ri -l | egrep "write.*block"
IO#write_nonblock
i know ruby-core is *always* looking for people to contribute for docs. you certainly get what you pay for with bloody open source eh?
Absolutely. The main reason that ruby documentation is "so deplorable" is because the people complaining about it are rarely the ones doing anything to improve it.
···
On Oct 13, 2007, at 18:01 , ara.t.howard wrote:
On Oct 13, 2007, at 6:31 PM, 7stud -- wrote:
On Oct 13, 2007, at 15:20 , 7stud -- wrote:
--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars
ara.t.howard wrote:
searching != knowing
Maybe in your native language they don't have question marks for
punctuation. If you don't know what they mean, perhaps you shouldn't
criticize?
besides, google pulls 'ruby write_noblock' up on the first hit
ruby - Google Search
+noblock&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-
US:official&client=firefox-a
Now that google has had time to index this page, the hits you claim to
be the Holy Grail for "write no_block" are actually links back to this
thread. I wonder if you post any information on this forum that isn't
erroneous.
Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.now you are being harsh again.
It sounds like you are suffering a case of sour grapes because I
corrected you in that other thread. If you are going to post poorly
written examples, which are also not responsive to a poster's question,
you will get corrected. If that hurts your feelings, maybe you should
construct better examples--or just not post at all.
Absolutely. The main reason that ruby documentation is "so
deplorable" is because the people complaining about it are rarely the
ones doing anything to improve it.
Yes, when a language is poorly designed and/or documented, blame the
people who are trying to learn it.
In article <E8AB2CFA-FDAF-47B3-A0BE-4C089960C5CF@dragonascendant.com>,
Christoffer Lernö <lerno@dragonascendant.com> writes:
Could this be due to me running
ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?
The funny thing was that only the socket returned by accept_nonblock
had this behaviour, that is
I think it is a Mac OS X problem.
Your example works without problem on GNU/Linux and FreeBSD
box but not on Mac OS X.
% cat z1.rb
require 'socket'
require 'fcntl'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
other = a.accept
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z1.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
0
Start Write
81560
End Write
The IO returned from accept is not nonblocking mode.
So write_nonblock makes it nonblocking mode at first.
% cat z2.rb
require 'socket'
require 'fcntl'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
other = a.accept_nonblock
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z2.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
4
Start Write
^Cz2.rb:8: Interrupt
The IO returned from accept_nonblock seems nonblocking mode.
So write_nonblock doesn't make it nonblocking mode.
But actually it is not nonblocking mode, I think.
Workaround:
Index: ext/socket/socket.c
···
===================================================================
--- ext/socket/socket.c (revision 13670)
+++ ext/socket/socket.c (working copy)
@@ -1464,6 +1464,24 @@
return init_inetsock(sock, Qnil, arg1, Qnil, Qnil, INET_SERVER);
}
+static void
+make_fd_nonblock(int fd)
+{
+ int flags;
+#ifdef F_GETFL
+ flags = fcntl(fd, F_GETFL);
+ if (flags == -1) {
+ rb_sys_fail(0);
+ }
+#else
+ flags = 0;
+#endif
+ flags |= O_NONBLOCK;
+ if (fcntl(fd, F_SETFL, flags) == -1) {
+ rb_sys_fail(0);
+ }
+}
+
static VALUE
s_accept_nonblock(VALUE klass, OpenFile *fptr, struct sockaddr *sockaddr, socklen_t *len)
{
@@ -1475,6 +1493,7 @@
if (fd2 < 0) {
rb_sys_fail("accept(2)");
}
+ make_fd_nonblock(fd2);
return init_sock(rb_obj_alloc(klass), fd2);
}
Anyway nonblocking flag of accepted fd is not portable.
http://cr.yp.to/docs/unixport.html
So setting it unconditionally is good thing for portability.
--
Tanaka Akira
On Sun, 14 Oct 2007 11:55:56 +0900, 7stud (who has been learning Ruby for
seven months) wrote to ara (who has been posting here since 2002, and who
contributed a core bug fix in his first two weeks):
It sounds like you are suffering a case of sour grapes because I
corrected you in that other thread. If you are going to post poorly
written examples, which are also not responsive to a poster's question,
you will get corrected. If that hurts your feelings, maybe you should
construct better examples--or just not post at all.
::Cough::
Uh. 7stud?
So, yeah.
You've come a little late to the party, you're speaking a little too
loudly, and you keep referring to the caviar as "bean dip".
It's awkward.
···
--
Jay Levitt |
Boston, MA | My character doesn't like it when they
Faster: jay at jay dot fm | cry or shout or hit.
http://www.jay.fm | - Kristoffer
Yeah, that seems to be it.
This workaround in the ruby code gets around the problem:
require 'socket'
require 'fcntl'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
other = a.accept_nonblock
other.fcntl(Fcntl::F_SETFL, other.fcntl(Fcntl::F_GETFL) & ~(Fcntl::O_NONBLOCK))
puts "Start Write"
other.write_nonblock("X" * 300000)
puts "End Write"
/C
···
On 14 Oct 2007, at 18:14, Tanaka Akira wrote:
In article <E8AB2CFA-FDAF-47B3-A0BE-4C089960C5CF@dragonascendant.com>,
Christoffer Lernö <lerno@dragonascendant.com> writes:Could this be due to me running
ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?
The funny thing was that only the socket returned by accept_nonblock
had this behaviour, that isI think it is a Mac OS X problem.
Your example works without problem on GNU/Linux and FreeBSD
box but not on Mac OS X.% cat z1.rb
require 'socket'
require 'fcntl'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
other = a.accept
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z1.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
0
Start Write
81560
End WriteThe IO returned from accept is not nonblocking mode.
So write_nonblock makes it nonblocking mode at first.% cat z2.rb
require 'socket'
require 'fcntl'
a = TCPServer.new(5000)
socket = TCPSocket.new("localhost", 5000)
other = a.accept_nonblock
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z2.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
4
Start Write
^Cz2.rb:8: InterruptThe IO returned from accept_nonblock seems nonblocking mode.
So write_nonblock doesn't make it nonblocking mode.
But actually it is not nonblocking mode, I think.