Ruby-dev summary 22877-23014

ruby-talk
1

Hi all,

This is a summary of ruby-dev mailing list in these days.

[ruby-dev:22859] Re: [ruby-talk:91705] io/nonblock

Nobuyoshi Nakada suggested new methods for nonblocking I/O:

* IO#nonblock?
* IO#nonblock = flag
* IO#nonblocking(flag) { .... }
* IO#nonblock { .... }
* IO#nonblock

TANAKA Akira is opposed to this idea, because:

* Nonblocking IO is not useful in most cases.
* Nonblocking flag is inherited to child processes,
  that is very dangerous.

He also suggested alternative solution, “stdio-sensitive sysread”.

This issue is still open.

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

 e.g.
 Process.spawn "ls -l /"

He also suggested that IO.popen should invoke processes without
shell interaction e.g. IO.popen directly calls command if the first
argument is an Array, as well as #exec does.

 # `*' is not expanded
 IO.open(["/bin/ls", "*"], "r")

Matz agreed with these ideas.

[ruby-dev:22911] core dump with /#{Thread.pass}/o

TANAKA Akira reported that /#{Thread.pass}/o causes segmentation fault.
The ruby interpreter edits an internal syntax tree when /#{…}/o is
seen, the value of /#{…}/o can be unexpectable with multithreads.
This problem is already discussed in [ruby-dev:11650] and [ruby-dev:21614],
and it is needed to make Mutex class builtin.

Akira committed temporary fix, but this issue is still open.

[ruby-dev:23002] drb/eq causes unexpected assertion fail

Kazuhiro Nishiyama reported that drb’s unit test failed at
test_drb.rb:5: test_05_eq. This problem is caused by the difference
of drb/eq.rb load order.

Masatoshi Seki, the author of drb, noted that he is thinking to require
drb/eq.rb by default.

– Minero Aoki

ruby-dev summary index:
http://i.loveruby.net/en/ruby-dev-summary.html

2

yuhu!
I always wanted this, even If I thought that should go in Kernel, like
fork() and exec(). Why this goes in Process ?

···

il Thu, 26 Feb 2004 18:25:30 +0900, Minero Aoki aamine@loveruby.net ha scritto::

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

3

TANAKA Akira is opposed to this idea, because:

* Nonblocking IO is not useful in most cases.

IO#print is not useful in most cases; IO#puts is sufficient. So we
should get rid of print. Same with read/write vs. sysread/syswrite.

The question should not be whether this method is useful in most cases,
but instead whether this method is useful in enough cases to warrant
including it.

* Nonblocking flag is inherited to child processes,
  that is very dangerous.

If a programmer creates a child process and uses a parent processes’s
file descriptor in the child, he’d better know what he is doing, whether
the non-blocking flag is set or not.

Paul

···

On Thu, Feb 26, 2004 at 06:25:30PM +0900, Minero Aoki wrote:

4

Hi,

At Thu, 26 Feb 2004 20:34:48 +0900,
gabriele renzi wrote in [ruby-talk:93740]:

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

yuhu!
I always wanted this, even If I thought that should go in Kernel, like
fork() and exec(). Why this goes in Process ?

Well, rather I’ll feel nice if there are also those two in
Process. Which is better, or both?

···

–
Nobu Nakada

5

In article 20040226144929.GP379@atdesk.com,
Paul Brannan pbrannan@atdesk.com writes:

The question should not be whether this method is useful in most cases,
but instead whether this method is useful in enough cases to warrant
including it.

My point is that sysread (or sysread like method which care stdio
buffer) is better than nonblocking read.

I think they are useful in similar cases.

If a programmer creates a child process and uses a parent processes’s
file descriptor in the child, he’d better know what he is doing, whether
the non-blocking flag is set or not.

I think usual program expects non-blocking flag is clear. For
example, following problem is caused because cvs (stdio) doesn’t
expect stderr is nonbloking.

http://groups.google.com/groups?th=e4df2fdc1f4f4950
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00041
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00186

Since there are much code which expect file descriptor which
non-blocking flag is clear, nonblocking-mode should be avoided if
possible. So Ruby should not recommend nonblocking IO by providing
easy-to-use method to make IO nonblocking. However someone who know
what he is doing can make IO nonbloking using IO#fcntl anyway.

···

–
Tanaka Akira

6

agreed, put 'em all in Process :slight_smile:

···

il Thu, 26 Feb 2004 21:55:44 +0900, nobu.nokada@softhome.net ha scritto::

Well, rather I’ll feel nice if there are also those two in
Process. Which is better, or both?

7

both.

i too, have always wanted this. IO.popen is ok - but for my uses not useful
since i lose stderr. using open3 has other problems…

-a

···

On Thu, 26 Feb 2004 nobu.nokada@softhome.net wrote:

Hi,

At Thu, 26 Feb 2004 20:34:48 +0900,
gabriele renzi wrote in [ruby-talk:93740]:

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

yuhu!
I always wanted this, even If I thought that should go in Kernel, like
fork() and exec(). Why this goes in Process ?

Well, rather I’ll feel nice if there are also those two in
Process. Which is better, or both?

–

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================

8

My point is that sysread (or sysread like method which care stdio
buffer) is better than nonblocking read.

I think they are useful in similar cases.

What about non-blocking writes?

(I rarely put an fd in non-blocking mode to read, because I can easily
enough keep my app from blocking on a read by simply not reading when
there isn’t data to read; if I put an fd in non-blocking mode, it is
almost always for writing).

If a programmer creates a child process and uses a parent processes’s
file descriptor in the child, he’d better know what he is doing, whether
the non-blocking flag is set or not.

I think usual program expects non-blocking flag is clear. For
example, following problem is caused because cvs (stdio) doesn’t
expect stderr is nonbloking.

http://groups.google.com/groups?th=e4df2fdc1f4f4950
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00041
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00186

So it’s not a good idea to use non-blocking IO with stdio. There are
plenty of other IO objects that I use non-blocking IO with, especially
sockets.

Since there are much code which expect file descriptor which
non-blocking flag is clear, nonblocking-mode should be avoided if
possible. So Ruby should not recommend nonblocking IO by providing
easy-to-use method to make IO nonblocking.

I have code that expects that the non-blocking flag is not clear. The
rule should be to not mix non-blocking fds with code that expects a
blocking fd and vice versa.

However someone who know what he is doing can make IO nonbloking using
IO#fcntl anyway.

This is true. And I’m not sure that:

socket.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

is any less clear to read then

socket.nonblock = true

except that Ruby might do something other than an fcntl under the hood
in the latter case, depending on the platform.

Paul

···

On Fri, Feb 27, 2004 at 01:59:38PM +0900, Tanaka Akira wrote:

9

In article 20040227152954.GS379@atdesk.com,
Paul Brannan pbrannan@atdesk.com writes:

What about non-blocking writes?

non-blocking writes needs O_NONBLOCK.
But it is difficult issue.

(I rarely put an fd in non-blocking mode to read, because I can easily
enough keep my app from blocking on a read by simply not reading when
there isn’t data to read; if I put an fd in non-blocking mode, it is
almost always for writing).

I imagine you want to avoid a situation like following.

% ruby -e ‘Thread.new { loop { STDERR.puts “e”; sleep 1 } }
print “x” * 8192
sleep 10’|sleep 10
e
-e:2:in write': Broken pipe (Errno::EPIPE) from -e:2:in print’
from -e:2

Since Ruby doesn’t use non-blocking writes on write operation, whole
process blocks.

Unfortunately Ruby doesn’t handle non-blocking writes well. Ruby may
lose some data on non-blocking writes.

% ruby -rfcntl -e ’
STDOUT.sync = true
STDOUT.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
STDOUT.write “x” * 4097
STDOUT.write “y”'|(sleep 1; cat )|od -c
0000000 x x x x x x x x x x x x x x x x

···

0010000 y
0010001

This is because stdio’s fflush lose data on EAGAIN.
stdio is not nonblocking friendly.

So, for fixing non-blocking writes problem, Ruby should

  1. abandon stdio and
  2. set O_NONBLOCK on write(2) temporally or use timer like DJB.
    http://cr.yp.to/unix/nonblock.html

I don’t known when Ruby abandon stdio.

Sigh.

I have code that expects that the non-blocking flag is not clear. The
rule should be to not mix non-blocking fds with code that expects a
blocking fd and vice versa.

Yes. However it is difficult to know a fd is not used as blocking
fd. stdio is used widely. Sometimes Ruby itself assumes fd blocking:
[ruby-talk:66196], [ruby-talk:93726].

Tanaka Akira