RCR for child execution

Looking on RubyGarden it seems that the RCR process there is “resting”, so
here are is a suggestion for discussion:

  1. A way to execute an external program capturing both stdout and stderr
    into a string

Syntax might be: %X{prog args} # capital X

This differs from %x{prog args 2>&1}

because it seems that Windows doesn’t support the latter syntax. At least,
not according to the following code I have seen:

tossErr = ''
tossErr = ' 2>/dev/null' if RUBY_PLATFORM =~ /linux/

system ("dot -Tcmap #{dotName} -o #{mapName}" + tossErr)

[taken from MiniRubyWiki. This is nasty and unportable because it doesn’t
do what is required on FreeBSD, Solaris etc.]

  1. A variant of IO.popen which captures both stdout and stderr down the
    same pipe, maybe with an extra option letter:

    c = IO.popen(“ls /foo”,“re”) # unidirectional
    c = IO.popen(“nroff -man”,“w+e”) # bidirectional

    Again, this would be just to be more portable than 2>&1

Rather than an option letter, maybe Ruby should just look for “2>&1” at the
end of the string, and if so strip it off and point the child’s stdout and
stderr to the same pipe? Then at least it could be relied on.

Regards,

Brian.

And also, the latter syntax doesn’t capture any output generated on stderr
by the shell itself - especially the common case where ‘prog’ does not
exist. You just get an empty string returned and a message written on
stderr.

It would also be nice if Ruby could raise an exception when prog does not
exist, instead of continuing blindly. Maybe it could exec the target program
directly and not go via a shell, in cases where the shell is not needed
(i.e. no pipes or redirection). But I can see how that would make things
inconsistent between ‘command pipelines’ which use the shell, and ‘single
commands’ which don’t. Or it might be sufficient just to check for the
appropriate exit status (127 in the case of /bin/sh and Unix).

Otherwise you might end up having a simple shell in Ruby itself, just enough
to do pipelines and redirection. The benefits would be proper exception
handling and less dependencies on /bin/sh, but it doesn’t really seem worth
the maintenance effort.

Regards,

Brian.

···

On Sat, May 10, 2003 at 09:14:35AM +0100, Brian Candler wrote:

  1. A way to execute an external program capturing both stdout and stderr
    into a string

Syntax might be: %X{prog args} # capital X

This differs from %x{prog args 2>&1}

because it seems that Windows doesn’t support the latter syntax.

I have some more to add to this issue.

I am embedding ruby into c++ and want to seperate Rubys output from C++.

AFAIK: When ruby spawns a child-process, then the child inherits
filedescriptor 1 & 2 from parent.

What im asking for is: is it possible for let the children inherit the
value of rb_stdout, rb_stderr instead ???

···


Simon Strandgaard

exist, instead of continuing blindly. Maybe it could exec the target program
directly and *not* go via a shell, in cases where the shell is not needed
(i.e. no pipes or redirection).

I've not understood : what do you think that ruby do actually ?

Guy Decoux

From the Unix point of view, the child can be passed any open Unix
filehandles that you like.

For doing this sort of stuff from C, I strongly recommend the book “Advanced
Programming in the Unix Environment” by the late Richard Stevens (Addison
Wesley). It’s a classic must-have reference books for anyone writing Unix
programs. In short, you open whatever files/pipes you like, then use the
function dup2() to assign them to fd 0/fd 1/fd 2.

If you look at the source for ‘Open3::popen3’ you’ll see a way that you can
do this from Ruby using IO#reopen. Note that rb_stdout and rb_stderr that
you mention are just the constants STDOUT and STDERR in ruby:

io.c: rb_define_global_const(“STDOUT”, rb_stdout);
io.c: rb_define_global_const(“STDERR”, rb_stderr);

These objects are wrappers, or containers, which contain the relevant fds,
or in fact FILE* objects which are a stdio abstraction which in turn contain
the fds. Look at prep_stdio in io.c:

prep_stdio(f, mode, klass)
FILE *f;
{

MakeOpenFile(io, fp);
fp->f = f;
fp->mode = mode;

}

rb_stdout = orig_stdout = prep_stdio(stdout, FMODE_WRITABLE, rb_cIO);

So whenever you do STDOUT.reopen you are really acting on the stdio object
“FILE *stdout” which in turn is a wrapper around an open file on fd 1.

Now, if you do
system(“foo”)
that’s the fd your child will share with the parent process as its stdout.

If you do
any=%x{foo}
then Ruby opens a new pipe and hands one end of it as fd 1 to the child
process, and reads the local end into a string. However it doesn’t touch fd
2, so the child process shares it.

The same applies with IO.popen(“foo”,“r”)

In the case of Open3.popen3 you get three separate pipes set up, and one end
of each of those pipes is dup2()'d into fd0, fd1 and fd2 of the child
process before exec’ing.

Is this any clearer??

Regards,

Brian.

···

On Sat, May 10, 2003 at 10:26:11PM +0900, Simon Strandgaard wrote:

I have some more to add to this issue.

I am embedding ruby into c++ and want to seperate Rubys output from C++.

AFAIK: When ruby spawns a child-process, then the child inherits
filedescriptor 1 & 2 from parent.

What im asking for is: is it possible for let the children inherit the
value of rb_stdout, rb_stderr instead ???

Maybe it could exec the target program
directly and not go via a shell, in cases where the shell is not needed
(i.e. no pipes or redirection).

This is what happens when you call system() with more then one argument.

system(“ls $HOME”)
Mail bin data doc downloads etc homework mnt opt prog tmp
=> true
system(“ls”, “$HOME”)
ls: $HOME: No such file or directory
=> false

Note: It’s a Good Idea™ to do this when possible because it avoids quoting
problems.

system(“cat Filename with spaces”)
cat: Filename: No such file or directory
cat: with: No such file or directory
cat: spaces: No such file or directory
=> false
system(“cat”, “Filename with spaces”)
cat: Filename with spaces: No such file or directory
=> false

Jason Creighton

···

On Sat, 10 May 2003 17:32:33 +0900 Brian Candler B.Candler@pobox.com wrote:

exist, instead of continuing blindly. Maybe it could exec the
target program
directly and not go via a shell, in cases where the shell is
not needed
(i.e. no pipes or redirection).

I’ve not understood : what do you think that ruby do actually ?

If it’s like perl (and I’m not sure that it is), but perl will exec
the program directly, unless it has shell meta-chars in the string,
in which case it will exec a shell to run the program.

I take it by your response, Ruby doesn’t do this…?

···

Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.

I have some more to add to this issue.

I am embedding ruby into c++ and want to seperate Rubys output from C++.

AFAIK: When ruby spawns a child-process, then the child inherits
filedescriptor 1 & 2 from parent.

What im asking for is: is it possible for let the children inherit the
value of rb_stdout, rb_stderr instead ???

[snip stdout/stderr]

Is this any clearer??

Problem:
Ruby-child-processes inherit stdout/stderr, not rb_stdout, rb_stderr.
From C/C++ I cannot use stdout/stderr.

This is my RCR proposal: Let child-processes inherit from
rb_stdout/rb_stderr, so that stdout/stderr is untouched :slight_smile:

···

On Sun, 11 May 2003 01:50:49 +0900, Brian Candler wrote:

On Sat, May 10, 2003 at 10:26:11PM +0900, Simon Strandgaard wrote:


Simon Strandgaard

I take it by your response, Ruby doesn't do this...?

ruby do exactly the same, this is why I've not understood this part of the
RCR

Guy Decoux

Ruby-child-processes inherit stdout/stderr, not rb_stdout, rb_stderr.
From C/C++ I cannot use stdout/stderr.

What do you mean, “from C/C++ I cannot use stdout/stderr”??

This is my RCR proposal: Let child-processes inherit from
rb_stdout/rb_stderr, so that stdout/stderr is untouched :slight_smile:

That makes no sense, because rb_stdout/rb_stderr are just wrappers (or
containers, or proxy-objects, call them what you will) on top of
stdout/stderr. They are the same Unix fds. The distinction you are making
is artificial.

This is what OO people call the “delegator pattern”: when I call
STDOUT.close, for example, it forwards a close() request to the libc object
FILE *stdout, which in turn is a container on fd 1.

If you want the child process to have a different files opened on fd 1
and/or fd 2, then after the fork() you simply reopen fd1/fd2. You can do
this in Ruby as follows:

pid = fork do
f = File.open(“/tmp/junk”,“w”)
STDOUT.reopen(f)
f.close
exec(“ls /dev”)
end
Process.waitpid(pid)
puts “I am done now (and my stdout is untouched)”

You should find that the output of ls has been put in /tmp/junk, but the
stdout of the parent process remains as it was.

Brian.

···

On Sun, May 11, 2003 at 01:27:31AM +0900, Simon Strandgaard wrote:

My apologies, I had misinterpreted the behaviour, and I see you are right:

irb(main):007:0> %x{asdfasdf 2>&1}
asdfasdf: not found
=> “”
irb(main):006:0> %x{asdfasdf}
(irb):6: command not found: asdfasdf
=> “”

Note the extra “(irb):6: command not found” in the second case; this is the
giveaway that the command is run without a shell. It’s a Ruby-generated
message (with a ruby line number) not a shell-generated message.

But in both casees, the command generates output to your Ruby program’s
stderr:

$ cat x.rb
a = %x{nonexist1}
p $?
b = %x{nonexist2 2>&1}
p $?

$ ruby x.rb 2>foo
32512
32512

$ cat foo
x.rb:1: command not found: nonexist1
nonexist2: not found

The error ‘command not found’ seems to say ‘exception!’ to me, but thinking
about it now I suppose it’s better to be consistent than to have exceptions
in some error situations and empty strings in other cases.

But at least I’d like to have a way to either capture this message into the
result string, or discard it, preferably without message about with STDERR
itself (which is what triggered this whole discussion off!)

Cheers,

Brian.

···

On Sat, May 10, 2003 at 11:55:21PM +0900, ts wrote:

I take it by your response, Ruby doesn’t do this…?

ruby do exactly the same, this is why I’ve not understood this part of the
RCR

Ruby-child-processes inherit stdout/stderr, not rb_stdout, rb_stderr.
From C/C++ I cannot use stdout/stderr.

What do you mean, “from C/C++ I cannot use stdout/stderr”??

Everything is OK when your application starts up, stdout/stderr works
fine. Then you initialize ruby and redirects its output elsewhere.
Now stdout/stderr doesn’t work any longer.

I tried restoring stdout/stderr to the console again, but this
affects the “system” call in ruby, see “process.c” that stdout/stderr
is being used.

After redirecting rubys output, you cannot use stdout/stderr.

cat main.cpp
#include <unistd.h> // pipe, select
#include <sys/ioctl.h> // FIONREAD
#include
#include
#include
#include <ruby.h> // rb_xxx

extern “C” {
VALUE rb_defout;
VALUE rb_stdout;
VALUE rb_stderr;
}

int main(int argc, char *argv) {
using namespace std;
int pipes[2];
if(pipe(pipes)) exit(-1);

    int bak_stdout = dup(fileno(stdout));
    int bak_stderr = dup(fileno(stderr));
    cout << "backups: stdout=" << bak_stdout <<
            " stderr=" << bak_stderr << endl;
    FILE *fd_out = fdopen(bak_stdout, "a");
    FILE *fd_err = fdopen(bak_stderr, "a");

    ruby_init();
    ruby_init_loadpath();
    VALUE params[2];
    params[0] = INT2FIX(pipes[1]);
    params[1] = rb_str_new2("w");
    VALUE w = rb_class_new_instance(2, params, rb_path2class("IO"));
    rb_gv_set("$defout", w);
    rb_gv_set("$stdout", w);
    rb_gv_set("$stderr", w);

    int new_stdout = dup2(bak_stdout, 1);
    int new_stderr = dup2(bak_stdout, 2);
    fprintf(fd_out, "new: stdout=%d stderr=%d\n", new_stdout, new_stderr);

    stdout = fdopen(new_stdout, "a"); // BOOM!
    stderr = fdopen(new_stderr, "a"); // BOOM!

    fprintf(fd_out, "bak_stdout\n");
    cout << "cout is working again" << endl;

    rb_eval_string("sleep 1; $stdout.puts 'aa';"
            " puts 'hello'; system('ls a.c xx'); sleep 1");

    close(pipes[0]);
    close(pipes[1]);
    return 0;

}

cat Makefile
RUBY=$(HOME)/install/language/ruby-1.8.0
LIBS=$(RUBY)/libruby-static.a -lcrypt
all:
g++ main.cpp -lc_r -I$(RUBY) $(LIBS) -Wall

Well, there is no thread to flush the pipe… but this is not important
here! What is important is that system(‘ls a.c xx’)'s output goes to
stdout/stderr, is mixed up with C’s output.

If I remove those 2 lines containing “BOOM!” then system’s output will go
to the pipe.

After redirection I cannot use stdout/stderr!

Making this possible would require changes to ruby… this is what my RCR
proposal is about.

···

On Sun, 11 May 2003 05:39:31 +0900, Brian Candler wrote:

On Sun, May 11, 2003 at 01:27:31AM +0900, Simon Strandgaard wrote:


Simon Strandgaard

Ruby-child-processes inherit stdout/stderr, not rb_stdout, rb_stderr.
From C/C++ I cannot use stdout/stderr.

What do you mean, “from C/C++ I cannot use stdout/stderr”??

Everything is OK when your application starts up, stdout/stderr works
fine. Then you initialize ruby and redirects its output elsewhere.
Now stdout/stderr doesn’t work any longer.

Right. Ruby’s STDOUT is your C++ application’s stdout, and Ruby’s STDERR
is your C++ application’s stderr. That’s because your application is your
application! The fact that it is written half in one language and half in
another is beside the point; at the end of the day it’s a single application
which runs under an O/S.

Tinkering with STDOUT in Ruby changes the ‘real’ underlying stdout, and
that’s what a fork/exec child shares with you unless you pass it something
else.

Now, when you do “puts” there is a layer of indirection because it writes to
$defout, which you can point elsewhere without touching stdout, but there’s
nothing like that for stderr: any Ruby-generated warnings go directly to
stderr, and any Ruby code which says
$stderr.puts(“my warning”)
clearly does the same.

But that’s an aside, as it’s not relevant to fork/exec children: if you have
changed $defout then the child won’t know, as the only thing it sees is the
O/S level stdout and stderr. You can open them as different files/pipes
after the fork and before the exec, or you can leave them alone, in which
case they are shared.

I tried restoring stdout/stderr to the console again, but this
affects the “system” call in ruby, see “process.c” that stdout/stderr
is being used.

It just flushes them to avoid problems which occur if buffers have partial
contents in them before the child starts up - otherwise the stored data
could be output twice (once when your app eventually flushes its buffer, and
once when the child eventually flushes its buffer)

Otherwise, “system” doesn’t do any writes to stdout/stderr. It’s the child
process which does so: any child is quite entitled to write to stdout and/or
stderr while it’s running, in fact that’s the normal way in which it would
interact with the system.

Regards,

Brian.

···

On Sun, May 11, 2003 at 09:29:43AM +0900, Simon Strandgaard wrote:

On Sun, May 11, 2003 at 01:27:31AM +0900, Simon Strandgaard wrote:

    stdout = fdopen(new_stdout, "a"); // BOOM!
    stderr = fdopen(new_stderr, "a"); // BOOM!

The particular BOOM I get is:

main.cpp:40: non-lvalue in assignment
main.cpp:41: non-lvalue in assignment

That’s because stdout is not a variable: on my FreeBSD box it is defined as

extern FILE __sF;

#define stdout (&__sF[1])

(so it’s not a pointer variable which can be reassigned, it is a pointer to
a static piece of storage)

To reopen stdout you need either ‘freopen’ (at the stdio.h layer) or ‘dup2’
(at the Unix API layer). I don’t know if there’s also a C++ way of doing
this by making a method call on ‘cout’.

Well, there is no thread to flush the pipe… but this is not important
here! What is important is that system(‘ls a.c xx’)'s output goes to
stdout/stderr, is mixed up with C’s output.

Yes, of course it is.

If you want the child to have a different stdout/stderr than yours, then
what you need to do is:
- fork
- (in child) reopen stdout/stderr
- (in child) exec
- (in parent) wait for child to finish

This is all that ‘system’ does, except it omits the ‘reopen stdout/stderr’
step.

Your logic is wrong: you are trying to reopen stdout/stderr before the fork,
which means you are affecting your main program (the parent) as well as the
child.

If you want to use a pipe, the logic is:

  • open the pipe(s)
  • fork
    (in child)
    • dup the writer end of each pipe to stdout/stderr
    • close the reader end
    • exec
      (in parent)
- close the writer end of each pipe
- read from the reader end of the pipes until they are closed
- wait for the child to finish

This is a sufficiently common operation that most Unixes have a function
“popen” which does it (try “man popen” on your system). But if you want
pipes for both stdout and stderr then usually you’ll have to do it the
longhand way.

See pages 223 and 435 of the Stevens book I referred you to. It sounds to me
like this would be a good investment! For example, after showing
step-by-step how to set up communication between parent and child using
pipes, the section about popen begins:

“Since a common operation is to create a pipe to another process, to
either read its output or send it input, the standard I/O library
has historically provided the popen and pclose functions. These two
functions handle all the dirty work that we’ve been doing ourselves:
the creation of a pipe, the fork of a child, closing the unused ends
of the pipe, execing a shell to execute the command, and waiting for
the command to terminate”

He then goes on to show not just how to use popen, but gives an
implementation of it.

Making this possible would require changes to ruby… this is what my RCR
proposal is about.

It sounds to me like you are actually asking for changes to Unix, but I
can’t see exactly what.

The Unix API has stood the test of time (30 years?!) so, no disrespect,
maybe you just need to learn a bit more about how to use it…

Regards,

Brian.

···

On Sun, May 11, 2003 at 09:29:43AM +0900, Simon Strandgaard wrote:

Hi,

I’m not sure if I understand what you care, but at least I want you to
know it’s the field I want to fix/change before the final 1.8.0
release.

						matz.
···

In message “Re: RCR for child execution” on 03/05/11, “Simon Strandgaard” 0bz63fz3m1qt3001@sneakemail.com writes:

Well, there is no thread to flush the pipe… but this is not important
here! What is important is that system(‘ls a.c xx’)'s output goes to
stdout/stderr, is mixed up with C’s output.

If I remove those 2 lines containing “BOOM!” then system’s output will go
to the pipe.

After redirection I cannot use stdout/stderr!

Making this possible would require changes to ruby… this is what my RCR
proposal is about.

Let me try to see if I can give a specific example of the problem that I
think you’re trying to solve:

- function A is written in C++
- function A calls function B, which is written in Ruby
- function B writes to stdout and stderr
- you want all function B's output to be captured in a string,
  without making any modifications to function B itself

Is that an accurate description of the problem?

If so, it’s straightforward to solve. Firstly, note that it doesn’t matter
which language A and B are written in. I would use the same basic approach
for these three cases:

  1. A and B both in Ruby
  2. A in C(++), B in Ruby
  3. A in C(++), B in C(++)

So let’s start with case 1.

def functionA
  result = functionB(99)
  puts "The output of B was #{result.inspect}"
end

def functionB(arg)
  puts "Hello, world! You passed in #{arg}"
  $stderr.puts "Hello on stderr"
  system("ls /nonexistent")     # so that ls outputs on stderr
end

function A

This doesn’t work as written of course, as we’re not building a result
string. The first line of functionB we could fix by pointing $defout to a
StringIO object, but the second two lines write directly to the Unix stderr
filehandle, and there’s nothing we can do about it. We can’t modify
functionB, and we certainly can’t modify “ls”!

Given that we are not allowed to make any modification to functionB, we have
to write a wrapper and run it in a separate process. That’s the only way we
can ensure that while functionB runs it has a different stdout and stderr,
which we can read into a string, without affecting our main program’s stdout
and stderr.

Here is the complete pure-Ruby solution:

def functionA
  result = capture_output { functionB(99) }
  puts "The output of B was #{result.inspect}"
end

def functionB(arg)
  puts "Hello, world! You passed in #{arg}"
  $stderr.puts "Hello on stderr"
  system("ls /nonexistent")     # so that ls outputs on stderr
end

def capture_output
  # We could use IO.popen("-") to do all this for us, except we want
  # both stdout and stderr attached to the same pipe at the child end
  rd, wr = IO.pipe
  pid = fork do
    rd.close
    STDOUT.reopen(wr)
    STDERR.reopen(wr)
    wr.close
    yield
  end
  wr.close
  result = rd.read
  rd.close
  Process.waitpid(pid)
  return result
end
  
functionA  #>> The output of B was "Hello, world! <... etc>"

That’s it. You can’t guarantee the exact order of the output lines when
mixing stdout and stderr in this way, because they tend to be flushed at
different times, but that’s probably not a concern. A more general version
of capture_output would accept a string to be passed to the child’s stdin,
and capture stdout and stderr to separate strings.

Now, you have two ways to solve case 2:
a. just use the Ruby capture_output that I have written above

def wrap_functionB(*args)
  capture_output { functionB(*args) }
end

The C++ program just calls wrap_functionB instead of calling functionB
directly. This can be done in a more general-purpose way:

def wrap(method,*args)
  capture_output { method.call(*args) }
end

so in C++ you first create a method object for the thing you want to call,
and then call ‘wrap’ with the method object and arguments.

b. rewrite “capture_output” in C++. It maps almost line-for-line to Unix API
calls, so I leave it as an exercise for the reader.

The solution for case 3, where C++ function A calls C++ function B, is the
same as 2b.

Are we getting somewhere now? :slight_smile:

Now, as to how useful this approach is in embedding: it might be, if you
have some Ruby code which you want to run which generates output that you
want to capture (your example of Test::Unit’s console runner). But you
wouldn’t want to do this unless absolutely necessary, because:

  • there is an efficiency overhead with starting new processes
  • functionB cannot modify any state which is shared with functionA, or even
    between subsequent invocations of functionB; nor can it pass back any
    return value (except a limited integer value can be passed back as the
    child exit status). All results would have to be passed back “in-band” on
    the child’s stdout.

So, you can’t have your cake and eat it. Either you run functionB as a
separate process, in which case you can capture its stdout/stderr
transparently, or you embed it properly (by calling functionB directly), in
which case it shares its stdout/stderr with the main program, because it
is part of the main program.

But there’s nothing to stop you calling capture_output in certain cases
only where it’s needed, and making direct method calls where you have
control over the Ruby code itself. The latter case is what I think of when
you say “embedded Ruby”.

Regards,

Brian.

    stdout = fdopen(new_stdout, "a"); // BOOM!
    stderr = fdopen(new_stderr, "a"); // BOOM!

The particular BOOM I get is:

main.cpp:40: non-lvalue in assignment
main.cpp:41: non-lvalue in assignment

That’s because stdout is not a variable: on my FreeBSD box it is defined as

All my boxes run FreeBSD5.0 :slight_smile:
I don’t get any warnings/errors.

extern FILE __sF;

#define stdout (&__sF[1])

(so it’s not a pointer variable which can be reassigned, it is a pointer to
a static piece of storage)

To reopen stdout you need either ‘freopen’ (at the stdio.h layer) or ‘dup2’
(at the Unix API layer).

from man freopen
FILE *
freopen(const char *path, const char *mode, FILE *stream);

I don’t know what “path” I should initialize freopen with, therefore
I use dup2 + fdopen.

int new_stdout = dup2(bak_stdout, 1);
stdout = fdopen(new_stdout, “a”);

Well, there is no thread to flush the pipe… but this is not important
here! What is important is that system(‘ls a.c xx’)'s output goes to
stdout/stderr, is mixed up with C’s output.

Yes, of course it is.

If you want the child to have a different stdout/stderr than yours, then
what you need to do is:
- fork
- (in child) reopen stdout/stderr
- (in child) exec
- (in parent) wait for child to finish

I know im terrible expressing myself in english, so be gentle with me :slight_smile:

Separation of ruby stdout/stderr from C/C++ stdout/stderr is what I want.
I run Ruby in the same thread as C++ (ruby embedded into C++).
This separation is working fine, as long as I don’t use system or
backquote.

Using system or backquote, then it goes BOOM. The problem as I see it is
localized in Rubys code - not mine! Ruby spawns a child which inherits
stdout/stderr. This child should instead inherit rb_stdout/rb_stderr
instead!!! Can this issue be fixed on Ruby ???

It sounds to me like you are actually asking for changes to Unix, but I
can’t see exactly what.

I have no problems with Unix… I think you underestimate me because of my
broken english… I have been programming my hole life… I just recently
started to poke around with unix-pipes. I am very familiar with sockets
so no big difference here. Thanks for putting so much effort in it anyway :slight_smile:

···

On Sun, 11 May 2003 19:12:17 +0900, Brian Candler wrote:

On Sun, May 11, 2003 at 09:29:43AM +0900, Simon Strandgaard wrote:


Simon Strandgaard

I think the best description of the problem I have come up with so far is:
http://ruby-talk.com/cgi-bin/scat.rb/ruby/ruby-talk/71191

Separation of ruby stdout/stderr from C/C++ stdout/stderr is what I want.
I run Ruby in the same thread as C++ (ruby embedded into C++).
This separation is working fine, as long as I don’t use system or
backquote.

Using system or backquote, then it goes BOOM. The problem as I see it is
localized in Rubys code - not mine! Ruby spawns a child which inherits
stdout/stderr. This child should instead inherit rb_stdout/rb_stderr
instead!!! Can this issue be fixed on Ruby ???

Thanks for Ruby :slight_smile:

···

On Mon, 12 May 2003 18:32:47 +0900, Yukihiro Matsumoto wrote:

I’m not sure if I understand what you care, but at least I want you to
know it’s the field I want to fix/change before the final 1.8.0
release.


Simon Strandgaard

Separation of ruby stdout/stderr from C/C++ stdout/stderr is what I want.

Using system or backquote, then it goes BOOM. The problem as I see it is
localized in Rubys code - not mine! Ruby spawns a child which inherits
stdout/stderr. This child should instead inherit rb_stdout/rb_stderr
instead!!! Can this issue be fixed on Ruby ???

It can’t be fixed easily; if rb_stdout were pointing at a StringIO object,
then this cannot be represented as a Unix fd.

I mean, I’m guessing you want something like ‘$deferr’ like ‘$defout’,
right? So that rb_warn sends to $deferr, and you can map this variable to
point to some other object?

OK, but what do you want to happen here?

 $deferr = StringIO.new(x)
 system("ls /nonexistent")

Now, the Ruby ‘system’ call is a direct analogue of the Unix ‘system’ call:
fork, exec, wait; nothing more.

What do you want instead? That Ruby should create two new pipes, pass these
to the child as stdout/stderr; then read any data generated on those pipes,
and pass them to $defout.write and $deferr.write respectively?

(Hence in the normal case where $deferr actually points to STDERR, the
message will still get written to STDERR but indirectly, by being read from
the child and then written to STDERR by the parent)

The problems I can see with this are:

  • It adds overhead even for the 99% of cases where this is not needed
  • It doesn’t follow the Unix API model, therefore violates POLS:
    “system” should not create pipes to the child. “popen” should.

But there are more fundamental problems. Firstly, your Ruby function can
call C at any time: it’s as easy to embed C in Ruby as Ruby in C. What
happens when the C function does a write to stdout or stderr? You can’t
possibily intercept it without modifying the C. What if the C in turn calls
system() to run a child process, and the child process writes to stderr?
Hard luck, it gets written to your main program’s stderr.

Secondly, you have to modify Ruby in every case where it might generate any
output to stderr, and recode it to use $deferr. For example:
output = %x(somecommand}

would have to generate an extra pipe for reading stderr, so that it could be
written to $deferr.

Thirdly and more importantly, what are you going to do about a ‘fork’
between two Ruby processes? Consider this:

pid = fork do
  sleep 5
  $deferr.puts "hello"
end
$deferr = StringIO.new(x)
Process.waitpid(pid)

By your logic, the child process when it writes to its own “private” Ruby
stderr should not interfere with the parent’s stderr, right? That means that
you are requiring Ruby not to do just a ‘fork’ where I’ve written ‘fork’,
but also to set up pipes with its child process, capture its stderr, and
write it to the local $deferr object.

This is IMO nonsense. An API should not try to do magical things like this,
because it will break code which tries to use the API properly.

So:

  • “fork” should just do a fork()
  • “system” should just do system() [fork, exec, wait]
  • “popen” should set up pipes, fork, exec, and map the pipe to an IO
    object.

This is what Ruby does already, of course.

Finally, and perhaps most importantly, a Ruby program is a Unix program
(when running under Unix!). It seems to me incredibly logical that a Ruby
program should have a STDOUT and a STDERR which map directly to the Unix
concepts of STDOUT and STDERR.

C Ruby


printf(“hello,world\n”); puts “hello, world”
fprintf(stderr,“Bad value\n”); STDERR.puts “Bad value”

That’s how it should be. In C, if you wanted to write your errors to a GUI
window instead (say), you’d write a function to do that, and you would call
that function instead of fprintf. In Ruby, it’s the same: you write a method
which puts the error in the window, and call that instead of STDERR.puts

put_error(window,“Bad value”); window.put_error(“Bad value”)

In both cases, if you want to take existing code which writes directly to
stderr, without modifying it, you can run it as a child process and capture
it using a pipe. It’s easy enough to code this in either language, in the
odd case where you need it.

At the end of the day, they’re both programming languages, they both map
onto an underlying operating system in a particular and familiar and logical
way, it seems to me stupid to try and bypass this magically, and it can’t
possibly be made to work in the general case anyway.

Brian.

···

On Mon, May 12, 2003 at 07:16:51AM +0900, Simon Strandgaard wrote:

Hi,

···

In message “Re: RCR for child execution” on 03/05/12, “Simon Strandgaard” 0bz63fz3m1qt3001@sneakemail.com writes:

main.cpp:40: non-lvalue in assignment
main.cpp:41: non-lvalue in assignment

That’s because stdout is not a variable: on my FreeBSD box it is defined as

All my boxes run FreeBSD5.0 :slight_smile:
I don’t get any warnings/errors.

stdin, stdout, stderr might not be assignable. It’s undefined
behavior. Although it seems to work on FreeBSD, not on other
platforms.

						matz.