Popen issues

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen("/bin/bash",“r+”) do |f|
f.puts "ls"
puts f.readlines
f.puts "date"
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen("/bin/bash",“r+”) do |f|
writer = f.dup
f.puts "ls\n"
f.close_write
puts f.readlines
w = writer.dup
w.puts "date\n"
w.close_write
puts f.readlines
end

Can someone advise?

Hal

~/tmp > ls
b.rb

~/tmp > cat b.rb
require ‘session’
bash = Session::Bash.new

puts ‘—’
cmd = ‘ls’
p cmd
o,e = bash.execute cmd
p o
p e

puts ‘—’
cmd = ‘date’
p cmd
bash.execute(cmd) do |o,e|
o and p o
e and p e
end

puts ‘—’
cmd = ‘find’
bash.errproc = lambda{|e| p e}
bash.outproc = lambda{|o| p o}
bash.execute cmd

~/tmp > ruby b.rb

···

On Wed, 28 Apr 2004, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines
f.puts “date”
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen(“/bin/bash”,“r+”) do |f|
writer = f.dup
f.puts “ls\n”
f.close_write
puts f.readlines
w = writer.dup
w.puts “date\n”
w.close_write
puts f.readlines
end

Can someone advise?

Hal


“ls”
“b.rb\n”
“”

“date”
“Tue Apr 27 15:40:21 MDT 2004\n”

“.\n”
“./b.rb\n”

it’s horribly more complex than it first appears. read the source to see why (RAA)

  • you don’t know whey the output of one command starts and the next begins
  • you can’t to non-blocking reads from threads
  • you can’t send commands w/o blocking entire processs unless carefull

i believe my package is thread safe - as in it doesn’t block them, not as in
many can use a session at once. at least i using it for many multi-threaded
apps.

this took my over a week to write - you may want to save the trouble - the
source is really straight forward to read. (IMHO)

-a

===============================================================================

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================

Hi,

I don’t know how to do it in pure Ruby, but if you are
using a GUI (Gtk) it is fairly easy.

pipe = IO.popen(“prog”, “w+”)
Gtk::input_add(pipe.to_i, Gdk::INPUT_READ) {
str = pipe.gets
puts "got " + str }

This will work only from the event-dispatch routine inside
Gtk.main().

Kristof

···

On Wed, 28 Apr 2004 06:06:30 +0900, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Hi,

It is maybe not a good idea to use /bin/bash as a pipe,
because it is an interactive interpreter, and not made
for pipe-communication. Normally you would just check
for an end of file byte, but bash will not send you
one. Is there any reason to use bash?

···

On Wed, 28 Apr 2004 06:06:30 +0900, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines
f.puts “date”
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen(“/bin/bash”,“r+”) do |f|
writer = f.dup
f.puts “ls\n”
f.close_write
puts f.readlines
w = writer.dup
w.puts “date\n”
w.close_write
puts f.readlines
end

Can someone advise?

Hal

Hi,

At Wed, 28 Apr 2004 06:06:30 +0900,
Hal Fulton wrote in [ruby-talk:98572]:

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines

IO#readlines reads all of the lines, i.e., blocks until the all
child processes exit or close the stdout. You probably can use
sysread instead, and IO#readpartial proposed by akr (see
[ruby-talk:96220]) would help you if it is implemented.

···


Nobu Nakada

I think putting the process in a pty will give you better control.

Dan

···

On Apr 27, 2004, at 17:06, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines
f.puts “date”
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen(“/bin/bash”,“r+”) do |f|
writer = f.dup
f.puts “ls\n”
f.close_write
puts f.readlines
w = writer.dup
w.puts “date\n”
w.close_write
puts f.readlines
end

Can someone advise?

Hal

Ara.T.Howard wrote:

[snip]

it’s horribly more complex than it first appears. read the source to see why (RAA)

  • you don’t know whey the output of one command starts and the next begins
  • you can’t to non-blocking reads from threads
  • you can’t send commands w/o blocking entire processs unless carefull

i believe my package is thread safe - as in it doesn’t block them, not as in
many can use a session at once. at least i using it for many multi-threaded
apps.

this took my over a week to write - you may want to save the trouble - the
source is really straight forward to read. (IMHO)

Very cool, Ara, thanks. I’ll take a look at it.

Hal

Hi,

It is maybe not a good idea to use /bin/bash as a pipe, because it is an
interactive interpreter, and not made for pipe-communication. Normally you
would just check for an end of file byte, but bash will not send you one.
Is there any reason to use bash?

really? mine does:

~ > ruby -e ‘p IO.popen(“bash -c exit”).eof?’
true

~ > uname -srm && bash --version
uname -srm && bash --version
Linux 2.4.20-30.73ncirt i686
GNU bash, version 2.05a.0(1)-release (i686-pc-linux-gnu)
Copyright 2001 Free Software Foundation, Inc.

how about you?

-a

···

On Wed, 28 Apr 2004, Kristof Bastiaensen wrote:

On Wed, 28 Apr 2004 06:06:30 +0900, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines
f.puts “date”
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen(“/bin/bash”,“r+”) do |f|
writer = f.dup
f.puts “ls\n”
f.close_write
puts f.readlines
w = writer.dup
w.puts “date\n”
w.close_write
puts f.readlines
end

Can someone advise?

Hal

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================

Of course, this makes sense. Thank you.

I don’t remember #readpartial but I will read about it.

Thanks,
Hal

···

nobu.nokada@softhome.net wrote:

Hi,

At Wed, 28 Apr 2004 06:06:30 +0900,
Hal Fulton wrote in [ruby-talk:98572]:

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines

IO#readlines reads all of the lines, i.e., blocks until the all
child processes exit or close the stdout. You probably can use
sysread instead, and IO#readpartial proposed by akr (see
[ruby-talk:96220]) would help you if it is implemented.

I think putting the process in a pty will give you better control.

Dan

dan-

have you seen my session package? do you think i should re-vamp this to run
all processes in a pty? i can’t think of a good reason NOT to at this point,
but aren’t currently…

-a

···

On Fri, 30 Apr 2004, Dan Janowski wrote:

On Apr 27, 2004, at 17:06, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines
f.puts “date”
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen(“/bin/bash”,“r+”) do |f|
writer = f.dup
f.puts “ls\n”
f.close_write
puts f.readlines
w = writer.dup
w.puts “date\n”
w.close_write
puts f.readlines
end

Can someone advise?

Hal

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================

Hi,

It is maybe not a good idea to use /bin/bash as a pipe, because it is an
interactive interpreter, and not made for pipe-communication. Normally you
would just check for an end of file byte, but bash will not send you one.
Is there any reason to use bash?

really? mine does:

~ > ruby -e ‘p IO.popen(“bash -c exit”).eof?’
true

how about you?

Indeed, if you call bash with an argument it does.
But if you call just bash, like in the previous example, it
doesn’t. Instead of IO.popen(“/bin/bash -c command”),
you could better do just IO.popen(“command”), (unless of
course you want bash specific features).

Kristof

···

On Tue, 27 Apr 2004 17:16:24 -0600, Ara.T.Howard wrote:

-a

On Wed, 28 Apr 2004 06:06:30 +0900, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines
f.puts “date”
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen(“/bin/bash”,“r+”) do |f|
writer = f.dup
f.puts “ls\n”
f.close_write
puts f.readlines
w = writer.dup
w.puts “date\n”
w.close_write
puts f.readlines
end

Can someone advise?

Hal

a-

I have not seen your session package (nose too close to grind stone at
the moment). I would not use ptys if you have code control over the
child, since any behavior issues can be worked out. In most other
cases, a pty has lots of advantages: stdio buffering turned off, good
process control since the master is effectively the controlling
terminal, no intermediate shell, etc. I would guess that character
throughput is less than with popen because of the pty stack, but I
could not tell you how much or if it matters.

-d

···

On Apr 29, 2004, at 15:14, Ara.T.Howard wrote:

On Fri, 30 Apr 2004, Dan Janowski wrote:

I think putting the process in a pty will give you better control.

Dan

dan-

have you seen my session package? do you think i should re-vamp this
to run
all processes in a pty? i can’t think of a good reason NOT to at this
point,
but aren’t currently…

-a

On Apr 27, 2004, at 17:06, Hal Fulton wrote:

Here’s something I’ve never figured out.

Suppose I want real two-way communication with a process via popen.
This seems doable.

But apparently the read hangs until I do a close_write; and if I
do that, I can’t write again. A flush doesn’t work.

Here’s what I was thinking (simplified):

io = IO.popen(“/bin/bash”,“r+”) do |f|
f.puts “ls”
puts f.readlines
f.puts “date”
puts f.readlines
end

I even tried this (following) abomination, to no avail:

io = IO.popen(“/bin/bash”,“r+”) do |f|
writer = f.dup
f.puts “ls\n”
f.close_write
puts f.readlines
w = writer.dup
w.puts “date\n”
w.close_write
puts f.readlines
end

Can someone advise?

Hal

========

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print
"\x3a\x2d\x29\x0a"”;done
=======================================================================
========

Kristof Bastiaensen wrote:

Indeed, if you call bash with an argument it does.
But if you call just bash, like in the previous example, it
doesn’t. Instead of IO.popen(“/bin/bash -c command”),
you could better do just IO.popen(“command”), (unless of
course you want bash specific features).

In my example, I was just using bash as an interactive
program.

Actually I tried bc first, with similarly no luck.

It’s not a question about bash or bc, but about popen.

Hal