What I'm trying to do is to pass some data
to a child process that execute an external process
and get back data from this external process.
···
---
PATH = "/usr/bin/highlight"
ARGS = "-f -l -t 8 -S %s"
LANG = "rb"
rd, wr = IO.pipe
if fork
# parent
rd.close
$stdout.reopen(wr)
wr.close
fd = File.open("/usr/lib/ruby/1.8/thread.rb")
str = fd.read
$stdout.write str
else
# child
wr.close
$stdin.reopen(rd)
rd.close
params = ARGS % LANG
cmd = "%s %s" % [PATH, params]
exec cmd
end
---
This works but child puts returned data to the standard output
(to the linux shell). I'd like to get back that output in the parent.
How can I do it?
--
Lawrence
http://www.oluyede.org/blog
harp:~ > cat a.rb
IO::popen('-') do |pipe|
if pipe
stdout = pipe.read
puts "parent got <#{ stdout }> from child"
else
exec 'echo', '-n', '42'
end
end
harp:~ > ruby a.rb
parent got <42> from child
you can also
STDERR.reopen '/dev/null'
in the child the shutup stderr - otherwise look at the code for open3 (it's
short) to see how to get a handle on both stdout and stderr of the child.
hth.
-a
···
On Thu, 12 May 2005, Lawrence Oluyede wrote:
What I'm trying to do is to pass some data
to a child process that execute an external process
and get back data from this external process.
---
PATH = "/usr/bin/highlight"
ARGS = "-f -l -t 8 -S %s"
LANG = "rb"
rd, wr = IO.pipe
if fork
# parent
rd.close
$stdout.reopen(wr)
wr.close
fd = File.open("/usr/lib/ruby/1.8/thread.rb")
str = fd.read
$stdout.write str
else
# child
wr.close
$stdin.reopen(rd)
rd.close
params = ARGS % LANG
cmd = "%s %s" % [PATH, params]
exec cmd
end
--
email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
renunciation is not getting rid of the things of this world, but accepting
that they pass away. --aitken roshi
===============================================================================
Ara.T.Howard@noaa.gov writes:
harp:~ > cat a.rb
IO::popen('-') do |pipe|
if pipe
stdout = pipe.read
puts "parent got <#{ stdout }> from child"
else
exec 'echo', '-n', '42'
end
end
harp:~ > ruby a.rb
parent got <42> from child
Works, but there's a problem. If I open the pipe in read/write
mode and exec a program that reads from stdin it doesn't work 
···
--
Lawrence
http://www.oluyede.org/blog
harp:~ > cat a.rb
IO::popen('-', 'r+') do |pipe|
if pipe
pipe.puts '42'
pipe.close_write
stdout = pipe.read.strip
#pipe.close
puts "parent got <#{ stdout }> from child"
else
exec 'cat'
end
end
harp:~ > ruby a.rb
parent got <42> from child
you aren't trying to run a program that requires a tty (like top) are you?
-a
···
On Fri, 13 May 2005, Lawrence Oluyede wrote:
Ara.T.Howard@noaa.gov writes:
harp:~ > cat a.rb
IO::popen('-') do |pipe|
if pipe
stdout = pipe.read
puts "parent got <#{ stdout }> from child"
else
exec 'echo', '-n', '42'
end
end
harp:~ > ruby a.rb
parent got <42> from child
Works, but there's a problem. If I open the pipe in read/write mode and exec
a program that reads from stdin it doesn't work
--
email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
renunciation is not getting rid of the things of this world, but accepting
that they pass away. --aitken roshi
===============================================================================
Ara.T.Howard@noaa.gov writes:
pipe.close_write
That's what I missed!
you aren't trying to run a program that requires a tty (like top) are you?
No no...
Thanks a lot!
···
--
Lawrence
http://www.oluyede.org/blog
How can I deal with large inputs to an external process
popen driven? With 5k of data sent to its standard
input blocks.
···
--
Lawrence
http://www.oluyede.org/blog
send the data down in a thread
data = IO::read 'huge_file'
sender = Thread::new(data){data.each{|line| pipe.puts line}}
you'd probably then want a reader to load a thread safe queue with the
returned data.
why don't you want to block?
-a
···
On Fri, 13 May 2005, Lawrence Oluyede wrote:
How can I deal with large inputs to an external process
popen driven? With 5k of data sent to its standard
input blocks.
--
email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
renunciation is not getting rid of the things of this world, but accepting
that they pass away. --aitken roshi
===============================================================================
You need to send the data in blocks lequal to the pipe buffer size.
Something like pipe.write(input.read(4096)) until input.eof?
Also, if the external process outputs to its stdout, it will block once it
has written pipe buffer bytes, so you need to read from the pipe aswell.
So, one thread for writing, one thread for reading.
Btw, maybe popened IOs should do this automatically under covers?
test_popen.rb
d = ' '*32000
IO.popen('cat','r+'){|c|
Thread.new{ c.write(d); c.close_write }
puts c.read.size
}
Here's a way to fix the hang:
class IO
class << self
alias_method :real_popen, :popen
def popen(*args,&block)
io = real_popen(*args)
def io.write(data)
i = wb = 0
(wb += super(data[i,4096]); i+=4096) while i < data.size
wb
end
if block_given?
block.call(io)
else
io
end
end
end
end
···
On 13.5.2005, at 23:40, Lawrence Oluyede wrote:
How can I deal with large inputs to an external process
popen driven? With 5k of data sent to its standard
input blocks.
--
Lawrence
http://www.oluyede.org/blog
Ara.T.Howard@noaa.gov writes:
send the data down in a thread
data = IO::read 'huge_file'
sender = Thread::new(data){data.each{|line| pipe.puts line}}
you'd probably then want a reader to load a thread safe queue with the
returned data.
That's what I was doing before with popen3, wrap the function with a Thread.
Let me try again
why don't you want to block?
Cause it hangs the UI (a webpage)
···
--
Lawrence
http://www.oluyede.org/blog
Ilmari Heikkinen <kig@misfiring.net> writes:
You need to send the data in blocks lequal to the pipe buffer size.
Something like pipe.write(input.read(4096)) until input.eof?
Tried send data line by line but it still hangs
Btw, maybe popened IOs should do this automatically under covers?
I think it does but my code works behind a Rails/cgi app and it doesn't work.
I'm planning to write a Ruby code replacement for the external process.
Thanks anyway
···
--
Lawrence
http://www.oluyede.org/blog
ah, why didn't you say so? 
that's __exactly__ why i designed my session
lib - to NOT hang tk apps. it's thread safe so you can do this:
class UI
def initialize
@session = Session::new
@stdout_widget = SomeWidget::new
@stderr_widget = SomeWidget::new
end
def button_pressed
command = get_system_command
Thread::new(session, command) do |s, c|
s.execute(c) do |stdout, stderr|
@stdout_widget.update stdout if stdout
@stderr_widget.update stderr if stderr
end
end
end
end
and this runs in the background. the block for stdout and stderr are handled
as output is produced. session runs commands in the shell and so you don't
have a handle on the stdin in the process, but this would be easy to solve via
require 'tempfile'
def run_background_command command, input
tmp = Tempfile::new rand.to_s
begin
tmp.write input
tmp.close
command = "#{ command } < #{ tmp.path }"
Thread::new do
@session.execute(command) do |stdout, stderr|
async_handle stdout if stdout
async_handle stderr if stderr
end
end
ensure
tmp.close! if tmp
end
end
...
dont_forget_to_join_this_thread = run_background_command 'ui_hanger.exe', 'abc'
...
dont_forget_to_join_this_thread.join
make sense?
http://www.codeforpeople.com/lib/ruby/session/
http://raa.ruby-lang.org/project/session/
hth.
-a
···
On Fri, 13 May 2005, Lawrence Oluyede wrote:
Ara.T.Howard@noaa.gov writes:
send the data down in a thread
data = IO::read 'huge_file'
sender = Thread::new(data){data.each{|line| pipe.puts line}}
you'd probably then want a reader to load a thread safe queue with the
returned data.
That's what I was doing before with popen3, wrap the function with a Thread.
Let me try again
why don't you want to block?
Cause it hangs the UI (a webpage)
--
email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
renunciation is not getting rid of the things of this world, but accepting
that they pass away. --aitken roshi
===============================================================================
Ara.T.Howard@noaa.gov writes:
ah, why didn't you say so? that's __exactly__ why i designed my session
lib - to NOT hang tk apps. it's thread safe so you can do this:
Seems cool but I don't get it work with my own
example. Popen/forks/threads/hangs are causing me a big headache in those
days 
Everything (popen and my own fork based code) seems to work except
when it's behind a web app.
I think I should write a Ruby replacement of the external process
···
--
Lawrence
http://www.oluyede.org/blog
Lawrence Oluyede <raims@dot.com> writes:
I think I should write a Ruby replacement of the external process
I think I can wait
Ara you are great, the 2.4.0 version of your session lib resolves
all my problems 
Thanks a lot
···
--
Lawrence
http://www.oluyede.org/blog