> I've never tried using this function before, but I am now using as part
> of a code compilation build script (using 'make' on UNIX).
>
> It looks something like this:
>
> ==================================
>
> stdin, stdout, stderr = Open3.popen3( 'make codename' )
> stdin.close
>
> puts "Reading STDOUT"
> outfile.puts stdout.read
> stdout.close
>
> ==================================
>
> I'm trying to get the output from the 'make' command (which is quite
> large), so I can dump it to a file for later diagnostics.
There are a couple ways to tackle what you want to do, but one big
problem with what you're doing here is that you're trying to read *all*
of the output from stdout into memory at once. I'm not sure how big
"quite large" is, but you should probably do this line by line or in
limited block sizes instead.
stdout.each_line do |line|
outfile.write(line)
end
If you use #each_line then I would also use #puts for output because both
are line oriented.
OR
while (data = stdout.read(1024)).size > 0
outfile.write(data)
end
That doesn't work because #read returns nil at EOF. You just need
while data = stdout.read(1024)
outfile.write(data)
end
OR, a tad more efficient
data = ""
while stdout.read(1024, data)
outfile.write(data)
end
> However, when it gets to the stdout.read, it seems to hang (or at least
> take so long I've never seen it finish).
>
> Any thoughts on why this is happening? Is there a better approach for
> me to try?
You never read from stderr in your example. I'm wondering if the
subprocess isn't actually blocking while trying to write something to
stderr. Because you never read from it, the buffer for the pipe
connected to stderr may fill up. If that were to happen while the
subprocess was writing to stderr, the subprocess would block while
trying to write and never get to close its end of the stdout pipe and
exit. Thus your read from stdout will block forever.
Right!
You need to read from both stdout and stderr in your script to avoid
this problem; however, this can get complicated since you need to avoid
blocking while reading from either one of them. You could also skip
stderr entirely and just use IO.popen, in which case the stderr output
will go to the terminal or wherever your script's stderr was wired to
go. Finally, you could use IO.popen while redirecting the subprocess'
stderr to its stdout so that you can read both streams at once:
IO.popen('make codename 2>&1', 'r') do |pipe|
pipe.each_line do |line|
outfile.write(line)
end
end
I wouldn't want to do that because then I cannot differenciate between
regular and error output.
Given your stated goal of sending the output of make to a file for later
diagnostics, you would probably be better off simply redirecting the
output directly to a file and skipping ruby entirely:
bash$ make codename >/path/to/make.log 2>&1
Agree.
The above when run in the bash shell would send both the stdout and
stderr into the file /path/to/make.log. If you still wanted to see the
output on screen at the same time as logging to the file, you can use
the tee program:
bash$ make codename 2>&1 | tee /path/to/make.log
The same thing happens here as before but you'll also see the output in
the terminal while make runs.
And here's a solution using open3:
require 'open3'
Open3.popen3( %w{make codename} ) do |stdin, stdout, stderr, t|
stdin.close
err_thr = Thread.new { IO.copy_stream(stderr, outfile) }
puts "Reading STDOUT"
IO.copy_stream(stdout, outfile)
err_thr.join
end
Note: I also used an Array for the make command invocation because that
avoids parsing issues in the shell because it omits the shell altogether
(see Process.spawn).
You might also prefer a more line based approach
def copy_lines(str_in, str_out)
str_in.each_line {|line| str_out.puts line}
end
Open3.popen3( 'make codename' ) do |stdin, stdout, stderr, t|
stdin.close
err_thr = Thread.new { copy_lines(stderr, $stderr) }
puts "Reading STDOUT"
copy_lines(stdout, $stdout)
err_thr.join
end
Kind regards
robert
···
On Wed, May 15, 2013 at 5:11 AM, Jeremy Bopp <jeremy@bopp.net> wrote:
On 05/14/2013 05:38 PM, Thomas Luedeke wrote:
--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/