To retrieve piped input to my program I can use something like
ARGF.readlines
However, if there is no piped data, this will wait for input from the
terminal- how can I avoid this?
Thanks,
Greg Weber
greg wrote:
To retrieve piped input to my program I can use something like
ARGF.readlinesHowever, if there is no piped data, this will wait for input from the
terminal- how can I avoid this?
You can check if $stdin is a terminal with
$stdin.tty?
Vince
you can do something like
stdin =
if STDIN.tty?
''
else
STDIN.read
end
but that is an extremely bad idea because running the program under cron, or
even using something like
program &
will cause it to hang
the best/safest approach is to __always__ construct the command-line to give
clear notice that stdin will be following. the most common way to do this is
to pass '-' on the command line. for example
program -
program --verbosity=4 -
etc. then, the code is simple and always correct:
# parse options first
stdin = ARGV.delete '-'
if stdin
buf = STDIN.read
# ...
else
# ...
end
regards.
-a
···
On Thu, 28 Sep 2006, greg wrote:
To retrieve piped input to my program I can use something like
ARGF.readlinesHowever, if there is no piped data, this will wait for input from the
terminal- how can I avoid this?
--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei
thanks, a
This kind of sucks though.
apparently in perl, if there is a piped input, '-' will show up
automatically in ARGV. I think I will propose this change to Ruby.
thanks, a
This kind of sucks though.
why? is this hard?
stdin = ARGV.delete '-'
apparently in perl, if there is a piped input, '-' will show up
automatically in ARGV. I think I will propose this change to Ruby.
eeeks! that's is pure __evil__!
consider i have many, many programs which do things like
convert infile outfile
convert - outfile # infile on stdin
convert infile - # outfile on stdout
convert - - # infile on stdin, outfile on stdout
convert --infiles=- # list of infiles on stdin, auto-name outfiles
this is standard unix practice (do a man on gzip, tar, etc). now, if ruby
just started willy nilly adding a '-' to the command line this technique would
not work because it's not just the existence of '-' but the __position__ of it
which can have meaning.
auto-munging of ARGV is a bad idea imho.
regards.
-a
···
On Thu, 28 Sep 2006, greg wrote:
--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei
Normally I would just nod my head in agreement with Ara, but not this time.
I just have to say yuck with regard to using '-' and the idea that command lines
should have have an explicit syntax to indicate that the program should read from
stdin.
If your program is prepared to deal with piped input, why are
you concerned about input coming from the keyboard? If I found a program
that worked as:
cat file | program
and as
program < file
but did not work as just
program
I would be surprised, to say the least.
Usually the issue is the other way around where an interactive program (vim, for
example) *needs* a tty device on stdin and may complain when STDIN.tty? fails, but
I'm not sure I understand the need to complain about STDIN being tied to a tty device
for a program that is just reading a stream of data.
The '-' hack just makes me shudder. Something like '/dev/stdin' would be marginally better
if you really need something like that (although not as portable).
···
On Sep 27, 2006, at 4:27 PM, greg wrote:
thanks, a
This kind of sucks though.
apparently in perl, if there is a piped input, '-' will show up
automatically in ARGV. I think I will propose this change to Ruby.
greg wrote:
thanks, a
This kind of sucks though.
Why? One performs a test to determine if there is a pipe. That seems
reasonable.
apparently in perl, if there is a piped input, '-' will show up
automatically in ARGV.
Have you done an experiment to see if this is true? In fact, the user of the
script must provide this symbol as an argument if he wants the read a pipe,
then the script must interpret the presence of this symbol as a signal to
commence reading a pipe, by the way a very common *nix convention. It's all
very non-automatic.
I think I will propose this change to Ruby.
No sense doing it in Ruby if it isn't done in Perl. It isn't done in Perl.
···
--
Paul Lutus
http://www.arachnoid.com
/ ...
apparently in perl, if there is a piped input, '-' will show up
automatically in ARGV. I think I will propose this change to Ruby.eeeks! that's is pure __evil__!
It also isn't true. IMHO Larry Wall wouldn't countenance such a thing.
consider i have many, many programs which do things like
convert infile outfile
convert - outfile # infile on stdin
convert infile - # outfile on stdout
convert - - # infile on stdin, outfile on stdout
convert --infiles=- # list of infiles on stdin, auto-name outfiles
While I understand the historical reason for such constructs, I much prefer:
convert infile outfile
convert /dev/stdin outfile # infile on stdin
convert infile /dev/stdout # outfile on stdout
convert # infile on stdin, outfile on stdout
convert --infiles=- # list of infiles on stdin, auto-name outfiles
Not sure about this last one. It certainly isn't all that common. You can get close
to this via:
cat filenames | xargs convert
with maybe an option to convert to cause it to send output to auto-named outfiles:
cat filenames | xargs convert --inplace
I realize that some Unix systems don't support /dev/stdin, /dev/stdout, though...
Gary Wright
···
On Sep 27, 2006, at 4:42 PM, ara.t.howard@noaa.gov wrote:
sorry, misread some perl stuff.
How about a global flag or constant
like $PIPE set to true if there is a pipe, and otherwise set to false?
it may well work, but hang forever in cron though:
this is as easy to do as
infile = get_infile # oops, returned nil
system "program #{ infile }" # hangs forever
that's also very problematic for prorgams which take two arguments:
cat infile | program - outfile
cat infile | program - - | outfile_filter > outfile
note that you cannot simply say that the abscence of infile means stdin and
similar for outfile because
program infile_or_outfile?
i'm sure one could skin this differently, i just have debugged many hung
processes that make assumptions based on STDIN.tty?
kind regards.
-a
···
On Thu, 28 Sep 2006 gwtmp01@mac.com wrote:
If your program is prepared to deal with piped input, why are you concerned
about input coming from the keyboard? If I found a program that worked as:cat file | program
and as
program < file
but did not work as just
program
--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei
> apparently in perl, if there is a piped input, '-' will show up
> automatically in ARGV. I think I will propose this change to Ruby.eeeks! that's is pure __evil__!
Yes I agree with this, but...
consider i have many, many programs which do things like
convert infile outfile
convert - outfile # infile on stdin
convert infile - # outfile on stdout
convert - - # infile on stdin, outfile on stdout
convert --infiles=- # list of infiles on stdin, auto-name outfilesthis is standard unix practice (do a man on gzip, tar, etc).
Standard but by no means universal. For example the mysql command
doesn't use or like it:
rick@frodo:/public/rubyscripts$ mysql -p -
Enter password:
ERROR 1049 (42000): Unknown database '-'
It either tries to use - as the database name or:
rick@frodo:/public/rubyscripts$ mysql -p depot_development -
mysql Ver 14.12 Distrib 5.0.22, for pc-linux-gnu (i486) using readline 5.1
...
Usage: mysql [OPTIONS] [database]
It tells me that I don't know the right syntax.
I don't think that it has been mentioned here that one reason for
detecting what's connected to stdin/stdout is useful when a tool
want's to have interactive and non-interactive modes:
Here's mysql in non-interactive mode:
rick@frodo:/public/rubyscripts$ echo "describe products;" | mysql -p
depot_development
Enter password:
Field Type Null Key Default Extra
id int(11) NO PRI NULL auto_increment
title varchar(255) YES NULL
description text YES NULL
image_url varchar(255) YES NULL
price decimal(8,2) YES 0.00
rick@frodo:/public/rubyscripts$
And here it is in interactive mode:
rick@frodo:/public/rubyscripts$ mysql -p depot_development
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5691 to server version:
5.0.22-Debian_0ubuntu6.06-log
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
···
On 9/27/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:
describe products;
+-------------+--------------+------+-----+---------+----------------+
Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
id | int(11) | NO | PRI | NULL | auto_increment |
title | varchar(255) | YES | | NULL | |
description | text | YES | | NULL | |
image_url | varchar(255) | YES | | NULL | |
price | decimal(8,2) | YES | | 0.00 | |
+-------------+--------------+------+-----+---------+----------------+
5 rows in set (0.01 sec)
Note that it not only puts out prompts but it changes the output to
be for human rather than computer consumption.
auto-munging of ARGV is a bad idea imho.
I agree but for other reasons.
As for programs hanging if you do
program &
Well, that's really a user error, and maybe even not that, maybe I
want to suspend program and then use fg to resume it.
And - doesn't really help this. I think that it's better in most
cases to solve it the other way around with something like
program </dev/null &
--
Rick DeNatale
My blog on Ruby
http://talklikeaduck.denhaven2.com/
greg wrote:
sorry, misread some perl stuff.
How about a global flag or constant
like $PIPE set to true if there is a pipe, and otherwise set to false?
Are you asking whether this is true, or whether it would be a good idea? If
the latter, I personally think stuff like this should be explicitly coded
by the programmer. There are situations where detecting the
presence/absence of a pipe may have side effects best avoided.
Of course, reasonable people may (will) differ.
···
--
Paul Lutus
http://www.arachnoid.com
thanks - that's good to know. larry++.
-a
···
On Thu, 28 Sep 2006, Paul Lutus wrote:
ara.t.howard@noaa.gov wrote:
/ ...
apparently in perl, if there is a piped input, '-' will show up
automatically in ARGV. I think I will propose this change to Ruby.eeeks! that's is pure __evil__!
It also isn't true. IMHO Larry Wall wouldn't countenance such a thing.
--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei
consider i have many, many programs which do things like
convert infile outfile
convert - outfile # infile on stdin
convert infile - # outfile on stdout
convert - - # infile on stdin, outfile on stdout
convert --infiles=- # list of infiles on stdin, auto-name outfilesWhile I understand the historical reason for such constructs, I much prefer:
convert infile outfile
convert /dev/stdin outfile # infile on stdin
convert infile /dev/stdout # outfile on stdout
convert # infile on stdin, outfile on stdout
agreed.
convert --infiles=- # list of infiles on stdin, auto-name outfiles
Not sure about this last one. It certainly isn't all that common. You can
get close to this via:
harp:~ > printf "stdin\nstdout\nstderr\n" | grep -i -f- a.rb
STDERR.puts "#{ msg } (#{ tid })"
STDIN.gets
we have lots of code that does this because they take thousands of infiles,
hdrfiles, etc...
cheers.
-a
···
On Thu, 28 Sep 2006 gwtmp01@mac.com wrote:
On Sep 27, 2006, at 4:42 PM, ara.t.howard@noaa.gov wrote:
--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei
STDIN.tty?
-a
···
On Thu, 28 Sep 2006, greg wrote:
sorry, misread some perl stuff.
How about a global flag or constant
like $PIPE set to true if there is a pipe, and otherwise set to false?
--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei
it may well work, but hang forever in cron though:
Hmm. Doesn't cron arrange for stdin and stdout to be /dev/null, in which case
the read will simply return EOF, correct?
I just tried some experiments on Mac OS X and it seems that cron arranges for
stdin to be an empty pipe for cron jobs.
In any case, clearly the program has been invoked incorrectly for that
context (background cron job) and it pretty much doesn't matter what structure
you choose for the command line arguments it is always possible to screw it
up and provide the wrong arguments or plumb the wrong input or output sources.
i'm sure one could skin this differently, i just have debugged many hung
processes that make assumptions based on STDIN.tty?
I'm not sure I understand. The OP was suggesting that the program was designed
to read from a pipe, so it really shouldn't care if it was reading from a tty
device, should it?
I think of this like duck typing in Ruby. As long as you can read lines
from the file descriptor and your program is designed to read lines of text,
should you really care if stdin is associated with a file, a pipe, a network
connection, or a tty device? Why not just read and process the data?
Gary Wright
···
On Sep 27, 2006, at 5:50 PM, ara.t.howard@noaa.gov wrote:
Thanks for all the info guys. Here is what I came up with.
This code seems invincible so far- it even works with &
require 'thread'
input = nil
t = Thread.new() do
input = STDIN.readlines
end
# do some stuff
# parse some options
while t.alive?
old = input
puts "thread running for 1 second"
Kernel.sleep(1)
if old == input
if input
puts "done collecting input"
else
puts "no input found on STDIN"
end
Kernel.sleep(1)
t.kill
break
else
puts "done collecting input"
Kernel.sleep(1)
end
end
puts input
Greg
Didn't you just say that this is a bad idea because of program & and
other cases?
Hmm. Doesn't cron arrange for stdin and stdout to be /dev/null, in which
case the read will simply return EOF, correct?I just tried some experiments on Mac OS X and it seems that cron arranges
for stdin to be an empty pipe for cron jobs.
hmmm. you are right. i had to look back at what had given me trouble before
- the crux of it was basically this
input_on_stdin = not STDIN.tty?
infile =
if input_on_stdin
STDIN.read
end
so, basically assuming that the user gave input on stdin simply because STDIN
was not a tty, which may or may not be true, as in the case where one is
running under cron or a system call from a daemon, etc.
i guess my point is that
'stdin is not a tty'
and
'input was given on stdin'
are two orthogonal ideas which, when merged, have bitten me in the past. so
that's why i've developed a preference for '-' to signal input on stdin - it's
non-ambiguous.
In any case, clearly the program has been invoked incorrectly for that
context (background cron job) and it pretty much doesn't matter what
structure you choose for the command line arguments it is always possible to
screw it up and provide the wrong arguments or plumb the wrong input or
output sources.
indeed.
I'm not sure I understand. The OP was suggesting that the program was
designed to read from a pipe, so it really shouldn't care if it was reading
from a tty device, should it?
well - that is true. maybe i confused things by making it more general. the
only reason i was making a fuss is that it's caused me problems - as i
mentioned. probably the noise wasn't worth it.
I think of this like duck typing in Ruby. As long as you can read lines
from the file descriptor and your program is designed to read lines of text,
should you really care if stdin is associated with a file, a pipe, a network
connection, or a tty device? Why not just read and process the data?
i think that you are quite right in this case.
cheers.
-a
···
On Thu, 28 Sep 2006 gwtmp01@mac.com wrote:
--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei