From getoptlong to optparse

ruby-talk
1

I have a command line program invoked like this:

  $ prog --host localhost --port 8000 login --password secret
  $ prog --host localhost --port 8000 list --open

....and so on. In other words:

  $ command [general options] subcommand [specific options]

Like darcs or cvs.

So far I've been doing fine with getoptlong:

    while ARGV.first =~ /^-/
      opt, arg = opts.get
      case opt
      when "--host"
        config["host"] = arg
      when "--port"
        config["port"] = arg.to_i
      end
    end

When the loop sees the first thing that does not begin with `-'
(option arguments don't count, they get eaten by opts.get), it assumes
it to be the subcommand, and leaves ARGV alone from that point on.
ARGV is later emptied by the subcommand specific options parsing.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Massimiliano

2

"Massimiliano Mirra - bard" <mmirra@libero.REMOVETHIS.it> schrieb im
Newsbeitrag news:87ekjcrnwy.fsf@prism.localnet...

I have a command line program invoked like this:

  $ prog --host localhost --port 8000 login --password secret
  $ prog --host localhost --port 8000 list --open

...and so on. In other words:

  $ command [general options] subcommand [specific options]

Like darcs or cvs.

So far I've been doing fine with getoptlong:

    while ARGV.first =~ /^-/
      opt, arg = opts.get
      case opt
      when "--host"
        config["host"] = arg
      when "--port"
        config["port"] = arg.to_i
      end
    end

I don't think the code above is proper GetoptLong code. I don't remember
exactly but the occurrence of ARGV in the loop makes me suspicious.

When the loop sees the first thing that does not begin with `-'
(option arguments don't count, they get eaten by opts.get), it assumes
it to be the subcommand, and leaves ARGV alone from that point on.
ARGV is later emptied by the subcommand specific options parsing.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Command line option parsing libs normally do that for you as long as you
don't have any special requirements. If you need to stop parsing options
after a special option #terminate might do what you need. There's a quite
complete example at the optparse rdoc page:
http://www.ruby-doc.org/stdlib/libdoc/optparse/rdoc/classes/OptionParser.html

Reconsidering your situation this is what I'd do: iterative parse all
remaining options starting with the full set at the beginning. Evaluate
the first non option, this is your next command. Eat it and go to the
first step only now remembering that options are stored for this command.
Repeat until there are no more args left or the first non option is not a
command (i.e. file name, ignorable or whatever).

Kind regards

    robert

3

You may want to take a look at cmdparse.rubyforge.org. It is based on optparse, but provides the semantics of subcommands as you called them. Your command line

  $ command [general options] subcommand [specific options]

is a prime example for the use of it. Have a look at the example at this page http://cmdparse.rubyforge.org/rdoc/classes/CommandParser.html

*hth*,
Thomas Leitner

···

On Tue, 02 Nov 2004 20:33:15 GMT Massimiliano Mirra - bard <mmirra@libero.REMOVETHIS.it> wrote:

I have a command line program invoked like this:

  $ prog --host localhost --port 8000 login --password secret
  $ prog --host localhost --port 8000 list --open

...and so on. In other words:

  $ command [general options] subcommand [specific options]

Like darcs or cvs.

So far I've been doing fine with getoptlong:

    while ARGV.first =~ /^-/
      opt, arg = opts.get
      case opt
      when "--host"
        config["host"] = arg
      when "--port"
        config["port"] = arg.to_i
      end
    end

When the loop sees the first thing that does not begin with `-'
(option arguments don't count, they get eaten by opts.get), it assumes
it to be the subcommand, and leaves ARGV alone from that point on.
ARGV is later emptied by the subcommand specific options parsing.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Massimiliano

4

Hi,

At Wed, 3 Nov 2004 05:33:48 +0900,
Massimiliano Mirra - bard wrote in [ruby-talk:118836]:

I have a command line program invoked like this:

  $ prog --host localhost --port 8000 login --password secret
  $ prog --host localhost --port 8000 list --open

....and so on. In other words:

  $ command [general options] subcommand [specific options]

Like darcs or cvs.

I had included an example file cmd.rb (and its incidental file
cmd-ls.rb), in sole packaged optparse, but they are not bundled
now.

#! /usr/bin/ruby
require 'optparse'
require 'pp'

@host = @port = nil

# general options
opt = OptionParser.new do |opt|
  opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
  opt.define(" general options:")
  opt.define('--host HOST', String) {|host| @host = host}
  opt.define('--port PORT', Integer) {|port| @port = port}
end

# specific option initializers
specific = {
  "login" => proc do |opt|
    @passwd = nil
    opt.define("--password PASSWORD", String) {|passwd| @passwd = passwd}
  end,
  "list" => proc do |opt|
    @open = false
    opt.define("--open") {|open| @open = true}
  end,
}
missing = proc {|opt|
  opt.define([" subcommand:", specific.keys.sort].join("\n\t"))
}

begin
  cmd = nil
  opt.order!(ARGV) {|cmd| opt.terminate}
  opt = OptionParser.new(opt) do |opt|
    unless cmd
      abort "#{$0}: command missing\n#{missing[opt]}"
    end
    subopt = specific.fetch(cmd) do
      abort "#{$0}: unknown command: #{cmd}\n#{missing[opt]}"
    end
    opt.define(" specific options for #{cmd}:")
    subopt[opt]
  end
  opt.parse!(ARGV)
rescue OptionParser::ParseError => e
  abort "#{e}\n#{opt}"
end
pp self

···

--
Nobu Nakada

5

"Robert Klemme" <bob.news@gmx.net> writes:

    while ARGV.first =~ /^-/
      opt, arg = opts.get
      case opt
      when "--host"
        config["host"] = arg
      when "--port"
        config["port"] = arg.to_i
      end
    end

I don't think the code above is proper GetoptLong code. I don't remember
exactly but the occurrence of ARGV in the loop makes me suspicious.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Command line option parsing libs normally do that for you as long as you
don't have any special requirements. If you need to stop parsing options
after a special option #terminate might do what you need.

Not exactly. I need to stop parsing when a special command is seen.
"--host" and "--port" in the above example might appear in a different
order or not appear at all, so they're not reliable.

Reconsidering your situation this is what I'd do: iterative parse all
remaining options starting with the full set at the beginning.

Bypassing optparse?

Evaluate the first non option, this is your next command.

This needs to go hand in hand with option parsing. Consider this:

  $ prog --opt1 arg1 --opt2 stuff --blah

Is `stuff' a subcommand or opt2's argument? That depends on opt2's
definition (which in GetoptLong means NO_ARGUMENT or
REQUIRED_ARGUMENT). Then again, if you're thinking that things get
muddy when opt2 has an optional argument, I do agree...

Eat it and go to the first step only now remembering that options
are stored for this command. Repeat until there are no more args
left or the first non option is not a command (i.e. file name,
ignorable or whatever).

How is this different from the ARGV loop above?

Massimiliano

6

Thomas Leitner <t_leitner@gmx.at> writes:

You may want to take a look at cmdparse.rubyforge.org. It is based
on optparse, but provides the semantics of subcommands as you called
them. Your command line

Seems what I need. The only drawback is that it would be an external
dependency, so I'll still try to bend optparse to my will :-), but
failing that I'll go with cmdparse.

Thanks!
Massimiliano

7

thanks nobu! i've been dying to know how to do this!

-a

#! /usr/bin/ruby
require 'optparse'
require 'pp'

@host = @port = nil

# general options
opt = OptionParser.new do |opt|
opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
opt.define(" general options:")
opt.define('--host HOST', String) {|host| @host = host}
opt.define('--port PORT', Integer) {|port| @port = port}
end

# specific option initializers
specific = {
"login" => proc do |opt|
   @passwd = nil
   opt.define("--password PASSWORD", String) {|passwd| @passwd = passwd}
end,
"list" => proc do |opt|
   @open = false
   opt.define("--open") {|open| @open = true}
end,
}
missing = proc {|opt|
opt.define([" subcommand:", specific.keys.sort].join("\n\t"))
}

begin
cmd = nil
opt.order!(ARGV) {|cmd| opt.terminate}
opt = OptionParser.new(opt) do |opt|
   unless cmd
     abort "#{$0}: command missing\n#{missing[opt]}"
   end
   subopt = specific.fetch(cmd) do
     abort "#{$0}: unknown command: #{cmd}\n#{missing[opt]}"
   end
   opt.define(" specific options for #{cmd}:")
   subopt[opt]
end
opt.parse!(ARGV)
rescue OptionParser::ParseError => e
abort "#{e}\n#{opt}"
end
pp self

--
Nobu Nakada

-a

···

On Wed, 10 Nov 2004 nobu.nokada@softhome.net wrote:
--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
When you do something, you should burn yourself completely, like a good
bonfire, leaving no trace of yourself. --Shunryu Suzuki

===============================================================================

8

<nobu.nokada@softhome.net> schrieb im Newsbeitrag
news:200411100738.iAA7cYlO006236@sharui.nakada.niregi.kanuma.tochigi.jp...

Hi,

At Wed, 3 Nov 2004 05:33:48 +0900,
Massimiliano Mirra - bard wrote in [ruby-talk:118836]:
> I have a command line program invoked like this:
>
> $ prog --host localhost --port 8000 login --password secret
> $ prog --host localhost --port 8000 list --open
>
> ....and so on. In other words:
>
> $ command [general options] subcommand [specific options]
>
> Like darcs or cvs.

I had included an example file cmd.rb (and its incidental file
cmd-ls.rb), in sole packaged optparse, but they are not bundled
now.

#! /usr/bin/ruby
require 'optparse'
require 'pp'

@host = @port = nil

# general options
opt = OptionParser.new do |opt|
  opt.banner = "Usage: #{$0} [general options] subcommand [specific

options]"

  opt.define(" general options:")
  opt.define('--host HOST', String) {|host| @host = host}
  opt.define('--port PORT', Integer) {|port| @port = port}
end

# specific option initializers
specific = {
  "login" => proc do |opt|
    @passwd = nil
    opt.define("--password PASSWORD", String) {|passwd| @passwd =

passwd}

  end,
  "list" => proc do |opt|
    @open = false
    opt.define("--open") {|open| @open = true}
  end,
}
missing = proc {|opt|
  opt.define([" subcommand:", specific.keys.sort].join("\n\t"))
}

begin
  cmd = nil
  opt.order!(ARGV) {|cmd| opt.terminate}
  opt = OptionParser.new(opt) do |opt|
    unless cmd
      abort "#{$0}: command missing\n#{missing[opt]}"
    end
    subopt = specific.fetch(cmd) do
      abort "#{$0}: unknown command: #{cmd}\n#{missing[opt]}"
    end
    opt.define(" specific options for #{cmd}:")
    subopt[opt]
  end
  opt.parse!(ARGV)
rescue OptionParser::ParseError => e
  abort "#{e}\n#{opt}"
end
pp self

Somehow I though Massimiliano will have multiple sub commands - apparently
that was wrong. Out of curiosity: What does the code above do in the
light of multiple sub commands? As far as I understand it, once a
subcommand is encountered options for this subcommand are defined and
would remain valid once another subcommand is seen. Is that right? Or is
this scheme incapable of dealing with multiple sub commands.

Kind regards

    robert

9

"Massimiliano Mirra - bard" <mmirra@libero.REMOVETHIS.it> schrieb im
Newsbeitrag news:877jp2roei.fsf@prism.localnet...

"Robert Klemme" <bob.news@gmx.net> writes:

>> while ARGV.first =~ /^-/
>> opt, arg = opts.get
>> case opt
>> when "--host"
>> config["host"] = arg
>> when "--port"
>> config["port"] = arg.to_i
>> end
>> end
>
> I don't think the code above is proper GetoptLong code. I don't

remember

> exactly but the occurrence of ARGV in the loop makes me suspicious.

>> I'd like to migrate to optparse, though. But how can I stop argument
>> parsing at a certain point in optparse?
>
> Command line option parsing libs normally do that for you as long as

you

> don't have any special requirements. If you need to stop parsing

options

> after a special option #terminate might do what you need.

Not exactly. I need to stop parsing when a special command is seen.
"--host" and "--port" in the above example might appear in a different
order or not appear at all, so they're not reliable.

Do you mean the problem is that you need to stop when a command is seen
*and* this command can occur anywhere, i.e. also behind an option with an
optional argument?

> Reconsidering your situation this is what I'd do: iterative parse all
> remaining options starting with the full set at the beginning.

Bypassing optparse?

No, using it.

> Evaluate the first non option, this is your next command.

This needs to go hand in hand with option parsing. Consider this:

  $ prog --opt1 arg1 --opt2 stuff --blah

Is `stuff' a subcommand or opt2's argument? That depends on opt2's
definition (which in GetoptLong means NO_ARGUMENT or
REQUIRED_ARGUMENT).

You have the same (or finer) level of control for option arguments in
optparse, too.

Then again, if you're thinking that things get
muddy when opt2 has an optional argument, I do agree...

Yes, they certainly get muddy at this point. They get as muddy as to be
undecidable if you have a command that is also a valid optional argument
of an option. You can't decide what it is other than defining a priority
for the resolution of this ambiguity.

I'm not as much an optparse expert to come up with a quick solution. If
there are only special values allowed for a certain option, this might
help if optparse has support for it. (I believe I remembered saw
something about enumeration arguments but I'm not 100% sure.)

> Eat it and go to the first step only now remembering that options
> are stored for this command. Repeat until there are no more args
> left or the first non option is not a command (i.e. file name,
> ignorable or whatever).

How is this different from the ARGV loop above?

You have two nested loops, one of them is hidden in optparse.

Kind regards

    robert

10

Hi,

At Wed, 10 Nov 2004 23:23:31 +0900,
Robert Klemme wrote in [ruby-talk:119714]:

Somehow I though Massimiliano will have multiple sub commands - apparently
that was wrong. Out of curiosity: What does the code above do in the
light of multiple sub commands? As far as I understand it, once a
subcommand is encountered options for this subcommand are defined and
would remain valid once another subcommand is seen. Is that right? Or is
this scheme incapable of dealing with multiple sub commands.

Do you mean this?

  prog.rb --host example.com --port 1024 login --pass secret list --open

Then loop subcommand parser until ARGV becomes empty using
#order! instead of #parse!.

--- prog.rb 2004-11-12 10:33:03.000000000 +0900
+++ mprog.rb 2004-11-12 10:47:16.000000000 +0900
@@ -6,5 +6,5 @@ require 'pp'

# general options
-opt = OptionParser.new do |opt|
+general = OptionParser.new do |opt|
   opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
   opt.define(" general options:")
@@ -30,6 +30,7 @@ missing = proc {|opt|
begin
   cmd = nil
- opt.order!(ARGV) {|cmd| opt.terminate}
- opt = OptionParser.new(opt) do |opt|
+ (opt = general).order!(ARGV) {|cmd| opt.terminate}
+ begin
+ OptionParser.new(general) do |opt|
     unless cmd
       abort "#{$0}: command missing\n#{missing[opt]}"
@@ -40,6 +41,6 @@ begin
     opt.define(" specific options for #{cmd}:")
     subopt[opt]
- end
- opt.parse!(ARGV)
+ end.order!(ARGV) {|cmd| opt.terminate}
+ end until ARGV.empty?
rescue OptionParser::ParseError => e
   abort "#{e}\n#{opt}"

···

--
Nobu Nakada

11

"Robert Klemme" <bob.news@gmx.net> writes:

>> while ARGV.first =~ /^-/
>> opt, arg = opts.get
>> case opt
>> when "--host"
>> config["host"] = arg
>> when "--port"
>> config["port"] = arg.to_i
>> end
>> end

Not exactly. I need to stop parsing when a special command is seen.
"--host" and "--port" in the above example might appear in a different
order or not appear at all, so they're not reliable.

Do you mean the problem is that you need to stop when a command is seen
*and* this command can occur anywhere, i.e. also behind an option with an
optional argument?

Yes and no. Yes: this command can occur anywhere after one or more
options. No: I have no options with optional arguments so far, so I
haven't dealt with this case which I suspect to be more complex.

> Evaluate the first non option, this is your next command.

This needs to go hand in hand with option parsing. Consider this:

  $ prog --opt1 arg1 --opt2 stuff --blah

Is `stuff' a subcommand or opt2's argument? That depends on opt2's
definition (which in GetoptLong means NO_ARGUMENT or
REQUIRED_ARGUMENT).

You have the same (or finer) level of control for option arguments in
optparse, too.

The kind of control I am trying to get was not given by GetoptLong per
se, but by the `while ARGV.first =~ /^-/', which breaks down parsing
in chunks where the subcommand can be detected.

Then again, if you're thinking that things get
muddy when opt2 has an optional argument, I do agree...

Yes, they certainly get muddy at this point. They get as muddy as to be
undecidable if you have a command that is also a valid optional argument
of an option. You can't decide what it is other than defining a priority
for the resolution of this ambiguity.

Yes. Or just disallowing optional arguments for generic options, the
need hasn't arisen so far and I doubt it will at least for this application.

> Eat it and go to the first step only now remembering that options
> are stored for this command. Repeat until there are no more args
> left or the first non option is not a command (i.e. file name,
> ignorable or whatever).

How is this different from the ARGV loop above?

You have two nested loops, one of them is hidden in optparse.

Sorry but I still can't make a picture of what you are suggesting.
Would you mind sketching a couple of lines of pseudocode?

Massimiliano

12

<nobu.nokada@softhome.net> schrieb im Newsbeitrag
news:200411120150.iAC1otlO017036@sharui.nakada.niregi.kanuma.tochigi.jp...

Hi,

At Wed, 10 Nov 2004 23:23:31 +0900,
Robert Klemme wrote in [ruby-talk:119714]:
> Somehow I though Massimiliano will have multiple sub commands -

apparently

> that was wrong. Out of curiosity: What does the code above do in the
> light of multiple sub commands? As far as I understand it, once a
> subcommand is encountered options for this subcommand are defined and
> would remain valid once another subcommand is seen. Is that right?

Or is

> this scheme incapable of dealing with multiple sub commands.

Do you mean this?

  prog.rb --host example.com --port 1024 login --pass secret list --open

Yes, that's exactly the scenario I was thinking of ("login" and "list" as
sub commands).

Then loop subcommand parser until ARGV becomes empty using
#order! instead of #parse!.

I got lost somewhere in the diff below. :slight_smile: What eaxtly is the difference
between "order!" and "parse!"?

Kind regards

    robert

--- prog.rb 2004-11-12 10:33:03.000000000 +0900
+++ mprog.rb 2004-11-12 10:47:16.000000000 +0900
@@ -6,5 +6,5 @@ require 'pp'

# general options
-opt = OptionParser.new do |opt|
+general = OptionParser.new do |opt|
   opt.banner = "Usage: #{$0} [general options] subcommand [specific

options]"

···

   opt.define(" general options:")
@@ -30,6 +30,7 @@ missing = proc {|opt|
begin
   cmd = nil
- opt.order!(ARGV) {|cmd| opt.terminate}
- opt = OptionParser.new(opt) do |opt|
+ (opt = general).order!(ARGV) {|cmd| opt.terminate}
+ begin
+ OptionParser.new(general) do |opt|
     unless cmd
       abort "#{$0}: command missing\n#{missing[opt]}"
@@ -40,6 +41,6 @@ begin
     opt.define(" specific options for #{cmd}:")
     subopt[opt]
- end
- opt.parse!(ARGV)
+ end.order!(ARGV) {|cmd| opt.terminate}
+ end until ARGV.empty?
rescue OptionParser::ParseError => e
   abort "#{e}\n#{opt}"

--
Nobu Nakada

13

"Massimiliano Mirra - bard" <mmirra@libero.REMOVETHIS.it> schrieb im
Newsbeitrag news:87k6sxr9zo.fsf@prism.localnet...

"Robert Klemme" <bob.news@gmx.net> writes:

>> >> while ARGV.first =~ /^-/
>> >> opt, arg = opts.get
>> >> case opt
>> >> when "--host"
>> >> config["host"] = arg
>> >> when "--port"
>> >> config["port"] = arg.to_i
>> >> end
>> >> end

>> Not exactly. I need to stop parsing when a special command is seen.
>> "--host" and "--port" in the above example might appear in a

different

>> order or not appear at all, so they're not reliable.
>
> Do you mean the problem is that you need to stop when a command is

seen

> *and* this command can occur anywhere, i.e. also behind an option with

an

> optional argument?

Yes and no. Yes: this command can occur anywhere after one or more
options. No: I have no options with optional arguments so far, so I
haven't dealt with this case which I suspect to be more complex.

Certainly. :slight_smile:

>> > Evaluate the first non option, this is your next command.
>>
>> This needs to go hand in hand with option parsing. Consider this:
>>
>> $ prog --opt1 arg1 --opt2 stuff --blah
>>
>> Is `stuff' a subcommand or opt2's argument? That depends on opt2's
>> definition (which in GetoptLong means NO_ARGUMENT or
>> REQUIRED_ARGUMENT).
>
> You have the same (or finer) level of control for option arguments in
> optparse, too.

The kind of control I am trying to get was not given by GetoptLong per
se, but by the `while ARGV.first =~ /^-/', which breaks down parsing
in chunks where the subcommand can be detected.

>> Then again, if you're thinking that things get
>> muddy when opt2 has an optional argument, I do agree...
>
> Yes, they certainly get muddy at this point. They get as muddy as to

be

> undecidable if you have a command that is also a valid optional

argument

> of an option. You can't decide what it is other than defining a

priority

> for the resolution of this ambiguity.

Yes. Or just disallowing optional arguments for generic options, the
need hasn't arisen so far and I doubt it will at least for this

application.

Yep, that works as well.

>> > Eat it and go to the first step only now remembering that options
>> > are stored for this command. Repeat until there are no more args
>> > left or the first non option is not a command (i.e. file name,
>> > ignorable or whatever).
>>
>> How is this different from the ARGV loop above?
>
> You have two nested loops, one of them is hidden in optparse.

Sorry but I still can't make a picture of what you are suggesting.
Would you mind sketching a couple of lines of pseudocode?

Hm... I'll attache a modified version of the optparse example which
yields this output

10:55:10 [ruby]: ruby optparse-example-2.rb --verbose cmd1 -i foo
<OpenStruct encoding="utf8" transfer_type=:auto verbose=true library=
inplace=false>
<OpenStruct extension=".foo" encoding="utf8" transfer_type=:auto
verbose=false library= inplace=true>
10:55:38 [ruby]:

Of course, additional handling has to be done for file name arguments but
I hope you get the picture.

Kind regards

    robert

optparse-example-2.rb (6.89 KB)

14

Hi,

At Fri, 12 Nov 2004 20:28:27 +0900,
Robert Klemme wrote in [ruby-talk:120033]:

> Then loop subcommand parser until ARGV becomes empty using
> #order! instead of #parse!.

I got lost somewhere in the diff below. :slight_smile: What eaxtly is the difference
between "order!" and "parse!"?

OptionParser has two kinds of behaviors; "in order" and
"permutation" mode. The former (#order! method) stops as soon
as a non-option argument is encountered and is same as
getopt_long(3) when the environment variable POSIXLY_CORRECT is
set, and the latter (#permute! method) processes all option
arguments at first and leaves non-option arguments. #parse!
method switches the two behavior by the environment variable.

···

--
Nobu Nakada

15

<nobu.nokada@softhome.net> schrieb im Newsbeitrag news:200411121250.iACCohlO024122@sharui.nakada.niregi.kanuma.tochigi.jp...

Hi,

At Fri, 12 Nov 2004 20:28:27 +0900,
Robert Klemme wrote in [ruby-talk:120033]:

> Then loop subcommand parser until ARGV becomes empty using
> #order! instead of #parse!.

I got lost somewhere in the diff below. :slight_smile: What eaxtly is the difference
between "order!" and "parse!"?

OptionParser has two kinds of behaviors; "in order" and
"permutation" mode. The former (#order! method) stops as soon
as a non-option argument is encountered and is same as
getopt_long(3) when the environment variable POSIXLY_CORRECT is
set, and the latter (#permute! method) processes all option
arguments at first and leaves non-option arguments. #parse!
method switches the two behavior by the environment variable.

Thanks, Nobu! Now I see much clearer.

    robert