# \[ANN\] Flopp - FLexible OPtion Parser
**URL:** https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585
**Category:** ruby-talk
**Created:** [22 April 2016 13:15 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585 "2016-04-22T13:15:57Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [22 April 2016 13:15 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/1 "2016-04-22T13:15:57Z")
Hi,
Just made a new fun gem for implementing command line parsers.
> **[ioquatix/samovar](https://github.com/ioquatix/samovar)**
>
> Contribute to samovar development by creating an account on GitHub.
It's designed to make it easy to generate decent command-lines while
supporting a wide range of parsing requirements (e.g. ARGV splits "--").
Please do let me know if you use it in a project so I can link to it. Yes,
the documentation is a bit crappy right now, I will work on that, feel free
to submit a PR too 🙂
Kind regards,
Samuel
---
### Author:  [@Kirk\_Haines2](https://rubytalk.org/u/Kirk_Haines2)
#### Post date: [22 April 2016 13:39 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/2 "2016-04-22T13:39:57Z")
Can you summarize what makes it different from some of the other existing
tools for this sort of thing -- OptionParse, Highline, Methadone, GetOpt,
Belafonte, Main, GLI, etc....?
Thanks!
Kirk Haines
> **···**
>
> On Fri, Apr 22, 2016 at 7:15 AM, Samuel Williams \< space.ship.traveller@gmail.com\> wrote:
>
> > Hi,
> >
> > Just made a new fun gem for implementing command line parsers.
> >
> > [GitHub - ioquatix/samovar](https://github.com/ioquatix/flopp)
> >
> > It's designed to make it easy to generate decent command-lines while
> > supporting a wide range of parsing requirements (e.g. ARGV splits "--").
> >
> > Please do let me know if you use it in a project so I can link to it. Yes,
> > the documentation is a bit crappy right now, I will work on that, feel free
> > to submit a PR too 🙂
---
### Author:  [@Bar\_Hofesh](https://rubytalk.org/u/Bar_Hofesh)
#### Post date: [22 April 2016 13:17 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/3 "2016-04-22T13:17:56Z")
Looks nice 🙂
I'll see if I can use it instead of optionparser in my proj
[http://www.aqua-mail.com](http://www.aqua-mail.com)
> **···**
>
> Sent with AquaMail for Android
>
> On April 22, 2016 4:16:19 PM Samuel Williams \<[space.ship.traveller@gmail.com](mailto:space.ship.traveller@gmail.com)\> wrote:
>
> Hi,
>
> Just made a new fun gem for implementing command line parsers.
>
> > **[ioquatix/samovar](https://github.com/ioquatix/samovar)**
> >
> > Contribute to samovar development by creating an account on GitHub.
>
> It's designed to make it easy to generate decent command-lines while supporting a wide range of parsing requirements (e.g. ARGV splits "--").
>
> Please do let me know if you use it in a project so I can link to it. Yes, the documentation is a bit crappy right now, I will work on that, feel free to submit a PR too 🙂
>
> Kind regards,
> Samuel
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [22 April 2016 13:20 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/4 "2016-04-22T13:20:38Z")
Bar, that would be awesome. It's still early days but I'd appreciate any
feedback. It should be super easy to use.
> **···**
>
> On 23 April 2016 at 01:17, Bar Hofesh \ wrote:
>
> > Looks nice 🙂
> > I'll see if I can use it instead of optionparser in my proj
> >
> > Sent with AquaMail for Android
> > [http://www.aqua-mail.com](http://www.aqua-mail.com)
> >
> > On April 22, 2016 4:16:19 PM Samuel Williams \< \> space.ship.traveller@gmail.com\> wrote:
> >
> > > Hi,
> > >
> > > Just made a new fun gem for implementing command line parsers.
> > >
> > > [GitHub - ioquatix/samovar](https://github.com/ioquatix/flopp)
> > >
> > > It's designed to make it easy to generate decent command-lines while
> > > supporting a wide range of parsing requirements (e.g. ARGV splits "--").
> > >
> > > Please do let me know if you use it in a project so I can link to it.
> > > Yes, the documentation is a bit crappy right now, I will work on that, feel
> > > free to submit a PR too 🙂
> > >
> > > Kind regards,
> > > Samuel
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [22 April 2016 14:06 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/5 "2016-04-22T14:06:03Z")
I tried a bunch of those and found them lacking in one way or another from
my requirements. I needed to support nested commands, ARGV splits, parsing
arbitrary tokens (in the future supporting autocomplete) and automatic
documentation for both options and nested commands. In the end I got sick
of fighting with Trollop so decided to roll my own. I will be using flopp
for all my command line tools now.
Parts of flopp are modelled after how git presents its options which I
found very logical.
> **···**
>
> On Saturday, 23 April 2016, Kirk Haines \ wrote:
>
> > On Fri, Apr 22, 2016 at 7:15 AM, Samuel Williams \< \> space.ship.traveller@gmail.com \> \\> wrote:
> >
> > > Hi,
> > >
> > > Just made a new fun gem for implementing command line parsers.
> > >
> > > [GitHub - ioquatix/samovar](https://github.com/ioquatix/flopp)
> > >
> > > It's designed to make it easy to generate decent command-lines while
> > > supporting a wide range of parsing requirements (e.g. ARGV splits "--").
> > >
> > > Please do let me know if you use it in a project so I can link to it.
> > > Yes, the documentation is a bit crappy right now, I will work on that, feel
> > > free to submit a PR too 🙂
> >
> > Can you summarize what makes it different from some of the other existing
> > tools for this sort of thing -- OptionParse, Highline, Methadone, GetOpt,
> > Belafonte, Main, GLI, etc....?
> >
> > Thanks!
> >
> > Kirk Haines
---
### Author:  [@Robert\_K1](https://rubytalk.org/u/Robert_K1)
#### Post date: [22 April 2016 16:08 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/6 "2016-04-22T16:08:56Z")
> I tried a bunch of those and found them lacking in one way or another from
> my requirements. I needed to support nested commands,
What exactly are "nested commands"? You mean like "apt get" and "apt
update" or git with sub commands? That could easily be solved with
any CL parser by fetching the command first and then deciding which
parser to use - or more generally what module / class to use to handle
the command. I would not want to have all option handling in one
place anyway when there are several sub commands - this soon gets
unmanageable.
> ARGV splits,
What is that?
> parsing arbitrary tokens (in the future supporting autocomplete)
OptionParser supports autocompletion:
$ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
"foo"}}.parse!; p ARGV' -- --foo abc
"foo"
["abc"]
$ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
"foo"}}.parse!; p ARGV' -- --fo abc
"foo"
["abc"]
$ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
"foo"}}.parse!; p ARGV' -- --f abc
"foo"
["abc"]
> and automatic
> documentation for both options and nested commands.
Automated documentation is done by OptionParser already.
> In the end I got sick of
> fighting with Trollop so decided to roll my own. I will be using flopp for
> all my command line tools now.
I find it preferable to use something in the standard library. But of
course everybody's mileage differs.
> Parts of flopp are modelled after how git presents its options which I found
> very logical.
>
> > Can you summarize what makes it different from some of the other existing
> > tools for this sort of thing -- OptionParse, Highline, Methadone, GetOpt,
> > Belafonte, Main, GLI, etc....?
Especially what makes it better than those solutions that do exist
already. I am just asking for a bit of advertising. 😉
Kind regards
robert
> **···**
>
> On Fri, Apr 22, 2016 at 4:06 PM, Samuel Williams \ wrote:
>
> > On Saturday, 23 April 2016, Kirk Haines \ wrote:
>
> --
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can
> - without end}
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [23 April 2016 02:03 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/7 "2016-04-23T02:03:18Z")
> What exactly are "nested commands"? You mean like "apt get" and "apt
> update" or git with sub commands? That could easily be solved with
> any CL parser by fetching the command first and then deciding which
> parser to use - or more generally what module / class to use to handle
> the command. I would not want to have all option handling in one
> place anyway when there are several sub commands - this soon gets
> unmanageable.
Yes, so we are both in agreement here. Having all the code in one place
quickly gets unmanageable. Flopp addresses this by making every command
(including the top level one) a separate Flopp::Command which encapsulates
the specific logic for parsing, exactly as you suggest. In this sense, you
can nest commands as many levels deep as you want, alias them in different
places with different names, etc.
However, because the nesting is aware of each other and not just a side
effect of the top level option parser, documentation is generated much more
seamlessly, and in general the parser produces a useful nested data
structure which lets you implement logic where it belongs w.r.t. a specific
command. The only example I have of this and where Flopp was originally
developed is in teapot.
> \> ARGV splits,
>
> What is that?
Just a name I came up for when you write "--" and expect the command to
only process stuff on the left while passing stuff on the right to a
script/sub-process, if any. In Flopp, you just write "split :rest" and you
get documentation, parsing, and the attribute exposed in the command
instance 🙂
> \> parsing arbitrary tokens (in the future supporting autocomplete)
>
> OptionParser supports autocompletion:
>
> $ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
> "foo"}}.parse!; p ARGV' -- --foo abc
> "foo"
> ["abc"]
> $ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
> "foo"}}.parse!; p ARGV' -- --fo abc
> "foo"
> ["abc"]
> $ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
> "foo"}}.parse!; p ARGV' -- --f abc
> "foo"
> ["abc"]
I meant auto-completion in the sense you can tab-complete at a given point
in the parser and it returns all valid tokens that could come next. In
order to do this you need to know the entire parser - and I don't think
this is really possible with OptionParser with nested commands, for
example. The behaviour shown above is not really what I was suggesting,
and, IMHO, is very dangerous. It might lead to people using a command, a
new option being introduced, and then that original command does something
different.. unless that command now just breaks because the option is
ambiguous, also a bad outcome.
Flopp is very explicit, but it's also easy to specify short and long
options, etc, such as:
option '-i/--in/--root \', "Set the root path for doing stuff"
This will set @options[:root] to whatever you specify with one of those
three options. There is no automatic short/long options, etc. The last
option is used as the key unless it's specified explicitly.
> \> and automatic
> \> documentation for both options and nested commands.
>
> Automated documentation is done by OptionParser already.
Not in a way that supports sub-commands easily AFAIK. It's also pretty
plain output. In Flopp, only one mode of output is supported at the moment,
but I want to have two modes of help output, --help and -h, long help and
short help respectively. Ideally, long help is more similar to a man page.
In addition, if you specify a command line, along with --help, it will
parse as much as possible into the help, with potential explanations, e.g.
if you choose --verbose --help, the --verbose option in the help output
could be documented. Ideally, the same with things like `one` and `many`
tokens, so that if you, say, wrote teapot --help build Run/MyProject it
could even give you application specific output relating to "Run/MyProject"
which has some description/documentation associated with it. This is a big
usability thing to me so I'll be working in this area next.
> \> In the end I got sick of
> \> fighting with Trollop so decided to roll my own. I will be using flopp
> for
> \> all my command line tools now.
>
> I find it preferable to use something in the standard library. But of
> course everybody's mileage differs.
Fair enough. My opinion is that the standard library should be simplified
and kept as small as possible. Otherwise, it's hard to version, change,
improve, etc. While OptionParser might be awesome (and it sounds like it is
for simple stuff), ideally it would be a separate gem.
> \> Parts of flopp are modelled after how git presents its options which I
> found
> \> very logical.
> \>
> \>
>
> \>\> Can you summarize what makes it different from some of the other
> existing
> \>\> tools for this sort of thing -- OptionParse, Highline, Methadone,
> GetOpt,
> \>\> Belafonte, Main, GLI, etc....?
>
> Especially what makes it better than those solutions that do exist
> already. I am just asking for a bit of advertising. 😉
I'm not sure I can to fair justice to a comparison of N different
approaches to option parsing. I'm sure they all have their good parts.
Flopp is something that will allow me to make awesome command line
interfaces for the various gems I made, with awesome --help.
- Awesome documentation for nested commands including colours. See the
picture on the github project page for an example.
- Convention over configuration for options to keep your high level
definition as simple as possible.
- Future features planned: command-line auto-completion, detailed help
(perhaps the ability to generate man/markdown pages too).
Some things that I haven't bothered with:
- Options can only have one value and it's not type checked for now (e.g.
this must be an int). Easily fixable but not really useful for me right
now. If you want to do validation, it could be done elsewhere for now.
- The greedy parser is structured in such a way that it doesn't allow you
to have "global" options. For example, what would
command --foo subcommand --foo -- --foo
mean? In Flopp, those are all different options. Other option parsers might
try to resolve this into one option, but for performance and specificity, I
didn't go down this route. That being said, this isn't a bad design, it's
got some benefits too. It might make sense for some options to have global
scope.. but then how do you deal with
command subcommand -- --help
Anyway, it's a work in progress, and I appreciate your feedback and
challenge to explain why it's a good idea.
> **···**
>
> > \> On Saturday, 23 April 2016, Kirk Haines \ wrote:
---
### Author:  [@Robert\_K1](https://rubytalk.org/u/Robert_K1)
#### Post date: [23 April 2016 10:55 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/8 "2016-04-23T10:55:05Z")
> > What exactly are "nested commands"? You mean like "apt get" and "apt
> > update" or git with sub commands? That could easily be solved with
> > any CL parser by fetching the command first and then deciding which
> > parser to use - or more generally what module / class to use to handle
> > the command. I would not want to have all option handling in one
> > place anyway when there are several sub commands - this soon gets
> > unmanageable.
>
> Yes, so we are both in agreement here. Having all the code in one place
> quickly gets unmanageable. Flopp addresses this by making every command
> (including the top level one) a separate Flopp::Command which encapsulates
> the specific logic for parsing, exactly as you suggest. In this sense, you
> can nest commands as many levels deep as you want, alias them in different
> places with different names, etc.
I can see how having support for the concept of sub commands in a
command line parser has its advantages. However, I do not think that
more than one level of nesting really makes sense. Wouldn't that soon
get confusing for the user?
> > \> ARGV splits,
> >
> > What is that?
>
> Just a name I came up for when you write "--" and expect the command to only
> process stuff on the left while passing stuff on the right to a
> script/sub-process, if any. In Flopp, you just write "split :rest" and you
> get documentation, parsing, and the attribute exposed in the command
> instance 🙂
I think that (or something similar?) is supported by OptionParser as well:
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
p ARGV' -x a b c
["a", "b", "c"]
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
p ARGV' -- -x a b c
"x"
["a", "b", "c"]
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
p ARGV' -- a -x b c
"x"
["a", "b", "c"]
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
p ARGV' -- a -- -x b c
["a", "-x", "b", "c"]
Note: the first "--" is for ruby itself.
> > \> parsing arbitrary tokens (in the future supporting autocomplete)
> I meant auto-completion in the sense you can tab-complete at a given point
> in the parser and it returns all valid tokens that could come next.
Are you saying that you want people to interactively complete options
if they are ambiguous?
> In order
> to do this you need to know the entire parser - and I don't think this is
> really possible with OptionParser with nested commands, for example. The
> behaviour shown above is not really what I was suggesting, and, IMHO, is
> very dangerous. It might lead to people using a command, a new option being
> introduced, and then that original command does something different.. unless
> that command now just breaks because the option is ambiguous, also a bad
> outcome.
I think that is perfectly OK because then you notice that your
invocation now has become ambiguous and can change the calling code
without silently introducing errors.
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--foo"){p
"f"}}.parse!; p ARGV' -- --foo
"f"
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--foo"){p
"f"}}.parse!; p ARGV' -- --fo
"f"
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--foo"){p
"f"}}.parse!; p ARGV' -- --f
"f"
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--foo"){p
"f"}.on("--for"){p "o"}}.parse!; p ARGV' -- --f
-e:1:in `\': ambiguous option: --f (OptionParser::AmbiguousOption)
$ echo $?
1
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--foo"){p
"f"}.on("--for"){p "o"}}.parse!; p ARGV' -- --fo
-e:1:in `\': ambiguous option: --fo (OptionParser::AmbiguousOption)
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--foo"){p
"f"}.on("--for"){p "o"}}.parse!; p ARGV' -- --foo
"f"
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--foo"){p
"f"}.on("--for"){p "o"}}.parse!; p ARGV' -- --for
"o"
> Flopp is very explicit, but it's also easy to specify short and long
> options, etc, such as:
>
> option '-i/--in/--root \', "Set the root path for doing stuff"
>
> This will set @options[:root] to whatever you specify with one of those
> three options. There is no automatic short/long options, etc. The last
> option is used as the key unless it's specified explicitly.
I like the implicit configuration Hash. OTOH I also like
OptionParser's solution where you have to decide yourself how you will
handle options:
options = {}
OptionParser.new do |opt|
opt.on '-i', '--in', '--root PATH', 'Set root path for doing stuff' do |value|
options[:root] = value
end
end.parse!
> > \> and automatic
> > \> documentation for both options and nested commands.
> >
> > Automated documentation is done by OptionParser already.
>
> Not in a way that supports sub-commands easily AFAIK. It's also pretty plain
> output. In Flopp, only one mode of output is supported at the moment, but I
> want to have two modes of help output, --help and -h, long help and short
> help respectively. Ideally, long help is more similar to a man page. In
> addition, if you specify a command line, along with --help, it will parse as
> much as possible into the help, with potential explanations, e.g. if you
> choose --verbose --help, the --verbose option in the help output could be
> documented. Ideally, the same with things like `one` and `many` tokens, so
> that if you, say, wrote teapot --help build Run/MyProject it could even give
> you application specific output relating to "Run/MyProject" which has some
> description/documentation associated with it. This is a big usability thing
> to me so I'll be working in this area next.
That's quite a lot whistles and bells for an option parser. 🙂 I am
not sure whether generating documentation should be an option parser's
task. I just does not feel right because generating documentation is a
built time task while option parsing is a run time task.
> > \> In the end I got sick of
> > \> fighting with Trollop so decided to roll my own. I will be using flopp
> > \> for
> > \> all my command line tools now.
> >
> > I find it preferable to use something in the standard library. But of
> > course everybody's mileage differs.
>
> Fair enough. My opinion is that the standard library should be simplified
> and kept as small as possible.
I disagree. You can compare C++ and Java to see why. Java's standard
library is very comprehensive and has solutions for a lot basic
functionality. In C++ you need a library for every mildly advanced
features which leads to a lot duplication and the trouble you get into
if you want to use one library for a functionality and another
component uses another library. Now you get bloat. Or you require
different version and the other lib requires another version. Or your
code and the library has incompatible data types so you end up doing a
lot conversions. ...
> Otherwise, it's hard to version, change,
> improve, etc. While OptionParser might be awesome (and it sounds like it is
> for simple stuff), ideally it would be a separate gem.
OptionParser can do quite a lot which is often overlooked because of
the minimalist documentation.
> > Especially what makes it better than those solutions that do exist
> > already. I am just asking for a bit of advertising. 😉
>
> I'm not sure I can to fair justice to a comparison of N different approaches
> to option parsing. I'm sure they all have their good parts.
I think comparing to what is in the standard library is due diligence. 😉
> Flopp is something that will allow me to make awesome command line
> interfaces for the various gems I made, with awesome --help.
>
> - Awesome documentation for nested commands including colours. See the
> picture on the github project page for an example.
> - Convention over configuration for options to keep your high level
> definition as simple as possible.
> - Future features planned: command-line auto-completion, detailed help
> (perhaps the ability to generate man/markdown pages too).
Couldn't read that section: my "awesome" detector blanked it out. 😉
> Some things that I haven't bothered with:
>
> - Options can only have one value and it's not type checked for now (e.g.
> this must be an int). Easily fixable but not really useful for me right now.
> If you want to do validation, it could be done elsewhere for now.
OptionParser has this built in. This is very convenient.
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--r=REGEX",
Regexp){|v|p v}}.parse!' -- -r 'a\*'
/a\*/
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("--r=REGEX",
Regexp){|v|p v}}.parse!' -- -r '\*a\*'
/usr/lib/ruby/2.1.0/optparse.rb:1741:in `initialize': target of repeat
operator is not specified: /\*a\*/ (RegexpError)
from /usr/lib/ruby/2.1.0/optparse.rb:1741:in `new'
from /usr/lib/ruby/2.1.0/optparse.rb:1741:in `block in \'
from /usr/lib/ruby/2.1.0/optparse.rb:366:in `call'
from /usr/lib/ruby/2.1.0/optparse.rb:366:in `conv\_arg'
from /usr/lib/ruby/2.1.0/optparse.rb:494:in `parse'
from /usr/lib/ruby/2.1.0/optparse.rb:1387:in `block in parse\_in\_order'
from /usr/lib/ruby/2.1.0/optparse.rb:1346:in `catch'
from /usr/lib/ruby/2.1.0/optparse.rb:1346:in `parse\_in\_order'
from /usr/lib/ruby/2.1.0/optparse.rb:1340:in `order!'
from /usr/lib/ruby/2.1.0/optparse.rb:1432:in `permute!'
from /usr/lib/ruby/2.1.0/optparse.rb:1454:in `parse!'
from -e:1:in `\'
> - The greedy parser is structured in such a way that it doesn't allow you to
> have "global" options. For example, what would
> command --foo subcommand --foo -- --foo
> mean? In Flopp, those are all different options. Other option parsers might
> try to resolve this into one option, but for performance and specificity, I
> didn't go down this route. That being said, this isn't a bad design, it's
> got some benefits too. It might make sense for some options to have global
> scope.. but then how do you deal with
> command subcommand -- --help
>
> Anyway, it's a work in progress, and I appreciate your feedback and
> challenge to explain why it's a good idea.
You are welcome!
Kind regards
robert
> **···**
>
> On Sat, Apr 23, 2016 at 4:03 AM, Samuel Williams \ wrote:
>
> --
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can
> - without end}
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [23 April 2016 02:20 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/9 "2016-04-23T02:20:15Z")
Okay sorry to do this but the name Flopp didn't sit with me well and it's
now renamed to Samovar. Sorry if this causes any inconvenience. The name
won't change from now on, I promise 😃
> **···**
>
> On 23 April 2016 at 14:03, Samuel Williams \ wrote:
>
> > What exactly are "nested commands"? You mean like "apt get" and "apt
> >
> > > update" or git with sub commands? That could easily be solved with
> > > any CL parser by fetching the command first and then deciding which
> > > parser to use - or more generally what module / class to use to handle
> > > the command. I would not want to have all option handling in one
> > > place anyway when there are several sub commands - this soon gets
> > > unmanageable.
> >
> > Yes, so we are both in agreement here. Having all the code in one place
> > quickly gets unmanageable. Flopp addresses this by making every command
> > (including the top level one) a separate Flopp::Command which encapsulates
> > the specific logic for parsing, exactly as you suggest. In this sense, you
> > can nest commands as many levels deep as you want, alias them in different
> > places with different names, etc.
> >
> > However, because the nesting is aware of each other and not just a side
> > effect of the top level option parser, documentation is generated much more
> > seamlessly, and in general the parser produces a useful nested data
> > structure which lets you implement logic where it belongs w.r.t. a specific
> > command. The only example I have of this and where Flopp was originally
> > developed is in teapot.
> >
> > > \> ARGV splits,
> > >
> > > What is that?
> >
> > Just a name I came up for when you write "--" and expect the command to
> > only process stuff on the left while passing stuff on the right to a
> > script/sub-process, if any. In Flopp, you just write "split :rest" and you
> > get documentation, parsing, and the attribute exposed in the command
> > instance 🙂
> >
> > > \> parsing arbitrary tokens (in the future supporting autocomplete)
> > >
> > > OptionParser supports autocompletion:
> > >
> > > $ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
> > > "foo"}}.parse!; p ARGV' -- --foo abc
> > > "foo"
> > > ["abc"]
> > > $ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
> > > "foo"}}.parse!; p ARGV' -- --fo abc
> > > "foo"
> > > ["abc"]
> > > $ ruby -r optparse -e 'OptionParser.new {|o| o.on("--foo"){|v|p
> > > "foo"}}.parse!; p ARGV' -- --f abc
> > > "foo"
> > > ["abc"]
> >
> > I meant auto-completion in the sense you can tab-complete at a given point
> > in the parser and it returns all valid tokens that could come next. In
> > order to do this you need to know the entire parser - and I don't think
> > this is really possible with OptionParser with nested commands, for
> > example. The behaviour shown above is not really what I was suggesting,
> > and, IMHO, is very dangerous. It might lead to people using a command, a
> > new option being introduced, and then that original command does something
> > different.. unless that command now just breaks because the option is
> > ambiguous, also a bad outcome.
> >
> > Flopp is very explicit, but it's also easy to specify short and long
> > options, etc, such as:
> >
> > option '-i/--in/--root \', "Set the root path for doing stuff"
> >
> > This will set @options[:root] to whatever you specify with one of those
> > three options. There is no automatic short/long options, etc. The last
> > option is used as the key unless it's specified explicitly.
> >
> > > \> and automatic
> > > \> documentation for both options and nested commands.
> > >
> > > Automated documentation is done by OptionParser already.
> >
> > Not in a way that supports sub-commands easily AFAIK. It's also pretty
> > plain output. In Flopp, only one mode of output is supported at the moment,
> > but I want to have two modes of help output, --help and -h, long help and
> > short help respectively. Ideally, long help is more similar to a man page.
> > In addition, if you specify a command line, along with --help, it will
> > parse as much as possible into the help, with potential explanations, e.g.
> > if you choose --verbose --help, the --verbose option in the help output
> > could be documented. Ideally, the same with things like `one` and `many`
> > tokens, so that if you, say, wrote teapot --help build Run/MyProject it
> > could even give you application specific output relating to "Run/MyProject"
> > which has some description/documentation associated with it. This is a big
> > usability thing to me so I'll be working in this area next.
> >
> > > \> In the end I got sick of
> > > \> fighting with Trollop so decided to roll my own. I will be using flopp
> > > for
> > > \> all my command line tools now.
> > >
> > > I find it preferable to use something in the standard library. But of
> > > course everybody's mileage differs.
> >
> > Fair enough. My opinion is that the standard library should be simplified
> > and kept as small as possible. Otherwise, it's hard to version, change,
> > improve, etc. While OptionParser might be awesome (and it sounds like it is
> > for simple stuff), ideally it would be a separate gem.
> >
> > > \> Parts of flopp are modelled after how git presents its options which I
> > > found
> > > \> very logical.
> > > \>
> > > \>
> > > \> On Saturday, 23 April 2016, Kirk Haines \ wrote:
> > >
> > > \>\> Can you summarize what makes it different from some of the other
> > > existing
> > > \>\> tools for this sort of thing -- OptionParse, Highline, Methadone,
> > > GetOpt,
> > > \>\> Belafonte, Main, GLI, etc....?
> > >
> > > Especially what makes it better than those solutions that do exist
> > > already. I am just asking for a bit of advertising. 😉
> >
> > I'm not sure I can to fair justice to a comparison of N different
> > approaches to option parsing. I'm sure they all have their good parts.
> >
> > Flopp is something that will allow me to make awesome command line
> > interfaces for the various gems I made, with awesome --help.
> >
> > - Awesome documentation for nested commands including colours. See the
> > picture on the github project page for an example.
> > - Convention over configuration for options to keep your high level
> > definition as simple as possible.
> > - Future features planned: command-line auto-completion, detailed help
> > (perhaps the ability to generate man/markdown pages too).
> >
> > Some things that I haven't bothered with:
> >
> > - Options can only have one value and it's not type checked for now (e.g.
> > this must be an int). Easily fixable but not really useful for me right
> > now. If you want to do validation, it could be done elsewhere for now.
> > - The greedy parser is structured in such a way that it doesn't allow you
> > to have "global" options. For example, what would
> > command --foo subcommand --foo -- --foo
> > mean? In Flopp, those are all different options. Other option parsers
> > might try to resolve this into one option, but for performance and
> > specificity, I didn't go down this route. That being said, this isn't a bad
> > design, it's got some benefits too. It might make sense for some options to
> > have global scope.. but then how do you deal with
> > command subcommand -- --help
> >
> > Anyway, it's a work in progress, and I appreciate your feedback and
> > challenge to explain why it's a good idea.
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [24 April 2016 02:24 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/10 "2016-04-24T02:24:03Z")
> I can see how having support for the concept of sub commands in a
> command line parser has its advantages. However, I do not think that
> more than one level of nesting really makes sense. Wouldn't that soon
> get confusing for the user?
Yes, having sub-commands in the parser is awesome.
How many levels deep you want depends on the command and how it needs to be structured. In any case, it's easy to support with a nested command parser.
> I think that (or something similar?) is supported by OptionParser as well:
>
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -x a b c
> ["a", "b", "c"]
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -- -x a b c
> "x"
> ["a", "b", "c"]
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -- a -x b c
> "x"
> ["a", "b", "c"]
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -- a -- -x b c
> ["a", "-x", "b", "c"]
>
> Note: the first "--" is for ruby itself.
I would argue that "supported" is tenuous at best here. The actual code here is:
DefaultList.long[''] = Switch::NoArgument.new {throw :terminate}
So basically the parser simply terminates after encoutering "--". There is no way to continue processing this. As an example,
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
p ARGV' -- a -- -x b c
["a", "-x", "b", "c"]
$ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
p ARGV' -- -- a -x b c
["a", "-x", "b", "c"]
So, how do you know what came before the -- and what came after? In Flopp/Samovar, this is explicit and part of the parser.
> Are you saying that you want people to interactively complete options
> if they are ambiguous?
Nope, options which are not fully specified are bad IMHO. I'm suggesting that it would be great if Samovar provided ZSH/Bash auto completion, which at some point it will because this would be incredibly useful in teapot.
> > In order
> > to do this you need to know the entire parser - and I don't think this is
> > really possible with OptionParser with nested commands, for example. The
> > behaviour shown above is not really what I was suggesting, and, IMHO, is
> > very dangerous. It might lead to people using a command, a new option being
> > introduced, and then that original command does something different.. unless
> > that command now just breaks because the option is ambiguous, also a bad
> > outcome.
>
> I think that is perfectly OK because then you notice that your
> invocation now has become ambiguous and can change the calling code
> without silently introducing errors.
Okay, well, I prefer code that doesn't break 🙂
> > Flopp is very explicit, but it's also easy to specify short and long
> > options, etc, such as:
> >
> > option '-i/--in/--root \', "Set the root path for doing stuff"
> >
> > This will set @options[:root] to whatever you specify with one of those
> > three options. There is no automatic short/long options, etc. The last
> > option is used as the key unless it's specified explicitly.
>
> I like the implicit configuration Hash. OTOH I also like
> OptionParser's solution where you have to decide yourself how you will
> handle options:
>
> options = {}
>
> OptionParser.new do |opt|
> opt.on '-i', '--in', '--root PATH', 'Set root path for doing stuff' do |value|
> options[:root] = value
> end
> end.parse!
They are both valid approaches. However, I prefer convention over configuration.
> > > > and automatic
> > > > documentation for both options and nested commands.
> > >
> > > Automated documentation is done by OptionParser already.
> >
> > Not in a way that supports sub-commands easily AFAIK. It's also pretty plain
> > output. In Flopp, only one mode of output is supported at the moment, but I
> > want to have two modes of help output, --help and -h, long help and short
> > help respectively. Ideally, long help is more similar to a man page. In
> > addition, if you specify a command line, along with --help, it will parse as
> > much as possible into the help, with potential explanations, e.g. if you
> > choose --verbose --help, the --verbose option in the help output could be
> > documented. Ideally, the same with things like `one` and `many` tokens, so
> > that if you, say, wrote teapot --help build Run/MyProject it could even give
> > you application specific output relating to "Run/MyProject" which has some
> > description/documentation associated with it. This is a big usability thing
> > to me so I'll be working in this area next.
>
> That's quite a lot whistles and bells for an option parser. 🙂 I am
> not sure whether generating documentation should be an option parser's
> task. I just does not feel right because generating documentation is a
> built time task while option parsing is a run time task.
Well, OptionParser generates documentation to some extent. In fact, most option parsers do. So, I'd say that in general people are using that functionality. In the case of generating help output, I personally believe that the help output needs to support the user as much as possible. I'm expecting to build on the nature of `git --help` which I personally really like.
I'd actually like to see interactive help. So, if you write
command --foo foo.txt --help
You might get some output like:
Usage: command [--foo \] \
--foo \ Compute the foo factor. Value: foo.txt
\ The tags to index from the foo file.
Possible options are: bob, dole, cat, cheese. (taken from foo.txt)
The same applies to commands like git checkout --help - it could interactively list available branches or tags, etc.
> OptionParser has this built in. This is very convenient.
>
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("--r=REGEX",
> Regexp){|v|p v}}.parse!' -- -r 'a\*'
> /a\*/
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("--r=REGEX",
> Regexp){|v|p v}}.parse!' -- -r '\*a\*'
> /usr/lib/ruby/2.1.0/optparse.rb:1741:in `initialize': target of repeat
> operator is not specified: /\*a\*/ (RegexpError)
> from /usr/lib/ruby/2.1.0/optparse.rb:1741:in `new'
> from /usr/lib/ruby/2.1.0/optparse.rb:1741:in `block in \'
> from /usr/lib/ruby/2.1.0/optparse.rb:366:in `call'
> from /usr/lib/ruby/2.1.0/optparse.rb:366:in `conv\_arg'
> from /usr/lib/ruby/2.1.0/optparse.rb:494:in `parse'
> from /usr/lib/ruby/2.1.0/optparse.rb:1387:in `block in parse\_in\_order'
> from /usr/lib/ruby/2.1.0/optparse.rb:1346:in `catch'
> from /usr/lib/ruby/2.1.0/optparse.rb:1346:in `parse\_in\_order'
> from /usr/lib/ruby/2.1.0/optparse.rb:1340:in `order!'
> from /usr/lib/ruby/2.1.0/optparse.rb:1432:in `permute!'
> from /usr/lib/ruby/2.1.0/optparse.rb:1454:in `parse!'
> from -e:1:in `\'
I agree and in the future if the need arises I will implement this in Samovar 🙂 For now though it's trivial to do the conversion and validation in the glue between the parser and your code. I agree though some semantic checking might be nice, e.g. \ would validate the path already exists. But, just blowing up with an error isn't that nice. I'd prefer some kind of help otuput which explains what value was invalid and why.
Kind regards,
Samuel
---
### Author:  [@David\_Gamba](https://rubytalk.org/u/David_Gamba)
#### Post date: [22 September 2016 05:28 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/11 "2016-09-22T05:28:38Z")
> - Awesome documentation for nested commands including colours. See the
picture on the github project page for an example.
> - Future features planned: detailed help (perhaps the ability to generate
man/markdown pages too).
I find that most option parsers out there are 90% code to generate a help
message!
When you are talking about a multi layered script, it is way faster and
simpler for a programmer to write a proper synopsis himself and then write
a proper man page for their program. Man pages have all the colors you want
already.
I seriously find it super annoying when I find a lazy programmer that used
one of these option parsers that generate multi page option descriptions
and it doesn't even pipe it to a pager for you.
Most of them don't really make sense anyway, their help message ends up
being: script-name [OPTIONS...]
And then a blast of options that you can't tell from the automated help if
they can be mixed together or not.
I even contributed a PR to the manpage backend for asciidoctor. So really,
man pages couldn't be easier to write.
I have taken a stab at the option parsing problem myself here:
> **[GitHub - DavidGamba/ruby-getoptions: Ruby option parser based on Perl’s...](https://github.com/DavidGamba/ruby-getoptions)**
>
> Ruby option parser based on Perl’s Getopt::Long. Contribute to DavidGamba/ruby-getoptions development by creating an account on GitHub.
If anything, you might find the introduction useful because it includes
what I believe is required from an option parser.
The API is very similar to that of Perl's GetOpt::Long.
I also took a stab at it in go: [GitHub - DavidGamba/go-getoptions: Fully featured Go (golang) command line option parser with built-in auto-completion support.](https://github.com/DavidGamba/go-getoptions)
And the difference there is that being a static type system, my API had to
be completely different to keep that type safety. I expanded the README
there a bit more.
Hope that gives you some good ideas and that it makes you consider not
mixing those two different concerns: Option parsing and documentation.
The Perl team had it right and that is what they initially used for those
git man pages you talked about. They have a different package to extract
the synopsis out of a script's man page, which is written in POD format,
called pod2usage. The programmer always wrote the synopsis and full man
page then.
--David
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [25 September 2016 13:33 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/12 "2016-09-25T13:33:13Z")
Robert, sorry, I never replied to you in detail.. here are a few comments.
> \>\> What exactly are "nested commands"? You mean like "apt get" and "apt
> \>\> update" or git with sub commands? That could easily be solved with
> \>\> any CL parser by fetching the command first and then deciding which
> \>\> parser to use - or more generally what module / class to use to handle
> \>\> the command. I would not want to have all option handling in one
> \>\> place anyway when there are several sub commands - this soon gets
> \>\> unmanageable.
> \>
> \> Yes, so we are both in agreement here. Having all the code in one place
> \> quickly gets unmanageable. Flopp addresses this by making every command
> \> (including the top level one) a separate Flopp::Command which
> encapsulates
> \> the specific logic for parsing, exactly as you suggest. In this sense,
> you
> \> can nest commands as many levels deep as you want, alias them in
> different
> \> places with different names, etc.
>
> I can see how having support for the concept of sub commands in a
> command line parser has its advantages. However, I do not think that
> more than one level of nesting really makes sense. Wouldn't that soon
> get confusing for the user?
It may do, but it's a simple and clean abstraction. How you choose to use
it is up to you. At least in the case of my utopia framework, I have both
server side and client side commands, e.g. 'utopia server create' is a
different command from 'utopia client create'.
> \>\> \> ARGV splits,
> \>\>
> \>\> What is that?
> \>
> \> Just a name I came up for when you write "--" and expect the command to
> only
> \> process stuff on the left while passing stuff on the right to a
> \> script/sub-process, if any. In Flopp, you just write "split :rest" and
> you
> \> get documentation, parsing, and the attribute exposed in the command
> \> instance 🙂
>
> I think that (or something similar?) is supported by OptionParser as well:
>
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -x a b c
> ["a", "b", "c"]
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -- -x a b c
> "x"
> ["a", "b", "c"]
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -- a -x b c
> "x"
> ["a", "b", "c"]
> $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> p ARGV' -- a -- -x b c
> ["a", "-x", "b", "c"]
>
> Note: the first "--" is for ruby itself.
Just printing out what's left over in ARGV isn't really helpful. In
Samovar, it's a first class citizen in the parser. It means that for the
sake of tab completion or other related functions, it's possible to
understand the intention of the programmer, not just "I parsed as much as I
could". The immediate benefit of this is that documentation is better, and
the parser knows what's going on explicitly so in the future autocompletion
would be possible to implement in a sound manor.
> \>\> \> parsing arbitrary tokens (in the future supporting autocomplete)
>
> \> I meant auto-completion in the sense you can tab-complete at a given
> point
> \> in the parser and it returns all valid tokens that could come next.
>
> Are you saying that you want people to interactively complete options
> if they are ambiguous?
Yes. Adding optional strong typing to samovar is the logical next step to
support this functionality. It's easy enough. The hard part is actually the
way different shells work when dealing with autocompletion.
> \> Flopp is very explicit, but it's also easy to specify short and long
> \> options, etc, such as:
> \>
> \> option '-i/--in/--root \', "Set the root path for doing stuff"
> \>
> \> This will set @options[:root] to whatever you specify with one of those
> \> three options. There is no automatic short/long options, etc. The last
> \> option is used as the key unless it's specified explicitly.
>
> I like the implicit configuration Hash. OTOH I also like
> OptionParser's solution where you have to decide yourself how you will
> handle options:
>
> options = {}
>
> OptionParser.new do |opt|
> opt.on '-i', '--in', '--root PATH', 'Set root path for doing stuff' do
> \>value\>
> options[:root] = value
> end
> end.parse!
This gets tedious and even error-prone when you have a complex command line
and lots of options. It's sort of the difference between mutable and
immutable w.r.t. option parsing.
> \>\> \> and automatic
> \>\> \> documentation for both options and nested commands.
> \>\>
> \>\> Automated documentation is done by OptionParser already.
> \>
> \> Not in a way that supports sub-commands easily AFAIK. It's also pretty
> plain
> \> output. In Flopp, only one mode of output is supported at the moment,
> but I
> \> want to have two modes of help output, --help and -h, long help and short
> \> help respectively. Ideally, long help is more similar to a man page. In
> \> addition, if you specify a command line, along with --help, it will
> parse as
> \> much as possible into the help, with potential explanations, e.g. if you
> \> choose --verbose --help, the --verbose option in the help output could be
> \> documented. Ideally, the same with things like `one` and `many` tokens,
> so
> \> that if you, say, wrote teapot --help build Run/MyProject it could even
> give
> \> you application specific output relating to "Run/MyProject" which has
> some
> \> description/documentation associated with it. This is a big usability
> thing
> \> to me so I'll be working in this area next.
>
> That's quite a lot whistles and bells for an option parser. 🙂 I am
> not sure whether generating documentation should be an option parser's
> task. I just does not feel right because generating documentation is a
> built time task while option parsing is a run time task.
A samovar is a vessel in which to contain tea. Samovar (the gem) is a
command line interface API for your program. It should make mapping from a
command line into your code as easy as possible. In that sense, the option
parser is just one aspect of this.
> \>\> \> In the end I got sick of
> \>\> \> fighting with Trollop so decided to roll my own. I will be using flopp
> \>\> \> for
> \>\> \> all my command line tools now.
> \>\>
> \>\> I find it preferable to use something in the standard library. But of
> \>\> course everybody's mileage differs.
> \>
> \> Fair enough. My opinion is that the standard library should be simplified
> \> and kept as small as possible.
>
> I disagree. You can compare C++ and Java to see why. Java's standard
> library is very comprehensive and has solutions for a lot basic
> functionality. In C++ you need a library for every mildly advanced
> features which leads to a lot duplication and the trouble you get into
> if you want to use one library for a functionality and another
> component uses another library. Now you get bloat. Or you require
> different version and the other lib requires another version. Or your
> code and the library has incompatible data types so you end up doing a
> lot conversions. ...
On this point, I understand what you are saying, and agree that the C++
standard library is pretty crappy. In my mind, though, there is a
distinction between things which can be implemented independently (e.g.
specific data structures, non-specific algorithmic code) and things that
should be in the core standard library (OS level APIs, e.g. file system,
networking, input, output, etc, core data structures). An option parser in
the standard library is in my mind, a little bit excessive, as it's not a
OS API, nor a core data structure.
> \> Otherwise, it's hard to version, change,
> \> improve, etc. While OptionParser might be awesome (and it sounds like it
> is
> \> for simple stuff), ideally it would be a separate gem.
>
> OptionParser can do quite a lot which is often overlooked because of
> the minimalist documentation.
>
> \> - Options can only have one value and it's not type checked for now (e.g.
> \> this must be an int). Easily fixable but not really useful for me right
> now.
> \> If you want to do validation, it could be done elsewhere for now.
>
> OptionParser has this built in. This is very convenient.
It's now implemented too.. Turns out it was more useful than I expected in
some cases 🙂
Thanks,
Samuel
> **···**
>
> On 23 April 2016 at 22:55, Robert Klemme \ wrote:
>
> > On Sat, Apr 23, 2016 at 4:03 AM, Samuel Williams \> \ wrote:
---
### Author:  [@Robert\_K1](https://rubytalk.org/u/Robert_K1)
#### Post date: [24 April 2016 09:35 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/13 "2016-04-24T09:35:25Z")
> Yes, having sub-commands in the parser is awesome.
Gosh! You did it again.
> > I think that is perfectly OK because then you notice that your
> > invocation now has become ambiguous and can change the calling code
> > without silently introducing errors.
>
> Okay, well, I prefer code that doesn't break 🙂
The code does not break, it does exactly what it is supposed to do: it
flags incorrect input. If input is ambiguous no machine can guess the
intention of the user so it's better to fail than to do "something".
> > That's quite a lot whistles and bells for an option parser. 🙂 I am
> > not sure whether generating documentation should be an option parser's
> > task. I just does not feel right because generating documentation is a
> > built time task while option parsing is a run time task.
>
> Well, OptionParser generates documentation to some extent.
It generates output for the help option. For me documentation is
something else, i.e. larger volumes of text with meaningful structure
etc.
> In fact, most
> option parsers do. So, I'd say that in general people are using that
> functionality. In the case of generating help output, I personally believe
> that the help output needs to support the user as much as possible. I'm
> expecting to build on the nature of `git --help` which I personally really
> like.
>
> I'd actually like to see interactive help. So, if you write
>
> command --foo foo.txt --help
>
> You might get some output like:
> Usage: command [--foo \] \
> --foo \ Compute the foo factor. Value: foo.txt
> \ The tags to index from the foo file.
> Possible options are: bob, dole, cat, cheese.
> (taken from foo.txt)
>
> The same applies to commands like git checkout --help - it could
> interactively list available branches or tags, etc.
That is exactly what I would consider whistles and bells. 🙂
On type safety:
> > OptionParser has this built in. This is very convenient.
> I agree and in the future if the need arises I will implement this in
> Samovar 🙂 For now though it's trivial to do the conversion and validation
> in the glue between the parser and your code.
Well, the same argument works for the implicit options hash: setting
options in a Hash is trivial to do in the glue code. 🙂
> I agree though some semantic
> checking might be nice, e.g. \ would validate the path
> already exists.
No option parser can do this on its own without additional help
because it does not know what the path is needed for. Could be
something that the program is supposed to create. And then you still
do not know whether the parent directory is required to be there or if
arbitrary levels of directories will be created by the program. etc.
This is really up to the program to validate.
> But, just blowing up with an error isn't that nice. I'd
> prefer some kind of help otuput which explains what value was invalid and
> why.
Yes, that would be better but can get quite complicated depending on
the target type. How far do you want to go with a regular expression?
I find the error message pretty good already - it's just the stack
trace which is ugly.
Cheers
robert
> **···**
>
> On Sun, Apr 24, 2016 at 4:24 AM, Samuel Williams \ wrote:
>
> --
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can
> - without end}
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [24 April 2016 11:09 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/14 "2016-04-24T11:09:44Z")
> The code does not break, it does exactly what it is supposed to do: it
> flags incorrect input. If input is ambiguous no machine can guess the
> intention of the user so it's better to fail than to do "something".
I'd argue this one point. If you have an option parser that allows an input
that later breaks the script, this is intrinsically a bad design. If you
think about this at large: a script deployed in thousands of environments,
suddenly breaks because people used a short-hand notation which was being
implicitly matched to a flag, but that is now ambiguous. Sure, it's great
that it recognizes the ambiguity, but it's not that great, IMHO, that it
was allowed in the first instance.
---
### Author:  [@Robert\_K1](https://rubytalk.org/u/Robert_K1)
#### Post date: [25 September 2016 17:01 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/15 "2016-09-25T17:01:18Z")
> Robert, sorry, I never replied to you in detail.. here are a few comments.
>
> > \> Just a name I came up for when you write "--" and expect the command to
> > \> only
> > \> process stuff on the left while passing stuff on the right to a
> > \> script/sub-process, if any. In Flopp, you just write "split :rest" and
> > \> you
> > \> get documentation, parsing, and the attribute exposed in the command
> > \> instance 🙂
> >
> > I think that (or something similar?) is supported by OptionParser as well:
> >
> > $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> > p ARGV' -x a b c
> > ["a", "b", "c"]
> > $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> > p ARGV' -- -x a b c
> > "x"
> > ["a", "b", "c"]
> > $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> > p ARGV' -- a -x b c
> > "x"
> > ["a", "b", "c"]
> > $ ruby -r optparse -e 'OptionParser.new{|o|o.on("-x"){p "x"}}.parse!;
> > p ARGV' -- a -- -x b c
> > ["a", "-x", "b", "c"]
> >
> > Note: the first "--" is for ruby itself.
>
> Just printing out what's left over in ARGV isn't really helpful.
The printing was just to illustrate which part of the command line
OptionParser is parsing in these scenarios.
> > \> Flopp is very explicit, but it's also easy to specify short and long
> > \> options, etc, such as:
> > \>
> > \> option '-i/--in/--root \', "Set the root path for doing stuff"
> > \>
> > \> This will set @options[:root] to whatever you specify with one of those
> > \> three options. There is no automatic short/long options, etc. The last
> > \> option is used as the key unless it's specified explicitly.
> >
> > I like the implicit configuration Hash. OTOH I also like
> > OptionParser's solution where you have to decide yourself how you will
> > handle options:
> >
> > options = {}
> >
> > OptionParser.new do |opt|
> > opt.on '-i', '--in', '--root PATH', 'Set root path for doing stuff' do
> > \>value\>
> > options[:root] = value
> > end
> > end.parse!
>
> This gets tedious and even error-prone when you have a complex command line
> and lots of options. It's sort of the difference between mutable and
> immutable w.r.t. option parsing.
This is always tedious - no matter what library or tool you use:
mapping multiple options which each might be relevant only for parts
of an application need some wiring done. There are more and less
verbose ways but the programmer does not get rid of that task.
> > That's quite a lot whistles and bells for an option parser. 🙂 I am
> > not sure whether generating documentation should be an option parser's
> > task. I just does not feel right because generating documentation is a
> > built time task while option parsing is a run time task.
>
> A samovar is a vessel in which to contain tea. Samovar (the gem) is a
> command line interface API for your program. It should make mapping from a
> command line into your code as easy as possible. In that sense, the option
> parser is just one aspect of this.
I am not sure what "mapping from a command line into your code" really
means. Still, generating documentation is a build or packaging time
task and not necessarily something do be done at runtime.
> > \> Fair enough. My opinion is that the standard library should be
> > \> simplified
> > \> and kept as small as possible.
> >
> > I disagree. You can compare C++ and Java to see why. Java's standard
> > library is very comprehensive and has solutions for a lot basic
> > functionality. In C++ you need a library for every mildly advanced
> > features which leads to a lot duplication and the trouble you get into
> > if you want to use one library for a functionality and another
> > component uses another library. Now you get bloat. Or you require
> > different version and the other lib requires another version. Or your
> > code and the library has incompatible data types so you end up doing a
> > lot conversions. ...
>
> On this point, I understand what you are saying, and agree that the C++
> standard library is pretty crappy. In my mind, though, there is a
> distinction between things which can be implemented independently (e.g.
> specific data structures, non-specific algorithmic code) and things that
> should be in the core standard library (OS level APIs, e.g. file system,
> networking, input, output, etc, core data structures). An option parser in
> the standard library is in my mind, a little bit excessive, as it's not a OS
> API, nor a core data structure.
Well, this is not surprising - considering the scope of functionality
you see in there. The command line parsing functionality I had in mind
was much more reduced from what your gem provides. I am pretty happy
with OptionParser in the std lib the way it is.
Cheers
robert
> **···**
>
> On Sun, Sep 25, 2016 at 3:33 PM, Samuel Williams \ wrote:
>
> > On 23 April 2016 at 22:55, Robert Klemme \ wrote:
> >
> > > On Sat, Apr 23, 2016 at 4:03 AM, Samuel Williams \>\> \ wrote:
>
> --
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can
> - without end}
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)
---
### Author:  [@Samuel\_Williams](https://rubytalk.org/u/Samuel_Williams)
#### Post date: [25 September 2016 13:14 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/16 "2016-09-25T13:14:43Z")
David, those are interesting thoughts.
Samovar has so far proved to be a boon in several projects I've refactored
(mostly coming from trollop):
- Teapot
\<[https://github.com/ioquatix/teapot/blob/master/lib/teapot/command.rb>](https://github.com/ioquatix/teapot/blob/master/lib/teapot/command.rb>); is
a build system and uses multiple top-level commands.
- Utopia
\<[https://github.com/ioquatix/utopia/blob/master/lib/utopia/command.rb>](https://github.com/ioquatix/utopia/blob/master/lib/utopia/command.rb>); is
a web application platform and uses nested commands.
- Synco
\<[https://github.com/ioquatix/lsync/blob/master/lib/synco/command.rb>](https://github.com/ioquatix/lsync/blob/master/lib/synco/command.rb>); is
a backup tool and sends commands across the network and has lots of options
with default values.
There are several functional improvements I want to make for 2.0 but so far
they are relatively minor (they won't affect the Samovar API in an
incompatible way). In all cases, the samovar command line has been
significantly easier to develop, manage and test because it forces a
specific structure to the command line code.
I like the idea of having a single authoritative source for the synopsis
and description. Does this belong in a man page? That is an interesting
question. You might be right here.. but hear me out:
In the current software development environment, assuming that a single
command is unique system-wide might be risky. For example, with Node you
have npm installing several different versions of the same package/command.
The same with Ruby and bundler, etc.
Therefore, if you type "man foobar" where foobar is installed in several
places with several different versions, what would you see?
In my mind, while there are some hassles (e.g. the pager issue you
mention), none of these are insurmountable.
While I like Samovar a lot in it's current incarnation, I'd like to make
some future changes:
- Support autocompletion out of the box, adding the necessary "type"
information to make this work (e.g. support named enumerations).
- Short "usage" vs longer "man" style documentation output.
- Allow flags to propagate upwards, from sub-commands until a command which
matches it is found (makes order of options less important if they can be
disambiguated).
In any case, I'm not convinced the man-page-first approach is a good one
given that commands might not be globally installed, yet man is a global
system-wide database.
> **···**
>
> On 22 September 2016 at 17:28, David Gamba \ wrote:
>
> > \> - Awesome documentation for nested commands including colours. See the
> > picture on the github project page for an example.
> > \> - Future features planned: detailed help (perhaps the ability to
> > generate man/markdown pages too).
> >
> > I find that most option parsers out there are 90% code to generate a help
> > message!
> >
> > When you are talking about a multi layered script, it is way faster and
> > simpler for a programmer to write a proper synopsis himself and then write
> > a proper man page for their program. Man pages have all the colors you want
> > already.
> >
> > I seriously find it super annoying when I find a lazy programmer that used
> > one of these option parsers that generate multi page option descriptions
> > and it doesn't even pipe it to a pager for you.
> > Most of them don't really make sense anyway, their help message ends up
> > being: script-name [OPTIONS...]
> > And then a blast of options that you can't tell from the automated help if
> > they can be mixed together or not.
> >
> > I even contributed a PR to the manpage backend for asciidoctor. So really,
> > man pages couldn't be easier to write.
> >
> > I have taken a stab at the option parsing problem myself here:
> > [GitHub - DavidGamba/ruby-getoptions: Ruby option parser based on Perl’s Getopt::Long](https://github.com/DavidGamba/ruby-getoptions)
> >
> > If anything, you might find the introduction useful because it includes
> > what I believe is required from an option parser.
> > The API is very similar to that of Perl's GetOpt::Long.
> >
> > I also took a stab at it in go: [https://github.com/](https://github.com/)
> > DavidGamba/go-getoptions
> > And the difference there is that being a static type system, my API had to
> > be completely different to keep that type safety. I expanded the README
> > there a bit more.
> >
> > Hope that gives you some good ideas and that it makes you consider not
> > mixing those two different concerns: Option parsing and documentation.
> >
> > The Perl team had it right and that is what they initially used for those
> > git man pages you talked about. They have a different package to extract
> > the synopsis out of a script's man page, which is written in POD format,
> > called pod2usage. The programmer always wrote the synopsis and full man
> > page then.
> >
> > --David
> >
> > Unsubscribe: \
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>);
---
### Author:  [@Robert\_K1](https://rubytalk.org/u/Robert_K1)
#### Post date: [25 April 2016 06:53 UTC](https://rubytalk.org/t/ann-flopp-flexible-option-parser/73585/17 "2016-04-25T06:53:39Z")
Right, this is the "never change a published API" rule. I had not
thought of that implication.
Kind regards
robert
> **···**
>
> On Sun, Apr 24, 2016 at 1:09 PM, Samuel Williams \ wrote:
>
> > > The code does not break, it does exactly what it is supposed to do: it
> > > flags incorrect input. If input is ambiguous no machine can guess the
> > > intention of the user so it's better to fail than to do "something".
> >
> > I'd argue this one point. If you have an option parser that allows an input
> > that later breaks the script, this is intrinsically a bad design. If you
> > think about this at large: a script deployed in thousands of environments,
> > suddenly breaks because people used a short-hand notation which was being
> > implicitly matched to a flag, but that is now ambiguous. Sure, it's great
> > that it recognizes the ambiguity, but it's not that great, IMHO, that it was
> > allowed in the first instance.
>
> --
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can
> - without end}
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)