ANN: MetaTags 1.0

[Ryan_Pavlik]

MetaTags 1.0

···

============

MetaTags is a rather simple ruby module. Its goal is the ability to
easily define and produce sets of tagged metadata in the code. The
code can then examine this metadata and use it as necessary.

Two full tag sets are provided: one for class info (such as the
class name, a description, and attributes), and one for method info
(such as method name, title, category, description, parameters,
etc.).

While inline documetation systems have a goal of parsing
documentation and providing human-readable output, the goal of
MetaTags is to produce documentation that is readable by the code
itself. The result of this could also be used to provide nice
human-readable documentation, as well.

Here is an example:

class_info <<-DOC
  !Class: MyClass
  !Desc:  This class does something

  !attr a: A: The "A" attribute
  !attr b: B
DOC
class MyClass
  attr_accessor :a
  attr_reader   :b

  method_info <<-DOC
    !Method:   initialize
    !Title:    Initialize
    !Category: constructors

    !Desc: Create a new MyClass object giving optional defaults
           for "a" and "b".

    !Parameters:
       * a: A: The "a" object; defaults to 5
       * b: B

    !param_flag b: width=80
  DOC
  def initialize(a, b)
    :
  end

  :
end

You can find MetaTags at the following location:

http://mephle.org/MetaTags/

–
Ryan Pavlik rpav@mephle.com

[Tobias1 (Tobias)]

Ryan Pavlik wrote:

  method_info <<-DOC
    !Method:   initialize
    !Title:    Initialize
    !Category: constructors

    !Desc: Create a new MyClass object giving optional defaults
           for "a" and "b".

    !Parameters:
       * a: A: The "a" object; defaults to 5
       * b: B

    !param_flag b: width=80
  DOC
  def initialize(a, b)
    :
  end

I’d rather like
method_info <<-DOC
Create a new MyClass object giving optional defaults
for “a” and “b”.

 Category: constructors
 Parameters:
   * b: B

DOC
def initialize(a=5,b=B.new) … end

I.e. less punctuation, make your “Desc:” tag the default, get method
name by overiding method_added

Or, maybe
doc('Create a new MyClass object giving optional defaults ',
‘for “a” and “b”’,
:category, ‘constructors’,
:parameters,
:a, ‘A: The “a” object; defaults to 5’,
:b, ‘B’)
def initialize(a=5,b=B.new)

···

• a: A: The “a” object; defaults to 5

–
Tobias
(email-address is anti-spammed, remove invalid parts)

[_Sean_O_Dell]

Ryan Pavlik wrote:

Here is an example:

class_info <<-DOC
  !Class: MyClass
  !Desc:  This class does something

  !attr a: A: The "A" attribute
  !attr b: B
DOC
class MyClass
  attr_accessor :a
  attr_reader   :b

  method_info <<-DOC
    !Method:   initialize
    !Title:    Initialize
    !Category: constructors

    !Desc: Create a new MyClass object giving optional defaults
           for "a" and "b".

    !Parameters:
       * a: A: The "a" object; defaults to 5
       * b: B

    !param_flag b: width=80
  DOC
  def initialize(a, b)
    :
  end

  :
end

Ryan, for what it’s worth, I like this pretty much how it works right
now. If I were to change anything it would be:

Make it so “!” is re-definable and ignore any text that appears before
it. This could be used as a generic source code parser which extracts
documentation, etc. from the source code (by parsing anything after a
comment character + a semaphor). As a stand-alone library, that could
have lots of other uses.

Sean O'Dell

[Aredridel1 (Aredridel)]

class_info <<-DOC
  !Class: MyClass
  !Desc:  This class does something

  !attr a: A: The "A" attribute
  !attr b: B
DOC

Is that YAML?!

Sure looks similar, but I’m hardly familiar with YAML yet.

If not, YAML might be the way to go…

[Ryan_Pavlik]

> > I'd rather like > method_info < Create a new MyClass object giving optional defaults > for "a" and "b". > > Category: constructors > Parameters: > * a: A: The "a" object; defaults to 5 > * b: B > DOC > def initialize(a=5,b=B.new) ... end > > I.e. less punctuation, make your "Desc:" tag the default, get method > name by overiding method_added

Unfortunately there’s nothing tying the method to the method_info
without telling it. In a few cases, I’ve used this in subclasses
without any method redefinition, just to make the wordking more
appropriate. (This is used to generate various forms.)

As for punctuation, I’ll make it so you can alter the regexp used for
splitting the documentation… but you may find you need something so
that any description strings that have a ": " in them don’t get split.

Or, maybe
doc('Create a new MyClass object giving optional defaults ',
‘for “a” and “b”’,
:category, ‘constructors’,
:parameters,
:a, ‘A: The “a” object; defaults to 5’,
:b, ‘B’)
def initialize(a=5,b=B.new)

Hah. The irony… this is similar to the old format I was using. In
fact, if you prefer this, it’d be trivial to write a function that fills
in the same structure merely using this form. After using it for
awhile, though, I found it to be much harder to both write (lots of
symbols) and look at (same reason) than something that’s mostly text.
Since you write one for every method, it tends to wear. Also, with
normal strings, you don’t have a lot of room for descriptions and the
like, and I ran into this limitation a number of times.

Thanks for the feedback,

···

On Mon, 8 Sep 2003 21:58:17 +0900 Tobias Peters tpeters@invalid.uni-oldenburg.de wrote:

–
Ryan Pavlik rpav@mephle.com

“Another perfectly calculated space-time
splice-n-splice. Now to get back to… wait a second.
I forgot to carry the TWO!” - 8BT

[why_the_lucky_stiff1 (why the lucky stiff)]

Looks like YAML just about. Save yourself the parsing and go for:

Call YAML::load inside method_info …

method_info <<-DOC
Method: initialize
Title: Initialize
Category: constructors
Desc: Create a new MyClass object giving optional defaults
for “a” and “b”.
Parameters:
- a: ‘A: The “a” object; defaults to 5’
- b: ‘B’
DOC

The parameter list comes back as an array. Lovely.

Even better, check out the %y{ … } patch:

http://whytheluckystiff.net/ruby/ruby-1.8.0-yamlstr.patch

And then:

method_info %y{
Method: initialize
Title: Initialize
…
}

_why

···

On Monday 08 September 2003 06:58 am, Tobias Peters wrote:

Ryan Pavlik wrote:

  method_info <<-DOC
    !Method:   initialize
    !Title:    Initialize
    !Category: constructors
    ... and so on ...

I’d rather like
method_info <<-DOC
Create a new MyClass object giving optional defaults
for “a” and “b”.

 Category: constructors
 ... and so on ...

[Ryan_Pavlik]

Ryan, for what it’s worth, I like this pretty much how it works right
now. If I were to change anything it would be:

Thanks… feedback is good.

Make it so “!” is re-definable and ignore any text that appears before
it. This could be used as a generic source code parser which extracts
documentation, etc. from the source code (by parsing anything after a
comment character + a semaphor). As a stand-alone library, that could
have lots of other uses.

I’ll definitely include this in the next release… probably on both a
global and a per-TagSet basis. Basically, it just splits the string on
a regexp, and the regexp is currently /^\s*!(\w+):?/. I just picked !
because I believe javadoc and doxygen use it, so it might be familiar.

In the next release I’ll also try to include a basic tool that generates
HTML as an example. This wasn’t my primary goal, but it’d probably be
useful.

thanks,

···

On Tue, 9 Sep 2003 04:59:32 +0900 “Sean O’Dell” <sean@cSePlsoAfMt.com [REMOVE_THE_SPAM]> wrote:

–
Ryan Pavlik rpav@mephle.com

“Another perfectly calculated space-time
splice-n-splice. Now to get back to… wait a second.
I forgot to carry the TWO!” - 8BT

[Ryan_Pavlik]

Parsing the tags is a simple str.split operation. The tricky bit is
parsing the content of the sections, which don’t necessarily look like
YAML at all. For instance:

!param_flags foo: x=5, foo, bar=false

Of course, MetaTags is mostly for defining the set of tags you’re using;
theoretically, you could subclass or dynamically modify TagSet to take a
YAML-preparsed string, and then apply the Section parsers to each YAML
string.

Parsing the given tag format is pretty trivial, though. MetaTags just
makes it easy to define a set of tags and their corresponding parsers:

text = TextSection.new
sym = SymbolSection.new
int = IntegerSection.new
ret = ReturnSection.new
par_long = ParametersSection.new
par_short = ParamSection.new
par_flags = ParamFlagsSection.new

TS = TagSet.new([“method”, “name”, sym, TAG_REQUIRED],
[“title”, nil, text],
[“category”, nil, sym],
[“order”, nil, int],
[“desc”, nil, text],
[“description”, “desc”, text],
[“param”, “params”, par_short],
[“parameters”, “params”, par_long],
[“returns”, nil, ret],
[“param_flags”, nil, par_flags])

You could, if desired, do TS = MyTagSet(…) in your application, and
get a different input format. What’s mostly important is that you get
the same information out in the end, as opposed to the input format
(since people obviously have different preferences).

···

On Tue, 9 Sep 2003 01:19:36 +0900 why the lucky stiff ruby-talk@whytheluckystiff.net wrote:

On Monday 08 September 2003 06:58 am, Tobias Peters wrote:

Ryan Pavlik wrote:

  method_info <<-DOC
    !Method:   initialize
    !Title:    Initialize
    !Category: constructors
    ... and so on ...

I’d rather like
method_info <<-DOC
Create a new MyClass object giving optional defaults
for “a” and “b”.

 Category: constructors
 ... and so on ...

Looks like YAML just about. Save yourself the parsing and go for:

–
Ryan Pavlik rpav@mephle.com

“Another perfectly calculated space-time
splice-n-splice. Now to get back to… wait a second.
I forgot to carry the TWO!” - 8BT

[_Sean_O_Dell]

Ryan Pavlik wrote:

Ryan, for what it’s worth, I like this pretty much how it works right
now. If I were to change anything it would be:

Thanks… feedback is good.

Make it so “!” is re-definable and ignore any text that appears before
it. This could be used as a generic source code parser which extracts
documentation, etc. from the source code (by parsing anything after a
comment character + a semaphor). As a stand-alone library, that could
have lots of other uses.

I’ll definitely include this in the next release… probably on both a
global and a per-TagSet basis. Basically, it just splits the string on
a regexp, and the regexp is currently /^\s*!(\w+):?/. I just picked !
because I believe javadoc and doxygen use it, so it might be familiar.

In the next release I’ll also try to include a basic tool that generates
HTML as an example. This wasn’t my primary goal, but it’d probably be
useful.

Or just dump it as XML and let people worry about transforming it into
HTML themselves. Generating XML is pretty simple; you just bracket
stuff by name and transform &, ', ", < and > character. That part is just:

class String
def escape_xml()
result = self.clone
result.gsub!(/[&‘"<>]/) do | m |
case m
when “&”
“&”
when "’"
“'”
when “"”
“&quit;”
when “<”
“<”
when “>”
“>”
end
end
end
end

… or perhaps someone knows a Rubyism that would be more efficient.

Sean O'Dell

···

On Tue, 9 Sep 2003 04:59:32 +0900 > “Sean O’Dell” <sean@cSePlsoAfMt.com [REMOVE_THE_SPAM]> wrote:

[Aredridel1 (Aredridel)]

Even better, check out the %y{ … } patch:

http://whytheluckystiff.net/ruby/ruby-1.8.0-yamlstr.patch

Matz: Any plans to make this standard? Looks like a winner to me.

If so, I’ll add the patch to the PLD 1.8.0 RPM.

Ari

[Tobias_Peters1 (Tobias Peters)]

> > I'd rather like > method_info < Create a new MyClass object giving optional defaults > for "a" and "b". > > Category: constructors > Parameters: > * a: A: The "a" object; defaults to 5 > * b: B > DOC > def initialize(a=5,b=B.new) ... end > > I.e. less punctuation, make your "Desc:" tag the default, get method > name by overiding method_added

Unfortunately there’s nothing tying the method to the method_info
without telling it.

Yes there is. method_added is what you need. See:

test_method_added.rb>>>>>>>>
METHOD_DOCS =
class Method_Doc
def initialize(data, klass)
@data = data
@klass = klass
METHOD_DOCS << self
end
attr_accessor :method_name
attr_reader :data, :klass
end

class Module
def method_doc(data)
@last_method_doc = Method_Doc.new(data, self);
end

alias pre_method_doc_method_added method_added

def method_added(name)
	(doc = @last_method_doc) && (doc.method_name = name)
	pre_method_doc_method_added(name)
end

end

class Example
method_doc <<-DOC
This does some serious init work
DOC
def initialize(a,b)
@a,@b=a,b
end
end

p METHOD_DOCS
<<<<<<<<<<<<test_method_added.rb<<<<<<<<<<<

$ ruby test_method_added
[#<Method_Doc:0x4013f34c
@method_name=:initialize,
@klass=Example,
@data=“\t\tThis does some serious init work\n”>]

@klass info comes from Module#method_doc, @method_name info comes from
Module#method_added

In a few cases, I’ve used this in subclasses
without any method redefinition, just to make the wordking more
appropriate. (This is used to generate various forms.)

I cannot follow.

Tobias

···

On Tue, 9 Sep 2003, Ryan Pavlik wrote:

On Mon, 8 Sep 2003 21:58:17 +0900 > Tobias Peters tpeters@invalid.uni-oldenburg.de wrote:

[David_A_Black2 (David A. Black)]

Hi –

Looks like YAML just about. Save yourself the parsing and go for:

I was thinking the same. Good idea.

Even better, check out the %y{ … } patch:

http://whytheluckystiff.net/ruby/ruby-1.8.0-yamlstr.patch

I think this is a really nifty patch, but I have a concern. If Ryan were
to use this syntax (and your patch), his library would require a modified
Ruby interpreter, which would severely limit the acceptance and usage of
the library. Also, being a one-off patch, we would be basically getting a
strong-hold on %y{}, which would present problems if the ruby interpreter
some day implemented %y{} for something else.

Now, you may say “let’s just add this to the core ruby interpreter.”

My opinion is that if we’re going to add special syntax for YAML, we
should have syntax for XML %xml{} (and potentially CSV %csv{} and other
formats). It doesn’t make sense to me to implement YAML or XML as a core
part of the language. I like the fact that we’ve got high
quality libraries for them, and I like the fact that these libraries are
included with the standard Ruby distribution, but I wouldn’t want to see
them creep into the actual language syntax.

Any other thoughts on this one?

I agree with your points (about both the strength of the libraries and
the pitfalls of patching) – and it also reminds me of an idea from
long ago that I never researched the feasibility of very deeply…
Namely, the idea of an extensible %_{} syntax, so that one could
define the behavior of %y{}, %x{}, and so on from within a Ruby
program or extension.

Perhaps someone steeped in the ways of the parser could comment on
how hard this would be (up to and including impossible).

David

···

On Tue, 9 Sep 2003, Chad Fowler wrote:

On Tue, 9 Sep 2003, why the lucky stiff wrote:

–
David Alan Black
home: dblack@superlink.net
work: blackdav@shu.edu
Web: http://pirate.shu.edu/~blackdav

[why_the_lucky_stiff1 (why the lucky stiff)]

yeah, not suggesting %y{…} as a dependancy. more of an egg rolled out into
the road… knowing it’ll get crushed… hoping drivers will only casually
observe… rather than erratic swerving and insistent toppling of telephone
poles.

_why

···

On Monday 08 September 2003 11:53 am, Chad Fowler wrote:

I think this is a really nifty patch, but I have a concern. If Ryan were
to use this syntax (and your patch), his library would require a modified
Ruby interpreter, which would severely limit the acceptance and usage of
the library.

[James_Britt4 (James Britt)]

Chad Fowler wrote:

···

On Tue, 9 Sep 2003, why the lucky stiff wrote:

Looks like YAML just about. Save yourself the parsing and go for:

I was thinking the same. Good idea.

Would it require, then, that you had to be extra careful about
whitespace and indentation to use MetaTags, since the data would have to
be well-formed YAML?

James

[HAL_9000]

Aredridel wrote:

Even better, check out the %y{ … } patch:

http://whytheluckystiff.net/ruby/ruby-1.8.0-yamlstr.patch

Matz: Any plans to make this standard? Looks like a winner to me.

If so, I’ll add the patch to the PLD 1.8.0 RPM.

Read over Chad’s comments about a dozen posts ago.

I share his concern, and I wouldn’t be surprised if Matz
did also. But IANYM.

Hal

[Mauricio_Fernndez]

Unfortunately there’s nothing tying the method to the method_info
without telling it.

Yes there is. method_added is what you need. See:

test_method_added.rb>>>>>>>>
[…]
<<<<<<<<<<<<test_method_added.rb<<<<<<<<<<<

$ ruby test_method_added
[#<Method_Doc:0x4013f34c
@method_name=:initialize,
@klass=Example,
@data=“\t\tThis does some serious init work\n”>]

@klass info comes from Module#method_doc, @method_name info comes from
Module#method_added

In a few cases, I’ve used this in subclasses
without any method redefinition, just to make the wordking more
appropriate. (This is used to generate various forms.)

I cannot follow.

I think he meant the method you suggest wouldn’t work if the method is
not redefined, but it makes sense to expand it as follows

METHOD_DOCS =
class Method_Doc
def initialize(data, klass)
@data = data
@klass = klass
METHOD_DOCS << self
end
attr_accessor :method_name
attr_reader :data, :klass
end

class Module
def method_doc(data, method = nil)
if method
d = Method_Doc.new data, self
d.method_name = method
@last_method_doc = nil # don’t mess up
else
@last_method_doc = Method_Doc.new(data, self);
end
end

alias pre_method_doc_method_added method_added

def method_added(name)
@last_method_doc ||= nil # get rid of warning with -w
(doc = @last_method_doc) && (doc.method_name = name)
@last_method_doc = nil # don’t modify twice the same docs
pre_method_doc_method_added(name)
end
end

class Example
method_doc <<-DOC
This does some serious init work
DOC
def initialize(a,b)
@a,@b=a,b
end

def foo
end
end

class Bla < Example
method_doc <<-DOC, :initialize # this breaks vim’s syntax colouring
This does some serious init work, but I explain it using a
terminology more appropriate for Bla.
DOC

def bar # this shouldn’t be documented
end
end

p METHOD_DOCS

···

On Tue, Sep 09, 2003 at 06:01:59PM +0900, Tobias Peters wrote:

–
_ _

__ __ | | ___ _ __ ___ __ _ _ __
'_ \ / | __/ __| '_ _ \ / ` | ’ \
) | (| | |__ \ | | | | | (| | | | |
.__/ _,|_|/| || ||_,|| |_|
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

We come to bury DOS, not to praise it.
– Paul Vojta, vojta@math.berkeley.edu

[Stanislav_Karchebny]

I think this is a really nifty patch, but I have a concern. If Ryan were
…
Now, you may say “let’s just add this to the core ruby interpreter.”
…
included with the standard Ruby distribution, but I wouldn’t want to see
them creep into the actual language syntax.

Any other thoughts on this one?

Actually, I wouldn’t want to creep the core language with various unrelated
things. But this one is really a nifty one and I would like to see it
available in the core distribution.

···

–
keep in touch. berkus. – http://lye.upnet.ru/

[HAL_9000]

dblack@superlink.net wrote:

[snip]

I agree with your points (about both the strength of the libraries and
the pitfalls of patching) – and it also reminds me of an idea from
long ago that I never researched the feasibility of very deeply…
Namely, the idea of an extensible %_{} syntax, so that one could
define the behavior of %y{}, %x{}, and so on from within a Ruby
program or extension.

Perhaps someone steeped in the ways of the parser could comment on
how hard this would be (up to and including impossible).

I also agree with Chad’s points.

As for your idea, I like it. As to its difficulty, my (uneducated)
opinion is: In the current Ruby implementation, impossible; in a
future one, difficult.

Hal

[why_the_lucky_stiff1 (why the lucky stiff)]

this wouldn’t be much of a problem. right now there’s a ‘default’ case on a
switch in parse.y:

      default:
    yyerror("unknown type of %string");
    return 0;

establish a USERSTR set of tokens and node types. (you can follow XSTR or
DXSTR around to see how.) Then, in rb_eval, add a case for NODE_MISCSTR,
which could check for the character in a global hash, passing the string
contents into a proc associated with that character. return the result of
the proc call from rb_eval.

the %y{…} hack could become:

add_delimited_prefix( ‘y’ ) do |str|
YAML::load( str )
end

_why

···

On Monday 08 September 2003 12:08 pm, dblack@superlink.net wrote:

Namely, the idea of an extensible %_{} syntax, so that one could
define the behavior of %y{}, %x{}, and so on from within a Ruby
program or extension.

Perhaps someone steeped in the ways of the parser could comment on
how hard this would be (up to and including impossible).