Using YAML as config

ruby-talk
1

Hi all -

I want to write a config file in YAML. In this file I will store a
small graph of objects in an array:

- !mydomain,mydate/Company
  name: TuesdayCo
  subthings:
    - !mydomain,mydate/Person
      name: Ruby Tuesday
    - !mydomain,mydate/Person
      name: Tuesday Weld
- !mydomain,myDate
  name: FridayCo
  subthings:
    etc.

Then I want to read in this file and get an array of Company objects,
each of which has an array of Person objects. I'm defining these
classes myself. I've tried using add_domain_type and hoped to get
instances of my classes, but instead I got instances of
YAML::DomainType.

How should I do this?

-larry

···

--
Posted via http://www.ruby-forum.com/.

2

Larry Edelstein wrote:

Hi all -

I want to write a config file in YAML. In this file I will store a
small graph of objects in an array:

- !mydomain,mydate/Company
  name: TuesdayCo
  subthings:
    - !mydomain,mydate/Person
      name: Ruby Tuesday
    - !mydomain,mydate/Person
      name: Tuesday Weld
- !mydomain,myDate
  name: FridayCo
  subthings:
    etc.

Then I want to read in this file and get an array of Company objects,
each of which has an array of Person objects. I'm defining these
classes myself. I've tried using add_domain_type and hoped to get
instances of my classes, but instead I got instances of
YAML::DomainType.

How should I do this?

Here's the only way I know to do this (I'm not sure if add_domain_type is another way):

require 'yaml'

# This assumes that all your objects will serialize as hashes ("maps")
module Base
   def to_yaml( opts = {} )
     YAML::quick_emit( object_id, opts ) do |out|
       out.map( taguri, to_yaml_style ) do |map|
         to_yaml_properties.each do |m|
           map.add( m, send( m ) )
         end
       end
     end
   end

   module BaseModuleMethods
     def yaml_new( klass, tag, val )
       unless Hash === val
         raise YAML::TypeError, "Invalid #{self}: " + val.inspect
         ## do more error checking here, if desired
       end

       name, type = YAML.read_type_class( tag, self )
       st = type.new
       val.each do |k,v|
         st.send( "#{k}=", v )
       end
       st
     end
   end

   def self.included mod
     mod.extend BaseModuleMethods
   end
end

class Company
   include Base

   attr_accessor :name, :subthings
   def initialize name = nil
     @name = name
   end

   yaml_as "tag:my.domain,2007:Company"

   def to_yaml_properties
     ["name", "subthings"]
   end
end

class Person
   include Base

   attr_accessor :name
   def initialize name = nil
     @name = name
   end

   yaml_as "tag:my.domain,2007:Person"

   def to_yaml_properties
     ["name"]
   end
end

co = Company.new "TuesdayCo"
co.subthings = [
   Person.new("Ruby Tuesday"),
   Person.new("Tuesday Weld")
]

co_yaml = co.to_yaml
puts co_yaml
# ==>
# --- !my.domain,2007/Company
# name: TuesdayCo
# subthings:
# - !my.domain,2007/Person
# name: Ruby Tuesday
# - !my.domain,2007/Person
# name: Tuesday Weld

co2 = YAML.load(co_yaml)
p co2

# ==>
#<Company:0xb7ce3cfc @subthings=[#<Person:0xb7ce44cc @name="Ruby Tuesday">, #<Person:0xb7ce3ef0 @name="Tuesday Weld">], @name="TuesdayCo">

# Check that error-checking works by
# giving the parser an array instead of a hash:
s = <<END
--- !my.domain,2007/Company
- 1
- 2
END

begin
   p YAML.load(s)
rescue YAML::TypeError => ex
   puts ex.message # Invalid Company: [1, 2]
end

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

3

Heya, larry! The easy way is like this:

  class Company
    yaml_as 'tag:mydomain.com,2007:Company'
  end

  class Person
    yaml_as 'tag:mydomain.com,2007:Person'
  end

_why

···

On Mon, Feb 12, 2007 at 10:21:54AM +0900, Larry Edelstein wrote:

- !mydomain,mydate/Company
  name: TuesdayCo
  subthings:
    - !mydomain,mydate/Person
      name: Ruby Tuesday
    - !mydomain,mydate/Person
      name: Tuesday Weld
- !mydomain,myDate
  name: FridayCo
  subthings:
    etc.
[...]

How should I do this?

4

Wow Joel - that's a lot of code and a lot to digest. I'm going through
it. Your response and _why's use the yaml_as attribute - I've never
even seen that before.

···

--
Posted via http://www.ruby-forum.com/.

5

Larry Edelstein wrote:

Wow Joel - that's a lot of code and a lot to digest. I'm going through it. Your response and _why's use the yaml_as attribute - I've never even seen that before.

It's nice to know that _why's much simpler code works. The other stuff I wrote is how I've handled validation and other customization before. I wonder if _why has a better way to do that too...

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

6

So I come to the part of my little app where I want to load and save a
config file. I want it to be human read/writeable too though - it's not
just for serialization. In fact, usually a human will write it, my
little app will read it, and may adjust it and write it out anew.

So in xml it would look like this
<channels>
  <channel>
    <name>foo</name>
    <id>3</id>
  </channel>
  <channel>
…
</channels>

I don't care about channels that's just a random root node, and I might
do it with attbs instead of elements, so:

<channel name="foo" id="3"/>

So I thought, lets use YAML instead - it's even easier to read and
write. Google, google, read blogs, docs, google, google, more docs, more
blogs, google to_yaml, fxri: hack hack, google to_yaml_type, fxri: hack,
hack, hack, google, google, yaml_as, hack.

Argh! Seems like day later, and I'm ready to go back to xml. But I'm
sure there's a simple way to do what I want, so I'll post here first and
hope. See, my class is Channel, like so:

class Channel; attr_accessor :name, :id; end

and want my array of channels to look something like this

- Channel
  name: foo
  id: 3
- Channel
  name: bar
  id: 4
…

But I just can't work out how to simply the yaml output to this stage.
And I don't want to override a dozen methods and do my own hash mapping.
The easiest way - thanks to why for posting this in the forum - seems to
be to use yaml_as:

class Channel
  yaml_as "tag:foobar,2007:Channel"
end

but then I get each entry as

--- !foolbar,2007/Channel
   name: foo
   id: 3

This is fairly simple, but remember that I want my entries to be
human-writeable the first time. That line of "---!foobar,2007…" is just
way too machiney for us poor humans to get right every time.

I can simplify the taguri like this
  yaml_as "tag::Channel"

to get

--- !Channel
...

and that's about as simple as I can get it without it failing to write
yaml. But loading it back in will fail to build the object - it will
give ma a YAML::DynamicType instead - which is what it always gives if
it doesn't recognize the object.
I guess the load method needs a valid taguri

(Side note - I'd never heard of a taguri before, but when I went to read
about it at http://www.taguri.org/ I was _very_ pleasantly surprised at
how quickly I was able to grok the whole thing by reading Sandro Hawke's
four-paragraph tutorial about his dog Taiko. Way to go Sandro - if I'd
been directed to the RFC first it'd be all over)

So anyway, the fact is that while taguris are great for distinguishing
Taiko from his master's descendents' pooches, and while it's very
simple, it's still overkill for me. I don't need a unique ID. I know
what "--- Channel" means in my little app. I don't want to submit my
Channel types to some great panoply of random little types each with
their own special pet name. I just want to damn well read and write it.

I've even considered parsing the text in to_yaml and YAML.load and
gsubbing "!foobar, 2007/" out and in again. But there's got to be a
better way. So far that better way is looking like xml. but there's got
to be a better rubyful way.

What is it?

···

--
Posted via http://www.ruby-forum.com/.

7

json?

http://json.rubyforge.org/

···

On 5/7/07, Rover Rhubarb <rover.rhubarb@gmail.com> wrote:

So far that better way is looking like xml. but there's got
to be a better rubyful way.

What is it?

--
Greg Donald
http://destiney.com/

8

So I come to the part of my little app where I want to load and save a
config file. I want it to be human read/writeable too though - it's not just
for serialization. In fact, usually a human will write it, my little app
will read it, and may adjust it and write it out anew.

So in xml it would look like this
<channels>
<channel>
   <name>foo</name>
   <id>3</id>
</channel>
<channel>
…
</channels>

I don't care about channels that's just a random root node, and I might
do it with attbs instead of elements, so:

<channel name="foo" id="3"/>

So I thought, lets use YAML instead - it's even easier to read and
write. Google, google, read blogs, docs, google, google, more docs, more
blogs, google to_yaml, fxri: hack hack, google to_yaml_type, fxri: hack,
hack, hack, google, google, yaml_as, hack.

Argh! Seems like day later, and I'm ready to go back to xml. But I'm
sure there's a simple way to do what I want, so I'll post here first and
hope. See, my class is Channel, like so:

class Channel; attr_accessor :name, :id; end

and want my array of channels to look something like this

- Channel
name: foo
id: 3
- Channel
name: bar
id: 4
…

But I just can't work out how to simply the yaml output to this stage.
And I don't want to override a dozen methods and do my own hash mapping.
The easiest way - thanks to why for posting this in the forum - seems to
be to use yaml_as:

class Channel
yaml_as "tag:foobar,2007:Channel"
end

but then I get each entry as

--- !foolbar,2007/Channel
  name: foo
  id: 3

This is fairly simple, but remember that I want my entries to be
human-writeable the first time. That line of "---!foobar,2007…" is just
way too machiney for us poor humans to get right every time.

I can simplify the taguri like this
yaml_as "tag::Channel"

to get

--- !Channel
...

and that's about as simple as I can get it without it failing to write
yaml. But loading it back in will fail to build the object - it will
give ma a YAML::DynamicType instead - which is what it always gives if
it doesn't recognize the object.
I guess the load method needs a valid taguri

(Side note - I'd never heard of a taguri before, but when I went to read
about it at http://www.taguri.org/ I was _very_ pleasantly surprised at
how quickly I was able to grok the whole thing by reading Sandro Hawke's
four-paragraph tutorial about his dog Taiko. Way to go Sandro - if I'd
been directed to the RFC first it'd be all over)

So anyway, the fact is that while taguris are great for distinguishing
Taiko from his master's descendents' pooches, and while it's very
simple, it's still overkill for me. I don't need a unique ID. I know
what "--- Channel" means in my little app. I don't want to submit my
Channel types to some great panoply of random little types each with
their own special pet name. I just want to damn well read and write it.

I've even considered parsing the text in to_yaml and YAML.load and
gsubbing "!foobar, 2007/" out and in again. But there's got to be a
better way. So far that better way is looking like xml. but there's got
to be a better rubyful way.

What is it?

there are simpler ways, but here is one that quite simple and also quite
extensible:

cfp:~ > ruby a.rb
- Foo:
     b: 2
- Foo::Bar:
     b: 1
     c: 1
#<Foo:0xb75aadc4 @b=2, @a=4>
#<Foo::Bar:0xb75aacc0 @b=1, @c=1, @a=40>

cfp:~ > cat a.rb

···

On Tue, 8 May 2007, Rover Rhubarb wrote:
     a: 4
     a: 40
#
# define some classes which impliment the required '.from_hash' and '#to_hash'
# duckface our ObjectList class will expect
#
   class Foo
     def self.from_hash hash
       new hash['a'], hash['b']
     end
     def initialize a, b
       @a, @b = a, b
     end
     def to_hash
       { 'a' => @a, 'b' => @b }
     end
   end
   class Foo::Bar
     def self.from_hash hash
       new hash['a'], hash['b'] , hash['c']
     end
     def initialize a, b, c
       @a, @b, @c = a, b, c
     end
     def to_hash
       { 'a' => @a, 'b' => @b, 'c' => @c }
     end
   end
#
# an ObjectList class adapted to store any objects which respond_to 'to_hash'
# and whose class responds_to 'from_hash'
#
   require 'yaml'
   class ObjectList < ::Array
     attr 'path'
     def initialize path
       load path
     end
     def load path
       @path = path
       replace YAML.load(( IO.read @path rescue '' ))
       map!{|entry| obj_from_entry entry}
     end
     def save path = @path
       yaml = map{|obj| entry_from_obj obj}.to_yaml
       open(path, 'w'){|fd| fd.write yaml}
     end
     def obj_from_entry entry
       classname, hash = entry.to_a.first
       constant_get(classname).from_hash hash
     end
     def entry_from_obj obj
       classname, hash = obj.class.name, obj.to_hash
       { classname => hash }
     end
     def constant_get hierachy
       ancestors = hierachy.split %r/::confused:
       parent = Object
       while((child = ancestors.shift))
         klass = parent.const_get(child) and parent = klass
       end
       klass
     end
   end
#
# setup a object_list object
#
   object_list = ObjectList.new 'object_list'
#
# store a few instances in the object_list
#
   object_list << Foo.new(4,2)
   object_list << Foo::Bar.new(40, 1, 1)
#
# save the object_list
#
   object_list.save
#
# show that save works
#
   puts IO.read(object_list.path)
#
# show that loading works
#
   object_list = ObjectList.new 'object_list'
   p object_list.first
   p object_list.last

-a
--
be kind whenever possible... it is always possible.
- the dalai lama

9

Try:

  class Channel
    yaml_as "x-private:Channel"
  end

And you get:

  --- !!Channel
  id: 3
  name: foo

G'luck!!

_why

···

On Tue, May 08, 2007 at 04:14:24AM +0900, Rover Rhubarb wrote:

I can simplify the taguri like this
  yaml_as "tag::Channel"

to get

--- !Channel
...

and that's about as simple as I can get it without it failing to write
yaml. But loading it back in will fail to build the object - it will
give ma a YAML::DynamicType instead - which is what it always gives if
it doesn't recognize the object.

10

Let me add:

- I meant DomainType, not dynamic type

- The use of yaml_as is redundant since I can get the same effect
without it - where my objects are dumped as

- !ruby/object:Channel

instead of my own silly name.

Still not good enough: how do I get rid of the !ruby/object
machine-talk. (Is it machine-talk, or was YAML designed by bushmen of
the Kalahari, where the scripting language of choice is "click-ruby")

···

--
Posted via http://www.ruby-forum.com/.

11

Hope this helps:

class Channel < Struct.new(:name, :id)
   def self.parse(io)
     io.inject(Array.new) do |channels, line|
       if line =~ /\bChannel\b/
         channels << new
       elsif not channels.empty? and line =~ /\A\s+(name|id):\s+(.+)/
         channels.last.send("#{$1}=", $2)
       end
       channels
     end
   end
end

if __FILE__ == $PROGRAM_NAME
   p Channel.parse(DATA)
end

__END__
- Channel
   name: foo
   id: 3
- Channel
   name: bar
   id: 4

James Edward Gray II

···

On May 7, 2007, at 2:14 PM, Rover Rhubarb wrote:

and want my array of channels to look something like this

- Channel
  name: foo
  id: 3
- Channel
  name: bar
  id: 4

12

Thanks all, for the effort made in replying. I'm pleased to see some
solutions, and don't want to seem ungrateful, but I'm still a little
disillusioned that xml is looking more straightforward even now.

_why: thanks for the x-private tip, but I still have to put
"click-click-Channel". Simple enough to read, but honestly to the
average config hand-coder the xml <Channel> is easier to remember and
understand than the arcane "--- !!Channel". YAML should beat XML hands
down - this is one case where it doesn’t. I know it's not the same
because I'm going to have to parse the XML and build the objects -
whereas YAML is going to deserialize into lovely objects right there on
the spot - but still…

James: thanks for giving me a solution that does what I asked exactly.
But to be honest here I wanted to use an existing format, not invent
one. (I guess it could be argued that by wanting to customize my YAML to
get rid of the ---!! I am trying to invent one - but I really want that
to be an existing feature of YAML)

Greg: funny you mention json because I had a similar experience with it
recently. I was coding some ajax for the first time (building a progress
indicator) and I decided to pass my simple values as properties of a
javascript object using json. Hours of tricky debugging later (very hard
to find those bugs in the json code) I got it working but decided next
time I'd use XML. The extra parsing step might cost more, but the bugs
would be easier to find.

I guess _why has the closest to what I was looking for. x-private is
clearly even intended to be used for this. (BTW sorry to bellyache
about docs, since I know its enough effort to code the libs in the first
place, but this can't be an uncommon request so maybe the x-private
could get more airtime in the docs somewhere). And I guess what I'd
really like is for the x-private tag in yaml_as to allow the --- !! to
be skipped to.

thanks again all

···

--
Posted via http://www.ruby-forum.com/.

13

YAML should beat XML hands down - this is one case where it doesn’t.

I think mine is an unpopular opinion among Rubyists, but I find XML easier to write, as a human, than YAML. YAML just has too many rules I have to remember. XML is definitely wordier, but I can keep the rules in my head as I work. My editor also makes managing those tags fairly easy.

If all I'm doing is a simple edit here and there though, YAML is easier on the eyes, for sure.

I know it's not the same because I'm going to have to parse the XML and build the objects

I'm pretty sure there's a library or two that handles that:

http://raa.ruby-lang.org/cat.rhtml?category_major=Library;category_minor=XML

James: thanks for giving me a solution that does what I asked exactly.
But to be honest here I wanted to use an existing format, not invent
one. (I guess it could be argued that by wanting to customize my YAML to
get rid of the ---!! I am trying to invent one - but I really want that
to be an existing feature of YAML)

This is where we start to disagree.

You asked for simple and even showed examples of what you had in mind. I wrote code that parses it. Simple pays off at both ends.

You wanted to drop the computerese; my version did. In fact, mine doesn't need those dashes. With a one byte change, you could also make indenting the attributes optional. It gets better and better for the human who needs to write it and it's still not complex on your end.

Finally, I think it's easier to read or write than XML or YAML.

When your needs are big, of course you want to go with a proven format. What you showed is simple though and parsed just fine with a few lines of code. I don't see any shame in that.

James Edward Gray II

···

On May 8, 2007, at 5:17 AM, Rover Rhubarb wrote:

14

James Gray wrote:

I think mine is an unpopular opinion among Rubyists, but I find XML
easier to write, as a human, than YAML. YAML just has too many rules
I have to remember. XML is definitely wordier, but I can keep the
rules in my head as I work. My editor also makes managing those tags
fairly easy.

If all I'm doing is a simple edit here and there though, YAML is
easier on the eyes, for sure.

Amen. I might not have agreed with you had I just read about YAML, but
after going through this process I'm a born again xml convert.

I also agree YAML is easier on the eyes. If were just ser-deser I was
after, I'd probably go with YAML.

I know it's not the same because I'm going to have to parse the XML
and build the objects

I'm pretty sure there's a library or two that handles that:

http://raa.ruby-lang.org/cat.rhtml?
category_major=Library;category_minor=XML

Thanks for this tip.
Of these, http://clabs.org/clxmlserial.htm looks the most promising.

Note, btw, the home page of myxml
(http://raa.ruby-lang.org/project/myxml/\), where it states:
    (* use YAML instead, it's 100% better for small apps ;-))

This is where we start to disagree.

You asked for simple and even showed examples of what you had in
mind. I wrote code that parses it. Simple pays off at both ends.

You wanted to drop the computerese; my version did. In fact, mine
doesn't need those dashes. With a one byte change, you could also
make indenting the attributes optional. It gets better and better
for the human who needs to write it and it's still not complex on
your end.

Finally, I think it's easier to read or write than XML or YAML.

When your needs are big, of course you want to go with a proven
format. What you showed is simple though and parsed just fine with a
few lines of code. I don't see any shame in that.

James Edward Gray II

Fair point.

···

--
Posted via http://www.ruby-forum.com/\.