Ruby as a domain specific language?

ruby-talk
1

Hi

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I was
thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What I'm wondering is, are there any good resources on using Ruby as a
domain-specific language? Is there any documentation on the different
way in which Ruby can be extended to support domain-specific
constructs?

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like Ruby
might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is particularly
dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept of
translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would have
translations etc. associated with them.

Thanks in advance for your help,
Calum

2

* cnmaclean@hotmail.com <cnmaclean@hotmail.com> [2005-02-16 03:14:53 +0900]:

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I was
thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like Ruby
might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is particularly
dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept of
translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would have
translations etc. associated with them.

Do you mean someting like:

define_data {
  data_item { |t|
    t.name "My Data"
    t.coding "Language"
    t.length 2
  }

  files "." {
    decode_with "My Data"
  }
}

···

--
Jim Freeze
Code Red. Code Ruby

3

cnmaclean@hotmail.com wrote:

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I

was

thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What I'm wondering is, are there any good resources on using Ruby as

a

domain-specific language? Is there any documentation on the

different

way in which Ruby can be extended to support domain-specific
constructs?

There was once a post by Phil Tomson about a Quantum Design Language
which might be close to what you're after:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/99734

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like

Ruby

might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is

particularly

dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept

of

translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would

have

translations etc. associated with them.

The other option is to go with yaml for the data file and process it
with Ruby. It might not be the best thing for human editing (as it's
whitespace significant) but it should be very easy to define
hierarchical constructs to be later processed by Ruby.

HTH,
Assaph

4

In article <1108491215.889243.288190@o13g2000cwo.googlegroups.com>,

Hi

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I was
thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What I'm wondering is, are there any good resources on using Ruby as a
domain-specific language? Is there any documentation on the different
way in which Ruby can be extended to support domain-specific
constructs?

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like Ruby
might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is particularly
dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept of
translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would have
translations etc. associated with them.

Thanks in advance for your help,
Calum

I would tend to say it more like: Ruby for creating a domain specific
language (DSL).

Ruby excells in creating DSLs primarily because of it's code blocks. I've
created a couple of DSLs with Ruby:

QDL (Quantum Description Language) - a language for describing quantum
logic circuits. See:
http://nexp.cs.pdx.edu/~qc/cgi-bin/view/QC/QuantumDesignLanguage

RHDL (Ruby Hardware Description Langauge) - a language for describing
hardware (and simulating concurrency). See:
http://www.aracnet.com/~ptkwt/ruby_stuff/RHDL/

Code blocks allow you to create a very natural-looking DSL directly
in Ruby without need of a special parser.

There's also Rake which could definately fall into the catagory of a DSL.

Phil

···

<cnmaclean@hotmail.com> wrote:

5

In our current (proprietary) language, it's something like this:

Define File 1234
  Name = "My File"
  Parent = 5678
  DataItem = "DataItem 1", Decimal, 10 bytes
  DataItem = "DataItem 2", Hex, 10 bytes

This format allows you to do simple things fairly easily, but it might
be impossible to do more complicated things.

Currently, the format is almost entirely declarative. You can't have
any logic (e.g. determining the second data item based on the contents
of the first), and there's only a very primitive and limited form of
variables. Doing some new means extending the language definition.

I'd also like (if possible) to allow declaring files hierarchically, so
that I can define File 1234 in the scope of 5678.

As I said, I'm also just looking for general articles/mailings etc. on
using Ruby as a domain-specific language - examples, showing what's
possible/allowed etc.

Thanks,
Calum

6

Phil Tomson wrote:

RHDL (Ruby Hardware Description Langauge) - a language for describing hardware (and simulating concurrency). See:
Aracnet; Internet Services. Hotel bookings online

I add mine:

Ruby for logic formulas:

http://www.ntecs.de/viewcvs/viewcvs/ForSys/

I actually split the DSL into two parts. The "clean"-part, where I use "regular" Ruby constructs and the other part where I make heavy use of const_missing and method_missing, this generates objects of classes defined in the "clean"-part.

Clean:

   Impl[
     ForAll[ Var[:x], Pred[:P, Var[:x], Var[:y]] ],
     Exists[ Var[:y], Pred[:Q, Var[:y]] & ForAll[ Var[:z], Pred[:R] ] ]
   ]

"Un-Clean":

   V::x { P(:x, :y) } >= E::y { Q(:y) & V::z { R() } }

Both is actually valid Ruby!

Regards,

   Michael

7

* cnmaclean@hotmail.com <cnmaclean@hotmail.com> [2005-02-16 05:49:56 +0900]:

In our current (proprietary) language, it's something like this:

Define File 1234
  Name = "My File"
  Parent = 5678
  DataItem = "DataItem 1", Decimal, 10 bytes
  DataItem = "DataItem 2", Hex, 10 bytes

Currently, the format is almost entirely declarative. You can't have
any logic (e.g. determining the second data item based on the contents
of the first), and there's only a very primitive and limited form of
variables. Doing some new means extending the language definition.

I'd also like (if possible) to allow declaring files hierarchically, so
that I can define File 1234 in the scope of 5678.

As I said, I'm also just looking for general articles/mailings etc. on
using Ruby as a domain-specific language - examples, showing what's
possible/allowed etc.

Sorry, but I don't know of any write-ups on this. However, it
was a hot topic at the last Ruby Conf, and I bet we see at least
one presentation on this at the next Ruby Conf.

Maybe I can help with some examples, or better yet, write
your example in a Ruby DSL form.

  define_file 1234 do |f|
    f.name "My File"
    f.parent 5678
    f.data_item "DataItem 1", Decimal, 10.bytes
    f.data_item "DataItem 2", Hex, 10.bytes
  end

Here, we have defined #name, #parent and #data_item as methods.
As these are constructed, references can be constructed to give
you the 'smarts' that you are looking for.
With these smarts, we can to do programmatical things like iteration:

  define_file 5678 do |f|
    f.name "Parent File"
    f.parent 1
    f.data_item :inherit_from_parent
    f.children { |child| child.do_whatever }
  end

···

--
Jim Freeze
Code Red. Code Ruby

8

cnmaclean@hotmail.com wrote:

In our current (proprietary) language, it's something like this:

Define File 1234
        Name = "My File"
        Parent = 5678
        DataItem = "DataItem 1", Decimal, 10 bytes
        DataItem = "DataItem 2", Hex, 10 bytes

Ruby would let you come fairly close:

···

#-----------------------------------------------------------------------#

class DataFile
  def initialize(file_id, &block)
    @file_id = file_id
    $registry[file_id] = self
    @data =
    instance_eval(&block)
  end
  
  def name(name)
    @name = name
  end

  def parent(pid)
    @parent = pid
  end

  def data_item(*args)
    p args
    @data << args
  end
end

def define(klass, *args, &block)
  klass.new(*args, &block)
end

$registry = {}

define(DataFile, 1234) {
  name "My File"
  parent 5678
  data_item "DataItem 1", :Decimal, "10 bytes"
  data_item "DataItem 2", :Hex, "10 bytes"
}

p $registry

#-----------------------------------------------------------------------#

If you don't like the string quoting in "10 bytes" you can do

class Fixnum
  def bytes
    self
  end
end

and then write 10.bytes

martin

9

* cnmaclean@hotmail.com <cnmaclean@hotmail.com> [2005-02-16 05:49:56 +0900]:

In our current (proprietary) language, it's something like this:

Define File 1234
Name = "My File"
Parent = 5678
DataItem = "DataItem 1", Decimal, 10 bytes
DataItem = "DataItem 2", Hex, 10 bytes
   

Actually, quick and dirty domain language in Ruby is very easy. E.g., you can write in less than a day some Ruby that will take this:

class File1234Parser < FileParser::Base
  define_file 1234,
    :name => 'My File',
    :Parent => 5678 {
      decimal "DataItem 1", 10.bytes
      hex "DataItem 2", 10.bytes
  }
end

File1234Parser.new(open('~/files/foo.1234')).parse
... or whatever...

This would work by invoking a method FileParser::Base.define_file, that would take whatever arguments you give it, make some Ruby code out of it and then

self.module_eval <<-EOL
  def parse
    <code generated from arguments>
  end
  ...
EOL

For things like parsing and/or generating a flat, fixed-width file, this can be done in a day (I did it). I also used the same approach once to generate a test oracle for a fairly complicated billing application. The "price definition files" there looks like:

class GprsPromotion << StandardPricePlan
  market_rates(:DEFAULT_PREPAID_RATES)
  rate :voice => flat_rate(1.00.ntd.per_minute, :rounding_factor => 15.seconds, :round => :up)
  rate :sms => event_rate(2.00.ntd.per_event)
  # etc
end

You can show it to someone who doesn't know Ruby and s/he will understand it.

It is also very easy to add special behaviors, because you can

define_file :Foo
alias :generic_parse :parse
def parse
  generic_parse
  do.some { ruby.wizardry } if necessary?
end

The dark side is that it all has to be valid Ruby syntax, and that is a bit more commas and braces than in the perfect world.

···

--
Best regards,

Alexey Verkhovsky

Ruby Forum: http://ruby-forum.org (moderator)
RForum: http://rforum.andreas-s.net (co-author)
Instiki: http://instiki.org (maintainer)