Help parsing simple structured data

ruby-talk
1

Nuby Ruby wrote:

Dan,

That works perfectly, thank you a lot! This type of data I end up dealing
with day-in and day-out, and this will help a lot.

Craig

Nuby Ruby wrote:

I'm rather new to Ruby and am trying to figure out how to parse some
structured data.. This is pretty much the most common problem I run into
and it'd be *really* helpful to see an example to wrap my head around

blocks

& hashes

The data I'm working with is the output of `df -h` which looks like this

:

Filesystem Size Used Avail Use% Mounted on
/dev/md0 9.9G 1.6G 7.8G 17% /
/dev/md1 60G 154M 57G 1% /data
/dev/sda1 99M 12M 83M 13% /boot

In the end what I'd like to end up with is to be able to say print
hash[md0(size)] and have it
return 9.9G or hash[sda1(Mount)] and have it show the mountpoint of

/boot

If anyone could help I'd really appreciate it.

Craig

Try this:

rows = `df -h`.split("\n")[1..-1]
hash = Hash.new {|hash, key| hash[key] = {}}
rows.each do |row|
   data = row.split /\s+/
   device = data[0].sub(%r|^/dev/|, "")
   %w{size used avail use mount}.each_with_index do |field, column|
     hash[device][field] = data[column+1]
   end
end

After running it, you should be able to find your data in
hash["md0"]["size"] or hash["sda1"]["mount"].

Does this work for you?

Dan

Craig,

By the way, I just realized that my script assumes that the mount point has no spaces. Nothing horrible will happen if it does, but hash[device]["mount"] will only contain the first word, like "/d/My" of "/d/My Documents". If you want to fix that, you could replace a few of the lines in the above script with these:

>> %w{size used avail use}.each_with_index do |field, column|
>> hash[device][field] = data[column+1]
>> end
>> hash[device][mount] = data[5..-1].join(" ")

Have fun,
Dan

···

On 8/28/07, Dan Zwell <dzwell@gmail.com> wrote:

2

In my situation it'll always be safe to assume there are no mount points,
but I'll grab your code anyways in case I ever run it on a windows or mac
box.

BTW, here's the code I'm working on, thanks for the tidbit, this little bit
of processing is the bit of code I use most in my daily life, and the
hardest for me to wrap my head around when I play with new langauges.

#!/usr/bin/env ruby

require 'rubygems'
require 'getoptlong'
require 'net/ssh'

opts = GetoptLong.new(
    [ "--host", "-h", GetoptLong::REQUIRED_ARGUMENT]
  )

hostname = nil
opts.each do |opt, arg|
  case opt
  when '--host'
    hostname = arg
  end
end

if hostname == nil
  puts "Missing --host argument"
  puts "usage: `ssh-test.rb --host hostname"
  exit 0
end

Net::SSH.start( hostname,
                :password=>'somepassword',
                :port=>22,
                :username=>'rubyuser'
                ) do |session|
                  shell = session.shell.open
                  shell.df "-h"
                  sleep 0.5

                  disks = shell.stdout.split("\n")[1..-1] while shell.stdout
?

                  hash = Hash.new {|hash, key| hash[key] = {}}
                  disks.each do |disk|
                    data = disk.split /\s+/
                    device = data[0].sub(%r|^/dev/|, "")
                    %w{size used avail use mount}.each_with_index do

field, column|

                      hash[device][field] = data[column+1]
                    end
                  end

                  print hash["md0"]["mount"]
                  puts "\n"
end

···

On 8/28/07, Dan Zwell <dzwell@gmail.com> wrote:

Nuby Ruby wrote:
> Dan,
>
> That works perfectly, thank you a lot! This type of data I end up
dealing
> with day-in and day-out, and this will help a lot.
>
> Craig
>
> On 8/28/07, Dan Zwell < dzwell@gmail.com> wrote:
>> Nuby Ruby wrote:
>>> I'm rather new to Ruby and am trying to figure out how to parse some
>>> structured data.. This is pretty much the most common problem I run
into
>>> and it'd be *really* helpful to see an example to wrap my head around
>> blocks
>>> & hashes
>>>
>>> The data I'm working with is the output of `df -h` which looks like
this
>> :
>>>
>>> Filesystem Size Used Avail Use% Mounted on
>>> /dev/md0 9.9G 1.6G 7.8G 17% /
>>> /dev/md1 60G 154M 57G 1% /data
>>> /dev/sda1 99M 12M 83M 13% /boot
>>>
>>> In the end what I'd like to end up with is to be able to say print
>>> hash[md0(size)] and have it
>>> return 9.9G or hash[sda1(Mount)] and have it show the mountpoint of
>> /boot
>>> If anyone could help I'd really appreciate it.
>>>
>>> Craig
>>>
>> Try this:
>>
>> rows = `df -h`.split("\n")[1..-1]
>> hash = Hash.new {|hash, key| hash[key] = {}}
>> rows.each do |row|
>> data = row.split /\s+/
>> device = data[0].sub(%r|^/dev/|, "")
>> %w{size used avail use mount}.each_with_index do |field, column|
>> hash[device][field] = data[column+1]
>> end
>> end
>>
>> After running it, you should be able to find your data in
>> hash["md0"]["size"] or hash["sda1"]["mount"].
>>
>> Does this work for you?
>>
>> Dan
>>
>>
>
Craig,

By the way, I just realized that my script assumes that the mount point
has no spaces. Nothing horrible will happen if it does, but
hash[device]["mount"] will only contain the first word, like "/d/My" of
"/d/My Documents". If you want to fix that, you could replace a few of
the lines in the above script with these:

>> %w{size used avail use}.each_with_index do |field, column|
>> hash[device][field] = data[column+1]
>> end
>> hash[device][mount] = data[5..-1].join(" ")

Have fun,
Dan

3

# of processing is the bit of code I use most in my daily life, and the
# hardest for me to wrap my head around when I play with new langauges.

you sound like an admin or you do admin tasks daily.

just want to tell you that there's a gem called ruport used for reporting task. ..just in case it may help you.

this is just a simple example. i snipped the initial echos fr irb. the last outputs will give you an idea...

so,

···

From: Nuby Ruby [mailto:nubyruby@gmail.com]

require 'rubygems'
require 'ruport'

data =`df -h`.split "\n"
header=data.shift.split " ",6
data = data.map{|line| line.split}
report = Ruport::Data::Table.new :data => data, :column_names => header

puts report

+-------------------------------------------------------------------------------+

Filesystem | Size | Used | Avail | Use% | Mounted on |

+-------------------------------------------------------------------------------+

/dev/hda1 | 3.8G | 3.1G | 566M | 85% | / |
varrun | 125M | 68K | 125M | 1% | /var/run |
varlock | 125M | 4.0K | 125M | 1% | /var/lock |
udev | 125M | 76K | 125M | 1% | /dev |
devshm | 125M | 0 | 125M | 0% | /dev/shm |
lrm | 125M | 18M | 107M | 15% | /lib/modules/2.6.15-28-686/volatile |
/dev/hdb1 | 19G | 6.9G | 13G | 36% | /disk2 |
/dev/hdd1 | 2.0G | 731M | 1.3G | 37% | /disk3-1 |
/dev/hdd2 | 2.1G | 65M | 1.9G | 4% | /disk3-2 |

+-------------------------------------------------------------------------------+
=> nil

puts report.column("Filesystem")

/dev/hda1
varrun
varlock
udev
devshm
lrm
/dev/hdb1
/dev/hdd1
/dev/hdd2
=> nil

puts report.column("Size")

3.8G
125M
125M
125M
125M
125M
19G
2.0G
2.1G
=> nil

there are a lot of fxns in ruport that allow you to sum on columns, swap them, remove/add columns, drill down on records, etc...

many thanks to GBrown and MMilner, et al for ruport.
kind regards -botp

4

just want to tell you that there's a gem called ruport used for reporting task. ..just in case it may help you.

+1 for ruport, it makes these things very easy to achieve, with less
code.

FWIW, you'll get extra features at no cost, like:

File.open('report.pdf','w') do |file|
  file << table.to_pdf
end

cheers

Thibaut

5

Oh man this is awesome. I love ruby, I spent the last hour playing with
ruport, and am in love with it's simplicity.. The reason I'm learning Ruby
is to make it easier for me to get off my butt and write reports.

···

On 8/29/07, Peña, Botp <botp@delmonte-phil.com> wrote:

From: Nuby Ruby [mailto:nubyruby@gmail.com]
# of processing is the bit of code I use most in my daily life, and the
# hardest for me to wrap my head around when I play with new langauges.

you sound like an admin or you do admin tasks daily.

just want to tell you that there's a gem called ruport used for reporting
task. ..just in case it may help you.

this is just a simple example. i snipped the initial echos fr irb. the
last outputs will give you an idea...

so,

>require 'rubygems'
>require 'ruport'

>data =`df -h`.split "\n"
>header=data.shift.split " ",6
>data = data.map{|line| line.split}
>report = Ruport::Data::Table.new :data => data, :column_names => header

>puts report

+-------------------------------------------------------------------------------+
> Filesystem | Size | Used | Avail | Use% | Mounted
on |

+-------------------------------------------------------------------------------+
> /dev/hda1 | 3.8G | 3.1G | 566M | 85% |
/ |
> varrun | 125M | 68K | 125M | 1% |
/var/run |
> varlock | 125M | 4.0K | 125M | 1% |
/var/lock |
> udev | 125M | 76K | 125M | 1% |
/dev |
> devshm | 125M | 0 | 125M | 0% |
/dev/shm |
> lrm | 125M | 18M | 107M | 15% |
/lib/modules/2.6.15-28-686/volatile |
> /dev/hdb1 | 19G | 6.9G | 13G | 36% |
/disk2 |
> /dev/hdd1 | 2.0G | 731M | 1.3G | 37% |
/disk3-1 |
> /dev/hdd2 | 2.1G | 65M | 1.9G | 4% |
/disk3-2 |

+-------------------------------------------------------------------------------+
=> nil

> puts report.column("Filesystem")
/dev/hda1
varrun
varlock
udev
devshm
lrm
/dev/hdb1
/dev/hdd1
/dev/hdd2
=> nil

> puts report.column("Size")
3.8G
125M
125M
125M
125M
125M
19G
2.0G
2.1G
=> nil

there are a lot of fxns in ruport that allow you to sum on columns, swap
them, remove/add columns, drill down on records, etc...

many thanks to GBrown and MMilner, et al for ruport.
kind regards -botp

6

Or as of today:

table.to_pdf(:file => "report.pdf)

···

On Aug 29, 7:49 am, Thibaut Barrère <thibaut.barr...@gmail.com> wrote:

> just want to tell you that there's a gem calledruportused for reporting task. ..just in case it may help you.

+1 forruport, it makes these things very easy to achieve, with less
code.

FWIW, you'll get extra features at no cost, like:

File.open('report.pdf','w') do |file|
  file << table.to_pdf
end