Extracting values from consecutive lines

Hello,

This is a basic question, as I'm very new to Ruby. I have a text file 'input.dat' containing the following data and text:

some text
some text...

angle value1 value2

156.00 3.3688E-03 1.7040E-03
157.00 3.2919E-03 1.7118E-03
158.00 3.2140E-03 1.7190E-03
159.00 3.1354E-03 1.7258E-03
160.00 3.0560E-03 1.7320E-03
161.00 2.9760E-03 1.7378E-03
162.00 2.8956E-03 1.7431E-03
163.00 2.8148E-03 1.7479E-03
164.00 2.7338E-03 1.7523E-03
165.00 2.6526E-03 1.7562E-03
166.00 2.5714E-03 1.7597E-03
167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive lines (say, from angle = 158 to 165). These values should be stored in some array or vector. How can I do that in Ruby?

Many thanks,

baptiste

# Make an array of arrays of numbers
# based on scanning for non-whitespace characters
# (all strings will show up as 0.0)
values = IO.readlines( 'input.dat' ).map{ |line|
  line.scan( /\S+/ ).map{ |str| str.to_f }
}

my_range = values.select{ |angle, _|
  (158..165).include?( angle )
}

require 'pp'
pp my_range
#=> [[158.0, 0.003214, 0.001719],
#=> [159.0, 0.0031354, 0.0017258],
#=> [160.0, 0.003056, 0.001732],
#=> [161.0, 0.002976, 0.0017378],
#=> [162.0, 0.0028956, 0.0017431],
#=> [163.0, 0.0028148, 0.0017479],
#=> [164.0, 0.0027338, 0.0017523],
#=> [165.0, 0.0026526, 0.0017562]]

···

On Oct 17, 10:03 am, baptiste Auguié <ba...@exeter.ac.uk> wrote:

> angle value1 value2

> 156.00 3.3688E-03 1.7040E-03
> 157.00 3.2919E-03 1.7118E-03
> 158.00 3.2140E-03 1.7190E-03
> 159.00 3.1354E-03 1.7258E-03
> 160.00 3.0560E-03 1.7320E-03
> 161.00 2.9760E-03 1.7378E-03
> 162.00 2.8956E-03 1.7431E-03
> 163.00 2.8148E-03 1.7479E-03
> 164.00 2.7338E-03 1.7523E-03
> 165.00 2.6526E-03 1.7562E-03
> 166.00 2.5714E-03 1.7597E-03
> 167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive
lines (say, from angle = 158 to 165). These values should be stored
in some array or vector. How can I do that in Ruby?

Many ways to approach it of course.
You could use CSV or FasterCSV since your data file is lines (rows) with columns separated by tabs or spaces.
CSV or FasterCSV might be more useful to do other things also with the data file.
but you can also just read the first 3 or 4 bytes of each line to find the lines you want.

···

On Oct 17, 2007, at 11:03 AM, baptiste Auguié wrote:

Hello,

This is a basic question, as I'm very new to Ruby. I have a text file 'input.dat' containing the following data and text:

some text
some text...

angle value1 value2

156.00 3.3688E-03 1.7040E-03
157.00 3.2919E-03 1.7118E-03
158.00 3.2140E-03 1.7190E-03
159.00 3.1354E-03 1.7258E-03
160.00 3.0560E-03 1.7320E-03
161.00 2.9760E-03 1.7378E-03
162.00 2.8956E-03 1.7431E-03
163.00 2.8148E-03 1.7479E-03
164.00 2.7338E-03 1.7523E-03
165.00 2.6526E-03 1.7562E-03
166.00 2.5714E-03 1.7597E-03
167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive lines (say, from angle = 158 to 165). These values should be stored in some array or vector. How can I do that in Ruby?

Many thanks,

baptiste

p IO.readlines( 'input.dat').grep( /^\d/ ).
  map{|s| s.strip.split.map{|x| x.to_f}}.select{|a|
    a.first.between?(158,165) }

···

On Oct 17, 11:03 am, baptiste Auguié <ba...@exeter.ac.uk> wrote:

Hello,

This is a basic question, as I'm very new to Ruby. I have a text file
'input.dat' containing the following data and text:

> some text
> some text...

> angle value1 value2

> 156.00 3.3688E-03 1.7040E-03
> 157.00 3.2919E-03 1.7118E-03
> 158.00 3.2140E-03 1.7190E-03
> 159.00 3.1354E-03 1.7258E-03
> 160.00 3.0560E-03 1.7320E-03
> 161.00 2.9760E-03 1.7378E-03
> 162.00 2.8956E-03 1.7431E-03
> 163.00 2.8148E-03 1.7479E-03
> 164.00 2.7338E-03 1.7523E-03
> 165.00 2.6526E-03 1.7562E-03
> 166.00 2.5714E-03 1.7597E-03
> 167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive
lines (say, from angle = 158 to 165). These values should be stored
in some array or vector. How can I do that in Ruby?

Many thanks,

baptiste

Hi,

angle value1 value2
156.00 3.3688E-03 1.7040E-03
157.00 3.2919E-03 1.7118E-03
158.00 3.2140E-03 1.7190E-03
159.00 3.1354E-03 1.7258E-03
160.00 3.0560E-03 1.7320E-03
161.00 2.9760E-03 1.7378E-03
162.00 2.8956E-03 1.7431E-03
163.00 2.8148E-03 1.7479E-03
164.00 2.7338E-03 1.7523E-03
165.00 2.6526E-03 1.7562E-03
166.00 2.5714E-03 1.7597E-03
167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive lines
(say, from angle = 158 to 165). These values should be stored in some array
or vector. How can I do that in Ruby?

Anybody considered ranges?

  r = 158..165
  file.each { |l|
    ang, val1, val2 = l.split.map { |x| Float x }
    r === ang and do_sth_with val1, val2
  }

  r = 158..165
  file.each { |l|
    ang, val1, val2 = l.split.map { |x| Float x }
    case ang
      when r then do_sth_with val1, val2
    end
  }

Does anybody remember flip-flops?

  file.each { |l|
    ang, val1, val2 = l.split.map { |x| Float x }
    iang = ang.floor
    if (iang==158)..(iang==165) then
      do_sth_with val1, val2
    end
  }

The floor call is not actually beautiful here but always
keep in mind equality shouldn't be tested for floats.
Flip-flops need equality.

Bertram

···

Am Donnerstag, 18. Okt 2007, 01:03:21 +0900 schrieb baptiste Auguié:

--
Bertram Scharpf
Stuttgart, Deutschland/Germany
http://www.bertram-scharpf.de

p IO.readlines( 'input.dat').grep( /^\d/ ).
  map{|s| s.strip.split.map{|x| x.to_f}}.select{|a|
    a.first.between?(158,165) }

···

On Oct 17, 11:03 am, baptiste Auguié <ba...@exeter.ac.uk> wrote:

Hello,

This is a basic question, as I'm very new to Ruby. I have a text file
'input.dat' containing the following data and text:

> some text
> some text...

> angle value1 value2

> 156.00 3.3688E-03 1.7040E-03
> 157.00 3.2919E-03 1.7118E-03
> 158.00 3.2140E-03 1.7190E-03
> 159.00 3.1354E-03 1.7258E-03
> 160.00 3.0560E-03 1.7320E-03
> 161.00 2.9760E-03 1.7378E-03
> 162.00 2.8956E-03 1.7431E-03
> 163.00 2.8148E-03 1.7479E-03
> 164.00 2.7338E-03 1.7523E-03
> 165.00 2.6526E-03 1.7562E-03
> 166.00 2.5714E-03 1.7597E-03
> 167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive
lines (say, from angle = 158 to 165). These values should be stored
in some array or vector. How can I do that in Ruby?

Many thanks,

baptiste

#!awk -f
/^158/, /^165/ { count++
  for (i=1; i<=NF; i++)
    a[count, i] = $i + 0
}
END {
  for (i=1; (i,1) in a; i++)
  { for (j=1; (i,j) in a; j++)
      printf "%f ", a[i,j]
    print
  }
}

···

On Oct 17, 11:03 am, baptiste Auguié <ba...@exeter.ac.uk> wrote:

Hello,

This is a basic question, as I'm very new to Ruby. I have a text file
'input.dat' containing the following data and text:

> some text
> some text...

> angle value1 value2

> 156.00 3.3688E-03 1.7040E-03
> 157.00 3.2919E-03 1.7118E-03
> 158.00 3.2140E-03 1.7190E-03
> 159.00 3.1354E-03 1.7258E-03
> 160.00 3.0560E-03 1.7320E-03
> 161.00 2.9760E-03 1.7378E-03
> 162.00 2.8956E-03 1.7431E-03
> 163.00 2.8148E-03 1.7479E-03
> 164.00 2.7338E-03 1.7523E-03
> 165.00 2.6526E-03 1.7562E-03
> 166.00 2.5714E-03 1.7597E-03
> 167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive
lines (say, from angle = 158 to 165). These values should be stored
in some array or vector. How can I do that in Ruby?

Many thanks,

baptiste

thanks so much!

I'll try and use this array with rgsl now...

Best regards,

baptiste

···

On 17 Oct 2007, at 17:25, Phrogz wrote:

On Oct 17, 10:03 am, baptiste Auguié <ba...@exeter.ac.uk> wrote:

angle value1 value2

156.00 3.3688E-03 1.7040E-03
157.00 3.2919E-03 1.7118E-03
158.00 3.2140E-03 1.7190E-03
159.00 3.1354E-03 1.7258E-03
160.00 3.0560E-03 1.7320E-03
161.00 2.9760E-03 1.7378E-03
162.00 2.8956E-03 1.7431E-03
163.00 2.8148E-03 1.7479E-03
164.00 2.7338E-03 1.7523E-03
165.00 2.6526E-03 1.7562E-03
166.00 2.5714E-03 1.7597E-03
167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive
lines (say, from angle = 158 to 165). These values should be stored
in some array or vector. How can I do that in Ruby?

# Make an array of arrays of numbers
# based on scanning for non-whitespace characters
# (all strings will show up as 0.0)
values = IO.readlines( 'input.dat' ).map{ |line|
  line.scan( /\S+/ ).map{ |str| str.to_f }
}

my_range = values.select{ |angle, _|
  (158..165).include?( angle )
}

require 'pp'
pp my_range
#=> [[158.0, 0.003214, 0.001719],
#=> [159.0, 0.0031354, 0.0017258],
#=> [160.0, 0.003056, 0.001732],
#=> [161.0, 0.002976, 0.0017378],
#=> [162.0, 0.0028956, 0.0017431],
#=> [163.0, 0.0028148, 0.0017479],
#=> [164.0, 0.0027338, 0.0017523],
#=> [165.0, 0.0026526, 0.0017562]]

_____________________________

Baptiste Auguié

Physics Department
University of Exeter
Stocker Road,
Exeter, Devon,
EX4 4QL, UK

Phone: +44 1392 264187

http://newton.ex.ac.uk/research/emag
http://projects.ex.ac.uk/atto
______________________________

Gavin Kistner wrote:

> 164.00 2.7338E-03 1.7523E-03
> 165.00 2.6526E-03 1.7562E-03
> 166.00 2.5714E-03 1.7597E-03
> 167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive
lines (say, from angle = 158 to 165). These values should be stored
in some array or vector. How can I do that in Ruby?

# Make an array of arrays of numbers
# based on scanning for non-whitespace characters
# (all strings will show up as 0.0)
values = IO.readlines( 'input.dat' ).map{ |line|
  line.scan( /\S+/ ).map{ |str| str.to_f }
}

my_range = values.select{ |angle, _|
  (158..165).include?( angle )
}

require 'pp'
pp my_range
#=> [[158.0, 0.003214, 0.001719],
#=> [159.0, 0.0031354, 0.0017258],
#=> [160.0, 0.003056, 0.001732],
#=> [161.0, 0.002976, 0.0017378],
#=> [162.0, 0.0028956, 0.0017431],
#=> [163.0, 0.0028148, 0.0017479],
#=> [164.0, 0.0027338, 0.0017523],
#=> [165.0, 0.0026526, 0.0017562]]

On my system, this is faster:

start = '> 163.00'
stop = '> 166.00'
results =
get_line = false

File.foreach("data.txt") do |line|
  test_field = line[0, start.length]

  if test_field == start
    get_line = true
  end

  if get_line
    results << line.split()[1..-1].map{|str| str.to_f}

    if test_field == stop
      break
    end

  end
end

···

On Oct 17, 10:03 am, baptiste Augui� <ba...@exeter.ac.uk> wrote:

--
Posted via http://www.ruby-forum.com/\.

William James wrote:

p IO.readlines( 'input.dat').grep( /^\d/ ).
  map{|s| s.strip.split.map{|x| x.to_f}}.select{|a|
    a.first.between?(158,165) }

Better. Still slower.

···

--
Posted via http://www.ruby-forum.com/\.

William James wrote:

···

On Oct 17, 11:03 am, baptiste Auguié <ba...@exeter.ac.uk> wrote:
> Hello,
>
> This is a basic question, as I'm very new to Ruby. I have a text file
> 'input.dat' containing the following data and text:
>
> > some text
> > some text...
>
> > angle value1 value2
>
> > 156.00 3.3688E-03 1.7040E-03
> > 157.00 3.2919E-03 1.7118E-03
> > 158.00 3.2140E-03 1.7190E-03
> > 159.00 3.1354E-03 1.7258E-03
> > 160.00 3.0560E-03 1.7320E-03
> > 161.00 2.9760E-03 1.7378E-03
> > 162.00 2.8956E-03 1.7431E-03
> > 163.00 2.8148E-03 1.7479E-03
> > 164.00 2.7338E-03 1.7523E-03
> > 165.00 2.6526E-03 1.7562E-03
> > 166.00 2.5714E-03 1.7597E-03
> > 167.00 2.4902E-03 1.7628E-03
>
> I would like to extract the 3 columns of data for several consecutive
> lines (say, from angle = 158 to 165). These values should be stored
> in some array or vector. How can I do that in Ruby?
>
> Many thanks,
>
> baptiste

p IO.readlines( 'input.dat').grep( /^\d/ ).
  map{|s| s.strip.split.map{|x| x.to_f}}.select{|a|
    a.first.between?(158,165) }

Please pardon my perverse prolixity. I threw in a "strip" even though
my subconscious told me it was superfluous.

p IO.readlines( 'input.dat').grep( /^\d/ ).
  map{|s| s.split.map{|x| x.to_f}}.select{|a|
    a.first.between?(158,165) }

Hi --

angle value1 value2

156.00 3.3688E-03 1.7040E-03
157.00 3.2919E-03 1.7118E-03
158.00 3.2140E-03 1.7190E-03
159.00 3.1354E-03 1.7258E-03
160.00 3.0560E-03 1.7320E-03
161.00 2.9760E-03 1.7378E-03
162.00 2.8956E-03 1.7431E-03
163.00 2.8148E-03 1.7479E-03
164.00 2.7338E-03 1.7523E-03
165.00 2.6526E-03 1.7562E-03
166.00 2.5714E-03 1.7597E-03
167.00 2.4902E-03 1.7628E-03

I would like to extract the 3 columns of data for several consecutive
lines (say, from angle = 158 to 165). These values should be stored
in some array or vector. How can I do that in Ruby?

# Make an array of arrays of numbers
# based on scanning for non-whitespace characters
# (all strings will show up as 0.0)
values = IO.readlines( 'input.dat' ).map{ |line|
line.scan( /\S+/ ).map{ |str| str.to_f }
}

Or:

   require 'scanf'
   values = IO.readlines('input.dat').map {|line| line.scanf("%f%f%f") }

(Assuming the > at the beginning isn't really part of it -- otherwise
">%f%f%f".)

I'm not entering the speed contest :slight_smile: but scanf might be nice for the
conversions.

David

···

On Thu, 18 Oct 2007, Phrogz wrote:

On Oct 17, 10:03 am, baptiste Auguié <ba...@exeter.ac.uk> wrote:

--
Upcoming training from Ruby Power and Light, LLC:
   * Intro to Ruby on Rails, Edison, NJ, October 23-26
   * Advancing with Rails, Edison, NJ, November 6-9
Both taught by David A. Black.
See http://www.rubypal.com for more info!

William James wrote:

#!awk -f
/^158/, /^165/ { count++
  for (i=1; i<=NF; i++)
    a[count, i] = $i + 0
}
END {
  for (i=1; (i,1) in a; i++)
  { for (j=1; (i,j) in a; j++)
      printf "%f ", a[i,j]
    print
  }
}

baptiste Auguié wrote:

How can I do that in Ruby?

If that wasn't clear enough for you, here's another clue: look at the
title of the group you posted to. Does it say 'awk' anywhere?

···

--
Posted via http://www.ruby-forum.com/\.

# William James wrote:
# > p IO.readlines( 'input.dat').grep( /^\d/ ).
# > map{|s| s.strip.split.map{|x| x.to_f}}.select{|a|
# > a.first.between?(158,165) }

···

From: 7stud -- [mailto:bbxx789_05ss@yahoo.com]
#
# Better. Still slower.

how about this,

a=[]
ARGF.each do |line|
  if line =~ /158/ .. line =~ /165/
    a << line.split.map{|x|x.to_f}
  end
end

'just joining the fun in ruby :slight_smile:
kind regards -botp

William James wrote:

William James wrote:

>
> > 166.00 2.5714E-03 1.7597E-03
p IO.readlines( 'input.dat').grep( /^\d/ ).
  map{|s| s.strip.split.map{|x| x.to_f}}.select{|a|
    a.first.between?(158,165) }

Please pardon my perverse prolixity. I threw in a "strip" even though
my subconscious told me it was superfluous.

p IO.readlines( 'input.dat').grep( /^\d/ ).
  map{|s| s.split.map{|x| x.to_f}}.select{|a|
    a.first.between?(158,165) }

I took out your superfluous grep() too, but it doesn't help your cause
enough.

Peña, Botp wrote:

how about this,

a=
ARGF.each do |line|
  if line =~ /158/ .. line =~ /165/
    a << line.split.map{|x|x.to_f}
  end
end

'just joining the fun in ruby :slight_smile:
kind regards -botp

A winner by a nose. :slight_smile:

···

--
Posted via http://www.ruby-forum.com/\.

Gavin Kistner wrote:
>> > 164.00 2.7338E-03 1.7523E-03
>> > 165.00 2.6526E-03 1.7562E-03
>> > 166.00 2.5714E-03 1.7597E-03
>> > 167.00 2.4902E-03 1.7628E-03

>> I would like to extract the 3 columns of data for several consecutive
>> lines (say, from angle = 158 to 165). These values should be stored
>> in some array or vector. How can I do that in Ruby?

> # Make an array of arrays of numbers
> # based on scanning for non-whitespace characters
> # (all strings will show up as 0.0)
> values = IO.readlines( 'input.dat' ).map{ |line|
> line.scan( /\S+/ ).map{ |str| str.to_f }
> }

> my_range = values.select{ |angle, _|
> (158..165).include?( angle )
> }

> require 'pp'
> pp my_range
> #=> [[158.0, 0.003214, 0.001719],
> #=> [159.0, 0.0031354, 0.0017258],
> #=> [160.0, 0.003056, 0.001732],
> #=> [161.0, 0.002976, 0.0017378],
> #=> [162.0, 0.0028956, 0.0017431],
> #=> [163.0, 0.0028148, 0.0017479],
> #=> [164.0, 0.0027338, 0.0017523],
> #=> [165.0, 0.0026526, 0.0017562]]

On my system, this is faster:

start = '> 163.00'

If you can't guess that "> " isn't actually part of the data,
you could have deduced it from the fact that the o.p. gave
Gavin's program the seal of approval. Gavin's program won't
work if the lines start with "> ". Your program won't work
since the lines don't start with "> ".

stop = '> 166.00'
results =
get_line = false

File.foreach("data.txt") do |line|
  test_field = line[0, start.length]

  if test_field == start
    get_line = true
  end

  if get_line
    results << line.split()[1..-1].map{|str| str.to_f}

    if test_field == stop
      break
    end

  end
end

Have you thought about doing all of your programming in COBOL?

···

On Oct 17, 12:32 pm, 7stud -- <bbxx789_0...@yahoo.com> wrote:

> On Oct 17, 10:03 am, baptiste Augui? <ba...@exeter.ac.uk> wrote: