Simple match of items in 2 arrays

ruby-talk
1

Hi

Inexperienced ruby user question:

I have 2 large csv files:
1) Zipdata = basically a frequency table of zipcodes:
02115, 10
64108, 9
99234, 8 etc

2) Ziplookup (ziplu) = several column table of data assoc with
zipcodes:
02115, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty
55021, +55.5454, - 64,8585, Kansas City, MO, Jackson Cty

I am trying to take each entry in the zipdata and compare the zipcode
to the lookup table
When a match is found, combine the data into a new line /
multidimensional hash table eg -
02115, 10, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty

Here is what I have:
outf = File.read "ZIP_CODES.txt"
ziplu=[]
zipdata = []
result = []

outf.each {|e| ziplu.push e.chomp}

# open the csv file of zip code occurences and their frequency (zips),
and add each zipcode to an array (zipdata)
doc = File.read "zips.txt"
doc.each {|f| zipdata.push f}

#compare each line from zipsdata to the lookup file and put into new
array
result.push(zipdata.each {|a| ziplu.find_all{|x| /a/ =~ x})

I can't get the comparison to work, and don't know what format to put
the result into (hash, array of arrays, file) - from which I can
easily output the results (ie., lat long, city, state) by zipcode...

Any guidance appreciated..

CLS

2

Charles L. Snyder wrote:

I can't get the comparison to work, and don't know what format to put
the result into (hash, array of arrays, file) - from which I can
easily output the results (ie., lat long, city, state) by zipcode...

Any guidance appreciated..

Charles,

I don't konw if this is the best way of doing it, but it works at least.
btw, your file has errors. It the code supposed to deal with them?
e.g. commas in the latitude, spaces after minus.
Have you considered using a database table?

Christer

class ZipCodeReader
  ZipCode = Struct.new(:zipcode, :lat, :lng, :address, :state, :city,
:freq)
  def read(filename)
    @zipcode = {}
    IO.foreach(filename) do |line|
      arr = line.chomp.split(",")
      zipcode = ZipCode.new(arr[0], arr[1].to_f, arr[2].to_f,
arr[3].lstrip, arr[4].lstrip, arr[5].lstrip, 0)
      @zipcode[arr[0]] = zipcode
    end
  end

  def readfreq(filename)
    IO.foreach(filename) do |line|
      arr = line.chomp.split(",")
      @zipcode[arr[0]].freq = arr[1].lstrip.to_i unless
@zipcode[arr[0]].nil?
    end
  end

  def dump
    arr=@zipcode.sort
    arr.each {|x| print x.to_s + "\n"}
  end
end

zipcode=ZipCodeReader.new
zipcode.read("zipcode.txt")
zipcode.readfreq("zipdata.txt")
zipcode.dump

···

--
Posted via http://www.ruby-forum.com/\.

3

Hi,

You can use the CSV module. As Christer already pointed out, you have errors in your CSV file.

   Robin

require 'csv'
require 'yaml'

zips = {}

ziplookup = CSV.parse(File.read('ziplu'), ', ')
zipdata = CSV.parse(File.read('zipdata'), ', ')

zipdata.each do |data|
   zip_code = data.first
   h = {}
   zips[zip_code] = h
   h[:frequency] = data[1].to_i
   lookup = ziplookup.find{ |lookup| lookup.first == zip_code }
   next unless lookup
   # something like this:
   h[:lat] = lookup[1].gsub(',', '.').to_f
   h[:long] = lookup[2].gsub(',', '.').to_f
   h[:city] = lookup[3]
end

y zips

Charles L. Snyder wrote:

···

Hi

Inexperienced ruby user question:

I have 2 large csv files:
1) Zipdata = basically a frequency table of zipcodes:
02115, 10
64108, 9
99234, 8 etc

2) Ziplookup (ziplu) = several column table of data assoc with
zipcodes:
02115, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty
55021, +55.5454, - 64,8585, Kansas City, MO, Jackson Cty

I am trying to take each entry in the zipdata and compare the zipcode
to the lookup table
When a match is found, combine the data into a new line /
multidimensional hash table eg -
02115, 10, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty

Here is what I have:
outf = File.read "ZIP_CODES.txt"
ziplu=
zipdata =
result =

outf.each {|e| ziplu.push e.chomp}

# open the csv file of zip code occurences and their frequency (zips),
and add each zipcode to an array (zipdata)
doc = File.read "zips.txt"
doc.each {|f| zipdata.push f}

#compare each line from zipsdata to the lookup file and put into new
array
result.push(zipdata.each {|a| ziplu.find_all{|x| /a/ =~ x})

I can't get the comparison to work, and don't know what format to put
the result into (hash, array of arrays, file) - from which I can
easily output the results (ie., lat long, city, state) by zipcode...

Any guidance appreciated..

CLS

4

Robin Stocker wrote:

···

Hi,

You can use the CSV module. As Christer already pointed out, you have errors in your CSV file.

  Robin

require 'csv'
require 'yaml'

zips = {}

ziplookup = CSV.parse(File.read('ziplu'), ', ')
zipdata = CSV.parse(File.read('zipdata'), ', ')

zipdata.each do |data|
  zip_code = data.first
  h = {}
  zips[zip_code] = h
  h[:frequency] = data[1].to_i
  lookup = ziplookup.find{ |lookup| lookup.first == zip_code }
  next unless lookup
  # something like this:
  h[:lat] = lookup[1].gsub(',', '.').to_f
  h[:long] = lookup[2].gsub(',', '.').to_f
  h[:city] = lookup[3]
end

y zips

Charles L. Snyder wrote:

Hi

Inexperienced ruby user question:

I have 2 large csv files:
1) Zipdata = basically a frequency table of zipcodes:
02115, 10
64108, 9
99234, 8 etc

2) Ziplookup (ziplu) = several column table of data assoc with
zipcodes:
02115, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty
55021, +55.5454, - 64,8585, Kansas City, MO, Jackson Cty

I am trying to take each entry in the zipdata and compare the zipcode
to the lookup table
When a match is found, combine the data into a new line /
multidimensional hash table eg -
02115, 10, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty

Here is what I have:
outf = File.read "ZIP_CODES.txt"
ziplu=
zipdata =
result =

outf.each {|e| ziplu.push e.chomp}

# open the csv file of zip code occurences and their frequency (zips),
and add each zipcode to an array (zipdata)
doc = File.read "zips.txt"
doc.each {|f| zipdata.push f}

#compare each line from zipsdata to the lookup file and put into new
array
result.push(zipdata.each {|a| ziplu.find_all{|x| /a/ =~ x})

I can't get the comparison to work, and don't know what format to put
the result into (hash, array of arrays, file) - from which I can
easily output the results (ie., lat long, city, state) by zipcode...

Any guidance appreciated..

CLS

5

Robin Stocker wrote:

Hi,

You can use the CSV module. As Christer already pointed out, you have
errors in your CSV file.

   Robin

require 'csv'
require 'yaml'

zips = {}

ziplookup = CSV.parse(File.read('ziplu'), ', ')
zipdata = CSV.parse(File.read('zipdata'), ', ')

zipdata.each do |data|
   zip_code = data.first
   h = {}
   zips[zip_code] = h
   h[:frequency] = data[1].to_i
   lookup = ziplookup.find{ |lookup| lookup.first == zip_code }
   next unless lookup
   # something like this:
   h[:lat] = lookup[1].gsub(',', '.').to_f
   h[:long] = lookup[2].gsub(',', '.').to_f
   h[:city] = lookup[3]
end

y zips

I'd change this a bit:
- read only lookup zips into mem and process the other line by line
(saves mem)
- use a hash for lookup

# untested
require 'csv'

zips = File.open('ziplu') do |io|
  h = {}
  io.each_line do |line|
    zip, freq = CSV.parse_line
    h[zip]=freq
  end
  h
end

File.open('zipdata') do |io|
  io.each_line do |line|
    rec = CSV.parse_line line
    freq = zips[rec[0]] or next
    rec[1,0]=freq
    puts rec.join ','
  end
end

HTH

Kind regards

    robert

···

Charles L. Snyder wrote:

Hi

Inexperienced ruby user question:

I have 2 large csv files:
1) Zipdata = basically a frequency table of zipcodes:
02115, 10
64108, 9
99234, 8 etc

2) Ziplookup (ziplu) = several column table of data assoc with
zipcodes:
02115, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty
55021, +55.5454, - 64,8585, Kansas City, MO, Jackson Cty

I am trying to take each entry in the zipdata and compare the zipcode
to the lookup table
When a match is found, combine the data into a new line /
multidimensional hash table eg -
02115, 10, +43.59906, -75,99343, Hoboken, NJ, Johnson Cty

Here is what I have:
outf = File.read "ZIP_CODES.txt"
ziplu=
zipdata =
result =

outf.each {|e| ziplu.push e.chomp}

# open the csv file of zip code occurences and their frequency
(zips), and add each zipcode to an array (zipdata)
doc = File.read "zips.txt"
doc.each {|f| zipdata.push f}

#compare each line from zipsdata to the lookup file and put into new
array
result.push(zipdata.each {|a| ziplu.find_all{|x| /a/ =~ x})

I can't get the comparison to work, and don't know what format to put
the result into (hash, array of arrays, file) - from which I can
easily output the results (ie., lat long, city, state) by zipcode...

Any guidance appreciated..

CLS

6

Thanks everyone - Robert's method works for me with minor alteration.
The errors in the example were not in the actual file.

BTW:
Christer's answer gives me:
"undefined method `lstrip' for nil:NilClass (NoMethodError)"

Robin's method gives me:
private method `gsub' called for nil:NilClass (NoMethodError)

Thanks to all !

7

Charles L. Snyder wrote:

Thanks everyone - Robert's method works for me with minor alteration.
The errors in the example were not in the actual file.

BTW:
Christer's answer gives me:
"undefined method `lstrip' for nil:NilClass (NoMethodError)"

Robin's method gives me:
private method `gsub' called for nil:NilClass (NoMethodError)

Thanks to all !

CLS,

can you give me some problem input?

Christer

···

--
Posted via http://www.ruby-forum.com/\.