Remote require

I know there are other solutions to this out there. Just wanted to
share my work/play at it. I was trying to make it reletvely secure
without being too much a pain to use. This is was I arrived at.

Usage looks like this:

  remote_source 'fruitapp',
'http://roll.rubyforge.org/source/fruitapp/lib/fruitapp/1.0.0/'
  remote_dependency '9de3cd6daece5f07aa3ddb0319a21415', 'tryme.rb'

  require 'fruitapp/tryme.rb'

The MD5 checksum of course is there to help ensure you get what you
expect. Of course that's the most time consuming part becasue you have
to keep those upto date with new versions.

Anyway, just thought I'd share it just in case it interested anyone.
Here's the code:

# --- remote_require.rb

require 'open-uri'
require 'digest/md5'
require 'fileutils'

REMOTE_CACHE = File.expand_path( '~/.lib/site_ruby/1.8/' )
FileUtils.mkdir_p REMOTE_CACHE unless File.directory? REMOTE_CACHE
$:.unshift REMOTE_CACHE

$current_source = nil
$current_source_stack = []
$remote_source = nil
$remote_dependency = {}

module Kernel

  class ChecksumError < LoadError ; end

  def remote_source( ref, uri )
    $remote_source = { :ref=>ref.chomp('/'), :uri=>uri.chomp('/') }
  end

  def remote_dependency( md5sum, file )
    raise "dependency without source" unless $remote_source
    $remote_dependency[ File.join( $remote_source[:ref], file ) ] = {
      :ref => $remote_source[:ref],
      :file => file,
      :uri => $remote_source[:uri],
      :md5 => md5sum
    }
  end

  req = method(:require)

  define_method :require do |fname|
    if r = $remote_dependency[fname]
      $current_source_stack << $current_source
      $current_source = r
    end
    begin
      req.call fname
    rescue LoadError => e
      # Bit of a shortcoming here since it's not very efficient to
      # be searching an remote location for multiple matches.
      # .so suffix must be specified explicity on the remote end.
      fname = fname + '.rb' unless fname =~ /\.rb$/ or fname =~ /\.so$/
      raise e unless $current_source
      s = $current_source
      url = fname.sub( /^#{s[:ref]}/, s[:uri] )
      $stderr << "remote require -- " + url if $DEBUG
      rf = URI.parse( url ).read
      if r
        unless Digest::MD5.new( rf ).hexdigest == s[:md5] # TODO
checksum
          raise ChecksumError.new(e)
        end
      end
      newfile = File.join( REMOTE_CACHE, fname )
      newdir = File.dirname( newfile )
      FileUtils.mkdir_p newdir
      File.open( newfile, 'w+' ) do |f|
        f << rf
      end
      retry
    ensure
      $current_source = $current_source_stack.pop if r
    end
  end

end

I think this would make a really nice gem. I love the idea! Maybe a future revision could take a regular md5sum output file?
-Mat

···

On Aug 3, 2006, at 1:05 PM, transfire@gmail.com wrote:

I know there are other solutions to this out there. Just wanted to
share my work/play at it. I was trying to make it reletvely secure
without being too much a pain to use. This is was I arrived at.

Usage looks like this:

  remote_source 'fruitapp',
'http://roll.rubyforge.org/source/fruitapp/lib/fruitapp/1.0.0/&#39;
  remote_dependency '9de3cd6daece5f07aa3ddb0319a21415', 'tryme.rb'

  require 'fruitapp/tryme.rb'

The MD5 checksum of course is there to help ensure you get what you
expect. Of course that's the most time consuming part becasue you have
to keep those upto date with new versions.

Anyway, just thought I'd share it just in case it interested anyone.
Here's the code:
[snip: source sode]

Mat Schaffer wrote:

> I know there are other solutions to this out there. Just wanted to
> share my work/play at it. I was trying to make it reletvely secure
> without being too much a pain to use. This is was I arrived at.
>
> Usage looks like this:
>
> remote_source 'fruitapp',
> 'http://roll.rubyforge.org/source/fruitapp/lib/fruitapp/1.0.0/&#39;
> remote_dependency '9de3cd6daece5f07aa3ddb0319a21415', 'tryme.rb'
>
> require 'fruitapp/tryme.rb'
>
> The MD5 checksum of course is there to help ensure you get what you
> expect. Of course that's the most time consuming part becasue you have
> to keep those upto date with new versions.
>
> Anyway, just thought I'd share it just in case it interested anyone.
> Here's the code:
> [snip: source sode]

I think this would make a really nice gem. I love the idea! Maybe a
future revision could take a regular md5sum output file?

I'll work on that. Also, someone else wrote me and suggested sha256
instead of md5 for greater security -- good idea. I'm not sure how to
generate the equivalent sha256 output file on my ubuntu box though, all
I see is sha1sum.

T.

···

On Aug 3, 2006, at 1:05 PM, transfire@gmail.com wrote:

-Mat

I'd imagine your best bet here would be to use RSA or something
similar for signing. That way you'd just have to include a public key
in your remote_source definition and have the author of the module
provide signatures for each file. You could have different versions
with no troubles, provided they were appropriately signed.

Sam

···

On 8/6/06, Trans <transfire@gmail.com> wrote:

Mat Schaffer wrote:
> On Aug 3, 2006, at 1:05 PM, transfire@gmail.com wrote:
> I think this would make a really nice gem. I love the idea! Maybe a
> future revision could take a regular md5sum output file?

I'll work on that. Also, someone else wrote me and suggested sha256
instead of md5 for greater security -- good idea. I'm not sure how to
generate the equivalent sha256 output file on my ubuntu box though, all
I see is sha1sum.

Sam Gentle wrote:

> Mat Schaffer wrote:
> > I think this would make a really nice gem. I love the idea! Maybe a
> > future revision could take a regular md5sum output file?
>
> I'll work on that. Also, someone else wrote me and suggested sha256
> instead of md5 for greater security -- good idea. I'm not sure how to
> generate the equivalent sha256 output file on my ubuntu box though, all
> I see is sha1sum.

I'd imagine your best bet here would be to use RSA or something
similar for signing. That way you'd just have to include a public key
in your remote_source definition and have the author of the module
provide signatures for each file. You could have different versions
with no troubles, provided they were appropriately signed.

I'm not as sure, albeit I'm not experienced enough to to certain. Could
you eleborate on how it would work? When you say signitures for each
file what are we takling about exactly?

Thanks,
T.

···

On 8/6/06, Trans <transfire@gmail.com> wrote:
> > On Aug 3, 2006, at 1:05 PM, transfire@gmail.com wrote:

Sam

Well, essentially the way it works is the author of a module would
generate a public/private key pair, distribute the public key, and use
the private key to encrypt hashes of their modules (ie. sign them).
Anyone with a copy of the public key could then verify that those
modules were signed by the same private key (made by the same person,
essentially...)

The idea being that for each file the author also makes available a
signature file (or even includes them in the script itself, although
that'd be a bit more complex), and the script uses those signatures to
verify.

If you have access to a copy of openssl, you can play around with this:

$ openssl genrsa > key.priv
$ openssl rsa -pubout <key.priv >key.pub
writing RSA key
$ echo "hello world" > myfile.txt
$ openssl dgst -sha1 -sign key.priv <myfile.txt >myfile.sig
$ openssl dgst -sha1 -verify key.pub -signature myfile.sig <myfile.txt
Verified OK

There's one major problem, however: it won't protect against the
author(s) going crazy and adding backdoors after you've trusted them,
or having their private keys stolen by sneaky attackers. It does,
however, mean that you don't have to worry about updating for new
versions.

Sam

···

On 8/7/06, Trans <transfire@gmail.com> wrote:

Sam Gentle wrote:
> On 8/6/06, Trans <transfire@gmail.com> wrote:
> > Mat Schaffer wrote:
> > > On Aug 3, 2006, at 1:05 PM, transfire@gmail.com wrote:
> > > I think this would make a really nice gem. I love the idea! Maybe a
> > > future revision could take a regular md5sum output file?
> >
> > I'll work on that. Also, someone else wrote me and suggested sha256
> > instead of md5 for greater security -- good idea. I'm not sure how to
> > generate the equivalent sha256 output file on my ubuntu box though, all
> > I see is sha1sum.
>
> I'd imagine your best bet here would be to use RSA or something
> similar for signing. That way you'd just have to include a public key
> in your remote_source definition and have the author of the module
> provide signatures for each file. You could have different versions
> with no troubles, provided they were appropriately signed.

I'm not as sure, albeit I'm not experienced enough to to certain. Could
you eleborate on how it would work? When you say signitures for each
file what are we takling about exactly?