Opening a stream to multiple (possibly zipped) files

ruby-talk
1

Hello,

I am interested in opening a single data stream to multiple (possibly
gzipped) files, and was wondering if you could do that elegantly in
Ruby. I have some dodgy old Perl code that works, but it isn't really
elegant/robust/portable:

http://pastie.org/963613

Cheers,

Martin

···

--
Posted via http://www.ruby-forum.com/.

2

Hi

Hello,

I am interested in opening a single data stream to multiple (possibly
gzipped) files, and was wondering if you could do that elegantly in
Ruby. I have some dodgy old Perl code that works, but it isn't really
elegant/robust/portable:

http://pastie.org/963613

There are bindings in ruby for zlib:
http://ruby-doc.org/core/classes/Zlib/GzipReader.html
And facets has a wrapper around it:
http://facets.rubyforge.org/apidoc/api/more/classes/Zlib.html
Maybe you could reimplement the Perl code trying to decompress always
and trapping the exception when the file is not compressed? Something
like this:

irb1.9 -rfacets/zlib -rzlib -rstringio
irb(main):001:0> Zlib.decompress("plain text")
Zlib::GzipFile::Error: not in gzip format

···

On Mon, May 17, 2010 at 3:49 PM, Martin Hansen <mail@maasha.dk> wrote:

--
Luis Parravicini
http://ktulu.com.ar/blog/

3

Maybe you could reimplement the Perl code trying to decompress always
and trapping the exception when the file is not compressed? Something
like this:

irb1.9 -rfacets/zlib -rzlib -rstringio
irb(main):001:0> Zlib.decompress("plain text")
Zlib::GzipFile::Error: not in gzip format

That is a good idea (I don't think you can rescue exceptions in Perl).
However, I am still uncertain if wise/possible and how to get a single
file handle on a stream from multiple files. I need to do
zipping/unzipping on the fly since I want to work on huge files.

Martin

···

--
Posted via http://www.ruby-forum.com/\.

4

Maybe you could reimplement the Perl code trying to decompress always
and trapping the exception when the file is not compressed? Something
like this:

irb1.9 -rfacets/zlib -rzlib -rstringio
irb(main):001:0> Zlib.decompress("plain text")
Zlib::GzipFile::Error: not in gzip format

That is a good idea (I don't think you can rescue exceptions in Perl).
However, I am still uncertain if wise/possible and how to get a single
file handle on a stream from multiple files. I need to do
zipping/unzipping on the fly since I want to work on huge files.

This should work :

require "zlib"

def openf(fn)
  File.open(fn) do |f|
    sig = f.read(3)
    f.rewind
    if sig.bytes.to_a == [31, 139, 8] # GZip file signature
      gz = Zlib::GzipReader.new(f)
      yield gz
      gz.close rescue nil
    else
      yield f
    end
  end
end
      

openf("toto.gz") { |f| puts f.read }

aljezalkezajlkezaj
=> nil

openf("toto2.txt") { |f| puts f.read }

iappoeipozaipza
=> nil

(You'd want to work on the exception handling etc, of course.)

Fred

···

Le 18 mai à 08:11, Martin Hansen a écrit :
--
Although I may flirt With all kinds of dirt To the point of disease
Now I want release From all this decay Take it away
And somewhere There's someone who cares With a heart of gold
To have and to hold (Depeche Mode, To Have and to Hold)