Open-uri and ftp problem

Hi!
following script (with an existing ftp address) throws an exception.

#file simple_uri
require 'open-uri'

ftp = "ftp://" # ftp address left out in this mail,
                # but one that I can acces
open( ftp ){ |f|
   f.read
}
# end of file
produces:

:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in `initialize': No such file or directory - /dev/null (Errno::ENOENT)
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in `open_uri_original_open'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in `open'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/net/ftp.rb:497:in `getbinaryfile'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:600:in `direct_open'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:169:in `open_loop'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:164:in `catch'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:134:in `open_uri'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:424:in `open'
  from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:85:in `open'
  from simple_uri.rb:3
>Exit code: 1
]

RTFS'ing *cough* I find that open-uri does this:

[snip]
ftp = Net::FTP.open(self.host)
ftp.login(user, passwd)
# line 600 in open-uri
ftp.getbinaryfile(self.path, '/dev/null', Net::FTP::DEFAULT_BLOCKSIZE)
[snap]

Docs for Net::FTP#getbinaryfile say:

···

--
   getbinaryfile(remotefile, localfile = File.basename(remotefile), blocksize = DEFAULT_BLOCKSIZE) {|data| ...}

Retrieves remotefile in binary mode, storing the result in localfile. If a block is supplied, it is passed the retrieved data in blocksize chunks.
---

Since I am on Windows, there is no /dev/null to 'store' the file in.

any clues on what I am doing wrong?

regards,
Henrik

In article <41658592.6070102@gmx.net>,
  Henrik Horneber <ryco@gmx.net> writes:

RTFS'ing *cough* I find that open-uri does this:

[snip]
ftp = Net::FTP.open(self.host)
ftp.login(user, passwd)
# line 600 in open-uri
ftp.getbinaryfile(self.path, '/dev/null', Net::FTP::DEFAULT_BLOCKSIZE)
[snap]

Docs for Net::FTP#getbinaryfile say:
--
   getbinaryfile(remotefile, localfile = File.basename(remotefile),
blocksize = DEFAULT_BLOCKSIZE) {|data| ...}

Retrieves remotefile in binary mode, storing the result in localfile. If
a block is supplied, it is passed the retrieved data in blocksize chunks.
---

Since I am on Windows, there is no /dev/null to 'store' the file in.

any clues on what I am doing wrong?

I think Net::FTP#getbinaryfile should accept nil as the localfile
argument.

If so, open-uri can avoid /dev/null.

···

--
Tanaka Akira

Tanaka Akira wrote:

I think Net::FTP#getbinaryfile should accept nil as the localfile
argument.

If so, open-uri can avoid /dev/null.

Hi!

Thanks for your reply!

Unfortunately, Net::FTP#getbinaryfile does not accept nil, as far as I understand the source, it passes the parameter to open, and dies with

H:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in `open_uri_original_open': cannot convert nil into String (TypeError)

When I change

ftp.getbinaryfile(self.path, '/dev/null', Net::FTP::DEFAULT_BLOCKSIZE) {|str|

to

ftp.getbinaryfile(self.path) {|str|

in open-uri.rb, it only falls back to Net::FTP#getbinaryfile's default value, which simply is the basename of the remote file, so that I end up with the file on my hard drive. (This sounds strange if you don't know the context, but this gets called from open("ftp://") with a block, so I want to handle the data myself without temporary storage on disk) .

I'm stuck. Would this not be a problem if I were using the cygwin build, because there might exist some fake /dev/null?

regards,
Henrik

You could just modify the definition of FTP.getbinaryfile (either by
modifying the source or overriding it in your code) like so:

    def getbinaryfile(remotefile, localfile = File.basename(remotefile),
                      blocksize = DEFAULT_BLOCKSIZE, &block) # :yield:
data
      if @resume
        rest_offset = File.size?(localfile)
        f = open(localfile, "a")
      elsif localfile == '/dev/nul' or localfile == nil
        rest_offset = nil
        f = nil
      else
        rest_offset = nil
        f = open(localfile, "w")
      end
      begin
        f.binmode if f
        retrbinary("RETR " + remotefile, blocksize, rest_offset) do

data>

          f.write(data) if f
          yield(data) if block
        end
      ensure
        f.close if f
      end
    end

And that should do what you want. If this is something you will be
using for a while, I'd suggest negotiating/propagating the change to the
appropriate maintainers, so you don't have to revisit the fix every time
you upgrade.

-- Markus

···

On Fri, 2004-10-08 at 10:10, Henrik Horneber wrote:

Tanaka Akira wrote:

>
> I think Net::FTP#getbinaryfile should accept nil as the localfile
> argument.
>
> If so, open-uri can avoid /dev/null.

Hi!

Thanks for your reply!

Unfortunately, Net::FTP#getbinaryfile does not accept nil, as far as I
understand the source, it passes the parameter to open, and dies with

H:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in
`open_uri_original_open': cannot convert nil into String (TypeError)

When I change

ftp.getbinaryfile(self.path, '/dev/null', Net::FTP::DEFAULT_BLOCKSIZE)
{|str|

to

ftp.getbinaryfile(self.path) {|str|

in open-uri.rb, it only falls back to Net::FTP#getbinaryfile's default
value, which simply is the basename of the remote file, so that I end up
with the file on my hard drive. (This sounds strange if you don't know
the context, but this gets called from open("ftp://") with a block, so I
want to handle the data myself without temporary storage on disk) .

I'm stuck. Would this not be a problem if I were using the cygwin build,
because there might exist some fake /dev/null?

regards,
Henrik

Markus wrote:

You could just modify the definition of FTP.getbinaryfile (either by
modifying the source or overriding it in your code) like so:

[snip code]

And that should do what you want. If this is something you will be
using for a while, I'd suggest negotiating/propagating the change to the
appropriate maintainers, so you don't have to revisit the fix every time
you upgrade.

Hi!

To be honest, I do not have a real use for it at all. I stumbled over it while trying to hack a download progress display into gems. I simply wondered if my code worked for ftp downloads, too. So, if I understand this correctly, as soon as somebody starts to publish gems per anonymous ftp, some of us windows users are in for a surprise, because gems uses open-uri like I did in my example.

Given my track record, it's perfectly possible that I misunderstand something here tho.

regards,
Henrik