Downloading a file

ruby-talk
1

hello what is the best way to download a file?

if the process get interrupted like the network goes down for a little while, is there a way to continue the download from the point it left off?

I am seeking various solutions, thanks!

···

--
Kind Regards,
Rajinder Yadav | DevMentor.org | Do Good! ~ Share Freely

GNU/Linux: 2.6.35-23-generic
Kubuntu x86_64 10.10 | KDE 4.5.1
Ruby 1.9.2p0 | Rails 3.0.3

2

what protocal shall be used?
consider:

Net::HTTP
Net::FTP

···

2010/12/20 Rajinder Yadav <devguy.ca@gmail.com>:

hello what is the best way to download a file?

if the process get interrupted like the network goes down for a little
while, is there a way to continue the download from the point it left off?

3

Rajinder Yadav wrote in post #969503:

hello what is the best way to download a file?

if the process get interrupted like the network goes down for a little
while, is there a way to continue the download from the point it left
off?

Use wget [-c]

···

--
Posted via http://www.ruby-forum.com/\.

4

ruby has a wget command? i want to do this from ruby code, and with no
backquote either

···

On Mon, Dec 20, 2010 at 6:11 AM, Brian Candler <b.candler@pobox.com> wrote:

Rajinder Yadav wrote in post #969503:

hello what is the best way to download a file?

if the process get interrupted like the network goes down for a little
while, is there a way to continue the download from the point it left
off?

Use wget [-c]

--
Kind Regards,
Rajinder Yadav | DevMentor.org | Do Good! ~ Share Freely

GNU/Linux: 2.6.35-22-generic
Kubuntu x86_64 10.10 | KDE 4.5.1
Ruby 1.9.2p0 | Rails 3.0.1

5

Rajinder Yadav wrote in post #969592:

Use wget [-c]

ruby has a wget command? i want to do this from ruby code, and with no
backquote either

I was thinking of:

    system("wget ....")

If you don't have wget on your system, I'm afraid I don't know of an
existing Ruby library which does download-to-file with restarting of
partial transfers.

You can cobble something together using Net::HTTP, but to do the
restarts you'll have to use the set_range method to add the Range:
header - see RFC 2616 section 14.35.

Note that old HTTP servers may ignore the Range: header, so you should
check that the response content_range is what you asked for.

···

--
Posted via http://www.ruby-forum.com/\.

6

well if i just want to download a file, is there something i can use
that is simple?

i tired using open-uri, but when i read and then save the file it mess
up the file? something like this

require 'open-uri'
url = 'http://www.openssl.org/source/openssl-1.0.0c.tar.gz&#39;
file = open(url)
c = file.read

f = File.open('openssl-1.0.0c.tar.gz', 'w')
f.write c
f.close

···

On Mon, Dec 20, 2010 at 1:10 PM, Brian Candler <b.candler@pobox.com> wrote:

Rajinder Yadav wrote in post #969592:

Use wget [-c]

ruby has a wget command? i want to do this from ruby code, and with no
backquote either

I was thinking of:

system("wget ....")

If you don't have wget on your system, I'm afraid I don't know of an
existing Ruby library which does download-to-file with restarting of
partial transfers.

You can cobble something together using Net::HTTP, but to do the
restarts you'll have to use the set_range method to add the Range:
header - see RFC 2616 section 14.35.

Note that old HTTP servers may ignore the Range: header, so you should
check that the response content_range is what you asked for.

--
Kind Regards,
Rajinder Yadav | DevMentor.org | Do Good! ~ Share Freely

GNU/Linux: 2.6.35-22-generic
Kubuntu x86_64 10.10 | KDE 4.5.1
Ruby 1.9.2p0 | Rails 3.0.1

7

i'm wondering if there is libcurl solution out there that will
download files, both binary and text?

···

On Mon, Dec 20, 2010 at 1:19 PM, Rajinder Yadav <devguy.ca@gmail.com> wrote:

On Mon, Dec 20, 2010 at 1:10 PM, Brian Candler <b.candler@pobox.com> wrote:

Rajinder Yadav wrote in post #969592:

Use wget [-c]

ruby has a wget command? i want to do this from ruby code, and with no
backquote either

I was thinking of:

system("wget ....")

If you don't have wget on your system, I'm afraid I don't know of an
existing Ruby library which does download-to-file with restarting of
partial transfers.

You can cobble something together using Net::HTTP, but to do the
restarts you'll have to use the set_range method to add the Range:
header - see RFC 2616 section 14.35.

Note that old HTTP servers may ignore the Range: header, so you should
check that the response content_range is what you asked for.

well if i just want to download a file, is there something i can use
that is simple?

i tired using open-uri, but when i read and then save the file it mess
up the file? something like this

require 'open-uri'
url = 'http://www.openssl.org/source/openssl-1.0.0c.tar.gz&#39;
file = open(url)
c = file.read

f = File.open('openssl-1.0.0c.tar.gz', 'w')
f.write c
f.close

--
Kind Regards,
Rajinder Yadav | DevMentor.org | Do Good! ~ Share Freely

GNU/Linux: 2.6.35-22-generic
Kubuntu x86_64 10.10 | KDE 4.5.1
Ruby 1.9.2p0 | Rails 3.0.1

8

Try

file = open(url, "rb")

instead of

file = open(url)

and

f = File.open('openssl-1.0.0c.tar.gz', 'wb')

instead of

f = File.open('openssl-1.0.0c.tar.gz', 'w')

. You're downloading a binary file, and Ruby will do bad things to
newlines in it if you don't pass the "b" for binary.
Additionally you should have a look at the block form of #open.

Vale,
Marvin

···

Am 20.12.2010 19:19, schrieb Rajinder Yadav:

i tired using open-uri, but when i read and then save the file it mess
up the file? something like this

require 'open-uri'
url = 'http://www.openssl.org/source/openssl-1.0.0c.tar.gz&#39;
file = open(url)
c = file.read

f = File.open('openssl-1.0.0c.tar.gz', 'w')
f.write c
f.close

9

Good Morning

i'm wondering if there is libcurl solution out there that will
download files, both binary and text?

I'm sure you might have heard of this great new website that was created 12+
years ago - GOOGLE. At that very website if someone like yourself entered -
ruby download file - you might find these things called websites that
actually contain information pertaining to the exact subject you asked
Google about.

John

P.S. Just for fun you may also wish to get wacky and try - ruby libcurl -
who knows what those 11 letters will find for you......

10

Thanks Vale,

that makes sense, i also just found out I can use context_type() to
figure out the file type!

···

On Mon, Dec 20, 2010 at 2:43 PM, Quintus <sutniuq@gmx.net> wrote:

Am 20.12.2010 19:19, schrieb Rajinder Yadav:

i tired using open-uri, but when i read and then save the file it mess
up the file? something like this

require 'open-uri'
url = 'http://www.openssl.org/source/openssl-1.0.0c.tar.gz&#39;
file = open(url)
c = file.read

f = File.open('openssl-1.0.0c.tar.gz', 'w')
f.write c
f.close

Try

file = open(url, "rb")

instead of

file = open(url)

and

f = File.open('openssl-1.0.0c.tar.gz', 'wb')

instead of

f = File.open('openssl-1.0.0c.tar.gz', 'w')

. You're downloading a binary file, and Ruby will do bad things to
newlines in it if you don't pass the "b" for binary.
Additionally you should have a look at the block form of #open.

Vale,
Marvin

--
Kind Regards,
Rajinder Yadav | DevMentor.org | Do Good! ~ Share Freely

GNU/Linux: 2.6.35-22-generic
Kubuntu x86_64 10.10 | KDE 4.5.1
Ruby 1.9.2p0 | Rails 3.0.1

11

WOW IS TODAY ASSHOLE DAY, I MISSED THE MEMO???

···

On Mon, Dec 20, 2010 at 1:42 PM, John W Higgins <wishdev@gmail.com> wrote:

Good Morning

i'm wondering if there is libcurl solution out there that will
download files, both binary and text?

I'm sure you might have heard of this great new website that was created 12+
years ago - GOOGLE. At that very website if someone like yourself entered -
ruby download file - you might find these things called websites that
actually contain information pertaining to the exact subject you asked
Google about.

John

P.S. Just for fun you may also wish to get wacky and try - ruby libcurl -
who knows what those 11 letters will find for you......

--
Kind Regards,
Rajinder Yadav | DevMentor.org | Do Good! ~ Share Freely

GNU/Linux: 2.6.35-22-generic
Kubuntu x86_64 10.10 | KDE 4.5.1
Ruby 1.9.2p0 | Rails 3.0.1

12

Thanks Marvin, tired eyes got the name wrong =)

···

On Mon, Dec 20, 2010 at 2:58 PM, Rajinder Yadav <devguy.ca@gmail.com> wrote:

On Mon, Dec 20, 2010 at 2:43 PM, Quintus <sutniuq@gmx.net> wrote:

Am 20.12.2010 19:19, schrieb Rajinder Yadav:

i tired using open-uri, but when i read and then save the file it mess
up the file? something like this

require 'open-uri'
url = 'http://www.openssl.org/source/openssl-1.0.0c.tar.gz&#39;
file = open(url)
c = file.read

f = File.open('openssl-1.0.0c.tar.gz', 'w')
f.write c
f.close

Try

file = open(url, "rb")

instead of

file = open(url)

and

f = File.open('openssl-1.0.0c.tar.gz', 'wb')

instead of

f = File.open('openssl-1.0.0c.tar.gz', 'w')

. You're downloading a binary file, and Ruby will do bad things to
newlines in it if you don't pass the "b" for binary.
Additionally you should have a look at the block form of #open.

Vale,
Marvin

Thanks Vale,

that makes sense, i also just found out I can use context_type() to
figure out the file type!

--
Kind Regards,
Rajinder Yadav