Download a file piece by piece

Hello people :-),

i have a question about downloading files. I want to download a file and print out how many bytes are downloaded. That's important to me, because i want to have a progress bar later.

The problem is that it looks like my code downloads the whole file and after that it executes the while loop - this behavior is useless for me.

does anyone know how to fix it or knows a better solution?

Thanks,
  Patrick

A working example:

require 'open-uri'

FILE_BUFFER_SIZE = 65536
filename = "dailysourcecode-39523-11-30-2006.mp3"
url = "http://m-uk.podshow.com/media/21/episodes/39523/dailysourcecode-39523-11-30-2006.mp3"

open(url, "r") do |input|
     open(filename, "w") do |output|
         bytes_downloaded = 0
         while (buffer = input.read(FILE_BUFFER_SIZE))
             output.write(buffer)
             bytes_downloaded += FILE_BUFFER_SIZE
             puts bytes_downloaded
         end
     end
end

It would be nice if there was an easier way to handle the data as it
comes in; but if you just need to do a progress bar, Kernel.open takes
two optional parameters just for this purpose:

:content_length_proc - gets called initially w/ the document size, if it
    could be obtained in the content-length header

:progress_proc - gets called with the size of the data read so far.

It's in the rdoc for open-uri under OpenURI::OpenRead#open.

···

On 11/30/06, Patrick Plattes <patrick@erdbeere.net> wrote:

i have a question about downloading files. I want to download a file and
print out how many bytes are downloaded. That's important to me, because
i want to have a progress bar later.

The problem is that it looks like my code downloads the whole file and
after that it executes the while loop - this behavior is useless for me.

--
Lou.

Patrick Plattes wrote:

Hello people :-),

i have a question about downloading files. I want to download a file and
print out how many bytes are downloaded. That's important to me, because
i want to have a progress bar later.

The problem is that it looks like my code downloads the whole file and
after that it executes the while loop -

No, actually, it is that the results are not printed out when you expect
them to be. The code is running as you wrote it, but the result printing is
being delayed. See below for the reason.

this behavior is useless for me.

does anyone know how to fix it or knows a better solution?

This is a very common beginner problem. The issue is that the main thread of
your application is too busy downloading the file to print anything out.
After the file is completely downloaded, your code exits, which gives the
environment freedom to do other things like print some data on the display.

To get a running account of your activity, you need to learn how to use
threads, and even then, because of how Ruby handles threads, showing a
progress bar is not a slam dunk.

···

--
Paul Lutus
http://www.arachnoid.com

We understand that you are very smart and have been programming for 30
years, but if you don't have anything constructive to add, like a reference
to learning how to use threads, could you please quit cluttering my inbox
with your rants? Does anyone else have visions of an old man waving his
cane around screaming at the kids to get off his lawn?
By the way, I like the ruby stuff on your web page, even if the vast
majority is way over my head.

···

On 11/30/06, Paul Lutus <nospam@nosite.zzz> wrote:

To get a running account of your activity, you need to learn how to use
threads, and even then, because of how Ruby handles threads, showing a
progress bar is not a slam dunk.

Louis J Scoras schrieb:

It would be nice if there was an easier way to handle the data as it
comes in; but if you just need to do a progress bar, Kernel.open takes
two optional parameters just for this purpose:

:content_length_proc - gets called initially w/ the document size, if it
   could be obtained in the content-length header

:progress_proc - gets called with the size of the data read so far.

It's in the rdoc for open-uri under OpenURI::OpenRead#open.

Thanks! I'm happy. It works really great :slight_smile:

Have a nice day,
  Patrick