I’ve downloaded WebFetcher from raa, and wanna recursively download a
portion of a site, but don’t know how to maintain the path structure, 'cos
when there’re two files with the same name, the file gets rename, saving
them in the same directory as the website did would solve that problem.
I’ve tried something like this, and obviously didn’t work:
page = Page.url(‘http://www.example.com/index/’)
page.recurse(2).images.save('example + page.dirname, &SAVE)
Robo
You might like to try using ‘httpsnapshot’ instead (also from RAA). Here’s
what I got when I pointed it at www.rubycentral.com/book :
$ ls /u/downloads/pickaxe/
builtins.html ref_c_falseclass.html ref_m_enumerable.html
classes.html ref_c_file.html ref_m_errno.html
dot.gif ref_c_file__stat.html ref_m_filetest.html
ext_ruby.html ref_c_fixnum.html ref_m_gc.html
ext_tk.html ref_c_float.html ref_m_kernel.html
foreword.html ref_c_hash.html ref_m_marshal.html
index-0000.html ref_c_integer.html ref_m_math.html
index.html ref_c_io.html ref_m_objectspace.html
intro.html ref_c_matchdata.html ref_m_process.html
irb.html ref_c_method.html roadmap.html
language.html ref_c_module.html rubyworld.html
lib_network.html ref_c_nilclass.html support.html
lib_patterns.html ref_c_numeric.html taint.html
lib_standard.html ref_c_object.html trouble.html
lib_windows.html ref_c_proc.html tut_classes.html
ospace.html ref_c_range.html tut_containers.html
pr_style.css ref_c_regexp.html tut_exceptions.html
preface.html ref_c_string.html tut_expressions.html
rdtool.html ref_c_struct.html tut_io.html
ref_c_array.html ref_c_struct__tms.html tut_methods.html
ref_c_bignum.html ref_c_symbol.html tut_modules.html
ref_c_binding.html ref_c_thread.html tut_stdtypes.html
ref_c_class.html ref_c_threadgroup.html tut_threads.html
ref_c_continuation.html ref_c_time.html visible_space.gif
ref_c_dir.html ref_c_trueclass.html web.html
ref_c_exception.html ref_m_comparable.html win32.html
All the internal hyperlinks seem to work properly.
Regards,
Brian.
On Mon, Jun 30, 2003 at 06:56:58PM +0900, Robo wrote:
I’ve downloaded WebFetcher from raa, and wanna recursively download a
portion of a site, but don’t know how to maintain the path structure, 'cos
when there’re two files with the same name, the file gets rename, saving
them in the same directory as the website did would solve that problem.I’ve tried something like this, and obviously didn’t work:
page = Page.url(‘http://www.example.com/index/’)
page.recurse(2).images.save('example + page.dirname, &SAVE)