Net::FTP questions

Hello...

I am attempting to write a client side script that mirrors a user's
/home/* on a remote server. Granted, I *could* use ftpmirror in my
Ubuntu distribution, but I'd like to get more familiar with network
and filesystem operations in Ruby.

The basic algorithm as I see it would be to logon, get a list of all
files and folders via the Net::FTP.list(*args) command, run it through
a block or a for loop or some other iterator, ducking into
subdirectories as needed (and mkdir-ing them on the client side), and
doing a get() of all the files in each subdirectory.

Here's where my problem comes in. How can I tell from the list array
whether an item is a directory or a file to be copied? Is there
something I can use (such as an FTP command) to catch a clue on how to
implement this? I know this is possible, as ftpmirror is done with
PERL and Ruby can do anything PERL can do - better :slight_smile:

Google is no help, and neither is running searches on ruby-talk
itself... Anyone done anything similar and have any tips? Any advice
whatsoever is appreciated.

Lincoln Anderson

Hi,

···

Le mardi 12 septembre 2006 à 07:39 +0900, Lincoln Anderson a écrit :

Here's where my problem comes in. How can I tell from the list array
whether an item is a directory or a file to be copied? Is there
something I can use (such as an FTP command) to catch a clue on how to
implement this? I know this is possible, as ftpmirror is done with
PERL and Ruby can do anything PERL can do - better :slight_smile:

The output of FTP#ls should be something like :
drwxr-x--- 2 login ftp 96 Sep 11 16:07 mydir
-rwxr-x--- 2 login ftp 96 Sep 11 16:07 myfile

So you will know an entry is a directory if the string begins by "d".

Vincent

Lincoln Anderson wrote:

Here's where my problem comes in. How can I tell from the list array
whether an item is a directory or a file to be copied? Is there
something I can use (such as an FTP command) to catch a clue on how to
implement this? I know this is possible, as ftpmirror is done with
PERL and Ruby can do anything PERL can do - better :slight_smile:
  

Here's a class I defined to use in a script I run that moves files between various Unix and Windows servers at my company. Haven't looked at the code in a while, so it might be ugly, but it works!

class FTPLocation
    def initialize(host, userid, password, path)
        @host, @userid, @password = host, userid, password
                 if path.nil?
            @path = './'
        else
            @path = path
        end
    end

···

#-----------------------------------------------------------------------
    # to_s
    #-----------------------------------------------------------------------
    def to_s
        return '%s - %s' % [@host, @path]
    end
     
    #-----------------------------------------------------------------------
    # connect
    #-----------------------------------------------------------------------
    def connect
        @ftp = Net::FTP.new(@host, @userid, @password)
# @ftp.debug_mode = true
        @ftp.chdir(@path)
    end
     
    #-----------------------------------------------------------------------
    # empty?
    #-----------------------------------------------------------------------
    def empty?
        @ftp.list.each do |entry|
            next if entry[0] == 'd'
            return false
        end
        return true
    end
     
    #-----------------------------------------------------------------------
    # each_file
    #-----------------------------------------------------------------------
    def each_file
        t = Time.now
        @ftp.list.each do |entry|
            next if entry[0] == 'd'
            filename = entry.split(' ')[-1]
            next if t - @ftp.mtime(filename) < 300
            yield filename
        end
    end
     
    #-----------------------------------------------------------------------
    # file_exists?
    #-----------------------------------------------------------------------
    def file_exists?(filename)
        @ftp.list.each do |entry|
            next if entry[0] == 'd'
            return true if filename == entry.split(' ')[-1]
        end
        return false
    end
     
    #-----------------------------------------------------------------------
    # get_file
    #-----------------------------------------------------------------------
    def get_file(filename, dest_path, xfer_type)
        if xfer_type == 'binary'
            @ftp.getbinaryfile(filename, dest_path)
        else
            @ftp.gettextfile(filename, dest_path)
        end
    end
     
    #-----------------------------------------------------------------------
    # put_file
    #-----------------------------------------------------------------------
    def put_file(source_path, filename, xfer_type)
        if xfer_type == 'binary'
            @ftp.putbinaryfile(source_path, filename)
        else
            @ftp.puttextfile(source_path, filename)
        end end
     
    #-----------------------------------------------------------------------
    # delete_file
    #-----------------------------------------------------------------------
    def delete_file(filename)
        @ftp.delete(filename)
    end
     
    #-----------------------------------------------------------------------
    # close
    #-----------------------------------------------------------------------
    def close
        @ftp.close if @ftp
    end
end

Confidentiality Notice: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. If you are not the intended recipient(s), you are hereby notified that any dissemination, unauthorized review, use, disclosure or distribution of this email and any materials contained in any attachments is prohibited. If you receive this message in error, or are not the intended recipient(s), please immediately notify the sender by email and destroy all copies of the original message, including attachments.

Vincent Arnoux wrote:

Hi,

:

Here's where my problem comes in. How can I tell from the list
array whether an item is a directory or a file to be copied? Is
there something I can use (such as an FTP command) to catch a
clue on how to implement this? I know this is possible, as
ftpmirror is done with PERL and Ruby can do anything PERL can do
- better :slight_smile:

The output of FTP#ls should be something like : drwxr-x--- 2
login ftp 96 Sep 11 16:07 mydir -rwxr-x--- 2 login ftp
96 Sep 11 16:07 myfile

So you will know an entry is a directory if the string begins by
"d".

Vincent

Yeah, I noticed that after my post... I guess I was trying to
complicate things :slight_smile:

Thanks,
Lincoln Anderson

···

Le mardi 12 septembre 2006 à 07:39 +0900, Lincoln Anderson a écrit

Lincoln Anderson wrote:

Here's where my problem comes in. How can I tell from the list array
whether an item is a directory or a file to be copied? Is there
something I can use (such as an FTP command) to catch a clue on how to
implement this? I know this is possible, as ftpmirror is done with
PERL and Ruby can do anything PERL can do - better :slight_smile:

Perhaps this can be useful to browse the FTP server tree (directory
names finish by '/', others are files):

def recursive_listing(ftp, path)
        liste_ftp = Array.new
        list = ftp.ls(path)
        if list.length == 0
                return
        end
        if list[0].split[0] == 'total'
                list.delete_at(0)
        end
        if list[0].split[8] == '.'
                list.delete_at(0)
        end
        if list[0].split[8] == '..'
                list.delete_at(0)
        end
       list.each { |str|
               if str.split[0].index('d') == 0
                       liste_ftp += [path + "/" + str.split[8] + "/"]
                       liste_ftp += [recursive_listing(ftp, path + "/"
+ str.split[8])]
               else
                       liste_ftp += [path + "/" + str.split[8]]
               end
       }
       return liste_ftp.flatten
end

ftp = Net::FTP.new('ftp_serveur')
ftp.login('login', 'passwd')
ftp_list = recursive_listing(ftp, 'starting_directory')
ftp.close

It might need some corrections depending your FTP server configuration.

Cheers,
Vincent

···

Le mercredi 13 septembre 2006 à 00:14 +0900, James Cribbs a écrit :

Vincent Arnoux wrote:

Lincoln Anderson wrote:

Here's where my problem comes in. How can I tell from the list
array whether an item is a directory or a file to be copied? Is
there something I can use (such as an FTP command) to catch a
clue on how to implement this? I know this is possible, as
ftpmirror is done with PERL and Ruby can do anything PERL can do
- better :slight_smile:

Perhaps this can be useful to browse the FTP server tree (directory
names finish by '/', others are files):

def recursive_listing(ftp, path) liste_ftp = Array.new list =
ftp.ls(path) if list.length == 0 return end if list[0].split[0]
== 'total' list.delete_at(0) end if list[0].split[8] == '.'
list.delete_at(0) end if list[0].split[8] == '..' list.delete_at(0)
end list.each { |str| if str.split[0].index('d') == 0 liste_ftp +=
[path + "/" + str.split[8] + "/"] liste_ftp +=
[recursive_listing(ftp, path + "/" + str.split[8])] else liste_ftp
+= [path + "/" + str.split[8]] end } return liste_ftp.flatten end

ftp = Net::FTP.new('ftp_serveur') ftp.login('login', 'passwd')
ftp_list = recursive_listing(ftp, 'starting_directory') ftp.close

It might need some corrections depending your FTP server
configuration.

Cheers, Vincent

That looks exactly like what I'm trying to do for the directory
hierarchy mirror. I'm still learning to think in Ruby... I keep going
back to C/C++ solutions and other heavy hitter languages... I'll tweak
that and see if I can get it working. I'll post my results when I get
it running (or I have any other insurmountable problems...).

Thanks,
Lincoln Anderson

···

Le mercredi 13 septembre 2006 à 00:14 +0900, James Cribbs a écrit :