Recursing through directories

Hi there

I have a simple script (untested)

require 'ftools'
Dir.entries( '/Volumes/' ).each |item| do
   File.move( item, item.gsub( /\//, '-' ) ) if File.file? item
end

The idea is to remove slashes from file names, I can later extend this to include other characters also. But how the deuce can I make it go in to each directoy to make sure each file is renamed even many subdirectories down?

Many thanks

Gabriel

Alle mercoledì 26 settembre 2007, Gabriel Dragffy ha scritto:

Hi there

I have a simple script (untested)

require 'ftools'
Dir.entries( '/Volumes/' ).each |item| do
   File.move( item, item.gsub( /\//, '-' ) ) if File.file? item
end

The idea is to remove slashes from file names, I can later extend
this to include other characters also. But how the deuce can I make
it go in to each directoy to make sure each file is renamed even many
subdirectories down?

Many thanks

Gabriel

I think that the Find module is what you need (ri Find).

Stefano

Gabriel Dragffy wrote:

require 'ftools'
Dir.entries( '/Volumes/' ).each |item| do
   File.move( item, item.gsub( /\//, '-' ) ) if File.file? item
end

The idea is to remove slashes from file names, I can later extend
this to include other characters also. But how the deuce can I make
it go in to each directoy to make sure each file is renamed even many
subdirectories down?

Use Dir['/Volumes/**/*'] (or even *-* to only get files with a slash in them)
instead of Dir.entries.

HTH,
Sebastian

···

--
Jabber: sepp2k@jabber.org
ICQ: 205544826

Will this operate recursively?

Many thanks

Gabriel

···

On 26 Sep 2007, at 16:33, Sebastian Hungerecker wrote:

Gabriel Dragffy wrote:

require 'ftools'
Dir.entries( '/Volumes/' ).each |item| do
   File.move( item, item.gsub( /\//, '-' ) ) if File.file? item
end

The idea is to remove slashes from file names, I can later extend
this to include other characters also. But how the deuce can I make
it go in to each directoy to make sure each file is renamed even many
subdirectories down?

Use Dir['/Volumes/**/*'] (or even *-* to only get files with a slash in them)
instead of Dir.entries.

Gabriel Dragffy ha scritto:

Gabriel Dragffy wrote:

require 'ftools'
Dir.entries( '/Volumes/' ).each |item| do
   File.move( item, item.gsub( /\//, '-' ) ) if File.file? item
end

The idea is to remove slashes from file names, I can later extend
this to include other characters also. But how the deuce can I make
it go in to each directoy to make sure each file is renamed even many
subdirectories down?

Use Dir['/Volumes/**/*'] (or even *-* to only get files with a slash in them)
instead of Dir.entries.

Will this operate recursively?

Many thanks

Gabriel

Yes.

Dir['/Volumes/**/*'] will match the directories recursively.

Bye.
Andrea

···

On 26 Sep 2007, at 16:33, Sebastian Hungerecker wrote:

Thank you for your help. I have a little trouble getting this script to operate properly and its given me an error I haven't come across before...

  Dir['/Volumes/**/*'].each { |file| File.move( file, gsub( /\//, "-") ) if File.file?( file ) }
TypeError: $_ value need to be String (nil given)
         from (irb):4:in `gsub'
         from (irb):4
         from (irb):4:in `each'
         from (irb):4

What might this mean?

Many thanks

Gabriel

···

On 26 Sep 2007, at 19:04, Andrea Fazzi wrote:

Will this operate recursively?

Many thanks

Gabriel

Yes.

Dir['/Volumes/**/*'] will match the directories recursively.

Bye.
Andrea

You're not providing a receiver for the gsub() method. Having said that, though, what it seems like you're trying to do is almost certainly a bad idea. It looks like you're prepared to move all files into the current directory. If that's not what you intended, then it's probably very good that you got an error.

-Rob

Rob Biedenharn http://agileconsultingllc.com
Rob@AgileConsultingLLC.com

···

On Sep 27, 2007, at 8:37 AM, Gabriel Dragffy wrote:

On 26 Sep 2007, at 19:04, Andrea Fazzi wrote:

Will this operate recursively?

Many thanks

Gabriel

Yes.

Dir['/Volumes/**/*'] will match the directories recursively.

Bye.
Andrea

Thank you for your help. I have a little trouble getting this script to operate properly and its given me an error I haven't come across before...

Dir['/Volumes/**/*'].each { |file| File.move( file, gsub( /\//, "-") ) if File.file?( file ) }
TypeError: $_ value need to be String (nil given)
        from (irb):4:in `gsub'
        from (irb):4
        from (irb):4:in `each'
        from (irb):4

What might this mean?

Many thanks

Gabriel

I'm sorry, you are so right. I really can't seem to get the hang of this. You're correct in that it is moving all files in to the current directory. How can I make it so it just renames them in place, and which resource should I be ideally reading to understand this better?

Best regards

Gabriel

···

On 27 Sep 2007, at 14:14, Rob Biedenharn wrote:

You're not providing a receiver for the gsub() method. Having said that, though, what it seems like you're trying to do is almost certainly a bad idea. It looks like you're prepared to move all files into the current directory. If that's not what you intended, then it's probably very good that you got an error.

-Rob

If you want to rename files having names that meet some pattern, why don't you start with print the names like this (files having "foo" somewhere in their name):

Dir['**/*'].each {|f| puts f if f =~ /foo/ }

but beware that Dir['**/*'] is going to plow through the directory structure and make an array (a big one) of all the file names before the block gets to see any of those names.

Perhaps you'd get some better answers if you post what you're trying to accomplish, the attempt you've made (or what you think should work), and the actual results you see.

Also, note that my example starts from the current directory (which was ~/ in my case). It's always better to get a script like yours working on a small bit of known data (and perhaps be non-destructive) before turning it loose on your entire disk. (In my case, it found many files that were throwaway "foo" files, but also many with "foot", "footnote", "food", and "footer" that were clearly not throwaway files.)

-Rob

Rob Biedenharn http://agileconsultingllc.com
Rob@AgileConsultingLLC.com

···

On Sep 27, 2007, at 10:55 AM, Gabriel Dragffy wrote:

On 27 Sep 2007, at 14:14, Rob Biedenharn wrote:

You're not providing a receiver for the gsub() method. Having said that, though, what it seems like you're trying to do is almost certainly a bad idea. It looks like you're prepared to move all files into the current directory. If that's not what you intended, then it's probably very good that you got an error.

-Rob

I'm sorry, you are so right. I really can't seem to get the hang of this. You're correct in that it is moving all files in to the current directory. How can I make it so it just renames them in place, and which resource should I be ideally reading to understand this better?

Best regards

Gabriel

Hi Rob

Thanks for your help.

What I am trying to achieve is replace the slashes (/) in file/directory names under a certain directory. So, for example, I want to recursively rename offending files that are under /Volumes, but I don't want them moved from their current directory... just renamed. What's the best way to go about that?

Regards

Gabe

···

On 27 Sep 2007, at 16:35, Rob Biedenharn wrote:

If you want to rename files having names that meet some pattern, why don't you start with print the names like this (files having "foo" somewhere in their name):

Dir['**/*'].each {|f| puts f if f =~ /foo/ }

but beware that Dir['**/*'] is going to plow through the directory structure and make an array (a big one) of all the file names before the block gets to see any of those names.

Perhaps you'd get some better answers if you post what you're trying to accomplish, the attempt you've made (or what you think should work), and the actual results you see.

Also, note that my example starts from the current directory (which was ~/ in my case). It's always better to get a script like yours working on a small bit of known data (and perhaps be non-destructive) before turning it loose on your entire disk. (In my case, it found many files that were throwaway "foo" files, but also many with "foot", "footnote", "food", and "footer" that were clearly not throwaway files.)

This doesn't make sense. There aren't slashes in the name, those are separating directories along the path to the file.

/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/another.yml
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/hash2ostruct.rb
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/openstruct.yml
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/recursive.yml

These files have some directories containing spaces in the name, but the name of each file is just the last part (another.yml, hash2ostruct.rb, openstruct.yml, recursive.yml). If I go to my home directory and rename 'code' to be 'source', I haven't changed the names of the files, but I *have* changed the path needed to reach them.

If you really *do* have files with '/' in their names, chances are you need to use a low-level utility to recover from that mess. If you're just seeing in the Mac OS X Finder that there are "slashes" in the name of a file, perhaps you're seeing an artifact of the Mac's filesystem. If you open a Terminal, you'll see that there the file has a ':' in the place that Finder shows you '/'.

Perhaps you can find these files (like I said, see if you find them properly before to operate on them) with Ruby like this:
  ruby -e 'puts Dir["**/*:*"]'
But know that there are a great many files that *NEED* to contain ':' created by iChat, AddressBook, Safari, and others. (You might not want to mess with anything under a Library directory :wink:

-Rob

Rob Biedenharn http://agileconsultingllc.com
Rob@AgileConsultingLLC.com

···

On Sep 28, 2007, at 5:04 AM, Gabriel Dragffy wrote:

On 27 Sep 2007, at 16:35, Rob Biedenharn wrote:

If you want to rename files having names that meet some pattern, why don't you start with print the names like this (files having "foo" somewhere in their name):

Dir['**/*'].each {|f| puts f if f =~ /foo/ }

but beware that Dir['**/*'] is going to plow through the directory structure and make an array (a big one) of all the file names before the block gets to see any of those names.

Perhaps you'd get some better answers if you post what you're trying to accomplish, the attempt you've made (or what you think should work), and the actual results you see.

Also, note that my example starts from the current directory (which was ~/ in my case). It's always better to get a script like yours working on a small bit of known data (and perhaps be non-destructive) before turning it loose on your entire disk. (In my case, it found many files that were throwaway "foo" files, but also many with "foot", "footnote", "food", and "footer" that were clearly not throwaway files.)

Hi Rob

Thanks for your help.

What I am trying to achieve is replace the slashes (/) in file/directory names under a certain directory. So, for example, I want to recursively rename offending files that are under /Volumes, but I don't want them moved from their current directory... just renamed. What's the best way to go about that?

Regards

Gabe

Gabriel Dragffy wrote:

What I am trying to achieve is replace the slashes (/) in file/directory
names under a certain directory. So, for example, I want to recursively
rename offending files that are under /Volumes, but I don't want them
moved from their current directory... just renamed. What's the best way
to go about that?

I guess you could do something like

Dir['startdir/**/'].each { |dirname|
  Dir.new(dirname).each{ |file_basename|
    #manipulate file_basename => new_name
    oldpath = File.join(dirname, file_basename)
    newpath = File.join(dirname, new_name)
    File.move(oldpath, newpath)
  }
}

Note that the outer iterator will only yield you directory names, while
the inner one will only yield file basenames inside those directories.
So you can treat file names, even ones containing slashes on their own,
and then combine the results with their respective directory names.

mortee

PS.: I could not figure out how to create a file with a / in its name in
those few minutes so I could not test if this actually works if you have
such files...

Rob, you are spot on! When I view the files in Finder they have slashes in the name. However, I need to backup these files to a linux files server using Rsync (because rsync won't copy everything but only the changes) but it keeps conking out with errors. If I run cp -rv it fails with errors like:
cp: /Volumes/Cecil/02photography/1997_past/::PANORMANIA: Invalid argument
When I look at the file in Finder it has a name with slashes in it. My immediate thought was to automatically rename all files like this. What would suggest I do?

Best regards

Gabriel

···

On 28 Sep 2007, at 14:17, Rob Biedenharn wrote:

Hi Rob

Thanks for your help.

What I am trying to achieve is replace the slashes (/) in file/directory names under a certain directory. So, for example, I want to recursively rename offending files that are under /Volumes, but I don't want them moved from their current directory... just renamed. What's the best way to go about that?

Regards

Gabe

This doesn't make sense. There aren't slashes in the name, those are separating directories along the path to the file.

/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/another.yml
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/hash2ostruct.rb
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/openstruct.yml
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/recursive.yml

These files have some directories containing spaces in the name, but the name of each file is just the last part (another.yml, hash2ostruct.rb, openstruct.yml, recursive.yml). If I go to my home directory and rename 'code' to be 'source', I haven't changed the names of the files, but I *have* changed the path needed to reach them.

If you really *do* have files with '/' in their names, chances are you need to use a low-level utility to recover from that mess. If you're just seeing in the Mac OS X Finder that there are "slashes" in the name of a file, perhaps you're seeing an artifact of the Mac's filesystem. If you open a Terminal, you'll see that there the file has a ':' in the place that Finder shows you '/'.

Perhaps you can find these files (like I said, see if you find them properly before to operate on them) with Ruby like this:
  ruby -e 'puts Dir["**/*:*"]'
But know that there are a great many files that *NEED* to contain ':' created by iChat, AddressBook, Safari, and others. (You might not want to mess with anything under a Library directory :wink:

I think that you are running into trouble with rsync's use of a : or two for special syntax to identify a file on a different host.

rsync(1) rsync(1)

NAME
        rsync - faster, flexible replacement for rcp

SYNOPSIS
        rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST

        rsync [OPTION]... [USER@]HOST:SRC DEST

        rsync [OPTION]... SRC [SRC]... DEST

        rsync [OPTION]... [USER@]HOST::SRC [DEST]

        rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST

        rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]

        rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST

You may have to quote the file name of supply an explicit host portion even if it matches the default. You might alternatively create an incremental tar archive of the changes and then back up the tar file to the linux box. (...and either way it's not really a Ruby problem.)

-Rob

Rob Biedenharn http://agileconsultingllc.com
Rob@AgileConsultingLLC.com

···

On Sep 28, 2007, at 2:42 PM, Gabriel Dragffy wrote:

On 28 Sep 2007, at 14:17, Rob Biedenharn wrote:

Hi Rob

Thanks for your help.

What I am trying to achieve is replace the slashes (/) in file/directory names under a certain directory. So, for example, I want to recursively rename offending files that are under /Volumes, but I don't want them moved from their current directory... just renamed. What's the best way to go about that?

Regards

Gabe

This doesn't make sense. There aren't slashes in the name, those are separating directories along the path to the file.

/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/another.yml
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/hash2ostruct.rb
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/openstruct.yml
/Volumes/Macintosh HD/Users/rab/code/ruby/quiz/081-Hash to OpenStruct/recursive.yml

These files have some directories containing spaces in the name, but the name of each file is just the last part (another.yml, hash2ostruct.rb, openstruct.yml, recursive.yml). If I go to my home directory and rename 'code' to be 'source', I haven't changed the names of the files, but I *have* changed the path needed to reach them.

If you really *do* have files with '/' in their names, chances are you need to use a low-level utility to recover from that mess. If you're just seeing in the Mac OS X Finder that there are "slashes" in the name of a file, perhaps you're seeing an artifact of the Mac's filesystem. If you open a Terminal, you'll see that there the file has a ':' in the place that Finder shows you '/'.

Perhaps you can find these files (like I said, see if you find them properly before to operate on them) with Ruby like this:
  ruby -e 'puts Dir["**/*:*"]'
But know that there are a great many files that *NEED* to contain ':' created by iChat, AddressBook, Safari, and others. (You might not want to mess with anything under a Library directory :wink:

Rob, you are spot on! When I view the files in Finder they have slashes in the name. However, I need to backup these files to a linux files server using Rsync (because rsync won't copy everything but only the changes) but it keeps conking out with errors. If I run cp -rv it fails with errors like:
cp: /Volumes/Cecil/02photography/1997_past/::PANORMANIA: Invalid argument
When I look at the file in Finder it has a name with slashes in it. My immediate thought was to automatically rename all files like this. What would suggest I do?

Best regards

Gabriel