Quickest way to get md5 of a file?

ruby-talk
1

Hi,

I wanna get the md5 checksums of files from within Ruby code.
I realize I can simply loop, read chunks of the file and
update an MD5 object but that would be very much slower than
using an external MD5 checksum program and I have very many
files to check.

Is there some trick to speed this up or should we consider adding
a checksum_of_file method to Digest?

Regards,

Karsten

···

–
http://fastmail.fm - Same, same, but different…

2

Hik

···

In message “Quickest way to get md5 of a file?” on 02/12/22, coma_killen@fastmail.fm coma_killen@fastmail.fm writes:

I wanna get the md5 checksums of files from within Ruby code.
I realize I can simply loop, read chunks of the file and
update an MD5 object but that would be very much slower than
using an external MD5 checksum program and I have very many
files to check.

Is there some trick to speed this up or should we consider adding
a checksum_of_file method to Digest?

ruby -r digest/md5 -e ‘ARGV.each{|f|print f, " "; puts Digest::MD5.hexdigest(File.read(f))}’ *

						matz.
3

“Yukihiro Matsumoto” matz@ruby-lang.org wrote in message
news:1040550973.608293.14507.nullmailer@picachu.netlab.jp…

Is there some trick to speed this up or should we consider adding
a checksum_of_file method to Digest?

ruby -r digest/md5 -e ‘ARGV.each{|f|print f, " "; puts
Digest::MD5.hexdigest(File.read(f))}’ *

This doesn’t work on Windows 2K, Ruby 1.6.7 or 1.7.3-7
It says that ‘f’ is not a valid file.
File.read(f) doesn’t work on 1.6.7 (use File.new(f).read but it does work on
1.7.3-7)

require 'digest/md5’
v = [‘test.rb’]
v.each{|f|print f, " "; puts Digest::MD5.hexdigest(File.new(f).read)}

Mikkel

4

You can escape the vertical line using caret like this.

C:>ruby -r digest/md5 -e ‘ARGV.each{^|f^| … }’ *

However, I couldn’t find any documents for this special
usage of the caret on Windows. I found this useful tips
by trial and error. Does anyone know where the official
document is?

···

On Sun, 22 Dec 2002 14:00:45 +0100 “MikkelFJ” mikkelfj-anti-spam@bigfoot.com wrote:

ruby -r digest/md5 -e ‘ARGV.each{|f|print f, " "; puts
Digest::MD5.hexdigest(File.read(f))}’ *

This doesn’t work on Windows 2K, Ruby 1.6.7 or 1.7.3-7
It says that ‘f’ is not a valid file.

–
Shusaku tsyk@yk.rim.or.jp

5

“MikkelFJ” mikkelfj-anti-spam@bigfoot.com writes:

ruby -r digest/md5 -e ‘ARGV.each{|f|print f, " "; puts
Digest::MD5.hexdigest(File.read(f))}’ *

This doesn’t work on Windows 2K, Ruby 1.6.7 or 1.7.3-7
It says that ‘f’ is not a valid file.
File.read(f) doesn’t work on 1.6.7 (use File.new(f).read but it does work on
1.7.3-7)

ruby -r digest/md5 -le “ARGV.each{|f| print f, ’ ', Digest::MD5.hexdigest(open(f, ‘rb’).read)}” *

···

–
eban

6

“WATANABE Hirofumi” eban@os.rim.or.jp wrote in message
news:20021223010249.588814.eban@os.rim.or.jp…

ruby -r digest/md5 -le “ARGV.each{|f| print f, ’ ',
Digest::MD5.hexdigest(open(f, ‘rb’).read)}” *

fails on directories though

Mikkel

7

“MikkelFJ” mikkelfj-anti-spam@bigfoot.com wrote in message

fails on directories though

ruby -r digest/md5 -le “ARGV.each{|f| (print f, ’ ',
Digest::MD5.hexdigest(open(f, ‘rb’).read)) unless File.stat(f).directory? }”

···

But this will still fail on files which do not have access, like on my
machine (Win XP Pro):

ruby 1.7.3 (2002-11-17) [i386-mswin32]

-e:1:in initialize': Permission denied - "hiberfil.sys" (Errno::EACCES) from -e:1:inopen’

8

“MikkelFJ” mikkelfj-anti-spam@bigfoot.com writes:

ruby -r digest/md5 -le “ARGV.each{|f| print f, ’ ',
Digest::MD5.hexdigest(open(f, ‘rb’).read)}” *

fails on directories though

ruby -r digest/md5 -e “ARGV.each{|f|test ?f,f and puts Digest::MD5.hexdigest(open(f, ‘rb’).read)+’ '+f rescue Errno}” ?

···

–
eban

9

Take 2 (no more one liner and slick like WATANABE’s next post) :

require 'digest/md5’
ARGV.each{|f|
begin
printf("%30s : ",f); puts Digest::MD5.hexdigest(File.read(f)) ;

^^^^^

Assuming no file names are lareger than 30 chars

rescue
puts "…skipping (may be dir or Access denied)"
next
end
}

“Shashank Date” sdate@kc.rr.com wrote in message
news:X8nN9.15609$Zt4.173973@twister.rdc-kc.rr.com…

“MikkelFJ” mikkelfj-anti-spam@bigfoot.com wrote in message

fails on directories though

ruby -r digest/md5 -le “ARGV.each{|f| (print f, ’ ',
Digest::MD5.hexdigest(open(f, ‘rb’).read)) unless
File.stat(f).directory? }”

···

But this will still fail on files which do not have access, like on my
machine (Win XP Pro):

ruby 1.7.3 (2002-11-17) [i386-mswin32]

-e:1:in initialize': Permission denied - "hiberfil.sys" (Errno::EACCES) from -e:1:inopen’