I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.
def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
One thing that you should do is get rid of that rescue block, that *should speed it up some:
- begin
- File.exist?(_fname) if File.delete(_fname)
- rescue
- end
+ File.delete(_fname) if File.exist?
~Wayne
s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator
On Sep 20, 2007, at 18:05 , Ruby Maniac wrote:
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.
Write a C extension. Seriously, Ruby is pretty darn slow at iterating
over each byte of a file. The I/O is reasonable, but the byte
iteration is terrible. You might get a small speedup by using
File#read instead of File#each_byte:
while (buffer = f.read(BUF_SIZE))
buffer.each_byte {|b| ... }
end
I got about a 5% speedup using an 8K buffer.
You also may want to consider the idiom:
File.open(filename) do |f|
...
end
This will automatically close the file at the end of the block.
def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
You may want to double check the previous line.
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
So you don't need to get the original file data back, right? If you
do, you may want to use xor instead, and if you don't, then why not
just write zeros?
On Sep 20, 6:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
end
f.close()
ff.close()
end
This is equivalent to
if File.delete(_fname)
File.exist?(_fname)
end
That's not what you want. Furthermore, there's no need
to delete the file. If you open it in mode "wb", the
existing contents will be obliterated.
On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
File.exist?(_fname) if File.delete(_fname)
Ruby Maniac wrote:
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
About 2.8 times as fast.
def scramble( fname )
_fname = fname + ".scrambled"
mask = 0x8080808080808080
data = File.open( fname, "rb" ){|f| f.read }
pad = (8 - (data.size % 8)) % 8
data << "p" * pad
File.open( _fname, "wb"){|ff|
array = data.unpack('Q*')
array.map!{|x| x | mask }
ff.write( array.pack('Q*')[0 ... -pad] )
}
end
Hi,
At Fri, 21 Sep 2007 07:05:10 +0900,
Ruby Maniac wrote in [ruby-talk:270110]:
def scramble(fname)
_fname = fname + ".scrambled"
File.new(fname, "rb") do |f|
File.new(_fname, "wb+") do |ff|
ff.write(f.read.tr("\0-\177", "\200-\377"))
end
end
end
--
Nobu Nakada
If you actually want speed, use a compiled language.
This FreeBasic code is about 160 times as fast and
demonstrates that using Python for any reason
whatsoever is unjustified.
Use Ruby when ease of programming is paramount;
use something like FreeBasic or LuaJIT when speed
is paramount. Never submit to Python perversion.
sub scramble( filename as string )
dim outname as string
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byte
outname = filename & ".scrambled"
in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open outname for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close
end sub
On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
Ruby Maniac wrote:
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
If you google for
ps2QrAMAAAA6_jCuRt2JEIpn5Otqf_w0
http://www.google.com/search?q=ps2QrAMAAAA6_jCuRt2JEIpn5Otqf_w0
which is the google posting-account number for "Ruby Maniac" in his headers you find this guy is a profi troll. Some of his posts to other groups are racist, sexist and highly offensive.
I suggest no further responses to his posts.
Any way to get this id filtered from the list?
B
Benchmark it so you can find the bottlenecks.
On Sep 20, 2007, at 8:50 PM, Brian Adkins wrote:
On Sep 20, 6:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.Write a C extension. Seriously, Ruby is pretty darn slow at iterating
over each byte of a file. The I/O is reasonable, but the byte
iteration is terrible. You might get a small speedup by using
File#read instead of File#each_byte:while (buffer = f.read(BUF_SIZE))
buffer.each_byte {|b| ... }
endI got about a 5% speedup using an 8K buffer.
You also may want to consider the idiom:
File.open(filename) do |f|
...
endThis will automatically close the file at the end of the block.
def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)You may want to double check the previous line.
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }So you don't need to get the original file data back, right? If you
do, you may want to use xor instead, and if you don't, then why not
just write zeros?end
f.close()
ff.close()
end
Hi,
Sat, 22 Sep 2007 17:38:51 +0900,
Nobuyoshi Nakada wrote in [ruby-talk:270305]:
At Fri, 21 Sep 2007 07:05:10 +0900,
Ruby Maniac wrote in [ruby-talk:270110]:
> def scramble(fname)
> _fname = fname + ".scrambled"
File.new(fname, "rb") do |f|
File.new(_fname, "wb+") do |ff|
Sorry, these are not "new" but "open".
--
Nobu Nakada
If that prolixity bothered you, look at a
version in the J programming language.
read =: 1!:1
write =: 1!:2
scramble =: verb define
((128 (23 b.) a. i. read <y) { a. ) write <y,'.scrambled'
)
scramble 'myfile'
On Sep 22, 5:16 am, William James <w_a_x_...@yahoo.com> wrote:
On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
> I am new to Ruby and I could use some expert advice as to how I can
> make this code run faster.> def scramble(fname)
> f = File.new(fname, "rb")
> _fname = fname + ".scrambled"
> begin
> File.exist?(_fname) if File.delete(_fname)
> rescue
> end
> ff = File.new(_fname, "wb+")
> for l in f
> l.each_byte{|c| ff.write((c | 0x80).chr) }
> end
> f.close()
> ff.close()
> endIf you actually want speed, use a compiled language.
This FreeBasic code is about 160 times as fast and
demonstrates that using Python for any reason
whatsoever is unjustified.Use Ruby when ease of programming is paramount;
use something like FreeBasic or LuaJIT when speed
is paramount. Never submit to Python perversion.sub scramble( filename as string )
dim outname as string
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byteoutname = filename & ".scrambled"
in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open outname for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close
end sub
Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).
Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.
On Sep 22, 12:23 am, William James <w_a_x_...@yahoo.com> wrote:
Ruby Maniac wrote:
> I am new to Ruby and I could use some expert advice as to how I can
> make this code run faster.> def scramble(fname)
> f = File.new(fname, "rb")
> _fname = fname + ".scrambled"
> begin
> File.exist?(_fname) if File.delete(_fname)
> rescue
> end
> ff = File.new(_fname, "wb+")
> for l in f
> l.each_byte{|c| ff.write((c | 0x80).chr) }
> end
> f.close()
> ff.close()
> endAbout 2.8 times as fast.
def scramble( fname )
_fname = fname + ".scrambled"
mask = 0x8080808080808080
data = File.open( fname, "rb" ){|f| f.read }
pad = (8 - (data.size % 8)) % 8
data << "p" * pad
File.open( _fname, "wb"){|ff|
array = data.unpack('Q*')
array.map!{|x| x | mask }
ff.write( array.pack('Q*')[0 ... -pad] )
}
end- Hide quoted text -- Show quoted text -
Okay I downloaded Free BASIC and installed it but your code does not
work !
On the one hand, FreeBASIC is certainly worth looking into but on the
other hand - there ain't no debugger than I know of unless you know of
a debugger for FreeBASIC.
On Sep 22, 3:16 am, William James <w_a_x_...@yahoo.com> wrote:
On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
> I am new to Ruby and I could use some expert advice as to how I can
> make this code run faster.> def scramble(fname)
> f = File.new(fname, "rb")
> _fname = fname + ".scrambled"
> begin
> File.exist?(_fname) if File.delete(_fname)
> rescue
> end
> ff = File.new(_fname, "wb+")
> for l in f
> l.each_byte{|c| ff.write((c | 0x80).chr) }
> end
> f.close()
> ff.close()
> endIf you actually want speed, use a compiled language.
This FreeBasic code is about 160 times as fast and
demonstrates that using Python for any reason
whatsoever is unjustified.Use Ruby when ease of programming is paramount;
use something like FreeBasic or LuaJIT when speed
is paramount. Never submit to Python perversion.sub scramble( filename as string )
dim outname as string
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byteoutname = filename & ".scrambled"
in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open outname for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close
end sub- Hide quoted text -- Show quoted text -
And the benchmark results for the above code on a 6.7M file:
$ time ruby -rscramble -e 'scramble_original("trace")'
real 0m11.626s
user 0m9.781s
sys 0m1.844s
$ time ruby -rscramble -e 'scramble_nobu("trace")'
real 0m0.099s
user 0m0.044s
sys 0m0.032s
116x speedup and 7 less loc
$ cat scramble.rb
def scramble_original(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
def scramble_nobu(fname)
File.open(fname, 'rb'){|in_f|
File.open(fname + '.scrambled', 'wb'){|out_f|
out_f.write(in_f.read.tr("\0-\177", "\200-\377"))
}
}
end
On 9/22/07, Nobuyoshi Nakada <nobu@ruby-lang.org> wrote:
Hi,
At Fri, 21 Sep 2007 07:05:10 +0900,
Ruby Maniac wrote in [ruby-talk:270110]:
> def scramble(fname)
> _fname = fname + ".scrambled"
File.new(fname, "rb") do |f|
File.new(_fname, "wb+") do |ff|
ff.write(f.read.tr("\0-\177", "\200-\377"))
end
end
> end--
Nobu Nakada
Finally got this FreeBASIC code to work but it reports a runtime of 3
seconds on a machine that reports 0.811 seconds for Python using Psyco
so this means FreeBASIC is not anywhere near as fast as Python
considering the Python code reads the whole file and then performs a
single String Translate and then write the whole file in 3 operations.
On Sep 22, 3:16 am, William James <w_a_x_...@yahoo.com> wrote:
On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
> I am new to Ruby and I could use some expert advice as to how I can
> make this code run faster.> def scramble(fname)
> f = File.new(fname, "rb")
> _fname = fname + ".scrambled"
> begin
> File.exist?(_fname) if File.delete(_fname)
> rescue
> end
> ff = File.new(_fname, "wb+")
> for l in f
> l.each_byte{|c| ff.write((c | 0x80).chr) }
> end
> f.close()
> ff.close()
> endIf you actually want speed, use a compiled language.
This FreeBasic code is about 160 times as fast and
demonstrates that using Python for any reason
whatsoever is unjustified.Use Ruby when ease of programming is paramount;
use something like FreeBasic or LuaJIT when speed
is paramount. Never submit to Python perversion.sub scramble( filename as string )
dim outname as string
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byteoutname = filename & ".scrambled"
in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open outname for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close
end sub- Hide quoted text -- Show quoted text -
Brad Phelan wrote:
Ruby Maniac wrote:
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
endIf you google for
ps2QrAMAAAA6_jCuRt2JEIpn5Otqf_w0
ps2QrAMAAAA6_jCuRt2JEIpn5Otqf_w0 - Google Search
which is the google posting-account number for "Ruby Maniac" in his headers you find this guy is a profi troll. Some of his posts to other groups are racist, sexist and highly offensive.
I don't think that's unique, is it? For example:
http://rubyurl.com/1VW
--
Alex
Nobuyoshi, you are holy!
Arlen
On Sat, 2007-09-22 at 17:38 +0900, Nobuyoshi Nakada wrote:
Hi,
At Fri, 21 Sep 2007 07:05:10 +0900,
Ruby Maniac wrote in [ruby-talk:270110]:
> def scramble(fname)
> _fname = fname + ".scrambled"
File.new(fname, "rb") do |f|
File.new(_fname, "wb+") do |ff|
ff.write(f.read.tr("\0-\177", "\200-\377"))
end
end
> end
OT, but, ... what's a "profi troll"? I Googled it but found nothing
useful.
Regards,
Jeremy Henty
On 2007-09-26, Brad Phelan <phelan@tttech.ttt> wrote:
... this guy is a profi troll.
Here is the fastest Ruby code I have been able to develop so far.
Note the use of the Hash caused the Ruby code to run 14% slower than
when the Hash was not used.
Also it is worth noting that I was able to whip-up some Python code
that runs this same problem in 2.278 secs using Python 2.5.1 and this
makes Python almost 30x faster than Ruby for this problem.
$h = Hash.new
(0..127).select {|x| $h[x.chr] = ((x & 0x7f) | 0x80).chr}
def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
def _scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write($h[c]) }
end
f.close()
ff.close()
end
The slowest section of code is iteration of each_byte where Ruby
spends 50% of its time. Python spends 67% of its time translating
characters.
Also the Python community was very quick to help me out given the same
amount of time with this problem.
After this exorcise I can easily see why Ruby is relegated to the
kinds of problems people are willing to use Ruby for versus the far
more intense set of problem domains in which Python can be more easily
used.
Don't take me wrong, I neither love nor hate any languages I encounter
- I simply perform benchmarks and choose the fastest of the breed and
then I tend to use whichever languages or tools perform better. I find
it easier to meet customer expectations when I begin with a set of
tools that already runs reasonably fast. For some this means Ruby
runs reasonably fast however for me this means Ruby comes in dead-last
and therefore may not be worth using versus other languages or tools
that benchmark faster than Ruby does.
On the other hand, I am also waiting for Ruby 1.9.0 and YARV to see if
Ruby can be made to perform better than Ruby 1.8.6. Having said this
Ruby 1.9.0 has been in the works for about 18 months by now with no
real date other than Nov 2007 but Ruby 1.9.0 was supposed to be
release sometime during 2006 also. Sadly for me, or happily depending
on where you stand on this issue - even if Ruby were made to run
faster than Python but it lacked a byte-code compiler so I don't have
to ship source code I would still spend my time coding Python along
with the D Language just so I don't have to ship source code.
Ruby is ideal for those who don't care about performance or for Open
Source Projects where source code will be shipped - all other uses
Python will be faster.
On Sep 20, 7:16 pm, John Joyce <dangerwillrobinsondan...@gmail.com> wrote:
On Sep 20, 2007, at 8:50 PM, Brian Adkins wrote:
> On Sep 20, 6:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
>> I am new to Ruby and I could use some expert advice as to how I can
>> make this code run faster.> Write a C extension. Seriously, Ruby is pretty darn slow at iterating
> over each byte of a file. The I/O is reasonable, but the byte
> iteration is terrible. You might get a small speedup by using
> File#read instead of File#each_byte:> while (buffer = f.read(BUF_SIZE))
> buffer.each_byte {|b| ... }
> end> I got about a 5% speedup using an 8K buffer.
> You also may want to consider the idiom:
> File.open(filename) do |f|
> ...
> end> This will automatically close the file at the end of the block.
>> def scramble(fname)
>> f = File.new(fname, "rb")
>> _fname = fname + ".scrambled"
>> begin
>> File.exist?(_fname) if File.delete(_fname)> You may want to double check the previous line.
>> rescue
>> end
>> ff = File.new(_fname, "wb+")
>> for l in f
>> l.each_byte{|c| ff.write((c | 0x80).chr) }> So you don't need to get the original file data back, right? If you
> do, you may want to use xor instead, and if you don't, then why not
> just write zeros?>> end
>> f.close()
>> ff.close()
>> endBenchmark it so you can find the bottlenecks.- Hide quoted text -
- Show quoted text -
Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.
I'm only just now reading about Psyco. But the speed gains described on the introduction page[1] seem about what one would
expect if similar techniques were used with Ruby:
"For common code, expect at least a 2x speed-up, more typically 4x.
But where Psyco shines is when running algorithmical code ..."
"The difference with the traditional approach to JIT compilers is
that Psyco writes several version of the same blocks (a block is a
bit of a function), which are optimized by being specialized to
some kinds of variables (a "kind" can mean a type, but it is more
general)."
So it could compile specialized versions of (portions of) a method
that would be called if the arguments are Floats or Fixnums.
I would imagine such a technique could work with Ruby, provided
one agrees to disallow various Numeric methods from being redefined.
For example, Ruby allows things like this:
class Fixnum
alias_method :old_times, :*
def *(other)
warn("i'm thinking: #{self} * #{other}")
old_times(other)
end
end
x = 6; y = 7
x * y
i'm thinking: 6 * 7
=> 42
I'd suppose if one is willing to give up the ability to modify
a certain set of Fixnum, Float, and perhaps String methods, one
could make the equivalent of Psyco for Ruby.
Indeed, there are a couple of already existing projects in
this general category:
http://rubyforge.org/projects/ruby2c/
http://rubyforge.org/projects/ruby2cext/
Regards,
Bill
From: "Ruby Maniac" <raychorn@hotmail.com>