# Ruby on Unix vs. Windows **URL:** \ **Category:** ruby-talk\ **Created:** [16 August 2005 21:00 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204 "2005-08-16T21:00:33Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Rick\_Nooner](https://avatars.discourse-cdn.com/v4/letter/r/f08c70/32.png) [@Rick\_Nooner](https://rubytalk.org/u/Rick_Nooner)\ **Post date:** [16 August 2005 21:00 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/1 "2005-08-16T21:00:33Z")
Yesterday at work we took an analysis program written in ruby that we had been running on a Solaris box (Sunblade 1500, 1 Gig RAM, 1.5 Ghz Sparc) and moved it to a windows box (HP D530, 1 Gig RAM, 2.8 Ghz Pentium) to do performance comparisons. The analysis builds a profile in memory of over 3.6 GB of data on disk. On the Solaris box, it takes about 35 mins and uses about 700 MB of RAM. It would not complete on the windows box using the full data set, bombing with "failed to allocate memory (NoMemoryError)". There was nearly 800 MB of RAM free on the windows box as well as having a 4 Gig swap available. Is windows that inefficient with memory allocation or is this a ruby implementation issue on windows? Before anyone asks, I would have done the comparison on a linux box if I had one available since what I really wanted to know was the speed boost that a Intel processor would give me. I didn't have one readly available, though. Thanks, Rick > **···** > > -- > Rick Nooner > [rick@nooner.net](mailto:rick@nooner.net) > [http://www.nooner.net](http://www.nooner.net) ---
**Author:** ![Bill\_Kelly](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@Bill\_Kelly](https://rubytalk.org/u/Bill_Kelly)\ **Post date:** [16 August 2005 21:53 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/2 "2005-08-16T21:53:15Z")
> Yesterday at work we took an analysis program written in ruby that we had been > running on a Solaris box (Sunblade 1500, 1 Gig RAM, 1.5 Ghz Sparc) and moved > it to a windows box (HP D530, 1 Gig RAM, 2.8 Ghz Pentium) to do performance > comparisons. > > The analysis builds a profile in memory of over 3.6 GB of data on disk. On > the Solaris box, it takes about 35 mins and uses about 700 MB of RAM. It > would not complete on the windows box using the full data set, bombing with > "failed to allocate memory (NoMemoryError)". There was nearly 800 MB of > RAM free on the windows box as well as having a 4 Gig swap available. > > Is windows that inefficient with memory allocation or is this a ruby > implementation issue on windows? Any chance the program does any large block allocations? Like more than 250 MB in one chunk? I've noticed windows performs poorly when a program churns lots of small memory allocations, then occasionally wants to grab a big block of memory. I have a test program (two, actually - one in ruby, the other in C using malloc) that doesn't fail on Linux and Darwin, and does fail on Windows. Even though the Linux and Darwin systems had less physical RAM and less swap than the Windows system with its 2GB ram and 4GB swap. When it fails on Windows, there's plenty of system memory still available - but the windows heap management has allowed the process virtual memory space to become so fragmented that there's no room to map the large block allocation into the process' virtual address space. Regards, Bill > **···** > > From: "Rick Nooner" \ ---
**Author:** ![TLOlczyk](https://avatars.discourse-cdn.com/v4/letter/t/9de053/32.png) [@TLOlczyk](https://rubytalk.org/u/TLOlczyk)\ **Post date:** [16 August 2005 23:56 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/3 "2005-08-16T23:56:14Z")
Look around for a Live CD distribution like Knoppix. But find one without ruby. Set aside a partition for Linux ( it can be Fat32 ) mount the partition somewhere and install ruby ( you DON'T want ruby running of the CD for benchmarks even hand waiving ones ). The hard part will be finding a Live CD without ruby. It's been a while, but I seem to remember that Intel CPUs make memory management for things like GC much harder, so it would be a good idea to compare OSs running on the same CPU. If not maybe you can scrounge an Alpha running NT and see if you get any different results on it. The reply-to email address is olczyk2002@yahoo.com. This is an address I ignore. To reply via email, remove 2002 and change yahoo to interaccess, > **···** > > On Wed, 17 Aug 2005 06:00:33 +0900, Rick Nooner \ wrote: > > > Yesterday at work we took an analysis program written in ruby that we had been > > running on a Solaris box (Sunblade 1500, 1 Gig RAM, 1.5 Ghz Sparc) and moved > > it to a windows box (HP D530, 1 Gig RAM, 2.8 Ghz Pentium) to do performance > > comparisons. > > > > The analysis builds a profile in memory of over 3.6 GB of data on disk. On > > the Solaris box, it takes about 35 mins and uses about 700 MB of RAM. It > > would not complete on the windows box using the full data set, bombing with > > "failed to allocate memory (NoMemoryError)". There was nearly 800 MB of > > RAM free on the windows box as well as having a 4 Gig swap available. > > > > Is windows that inefficient with memory allocation or is this a ruby > > implementation issue on windows? > > > > Before anyone asks, I would have done the comparison on a linux box if I > > had one available since what I really wanted to know was the speed boost > > that a Intel processor would give me. I didn't have one readly available, > > though. > > \*\* > Thaddeus L. Olczyk, PhD > > There is a difference between > \*thinking\* you know something, > and \*knowing\* you know something. ---
**Author:** ![Tesla](https://avatars.discourse-cdn.com/v4/letter/t/7cd45c/32.png) [@Tesla](https://rubytalk.org/u/Tesla)\ **Post date:** [17 August 2005 19:55 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/4 "2005-08-17T19:55:30Z")
Rick Nooner wrote: > Yesterday at work we took an analysis program written in ruby that we had been > running on a Solaris box (Sunblade 1500, 1 Gig RAM, 1.5 Ghz Sparc) and moved > it to a windows box (HP D530, 1 Gig RAM, 2.8 Ghz Pentium) to do performance > comparisons. > > The analysis builds a profile in memory of over 3.6 GB of data on disk. On > the Solaris box, it takes about 35 mins and uses about 700 MB of RAM. It > would not complete on the windows box using the full data set, bombing with > "failed to allocate memory (NoMemoryError)". There was nearly 800 MB of > RAM free on the windows box as well as having a 4 Gig swap available. > > Is windows that inefficient with memory allocation or is this a ruby > implementation issue on windows? > > Before anyone asks, I would have done the comparison on a linux box if I > had one available since what I really wanted to know was the speed boost > that a Intel processor would give me. I didn't have one readly available, > though. > > Thanks, > Rick My only experience with this is through SQL Server 2000. Large processes were not completing even though the machince had plenty of memory. Our theory was that it was a combination of NTFS and deframentation of the swap file. It seemed as though NTFS security was trying to keep track of everything and using even more memory. The swap file also could not respond fast enough to the calls because SQL Server was busy on the same disks. What we did was take and old 9gb scsi disk with FAT32 and set the swap file on to it. Not a perfect solution but it worked. > **···** > > -- > > Tesla - ---
**Author:** ![Rick\_Nooner](https://avatars.discourse-cdn.com/v4/letter/r/f08c70/32.png) [@Rick\_Nooner](https://rubytalk.org/u/Rick_Nooner)\ **Post date:** [16 August 2005 22:01 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/5 "2005-08-16T22:01:11Z")
You are describing exactly the behavior that I'm seeing. My program does a lot of small block allocations. How that's handled by Ruby under the hood, I don't know. If the allocator see's lots of small allocations so starts asking the OS for larger and larger chunks of memory, what you are describing could happen. If this has happened to you using malloc, it's probably beyond Ruby's control, unless Ruby replaces the default memory allocation mechanism. I'll write a test program to test this theory. Thanks, Rick > **···** > > On Wed, Aug 17, 2005 at 06:53:15AM +0900, Bill Kelly wrote: > > > Any chance the program does any large block allocations? > > Like more than 250 MB in one chunk? I've noticed windows > > performs poorly when a program churns lots of small > > memory allocations, then occasionally wants to grab a big > > block of memory. I have a test program (two, actually - > > one in ruby, the other in C using malloc) that doesn't > > fail on Linux and Darwin, and does fail on Windows. Even > > though the Linux and Darwin systems had less physical > > RAM and less swap than the Windows system with its 2GB > > ram and 4GB swap. When it fails on Windows, there's > > plenty of system memory still available - but the windows > > heap management has allowed the process virtual memory > > space to become so fragmented that there's no room to map > > the large block allocation into the process' virtual > > address space. > > -- > Rick Nooner > rick@nooner.net > > > **[Nooner Family Coming Soon...](https://www.nooner.net/)** > > > > The Nooner Family Website is coming soon... ---
**Author:** ![Joel\_VanderWerf1](https://avatars.discourse-cdn.com/v4/letter/j/94ad74/32.png) [@Joel\_VanderWerf1](https://rubytalk.org/u/Joel_VanderWerf1)\ **Post date:** [16 August 2005 22:13 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/6 "2005-08-16T22:13:05Z")
Rick Nooner wrote: > If this has happened to you using malloc, it's probably beyond Ruby's control, unless > Ruby replaces the default memory allocation mechanism. What about testing with cygwin or mingw32 ruby? I'm guessing those use different allocators than win32 ruby. > **···** > > -- >       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407 ---
**Author:** ![Russell\_Mok](https://avatars.discourse-cdn.com/v4/letter/r/b38774/32.png) [@Russell\_Mok](https://rubytalk.org/u/Russell_Mok)\ **Post date:** [16 August 2005 23:28 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/7 "2005-08-16T23:28:22Z")
Another option is SFU(www.interix.com). I believe there is a Ruby 1.8.2 binary package. I don't even have to worry about / or \, : or ; once I'm in the SFU environment. Joel VanderWerf wrote: > **···** > > > Rick Nooner wrote: > > > > \> If this has happened to you using malloc, it's probably beyond Ruby's control, unless > > \> Ruby replaces the default memory allocation mechanism. > > > > What about testing with cygwin or mingw32 ruby? I'm guessing those use > > different allocators than win32 ruby. > > > > -- > >       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407 ---
**Author:** ![Robert9](https://avatars.discourse-cdn.com/v4/letter/r/fbc32d/32.png) [@Robert9](https://rubytalk.org/u/Robert9)\ **Post date:** [16 August 2005 23:41 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/8 "2005-08-16T23:41:14Z")
Joel VanderWerf wrote: > Rick Nooner wrote: > > > If this has happened to you using malloc, it's probably beyond Ruby's control, unless > > Ruby replaces the default memory allocation mechanism. > > What about testing with cygwin or mingw32 ruby? I'm guessing those use > different allocators than win32 ruby. I would be curious to see if mingw32 has the same problems (remembering the big mingw vs msvc debates of yesteryear). As a Windows user this is a worrisome thing to me. If it is a "known" problem then is it going to be fixed at some point in time? Robert ---
**Author:** ![Bill\_Kelly](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@Bill\_Kelly](https://rubytalk.org/u/Bill_Kelly)\ **Post date:** [16 August 2005 23:55 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/9 "2005-08-16T23:55:01Z")
> As a Windows user this is > a worrisome thing to me. If it is a "known" problem then is it going to > be fixed at some point in time? I encountered this problem a year or so ago working on my employer's application which is all C / C++. Our customers sometimes browse very large images, upwards of 200 or 300 megs, which our software will attempt to load entirely into memory if requested to do so. When we contacted Mircosoft about it, they referred us to their Low Fragmentation Heap: [http://support.microsoft.com/kb/816542](http://support.microsoft.com/kb/816542) I haven't tried it yet - and I'd also be interested to learn more about any "drop in replacements" that Lothar hinted at. Regards, Bill > **···** > > From: "Robert" \ ---
**Author:** ![Lothar\_Scholz2](https://avatars.discourse-cdn.com/v4/letter/l/f14d63/32.png) [@Lothar\_Scholz2](https://rubytalk.org/u/Lothar_Scholz2)\ **Post date:** [17 August 2005 01:17 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/10 "2005-08-17T01:17:44Z")
Hello Bill, > I haven't tried it yet - and I'd also be interested to > learn more about any "drop in replacements" that Lothar > hinted at. For example take this one [http://www.nedprod.com/programs/Win32/ptmalloc2/index.html](http://www.nedprod.com/programs/Win32/ptmalloc2/index.html) > **···** > > -- > Best regards, emailto: scholz at scriptolutions dot com > Lothar Scholz [http://www.ruby-ide.com](http://www.ruby-ide.com) > CTO Scriptolutions Ruby, PHP, Python IDE 's ---
**Author:** ![Alex2](https://avatars.discourse-cdn.com/v4/letter/a/45deac/32.png) [@Alex2](https://rubytalk.org/u/Alex2)\ **Post date:** [17 August 2005 01:38 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/11 "2005-08-17T01:38:11Z")
Lothar Scholz wrote: > Hello Bill, > > > I haven't tried it yet - and I'd also be interested to > > learn more about any "drop in replacements" that Lothar > > hinted at. > > For example take this one > [http://www.nedprod.com/programs/Win32/ptmalloc2/index.html](http://www.nedprod.com/programs/Win32/ptmalloc2/index.html) Oh my! That's ptmalloc on Windows, isn't it?! That little thing seriously rocks in multi-threaded systems with a lot of dynamic memory allocation. It can resolve nasty scalability bottlenecks there by getting rid of a mutex in malloc/free. Not sure if it will do much good for Ruby though - Ruby threads are green, not native. If you get some results, please share with the list. Best regards, Alexey Verkhovsky ---
**Author:** ![Lothar\_Scholz2](https://avatars.discourse-cdn.com/v4/letter/l/f14d63/32.png) [@Lothar\_Scholz2](https://rubytalk.org/u/Lothar_Scholz2)\ **Post date:** [17 August 2005 01:46 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/12 "2005-08-17T01:46:44Z")
Hello Alex, > Oh my! That's ptmalloc on Windows, isn't it?! That little thing > seriously rocks in multi-threaded systems with a lot of dynamic memory > allocation. It can resolve nasty scalability bottlenecks there by > getting rid of a mutex in malloc/free. > Not sure if it will do much good for Ruby though - Ruby threads are > green, not native. Yes it will improve the runtime speed but it also has lower fragmentation then the original msvcrt.dll implementation. > **···** > > -- > Best regards, emailto: scholz at scriptolutions dot com > Lothar Scholz [http://www.ruby-ide.com](http://www.ruby-ide.com) > CTO Scriptolutions Ruby, PHP, Python IDE 's ---
**Author:** ![Lothar\_Scholz2](https://avatars.discourse-cdn.com/v4/letter/l/f14d63/32.png) [@Lothar\_Scholz2](https://rubytalk.org/u/Lothar_Scholz2)\ **Post date:** [17 August 2005 02:34 UTC](https://rubytalk.org/t/ruby-on-unix-vs-windows/20204/13 "2005-08-17T02:34:18Z")
Hello Lothar, > Yes it will improve the runtime speed but it also has lower Sorry, there is a "not" missing in this sentence. I meant: "Yes it will not improve..." > **···** > > -- > Best regards, emailto: scholz at scriptolutions dot com > Lothar Scholz [http://www.ruby-ide.com](http://www.ruby-ide.com) > CTO Scriptolutions Ruby, PHP, Python IDE 's