Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.
Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.
The default for the OS scheduler (even on Windows) is to distribute
runnable processes across all processors. Did you accidentally set
affinity to a single CPU?
Any ideas?!
What makes you sure it's a CPU issue? For example, a common
bottleneck is the file system. So if all your Ruby programs are IO
intensive and read / write on the same FS they will likely suffer from
this bottleneck.
You could easily check what the OS does by running a CPU only script,
e.g. on a bash prompt you can do a 30 second test with this:
for i in 1 2 3 4; do ruby -e "t=Time.now + 30; 1 while Time.now < t;
puts 'done $i'"& done
Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.
Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.
The default for the OS scheduler (even on Windows) is to distribute
runnable processes across all processors. Did you accidentally set
affinity to a single CPU?
Any ideas?!
What makes you sure it's a CPU issue? For example, a common
bottleneck is the file system. So if all your Ruby programs are IO
intensive and read / write on the same FS they will likely suffer from
this bottleneck.
You could easily check what the OS does by running a CPU only script,
e.g. on a bash prompt you can do a 30 second test with this:
for i in 1 2 3 4; do ruby -e "t=Time.now + 30; 1 while Time.now < t;
puts 'done $i'"& done
Kind regards
robert
--
remember.guy do |as, often| as.you_can - without end
Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.
Any ideas?!
On Windows 2003 or later, you can manually set CPU affinity using
start utility in the command prompt.
Probably. This depends on your network connectivity, how MySQL is
configured (connection limits), what the IO bandwidth of MySQL's
filesystem is and how MySQL is implemented (single threaded vs. multi
threaded etc.).
Looks like you have fallen into a classical performance optimization
scenario. Now you'll have to find out where time is spent...
Kind regards
robert
···
2008/10/15 Bryan Richardson <btricha@gmail.com>:
Thanks for the reply Robert. I'm accessing a MySQL database from the
scripts rather than the file system. Would this have the same
bottleneck?
--
remember.guy do |as, often| as.you_can - without end
Well, I'm not sure the 'start /affinity' approach is fixing my problem
either... the applications still seem to trade off computing time.
This might be for a couple of reasons:
1) Does Ruby try to be *smart* and consolidate all Ruby processes in one thread?
2) My Ruby applications are starting up and making calls to a Windows
application via an OLE interface, so maybe the referenced Windows
applications are running on the same processor.
Suggestions?
···
--
Thanks!
Bryan
On Wed, Oct 15, 2008 at 8:28 AM, Robert Klemme <shortcutter@googlemail.com> wrote:
2008/10/15 Bryan Richardson <btricha@gmail.com>:
Thanks for the reply Robert. I'm accessing a MySQL database from the
scripts rather than the file system. Would this have the same
bottleneck?
Probably. This depends on your network connectivity, how MySQL is
configured (connection limits), what the IO bandwidth of MySQL's
filesystem is and how MySQL is implemented (single threaded vs. multi
threaded etc.).
Looks like you have fallen into a classical performance optimization
scenario. Now you'll have to find out where time is spent...
Kind regards
robert
--
remember.guy do |as, often| as.you_can - without end
Well, I'm not sure the 'start /affinity' approach is fixing my problem
either... the applications still seem to trade off computing time.
Well, what did you expect? Even if you set affinity, OS scheduling still takes place. With other processes running you might actually negatively impact the whole system by restricting processor affinity.
This might be for a couple of reasons:
1) Does Ruby try to be *smart* and consolidate all Ruby processes in one thread?
No. Why should that be smart anyway (keep in mind that Ruby uses green threads)?
2) My Ruby applications are starting up and making calls to a Windows
application via an OLE interface, so maybe the referenced Windows
applications are running on the same processor.
You disclose details about your application piecemeal. It's difficult to analyze this remotely anyway but it's impossible when you do not know what other facts are missing.
Suggestions?
Keep on looking. Did you try how your OS scheduling works with the example I gave you earlier?
I didn't really know what to expect... I'm not a low-level OS guy so I
don't really understand how all that stuff works -- hence my
questions. I figured it was worth a shot, tried it, and commented on
my results.
I should have put the word smart in quotes before -- it was meant as
sarcasm. I don't think that would be the smart way to go, but I have
no clue how Ruby is implemented on Windows. I understand Ruby uses
green threads, but I'm not sure if that's from process to process or
machine-wide. Looks like it's from process to process based on your
answer of 'no' below.
I apologize for disclosing information piecemeal. It wasn't
intentional -- I just failed to think about the other application when
asking my question originally. As for the example you gave me to try
earlier... I have not tried it yet (I tried the easy one first). I'll
shut my trap now until I've heeded your advice.
···
--
Bryan
On Wed, Oct 15, 2008 at 3:14 PM, Robert Klemme <shortcutter@googlemail.com> wrote:
On 15.10.2008 18:29, Bryan Richardson wrote:
Well, I'm not sure the 'start /affinity' approach is fixing my problem
either... the applications still seem to trade off computing time.
Well, what did you expect? Even if you set affinity, OS scheduling still
takes place. With other processes running you might actually negatively
impact the whole system by restricting processor affinity.
This might be for a couple of reasons:
1) Does Ruby try to be *smart* and consolidate all Ruby processes in one
thread?
No. Why should that be smart anyway (keep in mind that Ruby uses green
threads)?
2) My Ruby applications are starting up and making calls to a Windows
application via an OLE interface, so maybe the referenced Windows
applications are running on the same processor.
You disclose details about your application piecemeal. It's difficult to
analyze this remotely anyway but it's impossible when you do not know what
other facts are missing.
Suggestions?
Keep on looking. Did you try how your OS scheduling works with the example
I gave you earlier?
I apologize for disclosing information piecemeal. It wasn't
No problem. I just wanted to explain my difficulties in coming up with better replies.
intentional -- I just failed to think about the other application when
asking my question originally. As for the example you gave me to try
earlier... I have not tried it yet (I tried the easy one first). I'll
shut my trap now until I've heeded your advice.
If you really want to dig through to the hear of the issue this won't be easy. You basically have a distributed application involving at least Ruby, Windows program(s) connected via OLE and MySQL. For a start you could try to profile the Ruby program (use -r profile) but chances are good that most time is spent outside your Ruby program. Good luck!
