As Ara said, in this case processes are the better choice for several
reasons: they have separation out of the box and they can make use of
multiple cores (which Ruby threads can't unless you use JRuby - this
may change with future 1.9 versions AFAIK).
Kind regards
robert
···
2008/6/30 barjunk <barjunk@attglobal.net>:
I've been hunting around for information regarding threads, and to me,
it seems confusing and conflicting.
What I'm trying to find out is...if I was going to start using threads
in Ruby, which version of Ruby should I be using.
I've seen folks say that I should use Ruby 1.9 and others say that it
is possible to use earlier versions. Nothing that I found seemed
definitive.
I'm new to all this, so this may be part of the problem.
What I'd like to accomplish is starting a main ruby instance, then
launch threads from that instance that run in their own sandbox.
At this point, I don't believe the threads need to talk with each
other, but it seems I could use some form of message passing to
accomplish this.
--
use.inject do |as, often| as.you_can - without end
As Ara said, in this case processes are the better choice for several
reasons: they have separation out of the box and they can make use of
multiple cores (which Ruby threads can't unless you use JRuby - this
may change with future 1.9 versions AFAIK).
This is an eventual goal, but I asked ko1 about it and such work has not started yet. It will be hard.
Processes are probably better under Ruby, but it's most definitely worth trying threads under JRuby first.
well, given that the difference between processes and threads is an incredibly small one for any modern os, and given that threads are *at least* 100 harder to write deterministic code for (as your bug reports regarding exception handling and ruby illustrate) i'd hazard a guess that processes are *almost* always the correct solution when robust code is desired. in otherwords i'd take the position that one should always use processes unless the reason becomes clear to use threads and, of course ,there are indeed reasons. this is mostly a comment on the limitations of programmers and not on platforms or languages, nevertheless the incredible ease of IPC with ruby makes it even more true imho.
Now I'm doing a server in Ruby using green threads (Ruby MRI).
In the future it's possible I try to migrate it to JRuby to use real threads.
Just a question: Will my code and for now green threads work correctly under
JRuby and green threads will become OS threads magically? or must I change my
code to work with OS threads?
Thanks for any explanation.
···
El Martes, 1 de Julio de 2008, Charles Oliver Nutter escribió:
Robert Klemme wrote:
> As Ara said, in this case processes are the better choice for several
> reasons: they have separation out of the box and they can make use of
> multiple cores (which Ruby threads can't unless you use JRuby - this
> may change with future 1.9 versions AFAIK).
This is an eventual goal, but I asked ko1 about it and such work has not
started yet. It will be hard.
Processes are probably better under Ruby, but it's most definitely worth
trying threads under JRuby first.
El Lunes, 30 de Junio de 2008, Joel VanderWerf escribió:
Zhukov Pavel wrote:
> On Mon, Jun 30, 2008 at 9:04 AM, Joel VanderWerf
>
>> I haven't used 1.9 much, but the impression I get is:
>>
>> - use 1.9 if you need _native_ threads (e.g. to take advantage of
>> multiple processors, or blocking system calls)
>
> really advantage on multiply processors? Ruby 1.9 does't use GIL???
I hope you mean "for this situation". Processes are definitely not the solution to all problems.
well, given that the difference between processes and threads is an incredibly small one for any modern os, and given that threads are *at least* 100 harder to write deterministic code for (as your bug reports regarding exception handling and ruby illustrate) i'd hazard a guess that processes are *almost* always the correct solution when robust code is desired. in otherwords i'd take the position that one should always use processes unless the reason becomes clear to use threads and, of course ,there are indeed reasons. this is mostly a comment on the limitations of programmers and not on platforms or languages, nevertheless the incredible ease of IPC with ruby makes it even more true imho.
The fact that Ruby's threading has many breakages and pitfalls does not mean threading in general is the wrong way to fix things. Java threading works extremely well, with the only real requirement that you must either synchronize or avoid access to shared resources. Power...responsibility...etc. You can't damn threading because the standard implementation of Ruby doesn't do it well.
Perhaps you're right that when you only have access to green threads that processes are the right way to go, since green threads don't really gain you anything other than simulated asynchrony. But native threads done right are as good as separate processes, with the bonus that you can share fast in-memory access to resources if you're willing to accept the synchronization cost and complexity.
- Charlie
···
On Jun 30, 2008, at 3:14 AM, Charles Oliver Nutter wrote:
Now I'm doing a server in Ruby using green threads (Ruby MRI).
In the future it's possible I try to migrate it to JRuby to use real threads.
Just a question: Will my code and for now green threads work correctly under JRuby and green threads will become OS threads magically? or must I change my code to work with OS threads?
You should not need to do anything and the threads will "just be native". We do that by hobbling native threads slightly so they check for those "unsafe" events like kill, raise, and critical.
One down side to native threads is that launching a thread in JRuby is a lot more costly than in MRI, but we also have a thread pool (somewhat experimental, but people are using it) to mitigate that cost:
Normal:
user system total real
control loop 0.005000 0.000000 0.005000 ( 0.006000)
Thread.new.join loop 2.569000 0.000000 2.569000 ( 2.569000)
-J-Djruby.thread.pool.enabled=true:
user system total real
control loop 0.009000 0.000000 0.009000 ( 0.009000)
Thread.new.join loop 0.655000 0.000000 0.655000 ( 0.654000)
yeah i agree 100% in principle. however i was programming java when stopping threads suddenly became depreciated, which i know you know all about, but for others
so doing something as simple as stopping a thread can be complicated. i can kill a process and all resources will be returned to the system. the fact that sun took quite a few years to figure this out, and that matz ruby had the bugs you recently found beg the question: if matz cannot do exceptions + threads right, if sun cannot get stopping a thread right for years, what chance do i have of writing code for, say, a web server that's supposed to run 24x7? i think modern languages are caving to the reality that most (aka average) programmers simply cannot program threads safely and are increasingly moving towards the message passing paradigm ousterhout has been raving about for years.
now having said that, i very often use ruby threads but often do so in a message passing fashion and even more often use those threads to spawn processes and achieve parallelism so i definitely am glad they are there (Thread.new{ curl } is ultra powerful). still, i can't help but feel they are destined to become relics - at least in the direct fashion we use them now.
On Jun 30, 2008, at 6:55 PM, Charles Oliver Nutter wrote:
The fact that Ruby's threading has many breakages and pitfalls does not mean threading in general is the wrong way to fix things. Java threading works extremely well, with the only real requirement that you must either synchronize or avoid access to shared resources. Power...responsibility...etc. You can't damn threading because the standard implementation of Ruby doesn't do it well.
Perhaps you're right that when you only have access to green threads that processes are the right way to go, since green threads don't really gain you anything other than simulated asynchrony. But native threads done right are as good as separate processes, with the bonus that you can share fast in-memory access to resources if you're willing to accept the synchronization cost and complexity.
--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama
El Martes, 1 de Julio de 2008, Charles Oliver Nutter escribió:
Iñaki Baz Castillo wrote:
> Now I'm doing a server in Ruby using green threads (Ruby MRI).
> In the future it's possible I try to migrate it to JRuby to use real
> threads.
>
> Just a question: Will my code and for now green threads work correctly
> under JRuby and green threads will become OS threads magically? or must I
> change my code to work with OS threads?
You should not need to do anything and the threads will "just be
native". We do that by hobbling native threads slightly so they check
for those "unsafe" events like kill, raise, and critical.
One down side to native threads is that launching a thread in JRuby is a
lot more costly than in MRI, but we also have a thread pool (somewhat
experimental, but people are using it) to mitigate that cost:
Normal:
user system total real
control loop 0.005000 0.000000 0.005000 ( 0.006000)
Thread.new.join loop 2.569000 0.000000 2.569000 ( 2.569000)
-J-Djruby.thread.pool.enabled=true:
user system total real
control loop 0.009000 0.000000 0.009000 ( 0.009000)
Thread.new.join loop 0.655000 0.000000 0.655000 ( 0.654000)
The fact that Ruby's threading has many breakages and pitfalls does not mean threading in general is the wrong way to fix things. Java threading works extremely well, with the only real requirement that you must either synchronize or avoid access to shared resources. Power...responsibility...etc. You can't damn threading because the standard implementation of Ruby doesn't do it well.
Perhaps you're right that when you only have access to green threads that processes are the right way to go, since green threads don't really gain you anything other than simulated asynchrony. But native threads done right are as good as separate processes, with the bonus that you can share fast in-memory access to resources if you're willing to accept the synchronization cost and complexity.
yeah i agree 100% in principle. however i was programming java when stopping threads suddenly became depreciated, which i know you know all about, but for others
The deprecation of thread stop, suspend, and exception raising was implemented precisely because of the shared resource requirements. If you can stop a thread in an environment where it may have been using resources other threads will use, it's impossible to know if those resources have been cleaned up or released safely. Sure, you can stop a process. The Java deprecations were done because it's provably impossible to share in-process resources and safely terminate threads at will.
The same goes for shared out-of-process resources, but since it's harder to share out-of-process resources it's harder to do serious damage. You can still corrupt files, orphan processes, or leave sibling processes waiting for data that will never arrive. You can even introduce exactly the same race conditions common to threading if you want multiple processes to perform atomic mutations of shared files or memory. If you have a large interconnected app with lots of processes communicating or using shared resources, arbitrarily nuking one of them can cause exactly the same headaches. It's a factor of resource sharing and interdependency, rather than anything specific to threading over processes.
now having said that, i very often use ruby threads but often do so in a message passing fashion and even more often use those threads to spawn processes and achieve parallelism so i definitely am glad they are there (Thread.new{ curl } is ultra powerful). still, i can't help but feel they are destined to become relics - at least in the direct fashion we use them now.
Probably not, but hopefully neither will typical IPC mechanisms, which are almost as painful to get right and make reliable. Threads are a low-level API, perhaps lower-level than day-to-day programmers should generally have to deal with. But it's absurd to say that processes can do everything threads can, otherwise we'd have a massive process bloat for almost every nontrivial applications we use. Threads have a place, though the ease in which resources can be shared often makes it a dangerous place to go. Let's not throw the threading baby out with the shared resource bathwater.
- Charlie
···
On Jun 30, 2008, at 6:55 PM, Charles Oliver Nutter wrote:
The Java deprecations were done because it's provably impossible to share in-process resources and safely terminate threads at will.
<snip>
i think java is correct to have done so, precisely because people have found it too hard to write safe code using those mechanisms but as you point out, you can do the same with processes while no OS has limited us yet. why? i think it's because sharing data between threads, or processes, is both dangerous and powerful. when someone builds an OS that supports message passing we'll see those operations limited on a processes too i bet but, for now, they are just too useful despite the danger and do work 'much' of the time which, for better or worse, seems to be the MO for many programming tasks.
Probably not, but hopefully neither will typical IPC mechanisms, which are almost as painful to get right and make reliable.
in fairness we're talking about ruby here where that is definitely not true. it's extremely painless to have reliable ipc with ruby using drb or com with sqlite as a message store.
. But it's absurd to say that processes can do everything threads can, otherwise we'd have a massive process bloat for almost every nontrivial applications we use.
for the record i'm not saying that - i'm saying that processes are a better starting point for most people wanting to gain parallelism in ruby for most problems. threads are appropriate at times too but, in ruby, the disadvantages like lack of cpu migration, blocking the entire process (in win), etc limit their usefulness - jruby excepted of course (no jab, it's a huge advantage jruby has over the mri)
On Jul 1, 2008, at 6:51 AM, Charles Oliver Nutter wrote:
--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama