I am new to Ruby. I am somewhat surprised that I was not able to find
much info on this question via google or searching of forums. I was
able to find a similar question posted a while back but no one answered
it. Perhaps threads aren't such an important topic in Ruby culture?
Anyway, to the point. I'd like to use green threads (NOT kernel
threads) in a Ruby program I'm writing. There is no reason for me to
use kernel threads, and every reason to use green ones.
Are green threads still available? If so, how do I access/use them
(Thread class says nothing about whether it is green or kernel - I
suspect kernel)? If not, why not?
If not available, are there any alternatives? I checked into fibers,
but they are not appropriate for what I want because if an exception
happens, it happens in the "main thread" I am executing, which I don't
want.
If jacques1 is right, then there are no green threads, just kernel
threads, and I'm out of luck (anyone know why they would do away with
green threads instead of keeping them and also adding kernel threads?).
I did a quick-and-dirty test here (ruby 1.9.3) to try to shed some
light. My assumption is that green threads should be incredibly cheap
to create (on the order of at least thousands per second), where as
kernel threads should be much, much slower.
(1..5000).each {Thread.new{}}
This takes around 12 seconds to execute on my machine. It seems far too
slow for green threads (I can create hundreds of thousands of green
processes in erlang per second). However, someone out there correct me
if I am wrong, but I would also expect kernel threads to be much slower
than the above, i.e. on the order of a few per second. I coded up a TCL
version of the same above, which I know uses kernel threads. It only
spawned a couple heavyweights per second.
I guess the results are indeterminate? Hopefully someone else will
chime in who knows definitively.
If you want to use 1.9, you're out of luck. That said, you aren't stating
your reasoning for why you would prefer green threads over native ones.
Native threads have a longer creation time and use marginally more memory
(they use ~20kB per thread last I measured), however they generally perform
better, particularly around I/O. On platforms like Linux scheduling is O(1)
so there's really no reason you can't create large numbers of threads.
There are definitely many reasons to prefer native threads over green
threads. This is the reason why you have seen a shift from green threads to
native threads in MRI, Rubinius, and the JVM.
Why do you want green threads?
···
On Wed, Jul 4, 2012 at 3:57 AM, rex goxman <lists@ruby-forum.com> wrote:
I'd like to use green threads (NOT kernel threads) in a Ruby program I'm
writing. There is no reason for me to
use kernel threads, and every reason to use green ones.
<<There are definitely many reasons to prefer native threads over green
threads. This is the reason why you have seen a shift from green threads
to
native threads in MRI, Rubinius, and the JVM.
Why do you want green threads?>>
There are many situations to prefer green threads over kernel
threads. It's why I'm somewhat surprised that Ruby is just ditching
green threads, vs. keeping them and deploying kernel threads along side
them. Perhaps it's a cultural thing. Rubyists might be more concerned
with things like web servers (Rails development) and what not, and
perhaps haven't found want or need for green threads so much (just a
guess).
One reason to prefer green threads is if your application doesn't need
speed up through concurrency/parallelism, rather you want a threaded
solution for algorithmic purposes, convenience purposes, or other
purposes. This is a crappy example, but say you need to do some
calculation which might fail or even crash, but you don't want (or need)
to check the inputs to the calculation, or wrap the thing up in an error
catching mechanism, or anything of this nature. You just want to
calculate it, and if it works, fine, and if it doesn't (or even crashes)
- fine, your main program continues to run. So spawn a green thread to
run the calculation. If it works, great. If it crashes, who cares?
You might say "do that with a kernel thread." But what if I want to
spawn this thread over and over again in a tight loop? I don't want or
need the monstrous overhead of a kernel thread here. I just want a
green thread.
Again, there's a whole host of situations where one might want to have
green threads vs. kernel threads. But I wasn't here to debate that - if
Rubyists want to throw green threads in the garbage, that's their
prerogative. I was just here to ask whether there were green threads or
not (in 1.9.*).
rex, why are you still here? You don't want to tell your actual reasons
for using green threads, you don't want to hear alternative solutions,
and the responses seem to annoy you.
Then what's the point of this whole discussion? The original question
seems to be answered by now.
Anyway, to the point. I'd like to use green threads (NOT kernel
threads) in a Ruby program I'm writing. There is no reason for me to
use kernel threads, and every reason to use green ones.
You could try linking Ruby with GNU pth (or GNU nPth). I think pth
comes with pthreads stubs, so maybe you can get that working (and
report back your experiences
I suspect pth will conflict with Fibers, however.
If not available, are there any alternatives? I checked into fibers,
but they are not appropriate for what I want because if an exception
happens, it happens in the "main thread" I am executing, which I don't
want.
I know you don't like fibers, but the "neverblock" gem uses Fibers
transparently under 1.9 to provide green threads. However, I'm
not sure how active "neverblock" development is these days.
If jacques1 is right, then there are no green threads, just kernel
threads, and I'm out of luck (anyone know why they would do away with
green threads instead of keeping them and also adding kernel threads?).
I did a quick-and-dirty test here (ruby 1.9.3) to try to shed some
light. My assumption is that green threads should be incredibly cheap
to create (on the order of at least thousands per second), where as
kernel threads should be much, much slower.
(1..5000).each {Thread.new{}}
This takes around 12 seconds to execute on my machine. It seems far too
slow for green threads (I can create hundreds of thousands of green
processes in erlang per second). However, someone out there correct me
if I am wrong, but I would also expect kernel threads to be much slower
than the above, i.e. on the order of a few per second. I coded up a TCL
version of the same above, which I know uses kernel threads. It only
spawned a couple heavyweights per second.
I guess the results are indeterminate? Hopefully someone else will
chime in who knows definitively.
Ruby 1.9 uses kernel threads and does not have green threads.
There is nothing like Erlang's lightweight processes in Ruby (or, for
that matter, in most of the languages).
This is a crappy example, but say you need to do some
calculation which might fail or even crash, but you don't want (or need)
to check the inputs to the calculation, or wrap the thing up in an error
catching mechanism, or anything of this nature. You just want to
calculate it, and if it works, fine, and if it doesn't (or even crashes)
- fine, your main program continues to run. So spawn a green thread to
run the calculation. If it works, great. If it crashes, who cares?
You want threads as a way to silently swallow exceptions? o_O I'm going to
go out on a limb and say that's a terrible application for threads...
You might say "do that with a kernel thread." But what if I want to
spawn this thread over and over again in a tight loop? I don't want or
need the monstrous overhead of a kernel thread here. I just want a
green thread.
If the time it takes to spawn a thread is really too much, use a thread
pool.
···
On Thu, Jul 5, 2012 at 6:38 AM, rex goxman <lists@ruby-forum.com> wrote:
Are you saying that green threads are used regardless of what is
happening with kernel threads and the GIL? The thread posted by fxn
seems to indicate this.
Ruby 1.9 uses kernel threads and does not have green threads.
There is nothing like Erlang's lightweight processes in Ruby (or, for
that matter, in most of the languages).
That is clear, but the definition of green thread depends on who schedules
them.
Is the GIL acting as a scheduler? Has MRI logic about what to lock next
therefore scheduling in practice *among the threads in its process*? Or is
my conjecture right that locking is mostly dumb and that ultimately the
kernel is the scheduler because the thread is native?
···
On Wed, Jul 4, 2012 at 2:30 PM, Peter Zotov <whitequark@whitequark.org>wrote:
There is nothing like Erlang's lightweight processes in Ruby (or, for
that matter, in most of the languages).
Well, for the purposes here, if Ruby's threads were green threads, yes
they would have been comparable to Erlang green processes. My emphasis
is far more on the "green" and far less on whether it is a process or a
thread.
As far as the definition of green and whether it has anything to do with
a scheduler or not, my definition of green is simply whether or not it
is a 'real' thread or simply a virtual thread which lives inside of the
vm.
Couldn't a similar design be achieved by using Fibers and a Thread pool that grabs work from a list of ready Fibers?
···
Am 05.07.2012 um 18:34 schrieb Tony Arcieri <tony.arcieri@gmail.com>:
On Thu, Jul 5, 2012 at 6:38 AM, rex goxman <lists@ruby-forum.com> wrote:
You might say "do that with a kernel thread." But what if I want to
spawn this thread over and over again in a tight loop? I don't want or
need the monstrous overhead of a kernel thread here. I just want a
green thread.
If the time it takes to spawn a thread is really too much, use a thread pool.
Are you saying that green threads are used regardless of what is
happening with kernel threads and the GIL? The link posted by fxn
seems to indicate this.
This looks like a mistake. Ruby 1.8 has green threads, Ruby 1.9 and
JRuby have native threads (though Ruby 1.9 has a GIL).
Are you saying that green threads are used regardless of what is
happening with kernel threads and the GIL? The thread posted by fxn
seems to indicate this.
I don't know enough to give a definitive answer (or plainly put, to
answer), but here's my interpretation of the post:
I believe the green threads in Ilya's post are the Ruby-land thread
objects, they are not necessarily green in the sense of who is doing the
scheduling. This is my interpretation a posteriori because of the way the
JVM is drawn.
They are definitely green threads in MRI 1.8 in the sense that MRI did
scheduling (you already know that).
Then, behind the scenes, they are mapped to native threads in MRI 1.9 and
JRuby. One-to-one. In the case of MRI there's a GIL. I guess technically a
GIL does not mean that MRI is doing the scheduling, it means MRI is holding
a lock, but the kernel schedules the native thread (because it is native).
Maybe there's a gray area in the definition of green thread here, I don't
know.
That is my interpretation, but I don't know enough to be certain. Please
someone correct me if that is wrong!
···
On Wed, Jul 4, 2012 at 1:45 PM, rex goxman <lists@ruby-forum.com> wrote:
On Wed, Jul 4, 2012 at 2:30 PM, Peter Zotov > <whitequark@whitequark.org>wrote:
Ruby 1.9 uses kernel threads and does not have green threads.
There is nothing like Erlang's lightweight processes in Ruby (or, for
that matter, in most of the languages).
That is clear, but the definition of green thread depends on who schedules
them.
Is the GIL acting as a scheduler? Has MRI logic about what to lock next
therefore scheduling in practice *among the threads in its process*? Or is
my conjecture right that locking is mostly dumb and that ultimately the
kernel is the scheduler because the thread is native?
The OP's problem is not the scheduler itself but the associated system
resources.
You can create 20k Erlang processes ("threads"). You cannot create 20k
pthreads. Just that.
