How do threads and join work?

ruby-talk
1

I'm reading the pickaxe and it says on p137 "When a Ruby program
terminates, all threads are killed, regardless of their states.
However, you can wait for a particular thread to finish by calling
that thread's Thread#join method. The calling thread will block until
the given thread is finished."

I wrote a small program to see if I understood how this works.

running = true
t = Thread.new do
  print "Thread started\n"
  while(running); end
  print "Thread finished\n"
end

t.join
puts "After join"
running = false
puts "Program finished"

The paragraph I quoted makes me think that it should print "Thread
started...After join...Thread finished...Program finished" It joins
the thread and continues processing, setting running to false, which
causes the thread's loop to end. Also not that it could say "Program
finished...Thread finished", I don't think there's a way to know for
sure.

Anyway, as you might guess, the actual output is "Thread started" and
it just hangs. I checked out the docs
(http://www.ruby-doc.org/core/classes/Thread.html#M001469) and it says
"Does not return until thr exits or until limit seconds have passed."

That explains why it's hanging - join is waiting for the thread to
exit. I'm confused as to the usefulness of threads in Ruby then. I
can see that if I were to create 5 threads, I can have them running
all at the same time, and then resume normal program execution.
However, more often I'd like to just perform some task in the
background while my program carries on as normal. In that case, a
Thread (to the extent that I know how to use it) is nothing more than
a method call.

Finally, I've tried putting Thread.pass inside the loop. Common sense
tells me it won't work because join still hasn't returned, but perhaps
Thread.pass is the answer if I'm not even supposed to be calling #join
in the first place.

I'd really appreciate some help in understanding this, and some
direction as to how to execute a task in the background.

Pat

2

Your thread never actually finished, because you joined on it before you set the value of running to false, so your code blocks at that point. You need to swap the last lines like this:

   running = true
   t = Thread.new do
     print "Thread started\n"
     while(running); puts "running = #{running}"; sleep 1; end
     puts "running = #{running}";
     print "Thread finished\n"
   end

   sleep 5

   puts "Before join"
   running = false
   t.join
   puts "After join"
   puts "Program finished"

I added two things into the while loop: a sleep so you don't run your CPU at 100% and a ticker.

Ashley

···

On Apr 23, 2006, at 11:06 am, Pat Maddox wrote:

running = true
t = Thread.new do
  print "Thread started\n"
  while(running); end
  print "Thread finished\n"
end

t.join
puts "After join"
running = false
puts "Program finished"

The paragraph I quoted makes me think that it should print "Thread
started...After join...Thread finished...Program finished" It joins
the thread and continues processing, setting running to false, which
causes the thread's loop to end. Also not that it could say "Program
finished...Thread finished", I don't think there's a way to know for
sure.

3

  puts "Before join"
  running = false
  t.join
  puts "After join"
  puts "Program finished"

Actually I just noticed that in my version the join is redundant. I only just woke up :slight_smile:

You would need join if you wrote something like this:

   running = true
   t = Thread.new do
     print "Thread started\n"
     #do_something
     sleep 10
     #do_something_else
     print "Thread finished\n"
   end

   puts "Before join"
   t.join
   puts "After join"
   puts "Program finished"

Hope that clears it up

Ashley

4

> running = true
> t = Thread.new do
> print "Thread started\n"
> while(running); end
> print "Thread finished\n"
> end
>
> t.join
> puts "After join"
> running = false
> puts "Program finished"
>
> The paragraph I quoted makes me think that it should print "Thread
> started...After join...Thread finished...Program finished" It joins
> the thread and continues processing, setting running to false, which
> causes the thread's loop to end. Also not that it could say "Program
> finished...Thread finished", I don't think there's a way to know for
> sure.

Your thread never actually finished, because you joined on it before
you set the value of running to false, so your code blocks at that
point.

Right, I mentioned that in my OP. I want to know if there's a way to
get join-like behavior - do not terminate the program until that
thread has finished executing - without blocking.

You need to swap the last lines like this:

   running = true
   t = Thread.new do
     print "Thread started\n"
     while(running); puts "running = #{running}"; sleep 1; end
     puts "running = #{running}";
     print "Thread finished\n"
   end

   sleep 5

   puts "Before join"
   running = false
   t.join
   puts "After join"
   puts "Program finished"

I added two things into the while loop: a sleep so you don't run your
CPU at 100% and a ticker.

This gives the output I want, but doesn't actually perform what I
want. The ultimate goal here is to basically tell my program "Go
start doing this in the background, and I'm going to keep working."

Pat

···

On 4/23/06, Ashley Moran <work@ashleymoran.me.uk> wrote:

On Apr 23, 2006, at 11:06 am, Pat Maddox wrote:

5

Sorry - missed your final point because I forgot to scroll down. In future I won't write replies first thing Sunday morning :slight_smile:

You can never join on your thread because it can only be terminated (cleanly) by setting the value of running to false

I don't see what's wrong with this (going back to your original code):

   running = true
   t = Thread.new do
     print "Thread started\n"
     while(running); end
     print "Thread finished\n"
   end

   do_work

   # Use one or: other of these depending on whether t is immortal
   running = false
   #t.join
   #puts "After join"

   puts "Program finished"

Maybe I have misunderstood whether you want to be in control of a thread that will run forever, or whether your while loop is just to simulate a long-running process?

Ashley

···

On Apr 23, 2006, at 12:33 pm, Pat Maddox wrote:

Right, I mentioned that in my OP. I want to know if there's a way to
get join-like behavior - do not terminate the program until that
thread has finished executing - without blocking.

This gives the output I want, but doesn't actually perform what I
want. The ultimate goal here is to basically tell my program "Go
start doing this in the background, and I'm going to keep working."

6

I've been using these two lines to capture text that only appears once, but the second line bothers me a little. Is there a one-line idiom for this that extracts groups like #scan but returns a string instead of an array so the second line isn't needed?

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )
text = text[ 0 ]

Thanks for any help,
Chris

7

Well this is all because I need to interact with a remote server that
sends asyncrhonous messages. I suppose the most common server that's
similar is an IRC server. You connect to the server, and then just
wait while it pumps out messages to you. The way I've implemented
this in other languages is to create a new thread which simply has an
infinite loop (checking a control condition, such as @connected) that
reads data from a socket. So in cases like this, the read call would
block until some data comes through the socket. Then it parses the
text and the loop continues. I'm trying to do the same thing with
Ruby.

Pat

···

On 4/23/06, Ashley Moran <work@ashleymoran.me.uk> wrote:

On Apr 23, 2006, at 12:33 pm, Pat Maddox wrote:

> Right, I mentioned that in my OP. I want to know if there's a way to
> get join-like behavior - do not terminate the program until that
> thread has finished executing - without blocking.
>
> This gives the output I want, but doesn't actually perform what I
> want. The ultimate goal here is to basically tell my program "Go
> start doing this in the background, and I'm going to keep working."

Sorry - missed your final point because I forgot to scroll down. In
future I won't write replies first thing Sunday morning :slight_smile:

You can never join on your thread because it can only be terminated
(cleanly) by setting the value of running to false

I don't see what's wrong with this (going back to your original code):

   running = true
   t = Thread.new do
     print "Thread started\n"
     while(running); end
     print "Thread finished\n"
   end

   do_work

   # Use one or: other of these depending on whether t is immortal
   running = false
   #t.join
   #puts "After join"

   puts "Program finished"

Maybe I have misunderstood whether you want to be in control of a
thread that will run forever, or whether your while loop is just to
simulate a long-running process?

Ashley

8

Any reason why

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is ).first

or

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )[0]

wouldn't work?

-Pawel

···

On 4/23/06, clc <clc1024@hotmail.com> wrote:

I've been using these two lines to capture text that only appears once, but
the second line bothers me a little. Is there a one-line idiom for this
that extracts groups like #scan but returns a string instead of an array so
the second line isn't needed?

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )
text = text[ 0 ]

9

I've been using these two lines to capture text that only appears once, but the second line bothers me a little. Is there a one-line idiom for this that extracts groups like #scan but returns a string instead of an array so the second line isn't needed?

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )
text = text[ 0 ]

If I understood the question correctly, here are my best ideas:

>> str = "skip this find this skip this"
=> "skip this find this skip this"
>> found = str.match(/skip this (find this) skip this/)[1]
=> "find this"
>> another_found = str[/find this/]
=> "find this"
>> yet_another_found = str[/skip this (find this) skip this/, 1]
=> "find this"

Hope that helps.

James Edward Gray II

···

On Apr 23, 2006, at 7:21 AM, clc wrote:

10

clc wrote:

I've been using these two lines to capture text that only appears once, but
the second line bothers me a little. Is there a one-line idiom for this
that extracts groups like #scan but returns a string instead of an array so
the second line isn't needed?

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )
text = text[ 0 ]

Thanks for any help,
Chris

text = string[/skip_this (keep_this) skip_this}is/, 1]

and if you don't need the skip_this,

text = string[/keep_this/]

Yoann

11

Adding to all the other replies: keep in mind that scan usually
returns multiple results which implies that you want to process all of
them.

If you know that there will be only one match of this in your string
or if you only want to work with the first match, I suggest to use =~
or String[%r{ skip_this ( keep_this ) skip_this }is, 1] as Yoann
showed.

Kind regards

robert

···

2006/4/23, clc <clc1024@hotmail.com>:

I've been using these two lines to capture text that only appears once, but
the second line bothers me a little. Is there a one-line idiom for this
that extracts groups like #scan but returns a string instead of an array so
the second line isn't needed?

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )
text = text[ 0 ]

--
Have a look: http://www.flickr.com/photos/fussel-foto/

12

Have you considered the possibility that you're either a) doing things in the wrong order or b) have your infinite loop in the wrong thread.

You can just make #join the last thing you do, that's effectively "non-blocking"

e.g.

t = Thread.start { ... }

# do other stuff in main thread

t.join # no code follows this anyway

Or if the thread should drive the running of the other stuff:

require 'mutex'
m = Mutex.new
is_running = true

t = Thread.start do
     while getting_stuff_or_whatever
         do_stuff( )
     end
     # we stopped getting stuff
     m.synchronize { is_running = false }
end

local_running = nil
m.synchronize { local_running = is_running }

while local_running
    do_other_stuff
    m.synchronize do
       unless is_running
           local_running = false
       end
    end
end

Of course this example is silly you should probably use a queue

···

On Apr 23, 2006, at 7:35 AM, Pat Maddox wrote:

Right, I mentioned that in my OP. I want to know if there's a way to
get join-like behavior - do not terminate the program until that
thread has finished executing - without blocking.

13

Ahh... I thought this was a newbie question hence patronising intial reply...

One possibility is the Queue object in the threads library:

   require 'thread'

   messages = Queue.new

   server = Thread.new do
     print "Server started\n"
     10.times do |i|
       sleep rand(5)
       messages << "Message #{i}"
     end
     messages << "Disconnect"
     print "Server finished\n"
   end

   client = Thread.new do
     while true
       message = messages.pop # blocks here
       puts "Received: #{message.inspect}"
       if message == "Disconnect"
         puts "Disconnecting from server"
         break
       end
     end
   end

   puts "Before join"
   client.join
   puts "After join"

   puts "Program finished"

This is based on the example in the RDoc

There's a load of different ways to synchronise data. My boss has my copy of the pickaxe but they are all cross referenced. Start with Monitor - it's what I used last, although that wasn't for a client-server application but for aggregating data from a group of individual threads.

Is this any help? (at last!)

Regards
Ashley

···

On Apr 23, 2006, at 1:03 pm, Pat Maddox wrote:

Well this is all because I need to interact with a remote server that
sends asyncrhonous messages. I suppose the most common server that's
similar is an IRC server. You connect to the server, and then just
wait while it pumps out messages to you. The way I've implemented
this in other languages is to create a new thread which simply has an
infinite loop (checking a control condition, such as @connected) that
reads data from a socket. So in cases like this, the read call would
block until some data comes through the socket. Then it parses the
text and the loop continues. I'm trying to do the same thing with
Ruby.

Pat

14

Well, now, that's just embarassing. Thanks!

···

----- Original Message ----- From: "Pawel Szymczykowski" <makenai@gmail.com>
To: "ruby-talk ML" <ruby-talk@ruby-lang.org>
Sent: Sunday, April 23, 2006 8:28 AM
Subject: Re: Scan-like Idiom to Capture a Group as a String Instead of an Array?

15

Hi --

I've been using these two lines to capture text that only appears once, but
the second line bothers me a little. Is there a one-line idiom for this
that extracts groups like #scan but returns a string instead of an array so
the second line isn't needed?

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )
text = text[ 0 ]

Any reason why

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is ).first

or

text = string.scan( %r{ skip_this ( keep_this ) skip_this }is )[0]

wouldn't work?

They'll do the same thing as Chris's example, though they all return
arrays rather than strings:

"abc".scan(/a(b)c/)[0]

=> ["b"]

so a further extraction would be necessary.

David

···

On Sun, 23 Apr 2006, Pawel Szymczykowski wrote:

On 4/23/06, clc <clc1024@hotmail.com> wrote:

--
David A. Black (dblack@wobblini.net)
Ruby Power and Light, LLC (http://www.rubypowerandlight.com)

"Ruby for Rails" PDF now on sale! Ruby for Rails
Paper version coming in early May!

16

James Edward Gray II wrote:

>> another_found = str[/find this/]
=> "find this"
>> yet_another_found = str[/skip this (find this) skip this/, 1]
=> "find this"

arg, sorry for the dupe.

Yoann

17

Thanks for clarifying the operation of scan -- I mistakenly thought I was getting a string when I was actually getting an array, so my original question wasn't even correct. Anyway, all of the methods listed below seem to work, and the last one seems especially elegant.

Thanks for the advice!
Chris

line = "name: value"

value = line.scan(%r{name: ([[:alnum:]]+)}i)[0][0]
puts "#{value.class}: #{value}"

value = line.scan(%r{name: ([[:alnum:]]+)}i).first.first
puts "#{value.class}: #{value}"

value = line.match(%r{name: ([[:alnum:]]+)}i)[1]
puts "#{value.class}: #{value}"

value = line[%r{name: ([[:alnum:]]+)}i, 1]
puts "#{value.class}: #{value}"

18

If your program does not have to do anything else you don't need threads at all.

If you actually want to do other stuff in parallel then it depends on
which part decices that your processing thread(s) are finished how you
do the termination code. If you give more information on this it's
easier for us to come up with a solution.

Kind regards

robert

···

2006/4/23, Pat Maddox <pergesu@gmail.com>:

Well this is all because I need to interact with a remote server that
sends asyncrhonous messages. I suppose the most common server that's
similar is an IRC server. You connect to the server, and then just
wait while it pumps out messages to you. The way I've implemented
this in other languages is to create a new thread which simply has an
infinite loop (checking a control condition, such as @connected) that
reads data from a socket. So in cases like this, the read call would
block until some data comes through the socket. Then it parses the
text and the loop continues. I'm trying to do the same thing with
Ruby.

--
Have a look: Robert K. | Flickr

19

Hi --

···

On Sun, 23 Apr 2006, clc wrote:

Well, now, that's just embarassing. Thanks!

I might have misunderstood your original question, but if you're happy
with a one-element array rather than a string, you could also do:

   text, = str.scan(/.../)

David

--
David A. Black (dblack@wobblini.net)
Ruby Power and Light, LLC (http://www.rubypowerandlight.com)

"Ruby for Rails" PDF now on sale! Ruby for Rails
Paper version coming in early May!

20

Oh, I didn't realize you could do that with parallel assignments too. Thanks for the help.

···

----- Original Message ----- From: <dblack@wobblini.net>
To: "ruby-talk ML" <ruby-talk@ruby-lang.org>
Sent: Sunday, April 23, 2006 8:52 AM
Subject: Re: Scan-like Idiom to Capture a Group as a String Instead of an Array?

I might have misunderstood your original question, but if you're happy
with a one-element array rather than a string, you could also do:

  text, = str.scan(/.../)