Threads, mutex, regexp and trouble

ruby-talk
1

Hello, rubyists.

I have a bit of a problem suing threads. The program reads in a list of
commands, then fires them off, via a telnet session, to another device,
collecting data as it goes.

On a one-by-one basis, everything is fine. the commands works,
everything is peachy. The problem is I have to do 250+ sessions in a
fairly limited time. Ok, use threads.

The problem is that I appear to be occassional tromping a variable, and
mutexing doesn’t seem to work. Sometimes its ok, often it not.

The code in question looks like:

  @threads << Thread.new(node) { |idx|
nn, ip, usr, pwd = idx.split(/:/)

begin
  bsn = BSN.new(ip, @timeout)

  # login and run each command
  if bsn.login then

    @cmds.each do |cmd|
      mutex.synchronize do
	  puts "~~>> cmd=#{cmd}"
	  bsn.cmd(cmd)
      end
    end
    bsn.logout
  else
    puts " !! Unable to reach node= #{nn}"
  end
  }
@threads.each { |thr| thr.join }

The problem is that the “cmd” variable seems to get mashed by a regexp
match within bsn.cmd(). A good run looks like:

… Running 2 commands on 2 nodes.
… threading now …
~~>> cmd=show subs isp=_eeua
~~>> cmd=show subs isp=_eeua

A bad run looks like:

… Running 2 commands on 2 nodes.
… threading now …
~~>> cmd=show subs isp=*_eeua
~~>> cmd=show subs isp=ebg1_eeua

These commands come from a text file and the line is “show subs
isp=*_eeua”.

The regexp does a match on data returned from the node. The first match
for the first node is “ebg1_eeua”. So, it seems that the regexp is
confusing the thread.

Any thoughts?

Thanks,

-mark.

2

“Mark Probert” probertm@nospam-acm.org schrieb im Newsbeitrag
news:Xns9475D4B7958E5probertmnospamacmorg@207.35.177.135…

Hello, rubyists.

I have a bit of a problem suing threads. The program reads in a list of
commands, then fires them off, via a telnet session, to another device,
collecting data as it goes.

On a one-by-one basis, everything is fine. the commands works,
everything is peachy. The problem is I have to do 250+ sessions in a
fairly limited time. Ok, use threads.

The problem is that I appear to be occassional tromping a variable, and
mutexing doesn’t seem to work. Sometimes its ok, often it not.

The code in question looks like:

  @threads << Thread.new(node) { |idx|

nn, ip, usr, pwd = idx.split(/:/)

begin
bsn = BSN.new(ip, @timeout)

login and run each command

if bsn.login then

@cmds.each do |cmd|
  mutex.synchronize do

puts “~~>> cmd=#{cmd}”
bsn.cmd(cmd)
end
end
bsn.logout
else
puts " !! Unable to reach node= #{nn}"
end
}
@threads.each { |thr| thr.join }

The problem is that the “cmd” variable seems to get mashed by a regexp
match within bsn.cmd(). A good run looks like:

… Running 2 commands on 2 nodes.
… threading now …
~~>> cmd=show subs isp=_eeua
~~>> cmd=show subs isp=_eeua

A bad run looks like:

… Running 2 commands on 2 nodes.
… threading now …
~~>> cmd=show subs isp=*_eeua
~~>> cmd=show subs isp=ebg1_eeua

These commands come from a text file and the line is “show subs
isp=*_eeua”.

The regexp does a match on data returned from the node. The first match
for the first node is “ebg1_eeua”. So, it seems that the regexp is
confusing the thread.

Any thoughts?

Is it guaranteed that all threads use the same instance via ‘mutex’, i.e.
does synchronization work properly? Or maybe you use gsub! instead of
gsub (which you should since apparently all threads share @cmds).

Additionally I’d add a rescue clause at the end of the thread’s main block
to catch and display any exceptions. Plus I’d put the bsn.logout into an
ensure clause:

@threads << Thread.new(node) { |idx|
nn, ip, usr, pwd = idx.split(/:/)

begin
bsn = BSN.new(ip, @timeout)

# login and run each command
if bsn.login then
  begin
    @cmds.each do |cmd|
      mutex.synchronize do
        puts "~~>> cmd=#{cmd}"
        bsn.cmd(cmd)
      end
    end
  ensure
    bsn.logout
  end
else
  $stderr.puts " !! Unable to reach node= #{nn}"
end

rescue Exception => e
$stderr.puts e
end
}

@threads.each { |thr| thr.join }

Btw: Your indentation was garbled. Better use spaces.

Kind regards

robert
3

“Robert Klemme” bob.news@gmx.net wrote:

Is it guaranteed that all threads use the same instance via ‘mutex’,
i.e. does synchronization work properly? Or maybe you use gsub!
instead of gsub (which you should since apparently all threads share
@cmds).

Thank you very much for your input. It is was the @cmds that was in fact
getting overwritten. I changed how I processed it, to rereading the file
for each thread, and the problem went away.

Thanks also for the exception handling suggestions. Implemented!

Regards,

-mark.

4

“Mark Probert” probertm@nospam-acm.org schrieb im Newsbeitrag
news:Xns947ED077718CDprobertmnospamacmorg@206.172.150.14…

Is it guaranteed that all threads use the same instance via ‘mutex’,
i.e. does synchronization work properly? Or maybe you use gsub!
instead of gsub (which you should since apparently all threads share
@cmds).

Thank you very much for your input. It is was the @cmds that was in
fact
getting overwritten. I changed how I processed it, to rereading the
file
for each thread, and the problem went away.

Rereading the file is a bad solution (performance wise). Why don’t you
just copy it? Depending on whether only the reference @cmd was
overwritte, the collection changed or even elements were changed one of
these three options is most appropriate (in order):

@threads << Thread.new(node, @cmds) { |idx, commands|
nn, ip, usr, pwd = idx.split(/:/)

begin
bsn = BSN.new(ip, @timeout)

# login and run each command
if bsn.login then
  begin
    commands.each do |cmd|
      mutex.synchronize do
        puts "~~>> cmd=#{cmd}"
        bsn.cmd(cmd)
      end
    end
  ensure
    bsn.logout
  end
else
  $stderr.puts " !! Unable to reach node= #{nn}"
end

rescue Exception => e
$stderr.puts e
end
}

@threads << Thread.new(node, @cmds.dup) { |idx, commands|
nn, ip, usr, pwd = idx.split(/:/)
…
end

@threads << Thread.new(node, Marshal.load(Marshal.dump(@cmds))) { |idx,
commands>
nn, ip, usr, pwd = idx.split(/:/)
…
end

Thanks also for the exception handling suggestions. Implemented!

Yeah, that’s an often seen problem: exceptions that terminate threads are
normally not shown so that threads seem to die silently.

Regards

robert
···

“Robert Klemme” bob.news@gmx.net wrote: