Is this a bug in Thread#kill?

ruby-talk
1

Hi all,

I find an interesting behavior of Thread#kill, I am not sure if this is a
bug.
Please tell me If my test code has any problems.

Require sys-proctable and parallel to test.

require 'open3'
require 'parallel'
require 'sys/proctable'

class MemoryOut < StandardError
end

jobs = *(1..10000)

Parallel.map(jobs, :in_threads=>20) do
    mem_thr = Thread.new do
        100.times do|i|
            begin
                Sys::ProcTable.ps
            rescue Errno::ENOENT, Errno::ESRCH
                # ProcTable bug
            end
            if i > 5
                raise MemoryOut
            end
            sleep 0.1
        end
    end

    open_thr = Thread.new do
        Open3.popen2e "sleep 2" do |i, o, w|
            w.value
        end
        #mem_thr.kill
    end

    begin
        mem_thr.join
    rescue MemoryOut
    end

    open_thr.join
end

This program seems OK and I can not find out the problem.

If I add back the commented out line "mem_thr.kill", then fds opened by
ProcTable can not be closed, finally causes the error "too many opened
files".

Use the command "lsof" to check this, can see many files under /proc are
not closed.
If I comment out the kill again, everything goes well.

Thanks,
Jianjun.

2

Hi all,

I find an interesting behavior of Thread#kill, I am not sure if this is a
bug.
Please tell me If my test code has any problems.

Usage of Thread#kill is discouraged. The proper way to terminate the
thread would be a thread safe flag that is checked inside the #times loop.
Wait, since apparently you want to prevent the mem_thr thread from running
too long and it's doing a repeated operation I'd just record a target time
and break if that is passed:

target = Time.now + 2

100.times do
  break if Time.now > target
  ...
done

Require sys-proctable and parallel to test.

require 'open3'
require 'parallel'
require 'sys/proctable'

class MemoryOut < StandardError
end

jobs = *(1..10000)

Parallel.map(jobs, :in_threads=>20) do
    mem_thr = Thread.new do
        100.times do|i|
            begin
                Sys::ProcTable.ps
            rescue Errno::ENOENT, Errno::ESRCH
                # ProcTable bug
            end
            if i > 5

That condition is odd: why don't you change the number of iterations?

                raise MemoryOut

Instead of raising I'd rather break.

            end
            sleep 0.1
        end
    end

    open_thr = Thread.new do
        Open3.popen2e "sleep 2" do |i, o, w|
            w.value
        end
        #mem_thr.kill
    end

open_thr is completely superfluous, why don't you just use sleep()?

    begin
        mem_thr.join
    rescue MemoryOut

    end

    open_thr.join
end

This program seems OK and I can not find out the problem.

Add

Thread.abort_on_exception = true

at the top. That will help debugging.

If I add back the commented out line "mem_thr.kill", then fds opened by

ProcTable can not be closed, finally causes the error "too many opened
files".

Use the command "lsof" to check this, can see many files under /proc are
not closed.
If I comment out the kill again, everything goes well.

I'd say this is rather an issue with Sys::ProcTable than with Thread#kill:
apparently that does not release resources properly under all
circumstances. Maybe executing with strace helps to hunt this down.

Kind regards

robert

···

On Fri, Jan 30, 2015 at 8:07 AM, Jianjun Mao <justmao945@gmail.com> wrote:

--
[guy, jim, charlie].each {|him| remember.him do |as, often| as.you_can -
without end}
http://blog.rubybestpractices.com/

3

Thanks for your reply Robert Klemme.

I implemented a function named "shell" to call external executables with
time and memory limit.
This piece of code is used to test the problem I described, so you may see
strange numbers and other things.

Usage of Thread#kill is discouraged. The proper way to terminate the

thread would be a thread safe flag that is checked inside the #times loop.

Yes, I see now. Thread#kill may prevent releasing sources. When use a flag,
I can make sure the line to release source is executed.

open_thr is completely superfluous, why don't you just use sleep()?

Yes, but that's where to execute external executables, just an example here.

I'd say this is rather an issue with Sys::ProcTable than with Thread#kill:

apparently that does not release resources properly under all
circumstances. Maybe executing with strace helps to hunt this down.

You can see the code here

The only function called to read file is IO.read, which guarantees the file
wil be closed after read. I see now. It seems that IO.read won't release
resources when the thread is killed, and it is unable to release it, as the
thread is killed! So that's the problem. You are right, Thread#kill is
dangerous.

···

On Fri, Jan 30, 2015 at 4:59 PM, Robert Klemme <shortcutter@googlemail.com> wrote:

On Fri, Jan 30, 2015 at 8:07 AM, Jianjun Mao <justmao945@gmail.com> wrote:

Hi all,

I find an interesting behavior of Thread#kill, I am not sure if this is a
bug.
Please tell me If my test code has any problems.

Usage of Thread#kill is discouraged. The proper way to terminate the
thread would be a thread safe flag that is checked inside the #times loop.
Wait, since apparently you want to prevent the mem_thr thread from running
too long and it's doing a repeated operation I'd just record a target time
and break if that is passed:

target = Time.now + 2

100.times do
  break if Time.now > target
  ...
done

Require sys-proctable and parallel to test.

require 'open3'
require 'parallel'
require 'sys/proctable'

class MemoryOut < StandardError
end

jobs = *(1..10000)

Parallel.map(jobs, :in_threads=>20) do
    mem_thr = Thread.new do
        100.times do|i|
            begin
                Sys::ProcTable.ps
            rescue Errno::ENOENT, Errno::ESRCH
                # ProcTable bug
            end
            if i > 5

That condition is odd: why don't you change the number of iterations?

                raise MemoryOut

Instead of raising I'd rather break.

            end
            sleep 0.1
        end
    end

    open_thr = Thread.new do
        Open3.popen2e "sleep 2" do |i, o, w|
            w.value
        end
        #mem_thr.kill
    end

open_thr is completely superfluous, why don't you just use sleep()?

    begin
        mem_thr.join
    rescue MemoryOut

    end

    open_thr.join
end

This program seems OK and I can not find out the problem.

Add

Thread.abort_on_exception = true

at the top. That will help debugging.

If I add back the commented out line "mem_thr.kill", then fds opened by

ProcTable can not be closed, finally causes the error "too many opened
files".

Use the command "lsof" to check this, can see many files under /proc are
not closed.
If I comment out the kill again, everything goes well.

I'd say this is rather an issue with Sys::ProcTable than with Thread#kill:
apparently that does not release resources properly under all
circumstances. Maybe executing with strace helps to hunt this down.

Kind regards

robert

--
[guy, jim, charlie].each {|him| remember.him do |as, often| as.you_can -
without end}
http://blog.rubybestpractices.com/