any idea why this script slows downs drastically as it runs and seems to leak
memory?
require ‘sync’
class Switch
ON, OFF, NEITHER = true, false, nil
def initialize state = OFF
extend Sync_m
@state = NEITHER
@observers = []
end
def on!
synchronize(:EX){
warn ‘on!’
@state = ON
notity_observers
}
end
def off!
synchronize(:EX){
warn ‘off!’
@state = OFF
notity_observers
}
end
def notity_observers
synchronize(:SH){
@observers.each do |o|
o.notify @state
end
}
end
def add_observer o
synchronize(:EX){
@observers << o
}
end
end
class SwitchToggle
def initialize switch
@switch = switch
@switch.add_observer self
end
def notify of
case of
when Switch::ON
Thread.new{ @switch.off! }
when Switch::OFF
Thread.new{ @switch.on! }
else
raise of.to_s
end
end
end
switch = Switch.new
toggle = SwitchToggle.new switch
switch.on!
STDIN.gets
-a
···
–
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
No idea, except that it seems to be related to the thread creation (maybe because it's within the sync block). The following works around it, and is probably more efficient anyway. I guess the semantics are close to the original.
require 'sync'
require 'thread'
class Switch
ON, OFF, NEITHER = true, false, nil
def initialize state = OFF
extend Sync_m
@state = NEITHER
@observers =
end
def on!
synchronize(:EX){
warn 'on!'
@state = ON
notity_observers
}
end
def off!
synchronize(:EX){
warn 'off!'
@state = OFF
notity_observers
}
end
def notity_observers
synchronize(:SH){
@observers.each do |o|
o.notify @state
end
}
end
def add_observer o
synchronize(:EX){
@observers << o
}
end
end
class SwitchToggle
def initialize switch
@switch = switch
@switch.add_observer self
@q = Queue.new
Thread.new do
loop {@q.pop.call}
end
end
def notify of
case of
when Switch::ON
later{ @switch.off! }
when Switch::OFF
later{ @switch.on! }
else
raise of.to_s
end
end
def later(&bl)
@q << bl
end
end
switch = Switch.new
toggle = SwitchToggle.new switch
switch.on!
STDIN.gets
···
ara.t.howard@noaa.gov wrote:
any idea why this script slows downs drastically as it runs and seems to leak
memory?
--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
ara.t.howard@noaa.gov writes:
any idea why this script slows downs drastically as it runs and seems to leak
memory?
First, I'm not a Ruby expert or something, but..
[Code skipped]
switch = Switch.new
Here you create the switch.
toggle = SwitchToggle.new switch
Here, toggle is created and registered as listener (in the constructor)
switch.on!
So now you switch it on.
Toggle gets notified, and creates a thread that switches it off.
But when that Thread switches it off, toggle gets notified, and starts a
thread that switches it on again..
Basically it's an infinite loop, and i suspect it's leaking because GC
doesn't kick in.
def count_objs
klasses = Hash.new 0
ObjectSpace.each_object do |o| klasses[o.class] += 1 end
return klasses.map { |k,v| "\t#{k}: #{v}" }.join("\n")
end
Thread.start { loop { puts count_objs; sleep 1 } }.join
Running this, then taking two snapshots, one after startup and one after its really slow:
$ diff -u a b
--- a 2006-07-05 15:28:29.000000000 -0700
+++ b 2006-07-05 15:28:12.000000000 -0700
@@ -2,17 +2,17 @@
Object: 3
Bignum: 3
fatal: 1
- String: 557
- Hash: 2
+ Hash: 17
+ String: 842
Module: 17
SwitchToggle: 1
Class: 153
- Array: 16
+ Array: 414
SystemStackError: 1
Float: 5
Binding: 1
- Thread: 4
+ Thread: 152
ThreadGroup: 1
IO: 3
- File: 2
+ File: 1
Switch: 1
Why aren't those threads getting GC'd?
···
On Jul 5, 2006, at 2:30 PM, ara.t.howard@noaa.gov wrote:
any idea why this script slows downs drastically as it runs and seems to leak
memory?
[...]
switch.on!
--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant
http://trackmap.robotcoop.com
No, the number of live threads (Thread.list) remains stable. Watching ObjectSpace shows the GC operating, but the number of thread instances never decreases.
···
On Jul 5, 2006, at 3:31 PM, Tom Rauchenwald wrote:
ara.t.howard@noaa.gov writes:
any idea why this script slows downs drastically as it runs and seems to leak
memory?
First, I'm not a Ruby expert or something, but..
[Code skipped]
switch = Switch.new
Here you create the switch.
toggle = SwitchToggle.new switch
Here, toggle is created and registered as listener (in the constructor)
switch.on!
So now you switch it on.
Toggle gets notified, and creates a thread that switches it off.
But when that Thread switches it off, toggle gets notified, and starts a
thread that switches it on again..
Basically it's an infinite loop, and i suspect it's leaking because GC
doesn't kick in.
--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant
http://trackmap.robotcoop.com
very interesting joel. thanks.
i seem to always end up using to Queue with threads these days anyhow - so i
guess this is fine...
i suppose it makes sense that one shouldn't create threads from inside a sync
block since it uses Thread.critical inside and, therefore, must assume only
only thread is running at times. i wonder if there is a pattern for safely
creating threads from within another taking into account
Thread.critical=true...
cheers.
-a
···
On Thu, 6 Jul 2006, Joel VanderWerf wrote:
No idea, except that it seems to be related to the thread creation (maybe
because it's within the sync block). The following works around it, and is
probably more efficient anyway. I guess the semantics are close to the
original.
--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama
it would be, but the synchronize prevents two threads from entering any of the
Switch methods at a time so only thread can actually be running at once.
-a
···
On Thu, 6 Jul 2006, Tom Rauchenwald wrote:
ara.t.howard@noaa.gov writes:
any idea why this script slows downs drastically as it runs and seems to leak
memory?
First, I'm not a Ruby expert or something, but..
[Code skipped]
switch = Switch.new
Here you create the switch.
toggle = SwitchToggle.new switch
Here, toggle is created and registered as listener (in the constructor)
switch.on!
So now you switch it on.
Toggle gets notified, and creates a thread that switches it off.
But when that Thread switches it off, toggle gets notified, and starts a
thread that switches it on again..
Basically it's an infinite loop, and i suspect it's leaking because GC
doesn't kick in.
--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama
this is pretty suspect
file: sync.rb
245 def sync_initialize
246 @sync_mode = UN
247 @sync_waiting =
248 @sync_upgrade_waiting =
249 @sync_sh_locker = Hash.new
250 @sync_ex_locker = nil
251 @sync_ex_count = 0
252 end
253
254 def initialize(*args)
255 sync_initialize
256 super
257 end
258
259 def sync_try_lock_sub(m)
260 case m
261 when SH
262 case sync_mode
263 when UN
264 self.sync_mode = m
265 sync_sh_locker[Thread.current] = 1
266 ret = true
note that unlocking never releases the reference! i'm going to test a patch
using something like
249 @sync_sh_locker = Hash.new{|h,k| h[k] = 1}
265 sync_sh_locker.delete Thread.current
... more later ...
-a
···
On Thu, 6 Jul 2006, Eric Hodel wrote:
Why aren't those threads getting GC'd?
--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama
check out my post on ruby-core eric...
-a
···
On Thu, 6 Jul 2006, Eric Hodel wrote:
Why aren't those threads getting GC'd?
--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama
i seem to always end up using to Queue with threads these days anyhow - so i
guess this is fine...
Yep. Thread w/o Queue for me would be like Car w/o Gas. Or maybe Car w/ $5/gallon gas.
i suppose it makes sense that one shouldn't create threads from inside a sync
block since it uses Thread.critical inside and, therefore, must assume only
only thread is running at times. i wonder if there is a pattern for safely
creating threads from within another taking into account
Thread.critical=true...
In this case, though, Thread.critical==false while in the notify block, so it can't be causing this problem, right?
···
ara.t.howard@noaa.gov wrote:
--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
Yes, but they are still queued up against the mutex. I completely
agree with Tom's analysis. Having an Observer trigger the
notification again is definitively a bad idea - at least if events are
not queued up.
Kind regards
robert
···
2006/7/6, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov>:
On Thu, 6 Jul 2006, Tom Rauchenwald wrote:
> ara.t.howard@noaa.gov writes:
>
>> any idea why this script slows downs drastically as it runs and seems to leak
>> memory?
>
> First, I'm not a Ruby expert or something, but..
>
> [Code skipped]
>
>> switch = Switch.new
>
> Here you create the switch.
>
>> toggle = SwitchToggle.new switch
>
> Here, toggle is created and registered as listener (in the constructor)
>
>> switch.on!
>
> So now you switch it on.
> Toggle gets notified, and creates a thread that switches it off.
> But when that Thread switches it off, toggle gets notified, and starts a
> thread that switches it on again..
> Basically it's an infinite loop, and i suspect it's leaking because GC
> doesn't kick in.
it would be, but the synchronize prevents two threads from entering any of the
Switch methods at a time so only thread can actually be running at once.
--
Have a look: http://www.flickr.com/photos/fussel-foto/
yes, it seems so. still, you were definitely on to something - if no threads
are created it works...
hmm. i'll have to read sync.rb a bit more - it's thick.
-a
···
On Wed, 5 Jul 2006, Joel VanderWerf wrote:
ara.t.howard@noaa.gov wrote:
i seem to always end up using to Queue with threads these days anyhow - so i
guess this is fine...
Yep. Thread w/o Queue for me would be like Car w/o Gas. Or maybe Car w/ $5/gallon gas.
i suppose it makes sense that one shouldn't create threads from inside a sync
block since it uses Thread.critical inside and, therefore, must assume only
only thread is running at times. i wonder if there is a pattern for safely
creating threads from within another taking into account
Thread.critical=true...
In this case, though, Thread.critical==false while in the notify block, so it can't be causing this problem, right?
--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama