I've got a script that submits a whole load of condor jobs, and has to
wait for each result to return in order to read it.
Would something like this do, or is there a better solution?
files.each do |file|
if FileTest::exists?(file)
# process the file
else
redo
end
end
Thanks!
Milo Thurston wrote:
I've got a script that submits a whole load of condor jobs, and has to
wait for each result to return in order to read it.
Would something like this do, or is there a better solution?
Maybe this (essentially the same but more compact)?
files.each do |file|
sleep 1 until FileTest::exists?(file)
if FileTest::exists?(file)
# process the file
else
redo
end
end
Do they have to be returned in the order they were submitted?
martin
Milo Thurston <nospam@linacreschoolofdefence.org> wrote:
I've got a script that submits a whole load of condor jobs, and has to
wait for each result to return in order to read it.
Would something like this do, or is there a better solution?
They can come back in any order.
I've also found that condor sometimes does not return them, so I'll
have to include a timeout in the loop. CPU usage is quite high, too.
Milo.
Martin DeMello <martindemello@yahoo.com> wrote:
Do they have to be returned in the order they were submitted?
You might be better off with a thread per loop, then, which fills an
entry into a global array or hash when its particular job finishes. That
way, if you want you can start processing files as they come in, and not
block on waiting for the next in sequence to get done.
martin
Milo Thurston <nospam@linacreschoolofdefence.org> wrote:
Martin DeMello <martindemello@yahoo.com> wrote:
> Do they have to be returned in the order they were submitted?They can come back in any order.
I've also found that condor sometimes does not return them, so I'll
have to include a timeout in the loop. CPU usage is quite high, too.
That's not something I've encountered yet.
Are there any on-line examples I could take a look at, to see
how it works?
Thanks.
Martin DeMello <martindemello@yahoo.com> wrote:
You might be better off with a thread per loop, then, which fills an
entry into a global array or hash when its particular job finishes. That
way, if you want you can start processing files as they come in, and not
block on waiting for the next in sequence to get done.
Check out http://www.rubycentral.com/book/tut_threads.html - Ruby's
threading mechanism is amazingly lightweight.
martin
Milo Thurston <nospam@linacreschoolofdefence.org> wrote:
Martin DeMello <martindemello@yahoo.com> wrote:
> You might be better off with a thread per loop, then, which fills an
> entry into a global array or hash when its particular job finishes. That
> way, if you want you can start processing files as they come in, and not
> block on waiting for the next in sequence to get done.That's not something I've encountered yet.
Are there any on-line examples I could take a look at, to see
how it works?
this already does it
Amazonギフト券買取は安全？違法ではない？ギフト券買取の際に気を付けたいポイントとリスクについて
check out the testsuite for examples
you would probably use it like:
watcher = FileSystemWatcher.new("condor_outdir", "*")
watcher.sleepTime = 1
watcher.start { |status,file|
if status == FileSystemWatcher::CREATED then
puts "#{ file } was created"
elsif status == FileSystemWatcher::MODIFIED then
puts "#{ file } was modified"
elsif status == FileSystemWatcher::DELETED then
puts "#{ file } was deleted"
end
}
cheers.
-a
On Tue, 8 Jun 2004, Milo Thurston wrote:
Martin DeMello <martindemello@yahoo.com> wrote:
You might be better off with a thread per loop, then, which fills an
entry into a global array or hash when its particular job finishes. That
way, if you want you can start processing files as they come in, and not
block on waiting for the next in sequence to get done.That's not something I've encountered yet.
Are there any on-line examples I could take a look at, to see
how it works?
Thanks.
EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it; and a weed grows, even though we do
not love it. --Dogen
===============================================================================
Excellent - thanks!
Just what I was looking for.
Ara.T.Howard <ahoward@noaa.gov> wrote:
this already does it
....your mail seems to be bouncing, but you could try
nsb at ceh ac uk
for your condor questions.
Hmmm...
It seems not to like being told to look for the creation of empty files:
/usr/lib/ruby/site_ruby/1.8/filesystemwatcher.rb:63:in `addFile': File
'/tmp/switchfinder/NC_000868.862310.T.16.out' either doesnt exist or
isnt readable (FSWatcher::InvalidFileError)
So, I've tried creating the files and getting it to look for
modifications, but it simply reads the empty files and returns an empty
results page to the browser.
Have you any idea why?
Thanks,
Milo.
Ara.T.Howard <ahoward@noaa.gov> wrote: