Solaris porting problem -- flock failure?

also, earlier you said that flock was ‘broken’ on solaris. what exactly did
you mean by that? supposedly solaris uses fcntl to implement flock so why
does ruby need it’s own? i’m sure there is an answer i’m just curious…

Which version of Solaris ?

SunOS 5.6 sun4u
SunOS 5.7 sun4u
SunOS 5.8 sun4u
SunOS 5.9 sun4u
etc…

i think we have them all. :wink:

If I’m remember correctly, on some version flock() work if the file is
open for writing.

all the above versions’ man pages claim this to be the case - a really weird
requirement since you may only wish to ensure repeatable reads…

You need a #rewind after you have got a lock.

huh? you mean when writing C programs or also from within ruby?

-a

ps. here is an update to my earlier post regarding a working coordination of
multiple processes/threads that works on linux and solaris. the earlier post
had one significant bug (though it still worked) :

the line

while ((file.flock File::LOCK_EX | File::LOCK_NB)) do
Thread.pass
end

should, of course, have been

while (not (file.flock File::LOCK_EX | File::LOCK_NB)) do
Thread.pass
end

note the ‘not’ !!

here is an updated version for anyone whos interested. again, i’d be
interested to hear if anyone ran it successfully on ‘your’ platform.

file : feeding_frenzy.rb

----CUT----
#!/usr/bin/env ruby

this program coordinates access to an output file for the purpose of

inserting a timestamp between multiple threads spawned from within multiple

forked processes. the coordination is considered successful if no

timestamps are inserted non-sequentially

require ‘thread’
require ‘ftools’

pid = Process.pid

n_process = (ARGV.shift or 4).to_i
n_thread = (ARGV.shift or 4).to_i
n_mark = (ARGV.shift or 2 << 9).to_i
outpath = (ARGV.shift or ‘/tmp/out’)
sortedpath = (ARGV.shift or ‘/tmp/sorted’)

File.rm_f outpath
File.rm_f sortedpath

n_mark_per_thread = n_mark / n_process / n_thread
cids =

$stdout.sync = true
$stdout.puts “================”
$stdout.puts “N_PROCESS : #{n_process}”
$stdout.puts “N_THREAD : #{n_thread}”
$stdout.puts “N_MARK : #{n_mark}”
$stdout.puts “OUTPATH : #{outpath}”
$stdout.puts “================”
$stdout.puts “\n#{pid} STARTED”

n_process.times {

cid =
  Process.fork {
    ppid = Process.ppid
    pid  = Process.pid
    sem  = Mutex.new
    $stdout.sync = true
    $stdout.puts "\t#{ppid}.#{pid} STARTED"
    threads = []
    File.open (outpath, 'a+') { |file|
file.sync = true
n_thread.times {
  Thread.new {
    sem.synchronize {
      tid = threads.size
      threads << Thread.current
      $stdout.puts "\t\t#{ppid}.#{pid}.#{tid} STARTED"
    }
    n_mark_per_thread.times {
      sem.synchronize {
	catch (:flock_eintr) {
	  begin
	    while (not (file.flock File::LOCK_EX | File::LOCK_NB)) do
	      Thread.pass
	    end
	  rescue
	    throw :flock_eintr
	  end
	}
	catch (:syswrite_eintr) {
	  begin
	    file.syswrite "%10.5f\n" % Time.now.to_f
	  rescue
	    throw :syswrite_eintr
	  end
	}
	catch (:flush_eintr) {
	  begin
	    file.flush
	  rescue
	    throw :flush_eintr
	  end
	}
	catch (:flock_eintr) {
	  begin
	    file.flock File::LOCK_UN
	  rescue
	    throw :flock_eintr
	  end
	}
      }
      Thread.pass
    }
  }
}
threads.each {|t| t.join}
    }
  }
 cids << cid

}

cids.each { |cid|
#Process.waitpid cid
Process.waitpid cid, Process::WUNTRACED
}
cids.each { |cid|
$stdout.puts “\t#{pid}.#{cid} FINISHED”
}

system ‘sync’

output = IO.readlines outpath
sorted = output.sort {|a,b| a.to_f <=> b.to_f}

if output != sorted
File.open (sortedpath, ‘w’) {|f| f.puts sorted }
$stdout.puts “\nERROR!”
$stdout.puts “SUGGEST ‘diff -c #{outpath} #{sortedpath}’”
else
$stdout.puts “\nSUCCESS!”
end
----CUT----

···

On Mon, 27 Jan 2003, ts wrote:

====================================

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

odd that it builds flock when it is not missing on this system?

flock() is broken on Solaris, this is why ruby emulate it with fcntl() (in
missing/flock.c)

Guy Decoux

You need a #rewind after you have got a lock.

              ^^^^^^^

Well, it was a #seek : sorry.

huh? you mean when writing C programs or also from within ruby?

also from ruby, if you open the file in update you must #seek to the end
after you have the lock

Guy Decoux

ts wrote:

“a” == ahoward ahoward@fsl.noaa.gov writes:

odd that it builds flock when it is not missing on this system?

flock() is broken on Solaris, this is why ruby emulate it with fcntl() (in
missing/flock.c)

On this solaris host, man flock says flock itself is implemented in
terms of fcntl locks:

  flock() applies or removes an  advisory  lock  on  the  file
  associated  with  the  file descriptor fd. The compatibility
  version of flock() has been implemented on top  of  fcntl(2)
  locking.  It does not provide complete binary compatibility.

But it appears ruby is doing its own emulation, just to be safe, I guess.

so, something like

f.flock File::LOCK_EX
f.seek f.size, IO::SEEK_END

is this even with

f = File.open path, ‘a+’

??

because i did not do this in my program (see earlier post) and it worked…
should i have?

do you have any idea why one seems to need to trap SystemCallError (EINTR) on
solaris while making system calls from within forked threads but seem not to
need to do so on linux? the solaris behviour kinda makes sense since you
could get SIGCLD, but it seems like this should not blow you process out of
kernel space… perhaps one could simply trap the signal to prevent this, but
then it would not be handled in the normal way…

-a

···

On Tue, 28 Jan 2003, ts wrote:

You need a #rewind after you have got a lock.
^^^^^^^

Well, it was a #seek : sorry.

huh? you mean when writing C programs or also from within ruby?

also from ruby, if you open the file in update you must #seek to the end
after you have the lock

====================================

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

  f = File.open path, 'a+'

The File is positionned at end of stream, if someone write in the file
when you are waiting for the lock you are not at end of stream.

Guy Decoux

tricky.

i wonder if the ruby flock impl could regonize this conditin based on the open
mode (‘a’ or ‘a+’) and do this for you… it seems like it would never be in
error.

alternatively, and probably better, a new method could be added such as
(incomplete example) :

class File
def File.flock_open (path, mode = ‘r’, flags = File::LOCK_SH)
f = File.open path, mode
while (not (f.flock flags))
Thread.pass
end
f.seek f.size, IO::SEEK_END if mode =~ /a+?/
return f unless block_given?
yield f
f.close
end
end

and this method could be delegated to if File.new was called with three
arguements…

-a

···

On Tue, 28 Jan 2003, ts wrote:

f = File.open path, ‘a+’

The File is positionned at end of stream, if someone write in the file
when you are waiting for the lock you are not at end of stream.

====================================

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================