Searching the web and books for information on this, I can’t seem to
find a definitive yes or no to my question:
“Is there a portable way to do file locking?”
Some of the problems that are mentioned are that NFS systems make
almost everything non-atomic, locking methods often depend on fcntl
which is not available on all systems, and the dreaded race
condition when test and set are non-atomic.
Ruby can be used on the Mac, PC, and Unix, so I’m really after
something that portable. I can’t use a Mutex because I need this to
be exclusive across process boundaries (several invokations of the
program).
My searching suggests this is a common problem, but the answer to it
is rare!
Thank you
Hugh
i been doing alot of experiments with locking myself, mainly on nfs systems
for some designs for a distributed work queue i’m working on, and have come to
largely the same conclusions. however, you defintely want fcntl based locking
for NFS systems. as far as i know any posix compliant sytem has fcntl but i’m
a windows dummy (windows people insert correction)…
you might want to check out a few things i’ve done - most of them were done
very quickly and further testing is in order but:
the tests i’ve been running (day at a time) consist of multiple processes on
multiple hosts competing to update a queue in an ordered fashion… if the
queue is ever out of order, or a marshall error is thrown, the test ‘fails’.
i also mark the times each node aquires the lock and gather stats on the
min/max/avg time required to obtain the lock. i’ve run using all three
methods above, plus system calls to lockfile, for my locking mechanism and
have the following observation
-
they all work on nfs - i get a core dump every now and again in the
liblockfile impl which is almost certainly a bug in my own code
-
lockd sucks at giving at sort of ‘even’ distribution to the processes,
what i generally see is one node hogging the lock for a while, then
eventually lockd seems to realize this and give it another node for a
while. for my uses this is not a big deal since the competition in
production would not actually be that fierce… it DOES work though with
a sufficiently new lockd impl or a rather expensive netap…
-
the max time between locks for 6 or so process competing for a fcntl based
lock on our systems is around 30 seconds
-
lockfile seems to work really well - given max/min/avg of about 1 sec for
all nodes. this really suprised me.
-
the big drawback to lockfiles is potential hangs and inability to grant
read-locks. there is serious locking package on CPAN which claims to do
this (read/write nfs safe lockfiles) at
File::NFSLock - perl module to do NFS (or not) locking - metacpan.org
the idea of this seems quite sketchy. i have not tested it.
if you are interested in my test code drop me a line - it’s one script that
you run on all the node, and a monitoring script that goes with it… nice
a terrible like my testing code tends to be…
in any case - i would think implementing the algorithim used by liblockfile in
ruby might be a good solution. the hard work at making things portable has
been done for you by matz and co. i made a stab at that (it’s in the lockfile
package) but it is NOT finished… i should probably take it out of there…
i’m very interested in any findings you have along these lines. please keep
us informed.
-a
···
On Thu, 15 Apr 2004, Hugh Sasse Staff Elec Eng wrote:
Searching the web and books for information on this, I can’t seem to
find a definitive yes or no to my question:
“Is there a portable way to do file locking?”
Some of the problems that are mentioned are that NFS systems make
almost everything non-atomic, locking methods often depend on fcntl
which is not available on all systems, and the dreaded race
condition when test and set are non-atomic.
Ruby can be used on the Mac, PC, and Unix, so I’m really after
something that portable. I can’t use a Mutex because I need this to
be exclusive across process boundaries (several invokations of the
program).
My searching suggests this is a common problem, but the answer to it
is rare!
Thank you
Hugh
–
EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================
As a side note on implementation, is there a particular reason you chose
to use filelocking when something like drb would allow you to do thread
atomic gets across a network without necessarily having shared disk
space? Let alone a shared tuplespace which does even more fun things
for you?
Charlie
that’s a good idea… i’ve thought of it myself, but our sysads are nazis
about opening any ports…
-a
···
On Wed, 14 Apr 2004, Charles Comstock wrote:
As a side note on implementation, is there a particular reason you chose
to use filelocking when something like drb would allow you to do thread
atomic gets across a network without necessarily having shared disk
space? Let alone a shared tuplespace which does even more fun things
for you?
Charlie
–
EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================
As a side note on implementation, is there a particular reason you chose
to use filelocking when something like drb would allow you to do thread
atomic gets across a network without necessarily having shared disk
space? Let alone a shared tuplespace which does even more fun things
for you?
Because I using this for (reading and) writing the config file which
tells other processes where to find my DRb server, not wanting to
open things up so widely by having a Ring server (which I don’t
fully understand yet anyway)…
Charlie
Hugh
···
On Thu, 15 Apr 2004, Charles Comstock wrote:
Hugh Sasse Staff Elec Eng wrote:
As a side note on implementation, is there a particular reason you chose
to use filelocking when something like drb would allow you to do thread
atomic gets across a network without necessarily having shared disk
space? Let alone a shared tuplespace which does even more fun things
for you?
Because I using this for (reading and) writing the config file which
tells other processes where to find my DRb server, not wanting to
open things up so widely by having a Ring server (which I don’t
fully understand yet anyway)…
Charlie
Hugh
Yea I don’t understand Ring servers either, I wish someone would explain
them. Is it doing a portscan or something, I just don’t follow how it
magically finds the Ringer server.
You could of course have a single config/reader writer server of course,
but I understand why that might be excessive.
My most recent DRb app does use shared home dirs for reading the server
location, but other than that I try to avoid that and send more over the
wire. I guess it justs feels more portable that way. It’s a fun little
project though, a client/agent/server for running a programming contest,
but ensuring that each contestee is not sharing cpu power with other
contest members by running there code in my userspace on there machine.
Works pretty slick, and is only about 250 lines of code total. It’s a
project I wouldn’t even undertake in another language, but in ruby it’s
a snap.
Charlie
···
On Thu, 15 Apr 2004, Charles Comstock wrote:
This is getting somewhat less portable… 
Hugh
···
On Thu, 15 Apr 2004, John Platte wrote:
On 2004 Apr 15, at 1:49, Ara.T.Howard wrote:
something like drb…
that’s a good idea… i’ve thought of it myself, but our sysads are
nazis
about opening any ports…
SSH tunneling?
for me it’s o.k. - we are all linux. see my other thread about pty/expect and
you’ll see where i’m headed there 
something like:
-
nfs mounted home dir for all processing node’s user
-
systems starts and generates a random passhprase
-
feeds passphrase into ssh-keygen/ssh-agent/ssh-add
-
copies id_rsa.pub to authorized_keys (remember nfs home dir)
-
now all nodes can tunnel drb through ssh
why generate the passphrase? because these machine routinues go down/come up
in the middle of the night and ssh-add would hang, waiting for a passphrase so
i could either
a) embed the passphrase in the source
b) randomly generate on startup so it is know only to the program
i realize this is terrible but what else can i do when you won’t open up ports
and can’t see to keep lockd running?
-a
···
On Thu, 15 Apr 2004, Hugh Sasse Staff Elec Eng wrote:
On Thu, 15 Apr 2004, John Platte wrote:
On 2004 Apr 15, at 1:49, Ara.T.Howard wrote:
something like drb…
that’s a good idea… i’ve thought of it myself, but our sysads are
nazis
about opening any ports…
SSH tunneling?
This is getting somewhat less portable…
–
EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================
Y’all are probably aware, but I should mention that PuTTY has quite a
complete SSH feature set for Windows (including ssh-agent), and is
MIT-licensed.
···
On 2004 Apr 15, at 9:54, Ara.T.Howard wrote:
SSH tunneling?
This is getting somewhat less portable…
for me it’s o.k. - we are all linux.
–
Ryan “John” Platte
Custom services, NIKA Consulting
http://nikaconsulting.com/
Yes, I’m using that, but have not done much with it
programmatically. But this seems a long way round, with dependencies,
to get file locking… Is there anything just using ruby that is
reliable?
Hugh
···
On Fri, 16 Apr 2004, John Platte wrote:
On 2004 Apr 15, at 9:54, Ara.T.Howard wrote:
SSH tunneling?
This is getting somewhat less portable…
for me it’s o.k. - we are all linux.
Y’all are probably aware, but I should mention that PuTTY has quite a
complete SSH feature set for Windows (including ssh-agent), and is
MIT-licensed.
File.flock should work - execpt for NFS. if you are on a hetrogeneous NFS
systems then i’d imagine you have much bigger problems though…
-a
···
On Fri, 16 Apr 2004, Hugh Sasse Staff Elec Eng wrote:
On Fri, 16 Apr 2004, John Platte wrote:
On 2004 Apr 15, at 9:54, Ara.T.Howard wrote:
SSH tunneling?
This is getting somewhat less portable…
for me it’s o.k. - we are all linux.
Y’all are probably aware, but I should mention that PuTTY has quite a
complete SSH feature set for Windows (including ssh-agent), and is
MIT-licensed.
Yes, I’m using that, but have not done much with it
programmatically. But this seems a long way round, with dependencies,
to get file locking… Is there anything just using ruby that is
reliable?
–
EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================