Confused about locking a file via file.flock(File::LOCK_EX)

I am writing a ruby appl under AIX where I need to
update the /etc/hosts table. I would like to make sure
that during my update nobody else can update that
file. I thought I found a solution to my problem with
the flock method. However, it does not appear to do
what I thought it will do. To test it, I wrote a
simple ruby piece of code that, first locks the file,
second spin-loop forever while I open another window
and vi the “locked” file. Well, I am able to vi and
change the file that is supposed to be locked.
ruby -v returns: ruby 1.6.7 (2002-03-01)
[powerpc-aix4.3.3.0]

Any ideas??

Thank you

···

Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

what I thought it will do. To test it, I wrote a
simple ruby piece of code that, first locks the file,
second spin-loop forever while I open another window
and vi the "locked" file. Well, I am able to vi and
change the file that is supposed to be locked.

Well, to understand the problem

svg% cat b.rb
#!/usr/bin/ruby
f = File.new("aa", "w")
f.flock(File::LOCK_EX)
puts "file locked #{Process.pid}"

fork do
   g = File.new("aa", "w")
   if !g.flock(File::LOCK_SH|File::LOCK_NB)
      puts "error : file locked #{Process.pid}"
   end
end
Process.wait

fork do
   g = File.new("aa", "w")
   g.puts "aa"
end
Process.wait
svg%

svg% cat aa
cat: aa: No such file or directory
svg%

svg% b.rb
file locked 25340
error : file locked 25341
svg%

svg% cat aa
aa
svg%

a lock work *only* if the process try to acquire a lock before writing in
the file.

In my example, the first fork test if the file is locked and give an error
if it can't have the lock.

The second fork, don't test and write directly in the file (it bypass the
lock)

In your case, vi do like my second fork : it don't try to acquire a lock
and write in the file

Guy Decoux

Flock only works within the same process. For example, if you had
2 concurrent versions of your program running, the second version
would wait for the first version to release the lock before it opened
the file. To my knowledge, there isn’t any guaranteed way to lock
all other processes out of a file, but I could be wrong.

HTH,
Travis Whitton whitton@atlantic.net

···

In article 20030725151907.12347.qmail@web20802.mail.yahoo.com, Ludwigi Beethoven wrote:

I am writing a ruby appl under AIX where I need to
update the /etc/hosts table. I would like to make sure
that during my update nobody else can update that
file. I thought I found a solution to my problem with
the flock method. However, it does not appear to do
what I thought it will do. To test it, I wrote a
simple ruby piece of code that, first locks the file,
second spin-loop forever while I open another window
and vi the “locked” file. Well, I am able to vi and
change the file that is supposed to be locked.
ruby -v returns: ruby 1.6.7 (2002-03-01)
[powerpc-aix4.3.3.0]

Any ideas??

Thank you


Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

Locking works between processes, but in general locks are “advisory”. That
means that process A and process B must both agree to use the locking
protocol. If process A locks a file, that doesn’t actually prevent process B
going ahead and writing to the file, if it doesn’t bother to get a lock
first.

It gets confusing when there are about 3 different locking mechanisms
(flock, lockf, and some ioctl-locking) but I don’t have my Stevens book with
me so I can’t give you the gory details :slight_smile:

Cheers,

Brian.

···

On Sat, Jul 26, 2003 at 12:40:51AM +0900, Travis Whitton wrote:

Flock only works within the same process.

Not true. You can have many different programs in many different
languages running as independent processes all using flock to
manage access to one or more files. However, flock is only an
advisory lock. It locks files so that no other process can
access them, IF that other process is bothering to use flock.

There is no mandatory file locking mechanism on UNIX.
It’s always possible to just ignore any locks and open a file
for writing.

-Mark

···

On Fri, Jul 25, 2003 at 03:20:16PM +0000, Travis Whitton wrote:

Flock only works within the same process.

Flock only works within the same process.

Not true. You can have many different programs in many different
languages running as independent processes all using flock to
manage access to one or more files.

I knew this. I guess I must have smoked a little too much crack
this morning… Time to go fsck my brain. Anyhoo, thanks for the
correction.

···

Travis Whitton whitton@atlantic.net

p.s., kidding about the crack.

Mark J. Reed wrote:

There is no mandatory file locking mechanism on UNIX.
It’s always possible to just ignore any locks and open a file
for writing.

I have to disagree. SVR2 had it way back when,
and Linux used to, last time I checked a few years ago.
As I recall, I could put a mandatory lock on a file, and even ‘cat’ would object.

* Mandatory locking
      o kernel enforces lock requests
      o access to records is controlled by the kernel
      o open, close, read, write system calls honor the locks
      o requires SGID bit set, with group-execute bit cleared
            + chmod u+rw,g+rws-x,o+rw file
      o in Linux, requires a MAND_LOCK mounted filesystem
            + mount -o remount,mand /filesystem
            + mount -o remount,nomand /filesystem
      o a write-lock prevents _any_ process from reading that section

I have no idea what AIX supports in this regard.

I guess I would have to grab the file and update it
rather quickly, and hope that nobody else is updating
it at the same time via some other way.

Thank you all for your reply.

···

Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software

The requirement was to prevent another user editing the same file with ‘vi’,
right?

On my system (FreeBSD), if I edit a file with vi, then try to edit the same
file in vi in another window, I get:

/etc/motd already locked, session is read-only.
/etc/motd: unmodified, readonly: line 1

This implies that there is some sort of locking going on.

Now, if I run this test program:

def testlock(filename)
f = File.open(filename)
f.flock(File::LOCK_EX)
sleep 30
f.close
end
testlock(“/etc/motd”)

then vi /etc/motd also reports that the file is locked. Is that not the case
for you?

If not, then the “update it rather quickly and hope” is most safely done by:

  • creating a copy of the file with a different name and modified contents
  • doing a ‘rename’ operation to replace the original file with it

This is an atomic operation - it will either succeed or fail. Of course
someone else who has it open in a buffer could overwrite it again, but at
least it will always be a complete and valid file, not corrupted by two
people writing to it simultaneously.

Regards,

Brian.

···

On Sat, Jul 26, 2003 at 08:57:00PM +0900, Ludwigi Beethoven wrote:

I guess I would have to grab the file and update it
rather quickly, and hope that nobody else is updating
it at the same time via some other way.

Well, I am using AIX version 4. I tried a similar
test and I was able to change and save /etc/hosts.
Perhaps there is a difference right there between
AIX and FreeBSD. As a matter of facts, I will
attach below a copy of the program that I am writing.
Perhaps someone can make suggestions.

The program will provide remote printing support to a
Windows/98 ws with a printer attached to it, managed
by a lpd server. The user comes in via a VPN that
provides a different IP address with every connection.
That’s the reason to update the /etc/hosts table.

If anyone knows an off the shelv software that will do
this then I will recommend that to the client. Althoug
it is more fun to write it myself as I learn ruby.

Of course I am not providing ALL the details, such us
the AIX printer config, which I think is irrelevant at
this point. For the UDP part I used a piece of code
from “The Ruby Way”.

Thank you

Here is the ruby code:

require “socket”

log=“/tmp/udpsrv.log”
begin
out = File.new(log, “a”)
rescue => e
puts “Problem with I/O operation: #{e}”
exit 1
else
time = Time.now
time1 = time.strftime(“%a%b%d%Y %H:%M:%S”)
out.puts time1 + " " + $0 + " Started Executing…"
PORT = 12321
time = Time.now
time1 = time.strftime(“%a%b%d%Y %H:%M:%S”)
out.puts time1 + " Listening on port: " + PORT.to_s

server = UDPSocket.open # Using UDP here…
server.bind(“cwsa”, PORT)
out.close

loop do
puts “Waiting for requests”
out = File.new(log, “a”)
time = Time.now
time1 = time.strftime(“%a%b%d%Y %H:%M:%S”)
out.puts time1 + " Wait for requests."

 text, sender = server.recvfrom(64)

 time = Time.now
 time1 = time.strftime("%a%b%d%Y %H:%M:%S")

 out.puts time1 + " Client domain: " +

sender[0].to_s
out.puts time1 + " Client port: " +
sender[1].to_s
out.puts time1 + " Client host: " +
sender[2].to_s
out.puts time1 + " Client IP adddress: " +
sender[3].to_s
out.puts time1 + " Client requests: " + text.to_s

 system("rm /tmp/newhosts 1>/dev/null 2>&1")    #

Remove old file
function, printer = text.chomp.split
if function.include?(“Enable”)
# Replace old definition from /etc/hosts table
entry for this printer
fileArr = IO.readlines(“/tmp/hosts”)
f = File.new(“/tmp/newhosts”, “w+”)
index = 0
found = false
fileArr.each do |line|
if line.include?(“VPNTEST”)
fileArr[index] = sender[3].to_s + " " +
“VPNTEST”
found = true
break
end
index += 1
end # End of do loop

    if !found    # Add to end of hosts file, if it

was not found
f.close # Firts close file
entry = sender[3].to_s + " " + “VPNTEST”
#system(“echo #{entry} >> /tmp/hosts”)
system(“mv /tmp/hosts /etc/hosts”) #
UNCOMMENT
else # Re-write hosts file back with the change IP
address
f.puts fileArr
f.close
#system(“mv /tmp/newhosts /tmp/hosts”)
system(“mv /tmp/newhosts /etc/hosts”)
end

    system("enable VPNTEST >>/tmp/dpsrv.log

2>>/tmp/udpsrv.log")
else
if function.include?(“Disable”)
system(“disable VPNTEST >>/tmp/dpsrv.log
2>>/tmp/udpsrv.log”)
else
out.puts time1 + " Request " + text.to_s + "
ignored."
end
end

 out.close

end
end

···

— Brian Candler B.Candler@pobox.com wrote:

On Sat, Jul 26, 2003 at 08:57:00PM +0900, Ludwigi > Beethoven wrote:

I guess I would have to grab the file and update
it
rather quickly, and hope that nobody else is
updating
it at the same time via some other way.

The requirement was to prevent another user editing
the same file with ‘vi’,
right?

On my system (FreeBSD), if I edit a file with vi,
then try to edit the same
file in vi in another window, I get:

/etc/motd already locked, session is read-only.
/etc/motd: unmodified, readonly: line 1

This implies that there is some sort of locking
going on.

Now, if I run this test program:

def testlock(filename)
f = File.open(filename)
f.flock(File::LOCK_EX)
sleep 30
f.close
end
testlock(“/etc/motd”)

then vi /etc/motd also reports that the file is
locked. Is that not the case
for you?

If not, then the “update it rather quickly and hope”
is most safely done by:

  • creating a copy of the file with a different name
    and modified contents
  • doing a ‘rename’ operation to replace the original
    file with it

This is an atomic operation - it will either succeed
or fail. Of course
someone else who has it open in a buffer could
overwrite it again, but at
least it will always be a complete and valid file,
not corrupted by two
people writing to it simultaneously.

Regards,

Brian.


Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

Well, I am using AIX version 4. I tried a similar
test and I was able to change and save /etc/hosts.

Perhaps try installing a newer version of vi - such as vim? Maybe the AIX
stock vi doesn’t attempt to get a lock on the file when opening it.

The program will provide remote printing support to a
Windows/98 ws with a printer attached to it, managed
by a lpd server. The user comes in via a VPN that
provides a different IP address with every connection.
That’s the reason to update the /etc/hosts table.

Does your lpd refuse a connection from someone who is not in /etc/hosts? If
so, could you not just list all the addresses they could come from in
/etc/hosts? If it’s a true VPN then that should be safe.

If the idea is to enable lpd for hosts which could be connecting from
anywhere on the Internet but need to authenticate themselves first, I would
probably use ssh tunnelling.

            ssh                    lpd

Client ----------------> gateway --------> printer
host host

Client side:
ssh -L 515:printerhost:515 username@gatewayhost

Gateway host:
just enable sshd

Printer host:
just put the gateway host IP address/name in /etc/hosts.lpd

Then, when the client prints to 127.0.0.1 port 515, it is relayed through
the ssh tunnel to the gateway host, which then opens a TCP connection to
port 515 on the printer host, and passes the traffic through. (The printer
host and gateway host can be the same machine, of course)

You get the advantages of a proper encrypted session and whatever
authentication you choose to use for ssh - passwords, RSA keys etc.

There are plenty of free ssh clients for Windows, most of which support
tunneling (try putty or teraterm pro + ttssh)

Regards,

Brian.

···

On Sun, Jul 27, 2003 at 12:27:42AM +0900, Ludwigi Beethoven wrote: