Forking and copying variables

ruby-talk
1

I have a script that sets a few variables and then forks. Each fork
executes the script from top to bottom. Is there any way for the forks
to get a copy of all the variables set before the fork, because each
fork thinks it;s running for the first time (and ya I see why because a
fork is a copy of the current process)?

Maybe forking isn't what I'm looking for, but the performance is so much
better than threading for this task.

Thanks for reading.

···

--
Posted via http://www.ruby-forum.com/.

2

Brandon Casci wrote:

I have a script that sets a few variables and then forks. Each fork
executes the script from top to bottom. Is there any way for the forks
to get a copy of all the variables set before the fork, because each
fork thinks it;s running for the first time (and ya I see why because a
fork is a copy of the current process)?

Maybe forking isn't what I'm looking for, but the performance is so much
better than threading for this task.

Thanks for reading.

Are you using the win32/process version of fork? If so, what you are seeing is one of the differences between that and native fork on unix/linux.

For example, this code:

   puts "foop"

   Process.fork do
     puts "fork"
   end

on linux produces the following output:

foop
fork

and on windows (requiring 'win32/process' first) produces:

foop
fork

···

--
        vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

3

Use the block form of fork, that will preserve all state:

10:14:12 [~]: ruby -e 'p $$; foo=1; 2.times do fork do
while foo < 10
print $$, ":", foo, "\n"
foo+=1
end
end end'
3840
3468:1
3468:2
3468:3
3468:4
3468:5
3468:6
3468:7
3468:8
3468:9
10:14:42 [~]: 3600:1
3600:2
3600:3
3600:4
3600:5
3600:6
3600:7
3600:8
3600:9

10:14:54 [~]:

Kind regards

  robert

···

On 05.04.2007 06:03, Brandon Casci wrote:

I have a script that sets a few variables and then forks. Each fork
executes the script from top to bottom. Is there any way for the forks
to get a copy of all the variables set before the fork, because each
fork thinks it;s running for the first time (and ya I see why because a
fork is a copy of the current process)?

Maybe forking isn't what I'm looking for, but the performance is so much
better than threading for this task.

4

I see!

I am on Windows. I was hoping to write something that would work on both
Windows and Linux so I tried to keep things simple by only using
Process.fork.

I'll read a little more on Win32 Process.create and see if this can get
me what I need. If so I can detect the platform and fork the appropriate
way on Linux or Windows.

···

--
Posted via http://www.ruby-forum.com/.

5

This seems like a typical IO buffering problem rather than
different semantics regarding process data shared across
forks.

It looks like in the Windows version stdout is buffered such
that 'foop' is not written before the fork. After the fork
the output buffer still has 'foop\n' in it. At some point
the parent flushes its buffer (with just 'foop\n') and then
later the child flushes its buffer (with 'foop\nfork\n')
giving you the result you see.

This is a common 'gotcha' in Unix if stdout is going to
a file and so it isn't line buffered. If stdout goes to
a terminal device it is line buffered and so the problem
doesn't occur since each call to puts flushes a line of
output.

I'm not a Windows guy but that's what seems to be happening
in Joel's example.

I'm still not sure what problem the original poster is
encountering. Some sample code and output would help.

Gary Wright

···

On Apr 5, 2007, at 12:19 AM, Joel VanderWerf wrote:

Are you using the win32/process version of fork? If so, what you are seeing is one of the differences between that and native fork on unix/linux.

For example, this code:

  puts "foop"

  Process.fork do
    puts "fork"
  end

on linux produces the following output:

foop
fork

and on windows (requiring 'win32/process' first) produces:

foop
fork

6

I don't think the state preserving semantics are different for block or
non-block forms:

$ cat fork.rb
foo = 1

if child = Process.fork
   puts "parent says: child is pid #{child}, foo is #{foo}"
else
   puts "child says: my pid is #{$$}, foo is #{foo}"
end

$ ruby fork.rb
child says: my pid is 7875, foo is 1
parent says: child is pid 7875, foo is 1

Gary Wright

···

On Apr 5, 2007, at 4:20 AM, Robert Klemme wrote:

On 05.04.2007 06:03, Brandon Casci wrote:

I have a script that sets a few variables and then forks. Each fork
executes the script from top to bottom. Is there any way for the forks
to get a copy of all the variables set before the fork, because each
fork thinks it;s running for the first time (and ya I see why because a
fork is a copy of the current process)?
Maybe forking isn't what I'm looking for, but the performance is so much
better than threading for this task.

Use the block form of fork, that will preserve all state:

7

Gary Wright wrote:

Are you using the win32/process version of fork? If so, what you are seeing is one of the differences between that and native fork on unix/linux.

For example, this code:

  puts "foop"

  Process.fork do
    puts "fork"
  end

on linux produces the following output:

foop
fork

and on windows (requiring 'win32/process' first) produces:

foop
fork

This seems like a typical IO buffering problem rather than
different semantics regarding process data shared across
forks.

It looks like in the Windows version stdout is buffered such
that 'foop' is not written before the fork. After the fork
the output buffer still has 'foop\n' in it. At some point
the parent flushes its buffer (with just 'foop\n') and then
later the child flushes its buffer (with 'foop\nfork\n')
giving you the result you see.

This is a common 'gotcha' in Unix if stdout is going to
a file and so it isn't line buffered. If stdout goes to
a terminal device it is line buffered and so the problem
doesn't occur since each call to puts flushes a line of
output.

I'm not a Windows guy but that's what seems to be happening
in Joel's example.

I'm still not sure what problem the original poster is
encountering. Some sample code and output would help.

The source for win32/process makes it clear what is going on:

       cmd = 'ruby -I "' + $LOAD_PATH.join(File::PATH_SEPARATOR) << '" "'
       cmd << File.expand_path($PROGRAM_NAME) << '" ' << ARGV.join(' ')
       cmd << ' child#' << @child_pids.length.to_s

       startinfo = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
       startinfo = startinfo.pack('LLLLLLLLLLLLSSLLLL')
       procinfo = [0,0,0,0].pack('LLLL')

       rv = CreateProcess(0, cmd, 0, 0, 1, 0, 0, 0, startinfo, procinfo)

It runs the same ruby script again in another process. It's not really fork in the unix sense.

···

On Apr 5, 2007, at 12:19 AM, Joel VanderWerf wrote:

--
        vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

8

Thanks for clarifying things Joel. As I said, I'm not a Windows guy...

Seems like a bad choice to use the name 'fork' for this behavior.
It is more like a fork/exec of the original script (in Unix
terminology). It would make more sense to me to have Process.fork
fail with an exception on Windows and come up with a different name
for this Windows-only behavior.

The pseudo-portability of Ruby code has always seemed a bit odd to
me. Obviously this sort of issue comes up with File IO, OS services,
process and thread semantics and so on.

I almost wish that the differences were more obvious via the class
structure rather than methods of the same class having different
semantics on different platforms.

Gary Wright

···

On Apr 5, 2007, at 2:54 PM, Joel VanderWerf wrote:

The source for win32/process makes it clear what is going on:

      cmd = 'ruby -I "' + $LOAD_PATH.join(File::PATH_SEPARATOR) << '" "'
      cmd << File.expand_path($PROGRAM_NAME) << '" ' << ARGV.join(' ')
      cmd << ' child#' << @child_pids.length.to_s

      startinfo = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
      startinfo = startinfo.pack('LLLLLLLLLLLLSSLLLL')
      procinfo = [0,0,0,0].pack('LLLL')

      rv = CreateProcess(0, cmd, 0, 0, 1, 0, 0, 0, startinfo, procinfo)

It runs the same ruby script again in another process. It's not really fork in the unix sense.

9

Gary Wright wrote:

The source for win32/process makes it clear what is going on:

      cmd = 'ruby -I "' + $LOAD_PATH.join(File::PATH_SEPARATOR) << '" "'
      cmd << File.expand_path($PROGRAM_NAME) << '" ' << ARGV.join(' ')
      cmd << ' child#' << @child_pids.length.to_s

      startinfo = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
      startinfo = startinfo.pack('LLLLLLLLLLLLSSLLLL')
      procinfo = [0,0,0,0].pack('LLLL')

      rv = CreateProcess(0, cmd, 0, 0, 1, 0, 0, 0, startinfo, procinfo)

It runs the same ruby script again in another process. It's not really fork in the unix sense.

Thanks for clarifying things Joel. As I said, I'm not a Windows guy...

Seems like a bad choice to use the name 'fork' for this behavior.
It is more like a fork/exec of the original script (in Unix
terminology). It would make more sense to me to have Process.fork
fail with an exception on Windows and come up with a different name
for this Windows-only behavior.

The pseudo-portability of Ruby code has always seemed a bit odd to
me. Obviously this sort of issue comes up with File IO, OS services,
process and thread semantics and so on.

I almost wish that the differences were more obvious via the class
structure rather than methods of the same class having different
semantics on different platforms.

I agree with all of the above. What the win32 fork does is useful in its own way, and it might even be useful to have something on unix/linux that has the same semantics (fork/exec original script). But it needs a different name...

···

On Apr 5, 2007, at 2:54 PM, Joel VanderWerf wrote:

--
        vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407