Threading on Win32 - at an impasse

(Bill Atkins) #1

I'm working on a project that would have to run several TCPServer's in
the background while accepting input from the user; it has to run on
both Win32 and UNIX.

It runs without issue on UNIX, but I can't think of any clean way to
get this arrangement to work on Win32. If I use threads for the
servers, these threads will all block while the main thread is waiting
for input for the user (I'm using readline, so I don't think the kbhit
and getch hack (mentioned in another thread I started) would work
without giving up the functionality of readline). The servers in the
threads will not handle input until readline gets a whole line from
STDIN. Win32::Thread doesn't support more than one thread at a time,
so Ruby's builtin Threads remain the only solution along these lines.

If I use separate processes for the servers, interprocess
communication gets a bit messy, and there is also the issue of
Win32::Process segfaulting whenever I attempt to kill a process (the
application must be able to kill its child processes). Separate
processes introduce a lot more complexity than is really necessary for
this project.

My other option is to package the Cygwin DLL and the Cygwin ruby
binaries with my application, since Cygwin doesn't have blocking
issues that arise with native Win32. The trouble is that I also have
a Gtk interface, and I suspect that running Gtk through Cygwin's X
server would be a nuisance, in addition to bloating the distribution
considerably.

Is there some elegant way to wait for user input while having
networking threads running in the background?

···

--
Bill Atkins

--
Bill Atkins

(David Vallner) #2

Citát Bill Atkins <batkins57@gmail.com>:

I'm using readline, so I don't think the kbhit and getch hack (mentioned in
another thread I started) would work without giving up the functionality of
readline).

Oooh, nested parentheses. Don't take this as a pro's advice, just first ideas I
get when I see someone using readline on Win32. Most of what readline does at
the end-user's level is handled by DOSKEY on Win32 if you use native buffered
input methods, so you might go off and use that in the Win32 version of the
project.

If you're using readline for some of its functionality I have no idea of as a a
person without any Unix programming experience whatsoever, please do ignore me
completely.

David Vallner

(Joel VanderWerf) #3

Bill Atkins wrote:

Is there some elegant way to wait for user input while having
networking threads running in the background?

Not very helpful, but what about hacking FXIrb and doing your user input
in a FXRuby window (on win32). I can't remember if FXIrb had readline
functionality though.

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

(Tanaka Akira) #4

In article <66b7e34b050810082824108293@mail.gmail.com>,
  Bill Atkins <batkins57@gmail.com> writes:

Is there some elegant way to wait for user input while having
networking threads running in the background?

I heard usa and nobu wrote a patch that Ruby's internal event driven
mechanism support Windows console. I don't know current status,
though.

···

--
Tanaka Akira

(Bill Atkins) #5

I'm using readine for its completion and history features.

···

On 8/10/05, david@vallner.net <david@vallner.net> wrote:

Citát Bill Atkins <batkins57@gmail.com>:

> I'm using readline, so I don't think the kbhit and getch hack (mentioned in
> another thread I started) would work without giving up the functionality of
> readline).

Oooh, nested parentheses. Don't take this as a pro's advice, just first ideas I
get when I see someone using readline on Win32. Most of what readline does at
the end-user's level is handled by DOSKEY on Win32 if you use native buffered
input methods, so you might go off and use that in the Win32 version of the
project.

If you're using readline for some of its functionality I have no idea of as a a
person without any Unix programming experience whatsoever, please do ignore me
completely.

David Vallner

--
Bill Atkins

(David Vallner) #6

Citát Bill Atkins <batkins57@gmail.com>:

I'm using readine for its completion and history features.

Hmm. DOSKEY does do history automagically, I also have a hunch IO#gets is
implemented using standard buffered IO calls. I do have serious doubts about
completion though.

David Vallner

(Bill Atkins) #7

All right, just in case anyone googles this later on, I was able to
get my application working with the help of win32-process. I used
separate processes instead of threads, because calling gets on stdin
seems to block all threads no matter what.

So it seems like separate processes are the way to go, at least until
Ruby 2.0 (or whenever it gets native threads).

Bill

···

On 8/11/05, david@vallner.net <david@vallner.net> wrote:

Citát Bill Atkins <batkins57@gmail.com>:

> I'm using readine for its completion and history features.

Hmm. DOSKEY does do history automagically, I also have a hunch IO#gets is
implemented using standard buffered IO calls. I do have serious doubts about
completion though.

David Vallner

--
Bill Atkins

(Ara.T.Howard) #8

can you post a baby example?

cheers.

-a

···

On Thu, 11 Aug 2005, Bill Atkins wrote:

All right, just in case anyone googles this later on, I was able to
get my application working with the help of win32-process. I used
separate processes instead of threads, because calling gets on stdin
seems to block all threads no matter what.

So it seems like separate processes are the way to go, at least until
Ruby 2.0 (or whenever it gets native threads).

--

email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
Your life dwells amoung the causes of death Like a lamp standing in a strong breeze. --Nagarjuna

===============================================================================

(Bill Atkins) #9

Sure. This solution might not work for everyone, because fortunately
I don't really have to do much interprocess communication aside from
being able to start and stop children at will. So I have a simulation
class:

require 'win32/process'

class Simulation
  .... blah blah blah

  def start
    @pid = Process.run "ruby bin/run_device.rb #@dev_name #@name"
  end

  def kill
    Process.kill 9, @pid if @pid
    @pid = nil
  end

   .......blah blah blah
end

And the bin/run_device.rb file loads up the appropriate simulation and
does its thing in the background. run_device.rb is the important part
- it is more or less my answer to Win32's lack of fork; it loads the
library files it needs and then starts up the simulator, so it's as if
I've forked the process, but not really. :slight_smile: run_device.rb's code is
long and not too interesting (and application-dependent in any case),
so I won't post it here.

Process.run is a custom function that Does The Right Thing based on
what OS the program is running on:

module Process
  def self.run app
    if RUBY_PLATFORM =~ /win32/
      puts "calling create"
      Process.create :app_name => app
    else
      puts "calling fork"
      fork do
        exec app
      end
    end
  end
end

If only Microsoft would implement fork/exec........

Sigh.

Hope that helps,
Bill

···

On 8/11/05, Ara.T.Howard <Ara.T.Howard@noaa.gov> wrote:

On Thu, 11 Aug 2005, Bill Atkins wrote:

> All right, just in case anyone googles this later on, I was able to
> get my application working with the help of win32-process. I used
> separate processes instead of threads, because calling gets on stdin
> seems to block all threads no matter what.
>
> So it seems like separate processes are the way to go, at least until
> Ruby 2.0 (or whenever it gets native threads).

can you post a baby example?

cheers.

-a
--

> email :: ara [dot] t [dot] howard [at] noaa [dot] gov
> phone :: 303.497.6469
> Your life dwells amoung the causes of death
> Like a lamp standing in a strong breeze. --Nagarjuna

--
Bill Atkins

(Ara.T.Howard) #10

it does - there is a win32-fork - guess you can't exec then? i don't really
know the semantic difference between win fork and nix fork.

thanks alot for the example - i've faced this same issue before and gave up.
:wink:

-a

···

On Thu, 11 Aug 2005, Bill Atkins wrote:

Sure. This solution might not work for everyone, because fortunately
I don't really have to do much interprocess communication aside from
being able to start and stop children at will. So I have a simulation
class:

require 'win32/process'

class Simulation
.... blah blah blah

def start
   @pid = Process.run "ruby bin/run_device.rb #@dev_name #@name"
end

def kill
   Process.kill 9, @pid if @pid
   @pid = nil
end

  .......blah blah blah
end

And the bin/run_device.rb file loads up the appropriate simulation and
does its thing in the background. run_device.rb is the important part
- it is more or less my answer to Win32's lack of fork; it loads the
library files it needs and then starts up the simulator, so it's as if
I've forked the process, but not really. :slight_smile: run_device.rb's code is
long and not too interesting (and application-dependent in any case),
so I won't post it here.

Process.run is a custom function that Does The Right Thing based on
what OS the program is running on:

module Process
def self.run app
   if RUBY_PLATFORM =~ /win32/
     puts "calling create"
     Process.create :app_name => app
   else
     puts "calling fork"
     fork do
       exec app
     end
   end
end
end

If only Microsoft would implement fork/exec........

Sigh.

Hope that helps,

--

email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
Your life dwells amoung the causes of death Like a lamp standing in a strong breeze. --Nagarjuna

===============================================================================

(Berger, Daniel) #11

Ara.T.Howard wrote:

it does - there is a win32-fork - guess you can't exec then? i don't really
know the semantic difference between win fork and nix fork.

thanks alot for the example - i've faced this same issue before and gave up.
:wink:

-a

There is no win32-fork. There's an implementation of fork in win32-process that creatively wraps CreateProcess(). Windows has no fork/exec, though it has CreateProcess() and 16 different flavors of spawn().

Regards,

Dan

(Bill Atkins) #12

There's a Process.fork in win32-process, but despite the name it does
not fork. :slight_smile:

When you call Process.fork (the win32-process version), it will
re-execute everything leading up to the fork call. From that point
on, the child process will execute the contents of the associated
block and the parent will continue after the block.

On UNIX, calling fork makes a copy of the program and all the data
associated with it in the process table. So you get an identical
process created and control literally forks at the point of the fork
call - the child process does one thing and the parent does something
else. UNIX fork does not re-run any code before that fork.

Technically, anything but UNIX fork is not really forking. Microsoft
products don't have this concept at all; you can only create brand-new
processes. I don't know of any real way to emulate forking without
access to the operating system itself, so Win32 users are probably
stuck as far as this goes.

Bill

···

On 8/11/05, Ara.T.Howard <Ara.T.Howard@noaa.gov> wrote:

On Thu, 11 Aug 2005, Bill Atkins wrote:

> Sure. This solution might not work for everyone, because fortunately
> I don't really have to do much interprocess communication aside from
> being able to start and stop children at will. So I have a simulation
> class:
>
> require 'win32/process'
>
> class Simulation
> .... blah blah blah
>
> def start
> @pid = Process.run "ruby bin/run_device.rb #@dev_name #@name"
> end
>
> def kill
> Process.kill 9, @pid if @pid
> @pid = nil
> end
>
> .......blah blah blah
> end
>
> And the bin/run_device.rb file loads up the appropriate simulation and
> does its thing in the background. run_device.rb is the important part
> - it is more or less my answer to Win32's lack of fork; it loads the
> library files it needs and then starts up the simulator, so it's as if
> I've forked the process, but not really. :slight_smile: run_device.rb's code is
> long and not too interesting (and application-dependent in any case),
> so I won't post it here.
>
> Process.run is a custom function that Does The Right Thing based on
> what OS the program is running on:
>
> module Process
> def self.run app
> if RUBY_PLATFORM =~ /win32/
> puts "calling create"
> Process.create :app_name => app
> else
> puts "calling fork"
> fork do
> exec app
> end
> end
> end
> end
>
> If only Microsoft would implement fork/exec........
>
> Sigh.
>
> Hope that helps,

it does - there is a win32-fork - guess you can't exec then? i don't really
know the semantic difference between win fork and nix fork.

thanks alot for the example - i've faced this same issue before and gave up.
:wink:

-a
--

> email :: ara [dot] t [dot] howard [at] noaa [dot] gov
> phone :: 303.497.6469
> Your life dwells amoung the causes of death
> Like a lamp standing in a strong breeze. --Nagarjuna

--
Bill Atkins

(Ara.T.Howard) #13

right-o. i guess what wondering what the semantics of this wrapper are - do
fd's get inherited, do parents need to reap chilren, etc. it seems that
something called CreateProcess)() must be alot like fork/exec, if not what's
the difference. i realize this is ot and i'm googling it now - but wondered
if anyone might comment on any of this specific to ruby or not.

cheers.

-a

···

On Thu, 11 Aug 2005, Daniel Berger wrote:

Ara.T.Howard wrote:

it does - there is a win32-fork - guess you can't exec then? i don't really
know the semantic difference between win fork and nix fork.

thanks alot for the example - i've faced this same issue before and gave up.
:wink:

-a

There is no win32-fork. There's an implementation of fork in win32-process that creatively wraps CreateProcess(). Windows has no fork/exec, though it has CreateProcess() and 16 different flavors of spawn().

--

email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
Your life dwells amoung the causes of death Like a lamp standing in a strong breeze. --Nagarjuna

===============================================================================

(Daniel Berger) #14

Ara.T.Howard wrote:

···

On Thu, 11 Aug 2005, Daniel Berger wrote:

> Ara.T.Howard wrote:
>
>> it does - there is a win32-fork - guess you can't exec then? i don't
>> really
>> know the semantic difference between win fork and nix fork.
>>
>> thanks alot for the example - i've faced this same issue before and gave
>> up.
>> :wink:
>>
>> -a
>
> There is no win32-fork. There's an implementation of fork in win32-process
> that creatively wraps CreateProcess(). Windows has no fork/exec, though it
> has CreateProcess() and 16 different flavors of spawn().

right-o. i guess what wondering what the semantics of this wrapper are - do
fd's get inherited, do parents need to reap chilren, etc. it seems that
something called CreateProcess)() must be alot like fork/exec, if not what's
the difference. i realize this is ot and i'm googling it now - but wondered
if anyone might comment on any of this specific to ruby or not.

A pretty good guide on the differences can be found at
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnucmg/html/UCMGch01.asp

It's a pretty good guide in general when trying to write portable code
between Windows and Unix.

Regards,

Dan

(daz) #15

Daniel Berger wrote:

[...]

   http://msdn.microsoft.com/library/en-us/dnucmg/html/ucmglp.asp

   (replacement link)

···

It's a pretty good guide in general when trying to write portable code
between Windows and Unix.

Regards,

Dan