Bug: threads and win32

Hatespyware · 23 December 2003 08:56

Hi. I use Arton X’s Activeruby package (ruby 1.8.0 (2003-08-04)
[i386-mswin32]), since it lets me do WSH stuff (ASP, COM, etc). I am
new to Ruby, and may have just made a luser error, but the following
does not work for me:

tt = Thread.new do
puts "listening…"
sleep 2
puts "listening…"
end

$stdin.each do |command|
break if command=~/quit/i or command=~/exit/i
tt.join if command=~/join/i
end

I expect it to print “listening…” twice, and it only prints it once.
If I type join, then I get the second message. The code works as
expected under Linux. I haven’t tried other Windows builds, because
they don’t include the WScript stuff.

Is this a common problem? Are there good workarounds? I thought Ruby
implemented the threads in some internal system independant way, so
this is confusing to me. Any help/info would be greatly appreciated.

thanks

Laurent_Julliard4 · 23 December 2003 11:36

I may be wrong but there is no workaround for this. Ruby Threads do
block on IO within windows.
IIRC that should be because ruby transparently uses select() to hnadle
IO, but select() on windows only works with sockets, and thus any
other IO blocks the whole interpreter. Feel free to hack ruby and fix
this

···

il 23 Dec 2003 00:52:14 -0800, hatespyware@yahoo.com (anon luker) ha scritto::

Is this a common problem? Are there good workarounds? I thought Ruby
implemented the threads in some internal system independant way, so
this is confusing to me. Any help/info would be greatly appreciated.

Hatespyware · 24 December 2003 00:21

gabriele renzi surrender_it@remove.yahoo.it wrote in message news:id8guvgqphehimf3igjj6cd4t0dlpg74i7@4ax.com…

il 23 Dec 2003 00:52:14 -0800, hatespyware@yahoo.com (anon luker) ha
scritto::

Is this a common problem? Are there good workarounds? I thought Ruby
implemented the threads in some internal system independant way, so
this is confusing to me. Any help/info would be greatly appreciated.

I may be wrong but there is no workaround for this. Ruby Threads do
block on IO within windows.
IIRC that should be because ruby transparently uses select() to hnadle
IO, but select() on windows only works with sockets, and thus any
other IO blocks the whole interpreter.
Yuck! On a hunch, I tried the cygwin version. It works, but very
slowly. Far too slow for socket programming. I must be missing
something, since the official tagline for Ruby includes “Ruby features
OS independent threading. Thus, for all platforms on which Ruby runs,
you also have multithreading, regardless of if the OS supports it or
not, even on MS-DOS! ;-)” What I want to do seems pretty basic,
though, so if Ruby supports threads and can’t meet my needs, then
something is wrong.

Feel free to hack ruby and fix
this
I’ll just use java or something for this project. Maybe I’ll be lucky
enough to get paid to hack on ruby one day… until then, I’m
constrained to using working solutions.

My followup question is: How does one do asynch io (ala sockets) in
Ruby under Windows? Threads seemed like the most natural way, but
nonblocking IO may work, too. Unfortunately, the langauge reference
that I have doesn’t really document all of the SO_* flags that the
recv line of socket methods support and a quick look at the constants
doesn’t show anything obvious like SO_NOBLOCK. Any advice would be
appreciated.

thanks

Park_Heesob3 · 24 December 2003 02:50

Hi,

Is this a common problem? Are there good workarounds? I thought Ruby
implemented the threads in some internal system independant way, so
this is confusing to me. Any help/info would be greatly appreciated.

I may be wrong but there is no workaround for this. Ruby Threads do
block on IO within windows.
IIRC that should be because ruby transparently uses select() to hnadle
IO, but select() on windows only works with sockets, and thus any
other IO blocks the whole interpreter.
Yuck! On a hunch, I tried the cygwin version. It works, but very
slowly. Far too slow for socket programming. I must be missing
something, since the official tagline for Ruby includes “Ruby features
OS independent threading. Thus, for all platforms on which Ruby runs,
you also have multithreading, regardless of if the OS supports it or
not, even on MS-DOS! ;-)” What I want to do seems pretty basic,
though, so if Ruby supports threads and can’t meet my needs, then
something is wrong.

Feel free to hack ruby and fix
this
I’ll just use java or something for this project. Maybe I’ll be lucky
enough to get paid to hack on ruby one day… until then, I’m
constrained to using working solutions.

My followup question is: How does one do asynch io (ala sockets) in
Ruby under Windows? Threads seemed like the most natural way, but
nonblocking IO may work, too. Unfortunately, the langauge reference
that I have doesn’t really document all of the SO_* flags that the
recv line of socket methods support and a quick look at the constants
doesn’t show anything obvious like SO_NOBLOCK. Any advice would be
appreciated.

thanks

Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I belive it will fix thread and blocking IO problem whinin Windows.

···

From: “anon luker” hatespyware@yahoo.com

@@ -1775,6 +1775,13 @@
rb_w32_select (int nfds, fd_set *rd, fd_set *wr, fd_set *ex,
struct timeval *timeout)
{

DWORD ms_total, limit;
HANDLE handles[MAXIMUM_WAIT_OBJECTS];
int handle_slot_to_fd[MAXIMUM_WAIT_OBJECTS];
int n_handles = 0, i;
fd_set aread, awrite, aexcept;
int retcode;
long r;
fd_set file_rd;
fd_set file_wr;
@@ -1783,6 +1790,28 @@
#endif /* USE_INTERRUPT_WINSOCK */
int file_nfds;

+#define SAFE_FD_ISSET(fd, set) (set != NULL && rb_w32_fdisset(fd, set))
+

/* calculate how long we need to wait in milliseconds */
if (timeout == NULL) {
ms_total = INFINITE;
} else {
ms_total = timeout->tv_sec * 1000;
ms_total += timeout->tv_usec / 1000;
}
/* build an array of handles for non-sockets */
for (i = 0; i < nfds; i++) {
if (SAFE_FD_ISSET(i, rd) || SAFE_FD_ISSET(i, wr) || SAFE_FD_ISSET(i, ex))
{
handles[n_handles] = (HANDLE)_get_osfhandle(i);
if ((DWORD)handles[n_handles] != 0xffffffff) {
handle_slot_to_fd[n_handles] = i;
n_handles++;
}
}
}
if (!NtSocketsInitialized) {
StartSockets();
}
@@ -1799,10 +1828,49 @@
file_nfds += extract_file_fd(wr, &file_wr);
if (file_nfds)
{

// assume normal files are always readable/writable
// fake read/write fd_set and return value
if (rd) *rd = file_rd;
if (wr) *wr = file_wr;

FD_ZERO(&aread);
FD_ZERO(&awrite);
FD_ZERO(&aexcept);
```
   limit = GetTickCount() + ms_total;
```
```
   do {
```
```
DWORD wret;
```
```
       retcode = 0;
```

wret = MsgWaitForMultipleObjects(n_handles, handles, FALSE, retcode >

0 ? 0 : 100, QS_ALLEVENTS);
+

```
if (wret == WAIT_TIMEOUT) {
```
/* set retcode to 0; this is the default.
- select() may have set it to something else,
- in which case we leave it alone, so this branch
- does nothing */
;
```
   } else if (wret == WAIT_FAILED) {
```
if (retcode == 0) {
retcode = -1;
}
```
 } else {
```
if (retcode < 0) {
retcode = 0;
}
```
    for (i = 0; i < n_handles; i++) {
```

 if (WAIT_OBJECT_0 == WaitForSingleObject(handles[i], 0)) {

if (SAFE_FD_ISSET(handle_slot_to_fd[i], rd)) {
rb_w32_fdset(handle_slot_to_fd[i], &aread);
}
if (SAFE_FD_ISSET(handle_slot_to_fd[i], wr)) {
rb_w32_fdset(handle_slot_to_fd[i], &awrite);
}
if (SAFE_FD_ISSET(handle_slot_to_fd[i], ex)) {
rb_w32_fdset(handle_slot_to_fd[i], &aexcept);
}
retcode++;
```
 }
```
}
```
       }
```

   } while (retcode == 0 && (ms_total == INFINITE || GetTickCount() <

limit));

if (rd) *rd = aread;
if (wr) *wr = awrite;
return file_nfds;
}
===============================================================

Regards,

Park Heesob

Hatespyware · 24 December 2003 08:01

“Park Heesob” phasis@bcline.com wrote in message news:008901c3c9c8$aa7c9780$6a1e213d@intheline.com…

Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I belive it will fix thread and blocking IO problem whinin Windows.
Regards,

Park Heesob

Woohoo! Thanks, Park! I will give this a try.

Laurent_Julliard4 · 26 December 2003 01:17

have you tried it? If it works, why don’t you submit it to ruby-core ?
This would be really cool

BTW, happy xmas to everyone !

···

il Wed, 24 Dec 2003 11:50:14 +0900, “Park Heesob” phasis@bcline.com ha scritto::

Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I belive it will fix thread and blocking IO problem whinin Windows.

U.Nakamura1 · 26 December 2003 01:57

Hello,

In message “Re: bug: threads and win32”

···

on Dec.26,2003 10:17:32, surrender_it@rc1.vip.ukl.yahoo.com wrote:

Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I belive it will fix thread and blocking IO problem whinin Windows.

have you tried it? If it works, why don’t you submit it to ruby-core ?
This would be really cool

Of course, we (ruby windows version maintainers) are testing
this patch. and already found some problems.
(for example, `ruby test/runner.rb’ doesn’t run…)

Now we are trying to fix the problems to import this patch to
ruby 1.9 (and 1.8.2, if we can…).

Regards,

U.Nakamura usa@osb.att.ne.jp

Park_Heesob3 · 26 December 2003 04:05

Hi,

Hello,

In message “Re: bug: threads and win32”

Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I believe it will fix thread and blocking IO problem whinin Windows.

have you tried it? If it works, why don’t you submit it to ruby-core ?
This would be really cool

Of course, we (ruby windows version maintainers) are testing
this patch. and already found some problems.
(for example, `ruby test/runner.rb’ doesn’t run…)

Now we are trying to fix the problems to import this patch to
ruby 1.9 (and 1.8.2, if we can…).

Here is the patch for ruby 1.8.1.
At least, it works with test/runner.rb.

···

From: “U.Nakamura” usa@osb.att.ne.jp

on Dec.26,2003 10:17:32, <surrender_it@rc1.vip.ukl.yahoo.com> wrote:

=================================================================
— win32.c Fri Dec 26 12:47:35 2003
+++ win32.c Fri Dec 26 12:46:44 2003
@@ -1819,6 +1819,13 @@
rb_w32_select (int nfds, fd_set *rd, fd_set *wr, fd_set *ex,
struct timeval *timeout)
{

DWORD ms_total, limit;
HANDLE handles[MAXIMUM_WAIT_OBJECTS];
int handle_slot_to_fd[MAXIMUM_WAIT_OBJECTS];
int n_handles = 0, i;
fd_set aread, awrite, aexcept;
int retcode;
long r;
fd_set file_rd;
fd_set file_wr;
@@ -1827,6 +1834,26 @@
#endif /* USE_INTERRUPT_WINSOCK */
int file_nfds;

+#define SAFE_FD_ISSET(fd, set) (set != NULL && rb_w32_fdisset(fd, set))

/* calculate how long we need to wait in milliseconds */
if (timeout == NULL) {
ms_total = INFINITE;
} else {
ms_total = timeout->tv_sec * 1000;
ms_total += timeout->tv_usec / 1000;
}
/* build an array of handles for non-sockets */
for (i = 0; i <= nfds; i++) {
if (SAFE_FD_ISSET(i, rd) || SAFE_FD_ISSET(i, wr)) {
handles[n_handles] = (HANDLE)TO_SOCKET(i);
if ((DWORD)handles[n_handles] != 0xffffffff &&
!is_socket((SOCKET)handles[n_handles]) && (SOCKET)handles[n_handles]<20) {
/* 20 is trial & error */
handle_slot_to_fd[n_handles] = i;
n_handles++;
}
}
}
if (!NtSocketsInitialized) {
StartSockets();
}
@@ -1843,10 +1870,56 @@
file_nfds += extract_file_fd(wr, &file_wr);
if (file_nfds)
{
```
   if(n_handles>0) {
```
FD_ZERO(&aread);
FD_ZERO(&awrite);

     limit = GetTickCount() + ms_total;

```
     do {
```
```
DWORD wret;
```
```
       retcode = 0;
```

wret = MsgWaitForMultipleObjects(n_handles, handles, FALSE, retcode >

0 ? 0 : 100, QS_ALLEVENTS);
+

```
if (wret == WAIT_TIMEOUT) {
```
/* set retcode to 0; this is the default.
- select() may have set it to something else,
- in which case we leave it alone, so this branch
- does nothing */
;
```
   } else if (wret == WAIT_FAILED) {
```
if (retcode == 0) {
retcode = -1;
}
```
 } else {
```
if (retcode < 0) {
retcode = 0;
}
```
    for (i = 0; i < n_handles; i++) {
```

 if (WAIT_OBJECT_0 == WaitForSingleObject(handles[i], 0)) {

if (SAFE_FD_ISSET(handle_slot_to_fd[i], rd)) {
rb_w32_fdset(handle_slot_to_fd[i], &aread);
}
if (SAFE_FD_ISSET(handle_slot_to_fd[i], wr)) {
rb_w32_fdset(handle_slot_to_fd[i], &awrite);
}
retcode++;
```
 }
```
}
```
       }
```

     } while (retcode == 0 && (ms_total == INFINITE || GetTickCount()

< limit));

if (rd) *rd = aread;
if (wr) *wr = awrite;
```
   }
```
```
   else {
```
// assume normal files are always readable/writable
// fake read/write fd_set and return value
if (rd) *rd = file_rd;
if (wr) *wr = file_wr;
```
   }
```
return file_nfds;
}

===================================================================

Regards,

Park Heesob

Laurent_Julliard4 · 27 December 2003 02:11

thank you, and Park (and the php dev too)

···

il Fri, 26 Dec 2003 10:57:57 +0900, “U.Nakamura” usa@osb.att.ne.jp ha scritto::

Of course, we (ruby windows version maintainers) are testing
this patch. and already found some problems.
(for example, `ruby test/runner.rb’ doesn’t run…)

Now we are trying to fix the problems to import this patch to
ruby 1.9 (and 1.8.2, if we can…).

U.Nakamura1 · 26 December 2003 08:27

Hello,

In message “Re: bug: threads and win32”

Here is the patch for ruby 1.8.1.
At least, it works with test/runner.rb.

Oh, great!
It works fine on my machine, too.

Next, try this

require ‘timeout’

timeout(5) do
while true
p ‘a’
end
end

This script should print “a” many many times, and raise
Timeout::Error excption after 5 seconds.
But, with your patch, sometimes it prints only one “a”.

Sorry, I haven’t debug this problem yet. this is just a
report.

Regards,

···

on Dec.26,2003 13:05:50, phasis@bcline.com wrote:

U.Nakamura usa@osb.att.ne.jp

Park_Heesob3 · 26 December 2003 09:15

Hi,

Oh, great!
It works fine on my machine, too.

Next, try this

require ‘timeout’

timeout(5) do
while true
p ‘a’
end
end

This script should print “a” many many times, and raise
Timeout::Error excption after 5 seconds.
But, with your patch, sometimes it prints only one “a”.

Sorry, I haven’t debug this problem yet. this is just a
report.

OK.
I don’t want make another problems any more.
This path will fix only standard input blocking problem

···

From: “U.Nakamura” usa@osb.att.ne.jp

— win32.c Fri Dec 26 12:47:35 2003
+++ win32.c Fri Dec 26 12:46:44 2003
@@ -1819,6 +1819,13 @@
rb_w32_select (int nfds, fd_set *rd, fd_set *wr, fd_set *ex,
struct timeval *timeout)
{

DWORD ms_total, limit;
HANDLE handles[MAXIMUM_WAIT_OBJECTS];
int handle_slot_to_fd[MAXIMUM_WAIT_OBJECTS];
int n_handles = 0, i;
fd_set aread, awrite, aexcept;
int retcode;
long r;
fd_set file_rd;
fd_set file_wr;
@@ -1827,6 +1834,26 @@
#endif /* USE_INTERRUPT_WINSOCK */
int file_nfds;

+#define SAFE_FD_ISSET(fd, set) (set != NULL && rb_w32_fdisset(fd, set))

/* calculate how long we need to wait in milliseconds */
if (timeout == NULL) {
ms_total = INFINITE;
} else {
ms_total = timeout->tv_sec * 1000;
ms_total += timeout->tv_usec / 1000;
}
/* build an array of handles for non-sockets */
for (i = 0; i <= nfds; i++) {
if (SAFE_FD_ISSET(i, rd) || SAFE_FD_ISSET(i, wr)) {
handles[n_handles] = (HANDLE)TO_SOCKET(i);
if ((SOCKET)handles[n_handles]==TO_SOCKET(0)) { /* only treat stdin */
handle_slot_to_fd[n_handles] = i;
n_handles++;
}
}
}
if (!NtSocketsInitialized) {
StartSockets();
}
@@ -1843,10 +1870,56 @@
file_nfds += extract_file_fd(wr, &file_wr);
if (file_nfds)
{
```
   if(n_handles>0) {
```
FD_ZERO(&aread);
FD_ZERO(&awrite);

     limit = GetTickCount() + ms_total;

```
     do {
```
```
DWORD wret;
```
```
       retcode = 0;
```

wret = MsgWaitForMultipleObjects(n_handles, handles, FALSE, retcode >

0 ? 0 : 100, QS_ALLEVENTS);
+

```
if (wret == WAIT_TIMEOUT) {
```
/* set retcode to 0; this is the default.
- select() may have set it to something else,
- in which case we leave it alone, so this branch
- does nothing */
;
```
   } else if (wret == WAIT_FAILED) {
```
if (retcode == 0) {
retcode = -1;
}
```
 } else {
```
if (retcode < 0) {
retcode = 0;
}
```
    for (i = 0; i < n_handles; i++) {
```

 if (WAIT_OBJECT_0 == WaitForSingleObject(handles[i], 0)) {

if (SAFE_FD_ISSET(handle_slot_to_fd[i], rd)) {
rb_w32_fdset(handle_slot_to_fd[i], &aread);
}
if (SAFE_FD_ISSET(handle_slot_to_fd[i], wr)) {
rb_w32_fdset(handle_slot_to_fd[i], &awrite);
}
retcode++;
```
 }
```
}
```
       }
```

     } while (retcode == 0 && (ms_total == INFINITE || GetTickCount()

< limit));

if (rd) *rd = aread;
if (wr) *wr = awrite;
```
   }
```
```
   else {
```
// assume normal files are always readable/writable
// fake read/write fd_set and return value
if (rd) *rd = file_rd;
if (wr) *wr = file_wr;
```
   }
```
return file_nfds;
}

===================================================================
Regards,

Park Heesob

Topic		Replies	Views
Ruby Mulithreading ruby-talk	3	82	25 May 2009
[ANN] win32-thread 0.0.1 (or, do you feel lucky?) ruby-talk	5	95	16 December 2004
Blocking Calls on Win32 Ruby ruby-talk	6	156	23 February 2004
Is this a threading bug in Ruby-185-21 on Win32? ruby-talk	4	104	27 March 2008
Possible ruby bug involving threads and IO ruby-talk	2	103	4 January 2005

Bug: threads and win32

From: “anon luker” hatespyware@yahoo.com

Regards,

on Dec.26,2003 13:05:50, phasis@bcline.com wrote:

From: “U.Nakamura” usa@osb.att.ne.jp

Related Topics