Fw: Possible bug?

ruby-talk
1

I’m having some trouble sending to ruby-core, so I’ll send this here. Sorry
for the noise.

···

I apologize for not being able to supply much information on this one, but
I’ve noticed some funny behaviour in the pragmatic programmers’ Ruby 1.7.3
win32 build, which I’ve been unable to reproduce in other versions
(including cygwin and linux versions 1.6.x and 1.8). I don’t have the
necessary tools available to compile my own 1.8.0 for win32 (other than
cygwin), so I’ll post this here in hopes that someone else does.

In trying to create a Regex that exceeds the maximum buffer size, I get the
following error message in the pragprog build:

C:>ruby -v -e 'Regexp.new “(\d+)” * 254’
ruby 1.7.3 (2002-11-17) [i386-mswin32]
-e:1:in initialize': regular expression too big: /(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+h⌠@☻Y‼♣►궪☻¶π♂►╪■♂►HO ¿☻►Ö Ñ☻î⌠@☻╗#♣►╨G¿☻°♥ (RegexpError) from -e:1:innew’
from -e:1

I’m not surprised that the error is thrown, but I’m surprised by the strange
characters at the end of the regex string. Looks like ruby is reading off
of the end of the populated buffer. In the cygwin version I just built out
of the cvs HEAD, I get the expected:

$ ./ruby -v -e 'Regexp.new “(\d+)” * 254’
ruby 1.8.0 (2003-04-17) [i386-cygwin]
-e:1:in initialize': regular expression too big: /(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)(d+)( d+)( d+)(d (RegexpError) from -e:1:innew’
from -e:1

In the admittedly small amount of time that I’ve spent looking through the
source, I haven’t been able to find the potential source of this error. Can
anyone confirm that this is still an issue (or not) with a more recent win32
build?

Thanks,
Chad

2

Hello,

In message “Fw: Possible bug?”

I’m not surprised that the error is thrown, but I’m surprised by the strange
characters at the end of the regex string.

It’s a bug (or spec) of some implementations of vsnprintf(),
and ruby uses it for output error message.

In the admittedly small amount of time that I’ve spent looking through the
source, I haven’t been able to find the potential source of this error. Can
anyone confirm that this is still an issue (or not) with a more recent win32
build?

I’ve just committed the fix.

Regards,

···

on Apr.18,2003 14:08:13, chadfowler@chadfowler.com wrote:

U.Nakamura usa@osb.att.ne.jp

3

Hi,

···

At Fri, 18 Apr 2003 15:32:53 +0900, U.Nakamura usa@osb.att.ne.jp wrote:

In the admittedly small amount of time that I’ve spent looking through the
source, I haven’t been able to find the potential source of this error. Can
anyone confirm that this is still an issue (or not) with a more recent win32
build?

I’ve just committed the fix.

vsnprintf() and snprintf() are used many times. And they are
not provided by MSVCRT directly but defiend in win32/win32.h.
I’d propose to define them in win32/win32.c instead.

prefixing with rb_w32_ is necessary for mingw32.

Index: win32/win32.c

RCS file: /pub/cvs/ruby/src/ruby/win32/win32.c,v
retrieving revision 1.84
diff -u -2 -p -r1.84 win32.c
— win32/win32.c 11 Apr 2003 10:02:37 -0000 1.84
+++ win32/win32.c 18 Apr 2003 06:54:49 -0000
@@ -3275,4 +3275,24 @@ rb_w32_utime(const char *path, struct ut
CloseHandle(hDir);

  • return ret;
    +}

+int
+rb_w32_vsnprintf(char *buf, size_t size, const char *format, va_list va)
+{

  • int ret = _vsnprintf(buf, size, format, va);
  • if (size > 0) buf[size - 1] = 0;
  • return ret;
    +}

+int
+rb_w32_snprintf(char *buf, size_t size, const char *format, …)
+{

  • int ret;
  • va_list va;
  • va_start(va, format);
  • ret = vsnprintf(buf, size, format, va);
  • va_end(va);
    return ret;
    }
    Index: win32/win32.h
    ===================================================================
    RCS file: /pub/cvs/ruby/src/ruby/win32/win32.h,v
    retrieving revision 1.38
    diff -u -2 -p -r1.38 win32.h
    — win32/win32.h 4 Mar 2003 14:12:19 -0000 1.38
    +++ win32/win32.h 18 Apr 2003 06:54:32 -0000
    @@ -118,6 +118,6 @@ extern “C++” {
    #define write _write
    #endif
    -#define vsnprintf _vsnprintf
    -#define snprintf _snprintf
    +#define vsnprintf rb_w32_vsnprintf
    +#define snprintf rb_w32_snprintf
    #undef stat
    #define stat(path,st) rb_w32_stat(path,st)
    @@ -166,4 +166,7 @@ extern int rb_w32_rename(const char *
    extern char **rb_w32_get_environ(void);
    extern void rb_w32_free_environ(char **);

+extern int rb_w32_vsnprintf(char *, size_t, const char *, va_list);
+extern int rb_w32_snprintf(char *, size_t, const char *, …);

extern int chown(const char *, int, int);

–
Nobu Nakada

4

Hello,

In message “Re: Possible bug?”

···

on Apr.18,2003 16:04:35, nobu.nokada@softhome.net wrote:

In the admittedly small amount of time that I’ve spent looking through the
source, I haven’t been able to find the potential source of this error. Can
anyone confirm that this is still an issue (or not) with a more recent win32
build?

I’ve just committed the fix.

vsnprintf() and snprintf() are used many times. And they are
not provided by MSVCRT directly but defiend in win32/win32.h.
I’d propose to define them in win32/win32.c instead.

Is this problem only on win32 platforms?
I suspect that this is not right.

Regards,

U.Nakamura usa@osb.att.ne.jp

5

Hi,

···

At Fri, 18 Apr 2003 16:13:27 +0900, U.Nakamura usa@osb.att.ne.jp wrote:

vsnprintf() and snprintf() are used many times. And they are
not provided by MSVCRT directly but defiend in win32/win32.h.
I’d propose to define them in win32/win32.c instead.

Is this problem only on win32 platforms?
I suspect that this is not right.

Other platforms which don’t have them use missing/vsnprintf.c
instead. Win32 platforms use _vsnprintf()/_snprintf(), but
they are not vsnprintf()/snprintf() themselves.

–
Nobu Nakada

6

Hello,

In message “Re: Possible bug?”

vsnprintf() and snprintf() are used many times. And they are
not provided by MSVCRT directly but defiend in win32/win32.h.
I’d propose to define them in win32/win32.c instead.

Is this problem only on win32 platforms?
I suspect that this is not right.

Other platforms which don’t have them use missing/vsnprintf.c
instead. Win32 platforms use _vsnprintf()/_snprintf(), but
they are not vsnprintf()/snprintf() themselves.

(1) Some platforms have the correct (set null at the end of buffer)
(v)snprintf. It’s OK.
(2) Some platforms don’t have (v)snprintf. Ruby uses missing/vsnprintf.c.
It’s OK.
(3) Win32 platforms have _(v)snprintf, and they are broken.
Your patch totally fix the problem, I agree.
(4) I doubt some other platforms exist. The platforms have the wrong
(v)snprintf just like win32’s _(v)snprintf.

If (4) is right, we should fix all (v)snprintf call at all.
I am anxious about it, and this is that I want to say previous mail.

Regards,

···

on Apr.18,2003 16:37:58, nobu.nokada@softhome.net wrote:

U.Nakamura usa@osb.att.ne.jp

7

Hi,

···

At Fri, 18 Apr 2003 16:52:21 +0900, U.Nakamura usa@osb.att.ne.jp wrote:

vsnprintf() and snprintf() are used many times. And they are
not provided by MSVCRT directly but defiend in win32/win32.h.
I’d propose to define them in win32/win32.c instead.

Is this problem only on win32 platforms?
I suspect that this is not right.

Other platforms which don’t have them use missing/vsnprintf.c
instead. Win32 platforms use _vsnprintf()/_snprintf(), but
they are not vsnprintf()/snprintf() themselves.

(1) Some platforms have the correct (set null at the end of buffer)
(v)snprintf. It’s OK.
(2) Some platforms don’t have (v)snprintf. Ruby uses missing/vsnprintf.c.
It’s OK.
(3) Win32 platforms have _(v)snprintf, and they are broken.
Your patch totally fix the problem, I agree.
(4) I doubt some other platforms exist. The platforms have the wrong
(v)snprintf just like win32’s _(v)snprintf.

If (4) is right, we should fix all (v)snprintf call at all.

You’re right. But snprintf() is used so many times, isn’t it
better to make a function or a macro?

And also, err_snprintf() wrongly assumes that snprintf() never
overflow or returns truncated size.

–
Nobu Nakada

8

I would suggest commiting the fix in the win32 port and then adding a
rubicon test to try to catch it in other platforms. Since we don’t know of
any other affected platforms, this might be the best way. In any case, a
rubicon test to catch the bug wouldn’t be a bad idea.

Chad

···

----- Original Message -----
From: “U.Nakamura” usa@osb.att.ne.jp
To: “ruby-talk ML” ruby-talk@ruby-lang.org
Sent: Friday, April 18, 2003 1:22 PM
Subject: Re: Possible bug?

Your patch totally fix the problem, I agree.

(4) I doubt some other platforms exist. The platforms have the wrong
(v)snprintf just like win32’s _(v)snprintf.

9

Scripsit ille aut illa »U.Nakamura« usa@osb.att.ne.jp:

In message “Re: Possible bug?”

vsnprintf() and snprintf() are used many times. And they are
not provided by MSVCRT directly but defiend in win32/win32.h.
I’d propose to define them in win32/win32.c instead.

Is this problem only on win32 platforms?
I suspect that this is not right.

Other platforms which don’t have them use missing/vsnprintf.c
instead. Win32 platforms use _vsnprintf()/_snprintf(), but
they are not vsnprintf()/snprintf() themselves.

(1) Some platforms have the correct (set null at the end of buffer)
(v)snprintf. It’s OK.
(2) Some platforms don’t have (v)snprintf. Ruby uses missing/vsnprintf.c.
It’s OK.
(3) Win32 platforms have _(v)snprintf, and they are broken.
Your patch totally fix the problem, I agree.
(4) I doubt some other platforms exist. The platforms have the wrong
(v)snprintf just like win32’s _(v)snprintf.

If (4) is right, we should fix all (v)snprintf call at all.
I am anxious about it, and this is that I want to say previous mail.

Can’t one easily check in a configure script for that?

/* conftest.c */
#include <stdio.h>
int main()
{
char out = 2;
snprintf(&out, 1, “\001”);
return out;
}

possible outcomes:

  • cannot compile/link: no snprintf;
    • compile confgtest.c with -Dsnprintf=_snprintf
      • cannot compile/link: use missing/
      • 0: #define rb_snprintf _snprintf
      • 1: #define rb_snprintf_to_fix _snprintf
        #define rb_snprintf rb_fixed_w32_snprintf
      • otherwise: use missing/
  • 0: #define rb_snprintf snprintf
  • 1: #define rb_snprintf_to_fix snprintf
    #define rb_snprintf rb_fixed_w32_snprintf
  • otherwise: use missing/

Shouldn’t that fix the problem in any case?

··· 
on Apr.18,2003 16:37:58, <nobu.nokada@softhome.net> wrote:

–
Desweiteren fiel mir auf, das die Kommandozeile immer statt ‘?’ ein
Fragezeichen generierte.
[Da_David in t-online.homepage.technik; OE hat mal wieder versagt]

10

I forgot in my last mail to say “Thanks!” Every so often, I’m re-amazed at
the speed and responsivenes of how open source development works. :slight_smile:

Chad

···

----- Original Message -----
From: nobu.nokada@softhome.net
To: “ruby-talk ML” ruby-talk@ruby-lang.org
Sent: Friday, April 18, 2003 2:39 PM
Subject: Re: Possible bug?

Hi,

At Fri, 18 Apr 2003 16:52:21 +0900, > U.Nakamura usa@osb.att.ne.jp wrote:

vsnprintf() and snprintf() are used many times. And they are
not provided by MSVCRT directly but defiend in win32/win32.h.
I’d propose to define them in win32/win32.c instead.

Is this problem only on win32 platforms?
I suspect that this is not right.

Other platforms which don’t have them use missing/vsnprintf.c
instead. Win32 platforms use _vsnprintf()/_snprintf(), but
they are not vsnprintf()/snprintf() themselves.

(1) Some platforms have the correct (set null at the end of buffer)
(v)snprintf. It’s OK.
(2) Some platforms don’t have (v)snprintf. Ruby uses
missing/vsnprintf.c.
It’s OK.
(3) Win32 platforms have _(v)snprintf, and they are broken.
Your patch totally fix the problem, I agree.
(4) I doubt some other platforms exist. The platforms have the wrong
(v)snprintf just like win32’s _(v)snprintf.

If (4) is right, we should fix all (v)snprintf call at all.

You’re right. But snprintf() is used so many times, isn’t it
better to make a function or a macro?

And also, err_snprintf() wrongly assumes that snprintf() never
overflow or returns truncated size.

–
Nobu Nakada