Problem with popen on windows

Kroeger_Simon_ext · 8 September 2005 10:18

Hi all,

i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000.

When starting my GUI app with rubyw (because i don't like the empty
window) there is another window poping up whenever i call popen. (or use
backquotes to start another process)

Using just ruby.exe all seems fine (except the initial nasty empty
window)

Someone knows a way around?

cheers

Simon

Robert · 8 September 2005 11:16

Kroeger Simon (ext) wrote:

Hi all,

i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000.

When starting my GUI app with rubyw (because i don't like the empty
window) there is another window poping up whenever i call popen. (or
use backquotes to start another process)

Using just ruby.exe all seems fine (except the initial nasty empty
window)

Someone knows a way around?

Untested: maybe you can change COMSPEC env var in your ruby script to
point to another binary that won't open a window before executing the new
process. Another option might be to add a command line param to COMSPEC -
although I don't know whether that would work.

Kind regards

robert

Steve_Tuckner · 8 September 2005 15:32

Kroeger Simon (ext) wrote:

Hi all,

i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000.

When starting my GUI app with rubyw (because i don't like the empty
window) there is another window poping up whenever i call popen. (or use
backquotes to start another process)

Using just ruby.exe all seems fine (except the initial nasty empty
window)

Someone knows a way around?

cheers

Simon

Below was my attempt to say, here is how you should fix this issue. And then it didn't work. If I run it with ruby it works fine. When I run it with rubyw, it just goes away when I click run. Any ideas anyone? The system code and `` code is taken from the list a long time ago.

Steve Tuckner

···

---------------------------------------------
require "vr/vruby"
require "vr/vrcontrol"
require 'Win32API'

def system(command)
Win32API.new("crtdll", "system", ['P'], 'L').Call(command)
end

def `(command)
  popen = Win32API.new("crtdll", "_popen", ['P','P'], 'L')
  pclose = Win32API.new("crtdll", "_pclose", ['L'], 'L')
  fread = Win32API.new("crtdll", "fread", ['P','L','L','L'], 'L')
  feof = Win32API.new("crtdll", "feof", ['L'], 'L')
  saved_stdout = $stdout.clone
  psBuffer = " " * 128
  rBuffer = ""
  f = popen.Call(command,"r")
  while feof.Call( f )==0
      l = fread.Call( psBuffer,1,128,f )
      rBuffer += psBuffer[0..l]
  end
  pclose.Call f
  $stdout.reopen(saved_stdout)
  rBuffer
end

class MyForm < VRForm

    def construct
        addControl VRButton, "run", "run", 0, 0, 150, 40
        move 0, 0, 160, 73
    end

    def run_clicked
        messageBox(`dir`)
    end
end

VRLocalScreen.showForm MyForm
VRLocalScreen.messageloop

Christopher_Aldridge · 9 September 2005 01:09

I second this and have battled it numerous times. Finally, I've just given
up on it and execute the job with ruby.exe, and let a dos prompt sit there
ignorantly.

I've been sitting here hopelessly PRAYING that a fix for this will be
provided with ruby2.

*waits impatiently.

···

On 9/8/05, stevetuckner <stevetuckner@usfamily.net> wrote:

Kroeger Simon (ext) wrote:

>Hi all,
>
>i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000.
>
>When starting my GUI app with rubyw (because i don't like the empty
>window) there is another window poping up whenever i call popen. (or use
>backquotes to start another process)
>
>Using just ruby.exe all seems fine (except the initial nasty empty
>window)
>
>Someone knows a way around?
>
>cheers
>
>Simon
>
>
>
>
>
Below was my attempt to say, here is how you should fix this issue. And
then it didn't work. If I run it with ruby it works fine. When I run it
with rubyw, it just goes away when I click run. Any ideas anyone? The
system code and `` code is taken from the list a long time ago.

Steve Tuckner

---------------------------------------------
require "vr/vruby"
require "vr/vrcontrol"
require 'Win32API'

def system(command)
Win32API.new("crtdll", "system", ['P'], 'L').Call(command)
end

def `(command)
popen = Win32API.new("crtdll", "_popen", ['P','P'], 'L')
pclose = Win32API.new("crtdll", "_pclose", ['L'], 'L')
fread = Win32API.new("crtdll", "fread", ['P','L','L','L'], 'L')
feof = Win32API.new("crtdll", "feof", ['L'], 'L')
saved_stdout = $stdout.clone
psBuffer = " " * 128
rBuffer = ""
f = popen.Call(command,"r")
while feof.Call( f )==0
l = fread.Call( psBuffer,1,128,f )
rBuffer += psBuffer[0..l]
end
pclose.Call f
$stdout.reopen(saved_stdout)
rBuffer
end

class MyForm < VRForm

def construct
addControl VRButton, "run", "run", 0, 0, 150, 40
move 0, 0, 160, 73
end

def run_clicked
messageBox(`dir`)
end
end

VRLocalScreen.showForm MyForm
VRLocalScreen.messageloop

Simon_Kroger · 9 September 2005 20:18

stevetuckner wrote:
> [...]

Below was my attempt to say, here is how you should fix this issue. And then it didn't work. If I run it with ruby it works fine. When I run it with rubyw, it just goes away when I click run. Any ideas anyone? The system code and `` code is taken from the list a long time ago.

Steve Tuckner
[...cool code...]

From the msdn library:

Note The _popen function returns an invalid file handle, if used in a Windows program, that will cause the program to hang indefinitely. _popen works properly in a Console application. To create a Windows application that redirects input and output, read the section "Creating a Child Process with Redirected Input and Output" in the Win32 SDK.

hmmm ... i will read through the mentioned section ....
(it's not 100% clear to me if this means "don't start a windows program with popen" or "don't start any program from a windows program" - it would make kind of sense if redirecting stdin from a program without stdin would fail)

btw: on my computers (XP-Home) nothing 'goes away' but the cmd.exe window flickers as if ruby's popen would be used - strange.

From the msdn again:

..and asynchronously executes a spawned copy of the command processor..

this is the main problem i guess.

cheers

Simon

Christopher_Aldridge · 9 September 2005 01:53

I just found a fix that works for me.. I hope this helps someone else out...

Nonetheless, it does require a small vb script, which acts as a
parent for the child script.. to execute, the command is as follows:

[code]
WScript RunHide.vbs Cmd /C MyBatch.cmd
[/code]

The internal source to RunHide.vbs is as follows:

···

##############################################################################
[code]
' ___________________________________________________________________
'
' VBScript File: RunHide.vbs
' Author: Frank-Peter Schultze
'
' Updates: http://www.fpschultze.de/w5.htm
' Enhancement Req.
' and Bug Reports: support@fpschultze.de
'
' Built/Tested On: Windows 2000 SP2
' Requirements: OS: Windows NT4+
' WSH: 1.0+
'
' Purpose: Create a new process that executes in a hidden
' window.
'
' Syntax: RunHide Command [Arg1 ["Arg 2" [...
'
' State Changes:
'
' Assumptions And
' Limitations:
'
' Last Update: 2004-01-29
' ___________________________________________________________________
'

Option Explicit

On Error Resume Next

Dim oSh, sCmd, i

If Not IsWscript() Then
WScript.Echo "Please run this script using WSCRIPT."
WScript.Quit(1)
End If

If ParseCmdLine(sCmd) Then
Set oSh = CreateObject("WScript.Shell")
oSh.Run Trim(sCmd), 0, False
End If

WScript.Quit(0)

Private Function ParseCmdLine(sCmd)

Dim i

ParseCmdLine = False

  If WScript.Arguments.Count > 0 Then
    Select Case WScript.Arguments(0)
      Case "/?" : ShowUsage(True)
      Case "-?" : ShowUsage(True)
      Case "/h" : ShowUsage(True)
      Case "-h" : ShowUsage(True)
      Case "h" : ShowUsage(True)
      Case Else : For i = 0 to WScript.Arguments.Count -1
                    If InStr(WScript.Arguments(i), " ") Then
                      sCmd = sCmd & " " & Chr(34) &
WScript.Arguments(i) & Chr(34)
                    Else
                      sCmd = sCmd & " " & WScript.Arguments(i)
                    End If
                  Next
                  ParseCmdLine = True
    End Select
  Else
    ShowUsage(True)
  End If

ParseCmdLine = True

End Function

Private Function IsWscript()

IsWscript = False

  If InStrRev(LCase(WScript.FullName), "wscript.exe", -1) Then
    IsWscript = True
  End If

End Function

Private Sub ShowUsage(bExit)

Dim strHelp

  strHelp = "Creates a new process that executes in a hidden window." & _
            vbCrLf & vbCrLf & _
            "RunHide Command [Arg1 [" & Chr(34) & "Arg 2" & Chr(34) & " [..."
  MsgBox strHelp, 64, "RunHide Syntax"

  If bExit Then
    WScript.Quit(1)
  End If

End Sub

[/code]
##############################################################################

Here's the quick test case I wrote (3 files):
c:\temp\dosprompt.rb
c:\temp\RunHide.vbs
c:\temp\test.bat

[dosprompt.rb code]
puts "Hi"
IO.popen("cmd /c c:/temp/test.bat")
sleep 2
puts "bye"
[/dosprompt.rb code]

[test.bat code]
echo hello > c:\asdfxxx.txt
[/test.bat code]

[RunHide.vbs code was previously mentioned above/]

To validate that it did in fact run, just run this from a command
prompt or from start > run
"WScript c:\temp\RunHide.vbs Cmd /C "C:\temp\dosprompt.rb""

To validate it ran ok...
c:\asdfxxx.txt should exist and contain "hello"

This seems to easy so.. perhaps I missed something.. if not, I'll be
happy about this tomorrow morning. Let me know if this works out for
you guys..

Credit to RunHide goes to this guy:
http://www.fpschultze.de/w5.htm

thx

On 9/8/05, x1 <caldridge@gmail.com> wrote:

I second this and have battled it numerous times. Finally, I've just given
up on it and execute the job with ruby.exe, and let a dos prompt sit there
ignorantly.

I've been sitting here hopelessly PRAYING that a fix for this will be
provided with ruby2.

*waits impatiently.

On 9/8/05, stevetuckner <stevetuckner@usfamily.net> wrote:
>
> Kroeger Simon (ext) wrote:
>
> >Hi all,
> >
> >i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000.
> >
> >When starting my GUI app with rubyw (because i don't like the empty
> >window) there is another window poping up whenever i call popen. (or use
> >backquotes to start another process)
> >
> >Using just ruby.exe all seems fine (except the initial nasty empty
> >window)
> >
> >Someone knows a way around?
> >
> >cheers
> >
> >Simon
> >
> >
> >
> >
> >
> Below was my attempt to say, here is how you should fix this issue. And
> then it didn't work. If I run it with ruby it works fine. When I run it
> with rubyw, it just goes away when I click run. Any ideas anyone? The
> system code and `` code is taken from the list a long time ago.
>
> Steve Tuckner
>
> ---------------------------------------------
> require "vr/vruby"
> require "vr/vrcontrol"
> require 'Win32API'
>
> def system(command)
> Win32API.new("crtdll", "system", ['P'], 'L').Call(command)
> end
>
> def `(command)
> popen = Win32API.new("crtdll", "_popen", ['P','P'], 'L')
> pclose = Win32API.new("crtdll", "_pclose", ['L'], 'L')
> fread = Win32API.new("crtdll", "fread", ['P','L','L','L'], 'L')
> feof = Win32API.new("crtdll", "feof", ['L'], 'L')
> saved_stdout = $stdout.clone
> psBuffer = " " * 128
> rBuffer = ""
> f = popen.Call(command,"r")
> while feof.Call( f )==0
> l = fread.Call( psBuffer,1,128,f )
> rBuffer += psBuffer[0..l]
> end
> pclose.Call f
> $stdout.reopen(saved_stdout)
> rBuffer
> end
>
> class MyForm < VRForm
>
> def construct
> addControl VRButton, "run", "run", 0, 0, 150, 40
> move 0, 0, 160, 73
> end
>
> def run_clicked
> messageBox(`dir`)
> end
> end
>
> VRLocalScreen.showForm MyForm
> VRLocalScreen.messageloop
>
>
>
>

Simon_Kroger · 10 September 2005 22:08

Simon Kröger wrote:

[...]
hmmm ... i will read through the mentioned section ....

I did.

Ok, here is the 'solution': one has to use CreateProcess to spawn the child. Unfortunately this is a bit more complicated but thanks to Win32API its all achievable in pure ruby.

The stuff in $0 == __FILE__ builds a simple fox gui and starts cmd.exe if a button is pressed. It uses a pipe to write to the stdin of the child ('dir\n') and reads back the childs stdout via another pipe and dumps it to a text control. (The stderror is also mapped but not used)

It's still some work to make it look as easy as popen, but definitaly possible.

Here is the code:
(some lines are copied from this list and the net)

···

--------------------------------------------------------------------
require 'Win32API'

NORMAL_PRIORITY_CLASS = 0x00000020
STARTUP_INFO_SIZE = 68
PROCESS_INFO_SIZE = 16
SECURITY_ATTRIBUTES_SIZE = 12

ERROR_SUCCESS = 0x00
FORMAT_MESSAGE_FROM_SYSTEM = 0x1000
FORMAT_MESSAGE_ARGUMENT_ARRAY = 0x2000

HANDLE_FLAG_INHERIT = 1
HANDLE_FLAG_PROTECT_FROM_CLOSE =2

STARTF_USESHOWWINDOW = 0x00000001
STARTF_USESTDHANDLES = 0x00000100

def raise_last_win_32_error
   errorCode = Win32API.new("kernel32", "GetLastError", , 'L').call
   if errorCode != ERROR_SUCCESS
     params = [
       'L', # IN DWORD dwFlags,
       'P', # IN LPCVOID lpSource,
       'L', # IN DWORD dwMessageId,
       'L', # IN DWORD dwLanguageId,
       'P', # OUT LPSTR lpBuffer,
       'L', # IN DWORD nSize,
       'P', # IN va_list *Arguments
     ]

     formatMessage = Win32API.new("kernel32", "FormatMessage", params, 'L')
     msg = ' ' * 255
     msgLength = formatMessage.call(FORMAT_MESSAGE_FROM_SYSTEM +
       FORMAT_MESSAGE_ARGUMENT_ARRAY, '', errorCode, 0, msg, 255, '')

     msg.gsub!(/\000/, '')
     msg.strip!
     raise msg
   else
     raise 'GetLastError returned ERROR_SUCCESS'
   end
end

def create_pipe # returns read and write handle
   params = [
     'P', # pointer to read handle
     'P', # pointer to write handle
     'P', # pointer to security attributes
     'L'] # pipe size

createPipe = Win32API.new("kernel32", "CreatePipe", params, 'I')

read_handle, write_handle = [0].pack('I'), [0].pack('I')
sec_attrs = [SECURITY_ATTRIBUTES_SIZE, 0, 1].pack('III')

raise_last_win_32_error if createPipe.Call(read_handle,
write_handle, sec_attrs, 0).zero?

[read_handle.unpack('I')[0], write_handle.unpack('I')[0]]
end

def set_handle_information(handle, flags, value)
   params = [
     'L', # handle to an object
     'L', # specifies flags to change
     'L'] # specifies new values for flags

   setHandleInformation = Win32API.new("kernel32",
     "SetHandleInformation", params, 'I')
   raise_last_win_32_error if setHandleInformation.Call(handle,
     flags, value).zero?
   nil
end

def close_handle(handle)
closeHandle = Win32API.new("kernel32", "CloseHandle", ['L'], 'I')
raise_last_win_32_error if closeHandle.call(handle).zero?
end

def create_process(command, stdin, stdout, stderror)
   params = [
     'L', # IN LPCSTR lpApplicationName
     'P', # IN LPSTR lpCommandLine
     'L', # IN LPSECURITY_ATTRIBUTES lpProcessAttributes
     'L', # IN LPSECURITY_ATTRIBUTES lpThreadAttributes
     'L', # IN BOOL bInheritHandles
     'L', # IN DWORD dwCreationFlags
     'L', # IN LPVOID lpEnvironment
     'L', # IN LPCSTR lpCurrentDirectory
     'P', # IN LPSTARTUPINFOA lpStartupInfo
     'P'] # OUT LPPROCESS_INFORMATION lpProcessInformation

   startupInfo = [STARTUP_INFO_SIZE, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
     STARTF_USESTDHANDLES | STARTF_USESHOWWINDOW, 0,
     0, 0, stdin, stdout, stderror].pack('IIIIIIIIIIIISSIIII')

processInfo = [0, 0, 0, 0].pack('IIII')
command << 0

   createProcess = Win32API.new("kernel32", "CreateProcess", params, 'I')
   raise_last_win_32_error if createProcess.call(0,
     command, 0, 0, 1, 0, 0, 0, startupInfo, processInfo).zero?

hProcess, hThread, dwProcessId, dwThreadId = processInfo.unpack('LLLL')

close_handle(hProcess)
close_handle(hThread)

[dwProcessId, dwThreadId]
end

def write_file(hFile, buffer)
   params = [
     'L', # handle to file to write to
     'P', # pointer to data to write to file
     'L', # number of bytes to write
     'P', # pointer to number of bytes written
     'L'] # pointer to structure for overlapped I/O

written = [0].pack('I')
writeFile = Win32API.new("kernel32", "WriteFile", params, 'I')

raise_last_win_32_error if writeFile.call(hFile, buffer, buffer.size,
written, 0).zero?

written.unpack('I')
end

def read_file(hFile)
   params = [
     'L', # handle of file to read
     'P', # pointer to buffer that receives data
     'L', # number of bytes to read
     'P', # pointer to number of bytes read
     'L'] #pointer to structure for data

number = [0].pack('I')
buffer = ' ' * 255

readFile = Win32API.new("kernel32", "ReadFile", params, 'I')

return '' if readFile.call(hFile, buffer, 255, number, 0).zero?

buffer[0...number.unpack('I')[0]]
end

if $0 == __FILE__
require 'fox12'

include Fox

application = FXApp.new("popen", "popen")

main = FXMainWindow.new(application, "popen", nil, nil, DECOR_ALL,
0, 0, 500, 500, 10, 10, 10, 10, 10, 10)

   button = FXButton.new(main, "&Do it!", nil, nil, 0,
     FRAME_THICK|FRAME_RAISED|LAYOUT_FILL_X|LAYOUT_TOP|LAYOUT_LEFT,
     0, 0, 0, 0, 10, 10, 5, 5)

   frame = FXHorizontalFrame.new(main,
     FRAME_THICK|FRAME_SUNKEN|LAYOUT_FILL_X|LAYOUT_FILL_Y,
     0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

edit = FXText.new(frame, nil, 0,
LAYOUT_FILL_X|LAYOUT_FILL_Y|TEXT_READONLY, 0, 0, 0, 0)

edit.textColor, edit.backColor = 0xFFFFFF, 0

   button.connect(SEL_COMMAND) do
     cmd = 'cmd.exe'
     input = "dir\nexit\n"

     # create 3 pipes
     child_in_r, child_in_w = create_pipe
     child_out_r, child_out_w = create_pipe
     child_error_r, child_error_w = create_pipe

     # Ensure the write handle to the pipe for STDIN is not inherited.
     set_handle_information(child_in_w, HANDLE_FLAG_INHERIT, 0)
     set_handle_information(child_out_r, HANDLE_FLAG_INHERIT, 0)
     set_handle_information(child_error_r, HANDLE_FLAG_INHERIT, 0)

processId, threadId = create_process(cmd, child_in_r,
child_out_w, child_error_w)

     # we have to close the handles, so the pipes terminate with the process
     close_handle(child_out_w)
     close_handle(child_error_w)

write_file(child_in_w, input)

     while !(buffer = read_file(child_out_r)).empty?
       edit.appendText(buffer.gsub("\r", ''))
     end
   end

   application.create()
   main.show(PLACEMENT_SCREEN)
   application.run()
end
----------------------------------------------------------------------

i would love to see this taken further, eventually in the popen implementation?

cheers

Simon

Simon_Kroger · 10 September 2005 22:13

x1 wrote:

I just found a fix that works for me.. I hope this helps someone else out...
  Nonetheless, it does require a small vb script, which acts as a
parent for the child script.. to execute, the command is as follows:
  [code]
  WScript RunHide.vbs Cmd /C MyBatch.cmd
[/code]

Hmm, this might be a workaround but not realy a fix. At least not for my problem. I don't want to distribute .vbs and .bat files.

Perhaps my other post offers a solution for you, too?

It's not very polished yet, but it does work (at least on my machine :))

cheers

Simon

Christopher_Aldridge · 11 September 2005 02:54

Simon, this worked on my machine also --excellent!

---Question / Request:
I tried stripping out the GUI stuff so that I could implement this
'solution' into my current needs (to no avail)

I would like to use this as a replacement to popen(), could you help
me strip this down into popen() form?

After the read_file method, I tried keeping what appeared to do the
actual work but.. no luck.. any suggestions?
#------------------------------------------------------------------------------------------------->
    cmd = 'cmd.exe'
    input = "c:\\temp\\test.bat\nexit\n"
    # create 3 pipes
    child_in_r, child_in_w = create_pipe
    child_out_r, child_out_w = create_pipe
    child_error_r, child_error_w = create_pipe
    # Ensure the write handle to the pipe for STDIN is not inherited.
    set_handle_information(child_in_w, HANDLE_FLAG_INHERIT, 0)
    set_handle_information(child_out_r, HANDLE_FLAG_INHERIT, 0)
    set_handle_information(child_error_r, HANDLE_FLAG_INHERIT, 0)

processId, threadId = create_process(cmd, child_in_r,
child_out_w, child_error_w)

# we have to close the handles, so the pipes terminate with the process
close_handle(child_out_w)
close_handle(child_error_w)

write_file(child_in_w, input)
#------------------------------------------------------------------------------------------------->
# while !(buffer = read_file(child_out_r)).empty?
# edit.appendText(buffer.gsub("\r", ''))
# end

thank you very much!

···

On 9/10/05, Simon Kröger <SimonKroeger@gmx.de> wrote:

x1 wrote:

> I just found a fix that works for me.. I hope this helps someone else out...
>
> Nonetheless, it does require a small vb script, which acts as a
> parent for the child script.. to execute, the command is as follows:
>
> [code]
> WScript RunHide.vbs Cmd /C MyBatch.cmd
> [/code]

Hmm, this might be a workaround but not realy a fix. At least not for my
problem. I don't want to distribute .vbs and .bat files.

Perhaps my other post offers a solution for you, too?

It's not very polished yet, but it does work (at least on my machine :))

cheers

Simon

Simon_Kroger · 11 September 2005 20:31

x1 wrote:

Simon, this worked on my machine also --excellent!

---Question / Request:
I tried stripping out the GUI stuff so that I could implement this
'solution' into my current needs (to no avail)

I would like to use this as a replacement to popen(), could you help
me strip this down into popen() form?

After the read_file method, I tried keeping what appeared to do the
actual work but.. no luck.. any suggestions?

ok, here we go:

···

---------------------------------------
io = popen('ver')
puts io.read_all
---------------------------------------
better?

or:
---------------------------------------
io = popen('cmd')
io.write("ping localhost\nexit\n")

   while !(buffer = io.read).empty?
     print buffer
   end
---------------------------------------
?

note that read blocks if there is no data to read, but this is done in a ruby-thread friendly way. So you may put that in a new Thread and your gui is still responsive (i tried that with the fox gui).
(if you let it block on read_file the whole process blocks)

The same is true for read_all and you can have multiple childs at once.

here is the complete code:

-------------------------------------------------------------------------
require 'Win32API'

$stdout.sync = true