Question about sockets/listeners

ruby-talk
1

hi, i'm very new to ruby and so i'd be very grateful of any help on
this. I am writing a script that listens for UDP packets on a
particular port. This is very easy using 'socket'. I then want to make
a script that simulates lots of clients sending UDP packets to the
listener. These clients should send a UDP packet from a particular port
and then listen for a response on that same port. For this reason, I am
trying to run each simulated client in a different thread. A simplified
version of what i have done so far is shown below, however, when I run
this, the client stops sending packets after the 66th one. Can someone
explain this to me please? why is it stopping at 66? which limit am i
hitting?

Also, any suggestions on how to improve the code will be gratefully
received. Thanks

#### Listener.rb ####
require "socket"

socket = UDPSocket.open
socket.bind("",1500)

while true
  message, details = socket.recvfrom(512)
  p message
end

#### Clients.rb ####
require 'socket'

SERVER_IP = "127.0.0.1"
SERVER_PORT = 1500

packets=[]
threads =[]
# Build an array of packets
for i in 1..100
  packets << "packet#{i}"
end

# For Packet to be sent, start a new thread
for packets_to_send in packets
  sleep 0.1
  threads << Thread.new(packets_to_send) do |packet|

  #Generate a random port
  port = rand(60000)

  # Open a socket on that port
  socket = UDPSocket.open
    socket.bind("",port)

  # Send the packet
  puts "Sending #{packet}"
  socket.send(packet, 0, SERVER_IP, SERVER_PORT)

  while true
    message, message_details = socket.recvfrom(512)
  end

  end
end

···

--
Posted via http://www.ruby-forum.com/.

2

My first guess is that you are running out of file descriptors. It
looks like each thread creates a new socket, calls send, then calls
recvfrom repeatedly, never closing the socket and freeing the file
descriptor.

You should close the socket after receiving a response. Alternatively
you could see how to raise the per/process file descriptor limit for
your system (for bash, try looking at the ulimit command--the exact
command and syntax depends on the shell you are using).

Gary Wright

···

On Dec 15, 2007, at 9:38 AM, James Croft wrote:

A simplified
version of what i have done so far is shown below, however, when I run
this, the client stops sending packets after the 66th one. Can someone
explain this to me please? why is it stopping at 66? which limit am i
hitting?

3

You main thread may be existing before all the
threads have been started.

After creating all the threads you need to wait
for them to finish by calling t.join on each one.

Gary Wright

···

On Dec 15, 2007, at 9:38 AM, James Croft wrote:

why is it stopping at 66? which limit am i
hitting?

4

Using a nonthreaded approach, I got up to about 360 packets through on a
single run. (This program requires the Ruby/EventMachine library):

···

On Dec 15, 2007 9:38 AM, James Croft <crofty_james@hotmail.com> wrote:

hi, i'm very new to ruby and so i'd be very grateful of any help on
this. I am writing a script that listens for UDP packets on a
particular port. This is very easy using 'socket'. I then want to make
a script that simulates lots of clients sending UDP packets to the
listener. These clients should send a UDP packet from a particular port
and then listen for a response on that same port. For this reason, I am
trying to run each simulated client in a different thread. A simplified
version of what i have done so far is shown below, however, when I run
this, the client stops sending packets after the 66th one. Can someone
explain this to me please? why is it stopping at 66? which limit am i
hitting?

Also, any suggestions on how to improve the code will be gratefully
received. Thanks

#-----------------------------------------
require 'rubygems'
require 'eventmachine'

EM.epoll

SERVER = "127.0.0.1"
PORT = 1500

module Server
        def receive_data data
                send_data "I saw your #{data}"
        end
end

module Client
        def post_init
                @@counter ||= 0
                @@counter += 1
                send_datagram "Data packet #{@@counter}", SERVER, PORT
        end
        def receive_data data
                @@received ||= 0
                @@received += 1
                p "Received #{@@received}: #{data}"
        end
end

EM.run {
        EM.open_datagram_socket SERVER, PORT, Server
        (60000..60400).each {|n|
                EM.open_datagram_socket SERVER, n, Client
        }
}

#-------------------------------

I did two things differently: first, I didn't introduce a delay between each
new socket. If I had, then this program would probably have scaled much
farther. And second, I used sequential port numbers rather than random ones,
to avoid trying to open an already-open port, and to avoid trying to open a
privileged one.

5

Ok, I tweaked my test program somewhat:

···

On Dec 15, 2007 9:38 AM, James Croft <crofty_james@hotmail.com> wrote:

hi, i'm very new to ruby and so i'd be very grateful of any help on
this. I am writing a script that listens for UDP packets on a
particular port. This is very easy using 'socket'. I then want to make
a script that simulates lots of clients sending UDP packets to the
listener. These clients should send a UDP packet from a particular port
and then listen for a response on that same port. For this reason, I am
trying to run each simulated client in a different thread. A simplified
version of what i have done so far is shown below, however, when I run
this, the client stops sending packets after the 66th one. Can someone
explain this to me please? why is it stopping at 66? which limit am i
hitting?

#---------------------------------------
require 'rubygems'
require 'eventmachine'

EM.epoll

SERVER = "127.0.0.1"
PORT = 1500

module Server
        def receive_data data
                send_data "I saw your #{data}"
        end
end

module Client
        def post_init
                @@counter ||= 0
                @@counter += 1
                send_datagram "Data packet #{@@counter}", SERVER, PORT
        end
        def receive_data data
                @@received ||= 0
                @@received += 1
                p "Received #{@@received}: #{data}"
                close_connection
        end
end

EM.run {
        EM.open_datagram_socket SERVER, PORT, Server

        t = EM::PeriodicTimer.new(0.05) {
                n = rand(1000) + 60000
                EM.open_datagram_socket SERVER, n, Client
        }
}

#-----------------------------

Now I'm delaying before opening each client as you were, and I'm using
random ports chosen in a high range. I'm also closing client sockets after
they receive their responses. This version is now up to 10,000 packets and
it's still running strong.

Also, on a Linux kernel, I took out the close_connection statement and used
EPOLL to eliminate Ruby's limit of 1024 descriptors per process. I had to
add a mechanism to keep the random port number generator from giving the
number of an already-open port. That program is also still running at well
over 10,000 open sockets and packets now.

6

Gary Wright wrote:

You should close the socket after receiving a response.

Sorry, I should have said, I am expecting mulitple responses from the
server and hence I want to keep the socket open.

Gary Wright wrote:

You main thread may be existing before all the threads have been started.

The threads shouldn't be exiting because I have the 'while true' in
there. I have just tried joining the threads and it didn't fix it. It
might be the file descriptor thing, i'll have to have a read up about
it. Thanks

···

--
Posted via http://www.ruby-forum.com/\.