Testing - Network Layer Fault Injection in Ruby

ruby-talk
1

Hi all,

I've yet to find something that fills this hole in my test cases.

Does anyone do this, and if so, what tools do they use?

Thanks,

James.

2

I do some of this with custom stubs in memcache-client, see below. I've also simulated call failures (Errno::whatever) with slightly more powerful stubs that can take a block to raise the exception.

Here's a fake socket:

class FakeSocket

   attr_reader :written, :data

   def initialize
     @written = StringIO.new
     @data = StringIO.new
   end

   def write(data)
     @written.write data
   end

   def gets
     @data.gets
   end

   def read(arg)
     @data.read arg
   end

end

And here's a test using it:

   def test_flush_all_failure
     socket = FakeSocket.new
     socket.data.write "ERROR\r\n"
     socket.data.rewind
     server = FakeServer.new socket
     def server.host() "localhost"; end
     def server.port() 11211; end

     @cache.servers = []
     @cache.servers << server

     assert_raise MemCache::MemCacheError do
       @cache.flush_all
     end

     assert_equal "flush_all\r\n", socket.written.string
   end

···

On Oct 14, 2007, at 13:23 , James Tucker wrote:

I've yet to find something that fills this hole in my test cases.
Does anyone do this, and if so, what tools do they use?

--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars

3

Eric,

Thanks, that's somewhat the route I had started down for some of the units. I probably shouldn't have omitted from my orignal question, the fact that I'm also using OpenSSL.

Some of the specific issues I have been dealing with are caused by a harsh environment (trans-atlantic links and poor ADSL connections), and in some cases this causes irrecoverable blocking. I think I've managed to cover all of the blocking cases, but local tests are hard to implement, as you have to fake things like a full TCP connection timeout. (Which requires not sending and timing out packet data which normally the OS doesn't want you to do).

It's also difficult to find out where the blocks are, as the state renders the app essentially useless, and happens in a different environment than that of the testing / development stage.

As some of this connection state stuff is actually outside of pure ruby, I was somewhat hoping for an external solution, such as a fault injection proxy socket or similar. Do you know of any?

Regards,

James.

Eric Hodel wrote:

···

On Oct 14, 2007, at 13:23 , James Tucker wrote:

I've yet to find something that fills this hole in my test cases.
Does anyone do this, and if so, what tools do they use?

I do some of this with custom stubs in memcache-client, see below. I've also simulated call failures (Errno::whatever) with slightly more powerful stubs that can take a block to raise the exception.

Here's a fake socket:

class FakeSocket

  attr_reader :written, :data

  def initialize
    @written = StringIO.new
    @data = StringIO.new
  end

  def write(data)
    @written.write data
  end

  def gets
    @data.gets
  end

  def read(arg)
    @data.read arg
  end

end

And here's a test using it:

  def test_flush_all_failure
    socket = FakeSocket.new
    socket.data.write "ERROR\r\n"
    socket.data.rewind
    server = FakeServer.new socket
    def server.host() "localhost"; end
    def server.port() 11211; end

    @cache.servers = []
    @cache.servers << server

    assert_raise MemCache::MemCacheError do
      @cache.flush_all
    end

    assert_equal "flush_all\r\n", socket.written.string
  end

--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars

4

I don't I've not needed that level of fault tolerance.

···

On Oct 14, 2007, at 14:52 , James Tucker wrote:

Thanks, that's somewhat the route I had started down for some of the units. I probably shouldn't have omitted from my orignal question, the fact that I'm also using OpenSSL.

Some of the specific issues I have been dealing with are caused by a harsh environment (trans-atlantic links and poor ADSL connections), and in some cases this causes irrecoverable blocking. I think I've managed to cover all of the blocking cases, but local tests are hard to implement, as you have to fake things like a full TCP connection timeout. (Which requires not sending and timing out packet data which normally the OS doesn't want you to do).

It's also difficult to find out where the blocks are, as the state renders the app essentially useless, and happens in a different environment than that of the testing / development stage.

As some of this connection state stuff is actually outside of pure ruby, I was somewhat hoping for an external solution, such as a fault injection proxy socket or similar. Do you know of any?

--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars