# \[ANN\] nbfifo-0.0.0 - non blocking fifos for threads **URL:** \ **Category:** ruby-talk\ **Created:** [23 September 2005 22:57 UTC](https://rubytalk.org/t/ann-nbfifo-0-0-0-non-blocking-fifos-for-threads/21043 "2005-09-23T22:57:06Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![Ara.T.Howard6](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@Ara.T.Howard6](https://rubytalk.org/u/Ara.T.Howard6)\ **Post date:** [23 September 2005 22:57 UTC](https://rubytalk.org/t/ann-nbfifo-0-0-0-non-blocking-fifos-for-threads/21043/1 "2005-09-23T22:57:06Z")
URIS    http::/codeforpeople.com/lib/ruby/nbfifo NAME    nbfifo.rb SYSNOPSIS    thread and process safe non-blocking fifos for ruby.    when using fifos from ruby threads it is extremely easy to block the entire    process via some operation done on a fifo such as opening it or    reading/writing to/from it. rbfifo.rb is a proof-of-concept class showing    how fifos can be used from a ruby multi-threaded process in a way that    ensures no threaded operation will block the entire process. as an added    benefit the nbfifo class sets up fifos in such a way that they are durable    across process invocations so that a server may leave a connection open    while clients come and go and clients do not even block waiting for a server    to be on the other end of a fifo opened for writing. another feature is    that an eof signal can be sent down the fifo to signal end-of-file to    clients, who will have EOFError raised in there code, however they can    catch this error and simply reset the fifo to begin reading again. the    relationship of readers and writers is such that they may even exist in    the same process without deadlocking. CONCEPT    opening a fifo in 'r+' mode do not block as opening them with only 'r' or    'w' does. nbfifo opens all fifos in this way to avoid blocking a process on    an open. the reading and writing protocol 'blocks' in the sense that the    application does not need to handle EAGAIN or some such but not in a way    that blocks an entire process - the blocking is thread local. therefore all    one need do to accomplish non-blocking io is operate in ruby threads (which    are normally quite prone to a process-blocking operation killing all    threads).    the writing is done as follows:      - never write more than PIPE\_MAX bytes at a time to ensure atomicity. you        don't worry about this - data is chunked by the api for you.      - each msg is preceded by a byte indicating it's length. the length plus        the msg is less that PIPE\_MAX when combined.      - select is called before writing      - when select returns the msg is sent    the reading is done as follows:      - select is called to avoid reading 'no-data'. when pipes are opened in        'r+' mode a blind read with no data waiting will return the emtpy string        ("") and the application would end up busy-looping on this value unless        select is used.      - when select returns the number of bytes is read. then the message is        read. EXAMPLES > **ยทยทยท** > > ------------------------------------------ >    here's the producer process: >    ------------------------------------------ > >      jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 \> cat producer.rb >      require 'nbfifo' >      STDOUT.sync = true >      $VERBOSE = nil > >      # >      # die nicely on getchar >      # >        trap('INT'){ exit } >        Thread::new(){ Thread::main.exit if select [STDIN]} > >      # >      # this guy will stay running >      # >        Thread::new{ loop{ puts "random producer thread stays running..."; sleep 1 }} > >      # >      # loop doing an open, which does not block, and the sending two virtual >      # doccuments down the pipe but following the first with eof >      # >        loop do >          Thread::new do >            nbfifo = NBFifo::new 'fifo' >            2.times{ nbfifo.send Time::now; sleep 1 } >            nbfifo.send\_eof >            2.times{ nbfifo.send Time::now; sleep 1 } >          end.join >        end > >    ------------------------------------------ >    here's the concumer process >    ------------------------------------------ > >      jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 \> cat consumer.rb >      require 'nbfifo' >      STDOUT.sync = true >      $VERBOSE = nil > >      # >      # die nicely on getchar >      # >        trap('INT'){ exit } >        Thread::new(){ Thread::main.exit if select [STDIN]} > >      # >      # this guy will stay running >      # >        Thread::new{ loop{ puts "random consumer thread stays running..."; sleep 1 }} > >      # >      # just loop opening fifo, which does not block, and reading whatever data >      # can be read until end-of-file is hit. when it is reset and go again >      # >        loop do >          Thread::new do >            nbfifo = NBFifo::new 'fifo' >            begin >              2.times{ puts nbfifo.recv } >            rescue EOFError >              nbfifo.reset >              retry >            end >          end.join >        end > >    ------------------------------------------ >    running the consumer >    ------------------------------------------ > >      jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 \> ruby consumer.rb >      random consumer thread stays running... >      Fri Sep 23 15:35:31 MDT 2005 >      Fri Sep 23 15:35:32 MDT 2005 >      random consumer thread stays running... >      Fri Sep 23 15:35:35 MDT 2005 > >    ------------------------------------------ >    running the producer >    ------------------------------------------ > >      jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 \> sleep 3; ruby producer.rb >      random producer thread stays running... > >    ------------------------------------------ >    running the consumer after killing it >    ------------------------------------------ > >      jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 \> ruby consumer.rb >      random consumer thread stays running... >      Fri Sep 23 15:43:15 MDT 2005 >      Fri Sep 23 15:43:17 MDT 2005 >      random consumer thread stays running... >      Fri Sep 23 15:43:26 MDT 2005 > >    note that killing the consumer and restarting did not affect the producer at >    all. this is quite different than the way normal fifo operation is - in that >    case the reader must always open the fifo after a writer is on the other end >    and a writer - even if opening using O\_NONBLOCK will explode if no reader is >    on the other end. using this method neither open (on either end) will block >    but subsequent reads/writes will, but in a way that blocks only thread and not >    the entire process. in fact, you can easily open up a fifo for reading and >    writing in the same process, regardless of the order they are opened, and >    easily write well behaved threads that use this channel to communicate: > >    ------------------------------------------ >    running a producer and consumer at once >    ------------------------------------------ > >      jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 \> cat producer\_and\_consumer.rb >      require 'nbfifo' >      STDOUT.sync = true >      $VERBOSE = nil > >      trap('INT'){ exit } >      Thread::new(){ Thread::main.exit if select [STDIN]} > >      producer = >        Thread::new do >          nbfifo = NBFifo::new 'fifo' >          loop{ nbfifo.send Time::now; sleep 1 } >        end > >      consumer = >        Thread::new do >          nbfifo = NBFifo::new 'fifo' >          loop{ puts nbfifo.recv } >        end > >      sleep > >      jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 \> ruby producer\_and\_consumer.rb >      Fri Sep 23 15:47:13 MDT 2005 >      Fri Sep 23 15:47:14 MDT 2005 >      Fri Sep 23 15:47:15 MDT 2005 >      Fri Sep 23 15:47:16 MDT 2005 >      Fri Sep 23 15:47:17 MDT 2005 > >    note that this program would deadlock instantly using fifos in the normal way >    ('r', or 'w') and would blow up with Errno::ENXIO if File::NONBLOCK were >    specified to the open and the writer thread attempted the open before the >    reader. the other issue with using File::NONBLOCK for fifos is that reads >    give the empty string ("") whether we're at the end-of-file or if there is >    simply no data and we're forced into a busy loop. > > CAVEATS > >    like all my code these days - this is experimental. you io gurus out there >    will no doubt have comments and, for you, this process may seem easy (that's >    you tanaka) but it was giving me fits so i decided to wrap it up. > >    i've run the code on linux only. i'd be grateful for some bsd and solaris >    testers. also, i played around with gunwin32 utils (for mkfifo) on windows >    and couldn't get it worked - but it was close. please contact me off/online >    if you have helpful info in this area. > > WHY? > >    say you were writing a fastcgi replacement and wanted a multi-threaded pure >    ruby server which could handled multiple incming requests without blocking >    the entire process opening/reading/writing to/from fifos. then you'll >    really need something like this. > > AUTHOR > >    ara.t.howard@noaa.gov > > # -a -- > > > email :: ara [dot] t [dot] howard [at] noaa [dot] gov > > phone :: 303.497.6469 > > Your life dwells amoung the causes of death > > Like a lamp standing in a strong breeze. --Nagarjuna > > ===============================================================================