[ANN] tupelo-0.21

Tupelo is a language-agnostic tuplespace for coordination of distributed programs. It is designed for distribution of both computation and storage, on disk and in memory, with pluggable storage adapters.

Tupelo is inspired by Masatoshi Seki's Rinda in the Ruby standard library, which in turn is based on David Gelernter's Linda.

Installation:

gem install tupelo

Requirements:

ruby 2.0 or 2.1.

Details:

Example:

This program counts prime numbers in an interval by distributing the problem to a set of hosts:

     require 'tupelo/app/remote'

     hosts = %w{itchy scratchy lisa bart} # ssh hosts with key-based auth

     Tupelo.tcp_application do
       hosts.each do |host|
         remote host: host, passive: true, eval: %{
           require 'prime' # ruby stdlib for prime factorization
           loop do
             _, input = take(["input", Integer])
             write ["output", input, input.prime_division]
           end
         }
       end

       local do
         inputs = 1_000_000_000_000 .. 1_000_000_000_200

         inputs.each do |input|
           write ["input", input]
         end

         count = 0
         inputs.size.times do |i|
           _, input, factors = take ["output", Integer, nil]
           count += 1 if factors.size == 1 and factors[0][1] == 1
           print "\rChecked #{i}"
         end

         puts "\nThere are #{count} primes in #{inputs}"
       end
     end

This is pretty exciting! Can I ask what application (if any) inspired it?

···

On Fri, Mar 21, 2014 at 12:10 PM, Joel VanderWerf <joelvanderwerf@gmail.com>wrote:

Tupelo is a language-agnostic tuplespace for coordination of distributed
programs. It is designed for distribution of both computation and storage,
on disk and in memory, with pluggable storage adapters.

Tupelo is inspired by Masatoshi Seki's Rinda in the Ruby standard library,
which in turn is based on David Gelernter's Linda.

Installation:

gem install tupelo

Requirements:

ruby 2.0 or 2.1.

Details:

https://github.com/vjoel/tupelo

Example:

This program counts prime numbers in an interval by distributing the
problem to a set of hosts:

    require 'tupelo/app/remote'

    hosts = %w{itchy scratchy lisa bart} # ssh hosts with key-based auth

    Tupelo.tcp_application do
      hosts.each do |host|
        remote host: host, passive: true, eval: %{
          require 'prime' # ruby stdlib for prime factorization
          loop do
            _, input = take(["input", Integer])
            write ["output", input, input.prime_division]
          end
        }
      end

      local do
        inputs = 1_000_000_000_000 .. 1_000_000_000_200

        inputs.each do |input|
          write ["input", input]
        end

        count = 0
        inputs.size.times do |i|
          _, input, factors = take ["output", Integer, nil]
          count += 1 if factors.size == 1 and factors[0][1] == 1
          print "\rChecked #{i}"
        end

        puts "\nThere are #{count} primes in #{inputs}"
      end
    end

Thanks for asking, Avdi! I wish I had a dramatic "killer" app to answer your question, but....

The story is that I learned a bit of Javaspaces and Jini back around 2001-2, when a coworker was promoting it for service discovery (he was interested in fleets of UAVs, now known as drones, and supervisory aircraft). Happily, there was a Ruby version of Javaspaces, namely Rinda, that was easy to play with.

Since then, I used Rinda here and there as a lightweight way to set up a distributed dataflow (simulation and sensor data analysis stuff), or control access to resources. Last spring I started trying to make Rinda better and even got a few patches accepted. But I realized that Rinda's centralized architecture was always going to be a problem. Hence a rewrite with the minimum of central state and processing.

Would be interested in finding applications for it, though :slight_smile:

···

On 03/21/2014 10:25 AM, Avdi Grimm wrote:

This is pretty exciting! Can I ask what application (if any) inspired it?

On Fri, Mar 21, 2014 at 12:10 PM, Joel VanderWerf > <joelvanderwerf@gmail.com <mailto:joelvanderwerf@gmail.com>> wrote:

    Tupelo is a language-agnostic tuplespace for coordination of
    distributed programs. It is designed for distribution of both
    computation and storage, on disk and in memory, with pluggable
    storage adapters.

...

Since then, I used Rinda here and there as a lightweight way to set up a
distributed dataflow (simulation and sensor data analysis stuff), or
control access to resources. Last spring I started trying to make Rinda
better and even got a few patches accepted. But I realized that Rinda's
centralized architecture was always going to be a problem. Hence a
rewrite with the minimum of central state and processing.

Forgot to mention: the outcome of working on Rinda is encapsulated here:

This contains a version of rinda with some of the bugfix/performance patches that were merged into the Ruby stdlib and some other patches (mostly around optimistic concurrency and simple forms of transactions) that were not. There are also examples and a "tuple shell" to make it easy to start and connect to Rinda instances from the command line.

However, I don't recommend using my-rinda (or rinda for that matter). The ideas in my-rinda have been rolled into tupelo, and tupelo's transactions are more general and efficient.

···

On 03/21/2014 11:01 AM, Joel VanderWerf wrote: