# Data queuing

**URL:** <https://rubytalk.org/t/data-queuing/73804>\
**Category:** ruby-talk\
**Created:** [7 September 2016 06:23 UTC](https://rubytalk.org/t/data-queuing/73804 "2016-09-07T06:23:25Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![A\_Berger](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@A\_Berger](https://rubytalk.org/u/A_Berger)\
**Post date:** [7 September 2016 06:23 UTC](https://rubytalk.org/t/data-queuing/73804/1 "2016-09-07T06:23:25Z")

</div>

Hi  
It looks like Class queue disappeared in Ruby 2.1 (last seen in Rubydoc  
2.0.0)

Moreover what is the best solution (simplicity, performance) for a queue  
feeding different Ruby-processes?  
Having 1) fifo and 2) distibutor functionality.  
Data should stay in Ram (e.g. Array), but for the communication a pipe,  
network/socket or shared mem is possible.  
Something fast&proved out there, or do I have to do it by hand?  
Whats commonly used (best already with integrated locking)?

Thank you  
Berg

---

<div class="post-metadata">

**Author:** ![Raj\_Sahae](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/raj_sahae/32/1840_2.png) [@Raj\_Sahae](https://rubytalk.org/u/Raj_Sahae)\
**Post date:** [7 September 2016 06:32 UTC](https://rubytalk.org/t/data-queuing/73804/2 "2016-09-07T06:32:11Z")

</div>

I'm not sure what you mean. I looked at ruby-doc.org and I see the Queue  
class in the core library for 2.1, 2.2, and 2.3.

> **···**
>
> On Tue, Sep 6, 2016, 11:23 PM A Berger \<aberger7890@gmail.com\> wrote:
> 
> > Hi  
> > It looks like Class queue disappeared in Ruby 2.1 (last seen in Rubydoc  
> > 2.0.0)
> > 
> > Moreover what is the best solution (simplicity, performance) for a queue  
> > feeding different Ruby-processes?  
> > Having 1) fifo and 2) distibutor functionality.  
> > Data should stay in Ram (e.g. Array), but for the communication a pipe,  
> > network/socket or shared mem is possible.  
> > Something fast&proved out there, or do I have to do it by hand?  
> > Whats commonly used (best already with integrated locking)?
> > 
> > Thank you  
> > Berg
> > 
> > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> 
> --
> 
> - Raj

---

<div class="post-metadata">

**Author:** ![A\_Berger](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@A\_Berger](https://rubytalk.org/u/A_Berger)\
**Post date:** [7 September 2016 06:41 UTC](https://rubytalk.org/t/data-queuing/73804/3 "2016-09-07T06:41:29Z")

</div>

Hi  
I see it changed from stdlib to core,  
but nothing is found with search engine:

> **[ruby queue - Bing](https://www.bing.com/search?q=ruby%2Bqueue)**
>
> Intelligent search from Bing makes it easier to quickly find what you’re looking for and rewards you.

I want to use all cpu-cores, so processes look better than threads,  
so I cant use this queue.  
Better solution:  
Perhaps someone can change threads to utilize all cores?

Thanks  
Berg

> **···**
>
> Am 07.09.2016 08:32 schrieb "Raj Sahae" \<rajsahae@gmail.com\>:
> 
> > I'm not sure what you mean. I looked at ruby-doc.org and I see the Queue  
> > class in the core library for 2.1, 2.2, and 2.3.
> > 
> > On Tue, Sep 6, 2016, 11:23 PM A Berger \<aberger7890@gmail.com\> wrote:
> > 
> > > Hi  
> > > It looks like Class queue disappeared in Ruby 2.1 (last seen in Rubydoc  
> > > 2.0.0)
> > > 
> > > Moreover what is the best solution (simplicity, performance) for a queue  
> > > feeding different Ruby-processes?  
> > > Having 1) fifo and 2) distibutor functionality.  
> > > Data should stay in Ram (e.g. Array), but for the communication a pipe,  
> > > network/socket or shared mem is possible.  
> > > Something fast&proved out there, or do I have to do it by hand?  
> > > Whats commonly used (best already with integrated locking)?
> > > 
> > > Thank you  
> > > Berg
> > > 
> > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > 
> > --
> > 
> > - Raj
> > 
> > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);

---

<div class="post-metadata">

**Author:** ![Martin\_DeMello](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@Martin\_DeMello](https://rubytalk.org/u/Martin_DeMello)\
**Post date:** [7 September 2016 06:49 UTC](https://rubytalk.org/t/data-queuing/73804/4 "2016-09-07T06:49:12Z")

</div>

jruby sounds like the easiest way forward here

> **[Concurrency in jruby](https://github.com/jruby/jruby/wiki/Concurrency-in-jruby)**
>
> JRuby, an implementation of Ruby on the JVM. Contribute to jruby/jruby development by creating an account on GitHub.

> **···**
>
> On Tue, Sep 6, 2016 at 11:41 PM, A Berger \<aberger7890@gmail.com\> wrote:
> 
> > Hi  
> > I see it changed from stdlib to core,  
> > but nothing is found with search engine: [Bing](https://www.bing.com/search?q=)  
> > ruby+queue
> > 
> > I want to use all cpu-cores, so processes look better than threads,  
> > so I cant use this queue.  
> > Better solution:  
> > Perhaps someone can change threads to utilize all cores?
> > 
> > Thanks  
> > Berg  
> > Am 07.09.2016 08:32 schrieb "Raj Sahae" \<rajsahae@gmail.com\>:
> > 
> > > I'm not sure what you mean. I looked at ruby-doc.org and I see the Queue  
> > > class in the core library for 2.1, 2.2, and 2.3.
> > > 
> > > On Tue, Sep 6, 2016, 11:23 PM A Berger \<aberger7890@gmail.com\> wrote:
> > > 
> > > > Hi  
> > > > It looks like Class queue disappeared in Ruby 2.1 (last seen in Rubydoc  
> > > > 2.0.0)
> > > > 
> > > > Moreover what is the best solution (simplicity, performance) for a queue  
> > > > feeding different Ruby-processes?  
> > > > Having 1) fifo and 2) distibutor functionality.  
> > > > Data should stay in Ram (e.g. Array), but for the communication a pipe,  
> > > > network/socket or shared mem is possible.  
> > > > Something fast&proved out there, or do I have to do it by hand?  
> > > > Whats commonly used (best already with integrated locking)?
> > > > 
> > > > Thank you  
> > > > Berg
> > > > 
> > > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org  
> > > > ?subject=unsubscribe\>  
> > > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > > 
> > > --
> > > 
> > > - Raj
> > > 
> > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > 
> > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);

---

<div class="post-metadata">

**Author:** ![Bar\_Hofesh](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/bar_hofesh/32/2020_2.png) [@Bar\_Hofesh](https://rubytalk.org/u/Bar_Hofesh)\
**Post date:** [7 September 2016 06:47 UTC](https://rubytalk.org/t/data-queuing/73804/5 "2016-09-07T06:47:45Z")

</div>

Just use Rubinuios Or Jruby , they support really system threads without the GIL

[http://www.aqua-mail.com](http://www.aqua-mail.com)

> **···**
>
> Sent with AquaMail for Android
> 
> On September 7, 2016 9:41:43 AM A Berger \<[aberger7890@gmail.com](mailto:aberger7890@gmail.com)\> wrote:
> 
> Hi  
> I see it changed from stdlib to core,  
> but nothing is found with search engine: [https://www.bing.com/search?q=ruby+queue](https://www.bing.com/search?q=ruby+queue)
> 
> I want to use all cpu-cores, so processes look better than threads,  
> so I cant use this queue.  
> Better solution:  
> Perhaps someone can change threads to utilize all cores?
> 
> Thanks  
> Berg
> 
> Am 07.09.2016 08:32 schrieb "Raj Sahae" \<[rajsahae@gmail.com](mailto:rajsahae@gmail.com)\<[mailto:rajsahae@gmail.com](mailto:rajsahae@gmail.com)\>\>:
> 
> I'm not sure what you mean. I looked at ruby-doc.org\<[http://ruby-doc.org](http://ruby-doc.org)\> and I see the Queue class in the core library for 2.1, 2.2, and 2.3.
> 
> On Tue, Sep 6, 2016, 11:23 PM A Berger \<[aberger7890@gmail.com](mailto:aberger7890@gmail.com)\<[mailto:aberger7890@gmail.com](mailto:aberger7890@gmail.com)\>\> wrote:
> 
> Hi  
> It looks like Class queue disappeared in Ruby 2.1 (last seen in Rubydoc 2.0.0)
> 
> Moreover what is the best solution (simplicity, performance) for a queue feeding different Ruby-processes?  
> Having 1) fifo and 2) distibutor functionality.  
> Data should stay in Ram (e.g. Array), but for the communication a pipe, network/socket or shared mem is possible.  
> Something fast&proved out there, or do I have to do it by hand?  
> Whats commonly used (best already with integrated locking)?
> 
> Thank you  
> Berg
> 
> Unsubscribe: \<[mailto:ruby-talk-request@ruby-lang.org](mailto:ruby-talk-request@ruby-lang.org)\<[mailto:ruby-talk-request@ruby-lang.org](mailto:ruby-talk-request@ruby-lang.org)\>?subject=unsubscribe\>  
> \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk)\>  
> --
> 
> - Raj

---

<div class="post-metadata">

**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\
**Post date:** [7 September 2016 07:17 UTC](https://rubytalk.org/t/data-queuing/73804/6 "2016-09-07T07:17:41Z")

</div>

> I see it changed from stdlib to core,  
> but nothing is found with search engine:  
> [ruby queue - Search](https://www.bing.com/search?q=ruby+queue)

$ ruby --version  
ruby 2.2.5p319 (2016-04-26 revision 54774) [x86\_64-cygwin]  
$ ruby -e 'p Queue'  
Thread::Queue

> I want to use all cpu-cores, so processes look better than threads,  
> so I cant use this queue.  
> Better solution:  
> Perhaps someone can change threads to utilize all cores?

This work is under way AFAIK but it takes time. You can search for  
"ruby gil" to find out a bit more. Or use JRuby as Martin suggested.

Cheers

robert

> **···**
>
> On Wed, Sep 7, 2016 at 8:41 AM, A Berger \<aberger7890@gmail.com\> wrote:
> 
> --  
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can  
> - without end}  
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)

---

<div class="post-metadata">

**Author:** ![A\_Berger](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@A\_Berger](https://rubytalk.org/u/A_Berger)\
**Post date:** [7 September 2016 07:32 UTC](https://rubytalk.org/t/data-queuing/73804/7 "2016-09-07T07:32:01Z")

</div>

Ok - I still wanted to avoid using java...  
But thank you, maybe I should cancel that precondition.

Nevertheless are there solutions for "many-processes" - data-sharing out  
there?  
(Thinking a database is overkill, besides for mostly needed as queue its  
not well suited for that job, but possible if at once fetching+deleting  
seen rows)  
Shared mem has locking, but it's not "nice" (to implement/use)...  
Maybe this is the fastest possibility.

Does anybody have experience using network or pipes (or ready-to-use  
queuing-gems with locking/parallel-access awareness)?

thx Berg

> **···**
>
> Am 07.09.2016 08:49 schrieb "Martin DeMello" \<martindemello@gmail.com\>:
> 
> > jruby sounds like the easiest way forward here
> > 
> > [Concurrency in jruby · jruby/jruby Wiki · GitHub](https://github.com/jruby/jruby/wiki/Concurrency-in-jruby)
> > 
> > On Tue, Sep 6, 2016 at 11:41 PM, A Berger \<aberger7890@gmail.com\> wrote:
> > 
> > > Hi  
> > > I see it changed from stdlib to core,  
> > > but nothing is found with search engine: [Bing](https://www.bing.com/search?q=)  
> > > ruby+queue
> > > 
> > > I want to use all cpu-cores, so processes look better than threads,  
> > > so I cant use this queue.  
> > > Better solution:  
> > > Perhaps someone can change threads to utilize all cores?
> > > 
> > > Thanks  
> > > Berg  
> > > Am 07.09.2016 08:32 schrieb "Raj Sahae" \<rajsahae@gmail.com\>:
> > > 
> > > > I'm not sure what you mean. I looked at ruby-doc.org and I see the  
> > > > Queue class in the core library for 2.1, 2.2, and 2.3.
> > > > 
> > > > On Tue, Sep 6, 2016, 11:23 PM A Berger \<aberger7890@gmail.com\> wrote:
> > > > 
> > > > > Hi  
> > > > > It looks like Class queue disappeared in Ruby 2.1 (last seen in Rubydoc  
> > > > > 2.0.0)
> > > > > 
> > > > > Moreover what is the best solution (simplicity, performance) for a  
> > > > > queue feeding different Ruby-processes?  
> > > > > Having 1) fifo and 2) distibutor functionality.  
> > > > > Data should stay in Ram (e.g. Array), but for the communication a pipe,  
> > > > > network/socket or shared mem is possible.  
> > > > > Something fast&proved out there, or do I have to do it by hand?  
> > > > > Whats commonly used (best already with integrated locking)?
> > > > > 
> > > > > Thank you  
> > > > > Berg
> > > > > 
> > > > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org  
> > > > > ?subject=unsubscribe\>  
> > > > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > > > 
> > > > --
> > > > 
> > > > - Raj
> > > > 
> > > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org  
> > > > ?subject=unsubscribe\>  
> > > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > > 
> > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > 
> > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);

---

<div class="post-metadata">

**Author:** ![Martin\_DeMello](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@Martin\_DeMello](https://rubytalk.org/u/Martin_DeMello)\
**Post date:** [7 September 2016 08:15 UTC](https://rubytalk.org/t/data-queuing/73804/8 "2016-09-07T08:15:29Z")

</div>

Hard to say more without knowing what problem you're trying to solve, but  
see if resque does what you want

> **[GitHub - resque/resque: Resque is a Redis-backed Ruby library for creating...](https://github.com/resque/resque)**
>
> Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later. - GitHub - resque/resque: Resque is a Redis-backed Ruby library for c...

martin

> **···**
>
> On Wed, Sep 7, 2016 at 12:32 AM, A Berger \<aberger7890@gmail.com\> wrote:
> 
> > Ok - I still wanted to avoid using java...  
> > But thank you, maybe I should cancel that precondition.
> > 
> > Nevertheless are there solutions for "many-processes" - data-sharing out  
> > there?  
> > (Thinking a database is overkill, besides for mostly needed as queue its  
> > not well suited for that job, but possible if at once fetching+deleting  
> > seen rows)  
> > Shared mem has locking, but it's not "nice" (to implement/use)...  
> > Maybe this is the fastest possibility.
> > 
> > Does anybody have experience using network or pipes (or ready-to-use  
> > queuing-gems with locking/parallel-access awareness)?
> > 
> > thx Berg  
> > Am 07.09.2016 08:49 schrieb "Martin DeMello" \<martindemello@gmail.com\>:
> > 
> > > jruby sounds like the easiest way forward here
> > > 
> > > [Concurrency in jruby · jruby/jruby Wiki · GitHub](https://github.com/jruby/jruby/wiki/Concurrency-in-jruby)
> > > 
> > > On Tue, Sep 6, 2016 at 11:41 PM, A Berger \<aberger7890@gmail.com\> wrote:
> > > 
> > > > Hi  
> > > > I see it changed from stdlib to core,  
> > > > but nothing is found with search engine: [https://www.bing.com/search?q=](https://www.bing.com/search?q=)  
> > > > ruby+queue
> > > > 
> > > > I want to use all cpu-cores, so processes look better than threads,  
> > > > so I cant use this queue.  
> > > > Better solution:  
> > > > Perhaps someone can change threads to utilize all cores?
> > > > 
> > > > Thanks  
> > > > Berg  
> > > > Am 07.09.2016 08:32 schrieb "Raj Sahae" \<rajsahae@gmail.com\>:
> > > > 
> > > > > I'm not sure what you mean. I looked at ruby-doc.org and I see the  
> > > > > Queue class in the core library for 2.1, 2.2, and 2.3.
> > > > > 
> > > > > On Tue, Sep 6, 2016, 11:23 PM A Berger \<aberger7890@gmail.com\> wrote:
> > > > > 
> > > > > > Hi  
> > > > > > It looks like Class queue disappeared in Ruby 2.1 (last seen in  
> > > > > > Rubydoc 2.0.0)
> > > > > > 
> > > > > > Moreover what is the best solution (simplicity, performance) for a  
> > > > > > queue feeding different Ruby-processes?  
> > > > > > Having 1) fifo and 2) distibutor functionality.  
> > > > > > Data should stay in Ram (e.g. Array), but for the communication a  
> > > > > > pipe, network/socket or shared mem is possible.  
> > > > > > Something fast&proved out there, or do I have to do it by hand?  
> > > > > > Whats commonly used (best already with integrated locking)?
> > > > > > 
> > > > > > Thank you  
> > > > > > Berg
> > > > > > 
> > > > > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org  
> > > > > > ?subject=unsubscribe\>  
> > > > > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > > > > 
> > > > > --
> > > > > 
> > > > > - Raj
> > > > > 
> > > > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org  
> > > > > ?subject=unsubscribe\>  
> > > > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > > > 
> > > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org  
> > > > ?subject=unsubscribe\>  
> > > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > > 
> > > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);
> > 
> > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);

---

<div class="post-metadata">

**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\
**Post date:** [7 September 2016 08:09 UTC](https://rubytalk.org/t/data-queuing/73804/9 "2016-09-07T08:09:11Z")

</div>

The easy way out is to use Queue together with DRb and use processes  
instead of threads for concurrent work.

Cheers

robert

> **···**
>
> On Wed, Sep 7, 2016 at 9:32 AM, A Berger \<aberger7890@gmail.com\> wrote:
> 
> > Ok - I still wanted to avoid using java...  
> > But thank you, maybe I should cancel that precondition.
> > 
> > Nevertheless are there solutions for "many-processes" - data-sharing out  
> > there?  
> > (Thinking a database is overkill, besides for mostly needed as queue its not  
> > well suited for that job, but possible if at once fetching+deleting seen  
> > rows)
> 
> --  
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can  
> - without end}  
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)

---

<div class="post-metadata">

**Author:** ![Andy\_Jones](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/andy_jones/32/1829_2.png) [@Andy\_Jones](https://rubytalk.org/u/Andy_Jones)\
**Post date:** [7 September 2016 08:16 UTC](https://rubytalk.org/t/data-queuing/73804/10 "2016-09-07T08:16:23Z")

</div>

> \> Nevertheless are there solutions for "many-processes" - data-sharing out  
> \> there?  
> \> (Thinking a database is overkill, besides for mostly needed as queue its  
> not  
> \> well suited for that job, but possible if at once fetching+deleting seen  
> \> rows)
> 
> The easy way out is to use Queue together with DRb and use processes  
> instead of threads for concurrent work.

Always remembering that DRb is as insecure as an insecure thing.

The more secure, but definitely more painful, direction to go: message passing to/from a STOMP Server, or something like RabbitMQ, or Redis' with its publish/subscribe feature.

The nice thing about DRb is that it "magically" transmits objects and method calls from server to client. AFAIK there isn't anything that will give you that with those other messaging solutions. (Hmmm ... adding that to my "mad science" list...)

Click here to view Company Information and Confidentiality Notice.\<[http://www.jameshall.co.uk/index.php/small-print/email-disclaimer&gt](http://www.jameshall.co.uk/index.php/small-print/email-disclaimer&gt);

---

<div class="post-metadata">

**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\
**Post date:** [7 September 2016 09:38 UTC](https://rubytalk.org/t/data-queuing/73804/11 "2016-09-07T09:38:32Z")

</div>

The magic is called "Marshalling". 😉 This also immediately gives you  
the limitation of objects that you can transfer (e.g. File).

Kind regards

robert

> **···**
>
> On Wed, Sep 7, 2016 at 10:16 AM, Andy Jones \<Andy.Jones@jameshall.co.uk\> wrote:
> 
> > The nice thing about DRb is that it "magically" transmits objects and method calls from server to client. AFAIK there isn't anything that will give you that with those other messaging solutions. (Hmmm ... adding that to my "mad science" list...)
> 
> --  
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can  
> - without end}  
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)

---

<div class="post-metadata">

**Author:** ![A\_Berger](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@A\_Berger](https://rubytalk.org/u/A_Berger)\
**Post date:** [7 September 2016 10:54 UTC](https://rubytalk.org/t/data-queuing/73804/12 "2016-09-07T10:54:12Z")

</div>

Hi

Thanks for the different infos!

I'm searching for sg more small&fast, for msging like Twitter [inside one  
server:) ] (never used Twitter, so I don't know what it is today...).

Small data-pieces are exchanged, mostly between each two of many pids (n/2  
x 1:1 , bidir) and a server getting the results (n:1, unidir)  
Seems one big memory-area (array) would be best: using the same memory is  
faster than copying / packing / sending via network, and \*looks\* easier.  
Also the whole state can easily being saved.  
Are there no ready-used solutions for that yet?

Thank you  
Berg

> **···**
>
> Am 07.09.2016 10:16 schrieb "Andy Jones" \<Andy.Jones@jameshall.co.uk\>:
> 
> > \> \> Nevertheless are there solutions for "many-processes" - data-sharing  
> > out  
> > \> \> there?  
> > \> \> (Thinking a database is overkill, besides for mostly needed as queue  
> > its  
> > \> not  
> > \> \> well suited for that job, but possible if at once fetching+deleting  
> > seen  
> > \> \> rows)  
> > \>  
> > \> The easy way out is to use Queue together with DRb and use processes  
> > \> instead of threads for concurrent work.
> > 
> > Always remembering that DRb is as insecure as an insecure thing.
> > 
> > The more secure, but definitely more painful, direction to go: message  
> > passing to/from a STOMP Server, or something like RabbitMQ, or Redis' with  
> > its publish/subscribe feature.
> > 
> > The nice thing about DRb is that it "magically" transmits objects and  
> > method calls from server to client. AFAIK there isn't anything that will  
> > give you that with those other messaging solutions. (Hmmm ... adding that  
> > to my "mad science" list...)
> > 
> > Click here to view Company Information and Confidentiality Notice.\<  
> > [http://www.jameshall.co.uk/index.php/small-print/email-disclaimer&gt](http://www.jameshall.co.uk/index.php/small-print/email-disclaimer&gt);
> > 
> > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);

---

<div class="post-metadata">

**Author:** ![Andy\_Jones](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/andy_jones/32/1829_2.png) [@Andy\_Jones](https://rubytalk.org/u/Andy_Jones)\
**Post date:** [7 September 2016 09:56 UTC](https://rubytalk.org/t/data-queuing/73804/13 "2016-09-07T09:56:46Z")

</div>

> The magic is called "Marshalling". 😉 This also immediately gives you  
> the limitation of objects that you can transfer (e.g. File).

To be fair to the authors of DRb, there is \*slightly\* more going on than just Marshal::load( Marshal::dump )...

Click here to view Company Information and Confidentiality Notice.\<[http://www.jameshall.co.uk/index.php/small-print/email-disclaimer&gt](http://www.jameshall.co.uk/index.php/small-print/email-disclaimer&gt);

---

<div class="post-metadata">

**Author:** ![A\_Berger](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@A\_Berger](https://rubytalk.org/u/A_Berger)\
**Post date:** [7 September 2016 11:08 UTC](https://rubytalk.org/t/data-queuing/73804/14 "2016-09-07T11:08:40Z")

</div>

...  
At the BEGINNING all processes are fed with data, so the main-queue is  
unidirectional, but direction is switchable 😉  
(Which makes no difference if you see the SAME queue as 2 unidir queues ,  
but only one active; whereas the client-queues are really bidir: my idea  
was to put an receiver-adress into every packet, but 2 queues each seem to  
be more efficient)  
- There are many msg-streams, so I want to avoid duplicating the data.

Thanks  
Berg

---

<div class="post-metadata">

**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\
**Post date:** [7 September 2016 11:14 UTC](https://rubytalk.org/t/data-queuing/73804/15 "2016-09-07T11:14:56Z")

</div>

For the "magic" transfer of objects I think that is really all there  
is to it. You basically have ordinary objects which are sent via  
mashalling and other objects for which only a stub is transferred that  
then invokes methods on the remote original. That would have to be  
done for the Queue instance in this scenario btw.

Kind regards

robert

> **···**
>
> On Wed, Sep 7, 2016 at 11:56 AM, Andy Jones \<Andy.Jones@jameshall.co.uk\> wrote:
> 
> > > The magic is called "Marshalling". 😉 This also immediately gives you  
> > > the limitation of objects that you can transfer (e.g. File).
> > 
> > To be fair to the authors of DRb, there is \*slightly\* more going on than just Marshal::load( Marshal::dump )...
> 
> --  
> [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can  
> - without end}  
> [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)

---

<div class="post-metadata">

**Author:** ![A\_Berger](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@A\_Berger](https://rubytalk.org/u/A_Berger)\
**Post date:** [7 September 2016 12:10 UTC](https://rubytalk.org/t/data-queuing/73804/16 "2016-09-07T12:10:08Z")

</div>

Hi  
...and lastly the (local running) service for remotely accessing the local  
objects (same the other way round), if ever needed.

- would all be much easier than using SharedMemory :((

kind regards  
berg

> **···**
>
> Am 07.09.2016 13:15 schrieb "Robert Klemme" \<shortcutter@googlemail.com\>:
> 
> > On Wed, Sep 7, 2016 at 11:56 AM, Andy Jones \<Andy.Jones@jameshall.co.uk\> \> wrote:  
> > \>\> The magic is called "Marshalling". 😉 This also immediately gives you  
> > \>\> the limitation of objects that you can transfer (e.g. File).  
> > \>  
> > \> To be fair to the authors of DRb, there is \*slightly\* more going on than  
> > just Marshal::load( Marshal::dump )...
> > 
> > For the "magic" transfer of objects I think that is really all there  
> > is to it. You basically have ordinary objects which are sent via  
> > mashalling and other objects for which only a stub is transferred that  
> > then invokes methods on the remote original. That would have to be  
> > done for the Queue instance in this scenario btw.
> > 
> > Kind regards
> > 
> > robert
> > 
> > --  
> > [guy, jim, charlie].each {|him| remember.him do |as, often| as.you\_can  
> > - without end}  
> > [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/)
> > 
> > Unsubscribe: \<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe\>  
> > \<[http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt](http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt);