[ruby-talk:444064] is times() an async operation

Austin_Ziegler_via_r · 17 December 2022 07:53

for instance,

10.times do |s| puts s end
0
1
2
3
4
5
6
7
8
9
=> 10

is the arguments block run in async mode?

I have this question b/c my program quit earily. i don't know if it's due to somewhat async operations.

the code is like (connection to rabbitmq),

conn = Bunny.new
conn.start

ch = conn.create_channel
exchange = ch.fanout("myfanout",:durable => true)

100.times do |s|
exchange.publish(s.to_s)
end

It can't publish the entire 100 items to the broker, just some items.

Thanks.

···

______________________________________________
ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org
To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org
ruby-talk info -- Info | ruby-talk@ml.ruby-lang.org - ml.ruby-lang.org

Austin_Ziegler_via_r · 17 December 2022 14:26

... my program quit early. I don't know if it's due to somewhat async operations.

the code is like (connection to rabbitmq),

100.times do |s|
exchange.publish(s.to_s)
end

It can't publish the entire 100 items to the broker, just some items.

Take a look at:

github.com/ruby-amqp/bunny

losing published messages

opened 09:08AM - 05 May 21 UTC

closed 04:59PM - 24 May 21 UTC

padoauk

Hi. Recently I've started RabbitMQ and bunny. With the following code, I trie…d simple but large pub/sub test and found some messages are lost. I've executed the script below, which should publish 100000 messages, iteratatively with 3 seconds pause. Then the number of messages subscriber receives fluctuates. The fluctuation looks like random, e.g. [99804, 99815, 99809, 99836, 99823, 99864, 99815, 99809, 999821] and so on. When the sleep call, the comment line in the code below, is executed, the situation becomes pretty much better. When tried with different publisher code written in python with pika and the same subscriber, there's no fluctuation and subscriber receives all the messages. Therefore, the problem exists in the ruby publisher. Please let me know what is wrong ? ``` #!/usr/bin/env ruby require 'bunny' SrvIp = 'localhost' Topic = 'my.topic' NumID = 10000 NumIter = 10 # prepare messages of random byte array def messages(num: 256, len: 256) msg = [] num.times do msg.push( Random.bytes(len) ) end return msg end Msgbuffer = messages(num: 256, len: 256) # many routing keys Route_names = [] NumID.times do |i| rname = "AA#{'%05d'%i}" Route_names.push(rname) end connection = Bunny.new(hostname: SrvIp) connection.start channel = connection.create_channel exchange = channel.topic(Topic) pt = 0 tm0 = Time.now num = 0 NumIter.times do |i| Route_names.each do |nm| msg = Msgbuffer[pt % Msgbuffer.length] pt += 1 begin exchange.publish(msg, routing_key: nm) rescue => e puts e.to_s end num += 1 #sleep(0.000001) end end tm1 = Time.now duration = tm1.to_f - tm0.to_f puts "#messages: #{num}, duration: #{'%.2f' %(duration)} sec, avg: #{'%.2f' % (1000.0 * duration / num)} msg/msec" ``` Here's data of the environment: RabbitMQ: (Docker container) OS: Linux RabbitMQ version: 3.8.15 Node name: rabbit@rabbitmq Erlang configuration: Erlang/OTP 23 [erts-11.2.1] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] bunny: ruby: 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux] bunny (2.17.0)

Q: "the script below, which should publish 100000 messages, iteratatively with 3 seconds pause. Then the number of messages subscriber receives fluctuates. The fluctuation looks like random, e.g. [99804, 99815, 99809, 99836, 99823, 99864, 99815, 99809, 999821] and so on."

https://www.rabbitmq.com/publishers.html#data-safety
https://www.rabbitmq.com/confirms.html#publisher-confirms

···

On December 17, 2022 7:53:39 AM UTC, Henry R via ruby-talk <ruby-talk@ml.ruby-lang.org> wrote:
A: "The code in this example does not use publisher confirms. Data safety is a joint responsibility of RabbitMQ nodes and your own code. See Publisher Data Safety."

______________________________________________
ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org
To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org
ruby-talk info -- Info | ruby-talk@ml.ruby-lang.org - ml.ruby-lang.org

Austin_Ziegler_via_r · 18 December 2022 01:00

That makes sense to me. Thank you.

···

On December 17, 2022 7:53:39 AM UTC, Henry R via ruby-talk <ruby-talk@ml.ruby-lang.org> wrote:

>
> ... my program quit early. I don't know if it's due to somewhat async operations.
>
> the code is like (connection to rabbitmq),
>
> 100.times do |s|
> exchange.publish(s.to_s)
> end
>
> It can't publish the entire 100 items to the broker, just some items.
>

Take a look at:
losing published messages · Issue #611 · ruby-amqp/bunny · GitHub

Q: "the script below, which should publish 100000 messages, iteratatively with 3 seconds pause. Then the number of messages subscriber receives fluctuates. The fluctuation looks like random, e.g. [99804, 99815, 99809, 99836, 99823, 99864, 99815, 99809, 999821] and so on."

A: "The code in this example does not use publisher confirms. Data safety is a joint responsibility of RabbitMQ nodes and your own code. See Publisher Data Safety."

Publishers — RabbitMQ
Consumer Acknowledgements and Publisher Confirms — RabbitMQ

______________________________________________
ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org
To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org
ruby-talk info -- Info | ruby-talk@ml.ruby-lang.org - ml.ruby-lang.org

______________________________________________
ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org
To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org
ruby-talk info -- Info | ruby-talk@ml.ruby-lang.org - ml.ruby-lang.org