Resolving pipe deadlock

ruby-talk
41

I'm just gonna throw two ideas out there:

Thank you for the consideration!

p = Pipe.new
p << READ_TABLE
  << READ_XML
  << DO_SCIENTIFIC_CALCULATION
  << WRITE_TABLE
p.execute_processes

That was there all from the beginning: Martin hat the chaining idea in
his original posting:

And I added << in my first version.

p = Pipe.new | READ_TABLE | READ_XML | DO_SCIENTIFIC_CALCULATION | WRITE_TABLE
p.execute_processes

Yes, I had that in the version from yesterday. :-

Sorry. :slight_smile:

Btw. I am not sure whether the idea with the pipe symbol was actually
a good one because items are not identical (as with integer bit
arithmetic for example) and it's also not executed immediately (as the
shell does with a pipeline).

Cheers

robert

···

On Thu, Jan 23, 2014 at 4:13 AM, Ryan Davis <ryand-ruby@zenspider.com> wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

42

Also, as I've said before I'd rather stack the msg handling pipeline
class on top of Pipe. That way you keep concerns nicely separated and
can use class Pipe for other purposes as well. The way you did it you
created a class that can only handle MessagePack messages.

Again, I get it working one way and then there is some other superior
way :o) - I'll give it a shot.

Actually I mentioned that already - maybe it was not explicit enough. :slight_smile:

Now, I want two additional features.

First, I want a to_s method in the Pipe class that return a string with
the commands and options run like: "Pipe.new.add(:cat. input:
"foo").add(:save, output: "bar").add(:dump).run". Strings like this I
would like to log in a history file for documentation of commands run
and easy rerunning (in irb). I think there might be a problem with this,
with the lambdas giving the separation of the Pipe and commands (from
the Pipe class there is no way to see what is in the lambdas)?

There is generally no way to get at the contents of the lambdas.
However, if you generate the lambdas yourself (like in those CAT
methods we have discussed earlier) you can do something like:

def CAT(*args)
  l = lambda do ... end

  def l.to_s
    "CAT(#{args.inspect})" # or whatever
  end

  l
end

You could devise other schemes that would allow to execute the code
again. You could even write a file and create the lambda from there.
Lots of options.

Second, I would like to keep track of some basic statistics from each
command: number of records in and out, runtime, and keys seen (I will be
using records consisting of simple hashes). Each command could write a
temporary file and then the stats could be compiled after execution.
Alternative, the stats could be passed along the IO stream as the last
record, or each command should also have separate pipes inter process
communication of stats.

Without giving too much thought I would prefer the approach with the
statistics sent down the pipe at the end because it avoids fiddling
with temporary files.

Kind regards

robert

···

On Sat, Feb 1, 2014 at 3:18 PM, Martin Hansen <lists@ruby-forum.com> wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/