Hi!
Kiba is a Ruby-based solution to write data pipelines (data sync processes,
large or micro-batches, reformatting / interop etc).
Version 2 leverages Ruby's "Enumerator" to add the ability to yield an
arbitrary number of rows from transform steps. This means that ETL
components can be decoupled more & composed together much more easily.
* Release v2.0.0 · thbar/kiba · GitHub
* Introduce new StreamingRunner by thbar · Pull Request #44 · thbar/kiba · GitHub
Version 2 also now officially supports a block-form API that allows Kiba
jobs to be called programmatically from Sidekiq, Faktory, Rake etc (in
addition to the command line mode already available in version 1).
Happy data processing!
Thibaut