Maki Nage is a framework designed to work on streaming data. A Maki Nage application takes a stream of events as input, applies some transformations on these events, and returns another stream of events:
Maki Nage leverages two other projects as a foundation:
The structure of Maki Nage is the following one:
All transformation functions in Maki Nage are ReactiveX operators. RxSCi is a ReactiveX extension library, dedicated to data manipulation. The combination of ReactiveX and RxSci makes it very easy to deal with streams of events.
Kafka is used for multi-core and multi-machine scalability. Thanks to its usage, it is easy to start working on a relatively small development machine, then scale on a bigger machine or a small cluster, and finally scale on a cloud platform. All these steps with the same code base.
The key advantages of Maki Nage are:
Ease of use, thanks to declarative, extensible python APIs.
Code reuse from development up to deployment.
A unified paradigm for streaming and batch processing.