Examples of several popular frameworks for big data processing that can be used in Java applications. Each of the frameworks is demonstrated by implementing a simple pipeline for processing of data from devices that measure the air quality index for a given area. For simplicity we will assume that numeric data from the devices is either received in batches or in a streaming fashion. The following frameworks are demonstrated:
- Apache Spark
- Apache Flink
- Apache Storm
- Apache Ignite
- Hazelcast Jet
- Kafka Streams
- Pulsar Functions