Combining Multiple Streaming Queries to provide actionable insights.
You run the linter via
npm run lint:tsYou can automatically fix some issues via
npm run lint:ts:fixThe Streaming Query Hive can handle multiple streaming queries from different sources, and utilizes different streaming operators to process the data. The architecture is designed to be modular, allowing for easy integration of new sources and operators. An example architecture combining three different sources and the results from the RDF Stream Processing Agents to solve for a specific Parent Query is shown below:
In the architecuture, it is assumed that the query results from the RSSP Agents is being streamed to a MQTT topic. The MQTT topic is then consumed by the Streaming Query Hive, which processes the aggregated results using different streaming operators to solve for the Parent Query. The relationship between the queries to esatablish that the Child Queries are part of the Parent Query is established using the Query Containment [1] Relationship.
The Streaming Queries utilized in the architecture are described in the RSP-QL query language [2]. The tool utilized to find if the queries have the Query Containment [1] or the Query Isomorphism [3] is the RSP-QL Containment Checker [4]. The RSP-QL Containment Checker is a tool that checks if a query is contained in another query, and can be used to determine if the results of one query can be used to solve another query. The tool is designed to work with the RSP-QL query language, and builds on the work done by the SPeCS Solver [5] to support aggregation functions and the streaming semantics of the RSP-QL query language.
The MQTT broker can be easily changed with another broker, such as RabbitMQ or Kafka, in the future.
The resultant query results for the parent query can be streamed to a different MQTT topic, or can be stored in a database for further analysis. Moreover, the results can be reasoned over using a reasoning engine such as EYE-JS[6]. The reasoning engine can be used to infer new knowledge from the query results, and can be used to provide actionable insights from the data.
This code is copyrighted by Ghent University - imec and released under the MIT Licence
For any questions, please contact Kush or create an issue in the repository here.
[1]: Query Containment
[2]: RSP-QL Semantics: A Unifying Query Model to Explain Heterogeneity of RDF Stream Processing Systems
[3]: Matching RDF Graphs
[4]: RSP-QL Containment Checker
[5]: SPeCS Solver
[6]: EYE-JS