Towards FAIR Data Stream Processing Ecosystems

ACM International Conference on Distributed and Event-Based Systems (DEBS), pp. 203–206

Abstract

With the rapid growth of data from heterogeneous, distributed sources, data streams need to be increasingly processed in the cloud-edge continuum. Processing is distributed between diverse edge environments and homogeneous, but powerful data centers, to optimally utilize available resources, alleviate infrastructure bottlenecks and follow the principle of data locality. Compared to cloud infrastructure, compute and data resources on the edge are distributed across geographical regions, infrastructures and organizational units with independent data processing systems. However, existing data stream processing frameworks provide integrated systems, which require matching software components to be used across the whole, distributed infrastructure or even assume centralized control over all resources. We argue, that this cloud-like, centralized approach does not fit to the decentralized nature of the edge environment. Focusing on fully integrated systems, which are either limited to single organizational units or require a certain degree of homogeneity limits data sovereignty and the overall potential of distributed data stream processing on the edge. Instead, we propose to develop data stream processing as part of data ecosystems, and connect locally independent and sovereign systems through a lightweight set of common standards, protocols, and semantic descriptions.

Authors 1

1. Liam Tirpitz corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Aachen, Germany