Efficient optimized query mesh for data streams

Abstract

Most of query optimizers choose a single query plan for processing all the data based on the average data statistics. But this plan is usually not efficient with the uncertain stream datasets of modern applications as network monitoring, sensor networks and financial applications; where these data have continuous variations over time. In this paper we propose an optimized query mesh for data stream (OQMDS) frameworks. In which, process data streams over multiple query plans, each of them is optimal for the sub-set of data with the same statistics. The OQMDS solution depends on preparing multiple query plans and continuously chooses the best execution plan for each sub-set of incoming data streams based on their statistics. We also propose two optimization algorithms called Optimized Iterative Improvement Query Mesh (OII-QM) and Non-Search based Query Mesh (NS-QM) algorithms, to efficiently generate the multiple plans (the optimized QM solution) which are used to process the online data streams. Our experimental results show that, the proposed solution OQMDS improves the overall performance of data stream processing.