关键词：粗粒度；分布式存储；SQL；查询优化
摘 要：Shark is a research data analysis system built on a novel coarse-grained distributed shared-memory abstraction. Shark pairs query processing with deep data analysis, providing a unified system for easy data manipulation using SQL and pushing sophisticated analysis closer to its data. It scales to thousands of nodes in a fault-tolerant manner. Shark can answer queries over 40 times faster than Apache Hive and run machine learning programs on large datasets over 25 times faster than equivalent MapReduce programs on Apache Hadoop. Unlike previous systems, Shark shows that it is possible to achieve these speedups while retaining a MapReduce-like execution engine, with the fine-grained fault tolerance properties that such an engine provides. Shark additionally provides several extensions to its engine, including table and column-level statistics collection as well as a cost-based optimizer, both of which we describe in depth in this paper. Cost-based query optimization in some cases improves the performance of queries with multiple joins by orders of magnitude over Hive and over 2 times compared to previous versions of Shark.