GAP(GAP Benchmark Suite)

不幸的是，缺乏广泛使用的图形基准套件迫使每个研究出版物创建自己的评估方法，这通常会导致错误或不必要的差异。我们观察到的常见严重错误包括：使用非常小的输入图、仅使用单个输入图拓扑或使用低性能实现作为基线。这些方法论问题使得好的想法很难脱颖而出，并模糊了为什么这些想法是有益的背后的推理。 为了让研究界在加速图处理方面取得进展，能够正确可靠地比较结果非常重要。我们创建了 GAP 基准套件来标准化评估，以缓解我们观察到的方法问题。通过标准化，我们希望不仅可以让结果更容易比较，还可以防止常见的评估错误。我们提供了一个基准规范来标准化方法和一个高性能的参考实现作为基准。我们的基准是与我们的工作负载特征共同设计的，并且在社区反馈的指导下经历了多次修订。Unfortunately, the lack of a widely adopted graph benchmark suite forces every research publication to develop its own evaluation methodology, which often leads to errors or unnecessary discrepancies. Common serious errors we have observed include: using extremely small input graphs, relying solely on a single input graph topology, or employing low-performance implementations as baselines. These methodological issues make it difficult for promising ideas to stand out, and obscure the reasoning behind why these ideas are beneficial. To enable the research community to advance in accelerating graph processing, it is critically important to be able to compare results correctly and reliably. We have created the GAP Benchmark Suite to standardize evaluation and mitigate the methodological issues we observed. Through standardization, we aim not only to make results easier to compare, but also to prevent common evaluation errors. We provide a benchmark specification to standardize methodologies, alongside a high-performance reference implementation for benchmarking. Our benchmark was co-designed with our workload characteristics, and has undergone multiple revisions guided by community feedback.

搜集汇总

背景与挑战

背景概述

GAP基准套件旨在解决图处理研究中因缺乏标准化评估方法而导致的常见错误，如使用小输入图或低性能基线。它提供了一个标准化的基准规范和高性能参考实现，以促进研究结果的可靠比较，帮助加速图处理领域的进展。