Active DNS Project

The dataset comes in Avro format which can be processed by several tools like avro-tools or fastavro for Python. We are working on a JSON version of the data for applications that do not rely on Java or Hadoop. This data is freely available to the academic research community for incorporation into new or existing work. This dataset relies on active DNS and a new large scale system, Thales, which is able to systematically query and collect large volumes of active DNS data. The output of this system is a distilled dataset that can be easily used by the security community. Thales has been reliably active for more than six monts and collected many terabytes of DNS data, while causing only a handful of abuse complaints.本数据集采用阿夫罗（Avro）格式，可通过avro-tools或Python环境下的fastavro等多款工具进行处理。目前我们正在为不依赖Java或Hadoop的应用场景开发该数据集的JSON版本。本数据集可免费向学术研究社区开放，用于整合至新的或已有的研究工作中。本数据集依托主动域名系统（Domain Name System，DNS）以及全新的大规模系统Thales构建，该系统可系统性地查询并收集海量主动DNS数据。该系统的输出为经过提纯的数据集，可被安全社区便捷地使用。Thales系统已稳定运行超过六个月，累计收集了数太字节的DNS数据，且仅收到少量滥用投诉。

搜集汇总

背景与挑战

背景概述

Active DNS Project是由佐治亚理工学院提供的外部数据集，旨在通过每日主动查询DNS数据（包括COM、NAME等流行区域和Alexa Top 1M等域名列表）来支持DNS研究。数据以Avro格式提供，未来将提供JSON版本，免费供学术研究使用，已通过Thales系统收集超过数TB数据且滥用投诉较少。