DeepNets-1M

DeepNets-1M 数据集由表示为图的神经网络架构组成，其中节点是操作（卷积、池化等），边对应于通过网络的数据的前向传递流。 DeepNets-1M 拥有 100 万个训练架构和 1402 个分布内 (ID) 和分布外 (OOD) 评估架构：500 个验证和 500 个测试 ID 架构、100 个宽 OOD 架构、100 个深度 OOD 架构、100 个密集 OOD 架构，100 个没有批量标准化的 OOD 架构，以及 2 个预定义架构（ResNet-50 和 12 层 Visual Transformer）。对于 1402 评估架构，DeepNets-1M 在使用随机梯度下降 (SGD) 训练网络后，包括 CIFAR-10 和 ImageNet 上的网络精度。除了准确性之外，还包括评估架构的其他属性：噪声图像的准确性、推理和收敛时间。架构的这些属性可以训练神经架构搜索模型。 DeepNets-1M 用于训练和评估参数预测模型，例如 Graph HyperNetworks。这些模型可以在单个前向传递中预测给定网络（图）的所有参数，并且可以将结果与使用 SGD 优化参数进行比较。The DeepNets-1M dataset consists of neural network architectures represented as graphs, where nodes represent operations (e.g., convolution, pooling, etc.) and edges correspond to the forward pass data flow through the network. DeepNets-1M contains 1 million training architectures and 1402 in-distribution (ID) and out-of-distribution (OOD) evaluation architectures: 500 validation and 500 test ID architectures, 100 wide OOD architectures, 100 deep OOD architectures, 100 dense OOD architectures, 100 OOD architectures without batch normalization, and 2 predefined architectures (ResNet-50 and 12-layer Visual Transformer). For the 1402 evaluation architectures, DeepNets-1M includes the network accuracies on CIFAR-10 and ImageNet after training the networks with stochastic gradient descent (SGD). In addition to accuracy, other properties of the evaluation architectures are included: accuracy on noisy images, inference time and convergence time. These properties of architectures enable training neural architecture search models. DeepNets-1M is used to train and evaluate parameter prediction models such as Graph HyperNetworks. These models can predict all parameters of a given network (graph) in a single forward pass, and the results can be compared with those obtained by optimizing parameters using SGD.

搜集汇总

背景与挑战

背景概述

DeepNets-1M是一个由图结构神经网络架构组成的数据集，包含100万个训练架构和1402个评估架构，用于训练和评估参数预测模型，如Graph HyperNetworks，以在单个前向传递中预测网络参数并与SGD优化结果比较。