Learned Stereo

We present a passive stereo depth system that produces dense and accurate point clouds optimized for human environments, including dark, textureless, thin, reflective and specular surfaces and objects, at 2560x2048 resolution, with 384 disparities, in 30 ms. The system consists of an algorithm combining learned stereo matching with engineered filtering, a training and data-mixing methodology, and a sensor hardware design. Our architecture is 15x faster than approaches that perform similarly on the Middlebury and Flying Things Stereo Benchmarks. To effectively supervise the training of this model, we combine real data labelled using off-the-shelf depth sensors, as well as a number of different rendered, simulated labeled datasets. We demonstrate the efficacy of our system by presenting a large number of qualitative results in the form of depth maps and point-clouds, experiments validating the metric accuracy of our system and comparisons to other sensors on challenging objects and scenes. We also show the competitiveness of our algorithm compared to state-of-the-art learned models using the Middlebury and FlyingThings datasets.我们提出一款被动立体深度系统（passive stereo depth system），可生成稠密且精准的点云，专为人类日常环境优化，可适配低光、无纹理、薄型、反光及镜面表面与物体，分辨率达2560×2048，视差（disparities）数量为384，单帧处理时延仅30毫秒。该系统由三大核心模块构成：融合学习型立体匹配与工程化滤波的算法框架、训练与数据混合方法，以及传感器硬件设计。相较于在Middlebury与Flying Things立体基准测试集上表现相当的同类方法，我们的架构运行速度快15倍。为有效开展模型训练的监督，我们融合了两类标注数据集：一类是采用市售深度传感器标注的真实场景数据，另一类是多组不同的渲染模拟标注数据集。为验证系统的有效性，我们展示了大量以深度图与点云形式呈现的定性结果，开展了验证系统度量精度的定量实验，并针对挑战性物体与场景与其他传感器进行了对比测试。此外，我们基于Middlebury与FlyingThings数据集，将所提算法与当前最先进的学习型模型进行对比，证实了其竞争力。