VN-Lip-Reading

README – Dataset for "A Large-Scale Vietnamese Lip-Reading Dataset for Visual Speech Recognition"==================================================================

Created on: 2025-07-28Created by: Trung Nguyen Quoc, Researcher (VSB-TUO, [e-mail])Last update: 2024-04-23

------------------------------------------------------------------Basic information------------------------------------------------------------------

1. Journal article: A Large-Scale Vietnamese Lip-Reading Dataset for Visual Speech Recognition

2. DOI:10.5281/zenodo.15899222.

3. Contact information:   Name: Matej Sindelar   Institution: VSB – Technical University of Ostrava   E-mail: [e-mail]   ORCID: https://orcid.org/0009-0006-0556-507X

4. Dataset publication date: 2025-07-28

5. Place of publication: Ostrava, Czechia

------------------------------------------------------------------6. Dataset description------------------------------------------------------------------

The Vietnamese Lip Reading (VLR) dataset is a large-scale audiovisual corpus created to support word-level visual speech recognition in Vietnamese. It consists of segmented video clips paired with time-aligned transcriptions at the word level. The video content was collected from publicly available Vietnamese YouTube videos, selected for clear frontal face visibility and natural speaking conditions.

To ensure consistency and visual clarity, each video was automatically processed to center the speaker’s face within the frame. The accompanying transcriptions were generated using automated speech recognition and provide precise timing information for each word spoken. These aligned video-text pairs were drawn from a diverse set of speakers and recording scenarios, capturing the natural variability of Vietnamese spoken language in uncontrolled, real-world environments.

The final dataset includes approximately 116,000 segmented video–transcript pairs. Its scale captures the complexity of Vietnamese visual speech and provides a structured resource suitable for various modeling and analysis tasks.-------Directory Structure and File OrganizationThe VLR dataset is organized in a multi-level folder hierarchy to facilitate structured storage and scalable access. At the root level, the dataset is divided into seven folders named PART1 through PART7. This partitioning is purely practical and intended to support easier downloading, dataset access, and overall file management.

Each PART folder contains multiple utterance-level directories. These folders are uniquely named using a fixed-format convention combining a five-letter prefix (derived from the source playlist or video) and a numeric suffix (e.g., BAIGI1, DAYHO3). Each of these folders corresponds to a specific clip representing a single short utterance in Vietnamese. While these names resemble speaker identifiers, they do not encode speaker identity, and the dataset is not structured around speakers.

Within each utterance-level folder, there are multiple subfolders, each representing a specific word-segmented unit of the utterance. These subfolders (e.g., BACHA2, BAI3258) contain the core data of the dataset:

A face-focused, segmented video file (video.mp4), with audio removed.A word-level annotation file (aligned.csv), containing start and end timestamps for each word in the utterance.---Each video clip in the dataset is accompanied by an aligned CSV file containing word-level temporal annotations. Each file is structured with three columns:- word: the lexical unit corresponding to a spoken segment in the video.- start_time: the timestamp (in seconds) marking the onset of the word.- end_time: the timestamp (in seconds) indicating the offset of the word.--------------------------------------------------------------------------------7. Funding: This work was supported in part by the European Regional Development Fund under the project Research Platform for Digital Transformation and Society 5.0 CZ.02.01.01/00/23_021/0012599 within the Jan Amos Komensky Operational Program. This work was supported in part by the Ministry of Education of the Czech Republic (Project No. SP2026/075) and by the Ministry of Education and Training and Ho Chi Minh City Open University (Project No. B2025-MBS-01).