---
markdown_generated_at: "2026-11-10"
title: "Qualcomm Interactive Video Dataset (QIVD)"
url: "https://www.qualcomm.com/developer/software/qualcomm-interactive-video-dataset-qivd"
---

# Qualcomm Interactive Video Dataset (QIVD)

![](https://s7d1.scene7.com/is/image/dmqualcommprod/bg-grey-software-desktop-1)

[Get Software](https://www.qualcomm.com/developer/software/qualcomm-interactive-video-dataset-qivd/downloads)

[Download](https://www.qualcomm.com/developer/software/qualcomm-interactive-video-dataset-qivd/downloads)

Qualcomm AI Research presents the QIVD dataset, which enables the study of visual understanding with Vision-Language Models (VLMs) to answer questions about events occurring in videos. This long-standing goal in AI is a prerequisite for real-world AI assistants and humanoid robots to interact with humans in everyday situations.

This is useful for AI systems that must interpret and respond to real-time visual and audio inputs, or to interact with a user in the real world and understand what is occurring.

Product license agreement

Our dataset comprises 2900 data items, where each item contains:

- A video with audio of a user asking a question about the video.
- A transcription of a human-generated question asked during the recording, and a human-generated answer to that question.
- A ground-truth timestamp that marks when it's appropriate to answer the question.

![Qualcomm-image](https://s7d1.scene7.com/is/image/dmqualcommprod/dataset-diagram?$QC_Responsive$&fmt=png-alpha)

## Categories

Each item is assigned to one of 13 predefined semantic categories representing distinct visual reasoning capabilities:

| Category | Description |
|---|---|
| Action Attribute | Inquiries about the manner of a performed action, such as *Which hand did I use to wave?* or *How fast did I jump?*—tests the ability to recognize fine-grained characteristics of dynamic events. |
| Action Counting | Questions about an action's repetition frequency, such as *How many times did I clap?*— evaluates temporal reasoning and event segmentation capabilities. |
| Action Detection | Identifies the specific action performed, such as *What am I doing right now?*— assesses basic activity recognition in dynamic scenes. |
| Action Understanding | Questions the purpose or outcome of an action, such as *What does this gesture mean?* or *Why am I moving the chair?*—tests higher-level action interpretation and intention recognition. |
| Object Attributes | Inquiries about an object's characteristics, such as *What color is this book?* or *Is this cup empty or full?*—evaluates fine-grained visual perception of static properties. |
| Object Counting | Determines the number of objects present, such as *How many pens are on the table?*—tests quantitative reasoning and object individuation. |
| Object Detection | Identifies scene objects, such as *Is there a lamp in this room?*—assesses basic object recognition capabilities. |
| Object Referencing | Indirectly points to an object within the scene, such as *What am I pointing at?* or *What is behind me?*—evaluates spatial reasoning and deictic reference resolution. |
| Object Understanding | Questions about an object's nature or function, such as *What is this tool used for?*—tests semantic knowledge about objects beyond mere recognition. |
| Scene Understanding | Inquiries about the environment, such as *What room am I in?* or *Is it daytime or nighttime?*— evaluates holistic scene interpretation. |
| Audio-Visual | Questions that require audio information for a complete answer, such as *What sound am I making?* or *Am I speaking loudly or softly?*—tests cross-modal integration capabilities. |
| OCR | Extracts text from an object, such as *What does this sign say?*—evaluates the capability to recognize text in the real world and within the context of the conversation. |
| Subjective | Solicits general opinions about an object or scene, such as *Does this outfit look good?*—tests a model's ability to respond sensibly to subjective questions. |

## Dataset Details

| Item | Value |
|---|---|
| Size | 2900 Videos |
| Average Length | 5.10 seconds |
| Total Number of Frames | 443350 |
| Average FPS | 30 |
| Average Image Resolution for Frames | 640 x 382.29 |
| Vocabulary Size | 3624 Words / 3072 Tokens |
| Total Semantic Categories | 13 |
| Average Question Length (words) | 6.09 |
| Average Answer Length (words) | 7.23 |
| Average Short Answer Length (words) | 1.38 |
| Average Answer Timestamp (%) | 81.47% |
| Total Questions | 1661 |
| Questions with "where" | 47 |
| Questions with "how" | 512 |
| Questions with "what" | 1102 |
| Total Deictic References | 789 |
| Questions with "here" | 32 |
| Questions with "these" | 39 |
| Questions with "that" | 45 |
| Questions with "there" | 105 |
| Questions with "this" | 568 |
| Source | All the videos in the benchmark have been crowdsourced and then annotated by non-expert annotators. Following this, the videos undergo a rigorous quality check process and a semantic categorization process. |
| Language | English |

| Item | Average Answer Timestamp | Total Samples |
|---|---|---|
| Average Answer Timestamp and Total Samples | 81.47% | 2900 |
| Action Attributes | 84.31% | 155 |
| Action Counting | 92.22% | 225 |
| Action Detection | 85.46% | 440 |
| Action Understanding | 81.47% | 110 |
| Object Attributes | 79.52% | 562 |
| Object Counting | 78.41% | 286 |
| Object Detection | 76.95% | 211 |
| Object Referencing | 79.18% | 706 |
| Object Understanding | 80.63% | 79 |
| Scene Understanding | 79.91% | 38 |
| Audio-Visual | 90.09% | 22 |
| OCR | 83.04% | 23 |
| Subjective | 77.39% | 43 |

## Data Format

The dataset is distributed in two files:

- **videos.zip:** 2,900 video files in .mp4 format.
- **annotations.zip:** Single .json file with all video annotations. This file is structured as a list of 2,900 dictionary entries, where each corresponds to one video and includes the following metadata and labels:
  - **video**: Video filename (e.g., "00000000.mp4").
  - **question**: Question asked in the video (e.g., "What am I holding in my left hand?").
  - **answer**: Complete answer to the question (e.g., "You are holding a Rubik's cube in your left hand.").
  - **short_answer**: Concise version of the answer with only the key information (e.g., "A Rubik's cube").
  - **timestamp**: Optimal time in the video to answer the question (e.g., "00:04.4").
  - **category**: Category of the question (e.g., "object referencing").

## Dataset Citation Instructions

The dataset is intended for research purposes only. Please cite our [paper](https://arxiv.org/abs/2503.19356) if you use this dataset in your research.

@misc{pourreza2025visionlanguagemodelsanswerface,
title={Can Vision-Language Models Answer Face to Face Questions in the Real-World?},
author={Reza Pourreza and Rishit Dagli and Apratim Bhattacharyya and Sunny Panchal and Guillaume Berger and Roland Memisevic},
year={2025},
eprint={2503.19356},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2503.19356},
}

## Dataset license

This dataset is intended for research purposes only and to support and contribute to the graph research community. The quality of the configuration space design and the collected execution times may be suboptimal and should not be considered as reference performances of the target device but rather as representative of the problem at hand for research purposes.

[Data License Agreement - Research Use](https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/qvid-dataset-research-license-mar-27-2025.pdf)

## Qualcomm AI Research

At Qualcomm AI Research, we are advancing AI to make its core capabilities – perception, reasoning, and action – ubiquitous across devices. Our mission is to make breakthroughs in fundamental AI research and scale them across industries. By bringing together some of the best minds in the field, we're pushing the boundaries of what's possible and shaping the future of AI.

Qualcomm AI Research continues to invest in and support deep-learning research in computer vision. The publication of the AirLetters dataset for use by the AI research community is one of our many initiatives.

Find out more about [Qualcomm AI Research](https://developer.qualcomm.com/forums/software/ai-research-datasets).

For any questions or technical support, please contact us at [research.datasets@qti.qualcomm.com](mailto:research.datasets@qti.qualcomm.com)

*Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.*

## Connect with our communities

- [LinkedIn](https://www.linkedin.com/showcase/qualcomm-developer)
- [See what's new](https://x.com/Qualcomm_Dev)
- [YouTube](https://www.youtube.com/qualcommdev)
- [GitHub](https://github.com/qualcomm)
- ![](https://s7d1.scene7.com/is/image/dmqualcommprod/discord-icon-1?$QC_Responsive$&fmt=png-alpha) [Discord](https://discord.com/invite/qualcommdevelopernetwork)
- [Visit the forum](https://mysupport.qualcomm.com/supportforums/s/)

![](https://s7d1.scene7.com/is/image/dmqualcommprod/ai-hub-bg-desktop?fmt=webp-alpha&qlt=85)