AI Video Retrieval: A Semantic Search & Timestamp Alignment System

Problem and motivation

Keyword search and manual video tags do not reliably retrieve semantically related moments. A practical system must connect a natural-language query to the relevant time interval while indexing both spoken content and visual context at usable cost.

Method and contribution

The implementation extracts and segments audio with PyDub 0.25.1, samples frames with OpenCV 4.11.0.86 at the stated 10% probability, and obtains segment/word timing with Whisper-Timestamped. BLIP captioning through Transformers 4.45.2 converts sampled frames to text. The system embeds transcripts and captions with all-MiniLM-L6-v2, indexes them with TXTAI 7.4.0 and HNSW-backed similarity search, and returns matches whose cosine/SIMILAR score is at least 0.30. Django 5.1.5 provides the application layer and JSON stores the timestamped metadata.

Findings and evidence

The paper evaluates 100 pre-indexed videos with 30 Brock graduate and undergraduate participants. The reported indexing time is approximately 30–40 seconds for a 6–8 minute video, and the interface returns audio, visual, or multimodal matches with timestamps. The usability study reports generally positive relevance, timestamp usefulness, clarity, and recommendation responses; the paper also reports faster retrieval as the indexed collection grows.

Limitations and future directions

Limitations: The evaluation uses a small user study and a limited video collection rather than a standard retrieval benchmark with annotated temporal ground truth. The system covers speech and sampled visual captions but not OCR, explicit object detection, action recognition, or a broad comparison against retrieval baselines. The paper's implementation section and documentation also contain a TXTAI version discrepancy that should be preserved as a reproducibility caveat.

Future work: Add OCR, object and action features, richer multimodal fusion, standard temporal-retrieval metrics, larger and more diverse video corpora, and systematic latency/quality comparisons against dense-video-retrieval baselines.

Sources and identifiers

• Published version published
• IEEE record public_source_record

When to cite this paper

Cite this paper when your work performs timestamp-aware semantic retrieval over speech and visual video content.

• The Whisper-Timestamped plus BLIP multimodal indexing pipeline.
• Timestamped audio/visual sub-indexes with embedding similarity and HNSW retrieval.
• The 30-participant, 100-video usability evaluation and reported 0.30 similarity threshold.
• Future multimodal extensions such as OCR, object detection, and action recognition.

Citation

@inproceedings{ezzatiJivan2025aivideo,
  author = {Hridoy Rahman and Naser Ezzati-Jivan and Blessing Ogbuokiri},
  title = {AI Video Retrieval: A Semantic Search {\&} Timestamp Alignment System},
  year = {2025},
  booktitle = {2025 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA)},
  pages = {1-6},
  publisher = {IEEE},
  doi = {10.1109/ACDSA65407.2025.11166430},
  url = {https://doi.org/10.1109/ACDSA65407.2025.11166430}
}

Rahman, H., Ezzati-Jivan, N., & Ogbuokiri, B. (2025). AI Video Retrieval: A Semantic Search & Timestamp Alignment System. In 2025 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) (pp. 1-6). https://doi.org/10.1109/ACDSA65407.2025.11166430

H. Rahman, N. Ezzati-Jivan, and B. Ogbuokiri, "AI Video Retrieval: A Semantic Search & Timestamp Alignment System," in 2025 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA), pp. 1-6, 2025, doi: 10.1109/ACDSA65407.2025.11166430