OpenCV

Mentors: Student: Jorge Velez

I propose to integrate native text tokenization support into OpenCV’s DNN module so that Large Language Models (LLMs) can be run end-to-end directly in OpenCV. Currently, to feed text into models like GPT-2 or GPT-3, users must rely on external libraries (e.g. tiktoken) for tokenization. My project will create a built-in tokenizer utility—implemented in C++ with Python bindings—that can convert raw text into token IDs and decode model outputs back into text. The core deliverable is a new cv::dnn::Tokenizer class supporting Byte Pair Encoding (BPE), along with the ability to load typical vocabulary/merges files and handle special tokens. Once implemented, this will allow a seamless pipeline for text-based or multimodal models within OpenCV: raw text -> tokens -> DNN inference -> tokens -> decoded text. I will also write comprehensive documentation and test this functionality against known tokenizer outputs (e.g., Hugging Face), ensuring correctness and good performance. By adding native tokenization, OpenCV’s DNN module becomes more versatile for cutting-edge AI tasks involving both vision and language. This reduces external dependencies, makes it easier to deploy LLMs in pure C++ or Python environments, and helps unify image and text processing in a single library—further strengthening OpenCV as a comprehensive open-source toolkit.

Mentors: Student: Jinyue CHEN

Description: QR, Barcode and ArUco are all popular code in computer vision applications. OpenCV now support all of them, and can detect and decode them. But OpenCV still expect a better detector and decoder for them. If possible, one efficient deep detector for all kinds of codes can simplify the usage. If one efficent deep detector cannot be achieved, several deep models are also acceptable. Expected Outcomes: Train a deep detector for QR, Barcode and ArUco. Or train three different deep detectors for different codes specifically. The trained model should be easy to implement with the current algorithms in OpenCV on QR/Barcode/ArUco. A nice demo to show how to use the algorithm. Detailed report to demontrate if the trained detector(s) are better than the current solution in OpenCV.

Mentors: Student: shyama7004

I propose to enhance OpenCV’s multi-camera calibration pipeline by curating and organizing high-quality calibration datasets that include varied fiducial markers (chessboards, ArUco, ChArUco, etc.). I will develop automated testing scripts and visualization tools in both C++ and Python, ensuring calibration accuracy and robustness. This project will also deliver clear, user-friendly documentation and a demo video, making calibration more accessible to developers.

Mentors: Student: Suleyman Turkmen

The goal of this project is to improve OpenCV’s imgcodecs module by reducing code duplication, enhancing performance, and improving format support. Deliverables: Modular and maintainable imgcodecs module. Faster, more efficient image I/O. Improved support for modern image formats and metadata. Robust and complete JPEG XL implementation with animation.

Mentors: Student: Shaurya Kishore Panwar

This project presents a methodological approach to designing, implementing, and documenting a SLAM framework from scratch in Python. The primary objective is to build a fully functional, OpenCV-based SLAM system with clear documentation to facilitate reproducibility, future extensions, and adoption by the broader computer vision community. Traditional SLAM systems that rely on hand-crafted features (e.g., ORB, SIFT) often exhibit reduced robustness under challenging conditions such as viewpoint shifts, illumination changes, or motion blur. To address these limitations, we integrate modern learned features—ALIKED keypoints and descriptors (Zhao et al., 2022) combined with the LightGlue matcher (Lindenberger et al., 2023)—into a streamlined SLAM pipeline. This integration improves tracking stability and relocalization performance, especially in environments with significant photometric and geometric variations.