Explore Projects

This project proposes ProtoDiff, a diffusion-based deep learning model designed to denoise astronomical observations of protoplanetary disks from telescopes like ALMA and VLT. Leveraging recent advances in score-based generative modeling, the model will be trained on synthetic disk simulations with realistic noise. ProtoDiff aims to outperform traditional denoising pipelines in both quality and efficiency, enabling cleaner reconstructions of planet-forming environments. The final deliverables will include open-source code, pretrained models, and documentation to support reproducible astrophysical data processing and accelerate research in planet formation studies.

<p>FLINT is a modular tool that supports better land management and helps reduce greenhouse gas (GHG) emissions worldwide, and allows multiple models and data required to better understand land management to be brought together in one place. This project idea is for implementation of new models to monitor land sector GHG emissions and support sustainable land management. There are already several existing models that have been attached, but many more are required to make the system complete. This project involves specifically implementing the agricultural soils model that are used to estimate non-CO2 GHG emissions from agricultural practices.</p>

<p>The purpose of this project is to build a converter that can translate models built in Tensorflow, Keras, Pytorch, Mxnet, ONNX to mlpack's model format and vice-versa.</p> <p>With the advent of extremely deep neural architectures and their high training cost, transfer learning is the only way out. That said, the number of pre-trained models for mlpack is practically zero till now while more popular frameworks like Tensorflow has dozens of them.</p> <p>Models trained in mlpack can be converted and used in Tensorflow for better benchmarking and feature-testing. Moreover, it can open up the field for mlpack and make it as popular as the other frameworks mentioned.</p>

<p>There has been a recent surge in the development of open-source computational methods for simulating human evolution and analyzing human genome data. These provide many new opportunities for genomic research, but the integration of these different resources is currently poor. In particular, turning models of human history into evolutionary models is notoriously time-consuming and bug-prone, and it requires knowledge of the specifics of each simulation tool. The major goals of this project are to develop a library of widely used historical models that integrate across multiple simulation tools, and to develop more robust and user-friendly model specification tools to automate the workflow of genomic and evolutionary analyses.</p>

Within this project, I aim to create a new ML model based on the already existing MLGO work in register allocation to further enhance the performance of the ML-enabled live range eviction heuristic. I plan on utilizing graph based features, particularly a program dependence graph and the live range incidence graph to improve performance over the currently engineered features. Starting off, graph data will be exposed to MlRegAllocEvictAdvisor (first deliverable), then an initial model will be trained in the same manner as the previous ML model for live range eviction (second deliverable), and finally the model will be fine tuned and optimized for use in production (third deliverable).

<p>This project proposes the development of a module that uses Machine Learning to classify correctly the audio-based relative data which is extracted from music tracks. Python’s related Machine Learning (ML) library, scikit-learn, is used for the training of the data and the predictions of the classification results.</p> <p>This work replaces/reproduces the existing ML classification problem procedure that follows the gaia library with a new ML model infrastructure that uses the Python library, scikit-learn. This new infrastructure is built in high-level modeling, it reproduces classification process with the SVM model, and can be easily extended with other Machine Learning and Deep learning models that can be then compared between each other or even combined.</p>

This project aims to develop a new deep learning exercise within Robotics-Academy, focusing on end-to-end visual control of autonomous vehicles. The exercise will allow users to upload trained PyTorch/TensorFlow models that process camera input from a simulated car or drone in Gazebo and output control vehicle movement based on the model’s predictions. A web interface will enable model upload, real-time visualization, and performance monitoring. The exercise will include a demo model trained on synthetic data, a Gazebo simulation environment, and integration with ROS2. This exercise will leverage existing infrastructure from previous deep learning-based Robotics-Academy exercises, such as Human Detection and Digit Classifier.

This proposal aims to revolutionize the City of Boston's 311 service request system by integrating AI-based image recognition. The current process, while functional, can be cumbersome and prone to misclassification, leading to inefficient allocation of city resources. By enabling residents to submit requests through a simple photo, analyzed and categorized by an AI model, the system becomes more accessible, efficient, and user-friendly. The project will involve building and training an AI model using historical 311 data, developing a high-performance API for the model, and ensuring seamless integration with web and app interfaces for a broad user base. The deliverables include a trained AI model, a robust API, and a prototype app interface ready for city-wide deployment after testing and optimization.

<p>The main goal of this project is to implement a mechanism to be in sync with the latest human data submitted to dbSNP. Once imported, this information can be distributed via EVA implementations of the GA4GH APIs htsget and Beacon specifications, as well as the EVA website.</p> <h6>Acceptance criteria</h6> <p>Given a dbSNP FTP directory with the human variant information, the pipeline should parse JSONs for each chromosome and write the variants from the JSONs to the EVA archive</p> <h6>Needed tasks</h6> <ul> <li>Construct an object model for dbSNP and parse the JSONs to objects in that model</li> <li>Convert the objects in the dbSNP model to the EVA variant object model</li> <li>Construct a variant object by including its component objects from above. Use the Variant writer of variation-commons to write the variants to the EVA archive</li> </ul>

<p>Implement Distribution-Aware Coordinate Representation for Human Pose Estimation(DARK Pose) using TensorFlow library . The main goal of this project is to reproduce the research paper results using TensorFlow deep learning library and publish the model in to the TensorFlow hub. The proposed project consists of new approach called DARK pose estimation which will enable mainstream human key point detection models to perform with better accuracy compared to normal heat map oriented key-point detection.The experimented models have architecture based on Hourglass, SimpleBaseline and HRNet with different configuration. After the implementation of the proposed models the results will be reproduces as mentioned in the paper. DARK Pose paper: <a href="https://arxiv.org/abs/1910.06278" target="_blank">https://arxiv.org/abs/1910.06278</a></p>

An extension component to be implemented in MIT App Inventor which enables user to use the model that they trained in Teachable Machine in App Inventor. Users can simply embed their Teachable Machine models, by the link provided to them after they have trained their model.

The SOFIE (System for Optimized Fast Inference Code Emit) project is an initiative within the TMVA (Toolkit for Multivariate Data Analysis) framework in ROOT, which aims to enhance the efficiency and speed of inference on Machine Learning Models. SOFIE converts ML models trained in different frameworks such as ONNX, PyTorch, and Tensorflow, converting them into an Intermediate Representation (IR). This IR allows SOFIE to generate optimized C++ functions for fast and effective inference of neural networks and subsequently convert them into C++ header files, which can be used in plug-and-go style for inference. For GSoC 2025, this project aims to enhance the Keras parser to support models trained in the latest TensorFlow v2.18.0, which introduces NumPy 2.0 compatibility. As well as Integrate JAX/FLAX support, enabling SOFIE to generate C++ inference functions for models developed using JAX/FLAX

This project focuses on developing and evaluating end-to-end Graph Neural Network (GNN) models for low-momentum tau identification in the context of particle physics experiments, such as the Compact Muon Solenoid (CMS) at CERN. The project aims to improve tau identification performance by leveraging GNNs' ability to capture complex relationships between jet constituents. Additionally, this project will test and benchmark GNN inference on Graphics Processing Units (GPUs) to assess the potential for real-time analysis and integration with existing systems. Key tasks include literature review, data preprocessing, graph construction, model selection, implementation, training, evaluation, optimization, GPU inference testing, benchmarking, and documentation. The expected outcomes of the project are a trained end-to-end GNN model tailored for low-momentum tau identification, a comprehensive evaluation and optimization of the model, and a thorough benchmark of GNN inference on GPUs.

A library management add-on for managing Different online-offline FreeCAD components is a utility application for FreeCAD which allows an easy and structured management of components/models available online or offline and to easily be inserted into the FreeCAD document. Its basic functionalities is to facilitate individual component from the online repository to be able to browsed, downloaded, and extended by already available local components. For a Library management system, a robust repository of data and an local storage structure is must. This is important as user need to brows through the all the existing models which are the part of the repository and download only selective models. A structured local storage will not only also allow users to easily add their own models to the local library management system, but also to the online repository open to public. For maintaining the metadata of these, a well defined storage structure is required. All this need to have an proper interface to be operable. The interface for the above will be developed as both a Graphical Interface and module for python scripting.

<p>Sustainable Computing Research Lab has done researches and project to identify elephants from images and tag them. LabelLab projects is a generalized extension of this. LabelLab focus on creating an mobile and web application to classify and label all kind of animal without being specific to elephant in order satisfy more use cases. LabelLab mobile app’s motivation is to allow user to use the classification model in field using their handheld device. The mobile app will have features take a image and classify it using the classification model and show the necessary information.</p> <p>Currently the classification model is already build. But both the mobile client and backend to integrated mobile client and classification model has to be built from scratch. This is a new project, So requirements of the project should be identified precisely and should be planned for future scalability of the project.</p> <p>I wish to implement the mobile client in Flutter for both Android and iOS and the backend in Node.js.</p>

This proposal aims to develop the Gemma LLM Garage, a full-stack interface to manage datasets and fine-tune Gemma models. Its main goal is to abstract the technical complexity from the end user, empowering anyone with a computer to fine-tune their models. So far, I have implemented basic features such as fine-tuning with LoRA, a mechanism for uploading datasets, simple data augmentation options, a live loss graph, and a simple inference platform to test the fine-tuned model. The main deliverables are implementing data cleaning strategies, improving the data augmentation options, implementing a testing workbench for the fine-tuned models, and supporting alternative fine-tuning methods (e.g. QLoRA). The project prototype is currently deployed through Google Cloud and can be accessed here: https://gemma-garage.web.app/ A quick video demo is available here: https://drive.google.com/file/d/1Knt9a16NfJTUX5rddte_SHcMDKCzA8NS/view

This project aims to enhance the analysis of multispectral paintings by detecting hidden images and understanding their structural characteristics. While previous GSoC efforts utilized CNN-based Siamese networks for layer similarity learning, such models often struggle to capture high-level semantic and structural relationships critical to interpreting overpainted or restored artworks. To address this limitation, this project proposes a Graph Neural Network (GNN)-based Siamese architecture that models both local features and their spatial/spectral relationships in graph form. The pipeline will begin with preprocessing multispectral data into region-based graphs, followed by training a GNN-Siamese model on the Multispectral Imaging (MSI) dataset using weak supervision. The model will then be generalized to unlabeled datasets (e.g., CAVE, Harvard) through few-shot adaptation, and further enhanced via self-supervised pretraining using large-scale RGB art datasets (e.g., WikiArt, NGA), with pseudo-label generation for semi-supervised learning.

<p>The main topic can be divided into two parts:</p> <p>+Body and hand detection: I apply the lightweight Openpose model, MediaPipe library, and the optical flow algorithm. The combination can process in real-time and with a lightweight model, so it is suitable for edge devices.</p> <p>+Gesture recognition: There are two main approaches for this problem: image-appearance-based models and pose-based models. These two are different in types of input data. The first approach's input is only images, while the second one's input is extracted poses from photos.</p> <p>+Extension: Currently, the robot can only learn the gesture from the predefined patterns, which appear in the training data. However, the training dataset is not always available for all motions. Therefore, by applying some unsupervised techniques, the robot can recognize some gestures without any supervised dataset.</p> <p>In this project: Four components will be published: BodyHandJointsDetector, ImageBaseGesture-Recognition, PoseBasedGestureRecognition, and UnsupervisedGestureRecognition. I will also code the testing client for each approach.</p>

<p>This project proposes the design and integration of a deep learning framework into Audacity, with the focus of developing an interface for end users to interact with state of the art (SOTA) audio source separation models. Although the vast majority of SOTA source separation algorithms are built using Python-based frameworks, PyTorch’s C++ torchscript API provides an elegant solution for deploying python models in C++ applications. In Audacity, this would introduce the opportunity for the user to choose from a collection of domain-specific pretrained models, each suited for a different task (e.g. speech or music). Additionally, it would set the foundational code for future integration of deep models designed for different tasks, such as speech recognition and audio classification. The goal of this work is not only to integrate source separation algorithms into Audacity, but more importantly to lay the groundwork for an extensible suite of open-source, deep learning-based music information retrieval (MIR) tools hosted by the world’s most popular free and open-source audio editor.</p>

<p>Hey, I have been in constant touch with Mehul regarding my project on Multi-modal Egocentric Perception. I have already had a skype meet with him before drafting this final pre-proposal.</p> <p>Abstract: The idea of the project is to introduce multimodality in recognizing everyday activities and scenes. As per today, no work has been done now which includes multimodality into account (especially audio ) to determine the kind of activities and scenes that person is involved when it comes to egocentric perception. I have already built an audio-based model based on popular IEEE-DCASE-challenge which can successfully classify scenes into categories like ( person is walking in a park, driving in a car ) for egocentric views. I plan to extend my work as part of gsoc-2k18 to incorporate my model with video-based models and increase the scope of model from scenes to scenes+activities. The final breakdown of the steps is submitted in the pre-proposal attached above. The idea is to built pyscene-detect for egocentric videos, which would be a prominent contribution owing to growing research in the area of first person view videos.</p>

<p>Measuring the similarity between two geometric models is an important problem in diverse fields, including computer graphics, computer games, and geometric modeling. For example, they can be used in terms of benchmarks to determine the variance of a processed geometry from a given ground-truth model.</p> <p>One such similarity measure is the Hausdorff distance. Given two compact subsets A, B of a metric space, the one-sided Hausdorff distance between A and B is defined as the distance between an element a from A and an element b from B, where a is chosen to maximize the distance and b is chosen to minimize it. Intuitively speaking, the Hausdorff distance measures the maximum deviation between two models.</p> <p>The project consists of implementing the algorithm for interactive Hausdorff distance computation presented by Tang, Lee, and Kim. While it provides an approximation to the Hausdorff distance - as already present in the CGAL framework - it ensures the computed measure to be within a user-specified error bound. Thereby, the user can obtain as good results as necessary for the applications at hand within a fast and efficient pipeline.</p>

<p>ArviZ is a Python package for exploratory analysis of Bayesian models, from diagnostics to visualization. It is designed as a backend-agnostic tool with the goal to reach the widest user base and thus contribute to extend best practices among Bayesian inference practitioners.</p> <p>Two key problems in this field are model comparison and convergence analysis. Model comparison is not trivial because of the different structures and number of parameters of each model. Fortunately, there are some information criteria (i.e. leave-one-out cross-validation) that can be used for this task. Even though convergence is proven for infinite iterations, it is not the case for finite MCMC runs, which can be arbitrarily bad. Convergence assessment must take into account both intra- and inter-chain correlations.</p> <p>ArviZ implements many of these algorithms for diagnostic and comparison, at least at a preliminary level, but it still lacks plots and tools to ease and improve its interpretation. This project seeks to design and implement these tools. Moreover, it will pay special attention to testing and documentation with examples not only of the new functionalities, but also of the already implemented ones.</p>

<p>The Android Graphics Tools Team's GraphicsFuzz is a metamorphic testing framework for OpenGL and Vulkan graphics drivers. Among its tools are a shader generator, which mutates a shader program into variants that differ in machine code, but render a similar image to the original - and a shader reducer, which reduces shader code to focus on specific logic that a user is interested in. Used in tandem, these tools allow one to fuzz test shader compilers for flaws and easily reproduce them.</p> <p>The generator mutates a shader by performing "transformations" on the original code which complicate control/data flow. These transformations often make use of OpenGL's built-in functions and features to mutate expressions into equivalent forms.</p> <p>However, the generator and reducer are only aware of a subset of the OpenGL shader language, and do not support all of OpenGL's built-in math and transformation functions. This reduces the number of potential mutations and ultimately hampers GraphicsFuzz's capability of finding edge cases.</p> <p>This project aims to enhance GraphicsFuzz's support for OpenGL semantics and extend its capabilities with new transformations, then apply them to open-source drivers.</p>