Explore Projects

The objective of this project is to provide alias support in concerto language . For example import statement like "import {document as d} from Library" to be supported by the end of this project. This would allow us to import two models with same name in a single file by aliasing them to different names which is currently not supported in the language. Doing this would require changes in both the concerto-parser and concerto-runtime . The project requires compiler design and programming language knowledge. The stack used in project is NodeJs and written in typescript. The concerto-parse is written in PEGJS.

As machine learning models grow in size and complexity, there is a rising need for fast, efficient, and scalable ways to serve them in production environments. This project aims to build a fully functional pipeline that enables both training and inference using models optimized with OpenVINO and served using KServe. It will provide dynamic request parsing and serving through KServe. Real-time inference will be managed by the system, along with remote training using a friendly API. Deliverables are an efficient inferencing and training pipelines with pre/post-processing and production-ready deployment with OpenVINO being at the core.

The proposal aims to expand upon the advancements made in the 2023 project, which focused on introducing 8-bit model quantization to OpenCV-zoo. Building upon this foundation, I added Block-wise quantization APIs to further enhance the capabilities of the framework. The primary objective of this project is to develop a comprehensive command line tool within OpenCV-zoo and add inference support to blockwise quantization operations in OpenCV. This tool will facilitate block-wise quantization of ONNX models, offering users a robust and efficient method to optimize their models for deployment across various platforms.

This project aims to enhance Anomalib by integrating state-of-the-art (SOTA) anomaly detection models, including GLASS, GLAD, MuSc, for both one-class and multi-class classification. By incorporating these models, Anomalib will remainaligned with the latest advancements in anomaly detection, strengthening its benchmarking capabilities. Additionally, new benchmark datasets, such as BTAD (for images) and UBNormal (for videos), will be integrated to expand the framework’s applicability across industrial and medical domains. The project will also introduce advanced evaluation metrics, including mIoU-max and averaged mAD, to enable more comprehensive performance assessments. Comprehensive documentation, along with updated benchmarks and performance comparisons, will ensure ease of use and deployment readiness. These enhancements will establish Anomalib as a more robust and versatile framework for real-world anomaly detection applications.

This project integrates n8n, a no-code workflow automation platform, with OpenVINO Model Server to let users build AI-powered workflows that run on Intel GPUs and NPUs. The core is a custom n8n node package that connects to OVMS - users pick a model, choose a target device, and the node handles inference. The system is deployed through Podman Compose with one command. The primary workflow is a Smart Document Processing Pipeline: it watches a folder for incoming documents, runs text extraction on NPU, classifies and extracts entities using an LLM on GPU, and saves results locally. OpenVINO's AUTO plugin handles device selection automatically based on availability and load. Deliverables: custom n8n node package with GPU/NPU/AUTO device selection, Podman Compose deployment, the document processing pipeline, 2-3 additional workflow templates (RAG chatbot, multimodal content analysis), and documentation with installation guide and tutorials."

<p>Up until the recent COVID-19 pandemic the greatest social/medical crisis in the United States has been the opioid epidemic, taking 128 lives a day per the CDC. Two of the high-level BioGears objectives are lowering the barrier to create medical training content and training the military. I believe that a clinically relevant comprehensive model of the naloxone reversal of fentanyl-induced respiratory depression could serve both as a training tool for emergency medical personnel as well as a predictive tool for the dose and frequency of nasally administered naloxone. Current guidelines for nasal naloxone usage are ambiguous and dependent on a wait-and-see approach instead of a high-fidelity data-driven predictive model, which risks under-usage of the life-saving drug due to basic human error.</p>

This project addresses the inefficiency of full fine-tuning in classification tasks on small-scale datasets and low-resource environments within OpenVINO Training Extensions (OTX). To solve this, the project will introduce Parameter-Efficient Fine-Tuning (PEFT) techniques(specifically DoRA) and integrate lightweight Vision Transformer (ViT) backbones into the OTX framework. The solution will be validated through benchmarking on datasets of varying sizes to evaluate the trade-off between performance and training cost. Deliverables include the implementation of PEFT modules, integration of ViT-based backbones, and comprehensive benchmark results.

Stentor roeselii is a free-living ciliate species of the genus Stentor. It is a unicellular organism but shows remarkable hierarchy of decisions when faced with a stimulus leading us to question whether there is some level of cognition involved and whether the line between metabolism and cognition is fuzzy implying mitochondria may be the cognitive centre For the last 200 years research into human cognition and decision-making has revolved around rational choice theory, which assumes that actors are utility maximizers capable of searching and finding rational and optimal decisions. However human behavioral data doesn’t support the assumptions that we have infinite time nor infinite cognitive energy to search the full space of candidate choices to produce optimal decisions. Take for example a hiker, who has an infinite number of paths to choose from. It's irrational to assume that the hiker is capable of fully evaluating each of the infinite paths available to them to then choose the most rational/ optimal path. And yet that is the operating assumption for most modern cognitive models. To remedy this, we have the Wayfinding theory which along with an optimal-control based model can be used by researchers in private and academic settings to develop more accurate computational models of cognition in various life species

This project targets the expansion of OpenVINO's functionality within the Node.js ecosystem, the Node.js API which currently lacks comprehensive samples. The primary goal is to translate existing Python API samples to Node.js and to introduce new samples that harness the Node.js API, thus making OpenVINO's deep learning optimization capabilities more accessible to JavaScript developers. The project will entail a thorough analysis of Python samples to extract core functionalities like model loading, preprocessing, inference, and output processing, and replicate these using Node.js, adhering to its asynchronous nature and event-driven architecture. The endeavor will address gaps in the Node.js API through alternative solutions or the development of adding new APIs to the Node.js bindings . Deliverables include a suite of translated and new Node.js samples, and detailed documentation to aid developer usage, significantly enhancing OpenVINO’s utility across a broader range of development environments.

<p>Transformers are getting more and more attention in tasks outside of Natural Language Processing (NLP) and should be considered a general sequence to sequence model. Also in Computer Vision Transforms have shown to outperform or are on par with Convolutional Neural Networks (CNNs). Due to their increased practical importance and potential impact in the future, the goal of this project is to implement and reproduce the recent paper "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale" by Dosovitskiy et al. for the TensorFlow Model Garden.</p>

The current state-of-the-art tracking models are hampered by low speed, limiting their applicability on devices with limited computational power. Although Existing realtime object tracking models could reach high speeds on edge devices, their performance is poor. Consequently, the high-performance tracker with fast speed on edge devices is critical. I have two plans to solve this problem. The first Scheme is to use the pre-trained lightweight transformer as the tracking models' backbone and through the proper design, the tracking model not only has good performance because of the use of the pre-trained visual transformer but also because the visual transformer used is lightweight and does not slow down the model due to the huge amount of computation caused by the transformer. The second Scheme is to build a small tracking model and pre-train the small tracking model using the MAE pre-training method, and finally, choose the current best-performing transformer-based tracking model as the teacher model to train the student model. Eventually, two state-of-the-art real-time tracking models will be provided, along with their code, and they will be maintained over time.

The CICADA (Calorimeter Image Convolutional Anomaly Detection Algorithm) project aims to detect anomalous physics signatures without bias from theoretical models in proton–proton collisions at the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider. CICADA identifies anomalies in low-level calorimeter trigger data using a convolutional autoencoder, whose behavior is transferred to compact student models via knowledge distillation. Careful model design and quantization ensure sub-200 ns inference times on Field Programmable Gate Arrays (FPGAs). CICADA currently uses a distilled neural network model with layer-wise quantization to meet the strict sub-200ns latency required for the CMS Level-1 Trigger (L1T), trained using QKeras (a now-deprecated library for quantization-aware training). This GSoC project aims to enhance CICADA by integrating highly granular quantization (HGQ) techniques, which allow automatic, per-weight and per-bias quantization, providing finer control over model precision and resource usage. This project involves developing a HGQ version of the CICADA distilled model and evaluating its performance and resource consumption when deployed on a FPGA, while contributing to the open-source CICADA, HGQ2 and hls4ml frameworks as needed. Notably, the HGQ2 library is also compatible with JAX, opening the door for future integration with modern, hardware-friendly ML ecosystems. These contributions will benefit the broader fast machine learning community focused on deploying ML models in low-latency, resource-constrained environments.

1. Python scripts for IFC to OBJ conversion and text file generation: Objective: Develop Python scripts that take an input 3D IFC model and generate OBJ files and accompanying text files compatible with Radiance. Approach: - We can implement a Python script to parse and convert IFC geometry into OBJ format. - We can script to generate the necessary text files for Radiance, ensuring correct material and scene descriptions. 2. Settings for object filtering, object count limits, object geometry substitution, material mapping: Objective: Implement settings to control various aspects of the conversion process. Approach: - Create user-configurable settings in the Blender UI to handle object filtering based on criteria such as object type or properties. Allow users to set limits on the number of objects processed during the conversion. - Provide options for substituting complex geometry with simplified representations for efficient light simulation. - We can implement material mapping and substitution settings to ensure compatibility with Radiance. 3. Choose from preset sky settings and automatic materials: Objective: Enable users to select preset sky settings and automate material assignments for convenience. Approach: - Implement a dropdown or menu in the Blender UI to choose from a range of preset sky settings for light simulation. - Develop a script to automatically assign appropriate materials based on predefined parameters, simplifying the user's workflow.

OCR (Optical Character Recognition), a technology which targets recognizing text in images automatically, contains the following parts: document electronization, identity authentication, digital financial system. The project has two goals: quantizing the two API DBNet and CRNN-CTC in OpenCV, and designing a more lightweight version. I will replace the backbone of DBNet with GhostNet, which proved to be more lightweight than ShuffleNetV3. I have quantized the CRNN_VGG_BiLSTM_CTC model provided in the OpenCV Zoo, and I am now programming the backbone of CRNN with GhostNet. At the end, I will quantize the OCR models with ONNX quantization models.

<p>Integrating the existing Tellurium project with Apache Spark and by which the computations or simulations can be run in parallel. Developing an API for running different scenarios like parameter scanning, parameter fitting, sensitivity analysis etc.</p>

I am proposing to publish 4 Collections of models to TF-Hub containing a total of 23 models. Collections will be published from the following models- Multimodal Bottleneck Transformer (MBT), CoAtNet, Swin Transformer, and Video Swin Transformer. These models have been proposed recently and are getting lots of attention. Each collection will contain- TF2 implementation, pre-trained models, training scripts, templates for fine-tuning the pre-trained weights, and demos of downstream tasks using these models. The downstream tasks will cover action classification, audio classification, image classification, object detection, and semantic segmentation. The selected models have been built on transformers, they are efficient and their performance is SOTA or close to SOTA. Researchers will be benefited from these published Collections as the models are the latest and research is ongoing to further improve their performance. Developers will also be able to use these published models easily in their projects.

This project is about adding new AI models for LIT demos. The whole idea is to include different kinds of models as examples, so people from various backgrounds can easily see and understand model demos and learn to use LIT for their own custom models. I will be focusing on two models: a multilingual question answering model based on the TyDiQA dataset, and another model that generates images from text (the Dalle Mini model).

<p>DeepChem has enabled powerful and practical applications for machine learning in many disciplines of the natural sciences. While its name recognizes its historical origins as a tool for analyzing chemical molecules, its reach has more recently broadened to encompass neighboring fields such as materials science and biology. The goal of this proposal is to expand DeepChem’s burgeoning infrastructure to better support protein modeling.</p> <p>This proposal will add a simple codon featurizer to DeepChem, which will be the first DeepChem featurizer capable of directly processing RNAs and protein sequences.</p> <p>It will also add new model classes to DeepChem’s models directory. These classes will act as wrappers for models in Facebook’s Evolutionary Scale Modeling (ESM) repository. They will enable DeepChem users to use ESM’s Multiple Sequence Alignment transformer and Protein BERT models to perform efficient, scalable transformer learning.</p> <p>Also including carefully planned additions to DeepChem’s documentation, unit tests, tutorials, and the MoleculeNet benchmarking suite, this proposal will empower DeepChem users to perform large scale protein modeling while staying within our ecosystem.</p>

Problem: Symbolic regression is crucial for discovering underlying physical laws from data, but traditional methods are often computationally expensive or lack generalization. Current approaches struggle with efficiency and adaptability across diverse scientific datasets. Solution: This project aims to develop a foundation model for symbolic regression by integrating transformer-based architectures (e.g., SymbolicGPT) with evolutionary algorithms and learned concept libraries (LASR). The model will leverage large-scale pretraining and fine-tuning to generalize across mathematical and physical datasets, combining the strengths of language models and evolutionary search for interpretable, efficient equation discovery. Deliverables: A transformer-based symbolic regression model with mathematical priors and curriculum learning. Integration of a learned concept library for reusable abstractions. Benchmarks against SOTA models. Optimized, modular code for seamless LLM integration and open-source deployment.

For simulating a system that contains many variables and/or equations, traditionally computational engineers tend to focus on utilizing an appropriate method with realistic time and space complexity. The technique of model order reduction (MOR), however, aims at approximating the original model with reduced number of variables and/or equations and still keeping sufficient accuracy. MOR is especially useful in various industries, for example thermal-fluids engineering, micro-electro-mechanical systems and control, where large-scale simulations have to be performed. The methods of model order reduction can be classified into several classes including proper orthogonal decomposition, reduced basis, balanced truncation, etc. Several MOR libraries have been well developed in MATLAB and Python. For instance, pyMOR is a Python library that relies on the application of reduced basis methods to parameterized partial differential equations. MATLAB's model reducer supports pole-zero simplification, balanced truncation mode selection methods and so on. In Julia, the main package that enables symbolic modeling is ModelingToolkit.jl, whose key features consist of automatic transformation and structural simplification, but it needs further works on fast ML approximate transformations. So this project is targeted on implementing MOR methods as extended features for ModelingToolkit.jl. Besides implementation of methods of model order reduction, the expected deliverables will include documentation which shows both the improved simulation time and accuracy penalty for the application of different MOR methods on test problems from various disciplines.

The proposal titled "Event Classification With Masked Transformer Autoencoders" aims to enhance particle jet tagging by developing a Lorentz-equivariant Particle Transformer (Lorentz ParT) that adheres to the symmetries of special relativity. This hybrid architecture combines Lorentz Geometric Algebra Transformers (L-GATr) with Particle Transformers (ParT) to compute physical invariants, such as invariant mass, while tracking geometric orientations between particles. A key innovation is the introduction of Gated Particle Attention, which addresses residual bottlenecks in standard transformers by dynamically weighting contextual information against original features. The model is designed for a dual-head functionality, performing both self-supervised classification across ten unique jet classes and continuous mass regression using the JetClass dataset. To optimize learning, a Masked Autoencoder (MAE) approach is utilized to understand underlying jet symmetries before label-based training begins. Preliminary results indicate that the gated prototype can achieve a lower eta loss than original methodologies, although it currently faces challenges with "hallucinating noise" in reconstruction maps due to increased parameter counts. The proposed 12-week execution plan focuses on establishing mass regression, mitigating ROC curve issues, and validating the architecture against a scratch model to ensure stable performance

<p>ABioTrans is a bio-statistical/informatics tool, developed in R for gene expression analysis. The tool allows the user to directly read RNA-Seq data files deposited in the Gene Expression Omnibus or GEO database. It provides easy options for performing very commonly used statistical techniques, namely, Pearson and Spearman rank correlations, Principal Component Analysis (PCA), k-means and hierarchical clustering, Shannon entropy, Noise (square of the coefficient of variation), differential expression (DE) analysis, and gene ontology classifications</p> <p>This project aims to develop the 2nd version of ABioTrans. ABioTrans version-2 will be web-based and will provide extensive analysis options to the gene expression analysis results such as pathway enrichment analysis, gene function, t-SNE, Protein Protein Interaction, SOM , SVM as well as publication-ready visualization options including network visualization using Cytoscape.JS As the current version of Abiotrans is not online, it gets very difficult and time consuming process for the biologist who want to perform few analysis. As it's a very time consuming &amp; tedious process. So, taking it online will solve a huge problem.</p>

Move2Kube is a command-line tool for automating creation of Infrastructure as code (IaC) artifacts. It has inbuilt support for creating IaC artifacts for replatforming to Kubernetes/OpenShift. Move2Kube has a very plugin friendly architecture, users can write custom logic in the form of "Transformers" that Move2Kube can integrate seamlessly into its transformation pipeline. So far we have support for both Starlark and container image based transformers. We would like to support writing transformers as WASM modules that Move2Kube can run. WASM provides extensive sandboxing for security, it allows writing transformers in different language stacks like Rust, C/C++, etc. other than Golang, and WASM is just as lightweight and fast as Starlark. - Expected Outcome: - Implement a feature in Move2Kube CLI to allow running WASM modules as custom transformers.