Explore Projects

<p>The 52°North Triturus library, is extremely powerful and provides a lot of functionalities. This library is used in a lot of interesting projects such as terrainServer and worldviz. Currently, the terrainServer offers Web-based services providing perspective terrain views. The idea of this project is to extend Terrain model generation and analysis to practical scenarios, where it would be desirable to have Web-based access to analysis tools (e.g., difference surfaces, mass calculations, inclination analysis) and to model generation functions, using the Triturus library. In the Google Summer of Code program, we shall try to achieve Web-based access for few of these tools. This module will be named ‘terrainTools’ and will be a standalone project.</p>

<p>The statistical inference of properties of astronomical objects can be done using parametric modelling of Fourier products and light curves including common algorithms for time series. Introduction of current state-of-the-art methods for stochastic systems using automatic differentiation, variational inference can make the analysis of periodograms efficient and robust. In regard to (<a href="https://github.com/StingraySoftware/stingray/issues" target="_blank">https://github.com/StingraySoftware/stingray/issues</a>), the first goal of this project is to add automatic differentiation and modern optimization algorithms to increase the robustness of the existing infrastructure. The second goal is to extend the existing infrastructure to facilitate building fully probabilistic models using PyMC3/Tensorflow packages.</p>

<p>DBpedia, a crowd- and open-sourced community project extracting the content from Wikipedia, stores this information in a huge RDF graph. DBpedia Spotlight is a tool which delivers the DBpedia resources that are being mentioned in the document.</p> <p>Using DBpedia Spotlight to extract and disambiguate Named Entities from Wikipedia articles and then applying a topic modelling algorithm (e.g. LDA) with URIs of DBpedia resources as features would result in a model, which is capable of describing the documents with the proportions of the topics covering them. But because the topics are also represented by DBpedia URIs, this approach could result in a novel RDF hierarchy and ontology with insights for further analysis of the emerged subgraphs.</p> <p>The direct implication and first application scenario for this project would be utilizing the inference engine in DBpedia Spotlight, as an additional step after the document has been annotated and predicting its topic coverage.</p>

Gemma is a lightweight, open-source large language model by Google DeepMind. This project aims to build an intuitive web interface for fine-tuning Gemma models. The interface will allow users to upload datasets, configure hyperparameters, monitor training progress, and export trained models — all without writing a single line of code. By lowering the entry barrier, the UI will empower a broader range of users to experiment with and adapt large language models to their specific tasks.

This project builds an end-to-end Handwritten Text Recognition (HTR) pipeline for early modern Spanish manuscripts by placing a Vision-Language Model (VLM) at the center of every processing stage, not just as a late-stage corrector. The pipeline uses Qwen2.5-VL with LoRA-based multi-task fine-tuning across four stages: document analysis, literal reading, line-level OCR via a fine-tuned TrOCR model, and reconciliation/correction, where the VLM reconciles multiple reading sources using visual and contextual evidence. As a pre-GSoC contribution, I have built and validated this pipeline, achieving a median CER of 0.107 (~89.3% character accuracy) on the Rodrigo dataset with perfect transcription on well-preserved pages. The proposed GSoC work will transform this into a production-ready, multi-model application by abstracting the VLM backend to support multiple open-source models (Qwen2.5-VL 3B/7B/72B, InternVL), adding batch processing for multi-page manuscripts with PDF support, building a locally-run desktop application with a Gradio interface, packaging the pipeline as a pip-installable CLI tool, conducting comprehensive benchmarks and ablation studies across all model backends, releasing fine-tuned model weights on HuggingFace Hub, and providing a training pipeline for researchers to fine-tune on their own manuscript collections. The fully local execution model ensures data privacy for sensitive archival material, while the multi-model architecture allows users to select models based on their hardware constraints and accuracy requirements.

With WASI-NN plugins, WasmEdge is well-suited for running AI applications. However, AI applications are more than just the model. The application must pre-process data (such as images, audio and video) into TFLite / PyTorch formats, and convert the inference results back into application data in post-processing. Here are some examples: * The [mediapipe-rs](https://github.com/WasmEdge/mediapipe-rs) project provides a Rust SDK to build applications for the mediapipe AI models. * The [llama2.c](https://github.com/karpathy/llama2.c) application is [compiled to Wasm and runs in WasmEdge](https://medium.com/@michaelyuan_88928/running-llama2-c-in-wasmedge-15291795c470) to generate text using the [llama2](https://ai.meta.com/llama/) models. In this project, we would like to build a Rust SDK to support applications on the [YOLO models](https://pjreddie.com/darknet/yolo/). - Expected Outcome: - A Rust SDK that implements the pre-processing and post-processing functions required for the YOLO models. Those functions are implemented in OpenCV and Python in the official YOLO release. - Both image and video inputs should be supported. - Examples and documentation should be provided. - Recommended Skills: - OpenCV - Rust - Tensorflow / Pytorch - WebAssembly

<p>The goal of this project is to implement a package that will give the financial community tools to estimate, simulate, and test several MSGARCH models used in volatility (i.e., square root of conditional variance) forecasting. By relying on a hidden/latent variable, these models are able to switch among several processes for the conditional volatility and therefore, account for structural break in the volatility dynamics. MSGARCH have gained a huge interest in the financial risk management community over the recent years as they are better at forecasting volatility and provide more accurate risk measures. The package will follow the structure of rugarch since this is one of the most used packages for volatility modeling. The core will be implemented in C++ while simple R functions will facilitate usage of the package. Currently, no R package is available to estimate these models.</p>

LLMs rely on floating-point arithmetic for inference, but the Logarithmic Number System (LNS) offers a compelling alternative: multiplication becomes integer addition in the log domain, and 16-bit LNS (xlns16) uses only 2 bytes per element, half the memory bandwidth of float32. This project integrates the open-source xlnscpp C++ LNS library into ggml (the compute backend of llama.cpp) as a proper ggml_backend, enabling LLM inference to run entirely in xlns16 arithmetic as a proof of concept. The implementation introduces GGML_TYPE_LNS16 as a native ggml datatype and implements the 13 operations required for a complete transformer inference pass (matrix multiply, softmax, RMS norm, RoPE, SiLU, residual add, along with some others), all using xlns16 kernels. Activations stay in xlns16 format throughout inference; weights are converted dynamically from their stored quantised formats (Q4_K_M, etc.) per kernel invocation using O(1) lookup tables. The backend is validated at two levels: a unit test confirms sub-0.6% relative error on matrix multiply, and an end-to-end run on SmolLM2-135M-Instruct confirms the full transformer executes without crashing. Deliverables: (1) GGML_TYPE_LNS16 datatype registered in ggml; (2) a complete ggml_backend for xlns16 arithmetic covering all ops needed for LLM inference; (3) integration with llama.cpp; (4) a validation test suite; (5) an accuracy analysis comparing LNS and FP output across model layers

Strong gravitational lensing is a powerful probe of dark matter structure and cosmology, yet no foundation model exists that is purpose-built for lensing data with theoretical training guarantees. This proposal introduces WaveLens-JEPA, a self-supervised vision foundation model for gravitational lensing that extends LeJEPA (Balestriero & LeCun, 2025) to the astrophysical domain. The core idea is to decompose lensing images using a 2D Discrete Wavelet Transform before encoding, separating the physically meaningful frequency scales — Einstein ring morphology, subhalo perturbations, and ALP vortex structure — into orthogonal channels. A physics cross-attention module injects convergence-map priors from the lensing equation at every ViT block, while Einstein-Ring-Aware Masking (ERAM) targets the physically informative annular region during pre-training. Fourier phase regularization preserves substructure-discriminative spatial information that standard encoders discard. The training objective inherits provable collapse-free guarantees via Sketched Isotropic Gaussian Regularization (SIGReg), unlike all prior lensing SSL methods. The pre-trained backbone is evaluated on six downstream tasks: two substructure classification setups, lens finding, mass density regression, super-resolution, and a novel interpretability task via Wavelet Attribution Maps (WAM). Deliverables include: (1) open-source modular codebase, (2) pre-trained backbone weights on Hugging Face Hub, (3) a six-task evaluation benchmark, (4) a sim-to-real domain adaptation protocol targeting COSMOS HST data, and (5) a full research paper.

The DeepChem project aims to create high-quality, open-source tools to democratize the use of deep learning in drug discovery, materials science, quantum chemistry, and biology. Among its suite of various machine learning models, it has a wide range of graphs-based neural network model implementations aimed to solve the applications such as predicting the solubility of small drug-like molecules, binding affinity for small molecule to protein targets, analyzing protein structures, and extracting useful descriptors. This project seeks to bring a new tool to the DeepChem suite for solving message passing problems based on the recent advancements in GCNs research. This project aims to implement a Directed – Message Passing Neural Network (D-MPNN) model, a graph convolution network (GCN) built upon the existing Message Passing Neural Network (MPNN) model based on the base implementation in Chemprop.

Toolkit for Multivariate Analysis (TMVA) is a multi-purpose machine learning toolkit integrated into the ROOT scientific software framework, used in many particle physics data analysis and applications. Currently, we are developing a fast inference system in TMVA, called SOFIE, that takes takes ONNX model as input and produces compilation-ready standalone C++ scripts as output. These scripts will then provide users an easy way to deploy their deep learning models in their physics software and analysis frameworks. This project will focus on development of some missing deep learning operations which will allow to build more complex networks within TMVA for parsing the Transformer based models and Graph Net Models in SOFIE. Specifically, we propose to implement the inference functionality of some ONNX operators in the code generation format. The student can choose to build this based on existing implementations in TMVA or other existing machine learning software tools or build their own from scratch. The expected result is a working implementation of modular operators classes that implement the operators as defined by the ONNX standards in the code generation format. The project requires also to write the corresponding unit tests need to validate the written code.

<p>State-of-the-art Deep Learning models for the TensorFlow Model Garden implemented from the most recent research papers and trained on the datasets they are benchmarked on. My aim is to add some of the most exciting models to the Model Garden to help make model prototyping much faster and promote research using TF 2.x. The project proposes models such as FineGan and DETR along with other exciting theoretical models. The TF Model Garden makes building new models with state-of-the-art performance much easier especially with the new additions to TF 2.x that have made every part of the model training pipeline much more efficient. The models I have proposed to add will be trained till they converge to the benchmarks in the paper and will have detailed documentation and lucid Colab demos to make them extensible and easy to use.</p>

<p>In this proposed project we build JaxModel, which provides a high-level API for training and deploying Machine Learning Models using JAX framework. The main goal for JaxModel will be to provide an API for building, training, and testing deep learning models which are compatible with the existing DeepChem code base which already provides features like Molnet, Featurisers, Splitters, Transformers, and other deep learning infrastructures. During the period of GSOC, I will also be adding a notebook tutorial specifying the usage of JaxModel. If time permits, In the same tutorial I will also use an advanced JAX-based model preferably (Graph-based model or Physics Informed Neural Networks). This will be the beginning of implementing JAX-based models into the main DeepChem repo.</p>

This project addresses a key challenge in High Energy Physics: efficient and accurate end-to-end reconstruction of particle properties from detector data. In the CMS experiment at the Large Hadron Collider, particle collision events can be represented as high-resolution, multi-channel images. The goal is to simultaneously classify particle types and regress their invariant mass directly from these images. A major limitation of existing Vision Transformer (ViT) approaches is the quadratic computational complexity of softmax attention, which makes them inefficient for large-scale detector data. While linear attention mechanisms reduce this complexity, they often struggle to capture rich global interactions and multi-channel dependencies required for physics tasks. To address this, this project proposes the development of linear-attention-based Vision Transformers tailored for CMS E2E tasks, with a strong focus on extending the DepthViT architecture. The approach consists of three key components: Efficient Representation Learning A linear attention ViT with Masked Autoencoder (MAE) pretraining will be used to learn strong representations from large-scale unlabeled CMS data, improving downstream performance and convergence. Architecture Adaptation (DepthViT Extension) The DepthViT architecture will be adapted for supervised classification and regression by: Introducing dual-task heads for classification and mass regression Preserving depth-channel and rank structure in later layers to avoid information collapse Integrating Hierarchical Attention Permutable (HAP) blocks to enable efficient cross-window and cross-channel information flow while maintaining linear complexity Systematic Benchmarking The proposed models will be rigorously compared against standard and efficient transformer baselines (ViT, Swin, L²ViT, XCiT) using: Physics metrics (accuracy, MAE, R²) Efficiency metrics (FLOPs, memory, throughput)

The goal of this project is to build a model to recognize text in century-old documents with over 80% accuracy. To achieve this goal, I will carry out 2 sub-projects: improving the pre-existing model (SeqCLR [Aberdam, A., et al., 2021]) which I made as a test of this project, and building another model with different architecture and algorithms from the pre-existing model (PerSec [Hao Liu et al, 2022] or DiG [Mingkun Yang et al, 2022]). The first project, which is an improvement of the pre-existing model, contains verifying hypotheses about elements that hinder the model’s performance and finding solutions to those elements like the implementation of new layers or algorithms. The second project contains an investigation of a new model, implementing the model, and improving the model, which is like the first project’s procedure.

<p>This project aims to bring post-training integer quantization to OpenCV’s DNN module and perform inference using 8-bit integer inputs and fixed-point arithmetic. Using a simple quantize() function, any Net object can be quantized, resulting in roughly 1.5x faster inference, 4x reduction in memory consumption and accuracies close to floating-point inference.</p>

The Open Quartz Solar project is a PV forecasting model that leverages open data to generate forecasts for any location on Earth. Currently, the project has a single model and requires additional machine learning models to enhance its forecasting capabilities. In this GSoC project, my primary goal is to develop and train a more advanced forecasting model for PV power generation. This includes identifying and implementing models that are best suited for accurate energy output forecasting. Additionally, I aim to create a comprehensive forecast dataset that can be used to train various forecasting models, making it easier for organizations to add new models in the future.

<p>The goal of this project is to develop models and train them on various Radiology images. These models need to be made suitable to be run on low powered devices such as ARM devices and Android devices. For this, they will have to undergo quantization and pruning (during training as well as after training). A hybrid approach of these techniques will be experimented to generate substantial results.</p>

<p>SixTrack is a 6D particle tracking code used to compute the trajectories of individual relativistic charged particles in circular accelerators. It uses various physics models to apply the effect of different accelerator elements via mapping formulae. Sixtrack is missing a few models implementation which are required to achieve higher accuracy in particle tracking. This project will aim to implement and test these new models with the goal of adding them in the official code.</p>

<p>Colour is an open-source python package providing a comprehensive number of algorithms and datasets for colour-science. It also has support for different colour models, allowing easy conversion from one colourspace to another. A colour model is a way to define colour. It describes how a colour will appear on a computer screen or paper. I plan to work on improving this feature by implementing new colour models that are required and at the same time improving the existing ones.</p>

My project focuses on utilizing Diffusion Models for Gravitational Lensing Simulation, encompassing two primary tasks. The first task involves exploring diffusion models to generate strong gravitational lensing images, wherein I plan to investigate both DDPM and DDIM architectures. The second task consists of Creating a diverse dataset of simulated gravitational lensing images under various astrophysical conditions. I intend to implement conditional diffusion models and astronomical conditional gravitation lensing images with CLIP latent. The project's deliverables will include the code for all experiments, accessible through a GitHub repository.

PyTorch is becoming increasingly popular as the preferred backend for developing algorithms, thanks to its sustainability. DeepChem has decided to port its TensorFlow and Jax Models to PyTorch. This project aims to successfully port two of DeepChem's models. SeqToSeq and DTNN Model are being ported under this project. SeqToSeq is a neural network that maps variable-length input sequences to variable-length output sequences. DTNN stands for Deep Tensor Neural Network, which is a deep learning architecture that uses tensors to represent molecular structures for predicting chemical properties and activities. The implementation will include documentation changes, usage examples, and a tutorial if required.

<p><strong>Meaningful Adversarial Examples for Natural Language Models</strong></p> <p>A project to create adversarial examples and resulting counterfactuals for text classifiers using the relations in the word embedding vector space. This allows for meaningful alterations to be made in the input documents to test a model for biases. What would happen if the subject of this document was female instead of male? If it was a person of color instead of being white? How would a state of the art model change its results when such changes are made?</p> <p>This project means to address these questions by creating a framework that allows for the testing against such biases as well as the creation of augmented datasets to dissuade their development.</p>