Explore Projects

The goal for this project is to develop a NeRF model training pipeline within the OpenCV library. The end result will be a fully functioning data processing pipeline that is well integrated with the NeRF model training procedure, as well as online documentation that detail how to use the pipelines properly. There are 3 key deliverables for this project. First, implement data loaders that can process user-taken images of real-world scenes. This will involve using the COLMAP and OpenCV data processing pipelines in order to extract the camera poses and epipolar geometry of the scene; these are requirements for the model's ability to learn a meaningful representation of the given scene. Second, build, train, and test the actual NeRF model. This will involve using PyTorch's built-in CUDA support and extensive usage of the PyTorch library as a whole in order to incorporate critical components of the NeRF architecture such as positional encoding, ray sampling, and regularization into our model design. This step will also include rendering procedures that will allow users to extract images/videos, depth maps, and other logging information such as the training loss curve from the fully trained model. Third, create simple and intuitive documentation on how to use both the data loaders and the NeRF code base. This will involve creating a page on OpenCV's documentation (via Doxygen) that will list out specific terminal commands that the user should use to train the model. It will also include a summary of required packages, recommended environment configurations, and links to other helpful resources such as YouTube video demonstrations, which will all be geared towards increasing the usability and accessibility of the pipeline.

This project aims to enable the development of models that are suitably efficient for use in real-time interactive applications on embedded systems such as the BeagleBone-based Bela. At the project's core is difflogic, a recent technique that employs sparsely-connected network composed of basic logic gates to obtain small models and fast inference. The core goal of the project is to enable a convenient workflow for developing difflogic models and running them on the Bela, with an expected workflow of developing and training models on a larger machine, followed by exporting the model to C and cross-compiling it for the BeagleBone - either the main CPU or the PRUs. To support this, I will develop wrappers for exporting compiled difflogic models for use in the languages supported on Bela (C++, Pure Data, SuperCollider, Csound). I will then build out integrations and applications of difflogic for the rapid development of useful audio models, exploring the possibilities for combining difflogic networks with DDSP (differentiable digital signal processing) and cheap/lightweight techniques and toolkits, "porting" well-known ML architectures to difflogic networks, and training difflogic networks to approximate the behavior of existing neural networks (knowledge distillation). I will develop example applications involving sound analysis, classification, and synthesis, and experiment with interactive machine learning. Finally, I will apply these techniques to a creative application inspired by work on bytebeat and sound matching: generating networks of logic gates to approximate sounds and exploring the space of such sound-generating networks. I will train difflogic gates to reproduce particular sounds, treating the entire network as a bytebeat-style audio-generating function of time. Lastly, I will build an interactive application to explore the space of sound-generating networks (e.g. by mutating or morphing networks) and visualize the execution of logic gate networks.

Protein-ligand interactions play a crucial role in various biological processes, and accurately predicting protein binding sites, where ligand molecules interact, is essential for drug discovery and understanding protein function. Current approaches integrated in DeepChem, such as the use of (geometric) deep learning or molecules fingerprints, mainly focus on predicting protein-ligand binding affinity, and require complex data and/or are computationally expensive. This project focuses integrating ESM-2 (a state-of-the-art protein language model from Hugging Face) with DeepChem, for the purpose of protein binding site prediction. This integration will enable DeepChem library to utilize powerful protein representations learned by ESM-2. A detailed tutorial will be written to provide not only a workflow for streamlined binding site prediction, but a guide that empowers researchers to use protein language models for further tasks within DeepChem. This project broadens DeepChem's toolbox, making it more versatile for the drug discovery community, and ultimately accelerating progress in this area. The main deliverables are: A function for feature extraction using the pre-trained ESM-2 model. Integration of extracted features with binding site information. DeepChem model architecture for protein binding site prediction. Reports for model performance and evaluation metrics. Tutorial documenting the complete workflow, including code, data pre-processing steps, model training details, and visualizations of predicted binding sites (if possible). If time allows, an additional tutorial showcasing the application of ESM-2 protein representations for a different task (e.g., generating peptide binders for target proteins), or implementation of a LoRA wrapper for the ESM-2 model, along with a corresponding tutorial explaining its functionality will be developed. Alternatively, both will be documented as open issues on the DeepChem GitHub repository for future development.

One of Fossology's primary features is extracting licenses and license text from files. Traditional methods like text comparison, regular expressions, and SPDX identifiers can result in false positives, often requiring human review. This project aims to leverage recent advancements in Large Language Models (LLMs). LLMs can process text at a near-human level or beyond. In the domain of copyrights and licenses, models like Gemini-Pro, ChatGPT-3.5, and GPT4 have demonstrated exceptional accuracy in recognizing and converting licenses into formats like SPDX. Research across various fields, including medicine and coding, indicates that smaller, domain specific LLMs can outperform larger, general-purpose models in their respective areas. For this project, we'll fine-tune a relatively small LLM with parameters in the 2-7 billion range for optimal performance in license identification and other applications. Potential models include gemma (2B or 7B), Mistral (7B variant), LLaMA-2 (7B), phi-2 (2.7B) and others. This project involves selecting the best-suited model, utilizing existing datasets for fine-tuning, experimenting with different fine-tuning techniques based on data and computational resources, and finally, deploying the refined model.

<p>For sparse learning problems, such as sparse generalized linear models and sparse undirected graphical model estimation, the current R packages still have a lot of room for improvement in terms of speed and robustness, especially in high dimensional settings or when there’s multi-colinearity among features in the data. We aim to build an R package that can achieve state-of-the-art performance on three model classes (1) sparse generalized linear model estimation, (2) sparse nonparametric generalized linear model estimation and (3) sparse undirected graphical estimation under a novel active-set based second-order optimization algorithmic framework. Statistical inference, such as post-regularization p-value computation, will also be implemented as an important feature in our package.</p>

While Meshery's AWS model catalog contains auto-generated connections for individual services, it currently lacks the critical "cross-model" architectural relationships (like linking an EKS Cluster to EC2 Subnets, or an API Gateway to a Lambda Function) that Site Reliability Engineers need to properly visualize and manage real-world, multi-service cloud deployments on Kanvas. Solution: This project systematically defines these missing inter-model dependencies by combining deep AWS architectural expertise with Meshery's v1alpha3 relationship schema. This involves mapping complex multi-hop infrastructures and accurately structuring evaluation queries, metadata, and mutatorRef/mutatedRef patch paths. Deliverables: 1. A comprehensive audit of cross-model gaps in the current AWS catalog. Implementation of high-priority infrastructural relationships (Networking, IAM Permissions, Firewalls). 2. Definition of complex multi-hop chains (e.g., end-to-end Serverless Event Pipelines, EKS Full Stack dependencies). Visual validation of all new relationships within Kanvas and detailed contributor documentation to guide future model integrations. 3. Definition of complex multi-hop chains (e.g., end-to-end Serverless Event Pipelines, EKS Full Stack dependencies). 4. Visual validation of all new relationships within Kanvas and detailed contributor documentation to guide future model integrations.

This project aims to create a robust, high-performance parser combined with an easy-to-use macro system for the Teal programming language. Leveraging a bicameral macro model, the project clearly distinguishes the parsing and macro expansion phases whilst still keeping the syntax streamlined. This enables flexible, powerful compile-time code transformations while maintaining clarity and efficiency in compiler design and performance.

DetectionMetrics is a toolkit designed to unify and streamline the evaluation of perception models across different frameworks and datasets. The older version i.e DetectionMetrics v1 provides objective performance metrics such as mean average precision and mean inference time for object detection models. Now DetectionMetrics v2 was introduced in which the previous version was redesigned with an expanded focus on image and LiDAR segmentation. Currently, DetectionMetrics v2 functions as both a Python library and CLI, focusing on the quantitative evaluation of image and LiDAR segmentation models, with plans to expand into object detection. It supports PyTorch and TensorFlow models, along with multiple public datasets. Its modular design allows easy integration of new models and datasets. This project aims to bring back the key functionalities from v1 that were lost while redesigning, build a GUI and set up a CI workflow.

<p>TARDIS is an open-source code for rapid spectral modelling of supernovae using Monte Carlo methods. To run the simulation model, TARDIS requires some information such as velocity of shell structure, density, abundances, etc. These input parameters determine the composition of supernovae that the user wishes to explore.</p> <p>At present, the model composition is initialized by a configuration YAML file or a CSVY format file and it is impractical to adjust the composition without the two files. Therefore, users have to edit text files manually in order to change the model composition, which tends to be inefficient and inconvenient for them. In this case, a custom abundance widget is proposed to provide a more convenient way in supernovae exploration. The project aims on abundance customization and allows the user to develop custom model compositions graphically, which can then also be output to a file.</p>

The MBDyn Python Preprocessor aims to streamline the creation of input files for MBDyn simulations by bridging Python scripting with MBDyn's input format. Currently lacking support for all MBDyn entities, the preprocessor needs extension to cover all available entities. Additionally, integrating the preprocessor with Blendyn, a Blender add-on for post-processing MBDyn output, will provide a GUI based preprocessor for generating MBDyn input files. The project entails incremental development, starting with familiarization and planning, extending the entity library, managing model behavior, creating a model container, integrating with Blendyn, and finally, incorporating methods for aeroelastic models. These milestones culminate in deliverables that enable users to build entire MBDyn models using Python, including support for rotary-wing aircraft models, while ensuring completeness through thorough review and documentation.

Exoplanet Atmosphere Characterisation plays a vital role in understanding chemical compositions, weather patterns and habitability of the exoplanet. Traditional retrieval models take hours per planet so using machine learning for this is a perfect alternative. I have simulated a physically realistic spectral dataset of atmospheres under regimes like equilibrium transmission, disequilibrium transmission, non-isothermal emission, and photochemical emission with instrument specific noise injection. Then two models: disequilibrium-aware GRU model and Variational Autoencoder for Pressure Temperature Profiles are used to predict various atmospheric parameters. Additionally, to detect biosignatures we can use an unsupervised convolutional Autoencoder trained on normal equilibrium spectra to flag anomalous molecular signatures. Finally, I plan on validating all my models on real JWST observational data proving that these models are worthy of real world applications.

Casanovo is a transformer-based de novo peptide sequencer that is too slow for real-time instrument use. I will profile the new non-autoregressive pipeline, apply CUDA graphs, mixed precision, and FlashAttention-2 to reduce per-spectrum latency, then train lightweight student models via knowledge distillation to reach the 20-100 Hz real-time target required by modern mass spectrometers.

<p>My project aims to define a new similarity measure metric based on social coding semantics underlying the open source trace data to enrich the ability of Augur. The heterogeneous information network schema and network embedding techniques are introduced to capture the latent similarity information between repositories. This project will end up with some new computational models to transform those information into computable representation vectors with respect to every repository.</p>

<p>Among all the human writing communication systems and inspired by Google Arts &amp; Culture project Fabricius, we propose the creation of a framework to identify glyphs in Aztec/Central Mexican codex and classifying their Graphical Communication System (GCS) using Convolutional Neural Networks (CNN) Keras. The goal is to create a model with transformed glyphs in RGB-matrix samples to successfully train Deep Learning algorithms as a useful too for Aztec scholars.</p>

<p>The idea of this project is to make a software in which a user can make deep learning models in an easy way using a graphical user interface with backend supported by tensorflow. Through the graphical user interface, a user will able to add, delete, edit deep learning layers in a model. The main purpose of the project is to make the implementation of deep learning models quick and easy.</p> <p>The software will be built using electron-js which is a framework for building cross-platform desktop apps with HTML, CSS, and JavaScript. It will have a drag-and-drop feature to build deep learning models in the form of a graph which will then converted to a python code by the software. The generated code will then be executed in the child process which trains the deep learning model and sends the metrics data( loss, accuracy ) to the parent process which then plots the statistics.</p>

I will implement the yolov3-tiny model using mlpack. I will get Darknet-19 working, flesh out the API, build and train yolov3-tiny, publish the weights, and then test the model on constrained hardware, like the Raspberry PI 4, as well as testing it on my local GPU. I will then publish documentation on the implemented model, so that anybody can train and use the model, or download the pre-trained weights.

Building the JSON Schema Tooling View to improve both its design and functionality. The project involved a full transformation, making the view more dynamic and data-driven. Enhancements include advanced filtering options, better navigation, and a new data model. Key features now include integration with Bowtie for benchmarking, and tracking mechanisms to gather insights on user interactions and trends. This redesign aims to improve user experience, encourage wider adoption, and provide actionable metrics while respecting user privacy.

<p>Chrome OS enables fast and relatively low energy-consuming by utilizing TensorFlow Lite and hardware ML accelerators. The hardware accelerators are great and give good performance in terms of model inference. However, not all operators will be implemented by hardware. And in this case, we need to fallback to a generic solution (CPU backend) to guarantee the model will get run successfully.</p> <p>The key is to implement a high-performance and stable CPU backend and XNNPACK gives us the opportunity to do so by simply porting its implementation to Chrome OS codebase and align the API and internal calls with our NNAPI.</p> <p>By porting the most commonly used ops to Chrome OS in XNNPACK, we can run simple models like MobileNet more efficiently than the existing reference operation implementations. with little effort. This would be a big plus to the model and model inference in Chrome OS eco-system.</p>

The project aims to implement different GAN architectures ranging from classical models to fully quantum models, including hybrid models as well. And compare their performance with the goal of proving the quantum advantage. We work with different datasets that belong to High energy physics to prove the practicality of these quantum models in the NISQ era. We will explore different encoding techniques, Loss functions, Gradient algorithms and network evaluation methods.

The first genome-scale models of metabolism appeared in 1999 and 2000. In the following years fundamental Microbial Systems concepts were developed such as in silico methods and algorithms to analyze their properties. Genome-scale Metabolic models (GEMs) built on genome-scale metabolic network reconstructions (GENREs) are capable of deciphering the complex interactions occurring in bacteria, through predictions of the possible mechanism underlying these interactions. dingo is a Python package that supports a variety of methods to sample from the flux space of metabolic models, based on state-of-the-art random walks and rounding methods. Aim of this project is to enhance dingo by incorporating a preprocessing class to reduce and simplify metabolic networks, by removing certain reactions and metabolites. To identify correlated reactions, i.e. the flux of one depends on the others, I will develop a post-processing function to calculate correlation metrics and compare pairwise reaction fluxes. Additionally, another function will create graphs from the correlation matrix aiming to predict metabolic pathways. In the second half of the project, I will implement approaches for building microbial communities models. Two main approaches are to be implemented: Bag of Genomes and Bag of Genes. I will develop : a new class (“dingo.Community()”) that will read and store a community model of two microbes based on the user’s approach of choice. Its output will be then used as input to the dingo.PolytopeSampler to proceed with sampling steps. Sampling in the community flux space with the different sampling methods already supported by dingo and benchmarking of the community modeling approaches against real data will be delivered. Refactoring current dingo functions to adjust the new functionality will be done too.

<p>The project plan represents a very significant step forward for the factorAnalytics package by adding advanced methods to the fundamental factor models (FFM's) and to the time series factor models (TSFM's) functionality of the factorAnalytics package developed in the GSoC 2016 project. Furthermore the project includes the addition of Risk Budgeting (RMRB) and Monitoring Active Managers based on work of Mentor Dr. Tom Philips (see references), with his guidance.</p> <p>The overarching of the FFM development part of this plan is to replicate a large proportion of the non-proprietary models and model fitting and analysis methodology that is contained in commercial portfolio construction and risk management products such as MSCI Barra, Axioma, Northfield, etc. As such factorAnalytics will provide a platform for empirical research on the use fundamental factor models for improved portfolio performance and risk management, as well as providing a good stepping stone to the use of such a commercial product.</p>

The aim of this project is to enhance the Beam Yaml API by introducing ML and IO transforms to provide the users with more functionalities. The suggested set of transforms to be implemented are: 1. RunInference 2. ReadFromSpanner 3. WriteToSpanner 4. Enrichment Transforms I also want to add three use cases for the Yaml API, providing end-to-end pipelines demonstrating the use of the newly implemented transforms. The suggested use cases are: 1. Text processing with MLTransform and RunInference 2. Processing tabular data from Spanner 3. Enriching tabular customer data with Enrichment Transform. By expanding the capabilities of the Yaml API, the goal is to streamline the process of constructing and managing data pipelines using Apache Beam. This improvement will enable users to tackle a wider array of data processing tasks with greater ease and efficiency, potentially attracting a broader audience to leverage Apache Beam for their data processing needs.

Sometimes, in the context of quantum chemistry, the quantum state of a system in a crystal or molecule can be well described only by its occupation and/or spin operators. The goal of Model Hamiltonians, one of the APIs of the QC-Devs, is to create a framework for constructing the models and outputting them into a format that is possible for traditional Python packages solving Quantum Many-Body problems. In some cases, it is more convenient to think only about the occupation number of an atomic (or functional moiety), which constitutes an occupation-based Hamiltonian. The most general occupation-number Hamiltonian we consider is the generalized Pariser-Parr-Pople + pairing (PPP+P) Hamiltonian. For some other phenomena, it’s convenient to write the Hamiltonian only in function of the spin operators. This constitutes the Spin-based Hamiltonians and can be exemplified by the Heisenberg model. On the other hand, for some materials like superconductors, it’s necessary to combine both types of models at the same time, in a class known as occupation-and-spin-based Hamiltonians, where the most known is the t-J-U-V model, which can be expressed as a sum of the Heisenberg XXZ and the PPP hamiltonian. The package already includes implemented occupation-based and spin-based Hamiltonians. However, there is still a need for additional occupation- and spin-based Hamiltonians. The primary objective of this project is to implement occupation-and-spin-based Hamiltonians, which involves the following steps: 1. Establish a dictionary-based database to include standard constants defining various atoms and their connectivity types. 2. Develop an API that integrates the existing code with the occupation- and spin-based Hamiltonians. 3. Design and execute tests for the implemented API to ensure its reliability and functionality. 4. Develop utility functions to generate model Hamiltonians in formats compatible with external programs, such as FCIDump and Triqs.