TensorFlow

Mentors: Student: Seungjun Lee

This project aims to create a high-quality custom dataset using PaLM API and create a high-quality text summarization model by fine-tuning a pre-trained model with the custom dataset. The custom dataset will be a CSV file with 10K rows and 4 columns: id, prompt, document, and summary. The document should vary in different writing styles such as news articles, blog posts, legal papers and so on, and the length of a summary will be between 1/3 and 1/4 of the length of the original document. This will make the custom dataset to be far superior to most publicly available datasets like CNN-Daily-Mail or XSUM, which typically generate 1-2 sentence summaries. Then I will create a few text summarization models with different pre-trained models, improving each fine-tuned model’s accuracy by implementing strategy driven from bias/variance and error analysis. After creating a few fine-tuned models, I will pick the best performing one based on the evaluation metric. Based on this, I will work on creating a tutorial covering the whole process, such as creating a custom dataset using PaLM API, fine-tuning pre-trained models using Keras API, and selecting the best performing one based on a evaluation metric. As a result, by creating this comprehensive tutorial, the project will encourage other developers to explore/use Keras API and PaLM API in their projects.

Mentors: Student: Yassin Hashem

This project outlines the development of a Flutter plugin for MediaPipe, with a focus on a feature-first approach. The plan includes implementing the Text Task API, Audio Task API, and Vision Task API, which covers Hand Landmark Detection, Gesture Recognition, and Image Classification and Embedding. The project also involves testing and documenting each feature, as well as creating demo apps to showcase them. The project is estimated to take approximately 12 weeks, with each feature being developed and tested in a separate 1-3 week timeframe. The final week will be dedicated to finalizing documentation, creating code tutorials, and preparing the plugin for publication. The goal is to create a high-quality, well-documented plugin that is compatible with different platforms and easy to use for developers. The main goals for the finished result of the project are: - Multi-platform Flutter plugin for MediaPipe - The primary deliverable of the project is a Flutter plugin that is compatible with both iOS and Android. - Testing and documentation - Each feature must be thoroughly tested and documented to ensure that they work as intended and that developers can easily use them. - Demo apps - Creating demo apps to showcase each feature is an essential part of the project. - Interactive Codelab tutorials - While not required, the creation of interactive Codelab tutorials would be a valuable addition to the project, particularly for the documentation phase.

Mentors: Student: Mohammad Eid

The project aims to make a MediaPipe plugin for Flutter. The basic idea is to make an API for Flutter developers to use MediaPipe platform-specific frameworks without needing to worry about every platform's specific implementation. I aim to make this plugin robust and easily extensible so that we can add support for other platforms and other MediaPipe products later on. The main goal for the project this summer is to have Android support for the plugin along with a base plugin framework to build upon in the future.

Mentors: Student: Mishra, Anshuman

The objective of this project is to implement Token Classification tasks for the KerasNLP library. The focus will be on enhancing the tokenizers API to allow for greater flexibility with offset mapping use cases. The ultimate goal is to use the enhanced tokenizer for the implementation of a Token Classification Pipeline. This will involve creating separate preprocessing and task classes for all the models. Finally, the project will be supplemented with a Keras-io example demonstrating Token Classification using the KerasNLP library.

Mentors: Student: Pratyay Banerjee

The COVID-19 pandemic has increased awareness of hygiene risks associated with touchscreens, with reports indicating that 80% of people find them unhygienic. Touchless gesture-based intuitive systems can reduce transmission in public settings and workplaces, and offer a seamless and convenient experience. Touchless technology is expected to remain popular in various industries, such as retail, healthcare, and hospitality. In this proposal, I suggest developing an interactive web app using the Mediapipe Hands JS Solution API and simple human gestures to provide perfect contactless interactions with interfaces. The app will showcase an augmented transaction panel previewed on the screen, enabling users to perform essential CRUD operations of items through custom simple-to-use gestures, without physical touch. Both custom defined & pre-trained gestures from Mediapipe’s tasks-vision API will be utilized to classify gestures and trigger events on the interface. The project targets most platforms, primarily big screens, and may run on selective mobile devices with a camera module for input feed. All data taken via input video feed is deleted after returning inference and is computed directly on the client side, making it GDPR compliant. Once completed, the Web App will be uploaded to Codepen and/or deployed on Vercel.