Empowering lives through non-visual access to technology
Showing 2 of 2 projects. Click any project card for scope, mentors, and proposal studio.
Mentors: Student: Shubham Dilip Jain
<p>The internet today is rich in image-content, from entire websites like Instagram and Pinterest dedicated to curating and displaying images, to Facebook and Reddit that have large amounts of content in image form. Non-visual users find it challenging to navigate and use these websites for their intended purpose. The information in images, whether on the internet or stored locally, is also inaccessible to non-visual users. NVDA or NonVisual Desktop Access is a free, open-source, portable screen reader for Microsoft Windows. It already includes an OCR that can recognize text within images, however, it lacks any functionality to describe image content and allow users to interact with the various objects within images. This project aims to overcome these issues through two modules:</p> <ol> <li>A machine learning module to generate descriptive captions for images on the user’s screen or those manually inputted by the user.</li> <li>An object detection model that draws bounding boxes around recognized objects and outputs the object label when the user’s pointer enters a bounding box. The outputs of both these modules could be presented to the user using NVDA’s existing mechanisms such as voice or braille.</li> </ol>
Mentors: Student: Bill Dengler
<p>NVDA provides support for the Windows command console used by Command Prompt, PowerShell, and the Windows Subsystem for Linux. It allows users to read the console's contents using review cursor commands, and automatically reports console output as it arrives. While functional, this support has some shortcomings, leading to decreased NVDA performance and stability and, by extension, user productivity. This proposal seeks to take advantage of <a href="https://devblogs.microsoft.com/commandline/whats-new-in-windows-console-in-windows-10-fall-creators-update/" target="_blank">changes to the command console</a> introduced in the Windows 10 Fall Creators Update and refined in later releases, leading to an improved console experience for users on current Windows versions.</p>