
AI for the Arts and the Humanities
Showing 5 of 38 projects. Click any project card for scope, mentors, and proposal studio.
Mentors: Student: Vennela Varshini Anasoori
Studying team communication in simulated human-factors environments currently relies on manual annotation of hours of raw audio, a process that is time-consuming and subject to recall bias. This project builds TCAMP, a modular open-source pipeline that takes raw headset audio as input and outputs structured behavioral metrics grounded in validated Human Factors research. By combining speech enhancement, speaker diarization, transcription, and communication analytics, TCAMP gives research labs an automated, objective instrument for studying team coordination and performance in simulated environments.
Mentors: Student: vybhav72954
This project builds a modular audio pipeline for tabletop RPG session analysis within the HumanAI TableTalk framework. It extends mood detection with speaker diarization, narrative role classification, and semantic retrieval over recorded sessions. Deliverables include a Whisper-based transcription layer, an SVM/ensemble classifier for narrative voice tagging, a FAISS-backed semantic retrieval index, and a documented evaluation suite; enabling players to query sessions by emotional arc or character voice.
Mentors: Student: Runchu Wu
As AI assistants adopt humanlike names, conversational tone, and confidence framing, users form impressions of AI competence from interface design alone — independently of actual system capability. This can produce overtrust, underuse, or miscalibrated reliance. Most existing research measures trust through self-report scales, which cannot capture when or how quickly users actually defer to or override an AI recommendation. This project builds a modular, open-source web-based experimentation platform that makes behavioral trust measures tractable for researchers. Built in Next.js, the system supports randomized condition assignment, configurable manipulation of humanlike interface cues (agent name, tone, confidence framing), and a recommendation-acceptance task that produces clear reliance-versus-override outcomes under controlled AI accuracy. Fine-grained event logs — decisions, timestamps, response latency — are exported as clean JSON and CSV datasets, and a reproducible Python notebook reports reliance rate, override rate, and latency by condition. The platform is designed as reusable research infrastructure: cue logic is configuration-driven, task logic is decoupled from condition management, and the event schema is fully documented so future researchers can extend the system with new cues, tasks, and study conditions. Deliverables include a functional experimental prototype, cue manipulation framework, behavioral logging backend, sample dataset, analysis notebook, and complete documentation in a public GitHub repository.
Mentors: Student: Aniket Junghare
This project builds an end-to-end Handwritten Text Recognition (HTR) pipeline for early modern Spanish manuscripts by placing a Vision-Language Model (VLM) at the center of every processing stage, not just as a late-stage corrector. The pipeline uses Qwen2.5-VL with LoRA-based multi-task fine-tuning across four stages: document analysis, literal reading, line-level OCR via a fine-tuned TrOCR model, and reconciliation/correction, where the VLM reconciles multiple reading sources using visual and contextual evidence. As a pre-GSoC contribution, I have built and validated this pipeline, achieving a median CER of 0.107 (~89.3% character accuracy) on the Rodrigo dataset with perfect transcription on well-preserved pages. The proposed GSoC work will transform this into a production-ready, multi-model application by abstracting the VLM backend to support multiple open-source models (Qwen2.5-VL 3B/7B/72B, InternVL), adding batch processing for multi-page manuscripts with PDF support, building a locally-run desktop application with a Gradio interface, packaging the pipeline as a pip-installable CLI tool, conducting comprehensive benchmarks and ablation studies across all model backends, releasing fine-tuned model weights on HuggingFace Hub, and providing a training pipeline for researchers to fine-tune on their own manuscript collections. The fully local execution model ensures data privacy for sensitive archival material, while the multi-model architecture allows users to select models based on their hardware constraints and accuracy requirements.
Mentors: Student: Atif_khan
Fragmented cultural heritage artifacts — such as Mayan stelae displaced from their original sites — exist today as scattered, incomplete pieces that are nearly impossible to physically reassemble. This project builds an AI-driven virtual reconstruction pipeline for Stela #43 from the Naranjo archaeological site, operating on 3D-scanned .PLY fragment data. The system combines a Point Transformer V2 backbone for geometric feature learning, a Siamese network for fracture surface matching, a Graph Transformer for globally consistent assembly reasoning, and a Trimmed ICP alignment module with pose graph optimization. A synthetic fracture dataset with realistic surface degradation simulation addresses the limited availability of real labeled data. Every learned component has a geometric fallback, and uncertainty quantification ensures low-confidence matches are flagged for human expert review rather than committed automatically. Deliverables include: a modular PyTorch reconstruction pipeline, a reusable synthetic fracture data generation tool, trained model checkpoints, a robust coarse-to-fine alignment module, a global pose optimization module, a full quantitative evaluation report comparing results against the existing geometric baseline, and complete technical documentation. The system is designed to augment archaeologist judgment — not replace it — by pre-screening thousands of potential fragment matches and providing interpretable, auditable match explanations.