SignBridge — The Bidirectional ASL Communication Bridge Built to Break Accessibility Barriers

This is a submission for the GitHub Finish-Up-A-Thon Challenge

What I Built

SignBridge is an educational, bidirectional communication bridge that breaks down the walls between the deaf/hard-of-hearing and the speaking world. Built with Next.js 15 and React 19, it enables flawless two-way communication:Type → Sign: Anyone can type English text or use speech-to-text, which instantly commands an on-screen skeleton avatar to animate American Sign Language (ASL).Sign → Text: A deaf user signs directly into their web camera, and browser-local machine learning translates the signs into live text and sound transcripts.I built this because I firmly believe that hard-of-hearing children should never be isolated or constrained to specific teachers just because of a communication gap. SignBridge democratizes accessibility, unlocking fluid conversations in classrooms, medical centers, and everyday life. I even bought a custom domain to transform this project into a permanent, production-grade tool.

Demo

Live Application: sign-bridge.app

GitHub Repository: https://github.com/Cjay-Cyber-2/SignBridge/

Loom video: https://www.loom.com/share/b02c4abe8e2d44269ae2ebd1ae67f576

The Comeback Story

SignBridge was originally born during an intense, 24-hour Google Hackathon. Driven purely by adrenaline, the prototype focused heavily on establishing the core visual concept. I managed to integrate the sign.mt cloud pipeline for text-to-avatar rendering, but the application was heavily unoptimized, lacked deep accessibility structures, and the codebase was fragile. Because of the strict 24-hour constraint, the reciprocal side translating raw sign language from a user's web camera back into speech was left completely unfinished. The repo sat on my GitHub profile collecting dust as an incomplete dream.

The GitHub Finish-Up-A-Thon gave me the exact push I needed to take SignBridge from a fragile proof-of-concept to a robust, dual-directional platform. During this challenge, I completed the hardest part of the equation:Implemented Sign → Text Recognition: I successfully integrated MediaPipe Hands directly into the browser. The application can now track hand landmarks with each letters and numbers in the ASL in real-time, matching gesture arrays against supported vocabulary to output instant live transcripts.Upgraded the Stack & Accessibility: I refactored the entire frontend to Next.js 15, React 19, and TypeScript, while incorporating speech-to-text APIs so speaking users can simply talk aloud to generate text that the avater could sign.Resolved Dependency Nightmares: Fixed breaking packages by decoupling the pose-viewer pipeline using legacy-peer-deps configurations, ensuring zero-latency client rendering.I also added access to different sign languages for different countries.

My Experience with GitHub Copilot

Reviving a rushed, 24-hour codebase is usually a developer's nightmare, but GitHub Copilot acted as an expert co-pilot that expedited my development loop:Complex Math & Landmark Mapping: Copilot was instrumental when writing the calculations required to translate raw MediaPipe hand coordinate matrices into predictable tokenized vocabulary array matches. It saved me hours of manual canvas mathematical modeling.Upgrading to React 19: When standard dependencies threw peer conflict warnings on the React 19 runtime, Copilot generated clean TypeScript wrappers that safely handled client-side rendering without breaking the hydration pipeline.Boilerplate Architecture: Copilot rapidly generated Tailwind layouts for the dual-mode split screen, allowing me to focus entirely on machine learning performance instead of styling forms.StackFramework: Next.js 15, React 19, TypeScript, Tailwind CSSAvatar Engine: pose-viewer (Skeleton playback powered by sign.mt cloud functions)Computer Vision: MediaPipe Hands (In-browser real-time word recognition)