VoiceScribe: Elevating Transcriptions with AssemblyAI's Universal-2 Model

This is a submission for the AssemblyAI Challenge : Sophisticated Speech-to-Text.

What I Built

I built VoiceScribe, a cutting-edge speech-to-text application that leverages AssemblyAI's Universal-2 model to deliver precise, formatted, and highly contextual transcriptions. VoiceScribe is tailored to professionals and industries that require high-quality transcription with advanced features like proper noun recognition, timestamping, and seamless formatting.

Key features include:

High-Accuracy Transcriptions: Powered by Universal-2, VoiceScribe excels at converting complex audio into detailed and accurate text.

Automatic Formatting: Recognizes sentence structures, proper nouns, and numbers, ensuring polished results.

Timestamps for Context: Adds timestamps for every spoken segment to improve usability in meetings, interviews, and video editing workflows.

Searchable Archives: Enables keyword-based search within transcriptions for efficient information retrieval.

Multiple File Formats: Supports various audio formats, ensuring compatibility with diverse use cases.

Demo

https://sb1xsz849-zeh3--5173--d3acb9e1.local-credentialless.webcontainer.io/

Journey

To build this application, I integrated AssemblyAI’s Universal-2 Model API with a front-end built using React and a back-end using Node.js. Here’s how AssemblyAI enhanced my application:

Real-Time Speech Processing:

The Universal-2 model was critical for converting diverse audio inputs into text with high fidelity, even in noisy environments or with heavy accents.

Additional Prompts:

Summarization API: AssemblyAI’s summarization feature allowed me to generate concise outputs for meetings, interviews, and podcasts.

Topic Detection API: Incorporated to categorize audio into predefined topics, enhancing user experience for searching and organizing content.

Accessibility Features:

Leveraging AssemblyAI’s capabilities, I ensured that the app supports subtitles and closed captions for hearing-impaired users.

Challenges Faced

Noisy Audio: Mitigated transcription errors from background noise by preprocessing audio files.

Large File Sizes: Optimized file handling and API batching for long recordings.

Formatting Variability: Tuned the API integration to consistently produce human-readable text with minimal post-editing.

Future Plans

Adding real-time transcription for live events.

Integrating multilingual support for international users.

Developing mobile apps for transcription on the go.

This was a solo submission, but feedback and testing were conducted with [Teammate DEV Usernames, if any].

Thank you for the opportunity to participate in the AssemblyAI Challenge!