ClimateIQ AI – 11 Mux Features Power Climate Action for 2B+ Users

This is a submission for the DEV's Worldwide Show and Tell Challenge Presented by Mux

What I Built

ClimateIQ AI is a production-grade climate intelligence platform transforming satellite data into actionable insights for farmers, cities, and communities worldwide.

🌍 The Full Platform (Beyond Video)

ClimateIQ already has deep climate features:

Tech Stack: Next.js 14, TypeScript, Supabase, Google Gemini AI, Mapbox, NASA FIRMS, NREL PVWatts, Planet API, SoilGrids

📹 NEW: Community Hub (Powered by Mux)

The missing piece was community-generated content. Rural communities couldn't share drought videos. Cities couldn't broadcast awareness campaigns. Climate educators needed multilingual reach.

Mux solved this with 11 integrated features:

• 📹 Climate Stories – Upload video testimonials with auto-generated captions


• 🌍 Climate Live – Low-latency streaming for real-time expert consultations


• 🌐 AI Translation – Captions in Hindi, Spanish, French for global reach


• 🎙️ Audio Dubbing – ElevenLabs voice cloning + Mux multi-track audio

My Pitch Video

Demo

🔗 Live Demo: https://climate-iq-git-feature-omkardongres-projects.vercel.app/

📦 GitHub: https://github.com/omkardongre/climate-iq/tree/feature

Test Credentials:

• Email: [email protected]
  


• Password: 12345678
  

🔓 Without Login: View videos, watch with captions, switch dubbed audio tracks, watch live streams


🔐 With Login: Upload videos, generate chapters, translate captions, dub audio, go live

How to Test Mux Features (All from the UI)

1. 
   Upload Video: Go to Community Hub → Click "Share Your Story" button → Select a video → Watch it upload with progress bar


2. 
   AI Chapters: After upload completes, click on your video → Click "Generate AI Chapters" button → See chapters appear below the player


3. 
   Translate Captions: Click "Translate Captions" button → Select Hindi/Spanish from dropdown → After ~40 seconds, check the CC menu in the video player


4. 
   Dub Audio: Click "Dub Audio" button → Select language → Wait ~2 minutes → Check the Audio/Language menu in the player settings (gear icon)


5. 
   Live Streaming: Click "Go Live" button → Copy the RTMP URL and Stream Key → Use OBS or any streaming software to broadcast

The Story Behind It

The Climate Data Gap: Satellites collect terabytes of climate data daily. But farmers facing crop failures, cities battling pollution, and communities seeking education can't access it. ClimateIQ bridges this gap.

Why Video Was Missing:

• A community in India showing drought damage creates more impact than statistics


• Live broadcasts enable real-time expert consultations during crises


• 2B+ potential users need content in their language

Why Mux:

• We needed auto-captions for accessibility


• Low-latency for real-time consultations


• Multi-language support for global reach


• Production-ready infrastructure in days, not months

Technical Highlights

11 Mux Features Integrated:

Use of Mux (Additional Prize Category)

Mux isn't just hosting videos – it's the backbone of our entire community feature.

🎬 Beyond Basic Hosting

1. AI-Powered Chapters (@mux/ai SDK)

import { generateChapters } from '@mux/ai/workflows';

const chapters = await generateChapters(assetId, 'en', { provider: 'google' });

Climate videos get auto-chapters like "Impact on Agriculture" or "Mitigation Strategies" – helping users navigate long content.

2. Multi-Track Audio Dubbing

await muxVideo.assets.createTrack(assetId, {

  type: 'audio',

  url: supabaseAudioUrl,

  language_code: 'hi',

  name: 'AI Dubbed (hi)'

});

ElevenLabs generates dubbed audio → we add it to the Mux asset → users switch languages natively in the player.

3. Low-Latency Live Streaming

await muxVideo.liveStreams.create({

  reduced_latency: true,

  reconnect_window: 60,

  generated_subtitles: [{ language_code: 'en', name: 'English (auto)' }]

});

Users broadcast with low-latency. Experts provide real-time advice during live sessions.

4. Auto-Captions + AI Translation Pipeline

1. Upload with generated_subtitles → Mux generates English captions


2. Fetch VTT from https://stream.mux.com/{playbackId}/text/{trackId}.vtt
   


3. Gemini translates to Hindi/Spanish


4. Upload to Supabase + add via assets.createTrack()
   

📊 What Makes This Deep Integration

We're not just hosting videos. We're using:

• Official @mux/ai SDK for chapter generation


• Multi-Track Audio for dubbed languages


• Text Track API for translated captions


• Master Access for AI processing pipelines


• Low-Latency Live Streaming for real-time advice


• Custom Analytics Dimensions for climate insights

🌍 Real Impact

Mux handles encoding, CDN, multi-track, and AI – so we focus on climate impact.

❗ By submitting this project, I confirm that my video adheres to Mux's terms of service: https://www.mux.com/terms