Created for the Gemini Live Agent Challenge
The Problem: Children Are Losing the Habit of Reading
Children worldwide are spending more time on social media than ever before — and it's replacing reading, creative play, and healthy development.
U.S. teens spend an average of 4.8 hours per day on social media ()
ASEAN school-aged children (ages 6-14) spend 2.77 hours per day on screens, exceeding the recommended 2-hour limit ()
Children are drawn to screens because the content is engaging — colorful, animated, interactive. But most of that content is passive consumption. There is a lack of interactive, creative, educational digital experiences that match the engagement level of social media while actually benefiting children's development.
As a developer from Timor-Leste — where over 72% of the population is under 35 but children's book access remains extremely limited — I wanted to change that. What if a child could simply speak their idea and watch it transform into a fully illustrated, narrated storybook?
That's exactly what I built with KidStory: Storybook for Kids — an AI-powered platform that transforms screen time from passive consumption into active creation.
For Parents
Register an account, then create stories for your children to read and enjoy together.
For Kids
Let your child speak their own story idea and watch the AI bring it to life with pictures and narration.
Screenshot showing narrator voice selection
3. The Quiz Master: gemini-2.5-flash
After reading, kids can take an interactive quiz. I optimized this by using the text-only model for quiz generation (faster and more cost-effective), then adding TTS for audio:
// In /app/api/live-quiz/route.ts
const quizResponse = await ai.models.generateContent({
model: "gemini-2.5-flash", // Text-only for speed
contents: quizPrompt,
config: {
responseMimeType: "application/json",
},
});
// Then generate audio separately
const audioResponse = await generateAudio(question.text);
KidStory system architecture showing all components
Key Technical Challenges & Solutions
1. Streaming Interleaved Content
The biggest challenge was handling the interleaved stream of text and images. Gemini returns them mixed together, so I needed to:
// Parse the stream and separate text from images
for await (const chunk of result.stream) {
const parts = chunk.candidates?.[0]?.content?.parts || [];
for (const part of parts) {
if (part.text) {
// Handle story JSON text
storyBuffer += part.text;
} else if (part.inlineData) {
// Handle image data
const imageData = part.inlineData.data;
// Upload to Cloud Storage
// Send URL to client
}
}
}
2. Character Consistency
Kids can upload photos of themselves or loved ones to appear in the story. To maintain consistency across all pages:
// Compress and include reference images
const referenceImages = await Promise.all(
characterPhotos.map(async (photo) => {
const compressed = await compressImage(photo);
return {
inlineData: {
mimeType: "image/jpeg",
data: compressed,
},
};
}),
);
// Include in prompt
const prompt = {
role: "user",
parts: [
{ text: storyPrompt },
...referenceImages, // Gemini uses these for consistency
],
};
3. Real-time Progress Updates
To create the "magical painting" effect, I used Server-Sent Events (SSE) to stream updates to the client:
// Server: Send updates as they arrive
const encoder = new TextEncoder();
const stream = new ReadableStream({
async start(controller) {
// Send story chunks
controller.enqueue(encoder.encode(`data: ${JSON.stringify(chunk)}\n\n`));
// Send image URLs as they're uploaded
controller.enqueue(
encoder.encode(`data: ${JSON.stringify(imageUpdate)}\n\n`),
);
},
});
return new Response(stream, {
headers: {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
},
});
// Client: React hook to handle streaming
const useStoryGenerator = () => {
const [progress, setProgress] = useState({});
const eventSource = new EventSource("/api/generate-story");
eventSource.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.type === "chunk") {
// Update story text
} else if (data.type === "image") {
// Show image for specific page
setProgress((prev) => ({
...prev,
[`page_${data.pageNumber}`]: "complete",
}));
}
};
};
4. Voice Input for Kids
Using the Web Speech API, kids can speak their story ideas:
// Custom hook for voice recognition
const useVoiceInput = () => {
const recognition = new webkitSpeechRecognition();
recognition.continuous = true;
recognition.interimResults = true;
recognition.onresult = (event) => {
const transcript = Array.from(event.results)
.map((result) => result[0].transcript)
.join("");
setTranscript(transcript);
};
return { startListening, stopListening, transcript };
};
5. PDF Generation with Compression
Stories can be downloaded as PDFs. To keep file sizes manageable, I implemented image compression:
// Compress images before adding to PDF
async function compressImage(dataUrl: string): Promise<string> {
return new Promise((resolve) => {
const img = new Image();
img.onload = () => {
const canvas = document.createElement("canvas");
const maxWidth = 800;
let width = img.width;
let height = img.height;
if (width > maxWidth) {
height = (height * maxWidth) / width;
width = maxWidth;
}
canvas.width = width;
canvas.height = height;
const ctx = canvas.getContext("2d");
ctx.drawImage(img, 0, 0, width, height);
// Compress as JPEG with 70% quality
const compressed = canvas.toDataURL("image/jpeg", 0.7);
resolve(compressed);
};
img.src = dataUrl;
});
}
This reduced PDF sizes by 70-85% while maintaining good visual quality.
Screenshot of Cloud Run console showing deployed service
IAM Permissions
One challenge was setting up the right permissions. I automated this with a script:
#!/bin/bash
# fix_permissions.sh
# Grant Vertex AI access
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:${COMPUTE_SA}" \
--role="roles/aiplatform.user"
# Grant Firestore access
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:${COMPUTE_SA}" \
--role="roles/datastore.user"
# Grant Cloud Storage access
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:${COMPUTE_SA}" \
--role="roles/storage.objectAdmin"
and source on GitHub:
Live Demo: https://ai.kidstory.app
Video Demo:
Questions?
Feel free to reach out or open an issue on GitHub. I'd love to hear your thoughts and answer any questions about building with Gemini's interleaved output!
Made with ❤️ for children everywhere
SOCIAL SHARE CARD GENERATOR