Main technologies used

• Gemma 4 E2B


• llama.cpp


• FastAPI


• SQLite


• ChromaDB


• Vanilla JavaScript


• HTML/CSS

How I Used Gemma 4

Memoria uses Gemma 4 as the core local reasoning engine for the entire reading experience.

I used the Gemma 4 E2B model through a local llama.cpp OpenAI-compatible server, allowing the application to run fully offline without relying on cloud APIs.

Why Gemma 4 E2B?

I specifically chose Gemma 4 E2B because it was the best fit for a responsive local reading assistant.

The project needed:

• Fast inference speeds


• Low VRAM usage


• Good reasoning quality


• Reliable structured outputs


• Practical local deployment on consumer hardware

Gemma 4 E2B delivered the right balance between speed and capability, making it possible to provide near real-time responses for recaps, contextual Q&A, text simplification, and chapter processing while still running locally through llama.cpp.

This was especially important because the app performs many smaller AI tasks continuously in the background while the user reads.

What Gemma 4 Powers

Spoiler-Safe Recaps

Gemma summarizes chapter chunks into structured summaries and key events that help readers quickly reconnect with the story.

Character Memory

The model updates persistent character descriptions and remembers important events tied to each character across chapters.

Speaker Attribution

Gemma helps identify ambiguous dialogue speakers when rule-based systems fail.

Contextual Q&A

Readers can ask questions about the story, and Gemma answers using chapter-aware retrieval that avoids future spoilers.

Text Simplification

Selected passages can be rewritten into clearer modern English while preserving meaning and tone.

Technical Architecture

The frontend is a lightweight EPUB reader built with vanilla HTML, CSS, and JavaScript. It handles book uploads, chapter navigation, reading controls, themes, typography settings, and the AI interaction panel.

The backend is built with FastAPI and SQLite. It manages books, chapters, summaries, embeddings, character memory, retrieval, and streaming responses.

The AI stack runs fully locally using llama.cpp:

• Gemma 4 E2B runs as the local chat and reasoning model


• Nomic embeddings power semantic retrieval


• ChromaDB stores vector embeddings per book


• Background processing pipelines analyze chapters incrementally

The app processes books chapter-by-chapter instead of trying to load entire novels into context at once. Intermediate artifacts like summaries, character memory, embeddings, and speaker metadata are stored and reused throughout the reading experience.

This pipeline-first design makes the system faster, more grounded, and more practical for long-form reading.

Spoiler-Safe Retrieval

One of the biggest design goals was preventing accidental spoilers.

When a reader asks a question, Memoria retrieves only information from chapters the user has already completed. The retrieval system filters vector search results using reading progress before sending context to Gemma 4.

This allows the app to help readers remember earlier story details without revealing future events.

Challenges

Handling Long Books

Full novels are too large to send directly into a local model context window. I solved this by chunking chapters into smaller sections while carrying forward rolling summaries and character memory.

Structured Output Reliability

Local models sometimes wrap JSON outputs in extra formatting or explanations. To make the pipeline reliable, prompts were heavily constrained and the backend extracts valid JSON blocks safely before processing.

Speaker Attribution

Dialogue attribution in fiction is difficult because speakers are often implied instead of explicitly named. I used a hybrid approach where rules handle obvious cases while Gemma handles ambiguous dialogue using broader context.

Fully Local Deployment

The project depends on multiple services including Gemma 4, embedding models, Python environments, and vector databases. I automated the setup process using launcher scripts so the app can be started locally with minimal manual configuration.

Why Local AI Matters

One of the main goals of this project was accessibility and digital equity.

Readers should not need:

• expensive subscriptions


• cloud AI services


• constant internet access


• external data collection

By combining Gemma 4 with llama.cpp and local retrieval, Memoria creates a fully local AI reading companion that respects reader privacy while remaining accessible on consumer hardware.

This makes the project useful not only for individual readers, but also for classrooms, libraries, care settings, and offline learning environments.

Conclusion

Memoria demonstrates how Gemma 4 can power practical, privacy-friendly accessibility tools beyond chatbots.

Instead of replacing reading, the goal is to support readers — helping them stay connected to stories, remember context, and reduce cognitive load while preserving the experience of reading itself.

By combining Gemma 4 E2B, llama.cpp, retrieval, and structured processing pipelines, Memoria turns static EPUB books into adaptive reading experiences that can run entirely offline.