This is a submission for the Hermes Agent Challenge: Build With Hermes Agent
What I Built
A small backstory on my coding origins
I learnt coding by secretly studying from a python book pdf on my Computer Scientist father's work laptop. That very day I wrote a program that inputs a user's name and prints: "You are mad {name}!" and had a lot of fun pranking my brother using that script.
Since then, there have been very few moments where coding felt as magical, because the more you understand syntax, the more you understand the magic underneath your code.
That is, until you meet a genius piece of magic such as Hermes.
My app idea
I built a Fitness coach inside Hermes that learns and adapts based on your daily feedback. According to your goals, performance, and time allocated, it adjusts your current plan. For the purpose of this hackathon I tested it via terminal ui (tui) but I plan to release the polished version on chat apps such as telgram and whatsapp for painless daily checkins.
Demo
Code
My Tech Stack
Hermes + node.js. Kept it simple for this quick dive.
How I Used Hermes Agent
Building an AI application that feels truly personal requires more than just a clever prompt; it requires state, memory, and the ability to adapt over time. During a recent hackathon, I set out to build an autonomous AI fitness coach. Not just a chatbot that spits out generic workout templates, but a system that onboards a user, sets a multi-month timeline, generates habit blocks, and adjusts daily based on feedback.
To achieve this, I used Hermes, an agentic framework designed for stateful, long-running applications. Here is a breakdown of how I built it, the challenges faced, and why Hermes was the perfect tool for the job.
Why This App is a Perfect Fit for Hermes
Most LLM interactions are stateless. You ask a question, you get an answer, and the session ends. A fitness journey, however, is a deeply stateful process. It spans weeks or months and requires constant recalibration.
Hermes excels here because of its Skill Architecture. In Hermes, "skills" act as the agent's internal database and long-term memory. Instead of relying on the LLM to hopefully remember the user's goals across a 64k context window, I structured the coach's memory into strict JSON schemas:
-
user_fitness_profile: The foundational constraints (goals, phase, available days). -
master_timeline_skill: The 3 to 6-month master arc. -
habit_block_skill: 21-day executable units. -
weekly_workout_plan: The actual scheduled days.
When a user logs a /checkin saying a session was "Too easy", the Hermes Evaluation Agent reads these skills, decides on an adjustment, and surgically rewrites the weekly_workout_plan for the next day. Hermes handles the file I/O and state persistence autonomously, allowing the LLM to act as a true reasoning engine.
Experiences & Lessons Learned
1. Strongly Typed State is Non-Negotiable
Early on, I realized that if the LLM hallucinates the structure of the fitness plan, the entire application loop breaks. I created a skills.ts file with strict TypeScript interfaces for every step of the journey. By enforcing these schemas in the system prompts, Hermes consistently generated valid, parsable JSON that cleanly mapped to my application's logic.
2. The Rate Limit Reality: Batching vs. Conversational
Initially, I designed the onboarding flow to be highly conversational, asking the user 8 questions one by one. I quickly hit the Gemini API Free Tier rate limits (Resource Exhausted: 429). Hermes is an agentic framework, meaning it often makes multiple API calls (thoughts, tool usage, generation) per user interaction.
The Fix: I pivoted the architecture to a batched approach. I combined the questionnaire into a single, multi-part prompt. The user selects their timeline and phase in one go. This drastically reduced API calls, saved tokens, and bypassed the rate limits entirely without sacrificing the personalized feel.
3. Hardcoding the "Rules of the Game"
LLMs are great at reasoning, but terrible at strict constraints. Instead of asking the LLM to invent a workout split, I hardcoded the domain logic into the prompt:
- Lean Bulk -> 4-day Upper/Lower or 4-day Full Body
- Aggressive Bulk -> 6-day Arnold
- Recomposition -> 3-day Full Body
Hermes reads the user's state, applies these hard rules, and maps the days accordingly. This prevents bad prescriptions (like recommending a 6-day split to someone on an aggressive cut, which would lead to under-recovery).
Extending the Coach using Hermes
The current iteration is a solid Sprint 1, but the Hermes framework makes extending this app incredibly straightforward:
1. From Terminal to Telegram (and beyond)
Hermes has built-in gateway support. Transitioning this coach from my local terminal to a public Telegram bot doesn't require rewriting the core logic. By simply providing a Bot Token and starting the hermes gateway, the exact same stateful agent can interact with users via their phones. We can even use cron jobs to trigger automated /checkin prompts at 8 PM every night. If this idea picks up I want to deploy it as a paid app on whatsapp and telegram
2. Contextual Awareness via MCP (Model Context Protocol)
Life happens. People get injured or go on work trips. By utilizing Hermes's MCP support, the coach could integrate with a user's Google Calendar. If it sees a "Conference in Vegas" next week, the external_event_handler skill can preemptively adjust the master_timeline_skill, shifting heavy workout days around the travel schedule.
3. Adding Knowledge with RAG
Currently, to save context, the weekly plan only schedules the types of days (e.g., "Upper 1"). By hooking up a local vector database via Hermes tools, the coach could query a library of exercises, form videos, and rep ranges, injecting specific, customized workouts into the daily sessions just-in-time.
Conclusion
Building a fitness coach with Hermes proved that agentic frameworks are the key to moving beyond simple chatbots. By combining strict state management (Skills), hardcoded domain rules, and LLM reasoning, you can build applications that truly learn, adapt, and guide users over the long term.
Honestly, the only major limitation on such an app when using Hermes is the LLM tokens that you can spend. Outside of which Hermes in itself is a perfect fit for such feedback loops within the app. I look forward to exploring this library in depth and enjoy this feeling of creating what feels like magic using Agentic ai.