We Gave Our Coding Tutor a Thinking Style Detector — This Is What We Found

By Aman | Team Clarion | Code Mentor AI

"Six of us. Same loop problem. Same wrong answer. And the platform LeetCode, Scrimba, take your pick gave us all the same hint. The issue wasn't just that these platforms forgot our mistakes. It was that they never tried to understand how we think. Code Mentor AI started as our attempt to fix that. My job was building the AI that actually models how a student approaches a problem not just what they got wrong."

Beyond Mistake Logging: Understanding Thinking Patterns
Most AI tutoring features focus on outputs wrong answers, failed tests, error messages. I worked on something earlier in the pipeline: the Thinking Pattern Report, which analyses how a student approaches a problem based on their behaviour in the editor.

Using Monaco editor event listeners, we capture keydown/keyup events, detect pauses longer than 3 seconds, and track deletion-to-keystroke ratios. These raw events get batched and sent to a FastAPI endpoint every few seconds, then aggregated into features that describe thinking style not just correctness.

The NLP Pipeline
I designed and built the full NLP pipeline that runs across Code Mentor AI's AI features. Every mistake classification, every Socratic hint, every skill gap summary runs through Groq's API. The choice of Groq was deliberate latency matters in a live coding environment, and Groq's inference speed made the difference between hints that feel responsive and hints that feel like the system is thinking too hard.

The 120-token cap on hints was a deliberate choice. Longer responses became mini-lectures. Short, pointed questions kept students in problem-solving mode.

Dynamic AI Models
One of the more interesting engineering decisions was making the AI model selection dynamic. Different features have different latency and quality requirements the Socratic hint needs to be fast and sharp, the Skill Gap Summary can be slower and more thorough. We built a model routing layer that selects the right Groq model variant per feature based on these requirements.

The Thinking Pattern That Surprised Us
When we ran the Thinking Pattern Report on simulated sessions, the most common pattern wasn't 'trial-and-error' as we expected. It was what we called 'hesitant-systematic' students who paused long before writing anything, wrote carefully, but then deleted large chunks when they hit an error. That pattern correlated strongly with logic errors, not syntax errors. We hadn't expected a thinking style to map so cleanly to a mistake type.

The Lesson
Groq's speed changed what was possible. We had initially planned to run hint generation asynchronously — show the hint after a delay. With Groq, we could make it feel synchronous. Never underestimate how much latency shapes user behaviour in a learning product.

The Hindsight agent memory layer is what made past mistake context available to every Groq call. Without Hindsight, each LLM call would be stateless. With it, every call is grounded in the student's real history.

Team Clarion
• Aanchal & Pranati — Backend architecture & database
• Lakshay — Full frontend integration with Next.js
• Aman, Kinjal & Priyanshu — Dynamic AI models & AI assistance features