A some time ago I shipped a desktop app to generate LLM fine-tuning datasets. It worked: my Qwen2.5-Coder-7B fine-tune jumped from 55.5% → 72.3% on HumanEval. Whole pipeline ran on OpenRouter — pick a model, click Generate, get JSONL.
v1.0.3-beta ships multi-provider LLM support — Ollama, LM Studio, llama.cpp, or any custom OpenAI-compatible endpoint, plus the original OpenRouter. Mix and match: generate on your local Qwen3-14B, judge on a cheap cloud model. Or stay fully offline.
Here's what shipped, what was harder than I expected, and what I learned along the way.
What's new in v1.0.3-beta
One-click local LLM. Open Settings → Providers → "Auto-detect local". The app probes localhost:11434 (Ollama), 1234 (LM Studio), 8080 (llama.cpp). Anything that answers gets a one-click "Add" button. Onboarding for an offline-first user takes ~30 seconds.
Mixed mode. Each category can use its own provider. Gen on local Qwen2.5-Coder:14B, judge on cloud example GPT-4-mini. Or different generators per category — algorithm category on a code-specialised local model. The pipeline routes each call to the right backend automatically.
Custom endpoints. Any OpenAI-compatible URL works: vLLM, TGI, your buddy's self-hosted gateway. Paste base URL + optional bearer token, done.
Original release post (with HumanEval +16pp benchmark): [previous dev.to post]
Dataset (2,248 examples):
What's next
System tray version. Long generation runs (5000+ examples on local hardware = hours) deserve a quieter UX than a permanent open window. Tray icon, "next job ready" notification, click to bring back the dashboard.
Embedding provider picker. Right now dedup works multi-provider on the backend, but the UI only exposes the OpenRouter embedding models. Adding a small dropdown so local users can run dedup on nomic-embed-text via Ollama too.
Two new categories targeting LiveCodeBench and BigCodeBench. The previous post explained why those benchmarks barely moved (format mismatch on LCB, too-generic library category for BCB). Both fixes are in progress — algorithmic drill with edge-case coverage for LCB, library-API-precise taxonomy for BCB.
If you generate datasets locally — what model size are you using and what's your accept rate? Especially curious if anyone got real value out of <14B local models for dataset gen, because my tests said no but I'd love to be wrong.
Disclosure: I drafted this post with AI help — same way I built the app.
SOCIAL SHARE CARD GENERATOR