OrkaJS: The TypeScript Framework That Makes LLM Development Actually Simple
If you've tried building an LLM-powered application recently, you know the pain. The ecosystem is fragmented, frameworks are complex, and you're often locked into specific providers. After months of frustration, I built OrkaJS to solve these problems.
The Problem
Let me paint a picture. You want to add RAG (Retrieval-Augmented Generation) to your app. Simple, right?
Reality check:
- Learn a framework-specific API (steep learning curve)
- Get locked into one LLM provider
- Manually handle retries, caching, and error handling
- Write custom code for each vector database
- Spend weeks on infrastructure instead of features
What you actually want:
const result = await orka.ask({
question: 'How do I reset my password?',
knowledge: 'support-docs'
});
That's it. That's OrkaJS.
What is OrkaJS?
OrkaJS is a progressive TypeScript framework for building production-ready LLM systems. It's designed with three core principles:
Universal Compatibility - Works with ANY JS/TS framework (Next.js SSR, Express, NestJS, Fastify, AdonisJS, KoaJS, Hono...)
Pluggable Architecture - Swap LLMs or vector databases in 3 lines of code
Production-Ready - Resilience, caching, evaluation, and observability built-in
Quick Start (Literally 2 Minutes)
Installation
npm install orkajs
Basic RAG in 10 Lines
import { createOrka, OpenAIAdapter, MemoryVectorAdapter } from 'orkajs';
const orka = createOrka({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
vectorDB: new MemoryVectorAdapter(),
});
// Create knowledge base
await orka.knowledge.create({
name: 'docs',
source: ['OrkaJS is a TypeScript framework for LLM systems.'],
});
// Ask with RAG
const result = await orka.ask({
knowledge: 'docs',
question: 'What is OrkaJS?',
});
console.log(result.answer);
That's it. No complex configuration, no boilerplate, no vendor lock-in.
Why OrkaJS is Different
1. Works Everywhere
Same code, any framework:
// Next.js API Route
export async function POST(req: Request) {
const { question } = await req.json();
const answer = await orka.ask({ question, knowledge: 'docs' });
return Response.json({ answer });
}
// Express.js
app.post('/ask', async (req, res) => {
const { question } = req.body;
const answer = await orka.ask({ question, knowledge: 'docs' });
res.json({ answer });
});
// NestJS
@Post('ask')
async ask(@Body() body: AskDto) {
return this.orka.ask({ question: body.question, knowledge: 'docs' });
}
2. Swap Providers in Seconds
// Development: Use Ollama locally
const orka = createOrka({
llm: new OllamaAdapter({ model: 'llama2' }),
vectorDB: new MemoryVectorAdapter(),
});
// Production: Switch to OpenAI + Pinecone
const orka = createOrka({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
vectorDB: new PineconeAdapter({
apiKey: process.env.PINECONE_API_KEY,
indexHost: process.env.PINECONE_HOST
}),
});
Same API. Zero code changes.
3. Production Patterns Built-In
Automatic Retries with Exponential Backoff
import { ResilientLLM } from 'orkajs/resilience';
const llm = new ResilientLLM(
new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
{ maxRetries: 3, backoffMs: 1000 }
);
Multi-Provider Fallback
import { FallbackLLM } from 'orkajs/resilience';
const llm = new FallbackLLM([
new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
new AnthropicAdapter({ apiKey: process.env.ANTHROPIC_API_KEY }),
new OllamaAdapter({ model: 'llama2' }), // Local fallback
]);
Intelligent Caching
import { CachedLLM } from 'orkajs/cache';
import { RedisCache } from 'orkajs/cache/redis';
const llm = new CachedLLM(
new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
new RedisCache({ url: process.env.REDIS_URL })
);
Advanced Features
🤖 Intelligent Agents
OrkaJS includes multiple agent architectures:
ReAct Agent (Reasoning + Acting):
import { ReActAgent } from 'orkajs/agent/react';
const agent = new ReActAgent({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
tools: [searchTool, calculatorTool],
maxIterations: 5,
});
const result = await agent.run('What is the weather in Paris and what is 25 * 4?');
Plan & Execute Agent:
import { PlanAndExecuteAgent } from 'orkajs/agent/plan-and-execute';
const agent = new PlanAndExecuteAgent({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
tools: [webSearchTool, calculatorTool, weatherTool],
});
const result = await agent.run('Research competitors and create a pricing strategy');
🔄 Multi-Model Orchestration
Router (route queries to specialized models):
import { RouterLLM } from 'orkajs/orchestration';
const router = new RouterLLM({
router: new OpenAIAdapter({ model: 'gpt-4o-mini' }), // Fast router
routes: [
{
name: 'code',
llm: new OpenAIAdapter({ model: 'gpt-4o' }),
condition: (prompt) => prompt.includes('code') || prompt.includes('function'),
},
{
name: 'creative',
llm: new AnthropicAdapter({ model: 'claude-3-5-sonnet-20241022' }),
condition: (prompt) => prompt.includes('story') || prompt.includes('creative'),
},
],
fallback: new OpenAIAdapter({ model: 'gpt-4o-mini' }),
});
Consensus (multiple models vote):
import { ConsensusLLM } from 'orkajs/orchestration';
const consensus = new ConsensusLLM({
llms: [
new OpenAIAdapter({ model: 'gpt-4o' }),
new AnthropicAdapter({ model: 'claude-3-5-sonnet-20241022' }),
new MistralAdapter({ model: 'mistral-large-latest' }),
],
strategy: 'majority', // or 'unanimous'
});
📊 Built-in Evaluation
import { TestRunner } from 'orkajs/evaluation';
const runner = new TestRunner(orka);
await runner.test('RAG Quality', async () => {
const result = await orka.ask({
question: 'How do I reset my password?',
knowledge: 'support-docs',
includeContext: true,
});
runner.assert.relevance(result.answer, result.context, 0.8);
runner.assert.faithfulness(result.answer, result.context, 0.9);
runner.assert.noHallucination(result.answer, result.context);
});
await runner.run();
🔍 Advanced Retrievers
import { MultiQueryRetriever } from 'orkajs/retrievers';
const retriever = new MultiQueryRetriever({
vectorDB: new PineconeAdapter({ /* config */ }),
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
numQueries: 3, // Generate 3 variations of the query
});
const chunks = await retriever.retrieve('How to reset password?');
// Generates: "reset password", "change password", "password recovery"
// Returns combined unique results
Real-World Example: Customer Support Bot
Here's a complete example of a production-ready support bot:
import { createOrka, OpenAIAdapter, PineconeAdapter } from 'orkajs';
import { ResilientLLM } from 'orkajs/resilience';
import { CachedLLM } from 'orkajs/cache';
import { RedisCache } from 'orkajs/cache/redis';
// 1. Setup with production patterns
const llm = new CachedLLM(
new ResilientLLM(
new OpenAIAdapter({
apiKey: process.env.OPENAI_API_KEY,
model: 'gpt-4o-mini',
}),
{ maxRetries: 3, backoffMs: 1000 }
),
new RedisCache({ url: process.env.REDIS_URL })
);
const orka = createOrka({
llm,
vectorDB: new PineconeAdapter({
apiKey: process.env.PINECONE_API_KEY,
indexHost: process.env.PINECONE_HOST,
}),
});
// 2. Index your documentation
await orka.knowledge.create({
name: 'support-docs',
source: { path: './docs' }, // Auto-loads all files
chunkSize: 1000,
chunkOverlap: 200,
});
// 3. Handle user questions
export async function handleSupportQuestion(question: string) {
const { answer, context } = await orka.ask({
question,
knowledge: 'support-docs',
systemPrompt: 'You are a helpful support agent. Be concise and friendly.',
topK: 5,
includeContext: true,
});
return {
answer,
sources: context.map(c => ({
content: c.content.slice(0, 100),
score: c.score,
})),
};
}
Framework Comparison
| Feature | OrkaJS | LangChain | LlamaIndex |
|---|---|---|---|
| Learning Curve | ⭐⭐⭐⭐⭐ Low | ⭐⭐ High | ⭐⭐⭐ Medium |
| TypeScript-First | ✅ Native | ⚠️ Port | ⚠️ Python-first |
| Framework Agnostic | ✅ All JS/TS | ⚠️ Limited | ⚠️ Limited |
| Pluggable Providers | ✅ 3 lines | ⚠️ Complex | ⚠️ Complex |
| Production Patterns | ✅ Built-in | ❌ Manual | ❌ Manual |
| Tree-shaking | ✅ Optimized | ❌ No | ❌ No |
| Bundle Size | 🟢 Small | 🔴 Large | 🔴 Large |
Architecture Philosophy
OrkaJS is built on three pillars:
1. Intent-Based API
Focus on what you want, not how to do it:
// ✅ OrkaJS - Intent-based
await orka.ask({ question, knowledge: 'docs' });
// ❌ Other frameworks - Implementation-based
const vectorStore = await loadVectorStore();
const retriever = vectorStore.asRetriever();
const chain = RetrievalQAChain.fromLLM(llm, retriever);
const result = await chain.call({ query: question });
2. Adapter Pattern
Every external dependency is an adapter implementing a simple interface:
interface LLMAdapter {
generate(prompt: string, options?: GenerateOptions): Promise<string>;
embed(texts: string[]): Promise<number[][]>;
}
interface VectorDBAdapter {
upsert(chunks: Chunk[]): Promise<void>;
search(query: number[], topK: number): Promise<ChunkResult[]>;
}
Swap implementations without changing your code.
3. Progressive Enhancement
Start simple, add complexity only when needed:
// Day 1: Simple RAG
const answer = await orka.ask({ question, knowledge: 'docs' });
// Week 2: Add caching
const llm = new CachedLLM(baseAdapter, cache);
// Month 3: Add multi-provider fallback
const llm = new FallbackLLM([primary, secondary, local]);
// Month 6: Add custom evaluation
await runner.test('Quality', async () => { /* tests */ });
Check out the Quick Start Guide and build your first LLM-powered feature in 10 minutes.
SOCIAL SHARE CARD GENERATOR