KODA Format: A Schema-First Data Format to Reduce LLM Token Usage ( 40%)

When building applications with large language models (LLMs), one of the most overlooked costs is how structured data is represented.

Most systems use JSON.

And JSON is inefficient for LLM input.

What is KODA?

KODA (Knowledge-Oriented Data Abstraction) is a schema-first data format designed to reduce token usage when sending structured data to LLMs.

It works by:

• Defining structure once (schema-first)


• Encoding values positionally


• Eliminating repeated keys found in JSON

KODA is optimized for:

• RAG pipelines


• Tool calling systems


• Agent workflows


• High-volume structured LLM input

The Problem with JSON in LLM Pipelines

JSON repeats field names for every record.

Example:

[

  {"id": 1, "title": "Bug", "state": "open"},

  {"id": 2, "title": "Fix", "state": "closed"}

]

Each object repeats:

• id


• title


• state

If you send 1000 records:

• those keys are repeated 1000 times


• tokens are wasted


• costs increase


• context window shrinks

KODA Equivalent

KODA/1

@META

schemas:issue

counts:issue=3



@SCHEMA

issue:id title state



@DATA:issue

1|Bug|open

2|Fix|closed

No repeated keys.

Only structure + values.

Token Reduction Benchmark

Measured using a gpt-4o-mini tokenizer on real datasets.

Key insight

KODA performs best on large, repetitive structured data.

For small datasets, schema overhead can outweigh benefits.

Why This Matters

In LLM systems:

• Tokens = cost


• Tokens = latency


• Tokens = context capacity

Reducing tokens by ~30–40%:

• lowers API costs


• increases usable context


• improves system efficiency

How KODA Works

KODA separates:

• Schema → defined once


• Data → streamed positionally

This removes structural redundancy.

Quick Python Example

from koda import Schema, Field, encode



schema = Schema("user", [

    Field("id"),

    Field("name"),

    Field("email", optional=True),

    Field("active", default="true")

])



data = [

    {"id": 1, "name": "Alice", "email": "[email protected]"},

    {"id": 2, "name": "Bob"}

]



koda_str = encode(data, schema)

print(koda_str)

KODA vs JSON vs YAML vs TOON

When to Use KODA

Use KODA if you are:

• sending large structured datasets to LLMs


• building RAG pipelines


• working with tool calls or agents


• optimizing token usage in production systems

When NOT to Use KODA

Do not use KODA for:

• small datasets (1–2 records)


• irregular or deeply nested JSON


• human-authored configuration files

JSON is better in those cases.

Design Principles

• Schema-first design


• Positional encoding


• Deterministic parsing


• No repeated keys


• Optimized for LLM input

Is KODA a JSON Replacement?

No.

KODA is a transport format for LLM pipelines.

Typical workflow:

JSON → KODA → LLM

FAQ

What is KODA?

KODA is a schema-first data format that reduces token usage for structured data in LLM systems.

Is KODA better than JSON?

For LLM input, yes. For general use, JSON is still better.

Does KODA always reduce tokens?

No. It works best on large structured datasets.

Where should I use KODA?

RAG pipelines, tool calls, and structured LLM input.

Try It

GitHub: https://github.com/Om7035/koda

pip install koda

Final Thoughts

If you're sending structured data to LLMs, you're likely wasting tokens.

KODA is a simple way to reduce that overhead.

It’s not a replacement for JSON it’s an optimization layer for LLM pipelines.

Feedback and contributions are welcome.