Did you know that every ~20 questions you ask an AI chatbot consumes

roughly a 500ml bottle of water for data centre cooling?

As AI scales, so does its thirst. A huge chunk of this is pure

waste — because we ask LLMs the same things over and over. Every

redundant query is a real, physical cost.

I'm a 3rd year CS engineering student and I built EcoCache to

reduce and measure that waste.

What it does

EcoCache sits in front of your LLM API calls. Before hitting the

model, it checks whether a semantically similar question was already

answered. If yes — it returns the cached answer instantly. If no —

it calls the API and stores the result for next time.

It's not exact string matching. "What is TCP?" and "Can you explain

TCP protocols?" are recognised as the same question using vector

embeddings and cosine similarity.

See it in action

from ecocache.client import EcoCacheClient



client = EcoCacheClient()  # add your Gemini API key to .env



# First call — hits the API

r1 = client.chat("What is the difference between TCP and UDP?")

print(r1["source"])   # → "api"



# Similar question — served from cache, no API call made

r2 = client.chat("Can you explain TCP vs UDP protocols?")

print(r2["source"])      # → "cache"

print(r2["savings"])     # → water and carbon saved so far

The dashboard

It comes with a live dashboard that tracks savings in real time:

50% cache hit rate on my tests. Every cache hit = one fewer LLM

inference = ~5mL water and ~4g CO2 saved. Small numbers individually.

Meaningful at scale.

How it works under the hood

1. Query comes in


2. Sentence-transformers converts it to a 384-dimensional vector


3. FAISS searches for the nearest vector in the cache


4. If similarity > 0.85 — return cached response


5. If not — call the LLM, store the result

Try it

git clone https://github.com/GanugapatiSaiSowmya/ecocache

cd ecocache

python3.11 -m venv venv && source venv/bin/activate

pip install -r requirements.txt

GitHub: https://github.com/GanugapatiSaiSowmya/ecocache

This is v0.1; rough edges exist. I'm actively working on it and

I appreciate your feedback, issues, or contributions.

If you think responsible AI development matters, a star would mean

a lot to a broke college student trying to make a dent ⭐

python #ai #sustainability #opensource #climatetech