KNN Algorithm from Scratch -Cat vs Dog Image Classification in Python (Complete Experiment)

🧠 KNN Algorithm from Scratch — Real Image Classification Experiment

I recently built a K-Nearest Neighbors (KNN) algorithm from scratch in Python to perform cat vs dog image classification.

This project focuses on understanding how distance-based machine learning really works, rather than simply chasing accuracy.

🎯 What this project demonstrates

• Machine Learning from scratch
  


• KNN algorithm implementation
  


• Image classification using raw pixel vectors
  


• Supervised learning fundamentals
  


• How noise, distance, and neighbors affect predictions
  


• The curse of dimensionality in real experiments
  

🧪 The core experiment

After training the model, I discovered that removing only 5 noisy images from each class drastically improved prediction accuracy.

This happens because:

• KNN has no abstraction or learning of features
  


• All intelligence comes from the geometry of the data
  


• A few bad samples can distort the entire decision boundary
  

🧬 Why this matters

Most tutorials skip the uncomfortable truth about KNN:

• More data does not always mean better results
  


• Distance metrics define the model’s intelligence
  


• High-dimensional data behaves very unintuitively
  

Understanding this makes you a much stronger machine learning engineer.

📦 Complete Source Code

You can explore the full project here:


🔗 https://github.com/yogender-ai/knn-cat-dog-demo

🏁 Final thought

Before neural networks learn to see,


we must understand how distance learns to lie.

If you’re learning machine learning, this experiment is worth exploring.