Kubernetes Architecture: What Actually Happens Between `kubectl apply` and a Running Pod

Most of us run kubectl apply -f dozens of times a day without thinking about the machinery it sets in motion. But when something breaks, a Pod stuck in Pending, a Service that won't route, a Deployment that never converges, understanding that machinery is the difference between guessing and debugging.

In this article, I'll map the end-to-end flow onto the actual Kubernetes architecture, so you can see not just what happens, but which component is responsible at every step.

The Architecture at a Glance

Kubernetes is split into two planes:

• Control plane: the brain. It makes decisions: what should exist, where it should run, and whether reality matches intent.


• Worker nodes: the muscle. They run your actual workloads and report back.
  Here's the full picture, with the request flow numbered:
  
  

(1) apply YAML → API Server (5) Kubelet asks runtime to start container

(2) spec persisted in etcd (6) runtime pulls image & runs it

(3) controller reconciles spec (7) CNI assigns Pod IP, joins network

(4) scheduler assigns a node (8) Kubelet reports status back

Now let's walk through the flow, component by component.

Step 1: The Cluster Exists Before Your App Does

A Kubernetes cluster is the combination of a control plane and a set of worker nodes. The control plane components (API Server, etcd, Controller Manager, Scheduler) can run on dedicated nodes or, in managed offerings like RKE2/EKS/GKE/AKS, be entirely abstracted away from you. Either way, they're always there, always watching.

Step 2: You Declare Intent in YAML

You don't tell Kubernetes how to run your app, you describe what you want. Typically that's a set of manifests:

• Deployment: how many replicas, which image, update strategy


• Service: a stable virtual endpoint in front of ephemeral Pods


• ConfigMap/Secret: configuration decoupled from the image

This declarative model is the foundation of everything that follows. Kubernetes' whole job is to close the gap between your declared state and reality.

Step 3: kubectl apply -f Hits the API Server

Every interaction with the cluster - from you, from controllers, from the Kubelet - goes through the API Server. It's the single front door. It authenticates the request, authorizes it (RBAC), runs admission controllers, and validates the object.

Nothing talks to etcd directly except the API Server. That's an architectural choice worth remembering: it's what makes the whole system auditable and consistent.

Step 4: The Request Is Classified

The API Server works out what kind of operation this is:

• Creating a new object (a Pod, a Deployment)?


• Updating or deleting an existing one?


• Something that should wake up a controller?

For kubectl apply, it performs a three-way merge between your manifest, the last applied configuration, and the live object, which is why apply behaves differently from create or replace.

Step 5: The Spec Lands in etcd

Once validated, the object's spec is written to etcd, the distributed key-value store that holds the entire cluster state. etcd is the only stateful component in the control plane. If you can restore etcd, you can restore your cluster. Which is why etcd backups are non-negotiable in any production setup.

At this point, nothing is running yet. You've simply recorded intent.

Step 6: Controllers Notice the New Spec

The Controller Manager runs a collection of control loops, each watching the API Server for objects they care about. When your Deployment appears:

• The Deployment controller sees it and creates a ReplicaSet.


• The ReplicaSet controller sees the ReplicaSet and notices the actual Pod count (zero) doesn't match the desired count.

Controllers never talk to each other directly. They coordinate entirely through the API Server and etcd - watching state, comparing it to intent, and acting.

Step 7: Controllers Create Resources

The ReplicaSet controller creates the missing Pod objects. Crucially, these Pods exist in etcd but have no node assigned - they sit in Pending, waiting for the next actor in the chain.

Step 8: The Scheduler Assigns a Node

The Scheduler watches for exactly this: Pods with no nodeName. For each one, it runs a two-phase decision:

1. Filtering; which nodes can run this Pod? (enough CPU/memory, matching node selectors, tolerating taints)


2. Scoring; of those, which node is best? (spreading, affinity rules, resource balance)

It then binds the Pod to the winning node by - you guessed it - writing that decision back through the API Server.

Step 9: The Kubelet Takes Over

On the chosen node, the Kubelet (the node agent) sees a Pod has been bound to it. The Kubelet asks the container runtime - via the Container Runtime Interface (CRI) - to make it real.

Step 10: The Runtime Runs the Container

The runtime (containerd, CRI-O) pulls the image if it isn't cached, creates the container, and starts it inside the Pod sandbox. If you've ever debugged an ImagePullBackOff, this is the step where it happens.

Step 11: The CNI Plugin Wires Up Networking

The CNI plugin (Calico, Cilium, Flannel...) gives the Pod its network identity - an IP address on the cluster network. From this moment the Pod is reachable by any other Pod, on any node, without NAT. That flat network model is one of Kubernetes' core guarantees.

Step 12: kube-proxy Makes Services Work

kube-proxy (or its eBPF-based replacements) programs routing rules on every node so that traffic to a Service's virtual IP gets forwarded to healthy backend Pods. Services stay stable while Pods come and go - this is the layer that makes that illusion hold.

Step 13: Status Flows Back Upstream

The Kubelet continuously reports Pod status; phase, readiness, restart counts - back to the API Server, which persists it in etcd. This is what feeds readiness gates, kubectl get pods, and your monitoring stack.

Step 14: Self-Healing Kicks In

If a Pod crashes or gets deleted, the ReplicaSet controller notices the drift between desired and actual replica counts and creates a replacement. Nobody pages you; the control loop just closes the gap.

Step 15: The Loop Never Stops

This is the part that makes Kubernetes Kubernetes: the reconciliation loop runs forever. Watch, compare, act. Every controller, at every level, is continuously nudging the cluster toward your declared state.

Key Takeaways

1. The API Server is the only front door. Everything i.e users, controllers, Kubelets - communicates through it.


2. etcd stores intent, not just state. A running cluster is the consequence of what's in etcd.


3. No component commands another directly. Coordination happens through watches on shared state which is why the system is so resilient to individual component failures.


4. Debugging maps to the flow. Pending → scheduler problem. ImagePullBackOff → runtime/registry problem. Pod running but unreachable → CNI or kube-proxy problem.

Once this flow clicks, Kubernetes stops feeling like magic and starts feeling like what it actually is: a set of simple loops, each doing one job, converging together.

Thanks for reading! If you found this useful, follow me for more platform engineering deep dives.