Is Terraform for Kubernetes Applications Flawed? For Kubernetes, GitOps is The Way

As DevOps and platform engineers, we've been rightly conditioned to chant the mantra of "Infrastructure as Code." So, when it comes to deploying applications on Kubernetes, reaching for a familiar tool like Terraform seems logical. It promises a unified workflow to manage everything from VPCs to Helm charts. However, this is a seductive but ultimately flawed path. Using Terraform to manage the lifecycle of Kubernetes-native applications is a significant anti-pattern that creates friction, fragility, and works against the very design principles of Kubernetes itself.

It's time for a candid discussion. By forcing a tool designed for static infrastructure provisioning onto the dynamic, ever-reconciling world of Kubernetes, we are setting ourselves up for failure. The path to a more resilient, secure, and efficient platform lies in embracing the principles Kubernetes was built for, and that means adopting GitOps.

The Clash of Titans: Terraform State vs. Kubernetes Reconciliation

The core of the problem lies in a fundamental conflict of state management. Terraform operates on a discrete, snapshot-based model. It runs, compares the desired state in your HCL code to its stored state file (.tfstate), and generates a plan to converge the two. It is the undisputed source of truth.

Kubernetes, however, already has a powerful state management and reconciliation system. Its control plane continuously works to match the cluster's live state with the desired state declared in its etcd database. It’s a closed-loop system designed for constant, autonomous correction.

When you layer Terraform on top of this, you create two competing sources of truth. This inevitably leads to state drift.

Imagine a scenario: an autoscaler scales your deployment in response to traffic, or an engineer uses kubectl to debug a pod and changes a label. Terraform knows nothing about these events. Its state file is now a lie, a stale snapshot of a past reality. The next terraform plan will either report unexpected "drift" that an engineer must manually reconcile or, worse, it could blindly destroy and recreate resources, causing an outage because it tries to "fix" a change that was intentional and necessary. This isn't just inefficient; it's dangerous.

Beyond Provisioning: The Application Lifecycle

Terraform excels at the create-configure-destroy lifecycle of foundational infrastructure. But applications are not static. They are living systems that require sophisticated lifecycle management:

• Complex Deployments: How do you orchestrate a canary release or a blue-green deployment with Terraform? These patterns are foreign to its resource-centric model.


• Rollbacks: A rollback in Terraform often means re-running a previous configuration, which can translate to a disruptive destroy-and-recreate cycle. In a Kubernetes-native workflow, a rollback is a rapid, surgical pointer change to a previous ReplicaSet, often completing in seconds.


• Observability: Terraform provides basic logs, but it can't offer the rich, application-aware status reporting, health checks, and event history that a tool like ArgoCD provides directly from the cluster.

The GitOps Paradigm with ArgoCD: A Better Way

This is where GitOps shines. With a tool like ArgoCD, the Git repository becomes the single, unambiguous source of truth.

The architecture is both simple and powerful:

1. Desired State in Git: All manifests (Helm charts, Kustomizations, etc.) for your application are stored in a Git repository.


2. Continuous Reconciliation: The ArgoCD agent runs in your cluster, continuously comparing the live state against the desired state in Git.


3. Automatic Drift Correction: If it detects any deviation—whether from a manual kubectl change or a configuration error—it can automatically self-heal, reverting the cluster to the state defined in Git.

Every change, from a simple image tag update to a full-scale deployment, is managed through a pull request. This workflow provides a natural audit trail, enables peer review for operational changes, and empowers developers to manage their applications' lifecycles using the tools they already know and use.

The Final Frontier: What About Infrastructure Dependencies?

This is where the debate gets interesting. "Okay," you might say, "GitOps for the app, but I still need Terraform to create the app's S3 bucket, IAM role, or Cloud SQL database." This forces teams to manage application dependencies in separate repositories and pipelines, creating a new kind of fragmentation.

This is where we must distinguish between two classes of infrastructure:

1. Core Infrastructure: These are the foundational pillars of your platform. VPCs, Kubernetes clusters themselves, DNS zones, and top-level IAM policies. They have a massive blast radius and a slow change cadence. This is the perfect use case for Terraform. Its deliberate, plan-and-apply workflow is a feature, not a bug, when managing these critical resources.




2. Infrastructure Dependencies: These are resources tightly coupled to a single application. They should be created when the app is deployed and destroyed when it's removed. An S3 bucket for a specific microservice, a Pub/Sub topic for an event-driven workflow, or an IAM role for a single pod are prime examples.

Managing these dependencies with Terraform is clumsy. Why should an application team have to file a ticket or run a separate pipeline just to get a bucket?

Unifying the Stack with Crossplane and GitOps

This is where a tool like Crossplane completes the GitOps picture. Crossplane extends the Kubernetes API, allowing you to manage external resources as if they were native Kubernetes objects.

By installing a Crossplane provider for your cloud, you can define an S3Bucket or RDSPostgreSQLInstance directly in YAML, right alongside your Deployment and Service.

Now, the magic happens. You can place your application's Helm chart and its Crossplane dependency manifests in the same Git repository. ArgoCD, already watching that repo, will deploy everything in one go.

1. ArgoCD applies the manifests.


2. The Kubernetes API server receives the Deployment and the S3Bucket objects.


3. The Kubernetes scheduler deploys the application pods.


4. The Crossplane controller sees the S3Bucket object and provisions the actual bucket in your cloud provider.

The entire application stack, from its cloud dependencies to its Kubernetes configuration, is now a self-contained unit, managed through a single Git repository and a unified GitOps workflow. When you delete the ArgoCD application, it can trigger the deletion of the Kubernetes resources and instruct Crossplane to de-provision the corresponding cloud infrastructure, ensuring a clean, complete teardown.

Conclusion: Use the Right Tool for the Job

Using Terraform to manage Kubernetes applications is not just a matter of preference; it's a fundamental architectural mismatch. It's like using a screwdriver to hammer a nail—you might eventually get it in, but the process will be awkward, and the result will be fragile.

The path to a mature, scalable, and secure Kubernetes platform is clear:

• Use Terraform for what it excels at: Provisioning and managing your stable, core infrastructure.


• Use GitOps with ArgoCD for deploying and managing the dynamic lifecycle of your applications.


• Use Crossplane alongside ArgoCD to manage application-specific infrastructure dependencies, creating truly self-contained and portable application definitions.

By separating these concerns and embracing a cloud-native, GitOps-centric approach, we can build platforms that are not only more powerful but also more resilient, secure, and easier to maintain. It’s time to stop fighting the current and let Kubernetes be Kubernetes.