Overview
Kubernetes Operators simplify the management of complex applications on Kubernetes. In this guide, we'll walk through creating a simple Kubernetes Operator using the Operator Framework. We'll also cover setting up a local Kubernetes cluster with KIND (Kubernetes in Docker) and deploying the Operator to the KIND cluster.
Note: This guide assumes you know what Kubernetes and Docker are and that you have a Mac or Linux/WSL machine.
Prerequisites
You might want to install the following tools on your machine:
golang:brew install go
docker:brew install docker
kind:brew install kind
kubectx:brew install kubectx
operator-sdk:brew install operator-sdk
kubebuilder:brew install kubebuilder
Here are some extra tools which might be useful in the future:
k9s:brew install k9s
kustomize:brew install kustomize
helm:brew install helm
helmify:brew install arttor/tap/helmify
Creating a Kubernetes Cluster with KIND
Before we can deploy our Operator, we need a Kubernetes cluster to deploy it to. We'll use KIND to create a local Kubernetes cluster.
Create a KIND cluster:
kind create cluster
Configure kubectl to use the KIND cluster:
kind export kubeconfig
Switch to the correct cluster context:
kubectx kind-kind
Verify the cluster is running:
kubectl cluster-info
We now have a minimal local Kubernetes cluster running on our machine. You should be able to use kubectl or k9s to interact with the cluster like any other Kubernetes cluster.
Initializing the Operator Project
We can now build our Operator using the Operator Framework. We'll use the Operator SDK to scaffold a new Operator project and then generate Custom Resource APIs. It is recommended to create a new Git repository for your Operator project and choose a meaningful name for your Operator. You can commit your changes after each CLI command to better understand what the operator is generating.
Create a new Operator project:
operator-sdk init --plugins go/v3 --repo github.com/my-group/my-operator
Create a new Custom Resource Definition (CRD):
operator-sdk create api --group=example --version=v1alpha1 --kind=MyApp
This should leave you with a ready-to-use Operator project scaffolded by the Operator SDK. The three most important directories are:
api: Containing the types for your Custom Resources Definition
controllers: Containing the logic for your Operator
config/samples: Containing sample Custom Resource instances
Implementing the Operator
We'll now implement the Operator logic. The Operator Framework provides a high-level API for writing Operators in Go. This operator will watch for instances of the Custom Resource and create a Config Map for each instance. This is just a simple example to help you get started.
Defining the Custom Resource
Open the file api/v1alpha1/myapp_types.go and add the following code of the MyAppSpec struct:
// MyAppSpec defines the desired state of MyApp
type MyAppSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Name is the name of the config to be created of MyApp.
Name string `json:"name,omitempty"`
}
This will add a Name field to the Custom Resource Spec. We'll use this field to set the name of the Config Map.
We can also add a status field to the Custom Resource. This will be used to store the status of the Custom Resource. To do so, add the following code to the MyAppStatus struct:
// MyAppStatus defines the observed state of MyApp
type MyAppStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
}
To support visual feedback for the users who are using tools like Openlens, we can add a +kubebuilder:printcolumn annotation to the MyApp struct. To do so, add the following code to the MyApp struct:
// MyApp is the Schema for the myapps API
// +kubebuilder:printcolumn:name="Name",type="string",JSONPath=".spec.name",description="The name of the config map to be created"
// +kubebuilder:printcolumn:name="Status",type="string",JSONPath=".status.conditions[?(@.type==\"App\")].reason",description="The status of this resource"
type MyApp struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec MyAppSpec `json:"spec,omitempty"`
Status MyAppStatus `json:"status,omitempty"`
}
Implementing the Controller
Before we can implement the Controller, we need to add a dependency. To do so, open the file controllers/myapp_controller.go and add the following code to the imports:
import (
"context"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log"
examplev1alpha1 "github.com/my-group/my-operator/api/v1alpha1"
)
Open the file controllers/myapp_controller.go and add the following code to the Reconcile function (// TODO(user): your logic here):
// For more details, check Reconcile and its Result here:
// - https://pkg.go.dev/sigs.k8s.io/[email protected]/pkg/reconcile
func (r *MyAppReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
_ = log.FromContext(ctx)
// Get CRD
myApp := &examplev1alpha1.MyApp{}
if err := r.Get(ctx, req.NamespacedName, myApp); err != nil {
if apierrors.IsNotFound(err) {
log.Log.Info("MyApp not found. Ignoring since object must be deleted.")
return ctrl.Result{}, nil
}
log.Log.Error(err, "Failed to get MyApp.")
}
// Start the Reconciliation
conditions := &myApp.Status.Conditions
if len(*conditions) == 0 {
meta.SetStatusCondition(conditions, metav1.Condition{
Type: "App",
Status: metav1.ConditionUnknown,
Reason: "Initializing",
Message: "Starting reconciliation",
})
log.Log.Info("Condition", "Length", len(myApp.Status.Conditions))
if err := r.Status().Update(ctx, myApp); err != nil {
log.Log.Error(err, "Failed to update MyApp status")
return ctrl.Result{}, err
}
// Start the next cycle
return ctrl.Result{}, nil
}
// Act depending on the Condition. This is just a rough example.
currentCondition := (*conditions)[0].Reason
switch currentCondition {
case "Initializing":
// Create a ConfigMap
cm := &corev1.ConfigMap{}
err := r.Get(ctx, types.NamespacedName{Name: myApp.Name, Namespace: myApp.Namespace}, cm)
if err != nil {
if apierrors.IsNotFound(err) {
// No ConfigMap exists and we create one
cmInstance := &corev1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: myApp.Name,
Namespace: myApp.Namespace,
},
}
if err := r.Create(ctx, cmInstance); err != nil {
log.Log.Error(err, "Failed to create a new ConfigMap", "Namespace", myApp.Namespace, "Name", myApp.Name)
return ctrl.Result{}, err
}
} else {
// Some unknown error occurred
log.Log.Error(err, "Failed to get ConfigMap", "Namespace", myApp.Namespace, "Name", myApp.Name)
return ctrl.Result{}, err
}
}
// Update the status
log.Log.Info("Config Map created")
meta.SetStatusCondition(&myApp.Status.Conditions, metav1 Condition{
Type: "App",
Status: metav1.ConditionTrue,
Reason: "Available",
Message: "Config Map created",
})
if err := r.Status().Update(ctx, myApp); err != nil {
log.Log.Error(err, "Failed to update status")
return ctrl.Result{}, err
}
case "Unavailable":
// Retry depending on the error
case "Available":
// Everything is fine
default:
// Set State if State was unknown
meta.SetStatusCondition(conditions, metav1.Condition{
Type: "App",
Status: metav1.ConditionUnknown,
Reason: "Initializing",
Message: "Starting reconciliation",
})
if err := r.Status().Update(ctx, myApp); err != nil {
log.Log.Error(err, "Failed to update myApp status")
return ctrl.Result{}, err
}
}
return ctrl.Result{}, nil
}
The Operator will watch the Custom Resources and react to changes. The Reconcile function will be called for each change. The Reconcile function will check the status of the Custom Resource and act accordingly. In this example, we'll update the status of the Custom Resource and create a Config Map if it doesn't exist.
Creating a sample Custom Resource Instance
To test our implementation we'll create a sample Custom Resource instance. Open the file config/samples/example_v1alpha1_myapp.yaml and add the following code:
apiVersion: example.stammkneipe.dev/v1alpha1
kind: MyApp
metadata:
labels:
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp-sample
app.kubernetes.io/part-of: my-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: my-operator
name: myapp-sample
spec:
name: myapp-config-map
Testing the Operator locally
Deploy the CRDs to the cluster:
make install
Deploy the sample Custom Resource instance to the cluster:
kubectl apply -f config/samples
Start the Operator:
make run
You should now see the Operator's logs in your terminal. The final message should be Config Map created. You can now stop the operator and check your cluster for the Config Map:
kubectl get configMap myapp-sample -o yaml
Conclusion
Congratulations! You've successfully built a Kubernetes Operator using the Operator Framework on a local KIND cluster. You can extend this example by adding more features to your Operator and exploring advanced Operator Framework capabilities. I encourage you to play around with the Operator Framework and explore the possibilities of Operators on Kubernetes.
As you might have noticed, this Tutorial does not include any tests, packaging, or deploying the Operator to a real Kubernetes cluster. I'll cover these topics in future guides. So stay tuned!
SOCIAL SHARE CARD GENERATOR