Lädt...


🔧 Terraform stories.


Nachrichtenbereich: 🔧 Programmierung
🔗 Quelle: dev.to

SAP Kyma with dynamic OIDC credentials with HCP Terraform

HCP Terraform already supports dynamic credentials with Kubernetes providers with AWS and GCP platforms.

I have extended this support to SAP BTP, Kyma runtime clusters with SAP Business Technology Platform.

Let's see how...

1. Configure Kubernetes

Configure an OIDC identity provider with SAP Kyma cluster.

SAP Kyma supports the gardener's oidc-shoot-extension, thus, effectively allowing for an arbitrary number of OIDC providers in a single shoot cluster.

This has to be done upfront during the kyma cluster bootstrapping phase.

OpenIDConnect_HCP

locals {
  OpenIDConnect_HCP = jsonencode({
        "apiVersion": "authentication.gardener.cloud/v1alpha1",
        "kind": "OpenIDConnect",
        "metadata": {
            "name": "terraform-cloud"
        },
        "spec": {
            "issuerURL": "https://app.terraform.io",
            "clientID": "terraform-cloud",
            "usernameClaim": "sub",
            "usernamePrefix": "-",
            "groupsClaim": "terraform_organization_name",
            "groupsPrefix": ""
        }
  })
}
resource "terraform_data" "bootstrap-tfc-oidc" {
  triggers_replace = {
    always_run = "${timestamp()}"
  }

  # the input becomes a definition of an OpenIDConnect provider as a non-sensitive json encoded string 
  #
  input = [ 
      nonsensitive(local.OpenIDConnect_HCP) 
      ]

 provisioner "local-exec" {
   interpreter = ["/bin/bash", "-c"]
   command = <<EOF
     (
    KUBECONFIG=kubeconfig-headless.yaml
    NAMESPACE=quovadis-btp
    set -e -o pipefail ;\
    curl -LO https://dl.k8s.io/release/v1.31.0/bin/linux/amd64/kubectl
    chmod +x kubectl

    while ! ./kubectl get crd openidconnects.authentication.gardener.cloud --kubeconfig $KUBECONFIG; 
    do 
      echo "Waiting for OpenIDConnect CRD..."; sleep 1; 
    done
    ./kubectl wait --for condition=established crd openidconnects.authentication.gardener.cloud --timeout=480s --kubeconfig $KUBECONFIG
    crd=$(./kubectl get crd openidconnects.authentication.gardener.cloud --kubeconfig $KUBECONFIG -ojsonpath='{.metadata.name}' --ignore-not-found)
    if [ "$crd" = "openidconnects.authentication.gardener.cloud" ]
    then
      OpenIDConnect='${self.input[0]}'
      echo $(jq -r '.' <<< $OpenIDConnect)
      echo $OpenIDConnect

      echo | ./kubectl get nodes --kubeconfig $KUBECONFIG
      ./kubectl create ns $NAMESPACE --kubeconfig $KUBECONFIG --dry-run=client -o yaml | ./kubectl apply --kubeconfig $KUBECONFIG -f -
      ./kubectl label namespace $NAMESPACE istio-injection=enabled --kubeconfig $KUBECONFIG

      # a debug line until the OpenIDConnect CRD is installed via the oidc shoot extension
      #
      echo $(jq -r '.' <<< $OpenIDConnect ) >  bootstrap-kymaruntime-bot.json
      echo $OpenIDConnect | ./kubectl apply --kubeconfig $KUBECONFIG -n $NAMESPACE -f - 

    else
      echo $crd
    fi

     )
   EOF
 }
}

As a result, the following OpenIDConnect CR will become available in your kyma cluster.

OpenIDConnect

The OIDC identity resolves the authentication requests to the Kubernetes API. However, it must be first authorised to interact with the cluster API.

In order to do so, one must create custom cluster roles to the terraform OIDC identity in the kyma cluster with either "User" and/or "Group" subjects.

For OIDC identities coming from TFC (HCP Terraform), the role binding "User" value is formatted as follows:

organization:<MY-ORG-NAME>:project:<MY-PROJECT-NAME>:workspace:<MY-WORKSPACE-NAME>:run_phase:<plan|apply>.

I have opted for generating these RBAC identities in the initial kyma cluster terraform configuration, thus, adding both plan and apply phase identities to the initial kyma runtime environment configuration as administrators.

User identities

/ https://developer.hashicorp.com/terraform/cloud-docs/run/run-environment#environment-variables
//
variable "TFC_WORKSPACE_NAME" {
  // HCP Terraform automatically injects the following environment variables for each run. 
  description = "The name of the workspace used in this run."
  type        = string
}

variable "TFC_PROJECT_NAME" {
  // HCP Terraform automatically injects the following environment variables for each run. 
  description = "The name of the project used in this run."
  type        = string
}

variable "TFC_WORKSPACE_SLUG" {
  // HCP Terraform automatically injects the following environment variables for each run. 
  description = "The slug consists of the organization name and workspace name, joined with a slash."
  type        = string
}

// organization:<MY-ORG-NAME>:project:<MY-PROJECT-NAME>:workspace:<MY-WORKSPACE-NAME>:run_phase:<plan|apply>.
locals {
  organization_name = split("/", var.TFC_WORKSPACE_SLUG)[0]
  user_plan = "organization:${local.organization_name}:project:${var.TFC_PROJECT_NAME}:workspace:${var.TFC_WORKSPACE_NAME}:run_phase:plan"
  user_apply = "organization:${local.organization_name}:project:${var.TFC_PROJECT_NAME}:workspace:${var.TFC_WORKSPACE_NAME}:run_phase:apply"
}

This way, as soon as the kyma runtime environment has been provisioned, the required identities are in place in the kyma cluster.

After the kyma cluster has been bootstrapped with the HCP Terraform’s OIDC provider in place, one can bind RBAC roles to groups.

Group identity

resource "kubernetes_cluster_role_binding_v1" "oidc_role" {
  //depends_on = [ <list of dependencies> ] 

  metadata {
    name = "terraform-identity-admin"
  }
  //
  // Groups are extracted from the token claim designated by 'rbac_group_oidc_claim'
  //
  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "ClusterRole"
    name      = "cluster-admin"
  }
  subject {
    api_group = "rbac.authorization.k8s.io"
    kind      = "Group"
    name      = var.tfc_organization_name
    namespace = ""
  }  
}

Role bindings

2. Configure HCP Terraform

Required Environment Variables

HCP Terraform will require these two environment variables to enable kubernetes dynamic credentials

Variable Value Notes
TFC_KUBERNETES_PROVIDER_AUTH TFC_KUBERNETES_PROVIDER_AUTH[_TAG] true Must be present and set to true, or HCP Terraform will not attempt to authenticate to Kubernetes.
TFC_KUBERNETES_WORKLOAD_IDENTITY_AUDIENCE TFC_KUBERNETES_WORKLOAD_IDENTITY_AUDIENCE[_TAG] TFC_DEFAULT_KUBERNETES_WORKLOAD_IDENTITY_AUDIENCE The audience name in your cluster's OIDC configuration, such as kubernetes.

You can set these as workspace variables, or if you’d like to share one Kubernetes role across multiple workspaces, you can use a variable set.

3. Configure the provider

HCP Terraform will assign the tfc_kubernetes_dynamic_credentials variable the kubeconfig token valid for 90 minutes.

tfc_kubernetes_dynamic_credentials

 variable "tfc_kubernetes_dynamic_credentials" {
  description = "Object containing Kubernetes dynamic credentials configuration"
  type = object({
    default = object({
      token_path = string
    })
    aliases = map(object({
      token_path = string
    }))
  })
}

output "kube_token" {
  sensitive = true
  value = file(var.tfc_kubernetes_dynamic_credentials.default.token_path)
}

provider configuration

terraform {
/**/ 
  cloud {
    organization = "<organization>"


    workspaces {
      project = "terraform-stories"
      tags = ["runtime-context"]      
    }
  }
/**/ 
  required_providers {
    btp = {
      source  = "SAP/btp"
    }    
    # https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs
    kubernetes = {
      source  = "hashicorp/kubernetes"
    }
    # https://registry.terraform.io/providers/alekc/kubectl/latest/docs
    kubectl = {
      source  = "alekc/kubectl"
      //version = "~> 2.0"
    }
  }
}
provider "kubernetes" {
 host                   = var.cluster-endpoint-url
 cluster_ca_certificate = base64decode(var.cluster-endpoint-ca)
 token                  = file(var.tfc_kubernetes_dynamic_credentials.default.token_path)
}

provider "kubectl" {
 host                   = var.cluster-endpoint-url
 cluster_ca_certificate = base64decode(var.cluster-endpoint-ca)
 token                  = file(var.tfc_kubernetes_dynamic_credentials.default.token_path)
 load_config_file       = false

}

One can retrieve both host and cluster_ca_certificate from the kyma cluster kubeconfig as follows:

kyma cluster kubeconfig

locals {
  labels = btp_subaccount_environment_instance.kyma.labels
}

data "http" "kubeconfig" {

  depends_on = [btp_subaccount_environment_instance.kyma]

  url = jsondecode(local.labels)["KubeconfigURL"]

  lifecycle {
    postcondition {
      condition     = can(regex("kind: Config",self.response_body))
      error_message = "Invalid content of downloaded kubeconfig"
    }
    postcondition {
      condition     = contains([200], self.status_code)
      error_message = self.response_body
    }
  } 

}

# yaml formatted default (oid-based) kyma kubeconfig
locals {
  kubeconfig = data.http.kubeconfig.response_body

  cluster_ca_certificate = base64decode(local.kubeconfig.clusters[0].cluster.certificate-authority-data)
 host                   = local.kubeconfig.clusters[0].cluster.server
}

4. Examples

kyma cluster shoot_info

data "kubernetes_config_map_v1" "shoot_info" {
  metadata {
    name = "shoot-info"
    namespace = "kube-system"
  }
}

output "shoot_info" {
  value =  jsondecode(jsonencode(data.kubernetes_config_map_v1.shoot_info.data))
}
shoot_info = {
        domain            = "<shootName>.kyma.ondemand.com"
        extensions        = "shoot-auditlog-service,shoot-cert-service,shoot-dns-service,shoot-lakom-service,shoot-networking-filter,shoot-networking-problemdetector,shoot-oidc-service"
        kubernetesVersion = "1.30.6"
        maintenanceBegin  = "200000+0000"
        maintenanceEnd    = "000000+0000"
        nodeNetwork       = "10.250.0.0/16"
        nodeNetworks      = "10.250.0.0/16"
        podNetwork        = "100.64.0.0/12"
        podNetworks       = "100.64.0.0/12"
        projectName       = "kyma"
        provider          = "azure"
        region            = "westeurope"
        serviceNetwork    = "100.104.0.0/13"
        serviceNetworks   = "100.104.0.0/13"
        shootName         = "<shootName>"
    }
...

🔧 What are user stories? and How to Create User Stories.


📈 20.78 Punkte
🔧 Programmierung

📰 „Monster Hunter Stories“ und „Monster Hunter Stories 2: Wings of Ruin“ im Kurztest


📈 20.78 Punkte
📰 IT Nachrichten

📰 How to View Snapchat Stories Without them Knowing: Snapchat Stories Viewer


📈 20.78 Punkte
📰 IT Security Nachrichten

🎥 My Life in Short/Shirt Stories - The Time I Learned PenSpinning (~2007-2009) - Shirt Stories #1


📈 20.78 Punkte
🎥 IT Security Video

📰 Download Instagram Stories- Methods to Download Instagram Stories


📈 20.78 Punkte
📰 IT Security Nachrichten

📰 Instagram Stories Hits 150M Active Users, Adds Advertising To Instagram Stories


📈 20.78 Punkte
📰 IT Security Nachrichten

📰 Instagram's New Stories Are a Near-Perfect Copy of Snapchat Stories


📈 20.78 Punkte
📰 IT Security

📰 Instagram Stories Hits 150M Active Users, Adds Advertising To Instagram Stories


📈 20.78 Punkte
📰 IT Security Nachrichten

📰 Instagram's New Stories Are a Near-Perfect Copy of Snapchat Stories


📈 20.78 Punkte
📰 IT Security

🔧 Terraform stories.


📈 20.66 Punkte
🔧 Programmierung

🔧 Terraform Session 3: Let's Learn about Terraform State, Variables and Functions


📈 20.54 Punkte
🔧 Programmierung

🔧 Terraform-CodeGen0: A Terraform Code Generator


📈 20.54 Punkte
🔧 Programmierung

🔧 Understanding the Terraform Lifecycle: A Key Concept for HashiCorp Terraform Associate Exam


📈 20.54 Punkte
🔧 Programmierung

🔧 Understanding the Terraform Lifecycle: A Key Concept for HashiCorp Terraform Associate Exam


📈 20.54 Punkte
🔧 Programmierung

🔧 Terraform Remote Backend: How to Manage Terraform State File for Easier Collaboration across Teams


📈 20.54 Punkte
🔧 Programmierung

🔧 Terraform Tactics: A Guide to Mastering Terraform Commands for DevOps


📈 20.54 Punkte
🔧 Programmierung

🔧 Terraform - Using GitHub Copilot Chat with Terraform


📈 20.54 Punkte
🔧 Programmierung

🔧 Understanding Terraform: part 1 – What is Terraform?


📈 20.54 Punkte
🔧 Programmierung

🔧 Top Stories from the Microsoft DevOps Community – 2019.06.28


📈 10.39 Punkte
🔧 Programmierung

📰 Battlefield 1: Verschiedene spielbare Charaktere und Stories


📈 10.39 Punkte
📰 IT Nachrichten

📰 Lessons From Six Software Rewrite Stories


📈 10.39 Punkte
📰 IT Security Nachrichten

📰 Instagram Stories: Facebook lässt sich von Snapchat inspirieren


📈 10.39 Punkte
📰 IT Nachrichten

📰 Monster Hunter Rise im Crossover mit Monster Hunter Stories 2


📈 10.39 Punkte
📰 IT Nachrichten

📰 Capcom's Monster Hunter Stories 2: Wings of Ruin Gets a Release Date


📈 10.39 Punkte
📰 IT Security Nachrichten

📰 Vorbild Snapchat: Facebook-App mit "Stories" und Kamera-Effekten


📈 10.39 Punkte
📰 IT Nachrichten

📰 Instagram testet Stories nur für kleine Gruppen


📈 10.39 Punkte
📰 IT Nachrichten

🔧 Top Stories from the Microsoft DevOps Community – 2019.08.02


📈 10.39 Punkte
🔧 Programmierung

🔧 #WeArePlay | Meet George from the UK. More stories from Croatia, USA and Kenya.


📈 10.39 Punkte
🔧 Programmierung

📰 Stories from the SOC – Persistent malware


📈 10.39 Punkte
📰 IT Security Nachrichten

🍏 Stories Financial Analysts Tell – TMO Daily Observations 2023-03-07


📈 10.39 Punkte
🍏 iOS / Mac OS

matomo