🛠

Fixing ErrorApplicationGatewayForbidden in AKS + Application Gateway

When running:

• Azure Kubernetes Service
• Azure Application Gateway
• Azure Application Gateway Ingress Controller

You may encounter this error inside AGIC logs:

ErrorApplicationGatewayForbidden
StatusCode=403
Microsoft.Network/applicationGateways/read

This tutorial explains:

1. ✅ Why this happens
2. 🔍 How to diagnose it
3. 🛠 How to fix it properly
4. 🔐 How to prevent it next time

📌 The Error

From AGIC logs:

kubectl logs -n kube-system deploy/ingress-appgw-deployment

You’ll see something like:

Unexpected status code '403' while performing a GET on Application Gateway
AuthorizationFailed

And:

The client '<object-id>' does not have authorization to perform action
'Microsoft.Network/applicationGateways/read'

🧠 Root Cause

AGIC runs using a managed identity.

That identity must have Azure RBAC permissions to:

• Read Application Gateway
• Modify listeners
• Modify backend pools
• Modify routing rules

If those permissions are missing → AGIC cannot configure App Gateway → 403 error.

This usually happens when:

• App Gateway was created manually
• Cross-resource-group setup
• Subscription policies restrict automatic role assignment
• Infrastructure created in separate steps

🏗 How AGIC Actually Works (Important to Understand)

Flow:

Ingress YAML
   ↓
AGIC watches cluster
   ↓
Calls Azure API
   ↓
Modifies Application Gateway config

AGIC is essentially a controller that:

• Talks to Azure ARM API
• Updates App Gateway dynamically

Without RBAC → it cannot call Azure API.

🔍 Step 1 — Identify AGIC Managed Identity

Run:

az aks show \
  --resource-group <RG> \
  --name <AKS_NAME> \
  --query addonProfiles.ingressApplicationGateway.identity.objectId \
  -o tsv

Save the output:

AGIC_OBJECT_ID=<value>

That is the identity failing in the logs.

🔎 Step 2 — Verify Missing Role

Check current assignments:

az role assignment list \
  --assignee $AGIC_OBJECT_ID \
  -o table

You’ll likely see:

• No Contributor on App Gateway
• No Reader on Resource Group

That’s the problem.

🛠 Step 3 — Fix RBAC Properly

✅ 1. Assign Reader on Resource Group

az role assignment create \
  --assignee $AGIC_OBJECT_ID \
  --role Reader \
  --scope /subscriptions/<SUB_ID>/resourceGroups/<RG>

Why?

AGIC reads resource group metadata.

✅ 2. Assign Contributor on Application Gateway

Get App Gateway ID:

APPGW_ID=$(az network application-gateway show \
  --name <APPGW_NAME> \
  --resource-group <RG> \
  --query id -o tsv)

Assign Contributor:

az role assignment create \
  --assignee $AGIC_OBJECT_ID \
  --role Contributor \
  --scope $APPGW_ID

Why Contributor?

AGIC must:

• Update listeners
• Update backend pools
• Update HTTP settings
• Update routing rules

Reader is not enough.

🔄 Step 4 — Restart AGIC

kubectl rollout restart deployment ingress-appgw-deployment -n kube-system

Then check logs again:

kubectl logs -n kube-system deploy/ingress-appgw-deployment

The error should disappear.

You should now see:

Applied App Gateway configuration

🧪 Step 5 — Validate End-to-End

Test your endpoint:

https://<front-door-or-appgw-url>

Traffic should now:

Front Door (optional)
   ↓
Application Gateway
   ↓
AKS

No more retry loop.

🛡 Production Best Practice (Important)

Instead of giving full Contributor on the entire resource group:

🔐 Use Least Privilege

Scope Contributor only to:

/resourceGroups/<RG>/providers/Microsoft.Network/applicationGateways/<APPGW_NAME>

Even better:

Create a custom RBAC role limited to:

Microsoft.Network/applicationGateways/*

For finance / banking / regulated environments.

🚨 Common Variations of This Problem

🧠 Prevention Checklist (Use This Next Time)

When creating AKS with AGIC:

az aks create \
  --enable-addons ingress-appgw \
  --appgw-id <ID>

Immediately after:

1. Get AGIC identity
2. Assign:

• Reader on RG
• Contributor on App Gateway
  1. Verify with az role assignment list

Make this part of your infrastructure checklist.

🏦 Enterprise Architecture Insight

In large organizations:

• Network team owns Application Gateway
• Platform team owns AKS
• RBAC must be explicitly granted

This error is extremely common in enterprise environments.

Understanding it makes you significantly stronger in Azure architecture.

🎯 Final Summary

Problem

AGIC 403 AuthorizationFailed

Cause

Managed identity missing RBAC permissions

Fix

Assign:

• Reader → Resource Group
• Contributor → Application Gateway

Result

AGIC can successfully reconcile Ingress → App Gateway config