QAI Agent now does more than cluster failures and score PR risk. It alerts your team in Slack, explains why tests failed with AI-powered RCA, and generates the fix — all without leaving the PR.
A few weeks ago I wrote about QAI Agent — a GitHub Action that clusters test failures and scores PR risk.
The feedback was clear: clustering is useful. But developers wanted more. Not just what broke — why it broke, and how to fix it.
So we closed the loop.
Here's what happens now when your CI fails.
Step 1 — Slack finds you
You don't check CI. CI comes to you.
The moment a high-risk PR is detected, your team gets a Slack alert:
🔴 High Risk PR #28 — Do not merge
Risk: 0.60 · 8 test failures · 8 clusters
Recommendation: investigate failures first
No polling. No tab switching. It just appears in your team channel.
Step 2 — The PR comment tells you everything
You click "View in QAI Platform." The PR already has a full breakdown:
- 8 failures listed with exact errors
- 8 unique failure clusters identified
- RCA Analysis table — cause, confidence score, suggestion
- 💡 AI fix suggestions available
The RCA table is new. For each Playwright trace, QAI detects the failure category and confidence.
Rule-based detection, no cloud required. It runs locally on the GitHub Actions runner.
Step 3 — AI generates the fix
Click "View in QAI" → open the failing test → hit "Suggest fix."
The AI explains exactly what went wrong:
The price locator matches multiple elements (paragraph and span both showing $54.95), causing a strict mode violation that prevents the visibility check.
Then generates the fix:
await expect(page.locator('span').filter({ hasText: /\$\d+/ })).toBeVisible();
One click. Ready to apply.
The full loop
CI fails
→ Slack alert fires to your team channel
→ PR comment posts: clusters + RCA + confidence scores
→ AI fix suggestion on demand
→ Merge verdict: go or no-go
Before QAI: open CI → read logs → guess → fix → repeat.
After QAI: open Slack → click link → apply fix → merge.
Setup — still one step
Nothing changed on the setup side. Add one step after your tests run:
- name: QAI Agent
uses: useqai/qai-agent@v1
if: always()
with:
junit-path: 'test-results/results.xml'
trace-path: 'test-results/**/*.zip' # optional, enables RCA
For Slack alerts and AI fix suggestions, connect the cloud platform with two more lines:
qai-url: https://ingest.useqai.dev
qai-api-key: ${{ secrets.QAI_API_KEY }}
Get your free API key at useqai.dev — 30 seconds, no credit card.
GitHub Action — fully open source:
- PR comment with risk score ✅
- Failure clustering ✅
- Playwright trace RCA (rule-based, runs locally) ✅
- Block merges on high risk ✅
Cloud platform (useqai.dev):
- AI fix suggestions (LLM-powered) ✅
- Slack alerts for high-risk PRs ✅
- Historical trends + flakiness tracking ✅
- Cross-repo visibility ✅
Try it
🔧 GitHub Action: useqai/qai-agent on the Marketplace
📦 Source: github.com/useqai/qai-agent
📊 Dashboard: useqai.dev
💬 Live PR comment demo: PR #2
If you try it and hit any issues — open an issue or drop a comment here. Especially interested in non-Playwright frameworks and edge cases in JUnit parsing.