No one knows what regression covers.
So every release is a judgment call.
AI-powered, Human-led Quality
Stop hearing about bugs
from your customers.
Your quality partner. Our AI agents map your microservices, test every PR and check production after every deploy. Senior Quality engineers lead them and answer for what slips through.
An example workspace. Click around the sidebar to explore it.
Quality this week
2 of 3 releases shipped. One was held back before customers saw a double charge.Release 2026.41.2 is blocked on a critical billing bug. The fix is in review.
Bugs reported by customers2 last 30 dayswas 24 in the 30 days before
Full regression14 minwas 3 days, manual
Time to root cause1.5 dayswas 9 days
Releases this month110 rollbacks
Bugs reported by customers, per week
W34W35W36W37W38W39W40W41
BeforeWith Qualityguage
Needs your attention
Double-charge risk in 2026.41.2Caught in regression. Release held.
3 of 9 refund flows untestedHighest-risk gap this quarter
Staging ledger is 2 versions behind prodTest results for ledger are unreliable
Release 2026.41.2
Blocked · 1 critical
Release
readiness
readiness
Regression run #12871,184 / 1,1936 failed · 3 quarantined
Run time14m 22s24 parallel runners
Critical flows covered47 / 5881% · +5 this release
Tests added this release41from 37 PRs + 2 incidents
Release checks2 failing · 1 warning
PR checks37 of 37 PRs reviewed
0 openContract tests
Refund.amount type changed for 2 consumersDaily regression6 failures, 1 critical
Performance smokePOST /v1/charge p95 over budget
Prod sanity on 2026.41.1last run 18 min ago
22 / 22Recent releases
2026.41.2Blocked · double-charge risk
today2026.41.1Shipped · 22 / 22 prod checks
10 Oct2026.41.0Shipped · 1 flaky test quarantined
8 Oct2026.40.3Shipped · hotfix for INC-221
7 OctService map
Click a service● finding ● drift or gap
Test runs
| Run | Started | Trigger | Passed | Failed | Time |
|---|---|---|---|---|---|
| #1287 | Today 06:00 | Nightly | 1,184 / 1,193 | 6 | 14m 22s |
| #1286 | 12 Oct 18:52 | Release cut 2026.41.2 | 1,186 / 1,193 | 4 | 14m 09s |
| #1285 | 12 Oct 06:00 | Nightly | 1,152 / 1,152 | 0 | 13m 41s |
| #1284 | 11 Oct 06:00 | Nightly | 1,150 / 1,152 | 2 | 13m 55s |
Quarantined flaky tests3 · each has an owner and a fix date
payouts › bank_webhook_timeoutShared bank sandbox times out under load · owner: Payouts · fix by 15 Oct
since 9 Octonboarding › kyc_upload_large_fileTest KYC documents expired in staging · owner: Qualityguage · fix by 14 Oct
since 10 Octnotify › sms_delivery_callbackSMS provider sandbox rate-limits parallel runs · owner: Platform · fix by 16 Oct
since 11 OctOpen findings
Critical · Race conditionfound 11:42 IST
Customers can be charged twice when billing-svc retries a slow debit
payments/charges › charge_retry_after_timeout
✕ expected 1 ledger entry for idempotency_key ch_7Hq2Lx, found 2
reproduced 9 / 10 runs from prod trace 7f3a…e21
api-gateway
2,412 ms
charge-svc
214 ms
billing /debit
2,004 ms
billing retry
96 ms
ledger-svc
2 writes
Root cause: the retry after a 2,000 ms timeout drops the Idempotency-Key header. Introduced in PR #2291 (a41f9c2).
Business impact: ~0.4% of charges at peak loadJira PAY-1873 · owner: BillingTest TC-1841 added
High · Breaking API changefound in PR #2318
Changing Refund.amount to decimal breaks two services that still read it as an integer
contracts › refunds-svc → ledger-svc
✕ ledger-svc rejects amount "1250.50": expected integer (paise)
contracts › refunds-svc → email-receipts
✕ receipt shows ₹125050.00 instead of ₹1,250.50
The refunds team didn’t know these two services consume the field. The service map showed both. Agreed fix: add amount_minor and deprecate the old field over two releases.
Caught at PR review, before mergeOwners: Payments, Ledger, Growth4 contract tests added
Medium · Performanceperf smoke, release cut
POST /v1/charge p95 rose from 341 ms to 412 ms. The budget is 380 ms.
perf › charge_p95_under_budget (200 rps, 5 min)
✕ p95 412 ms > budget 380 ms (2026.41.1: 341 ms)
slowest span: charge-svc → fraud rules lookup, +58 ms
A new fraud rule runs an unindexed query on every charge. Suggested fix: add an index on rules(merchant_id, active).
Not a release blocker on its ownOwner: PaymentsBudget check runs on every release
INC-221 · Refunds stuck in “pending”
Timeline
Kafka cluster rebalances during routine broker upgrade
First customer ticket: refund not received
On-call paged after 9 tickets. No alert had fired.
Consumer restarted, backlog drained
Root cause reproduced in the QA environment
Fix shipped in 2026.40.3, tests added
Root cause
refunds-svc commits its Kafka offset before writing to the ledger. During a rebalance, in-flight messages were marked done but never written.
Why tests missed it: staging runs 1 partition, production runs 12. A rebalance never happened in staging.
So it can’t happen again
TC-1822 · rebalance during refundsRuns daily on a 12-partition QA cluster
passingAlert on refund consumer lagPages when lag exceeds 2 minutes
liveQA Kafka now matches prod12 partitions, same broker version
doneCoverage of critical business flows
| Business area | Covered flows | Risk of the gaps | |
|---|---|---|---|
| Payments | 18 / 20 | Medium | |
| Refunds | 6 / 9 | High | |
| Payouts | 8 / 11 | Medium | |
| Onboarding and KYC | 10 / 12 | Low | |
| Notifications | 5 / 6 | Low |
Next to cover, highest risk firstplanned this week
Partial refund after a chargebackMoney can leave twice · seen in 2 support tickets
MonRefund to an expired cardFalls back to a manual process with no test
TueMulti-currency refundFX rate taken at refund time, not charge time
WedYou already know the signs.
If three of these sound familiar, keep reading.
ConfidenceNobody can say how safe a release really is.
Half the suite is flaky.
So nobody trusts a red build, and real failures get ignored.
Regression takes hours.
So it’s the first thing skipped when a deadline gets close.
Environments and knowledgeTesting happens somewhere that isn’t production, by people who can’t see the whole product.
Staging is nothing like production.
Different versions, data and scale. Green in QA proves little.
Each team knows its own service.
Nobody knows the whole product path a customer request takes.
No Swagger, no API docs.
Every new test starts with a Slack thread or a Google search.
Speed and scaleCode ships faster every quarter. Testing doesn’t.
AI tools doubled your pull requests.
Testing stayed the same size.
Load tests almost never run.
No performance baseline, so nobody knows if this release is slower than the last.
ProductionWhen something breaks, finding out why takes longer than fixing it.
The prod bug won’t reproduce.
So it’s closed, and a month later it comes back.
Nobody can trace a request across services.
Logs live in ten places, so root cause takes days of guessing.
And every postmortem ends with the same question.
“Why didn’t QA catch this?”
These are system problems. We fix the system.
For everyone who signs off on a release.
Every leader asks a different question about quality. Pick yours and see exactly what lands on your desk.
Founder · CEO
“Will the next release embarrass us in front of a customer?”
- Judged on
- What they expect
- A straight answer every Monday: is the product getting better or worse?
- What Qualityguage delivers
- A one-page report in plain English. What customers hit, what we stopped, and what’s at risk next.
Weekly quality report · to: founders@acmeMon 09:00
Week 41: one serious bug stopped before release.
Customer-reported bugs0was 3 a week in Aug
Releases shipped2 of 31 held for safety
Downtime0 min63 min in W40
Stopped: a retry bug that would have charged some customers twice
At risk: refunds after chargebacks aren’t tested yet. Covered by Wednesday.
Fixed for good: last week’s stuck refunds can no longer recur
CTO · VP Engineering
“Can we ship faster without breaking production?”
- Judged on
- What they expect
- A go or no-go on every release, with the reason, before anyone deploys.
- What Qualityguage delivers
- An automatic release gate in Slack and CI. Every block comes with the failing test, the root cause and the owner.
#releases · SlackToday 11:46
Q
QualityguageAPP
🔴 Release 2026.41.2 is blocked. 1 critical finding.
Cause: PR #2291 by the Billing team. Fix in review. Reviewed by a Senior Quality engineer.
Double charge when billing retries a slow debitQG-412
PR checks · 37 of 37pass
Regression · 1,184 of 1,19314m 22s
Perf smoke · p95 412 ms, budget 380warn
View findingOverride with reason
Head of QA · QA Director
“How do I cover 40 services with a team of five?”
- Judged on
- What they expect
- Automation that keeps pace with development, a suite the team trusts, and a map of the whole product.
- What Qualityguage delivers
- Agents that write, run and maintain tests under your team’s review. Flaky tests fixed, and coverage measured per business flow to back every sign-off.
Suite health · acme-paymentsWeek 41
Automation backlog38was 212 in Aug
Flaky tests3quarantined, owned
Escaped defects1RCA done
Payments · 18 / 20 flows90%
Refunds · 6 / 9 flows67%
Payouts · 8 / 11 flows73%
Engineering managers · Tech leads
“What will this change break that my team can’t see?”
- Judged on
- What they expect
- To know what a PR breaks in other teams’ services before merge, not in the incident channel.
- What Qualityguage delivers
- A review on every pull request: the services it affects, the missing tests written for you, and a clear pass or fail.
PR #2318 · Refund amount as decimalGitHub
Q
qualityguage-botreviewed
⚠️ This change affects 2 services outside your team.
- amount: int // paise
+ amount: decimal // rupees
ledger-svc still parses it as an integerLedgeremail-receipts shows ₹125050.00Growth4 contract tests added to this PR1 failing
See suggested fixNotify owners
Head of Product
“Will the launch date hold, and will the feature actually work?”
- Judged on
- What they expect
- Test status per feature, in product language, a week before launch, not the night before.
- What Qualityguage delivers
- A launch readiness view. Every acceptance criterion is mapped to a test and marked proven, failing or pending.
Launch readiness · Partial refundsLaunch: 21 Oct
11 of 14 acceptance criteria tested79%
Refund part of an order to the original cardpassing
Customer gets an email with the refunded amountpassing
Partial refund after a chargebackfailing
Refund to an expired cardtest due Tue
Refund in a different currencytest due Wed
SRE · Support
“Will this incident happen again next month?”
- Judged on
- What they expect
- A root cause they can trust and proof the fix holds. Not another “cannot reproduce”.
- What Qualityguage delivers
- Every incident reproduced from production traces, root-caused across services, and locked in with a test that runs daily.
INC-221 · Refunds stuck in pendingRCA · 8 Oct
Customers affected1,91214 tickets
Root cause found1 dayreproduced in QA
Can it recur?NoTC-1822 runs daily
Offset committed before the ledger write. Lost during a Kafka rebalance.
Missed because staging had 1 partition, prod has 12
QA Kafka now matches prod. Lag alert added.
Two keys. One of them is yours.
The Qualityguage orchestrator runs in your cloud account. Opening it takes your key and ours. Your code and services stay under your team’s normal access.
Runs in your account.
The orchestrator, with its agents and runners, is deployed in your cloud. Your source code and customer data never leave it.
Nobody acts alone.
Every access to the orchestrator needs both keys. We can’t get in without you, and nobody can change it without us.
Never used for training.
We don’t train or fine-tune any model on your code, traffic or data.
No lock-in.
Tests are written in standard open-source frameworks and live in your repositories from day one.
We sign up for outcomes. Not hours.
Five measures, agreed with you before we start and reported to leadership every Monday.
of critical business flows tested end to end
Measured against the live service map, not a spreadsheet.
flaky tests in the regression suite
Every flaky test is quarantined, owned and fixed by a set date.
from push to a clear pass or fail
Including which services outside the team the change affects.
from a merged fix or incident to its regression test
So coverage grows with every change, not once a quarter.
production bugs that come back
Every escaped bug gets a root cause and a test that runs daily.
Example targets. Your numbers are set from your own baseline when we start.
Quality that keeps customers.
A live map of your services and a baseline for all five measures.