An AI agent recommended the same fix four times.

Confident. Well-reasoned. Impossible.

Lab · Microsoft Dublin · 10 Nov 2026 · 90 min show and tell

Letting AI operate production Azure: what it gets wrong

Damian Flynn · Microsoft MVP, Azure · Innofactor

Damian Flynn

Damian Flynn

Practice Lead & Principal Consultant (Azure), Innofactor

30+ years in enterprise IT · cloud since 2009 · Mayo, Ireland
Azure Infrastructure as Code · Azure Networking · 17 years an MVP

Microsoft Most Valuable Professional

AI doesn't fail loudly.

It fails plausibly.

Six incidents. Same steps each time.

1

The story

What happened in production. Customers anonymised.

2

The trap

The agent's mistake, replayed from a real session.

3

The guardrail

A real check, run live, catching it.

4

The rule

One line you can take home.

You launch Claude and fire away. Where does the guardrail fire?

AInstructionsCLAUDE.md / AGENTS.md. Advice. The model reads it and can still skip it.
BHooks.claude/settings.json. Claude Code runs them on every tool call, not the model. Exit 2 blocks the call and tells the model why.
CPlatformBranch protection, disabled pipelines, PR checks. Holds even when the agent isn't Claude.

Not a Claude trick. Every harness has a Layer B.

HarnessWhere it livesEventHow it blocks
Claude Code.claude/settings.jsonPreToolUseexit 2
GitHub Copilot CLI.github/hooks/*.jsonpreToolUse"permissionDecision": "deny"
OpenAI Codex.codex/hooks.jsonPreToolUsedeny
Gemini CLI.gemini/settings.jsonBeforeToolexit 2
Pi.pi/extensions/*.tspi.on("tool_call"){ block: true }
OpenCode.opencode/plugins/*.tstool.execute.beforethrow

Same guardrail script, six doors. Seen blocking live this week: 5 of 6. Untrusted folder? The hook silently doesn't fire.

The six

1

Empty is not unset

2

Green is not working

3

The world moved while I was thinking

4

Telephone

5

Who reviews the reviewer?

6

A freeze is a promise, not a lock

Incident 1 / 6

Empty is not unset

Recommended 4×.
Settable: never.
Lives in: hook · PreToolUse on writes  ·  → ./present.sh 1
Incident 1 · the scene

Where the agent looked, and where the setting really lives

Staff laptopsAzure VPN Client, Entra sign-in✓ the real fix: client profile
P2S
OpenVPN
VPN gatewayvpnClientConfiguration✗ vpnClientDnsServers: not a member
Hub networkinternal DNS resolvers
The agent read the gateway, got (empty), and recommended setting it. Four times.
In the sandbox: a VNet takes the same read/write test (one real member, one invented), and the published API spec is checked for the gateway's members.
Incident 2 / 6

Green is not working

Sev 1 alert fired.
Receivers: 0.
Lives in: hook · PostToolUse + a scheduled check  ·  → ./present.sh 2
Incident 2 · the scene

Three green lights, three dead systems

Bootstrap0 changed · 0 failed✗ no gate could run
presence
≠ capability
Import loopfailed 0✗ registry holds 1 of 6
VM data disk/healthz → 200✗ mount dead 2h16m
Sev 1 alertfired✓ detection worked
notify
Action groupreceivers: 0✗ nobody told
On-call—
In the sandbox: a fixture import loop that prints failed 0, and a real alert rule pointing at a real action group with zero receivers.
Incident 3 / 6

The world moved while I was thinking

Branch printers down.
~15 minutes.
Lives in: hook · PreToolUse on changes  ·  → ./present.sh 3
Incident 3 · the scene

An operator was mid-change. The agent was reading an old map.

Branch officesprinters, SD-WAN
Old hubpeering being removed
migrating
New hubfirewall + gateway
Spokeroute table: propagation off
T0 agent reads state  ·  T1 operator finishes a later phase  ·  T2 agent advises from T0: "turn propagation on"  →  traffic bypasses the firewall  →  printers down ~15 min
In the sandbox: two VNets and a route table. A script plays the operator and swaps the peering while the agent's advice is on screen.

Can my team share this? Yes. It's just files.

personal

Your machine

~/.claude/ · settings.local.json

Your experiments. Nobody else gets them, and nobody reviews them.

team

The repo

AGENTS.md · .claude/settings.json · hooks/ · skills/

Committed and reviewed in a PR, like code. Everyone who clones gets the same guardrails.

organisation

Managed policy

managed settings · requirements.toml · a shared plugin

Pushed by MDM or a template repo. Users can't switch it off.

Context travels the same way: conventions in AGENTS.md, runbooks and incident notes in the repo. Secrets never.

Incident 4 / 6

Telephone

One swapped pair.
Three tickets.
Lives in: hook · PreToolUse on publishing  ·  → ./present.sh 4
Incident 4 · the scene

Every check was right. The retelling wasn't.

Storage backendmeasured✓ network 464,013 B · platform 80,207 B
logged
Session logcorrect order✓
restated
from memory
Three ticketsnetwork 80,207 · platform 464,013✗ pair swapped
In the sandbox: fixture files only: the source table and the tickets an agent wrote from it. Nothing touches a real ticket system.
Incident 5 / 6

Who reviews the reviewer?

HIGH: blocker.
Actually: by design.
Lives in: platform · PR check  ·  → ./present.sh 5
Incident 5 · the scene

A second model checks the first. Who checks the second?

Author agentwrites a Bicep change
PR
Second modelCopilot · CodexHIGH / MEDIUM / LOW
findings
Verifyeach HIGH cites file:line; grep it✓ CONFIRMED / ✗ REFUTED
Mergeonly confirmed blockers block
In the sandbox: a small Bicep change with one real defect and one documented design choice, reviewed this week by Copilot (twice) and Codex.
Incident 6 / 6

A freeze is a promise, not a lock

Default route for
79 subscriptions.
Lives in: platform · not the agent  ·  → ./present.sh 6
Incident 6 · the scene

Two doors into one firewall. Only one was supposed to be open.

New codeTerraform✓ owns the firewall now
plan/apply
Hub firewalldefault route for 79 subscriptions
 
79 subscriptionsall egress
Old codeBicep · 'FREEZE' agreed✗ still mergeable
on merge
Old pipelineanother project · enabled✗ ran 5× the day before
deploys
Same firewallresurrects deleted rules
In the sandbox: a GitHub repo with two 'legacy deploy' workflows and a FREEZE note. A routine push still deploys, until the guardrail disables them.

Three of six were broken open by one plain question

“Don't the clients set DNS in their local profiles?”

“Can anyone still update it with the old code?”

And a simple question about whether a hosting option even had an address.
The guardrails exist to ask those questions every time, not when someone happens to.

Take these home

1Prove it's settable, not that it's empty
2Assert the outcome, not the exit code
3Re-read live state before you recommend
4Quote, don't paraphrase
5Every HIGH cites a line, and someone checks it
6Disable the old door

AI doesn't fail loudly. So check quietly, every time.

damianflynn.com · github.com/damianflynn · linkedin.com/in/damianflynn