Logging That Tells the Story: Evidence You Can Trust (Without Drowning in Data)
- Daniel Bertrand
- Mar 22
- 3 min read
Plain-English idea: Good AI logging shouldn’t be a firehose of technical crumbs—it should read like a short, reliable story you can replay: who asked, why it mattered, what tool acted, how much data moved, and where it went. When something feels off, those facts let you decide fast and fix with confidence.
Why “story-first” logging beats “more logs”
If an AI helper makes a mistake, you don’t just need what happened—you need why it happened and under whose authority. A thousand raw events won’t help if they can’t be stitched into a clear narrative. Log the plot points, not every footstep.
The seven plot points every AI action should record
Keep it simple and consistent across all agents:
Who – the agent’s name and owner (real person/team).
Why – the agent’s plan in one or two sentences (before it acts).
What – the tool/command it actually used (not just the final attachment).
Where – the data source it touched (CRM, HRIS, drive, inbox).
How much – a quick measure (rows/records touched, file count, file size).
To whom – exact destination (named folder, distribution list, system).
Under whose authority – was a human approval required and recorded?
Tip: capture a simple “plan vs. act” diff. If the plan said “review summary” but the act was “export full table,” that’s your red flag.
The two special labels that make investigations fast
Provenance: Did the request start from inside or from outside (email, web, upload, customer ticket)?
Sensitivity: Is the data Public, Internal, or Sensitive?
When an incident pops, you’ll immediately see patterns like “outside → sensitive → export” and respond accordingly.
What a “good” entry looks like (plain text, easy to read)
2026-02-05 14:30 Agent: FinOps
Plan: “Send monthly headcount summary to Finance-Metrics.”
Action: hris.export_headcount() Rows: 1,204
Provenance: internal Data: Internal
Destination: Finance-Metrics (mail list)
Approval: not required (per policy)
Owner: Pat Lee (Finance Ops)
If something goes wrong, you want a handful of entries like this—clear enough for legal, leadership, and IT to agree on the facts.
Privacy, safety, and retention (keep it respectful)
Don’t log raw sensitive data. Log counts and hashes, not the content itself.
Keep prompts/attachments—but sanitize. Store redacted copies so you can replay what the agent saw without exposing extras.
Make logs tamper-evident. Chain entries so deletions leave a scar.
Choose human-sized retention. Example: 90 days for routine entries, 1 year for Sensitive data actions or incidents.
Access is on a need-to-know basis. Treat the log like a sensitive system, not a public bulletin board.
Dashboards that actually help (no heatmaps required)
Outside → Sensitive → Export (last 30 days)
First-time actions by any agent (new tool, new destination)
After-hours big moves (large exports at 2 a.m.)
Approvals missing where policy says “required”
Top destinations (so you can retire unused distro lists and risky “catch-all” folders)
One-click “Incident Evidence Bundle”
When an alert fires, you don’t want to hunt. Your logging should export, in one go:
Timeline of the key entries
Graph of who asked whom, with approvals in between
Receipts of tool calls (what/where/how much)
Plan texts and approval notes
A short Executive Summary that leadership can read in 2 minutes
(If you’ve been following along, this bundles neatly with the “broker” and hand-off card we covered in previous posts)
A one-week rollout anyone can do
Day 1–2 – Pick a single format (like the example above) and apply it to your top 3 agents.
Day 3–4 – Turn on “plan vs. act” diffs and add the two labels (Provenance, Sensitivity).
Day 5 – Add a “Download Evidence Bundle” button in your ticketing workflow.
Day 6–7 – Sit down with one near-miss. Could you tell the story clearly? If not, tweak the format.
What each role owns
Executives: Approve a one-page policy that says “no agent in production without story-first logging.”
Managers: Ensure your team’s agents record the seven plot points and have named owners.
Front-line staff: If an agent’s plan doesn’t match what it did, report it—that’s how safeguards improve.
IT/Security: Enforce the format, keep logs tamper-evident, and add alerts for “outside → sensitive → export.”
How this fits broader guidance
