Don’t Let Models Parrot Your Secrets: Log Sanitization Needs Gates at Both Ends

Last month, the on-call bot I built for our inference service started gaining traction: when customers reported issues, it would pull relevant lines from production logs and feed them to the model to generate a diagnosis. Then, one day, a 32-character token appeared verbatim in a diagnostic Markdown report—it was an API key that a colleague had printed while dumping environment variables at startup, which had been sitting in the logs all along.

The key hadn’t expired yet, but the diagnostic Markdown had already been posted to an external group chat. I rotated the key within 30 seconds, but since the Markdown contained the key in plaintext and had been synced to our internal repository, I had to meticulously clean up every potential leakage channel I could think of.

The most painful insight from the post-mortem was this: that environment dump log had been sitting in production logs for three weeks. Every time someone manually investigated an issue, they scrolled right past it. Everyone “saw” it, but no one treated it as a risk. The risk was always there; we just lacked an automated gate to stop it. Relying on human eyes to “see it and skip it” is equivalent to having no protection at all.

Where the Holes Were

Before the incident, our pipeline operated on two flawed assumptions.

First, **“The model is human-like and uses judgment.”** We assumed it would independently decide which parts of the log shouldn’t be repeated. Not true. The model’s task is to incorporate context from the input into its response, and a 32-character key is exactly the kind of pattern it excels at copying verbatim. I tested this afterward: I fed it a log line containing a key five times, and it parroted the full key all five times, without fail.

Second, **“Input-side sanitization is enough.”** We did have some regex-based sanitization in place, but it only covered `sk-` prefixes and `Bearer` headers. It missed other types of hexadecimal tokens and an internal database DSN entirely.

The Fix: Four Hard Gates

We subsequently split sanitization into four layers. If any layer triggers a match, the data is blocked—there is no manual override to bypass this:

1. **Regex List (Input Side)**: Covers `sk-`, `AKIA`, JWTs (starting with `eyJ`), MySQL/Postgres connection strings, and key-value pairs like `password=` or `token=`. Whenever a new type of secret is discovered, rules are added the same day, and incident records immediately update the list.

2. **High-Entropy Heuristics (Input Side)**: Calculates Shannon entropy for continuous string segments ≥ 20 characters long. If the threshold is exceeded, the content isn’t deleted outright but flagged for manual review. This catches formats that regexes often miss, such as hexadecimal tokens and Base64-encoded keys.

3. **Output Rescan (Output Side)**: Before the diagnostic Markdown generated by the model is sent out, it passes through the same set of rules again. Matches are blocked immediately, and an alert is sent to the on-call group. This layer is the true safety net. In the first two weeks after launch, it caught two real incidents (one key, one phone number) at the output stage. The phone number was embedded in a normal sentence, so the input side hadn’t flagged it at all.

4. **Field Whitelist (Source Side)**: Instead of pulling the “last 500 lines,” the log ingestion tool now pulls only specified fields: timestamp, level, service name, error code, and message body. Environment dump lines like the one in the incident can no longer enter the model’s view.

We also paid tuition for false positives: the entropy rule triggered over 40 false alarms in the first two weeks, half of which were naturally hexadecimal trace IDs. We later added boundary checks (requiring whitespace or punctuation before the string), which reduced false positives to less than one per week, making the manual review workload manageable.

How to Prove the Gates Are Working

On the day the pipeline went live, I took a snapshot of the pre-incident logs (with keys replaced by dummy values) as a fixture for regression testing. Now, every time we modify sanitization rules, we run the fixture first and assert that all high-entropy segments corresponding to original keys are masked in the output.

This serves two purposes. First, **preventing regressions**: the longer the rule set, the more likely interaction bugs become. Once, when adding a JWT rule, the `eyJ` prefix match didn’t handle quoted JSON logs correctly, and the fixture immediately turned red, allowing us to catch the bug. Second, **building trust**: anyone taking over this pipeline doesn’t need to take my word that “all four layers are active”; they can run the fixture and see the results for themselves.

A quick note on cost: the output rescan runs synchronously in the generation path using a local rule engine, not an external LLM. This adds less than 200 milliseconds of latency per diagnosis. If we used an external model to audit the output, both latency and costs would multiply several times over.

Final Thoughts

- **Don’t trust the model’s “conscience.”** If you want certain types of text to “absolutely never appear in the output,” enforce it at the pipeline level: input sanitization + output rescan. Missing either end makes the gate merely decorative.

- **Real leaks are low-probability events, but they will happen.** The only question is whether your pipeline has a “final intercept.” If it does, the cost is a few hours of engineering work. If it doesn’t, the cost is a public incident and a batch of keys that need rotating.