The First 30 Seconds After Launch: Figure Out How to Roll Back First

9:00 AM on a Monday. You change one line of timeout configuration in the payment gateway, hit Enter, and the deployment completes. A quick scan of the logs looks clean. You breathe a sigh of relief and head off to your next meeting.

Thirty minutes later, payment callbacks start queuing up, and timeout errors flood the screen. That’s when you finally remember to ask the question: **How do we roll back?** If the answer is "we’ll just redeploy the previous version later," then you don’t actually have a rollback plan—you’re just assuming that rolling back will happen automatically.

This guide isn’t about architecture. It’s simply a 2-minute pre-flight checklist.

When to Use This

- Deploying code, changing configurations, toggling feature flags, or running database migrations

- Working in shared environments (staging, production, or "that server everyone uses")

- Operations that send external messages, write to external systems, or modify data

When Not to Use This

- Local experiments where failure has zero cost and you can just restart

- One-off copy changes where mistakes can be rewritten instantly

- Pure read operations (checking logs, viewing data)

If none of the "When to Use" criteria apply, skip the process—**don’t treat the process itself as work**.

The Checklist (2 Minutes)

1. **Write down the rollback command.** Literally write it down—in the commit message, ticket, or Slack. If you can’t specify the exact command, you don’t have a rollback plan; you only have a wish.

2. **Estimate rollback time.** A 30-second revert and a 40-minute data reconstruction are two different orders of risk. Any "rollback" taking longer than 10 minutes isn’t a rollback—it’s a disaster recovery plan. In such cases, you should use canary releases or feature switches instead of gambling.

3. **Preserve the scene.** Git tags, config diffs, table backups. Saying "I remember what the previous version looked like" equals having no backup.

4. **Can side effects be undone?** Messages sent, caches populated, downstream tables updated—these often cannot be rolled back. When side effects are irreversible, the issue isn’t that your rollback plan is inadequate, but that the operation itself should be made reversible (e.g., dry runs first, canary releases first).

5. **Record the old value.** Changing a timeout from 5s to 30s? Write "Old value: 5s" in the ticket. Human memory is unreliable at 3:17 AM.

Common Pitfalls

**Pitfall 1: Mistaking backups for rollbacks.** Having a backup doesn’t mean fast recovery. It’s common for recovery scripts to haven’t been tested in six months, for backups to be stored off-site, or for approval processes to delay retrieval. The acceptance criterion for a rollback is "Can we restore within 10 minutes right now?" not "Do we have a backup?"

**Pitfall 2: Enabling flags without verification.** A feature flag is itself a change. No one has tested whether the service starts correctly after the flag is turned off, or if it throws null pointer exceptions. Before treating a flag as a safety net, test it in isolation.

**Pitfall 3: Rolling back code but not data.** New code writes to a new column; after reverting, the old code reads from the old table, causing field mismatches. Database changes must either be backward-compatible or explicitly marked as "non-rollbackable." Mixing these approaches is the most dangerous path.

**Pitfall 4: Planning it but not writing it down.** In meetings, people say, "We’ll canary first, then roll back if needed," but forget the rollback half during execution. Your rollback plan must match the completeness of your deployment plan: same effort, same documentation location, same acceptance details. During an incident, the only thing you’ll have time to look at is what’s written down.

A 10-Second Self-Check

**"If it blows up, can I roll back within 10 minutes?"** Yes → You have a command, a backup reference, and a time estimate. No → Make it possible first, then launch.

Pre-Launch Ticket Template

## Rollback
Command:  / revert X=5s / disable flag: pay-timeout>
Estimated Time: 
Scene Preservation: 
Irreversible Side Effects: 

If you can’t fill in all five fields, it’s a red light.

Example: What a Timeout Rollback Looks Like

Ticket: Change payment callback timeout from 5 seconds to 30 seconds. The rollback section should look like this:

- Command: In the config panel, change `pay.cb.timeout` from 30 back to 5, or run `config rollback --key pay.cb.timeout`

- Estimated Time: 2 minutes

- Scene Preservation: Screenshot taken before the change, attached to the ticket

- Irreversible Side Effects: None—because this is a configuration change, not a data migration

Contrast this with a risky version: "Rollback: Redeploy the previous image." The problem is that no one has timed how long this action takes, it’s unclear how old the "previous" image is, and no one has considered what happens to new data written in the interim. It looks like a plan, but three question marks remain hanging. The second approach fills those gaps.

Conclusion

The value of a rollback plan isn’t realized on the day things go wrong, but before you even start: it’s that short written statement that forces you to clarify your "exit strategy" at 2:00 PM. The moment you find yourself unable to write it down is the moment with the highest information value before launch.