Example Postmortem
This postmortem is for a made-up incident at a made-up business. It uses a blameless format, which focuses on what went wrong with systems and processes instead of who to blame.
Postmortem: Ransomware on the File Server, Riverbend Veterinary Clinic
Summary
On a Saturday night, an attacker logged in to the clinic's network storage device (NAS) through its web admin page. The page could be reached from the internet and still used the default admin password. The attacker encrypted the shared drive that holds patient records and invoices. The clinic restored its files from a backup that was eight days old.
Impact
- The clinic ran on paper for two days
- Eight days of records had to be re-entered by hand, about 40 staff hours
- The clinic paid for emergency help from its IT contractor
- There was no evidence that data was copied, but it couldn't be ruled out, so clients were notified as required by state law
Timeline
- Saturday 11:40 PM: attacker logs in to the NAS admin page with the default username and password
- Sunday 12:15 AM: files on the shared drive start being encrypted
- Monday 7:30 AM: front desk staff can't open files and find a ransom note
- Monday 8:00 AM: office manager calls the IT contractor, who says to unplug the NAS from the network
- Monday 11:00 AM: contractor confirms ransomware and finds the admin login from an unknown overseas IP address in the NAS logs
- Tuesday: NAS is wiped, firmware is updated, and files are restored from the external backup drive
- Wednesday: clinic is back to normal, and staff finish re-entering records
Root cause and contributing factors
- Root cause: the NAS admin page was open to the internet with the default password
- The NAS was set up three years ago by a previous contractor, who turned on remote access during setup and never turned it off
- The firmware was two years out of date
- Nobody was responsible for the device, and it wasn't on any list of equipment
- Backups were weekly and manual, and depended on one person remembering to do them
What went well
- Staff reported the problem quickly and the NAS was unplugged within 30 minutes
- The backup drive was kept unplugged, so it wasn't encrypted
- The NAS logs showed exactly how the attacker got in
What went poorly
- No inventory of network devices
- No alerts for admin logins or for large numbers of files changing
- The backup schedule relied on memory
Action items
| Action | Owner | Due |
|---|---|---|
| Change the default password on every network device | IT contractor | 1 week |
| Turn off remote admin access and require the VPN instead | IT contractor | 1 week |
| Create an inventory of all network devices with an owner for each | Office manager and IT contractor | 2 weeks |
| Automate daily backups and keep one copy offline | IT contractor | 2 weeks |
| Turn on automatic firmware updates, or schedule a monthly check | IT contractor | 1 month |
| Compare the NAS and router settings against the vendors' hardening guides | IT contractor | 1 month |