Cyber Incident Response: When Systems Go Down

What the first hours look like when CAD, records or email are gone, who decides what comes back first, and how a department keeps answering calls while it happens.

Full guide planned · 15 min read

The short version

A cyber incident is a continuity problem before it is a technical one. The question on the first morning is not how the attacker got in. It is whether you can still take a call, dispatch a unit, alert a station, document a patient and pay your people, and for how long you can do all of that on paper. Most departments discover their real dependencies during the outage. Station alerting turns out to run through the same network as email. The pre-plans are in a cloud system nobody can reach. The ePCR queues locally but cannot transmit, so the hospital gets nothing. Payroll is a city system, and the city is also down. Restoration is a command decision, not an IT one. Someone has to say what comes back first, and that order should be written before the incident, not argued during it. Dispatch and alerting almost always lead. What follows them depends on your operation, and the useful exercise is to put the list in order now, with the people who would have to live with it. Recovery is also staged rather than instant. Systems come back partially, in a rough order, sometimes into a rebuilt environment, and a department can spend weeks operating in a mixed state where some things work and some do not. Plan for the mixed state, because that is where most of the time is actually spent. For EMS there is an added layer. Patient information is protected health information, and an incident touching an ePCR system brings notification and privacy obligations that run in parallel with the operational response. Know before the day comes who makes that call in your organisation. None of this is an argument for alarm. It is the same reasoning behind a pump failure plan or a mutual aid agreement: decide in advance how you keep working when a thing you rely on is unavailable.

Start here

If you read one thing on this subject, read this.

Official resourceCISA

#StopRansomware Guide

The federal guide to preventing ransomware and to responding when it happens.

Why it matters

The response checklist is worth reading before you need it, not during.

Official resourceCity of Dallas

After Action Review: May 2023 Ransomware Incident (PDF)

The city's own review of the May 2023 incident that affected police, fire and city systems.

Why it matters

A public-safety continuity case study from the organisation that lived it. Read it for system interdependency, manual operations, the order things were restored in, and how long staged recovery actually took — not as a reason to be frightened.

Go deeper

Official resourceCISA

Stop Ransomware

The federal hub for ransomware guidance, alerts and reporting.

Why it matters

Where to go on the day, including who to report to.

Official resourceNIST

Contingency Planning Guide (SP 800-34)

Planning to operate through a loss of systems, and to recover in a defined order.

Why it matters

The structure behind a restoration priority list you can agree before an incident.

Questions to ask your vendor

  1. If your system is unavailable, what is our documented fallback, and have you tested it with a customer our size?
  2. How quickly do you notify us of an incident affecting our data, and in what form?
  3. Where are our backups held, who can reach them, and can we restore without your help?
  4. Have you ever had to restore a customer of our size from backup? How long did it take?
  5. What does your system do when it loses connectivity, and what happens to the records created while it was offline?
  6. Which of your staff can access our production data, and how is that access logged?
  7. If we terminate, how do we get our data out, in what format, and how long do you keep a copy?
  8. Do you support MFA on every account, including administrative ones, at no extra cost?

This is a reading list, not a guide

Everything above was published by someone else, and is here because it is the clearest treatment of the subject we could find and verify. The Hub’s own guide to this topic is still being written. If you know a better source than the ones listed, that is worth telling us before it is.

Send a note