In a system where mistakes can hurt people, a green test result is dangerous if the test never checked the new code.
I took on the security delivery of a pharmacist workflow system for a medical organization. Patient medications, signatures, and visit records all ran through it. I expected to find defects. I did not expect my own checks to give me a false pass and 25 false alarms on the same day.
I've worked across healthcare, education, and long-term care for 20 years. This wasn't my first medical system. I wrote about moving a hospital to the cloud. This time, the question was whether I could sign my name to the system's security and behavior.
Why this system needed another pass
The previous vendor had built the features, but the security work wasn't finished. The hospital needed the system ready for an external acceptance review. Someone had to check the parts a working screen could hide.
A marketing site can usually recover from a broken page with a refresh. Here, a failed save could lose a pharmacist's signature. A weak permission check could expose patient records or allow a signature to be misused.
I took the work on alone, with AI doing much of the engineering and me responsible for the decision to deliver it.
Three defects behind normal-looking screens
- A field rename broke signature saving. Only one side of the change had been updated. A pharmacist could sign, see no error, and still have nothing saved.
- The second visit's signature flow went nowhere. The QR code and dialog led to a page with no backend logic. It looked connected, but had never worked end to end.
- Unauthenticated users could enumerate signature records. A person who had not logged in could try identifiers one by one.
Each defect could sit behind a screen that looked fine. The only way to catch them was to follow what a real user did and check what the system stored or returned.
Then I checked the checks
After fixing the defects, I ran the tests and saw green. I was about to call the system ready.
Instead, I asked what those tests had actually measured. On the same day, I found two opposite failures:
- One false pass: a green test had checked an old version. It never exercised my new change.
- 25 false alarms: red tests came from a broken test environment, not from defects in the system.
One test setup had an even deeper problem. A failing test could be silently rerun and treated as a pass. More than 30 of the 60-plus tests were affected, and the behavior had sat there for weeks.
Had I trusted the green result, I would have signed off on a medical system I hadn't actually verified.
I'd been caught by a similar problem while building a machine to check whether my AI followed its rules. This time I repaired the checks and deliberately broke code to see whether each relevant test would turn red. I connected 25 tests that had never run and checked the live pharmacist workflow. All 12 online verification items passed.
What delivery means to me now
| A common stopping point | What I checked before delivery |
|---|---|
| Features are built and tests are green | Tests cover the change and real behavior is correct |
| "It should be fine" | Deliberately break it and see whether the check catches it |
| The screen looks normal | Follow the user path and check the saved data |
| Hand over code | Hand over an acceptance report I can sign |
AI has made it faster to produce features. It hasn't removed the need to prove that a signature saves, that permissions hold, or that a test can detect a failure.
Questions I get
Did you use AI on this project?
Yes. AI wrote code and helped run tests. I made the delivery decision and checked the evidence myself.
Does this level of checking matter outside healthcare?
It matters wherever a failure has a real cost: financial workflows, compliance, personal data, and approvals.
Can one person handle security delivery for a medical system?
I handled this delivery alone with AI as an engineering tool. The crucial part was checking the system and the checks themselves.
I'm Young. I've spent 20 years working across healthcare, education, and long-term care. The medical organization in this story remains anonymous. If you have a system whose delivery needs evidence you can sign, book a free conversation with me.