A good phishing simulation program is a repeatable learning exercise, not a trap. Plan it around a few clear objectives, send realistic but fair messages to everyone, coach people right after a mistake, and track report rates alongside click rates. Then pair the program with technical controls, because simulations alone do not stop phishing.
This guide is written for managing partners, IT directors and security leads at law firms, oil and gas companies and other regulated mid-sized organizations. It covers how to set up a program, how often to run it, how to keep trust intact, which metrics matter, and what to watch for in your sector.
TL;DR
- Set objectives first: reduce risky behavior, raise the reporting rate, and feed findings into your risk register. Do not chase a low click rate for its own sake.
- Include everyone, from interns to partners and executives. Keep the tone non-punitive and tell staff that exercises happen.
- Coach immediately after a failed test with short, current, real-world content, not a lecture.
- Track click rate and report rate over time, and adjust for how hard each email was before you compare results.
- Treat simulations as one layer next to email filtering, endpoint protection and backups.
Table of Contents
- What a phishing simulation can and cannot do
- Plan the program before you send anything
- How often to run simulations and how to adjust difficulty
- Design realistic scenarios without losing trust
- Give feedback that changes behavior
- Measure what matters, and read it carefully
- Sector considerations: law firms and energy operators
- Our perspective: simulations are a risk signal, not a score
- Get help building a program that holds up
- Common mistakes to avoid
- FAQ
- Sources
What a phishing simulation can and cannot do
A phishing simulation sends staff a harmless, fake phishing message and records what they do with it: ignore it, report it, click it, or enter credentials. The goal is to teach and to measure. NIST SP 800-50 Rev. 1 lists phishing simulations among the experiential learning activities that let an organization observe user behavior and find vulnerabilities.
The case for doing it is real. The Verizon 2024 Data Breach Investigations Report found that 68% of breaches involved a non-malicious human element, a figure summarized here. CISA also advises organizations to educate personnel as a ransomware prevention measure and notes that security teams can test training effectiveness with simulated phishing emails.
Be realistic about the ceiling, though. An eight-month study of more than 19,500 employees at a large healthcare organization, presented at the 2025 IEEE Symposium on Security and Privacy, found minimal differences in simulation failure rates between trained and untrained users. The lesson is not to abandon simulations. It is to stop treating them as a shield. A program that only sends fake emails and logs clicks is the weak version. A program that builds a reporting habit, improves training content and feeds technical decisions is the strong one.
CISA's counter-phishing guidance says the same thing from the technical side: it recommends combining workforce training with technical precautions such as secure email gateways, outbound web-browsing protections and hardened endpoints.
Plan the program before you send anything
Most failed programs skip planning. They buy a tool, blast one template to everyone and report a click percentage to leadership. Start with these steps instead.
- Define objectives. Pick two or three: for example, raise the share of staff who report suspicious email, shorten the time from delivery to first report, and cut repeat clicks among high-risk roles.
- Choose a program structure. The California Department of Technology suggests four pillars: onboarding integration, quarterly refresher modules, workforce-wide simulations, and role-based training for high-risk personnel. That structure scales down well to a 40-person firm.
- Include everyone. The FTC advises that no employee is too important for security training, and that programs should cover interns, contractors and executive leadership. At law firms, partners often hold the broadest access and the most sensitive client files, so exempting them is the most expensive shortcut.
- Build a reporting path. Give staff a one-click report button or a single mailbox, and make sure someone triages reports the same day. A report that vanishes teaches people not to bother.
- Set a baseline. Run one moderate-difficulty simulation before any new training, so later results mean something.
- Get sign-off. Brief leadership, HR and, where relevant, legal counsel before the first send.
How often to run simulations and how to adjust difficulty
There is no single correct frequency, and no official body publishes one number. Treat any claim of a universal best interval with caution. What the evidence and common frameworks support is a rhythm you can sustain, tied to risk.
A workable starting point for most mid-sized organizations:
| Audience | Suggested rhythm | Why |
|---|---|---|
| All staff | Quarterly simulation, aligned with a short refresher module | Matches the quarterly refresher pillar in the California program guide |
| New hires | Within the first weeks, as part of onboarding | Newcomers have no habits yet and often lack context on who emails them |
| High-risk roles (finance, executive assistants, IT admins, partners) | More often, with role-specific scenarios | They are targeted more precisely and can authorize payments or access |
| Repeat clickers | Extra coaching, not extra punishment | A short one-to-one conversation outperforms another mass email |
Two cautions apply. First, more is not always better. Frequent, similar tests breed fatigue and cynicism, and employees begin to trade tips on how to spot the "test" rather than the threat. Second, vary the sending times and templates so people cannot predict a test.
For difficulty, start easy, then rise gradually. Mix in obvious phish (generic urgency, mismatched sender) with targeted ones (a realistic invoice, a shared-document request from a plausible vendor). Keep the difficulty of each email on record, because it determines how you read the results. We cover the reasoning behind risk-based cadence in our guide to security awareness training every 4 to 6 months.
Design realistic scenarios without losing trust
Realism is what makes a simulation worth running. It is also where programs do damage.
Match scenarios to the work people actually do
Generic lottery or gift-card emails teach little. Use messages that mirror your real threat picture:
- Law firms: a client or co-counsel sharing a document through a file link, a wire instruction change tied to a closing, a court filing notice, a request to "review" a conflict check. Our overview of types of law firm cyber threats maps the lures attackers use against legal teams.
- Oil and gas and energy: a vendor portal login, a purchase order, a safety bulletin, a contractor badge request, a message that appears to come from a field supervisor. Our guide on security awareness for energy workers goes deeper on field and plant-floor realities.
- Any firm: payroll and benefits notices, multi-factor authentication prompts, shipping alerts, and shared calendar invites.
Keep the exercise fair
NIST's guidance is direct on the human side. Simulations should be non-punitive and framed as learning opportunities, and organizations can reduce anxiety by communicating clearly that exercises happen at random. NIST also recommends that legal teams review planned exercises so bait content does not misuse real brand names or government entities in ways that could cause confusion or harm.
In practice, that gives you a short rulebook:
- Do not use bait about bonuses, layoffs, health results or family emergencies. It is cruel and invites complaints.
- Do not impersonate real agencies or named vendors without legal review.
- Never publish names of people who clicked, and never tie one result to discipline.
- Tell staff in advance that simulations are part of the program, and explain why.
- Escalate only for deliberate policy violations, such as repeatedly ignoring training, and handle those through normal HR channels.
Fairness also applies to how programs use data and automation. If your platform personalizes content or scores individuals, think through consent and data handling. A thoughtful read on ethics in AI-driven employee training covers the data, people and policy questions worth settling before you roll anything out.
Give feedback that changes behavior
The moment after a click is the best teaching moment you will get. Use it.
- Show a short landing page immediately. Three or four points: what gave it away, what to do instead, how to report.
- Keep it brief. A two-minute micro-lesson beats a 30-minute video. The FTC notes that effective training moves beyond boring, repetitive lectures and uses real-world stories and current headlines.
- Thank people who report. Positive reinforcement for reporting, including reporting real phish, builds the habit you want.
- Follow up with repeat clickers privately. Ask what made the message convincing. You will learn about workflow pressures that no dashboard shows.
- Share anonymized lessons firm-wide. A short monthly note with one sanitized real example keeps awareness alive between tests.
For managing partners, the governance side matters just as much. Our governance playbook for managing partners explains how leadership participation sets the tone for the whole firm.
Measure what matters, and read it carefully
Two numbers anchor most programs: the click rate (failure) and the report rate (success). The SANS Institute's benchmarking paper recommends tracking both over time and warns that results must be interpreted with caution.
The reason is that raw numbers mislead. SANS identifies at least 11 variables that affect benchmark validity, including program history, simulation difficulty and workforce sample size. A 4% click rate on an obvious email says little. A 12% click rate on a convincing, targeted one may reflect a stronger program.
The NIST Phish Scale addresses this. It rates how hard a specific email is to detect, so you can put click and report numbers in context rather than comparing different tests as if they were equal.
A practical metrics set
| Metric | What it tells you | How to use it |
|---|---|---|
| Click rate | Susceptibility to a given lure | Compare only like-for-like difficulty |
| Report rate | Active defense behavior | Aim to raise it every quarter |
| Time to first report | How fast your team can start response | Share with your incident response owner |
| Repeat clicker count | Where coaching is needed | Support people, do not shame them |
| Credential entry rate | Depth of the failure | Track separately from clicks |
| Real-phish reports | Whether skills transfer to live threats | Strongest sign the program works |
Report the trend, not a single result, to leadership. A rising report rate with a flat click rate is usually good news, because more people are alerting you even as tests get harder.
Sector considerations: law firms and energy operators
Law firms
Lawyers handle privileged data, and a successful phish can become a confidentiality problem as well as a security one. Tie your program to your ethical duties. Our guides on ABA cybersecurity guidelines compliance and client confidentiality controls show where training and simulation evidence support your obligations. Keep records of what you ran and how you responded, since clients increasingly ask for them in security questionnaires.
Oil, gas and energy
Energy organizations face a split environment: corporate IT on one side, operational technology (OT) on the other. Phishing is often the way into the first and a stepping stone to the second. The Department of Energy notes that even when a sector has not been a direct target, ripple effects from other attacks can still be debilitating.
Simulations are one training form among several. The DOE's CyberStrike program offers simulated real-world cyber event training for energy owners and operators, and DOE describes exercises such as Liberty Eclipse for oil and gas operators that help find security gaps and validate response plans with government partners. Use phishing results as inputs to those wider exercises. Our explainer on incident response tabletop exercises shows how to connect the two. Offshore and remote sites bring added constraints, covered in our offshore facility cybersecurity guide.
For field crews, keep scenarios short and mobile-friendly, and schedule them around shift patterns so people are not tested while on a rig or at a turnaround.
Back up your program with technical controls
Because some people will always click, plan for it. The FTC recommends keeping routine data backups not connected to the network to limit damage if phishing leads to ransomware. Add multi-factor authentication, email filtering and endpoint protection so a single click does not become a breach.
Our perspective: simulations are a risk signal, not a score
We see the same pattern with regulated clients. Leadership asks for a click rate, a tool produces one, and the number goes into a board slide without context. It tells the board almost nothing.
We treat phishing simulation as one input to a security program. The useful questions are these. Which roles are targeted most? How quickly does a report reach someone who can act? Do the lessons change policy, email controls or onboarding? A vCISO connects those answers to the risk roadmap, so a failed test becomes a specific control improvement and not a reprimand.
For law firms and energy operators, we also look at what the results mean for compliance evidence. An auditor or client wants to see a documented, repeated, risk-based program with leadership involvement. That is far more persuasive than a single dashboard screenshot.
And we favor fairness. People who feel safe reporting mistakes report more, and early reporting is what limits damage.
— vCISO
Get help building a program that holds up
If you want simulations tied to a real risk roadmap, CISOSafe can help. Our virtual CISO and compliance services cover security awareness programs, security risk assessments, policy and controls development, and audit and certification readiness, so your phishing results feed decisions and evidence, not just a report.
To discuss your firm or operation, start at cisosafe.com.
Common mistakes to avoid
- Testing only once a year. One annual test is a compliance checkbox, not a behavior program.
- Using a single template. People learn the template, not the threat.
- Punishing clickers. NIST warns against it, and it suppresses reporting.
- Skipping leadership. Partners and executives are prime targets.
- Comparing across tests without context. Use difficulty ratings before drawing conclusions.
- Ignoring the report path. If reports go unanswered, reporting stops.
- Relying on training alone. Pair it with technical controls and backups.
FAQ
How often should we run phishing simulations?
For most mid-sized organizations, a quarterly simulation tied to a short refresher is a sensible starting point, matching the quarterly refresher model in California's program guide. Increase frequency for high-risk roles and new hires. Adjust to your risk and your capacity to follow up properly.
Should employees be told about phishing simulations?
Yes, in general terms. Tell staff that the program exists, that tests are random and that the aim is learning, not discipline. NIST recommends clear communication of this kind. Do not announce exact dates, which would undermine the exercise.
What is a good click rate?
There is no universal target. SANS notes that results depend on difficulty, program maturity and sample size, so compare your own results over time using similar difficulty. A rising report rate is often a better sign of progress than a lower click rate.
Do phishing simulations actually work?
They help most when combined with fast feedback, a reporting habit and technical controls. A large 2025 healthcare study found little difference in failure rates between trained and untrained users, so do not rely on simulations alone.
Can we punish employees who repeatedly click?
NIST warns against punitive measures for people who fail simulations. Offer private coaching and look for process causes first. Reserve formal action for deliberate, repeated policy violations and handle it through HR.
Sources
- NIST SP 800-50 Rev. 1, Building a Cybersecurity and Privacy Learning Program
- A Step-By-Step Guide To Phishing Simulation Training (Forbes Tech Council)
- CISA, Ransomware Executive One-Pager and Technical Document
- New Study Examines the Effectiveness of Phishing Training Programs
- CISA, Counter-Phishing Capacity Enhancement Guide
- California Department of Technology, Phishing Awareness & Simulation Program Guide
- FTC, How readiness can help protect veteran-owned businesses
- SANS Institute, Effectively Benchmarking Your Phishing Awareness Program
- NIST Phish Scale User Guide
- DOE, Protecting Wind Energy Systems From Cyberattacks
- DOE, 2023 CyberStrike At-A-Glance
- DOE, CESER Supports the Oil and Natural Gas Subsector
- FTC, Cybersecurity for Small Business
- AI Ethics in Training: Data, People and Policy