Click Rate Is a Terrible Metric for Phishing Simulations
Why click rate falls short: The limitations of phishing simulation metrics
If you ask any IT department what they measure when they run a phishing simulation, they’ll almost always say click rate. They want to know what percentage of users “fell” for the fake attack. Click rate is such a common measure for phishing simulations that some of the biggest awareness training and attack simulation platforms tout a reduced click rate as their primary objective.
This doesn’t make much sense though, as click rate is a lousy metric for phishing simulations. To understand why, it’s important to dig into the reasons for running simulated phishing attacks in the first place.
Phishing as a Test
One common justification for running phishing simulations is that it is a test of an organization’s risk exposure. It tells you how vulnerable your organization is to social engineering attacks. In this view, a low click rate would be good.
However, sending out a single phishing email to “test” your organization doesn’t measure much of anything. If you get a 5% click rate, it doesn’t mean the other 95% of users are safe. The reality is that most people ignore most emails most of the time. There are a million reasons to not click on a phishing email that have nothing to do with cybersecurity. Was the email relevant and interesting? Did the person even read it? Did they care? A one-off click rate is basically meaningless.
You might think you could get around this by sending out multiple phishing emails over time, but the problem there is that the attacks are not standardized. If you get a 4% click rate on the first email and a 3% click rate on the second, is it because your organization got better, or because your email was less click-worthy? You won’t know.
Overall, running phishing simulations to measure your organization’s performance is not a fruitful exercise, and using click rate in that context is a quantifiable figure that disguises how little we really know.
Phishing as Training
The second common justification for running phishing simulations is that it is good awareness training. It teaches people how to identify and avoid phishing emails.1 This is the view that we hold at Pistachio.
If the goal is training, then you want as many people to fail as possible. In other words, you want a high click rate. Why? Because the people who fail phishing simulations are the least likely to fail in the future.
We can see this in our data. Users who “fail” an attack simulation are 50% less likely to fail the next phishing simulation they receive relative to the users who ignored the previous attack.2 Falling for a simulation, and receiving the immediate feedback Pistachio provides, really does help people to become safer. So we want people to fail!3
It’s no different than going to the gym, really. If you are always succeeding on all of your lifts then you’re never “pushing to failure” and you won’t get the results you want.4 You have to fail to improve.
A good phishing simulation program solution should aim to achieve a high percentage of users failing at least one attack, and also should ensure that users are exposed to a variety of different types of attacks at different difficulty levels.5 Click rate doesn’t capture that information, so it’s not a good metric.6
Focus on What Matters
Our aim at Pistachio is to provide your employees the training and experience they need to stay safe in the digital world. Sometimes we need to send out very challenging phishing emails to expose people to the latest techniques, and other times we need to send out emails that are obviously wrong because some people click those too. What each user needs is different, and Pistachio has to be adaptive to that.
Focusing on click rate would get in the way of that. It does not capture the objective of running phishing simulations. Next time someone asks to see the click rate on your phishing simulations, send them a link to this blog post instead.
It also reminds people that phishing emails exist, which is half the battle.↩
This data is imperfect because Pistachio’s attacks scale to each user, but the difficulty adjustments are incremental and all users start at different difficulty levels.↩
IT departments should keep this in mind when they get mad at people for falling for a fake phishing email.↩
For a more nuanced take on training to failure, you can listen to Jeff Nippard explain it.↩
If you only show the hardest attacks you get the highest click rate, but you don’t expose people to the easier emails that also trick people.↩
A 5% click rate could be the same 5% every time, or a different 5% every time. The former is bad, the latter is good.↩
We believe in a learn-by-doing approach over snooze-worthy videos.
Instead, Pistachio tailors real-world scenarios so your team can build their cyber skills in a safe environment.
Continue reading
Ready to reduce your human risk? Book your demo today.
“We lacked confidence that employees would recognise today’s phishing threats or respond correctly. Since using the Pistachio platform, they are more attentive and proactive in identifying potential risks. The solution is easy to manage, quick to implement, and highly effective.
“The platform is easy to implement and provides clear insights into employee phishing awareness. Its bite-sized learning encourages participation, saves administration time, and allows us to focus more on real threat detection.”
“We were looking for a practical way to improve cybersecurity awareness across the organization. Pistachio made training simple and engaging, and we’ve seen a noticeable improvement in employee awareness with minimal effort.”
“Pistachio removed the burden of planning campaigns, collecting data, and evaluating results. The program challenges each employee at their own level and provides clear, simple reporting. Pistachio has streamlined our awareness training and strengthened our organization’s cybersecurity.”
“Our goal was to help employees better recognize phishing emails and avoid malicious links. Pistachio made this easy and engaging, and we’ve seen a clear improvement in detection, fewer clicks on phishing links, and reduced data disclosure. With minimal ongoing maintenance, it’s an effective and low-effort solution.”
“Within weeks, Presence had flagged several incidents that could have otherwise been missed, including employees forwarding company data to personal accounts during offboarding. The alerts are reliable and actionable, so we've built a full investigation process around them. Although I was skeptical at first, the value was obvious almost immediately.”
“Pistachio has reduced our manual workload to zero with its automated phishing simulations and in‑box delivery of scenario‑based trainings. This has allowed our team to dedicate time to other important security initiatives while Pistachio handles our Security Awareness program seamlessly.”
“Presence automatically detects suspicious behaviour and gives us the visibility we need without constant manual oversight. Within the first few weeks we were already seeing alerts that would have otherwise gone unnoticed as the system learns what is usual and unusual behaviour for our environment.”
“Before Pistachio, security awareness was something we could only offer to a few customers. Today it’s part of every managed service we deliver, and we haven’t had to grow the team to make that happen.”
Fill out the form to book a demo:
Trusted by 1200+ customers & MSPs
