Measuring Security Awareness: Metrics That Actually Matter

Phishing simulations are running and training has been rolled out. Whether any of it works is another question, and sooner or later, usually when the next budget is due, leadership will ask for evidence. The good news: security awareness can be measured. The less good news: a single number tells you almost nothing. This article looks at the metrics that hold up and the conditions under which you should collect them.

Why a single number misleads

The click rate of one phishing simulation is the most frequently quoted and most frequently misread figure in awareness reporting. It depends heavily on how difficult the template was, when the simulation ran and which departments received it. A low click rate on an easily recognizable template says less than a higher one on a well-crafted, targeted scenario.

There is a second problem: someone who did not click has not necessarily learned anything. The email may simply have been overlooked or deleted unread. And turning a single metric into the sole target invites optimizing the number instead of the behavior, for example when simulations are quietly announced in advance so the results look good.

Meaningful insight only emerges when several metrics are read together over a longer period.

Four metrics that actually say something

• Click rate over time: What counts is not the result of one simulation but the trend across several simulations of comparable difficulty. If the click rate falls quarter after quarter while difficulty stays constant, that is a signal you can rely on.
• Report rate: The share of recipients who actively report a suspicious email. It may be the most underrated metric of all, because it measures desired behavior rather than the mere absence of mistakes. A rising report rate means your organization spots real attacks earlier.
• Time to first report: How quickly does the first report reach your security team? In a real attack, this window decides how much damage can be prevented.
• Training completion rates: They show whether your program reaches people at all. On their own they say nothing about behavior, but as a context metric they are indispensable. Without reach, there is no effect.

Reading the metrics together

Things get interesting when you combine the numbers. A falling click rate alongside a rising report rate is the pattern you want to see: employees recognize attacks and act correctly, too. High completion rates with a flat report rate, on the other hand, suggest content is being consumed but not landing in day-to-day work, a reason to rethink the format or frequency of your training.

It is also important to compare simulations by difficulty level rather than lumping all results together. Well-designed phishing simulations deliberately vary their scenarios. Your reporting has to reflect that variance, or it ends up comparing apples to oranges.

There is a regulatory angle too: NIS2 is in force, and organizations in scope must implement effective risk management measures, training included, and be able to demonstrate them. Credible awareness reporting is the foundation for that.

Anonymized reporting: works council and data protection as the framework

The obvious question is who clicked, and it points in the wrong direction. Evaluating simulations at the level of individual employees raises data protection issues and, in many European countries, requires involving the works council or other employee representatives. Above all, it undermines the goal itself: people who fear sanctions stop reporting incidents, and the report rate is your most valuable metric.

Anonymized, aggregated reporting is entirely sufficient for every KPI above, for example at the level of departments or locations, with minimum group sizes that rule out conclusions about individuals. You are measuring the program, not the person. Involve employee representatives and your data protection officer early. It builds trust, increases acceptance and ultimately improves the quality of your data.

From measuring to steering

Honestly measured awareness is not a surveillance tool but a steering tool: it shows where your program works, where it needs adjusting and how your organization's security level develops over time. Protectly provides exactly that foundation: click rates, report rates and completion rates over time, anonymized and works-council compliant. See how it looks on our Analytics & Reporting page, or experience it first-hand in a personal demo.