Measuring training effectiveness beyond click rates

Most security awareness programs measure success the same way: click rate. It's the easiest number to capture, it's the one that fits cleanly into a quarterly board slide, and it does have some signal value.

But click rate alone is a deeply misleading metric, and programs that optimize for it end up underperforming on the things that matter. Here are six metrics that — taken together — tell a much richer story about whether your program actually works.

1. Report rate

If you measure one thing besides click rate, measure this. Report rate is the percentage of test messages flagged via the official report mechanism, divided by the number sent.

Why it matters more than click rate:

• Click rate has a floor. You can't get to zero — there will always be a tail of users who click something. Beyond a certain point, the metric stops moving.
• Report rate has a ceiling that almost no organization hits. A mature program might see 35-50% report rate; many organizations sit at 5-10%.
• Click rate measures who got fooled. Report rate measures who engaged with the right reflex. The latter is what scales when real attacks come in.

A program that brings click rate from 12% to 8% but moves report rate from 7% to 28% has materially improved security posture, even though the click number didn't move dramatically.

2. Time-to-first-report

When you launch a campaign, how fast does the first user report it?

In a high-functioning program, the first report often comes within 5-15 minutes of the first send. In a low-functioning program, the first report might come hours later — or after multiple users have already clicked.

This metric is a leading indicator of cultural maturity. Programs with strong norms around reporting see TTFR drop steadily over time. Programs with weak norms see it stay flat or fluctuate randomly.

3. Repeat-clicker concentration

Not all clickers are created equal. In most organizations, a small fraction of users accounts for a large fraction of clicks. Identifying that population is high-leverage:

• The repeat-clickers (3+ clicks in 6 months) are usually well under 10% of users
• They often have outsized risk exposure (privileged access, financial responsibilities)
• Targeted intervention for this group has very high ROI

Track this metric explicitly. Programs that report "5% click rate across the org" hide the fact that their click rate is 0.5% for 90% of users and 30% for 10% of users. Those are very different risk pictures.

4. Real-attack catches

The hardest metric to capture but the most directly meaningful: how many real (non-simulated) attacks did your users report, and what happened to them?

This requires plumbing between your awareness platform and your SOC / IR / helpdesk. When a user reports a phishing message that turns out to be real, that's a data point worth tracking:

• Frequency tells you how often your workforce is the early-warning system for real threats
• Patterns tell you what kinds of attacks are landing
• Per-user data tells you who your strongest reporters are (worth thanking and recognizing)

Most organizations don't measure this because the plumbing is hard. Programs that figure out the plumbing get a metric that genuinely correlates with breach prevention.

5. Coverage and completion (with caveats)

The classic "% of workforce completed annual training" number. It's still useful — auditors and insurers ask about it — but should be:

• Disaggregated by department, role, and tenure
• Reported as engaged-completion (actually finished, not skipped) when possible
• Paired with content-relevance metrics (was this training role-appropriate?)

A 95% completion rate that's 95% click-through-skipping isn't doing what the number suggests. Modern platforms can measure engaged completion via quiz scores, content interaction time, and downstream behavior change.

6. Behavior change over time, per user

The hardest metric to capture but the most strategically valuable: does an individual user's behavior change as they go through the program?

Cohort-level metrics smooth over individual variance. A program might show a flat overall click rate while individual users are improving and being replaced by new hires who haven't trained yet. That's actually a successful program, but the aggregate metric hides it.

The more actionable view:

• For each user, track their click-and-report history over their tenure
• Identify users who improved (started clicking, stopped clicking) — what did the program do that worked?
• Identify users who didn't improve (kept clicking) — what's the targeted intervention?
• Identify users who're consistently strong reporters — recognize them, learn from their habits

What a good dashboard looks like

A dashboard that puts these together is more useful than any single-metric report:

Metric	Current quarter	YoY trend	Direction we want
Click rate	6.2%	-2.1pp	Down
Report rate	24%	+9pp	Up
Time-to-first-report	8 min	-22 min	Down
Repeat-clicker rate	4.1%	-1.3pp	Down
Real attacks caught (Q)	47	+18	Up
Engaged completion rate	89%	+6pp	Up

A board can read that dashboard. A CISO can present it. An auditor can sign off on it. And it actually tells you whether your program works.

What to do if your program only measures click rate

Three steps:

1. Add report rate this quarter. It's almost free if you have any modern simulation platform — most of them already capture it; you just need to surface it.

2. Capture time-to-first-report next. Slightly more work — your platform may need a small dashboard tweak — but again, the data is usually there.

3. Build the real-attack-catches plumbing in the next 6 months. This requires connecting your awareness platform to your SOC/IR. It's the most valuable metric long-term.

The shift from one-metric reporting to multi-metric reporting tends to land well with leadership — boards are tired of click-rate-only updates, and a richer dashboard lets you tell a more honest story about what's working and what isn't.

That honesty matters. Programs that report only what looks good get less funding when budgets tighten. Programs that report what's actually happening get believed when they ask for more.

---

Sources: Verizon 2025 Data Breach Investigations Report — Executive Summary, IBM Cost of a Data Breach Report 2025.