Measures reporting first, clicks second

Why it matters

Programs that track only completion and click rates struggle to tell whether behavior changed. Click rates also swing with email difficulty, while reporting shows active defense.

How to observe it

Which metric leads board and leadership reports.

Evidence

Supporting (3)

• Measuring the Effectiveness of U.S. Government Security Awareness Programs: A Mixed-Methods Study
  Jacobs et al., 2023 Supports

  Government security awareness programs lean heavily on compliance metrics such as training completion rates and struggle to find other ways to judge whether behavior actually changed.

• Phishing in Organizations: Findings from a Large-Scale and Long-Term Study
  Lain et al., 2022 Mixed

  Embedded training shown after failing a simulation did not make employees more resilient and could even increase susceptibility, while email warnings helped. Employees reporting suspicious emails worked as a fast, sustainable crowd-sourced detection system, with reporters staying active over long periods.

• Categorizing human phishing difficulty: a Phish Scale
  Steves et al., 2020 Foundational

  Click rates should be expected to vary with how hard a phishing email is for a given audience, especially when its premise fits the recipient's work context. The authors propose the NIST Phish Scale so programs can rate exercise difficulty and interpret click rates.