The evidence, for and against

Training interventions had no significant effect on click rates or reporting rates, with negligible effect sizes. Phish Scale difficulty did predict behavior (7% clicks on easy emails vs 15% on hard), and in 36-55% of campaigns reports arrived before clicks, though training did not improve this.

Embedded feedback shown only to people who fail has limited reach and weaker field effects than lab studies suggested. Sending delayed feedback to all employees after a simulation (non-embedded training) emerged as a more promising way to reduce vulnerability over time.

Susceptibility and intervention effectiveness varied sharply by staff group, and risk from a few phishing emails lingered about three days. Technical measures (filtering, in-email warnings) worked best, generic [EXTERNAL] tags did little, and some staff reacted to the simulation with fear, shame, guilt, and hostility.

Recent completion of annual awareness training had no significant link to failing phishing simulations, and embedded training produced only tiny differences in failure rates. Most users spent minimal time on training pages, and for some content types more training was associated with higher later failure rates.

Whatever benefit embedded training has comes from the nudge of being periodically reminded of the threat, not from the training content, which employees rarely read. Delaying training was as effective as immediate training, rewards did not improve behavior, and phishing looked like an attention problem rather than a knowledge problem.

Embedded training shown after failing a simulation did not make employees more resilient and could even increase susceptibility, while email warnings helped. Employees reporting suspicious emails worked as a fast, sustainable crowd-sourced detection system, with reporters staying active over long periods.

Click rates should be expected to vary with how hard a phishing email is for a given audience, especially when its premise fits the recipient's work context. The authors propose the NIST Phish Scale so programs can rate exercise difficulty and interpret click rates.

43% of participants clicked at least one simulated phishing email, with older women most susceptible. Young users' susceptibility dropped over the 21 days while older users' stayed flat, and the effectiveness of each persuasion technique and life-domain topic varied by age group. Older users also rated their own susceptibility lower than it was.

Across a simulation sent to about 62,000 employees, emails carrying authority cues raised the likelihood of clicking a suspicious link. Focus groups pointed to workplace factors, such as routine email habits and work pressures, that shape whether employees fall for spear phishing.

Facts-and-advice training beat no training only when presented by a security expert, while story-based training worked much better when told by a peer. Who delivers training can strongly change security outcomes.

Argues that people heed reliable, credible risk signals, but high false-positive security mechanisms teach users to ignore them. Instead of scaring, tricking or bullying users, security needs more accurate detection and less obstructive tools.

Differently framed embedded training messages had no significant effect on whether people clicked later spear-phishing emails, and many people either clicked every link or none regardless of training. Employees largely did not read the training materials.