The evidence, for and against

Embedded feedback shown only to people who fail has limited reach and weaker field effects than lab studies suggested. Sending delayed feedback to all employees after a simulation (non-embedded training) emerged as a more promising way to reduce vulnerability over time.

Recent completion of annual awareness training had no significant link to failing phishing simulations, and embedded training produced only tiny differences in failure rates. Most users spent minimal time on training pages, and for some content types more training was associated with higher later failure rates.

Whatever benefit embedded training has comes from the nudge of being periodically reminded of the threat, not from the training content, which employees rarely read. Delaying training was as effective as immediate training, rewards did not improve behavior, and phishing looked like an attention problem rather than a knowledge problem.

Embedded training shown after failing a simulation did not make employees more resilient and could even increase susceptibility, while email warnings helped. Employees reporting suspicious emails worked as a fast, sustainable crowd-sourced detection system, with reporters staying active over long periods.

Differently framed embedded training messages had no significant effect on whether people clicked later spear-phishing emails, and many people either clicked every link or none regardless of training. Employees largely did not read the training materials.

Embedded email-based training (PhishGuru) and a game (Anti-Phishing Phil) built on learning-science principles improved users' ability to recognize phishing. The authors frame user education as a complement to automated detection, noting that users are unmotivated and that training can raise false alarms on legitimate messages.