The evidence, for and against
6 studies · Download CSV
Embedded feedback shown only to people who fail has limited reach and weaker field effects than lab studies suggested. Sending delayed feedback to all employees after a simulation (non-embedded training) emerged as a more promising way to reduce vulnerability over time.
Content, Nudges and Incentives: A Study on the Effectiveness and Perception of Embedded Phishing Training Lain et al., Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS '24)Whatever benefit embedded training has comes from the nudge of being periodically reminded of the threat, not from the training content, which employees rarely read. Delaying training was as effective as immediate training, rewards did not improve behavior, and phishing looked like an attention problem rather than a knowledge problem.
"I didn't click": What users say when reporting phishing Pilavakis et al., Proceedings 2023 Symposium on Usable Security (USEC 2023)People who report suspected phishing typically describe evidence they noticed, possible impacts, what they did or did not do, and questions they have. Some build clear arguments for why the email is phishing and why the organization should act.
Teaching Johnny not to fall for phish Kumaraguru et al., ACM Transactions on Internet TechnologyEmbedded email-based training (PhishGuru) and a game (Anti-Phishing Phil) built on learning-science principles improved users' ability to recognize phishing. The authors frame user education as a complement to automated detection, noting that users are unmotivated and that training can raise false alarms on legitimate messages.
Conditions for intuitive expertise: A failure to disagree Kahneman & Klein, American Psychologist, 64(6), 515-526A skeptic of intuition (heuristics-and-biases) and a champion of it (naturalistic decision making) agree that intuitive judgments can be trusted only when the environment is regular enough to be learnable and the person has had lots of practice with rapid, clear feedback.
Psychological Safety and Learning Behavior in Work Teams Edmondson, Administrative Science Quarterly, 44(2)Introduces team psychological safety: a shared belief that it is safe to take interpersonal risks. Psychological safety, not team efficacy, was associated with learning behavior such as seeking feedback and discussing errors, and learning behavior mediated the link to team performance.