ChallengesField experiment · 2026
Rozema & Davis — Proceedings of the ACM Web Conference 2026 (WWW '26)
Training interventions had no significant effect on click rates or reporting rates, with negligible effect sizes. Phish Scale difficulty did predict behavior (7% clicks on easy emails vs 15% on hard), and in 36-55% of campaigns reports arrived before clicks, though training did not improve this.
MixedField experiment · 2026
Yin et al. — MIS Quarterly
Embedded feedback shown only to people who fail has limited reach and weaker field effects than lab studies suggested. Sending delayed feedback to all employees after a simulation (non-embedded training) emerged as a more promising way to reduce vulnerability over time.
MixedField experiment · 2025
Tolsdorf et al. — Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security (CCS '25)
Susceptibility and intervention effectiveness varied sharply by staff group, and risk from a few phishing emails lingered about three days. Technical measures (filtering, in-email warnings) worked best, generic [EXTERNAL] tags did little, and some staff reacted to the simulation with fear, shame, guilt, and hostility.
SupportsQualitative · 2025
Burda et al. — Information & Computer Security
The main reason employees report suspicious emails is wanting to protect and help the organization and coworkers, followed by responsibility, awareness of consequences, and feelings of insecurity. Reporters were more likely to report convincing, well-impersonated emails, showing they judge potential impact.
ChallengesRandomized controlled trial · 2025
Ho et al. — 2025 IEEE Symposium on Security and Privacy (SP)
Recent completion of annual awareness training had no significant link to failing phishing simulations, and embedded training produced only tiny differences in failure rates. Most users spent minimal time on training pages, and for some content types more training was associated with higher later failure rates.
MixedMixed methods · 2024
Lain et al. — Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS '24)
Whatever benefit embedded training has comes from the nudge of being periodically reminded of the threat, not from the training content, which employees rarely read. Delaying training was as effective as immediate training, rewards did not improve behavior, and phishing looked like an attention problem rather than a knowledge problem.
SupportsField experiment · 2024
Heiding et al. — IEEE Access, 12
Click-through was 19-28% for generic control phishing, 30-44% for GPT-4 generated emails, 69-79% for emails designed by hand using the V-Triad cognitive-bias rules, and 43-81% for GPT-4 combined with the V-Triad. Large language models were also fairly good at detecting phishing intent, sometimes beating humans, and cut attacker costs.
SupportsField experiment · 2024
Heiding et al. — arXiv:2412.00586
Fully AI-automated spear-phishing emails drew a 54% click-through rate, matching human experts (54%) and far above arbitrary control phishing (12%), a big jump from comparable AI results a year earlier. The AI's reconnaissance profiles were accurate and useful for 88% of targets, and AI can raise attacker profitability up to 50-fold at scale.
SupportsQualitative · 2023
Pilavakis et al. — Proceedings 2023 Symposium on Usable Security (USEC 2023)
People who report suspected phishing typically describe evidence they noticed, possible impacts, what they did or did not do, and questions they have. Some build clear arguments for why the email is phishing and why the organization should act.
SupportsMixed methods · 2023
Jacobs et al. — HCI International 2023 (HCI in Business, Government and Organizations), Lecture Notes in Computer Science
Government security awareness programs lean heavily on compliance metrics such as training completion rates and struggle to find other ways to judge whether behavior actually changed.
MixedField experiment · 2022
Lain et al. — 2022 IEEE Symposium on Security and Privacy (SP)
Embedded training shown after failing a simulation did not make employees more resilient and could even increase susceptibility, while email warnings helped. Employees reporting suspicious emails worked as a fast, sustainable crowd-sourced detection system, with reporters staying active over long periods.
FoundationalCase study · 2021
Althobaiti et al. — Proceedings of the ACM on Human-Computer Interaction (CSCW)
Handling phishing reports is a distributed process across several teams, each with narrow system access and knowledge. Sudden large campaigns flooded the help desk with reports, disrupting work and slowing mitigations and reflection.
SupportsField experiment · 2020
Reinheimer et al. — Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020)
After an awareness program, employees identified phishing and legitimate emails significantly better right away and at four months, but the gain was gone by six months. Reminders based on videos and interactive examples worked best and lasted at least another six months.
FoundationalMixed methods · 2020
Steves et al. — Journal of Cybersecurity
Click rates should be expected to vary with how hard a phishing email is for a given audience, especially when its premise fits the recipient's work context. The authors propose the NIST Phish Scale so programs can rate exercise difficulty and interpret click rates.
FoundationalMixed methods · 2020
Veiga et al. — Computers & Security, 92, 101713
Combines a scoping review with a 512-respondent industry survey to define information security culture, identifying 5 external and 20 internal influencing factors. Academic definitions were much broader than industry's, and strong cultures were linked to mutual trust and integrity.
SupportsQualitative · 2020
Wash — Proceedings of the ACM on Human-Computer Interaction (CSCW)
Experts detect phishing in three stages: making sense of the email and noticing small discrepancies, becoming suspicious when something (usually a link asking for action) triggers the phishing explanation, then investigating and deleting or reporting. Training should build this sensemaking process, not just checklists.
MixedLarge-scale observational · 2019
Heijden & Allodi — 28th USENIX Security Symposium (USENIX Security 19), pp. 1309-1326
Measuring Cialdini-style persuasion triggers in real reported phishing let the authors predict which attacks would draw the most clicks, enabling response teams to prioritize takedowns. Consistency and scarcity triggers were associated with more clicks, reciprocity appeared counterproductive, and authority, social proof and liking showed no clear trend.
MixedSystematic review · 2019
Renaud & Dupuis — Proceedings of the New Security Paradigms Workshop (NSPW '19)
A review of the wider fear-appeal literature finds real disagreement over whether fear appeals are helpful or advisable, and wide variation in how cyber security fear-appeal experiments are designed. The authors propose a standard protocol for such studies.
SupportsConceptual · 2019
Zimmermann & Renaud — International Journal of Human-Computer Studies, 131, 169-187
A problematization analysis finds government, industry and hacker discourse treats humans as the problem, with controls designed to constrain them. The authors propose 'Cybersecurity, Differently', which assumes people are well-intentioned and builds on what contributes to positive outcomes and resilience.
FoundationalQualitative · 2019
Ferreira & Teles — International Journal of Human-Computer Studies
Merging Cialdini, Gragg and Stajano & Wilson into one set of social-engineering persuasion principles, the authors found authority, strong affect, integrity and reciprocation were the most common in phishing subject lines. Strong-affect and authority emails leaned on 'you/your' wording, reciprocation on 'we/us/our'.
SupportsLab experiment · 2019
Parsons et al. — International Journal of Human-Computer Studies
In a role-play study of 985 people, emails using consistency and reciprocity were most effective while scarcity and social proof were least effective. People who scored as susceptible to a given principle were usually more fooled by emails using it, and age, computer time, social-proof susceptibility and impulsivity predicted detection ability.
MixedField experiment · 2019
Lin et al. — ACM Transactions on Computer-Human Interaction (TOCHI)
43% of participants clicked at least one simulated phishing email, with older women most susceptible. Young users' susceptibility dropped over the 21 days while older users' stayed flat, and the effectiveness of each persuasion technique and life-domain topic varied by age group. Older users also rated their own susceptibility lower than it was.
SupportsMixed methods · 2018
Williams et al. — International Journal of Human-Computer Studies
Across a simulation sent to about 62,000 employees, emails carrying authority cues raised the likelihood of clicking a suspicious link. Focus groups pointed to workplace factors, such as routine email habits and work pressures, that shape whether employees fall for spear phishing.
SupportsConceptual · 2018
Renaud et al. — Computers & Security, 78
Argues that governments shifting cyber risk onto individual citizens ('responsibilization'), by issuing advice and leaving consequences to them, is contributing to cybercrime's success. Proposes a more active risk-regulation regime instead.
SupportsLab experiment · 2018
Vishwanath et al. — Communication Research, 45(8), 1146-1166
Because training effects fade as people slip back into email routines, the authors built a model (SCAM) combining conscious cognitive processing, preconscious suspicion and habitual, automatic email use, and tested it across two phishing experiments. Email habits emerged as a key predictor of susceptibility alongside cognitive processing.
SupportsMixed methods · 2018
Vance et al. — MIS Quarterly, 42(2), 355-380
Attention to repeated security warnings declined measurably in the brain across a workweek, partially recovering between days. In the field, adherence to permission warnings fell over three weeks, while warnings whose appearance varied (polymorphic designs) substantially reduced this habituation.
SupportsField experiment · 2018
Wash & Cooper — Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
Facts-and-advice training beat no training only when presented by a security expert, while story-based training worked much better when told by a peer. Who delivers training can strongly change security outcomes.
SupportsQualitative · 2016
Stanton et al. — IT Professional, 18(5)
Although the interviews never asked about fatigue, over half of the 40 participants described it: resignation, loss of control, fatalism, risk minimization and decision avoidance. This fatigue fed their sense that following security advice has little benefit.
SupportsConceptual · 2015
Bada et al. — International Conference on Cyber Security for Sustainable Society, 2015 (arXiv:1901.02672, posted 2019)
Awareness campaigns fail when they only provide information: people must be able to understand and apply advice and be motivated to act, which requires attitude and intention change. Reviews persuasion techniques, including fear appeals, and lists factors behind campaign success or failure.
SupportsConceptual · 2015
Sasse — IEEE Security & Privacy, 13(3), 80-83
Argues that people heed reliable, credible risk signals, but high false-positive security mechanisms teach users to ignore them. Instead of scaring, tricking or bullying users, security needs more accurate detection and less obstructive tools.
ChallengesField experiment · 2014
Caputo et al. — IEEE Security & Privacy
Differently framed embedded training messages had no significant effect on whether people clicked later spear-phishing emails, and many people either clicked every link or none regardless of training. Employees largely did not read the training materials.
SupportsQualitative · 2014
Kirlappos et al. — Workshop on Usable Security (USEC 2014)
Beyond comply/not-comply, security-conscious employees who cannot follow policy build their own workarounds ('shadow security') that balance getting work done with managing risk. The authors recommend learning from these practices rather than stamping them out.
MixedSurvey · 2010
Johnston & Warkentin — MIS Quarterly, 34(3), 549-566
Fear appeals did increase people's intentions to adopt recommended security actions, but the effect varied across people and depended partly on self-efficacy, response efficacy, perceived threat severity and social influence.
FoundationalField experiment · 2010
Lally et al. — European Journal of Social Psychology, 40(6), 998-1009
Repeating a chosen behavior daily in a consistent context made it more automatic along an asymptotic curve. Time to reach 95% of peak automaticity ranged from 18 to 254 days, and missing a single day did not materially derail habit formation.
SupportsSurvey · 2010
Siponen & Vance — MIS Quarterly, 34(3), 487-502
Employees' rationalizations for rule-breaking ('neutralization' techniques from criminology) explained intentions to violate security policy better than deterrence theory's sanctions. The authors argue policies should address these rationalizations.
MixedMixed methods · 2010
Kumaraguru et al. — ACM Transactions on Internet Technology
Embedded email-based training (PhishGuru) and a game (Anti-Phishing Phil) built on learning-science principles improved users' ability to recognize phishing. The authors frame user education as a complement to automated detection, noting that users are unmotivated and that training can raise false alarms on legitimate messages.
FoundationalConceptual · 2009
Kahneman & Klein — American Psychologist, 64(6), 515-526
A skeptic of intuition (heuristics-and-biases) and a champion of it (naturalistic decision making) agree that intuitive judgments can be trusted only when the environment is regular enough to be learnable and the person has had lots of practice with rapid, clear feedback.
MixedSurvey · 2009
Herath & Rao — European Journal of Information Systems, 18(2)
Threat severity, response efficacy, self-efficacy and response costs shaped attitudes to security policy. Organisational commitment and social influence significantly drove compliance intentions, and available resources boosted self-efficacy. Employees underestimated how likely breaches were.
SupportsConceptual · 2009
Herley — Proceedings of the 2009 New Security Paradigms Workshop (NSPW '09)
Argues that users ignoring security advice is economically rational: advice imposes large, constant effort costs while its benefits are often speculative. For example, the time cost of everyone checking URLs would dwarf all phishing losses.
ChallengesSurvey · 2009
D'Arcy et al. — Information Systems Research, 20(1)
User awareness of security policies, SETA programs and computer monitoring each deterred intentions to misuse IS, working through perceived sanctions. Perceived severity of sanctions mattered more than certainty, and the effect varied with individuals' morality.
SupportsQualitative · 2008
Beautement et al. — Proceedings of the 2008 New Security Paradigms Workshop (NSPW '08)
Employees decide whether to comply by weighing the personal costs and benefits of compliance against perceived benefit to the organization. Proposes the 'Compliance Budget': a finite store of goodwill that security demands draw down and that must be managed.
FoundationalField experiment · 2008
Workman — Journal of the American Society for Information Science and Technology, 59(4), 662-674
Drawing on marketing-persuasion theory, a field study tested whether the factors that make marketing campaigns work also explain who falls for phishing and pretext phone attacks, and found that they do: dispositional factors tied to persuasion predicted victimization.
FoundationalConceptual · 2007
Wood & Neal — Psychological Review, 114(4), 843-863
Habits are learned links between context cues and responses that form through repetition and can then be triggered directly by those cues without the goal that originally drove them. Goals shape habits mainly by motivating the early repetition and by steering people toward cues, rather than by directly activating the habit.
FoundationalConceptual · 2000
Ryan & Deci — American Psychologist, 55(1), 68-78
Reviewing research on self-determination theory, the authors argue that people's intrinsic motivation and internalization of rules depend on meeting three basic needs: competence, autonomy and relatedness. Controlling environments undermine these needs and motivation, while supportive ones foster them.
FoundationalMixed methods · 1999
Edmondson — Administrative Science Quarterly, 44(2)
Introduces team psychological safety: a shared belief that it is safe to take interpersonal risks. Psychological safety, not team efficacy, was associated with learning behavior such as seeking feedback and discussing errors, and learning behavior mediated the link to team performance.
SupportsMixed methods · 1999
Adams & Sasse — Communications of the ACM, 42(12)
Insecure password practices (e.g., writing passwords down, linking passwords) stemmed from memory overload and poorly designed policies, not user carelessness. Security departments that withheld information and treated users as a threat worsened motivation and compliance.
FoundationalLab experiment · 1997
Bechara et al. — Science, 275(5304), 1293-1295
Healthy participants began choosing advantageously and showed anticipatory skin-conductance responses to risky options before they could explain which strategy was best. Patients with prefrontal damage never developed these signals and kept choosing badly even after knowing the right strategy.
FoundationalConceptual · 1977
Bandura — Psychological Review, 84(2), 191-215
People's expectations that they can successfully perform a behavior determine whether they try, how hard they work and how long they persist. These efficacy beliefs are built mainly through actual successful performance, and also through watching others, verbal persuasion and emotional arousal.