Research / Principles
Principle 8 of 8
Trained human instinct, the sense that a request is wrong, is the best defense against social engineering, and it improves with practice.
Click-through was 19-28% for generic control phishing, 30-44% for GPT-4 generated emails, 69-79% for emails designed by hand using the V-Triad cognitive-bias rules, and 43-81% for GPT-4 combined with the V-Triad. Large language models were also fairly good at detecting phishing intent, sometimes beating humans, and cut attacker costs.
Fully AI-automated spear-phishing emails drew a 54% click-through rate, matching human experts (54%) and far above arbitrary control phishing (12%), a big jump from comparable AI results a year earlier. The AI's reconnaissance profiles were accurate and useful for 88% of targets, and AI can raise attacker profitability up to 50-fold at scale.
People who report suspected phishing typically describe evidence they noticed, possible impacts, what they did or did not do, and questions they have. Some build clear arguments for why the email is phishing and why the organization should act.
Experts detect phishing in three stages: making sense of the email and noticing small discrepancies, becoming suspicious when something (usually a link asking for action) triggers the phishing explanation, then investigating and deleting or reporting. Training should build this sensemaking process, not just checklists.
Because training effects fade as people slip back into email routines, the authors built a model (SCAM) combining conscious cognitive processing, preconscious suspicion and habitual, automatic email use, and tested it across two phishing experiments. Email habits emerged as a key predictor of susceptibility alongside cognitive processing.
A skeptic of intuition (heuristics-and-biases) and a champion of it (naturalistic decision making) agree that intuitive judgments can be trusted only when the environment is regular enough to be learnable and the person has had lots of practice with rapid, clear feedback.
Habits are learned links between context cues and responses that form through repetition and can then be triggered directly by those cues without the goal that originally drove them. Goals shape habits mainly by motivating the early repetition and by steering people toward cues, rather than by directly activating the habit.
Healthy participants began choosing advantageously and showed anticipatory skin-conductance responses to risky options before they could explain which strategy was best. Patients with prefrontal damage never developed these signals and kept choosing badly even after knowing the right strategy.
Uses the reporting button or channel whenever a message feels wrong, whether or not they clicked.
Treats deadlines, threats, and 'act now' language as a signal to slow down, not speed up.
Confirms payment changes, credential requests, and odd asks by calling or messaging the person through a known, separate channel.
Stops when a message feels wrong, even before they can say exactly why.
Checks before acting when a message invokes a boss, executive, government body, or IT department.
Stays helpful, but confirms who is asking before sharing files, codes, or access.
Does not treat claims that coworkers already acted as a reason to act.
No longer relies on typos and bad grammar as the main warning sign.
Invests in filtering and warnings first, and treats trained people as the layer that catches what gets through.