Finding Security Awareness Training Balance

Earlier this year, the National Institute for Standards and Technology (NIST) published updated recommendations for phishing simulations in security awareness training programs. We discussed it on our Community page soon after the updated standards were released, but the substance of the change bears repeating.

“Practical exercises include no-notice social engineering attempts to collect information, gain unauthorized access, or simulate the adverse impact of opening malicious email attachments or invoking, via spear-phishing attacks, malicious web links.” – NIST SP 800-53, Rev. 5, Section 5.3 (pg. 60)

This update includes a recommendation for “no-notice” phishing simulations to be delivered at the beginning of security awareness training programs to more accurately gauge the readiness of a set of users to recognize a phishing attempt. The thinking is obviously that letting users in on the phishing simulation game will heighten suspicion of their inbox and skew baseline results. This concern can be thought as a spin-off of the well-studied “observer effect” known in many scientific fields: Observing the behavior of something necessarily changes that behavior.

While it might be tempting for a chief information security officer (CISO) or other IT professional to take high grades on a phishing simulation as a sign of a job well done, that can be a dangerous conclusion to draw. Phishing tests that are too easy do little to address a problem that’s become one of the most common methods of entry for ransomware attacks. If IT professionals grade on a curve here, they’re doing very little to improve their organization’s overall cyber resilience.

Combatting this false sense of confidence about users’ ability to spot phishing attacks requires making sure simulations aren’t too easy to spot.

What Makes a Phishing Simulation Too Easy?

After putting some thought into that question, NIST researchers published a paper last year in the Journal of Cybersecurity citing three key criteria for determining if a phishing simulation makes for good training.

According to the authors, “low click rates do not necessarily indicate training effectiveness and may instead mean the phishing emails” were:

1. Too obvious: Either errors were too overt, or these templates were running something akin to the Nigerian Prince scam. Either way, they won’t help an employee overcome today’s more sophisticated phishing attempts.
2. Not relevant to staff: We’re all busy at work, so deleting an email offering 25% off at Ed’s Golf Cart Repair Shop doesn’t mean a user is an expert at spotting scams. It just means there was nothing in the simulation that enticed the user to click.
3. The phish was repeated or was similar to one that was: Phish me once, shame on me … But, seriously, this drives home the importance of having a wide range of phishing templates. These programs work best when they’re ongoing, so it’s important to switch it up.

On the other hand, a phishing simulation is convincing if it does the following to some degree:

• Mimics a workplace process or practice
• Has workplace relevance
• Aligns with other situations or events, including those external to the workplace
• Presents consequences for not clicking (for example, buy gift cards or we lose the client)
• References targeted training, specific warnings or other exposure

Tip: NIST has devised a weighted version of this scale, “the phish scale,” that you can use to determine the difficulty of your simulations. A phishing simulation that has all of the above characteristics would be considered extremely difficult. That’s good, right?  Click on Page 2 to continue reading…