There are so many nuances necessary for considering, planning, and executing a phishing simulation, but fundamentally we have to start deeply believing and driving toward the overall outcome of the business’ key objectives.
More and more, organizations are asking themselves this very question and, in my opinion, there’s no definitive answer. There are so many nuances necessary for considering, planning, and executing a phishing simulation, but fundamentally we have to start deeply believing and driving toward the overall outcome of the business’ key objectives. But what are those? Low click rates? High click rates? No submitted credentials? Lots of submitted credentials? Many clients seem to lean toward the actions taken post a simulation becoming a measurable objective, which is great, but against what yardstick? If it’s off of a previous simulation, what was that based off of? This can be perplexing, but we’ve got to answer it, in as short a way as possible and as concisely as we can. This is what I’ve come up with.
When undertaking the responsibility of a phishing simulation, you have to start with the ‘what if’s’ and appropriately document actions for each of them. If they’re realized, it’s imperative that the documented actions and remedial steps from the planning phase are delivered upon, otherwise, what’s the point? But it’s not that simple. Throw in a multi-cultural group of international end-users, a rising millennial workforce, WFH and digital fatigue, overly detailed and antiquated security awareness training and you’ve got a massively complex task at hand.
You see, at a high-level (and having helped literally hundreds of customers through this journey) I’ve seen many strategies but ultimately these can be summarized in two main ways of execution; The ‘Cowboy’ way and the ‘Boy Scout’ way, each with their own pros and cons:
The Cowboy M/O:
Administrators choose an audience, usually company wide. They select the most sophisticated template, irrespective of insensitivity to certain topics, send it out and then log in 24hrs post-send, kind of review the data, proceeding to laugh at people who clicked a URL.
Pros:
This is precisely how criminals think. They don’t care about your feelings and if they want to bait your users with a particular hot or contentious topic they believe will work, they’re going to.
Quick to execute, with little barrier to entry.
Cons:
Rarely are the users who clicked followed up with, additionally trained or shown where they went wrong.
The results are presented at a high level as part of a security audit and the percentiles of each appropriate action are defined, but just not acted upon.
The Boy Scout M/O:
Administrators carefully decide on an audience (usually at random) but at the very least ensure its statistical relevance. They then use various ways to select a template, usually industry-based threat feeds, analyst reports, best practice documents from various vendors or a combination of these. Post send, they analyze the results with incredible depth, benchmarking departments, documenting clickers, etc. ultimately wanting to know why a particular user clicked it or submitted credentials.
Pros:
In most cases, this methodical approach bypasses unwanted heat from end-users because the templates used in the trickery don’t offend race, religion, gender, or opinion.
Documenting to this level of detail helps build an unbelievable deck of results, packed with post-simulation actions, remediation plans and compliance findings.
Cons:
Over-planning the pre and post actions of a simulation makes the execution of this tricky and cumbersome.
In most cases, this limits larger businesses to do this only once or twice a year, which kind of defeats its purpose. (Criminals attack your business hundreds of times, everyday).
Now, I know what you’re thinking, “Thanks for that. You’ve basically said no way is the right way and no matter what I do, I’m likely to either be the cause of a Twitter-storm because I hurt someone’s feelings, or I’m going to get fired for not executing”. Well, not so fast. The complexities of a companywide or even a targeted simulation extend just target audience and the appropriate sophistication levels.
You’ve got to consider wider influences, like:
Current ‘click’ behavior of various roles within your business – Most email gateways will report this to you.
Audience location – People in Russia don’t care about news stories RE: Boris Johnson’s shampoo regime.
Criminal activity in region – No one is attacking the native tribes of the Amazon.
Typical lures that previously caught your users, for real or in a simulation – think urgency, payment processing, deposits, etc.
Those are just the basics but give you a head start with the planning phase of a companywide phishing simulation. The next part is the execution and delivery. Once planned, you’ve now got to send this to your audience and this is incredibly daunting, because things can go wrong instantly and blow up spectacularly. Some advice here:
Always keep the simulation to only a handful of trusted individuals in the SecOps, IT and executive team.
Make sure you tag the mail with an invisible ID (usually a string of text that’s blanked).
Notify a mail administrator to setup a rule that routes reports of that simulation email (based on that ID) to a Helpdesk mailbox, monitored by a single, trusted Helpdesk analyst. This will alleviate some of the chaos that ensues in a global simulation campaign, but also give you insights into which users stuck to the right behavior.
Lastly, the data. Once this has run for 24, 48, 96, or however many hours or days you’d like it to, it’s time to start drilling into the data. Pull everything, as much as you can and start creating report items, like:
Overall Action (open, click, ignore) % of Overall Audience. If a large global audience, do this regionally.
Departmental statistics, either per region or BU, or both.
Click and/or report time. This is critical in understanding behavioral patterns of specific users, and groups of users.
Of all users who clicked, what is the mean access and privilege level to systems and networks, I.e., how exposed are you, if this were real?
Compare your report to data you can find online. Find similarities in organization size, industry, and region.
Running 3-5 of these per year, gives you an insurmountable array of data to build on, find patterns and ultimately ensure your overall awareness training strategy is helping this decline, rather than the contrary.
Comments