Pushing Left, Like a Boss – Part 6: Threat Modelling

The last security-related part of the Design Phase of the System Development Life Cycle (SDLC) that we will talk about in this blog, is threat modelling, affectionately known as “evil brainstorming”.

The purpose of threat modelling is to discuss the possible threats to your system, then to do your best to mitigate them, and if not, to manage or accept the risks. There are multiple formalized methods for doing this, which I will not discuss here, each one already has its’ own book, advocate or dedicated blog, likely doing a better job detailing it than I ever could. In fact, my friend Adam Shostack, wrote an amazing book about it. Check it out!

That said, dear reader, I want you to understand why threat modelling is important, who needs to do it, as well as when and how you can start.

In order to create a threat model, a representative from each project stakeholder group needs to be present, this means someone from the business/someone representing the customer, a security rep, and someone from the development team. Yes, someone from the tech team needs to be there; they often have the most-frightening threat ideas!

Then you discuss what the risks are to the system. “What keeps you up at night?”, “If you were going to attack your app, how would you do it?”, “What threat actors should we be aware of? Should we prepare for?”, etc. You want to look at the system from the viewpoint of an attacker, what could go wrong? How could the system be misused? How can we ensure we protect the user (including from us)? What is the worst-case scenario? This session can be incredibly formal (creating attack trees, for instance), or quite informal (which is how I would suggest you start, if you have never done one before). You can read about two informal threat modelling sessions I documented on my blog; Serverless with Bryan Hughes and Robots with Jesse Hones.

Once you have a list of concerns, you will need to evaluate which ones are more (or less) likely and which may require security testing of your app (to see if it is vulnerable or not). You also need to evaluate which ones matter more or less; not all risks are created equal. You may be surprised (and frightened) by the justifications for the value of each risk; recently I had to deliver the news that the potential damage was “absolutely catastrophic”. Even though the risk itself was ‘fairly unlikely”, the project team changed their course of action immediately once they understood the potential long-term ramifications.

When you have your list of risks, and how much each one matters, you need to plan. Will you mitigate (fix/remove) some of the issues? Will you manage some, by keeping an eye on them to see if they get worse? Will you accept some of the risks? Perhaps some are highly unlikely or pose only a tiny threat? Why spend funds on something that is not worrisome?

The entire process should be documented, especially the decisions you make at the end, with management sign off. These decisions must be made with management or someone that has the authority to make large decisions like this. A software developer cannot “accept the risk”, nor likely can you as a security engineer; it is probable that you will need a C-level executive to accept risks that are above the level “medium” or “low”.

If you are doing an iterative design, you will need to do several shorter threat modelling sessions, for new features or large changes to existing features. If you are doing a large waterfall-style approach, one thorough session should be enough (assuming no large changes afterward). You will need to decide this for yourself and your org.

I threat model and/or do design reviews all the time, and I really enjoy it. I genuinely feel that the experience I’ve had threat modelling has made me a much better software developer, and likely a slightly more thoughtful person.

If you like what you have read here and want to delve in deeper with threat modelling, I suggest first reading up on STRIDE, and of course read Alice and Bob! After that, using your favorite search engine, look up “OWASP Threat Modelling”, STRIDE, and PASTA, as a start down your new path.