In the previous article, ‘Building Security Champions’, we covered what champions are, why you need them, and our plan to make an amazing program.
The #1 most important rule of recruiting security champions is that you must attract them. Do not “voluntell” someone to be a security champion. That person is not going to do their best for you, and they certainly won’t enjoy the experience. Attract the right people instead of forcing them.
How does one ‘attract’ champions?
Use lunch and learns to teach about security
Arrange security training
Anyone who asks questions or attends all the events is a potential champion
Use interesting titles for events if you can
Add a note to your email signature, saying you are looking for champions
Put a sign on the fridge in the kitchen
Talk about it at the all-staff meeting
Send an email to all of IT
Pay attention to who responds, attends events, asks questions, and who is ‘always there'. Those are the people you need.
Adjust Your Attitude
Change your team’s mantra to “I am here to serve you” and your team will attract even more candidates. Saying “you are my customers” to the rest of IT if you are a security professional, is basically the truth. Plus, you always get more bees with honey.
#2 most important rule of recruiting: ensure their manager is on board. You don’t want this person to have to fight to do work for you or feel conflicted. Ensuring their manager is comfortable.
In the next article, we will talk about how to engage your champions (which will result in you finding even more).
Most of us that work in cybersecurity are well aware that there are not enough people to fill all of the positions that we have opened. There is a severe shortage of trained and experienced people who are capable of securing the systems that we must protect. Application security engineers, DevSecOps professionals, security architects, you name it, there’s a shortage.
We will never have the staff, budget or time to do all the security work we want to do.
One of the ways that we can address this is by scaling our security teams and programs. When I say scaling, I don't mean what you do to a fish after you catch it. I mean finding a way to do more, with less. This can involve automation, creating self-service systems, and many other potential solutions. In this series of blogs, we will discuss how you can solve this problem by building a security champions program for your organization.
What IS a security champion?
A Security Champion is a member of a team that takes on the responsibility of acting as the primary advocate for security within the team and acting as the first line of defense for security issues within the team.
Or more plainly:
The person who is most excited about security on a team. They want to read the book, fix the bug, or ask security questions. Every time.
Tell me more!
Security champions are your communicators. They deliver security messages to each dev team, teaching, sharing, and helping.
They are your point of contact, delivering messages to and from the security team, and keeping you up to date on what matters to your team.
They are your advocate. They perform security work, for their dev team, with your help.
They also advocate for security, asking questions in situations you would have been left out of. Raising concerns, you might have missed. They are a peer for everyone on their team and can influence in ways that you yourself cannot.
In the next few posts, we will cover how to build an amazing security champions program! We will follow this recipe:
If you want to learn it all right now, I have a conference talk on this topic already, which covers much of what these posts will cover. Feel free to watch it. I gave it at B-Sides Vancouver, an AMAZING community-led conference, close to where I live.
For those of you hiding under a rock, OWASP is an international non-profit foundation which supports over 100 open source projects, over 300 meetup chapters worldwide and regular large international conferences, all with the aim of helping everyone to build secure software. Since We Hack Purple's mission is very similar, we thought a partnership was very necessary!
As part of the OWASP & We Hack Purple partnership, all OWASP members are now provided free access to the Application Security Foundations Level 1 course from WHP! This introductory AppSec course will answer all your burning questions and define all the technical terms right at the start. Then we will set goals for your AppSec program at work as an exercise. After this, we dive deep into every type of application security activity and tool on the market while sprinkling you with quizzes and exercises. As a final project, we make an AppSec program action plan for you to bring back to work with you. This on-demand course is FREE for all OWASP members!
To access the course, read on, sign up with your OWASP.org email address, and start learning.We Hack Purple’s Application Security Foundations Level 1 course consists of 108 short modules. It is virtually trained by Tanya Janca, our head nerd at WHP. All you need to do is be an OWASP member and sign up! If you’re not a member, joining or renewing a lapsed OWASP membership is easy! Just remember to sign up using your owasp.org emailso you can complete the We Hack Purple registration process.Join OWASP as a new member, or renew your membership
** This article is for beginners in security or other IT folk, not experts. 😀
The mandate and purpose of every IT security team is to protect the confidentiality, integrity and availability of the systems and data that the company, government or organization that they work for. That is why the security team hassles you about having administrator rights on your work machine, won’t let you plug unknown devices into the network, and all the other things that feel inconvenient; they want to ensure these three things. We call it “The CIA Triad” for short.
Let’s examine this using examples with our friends Alice and Bob. Alice has type 1 diabetes and uses a tiny device implanted in her arm to check her insulin several times a day, while Bob has a ‘smart’ pacemaker that regulates his heart, which he accessed via a mobile app on this phone.
If Alice’s insulin measuring device was unavailable due to malfunction, tampering or if it ran out of batteries, she may end up being inconvenienced. Since Alice usually checks her insulin levels several times a day, but she is able to do manual testing of her insulin if she needs to, it is somewhat important to her that this service is available.
Bob, on the other hand, has irregular heartbeats from time to time, and never knows when his arrhythmia will strike. If Bob’s pacemaker was not available when his heart is behaving erratically, this could be a life or death situation if enough time elapses. It is very important that his pacemaker is available, and that it reacts in real-time when an emergency happens. Bob works for the federal government as a clerk managing secret and top-secret documents and has for many years. He is a proud grandfather and is trying hard to live a healthy live since his pacemaker was installed.
Alice is the CEO of a large fortune 500 company, and although she is not ashamed that she is a type 1 diabetic, she does not want this information to become public. She is often interviewed by the media and does public speaking, serving as a role model for many other women in business. Alice works hard to keep her personal life private, and this includes her health condition. She believes that some people within her organization are after her job and would do anything to try to portray her as ‘weak’ in efforts to undermine her authority. If her device where to accidentally leak her information, show itself on public networks, or if her account information became part of a breach, this would be highly embarrassing for her, and potentially damaging to her career. Keeping her private life private is very important to Alice.
Bob, on the other hand, is very open about his heart condition and happy to tell anyone that he has a pacemaker. He has a great insurance plan with the federal government and is very grateful that when he retires that he can continue with his plan, despite his pre-existing condition. Confidentiality is not a priority for Bob in this respect.
Integrity in data means that the data is correct and accurate. Integrity in a computer system means that the results it gives you are precise and factual. For Bob and Alice, this may be the most important of CIA factors: if either of their systems give them incorrect treatment it could result in death. For a human being (as opposed to a company or nation-state), there does not exist a more serious repercussion than end of life. The integrity of their health systems is crucial to ensuring they both remain in good health.
The next time you IT Security team is enforcing a new policy or asking for security requirements as part of your project, I challenge you to try to figure out which of the CIA triad they might be trying to protect with their actions. Sometimes it is a combination of 2 or of the triad. More importantly, I challenge you to ask yourself when you work if you are ensuring each of these three factors of security are being protected by your actions, as the IT Security team cannot succeed alone.
The real reason companies do software inventory exercises is to have a sense of control over their application portfolio. Because you need to KNOW that everything is tested, monitored and patched. That feeling of being certain, rather than ‘fairly confident’, is the difference between knowing that you are doing a good job of securing your software and just hoping that you are.
Most enterprise-level companies have hundreds or even thousands of assets, and do not have accurate inventory of them. This happens for many reasons; developers don’t always report their work, no one is assigned to keeping track of application updates, shadow IT, people not realizing that an API or SaaS (Software as a Service) qualifies as an ‘app’, APIs being split into multiple APIs and the new one(s) not being recorded, lost excel spreadsheets, other priorities getting in the way, employee turnover, etc. There are too many reasons to list, and the cause of the problem doesn’t matter as much as what can be done about it.
Why is having an incomplete inventory of your web assets a problem? Because you can’t protect something if you don’t know you have it.
You might be saying, “All of our applications are released via our standard DevOps Pipeline.” Sure, all the ones you are aware of go through it. What about the ones that you don’t know about?
As you might have guessed by this point; I was working at a company that did application inventory when I wrote this article. Trust me dear reader, even if I don’t work there anymore inventory is still a very important issue.
Story from Tanya:
Back in the day when I was a software developer, one of the government departments that I worked for hired a consultant to do ‘application portfolio management’, his name was Sanjeev. Sanjeev was the coolest guy in our office, well-dressed, charming, always in the right meetings with the powerful people. As you can guess, Sanjeev was paid a lot more than I was. His entire project, for the whole year, was to interview my 14 team members and I about which apps we had, to get links to where they were, and gather any documentation we had about them. He was a patient guy, talking to developers all day and asking us for information that we just didn’t have. Also, none of us thought what he was doing was important, and he was sometimes treated as a pest.
Long story short, we thought we had 200 apps, but he only found evidence of 72. Knowing what I know now…. I wonder how many we really had.
In order to perform a manual application inventory you could interview members of your dev teams and ask them what apps they have. Although having good communication with the dev team is always a good idea, this would be a long and tedious process if you are at a large company. If you are reading this blog, you are likely to be looking for a more technical solution, so let’s look at how we can try to automate this.
For a DevOps shop (those using automated pipelines for releases), inventorying all of the apps and APIs released from various pipelines would be a good starting point. For example: add triggers to the git `pre-push` hook or to a Jenkins pipeline. Any that are not released via a pipeline would not be included, but ideally this would cover the majority of your newer applications.
From there you could look at this from a network angle. If you have the IP range(s) for your network and you have permission to scan them, you may run a network scanning tool such as nmap or masscan against the IP range and look for open ports on TCP port 80 (HTTP) and 443 (HTTPS). If either of these ports are open, they are likely running something web-related. From there you could use an automated tool such as EyeWitness to visit each one of the IP addresses, one at a time, and take a snapshot of the page to see what’s there. Or you could hire someone and have them visit each one manually, to find out what is there.
There are a few problems with the network scanning approach.
Web applications are not restricted to running only on port 80 or 443 (ask most developers where they test locally, and they’ll probably tell you port 8080). Web servers are capable of having several APIs and apps on them as well; they could be separated by ports or through the host name sent in the request. This means that you may not even see all the applications or APIs at a given IP address with a simple network scan.
You may not even be able to reach all your IP addresses due to firewalls, VLANS, or physical network separation.
What if your applications are hosted in the cloud? In cloud environments, IP addresses change frequently, and applications cycle rapidly (for example — serverless applications). You’re certainly not going to scan all of your cloud provider’s network hosts; you neither have the time (that’s a lot of IP addresses) nor the legal permission (remember — the “cloud” is just someone else’s computer) to do so.
Another approach could be to do an external scan of your domain name(s).If you know all of your domain names, you could use a tool like OWASP Zap or Burp Suite to crawl all the links on your domain and have it find as many pages as it can.
Again, there are issues with this method as well, mostly around visibility.
Crawling software can only see what it has access to. If there is no link on a page to an API call, it will never see that API. Are you using MFA (multi-factor authentication)? If so, the crawler won’t be able to get around that (it’s literally what MFA was designed for). Do you have authentication or authorization in your applications? If so, you probably don’t want to write evergreen login credentials into an application (this is also known as a “backdoor”) and then give those credentials to a crawler. That crawler may also hit some sensitive parts of your application (e.g. admin functionality) and wreak wanton destruction to your database, users, or critical business functions.
An external network agent will only be able to access what an external network user will be able to access. This means that it does not give a complete and accurate view of your applications.
If your domain list is incomplete, or there are subdomains you didn’t know about, again — the crawler will completely miss these too.
These approaches all come down to two main problems:
1. These actions require significant overhead to continuously update your targets and initiate scans.
2. You will never be able to get a complete view of your applications and their functionality.
I ask you to ponder the following question:
How do you create and maintain your application inventory?
In this series we are discussing how to get your technical training approved at work. This is not the first article, and you may want to go back and read it from the start.
In the previous article, we talked about how we need to explain to our boss not only which training we want, but we must overcome any objections, if we are going to get it approved. Let’s look at the second objection in our list.
Back in the day I requested approval to take a web-app hacking course, and I recall my boss saying, “You don’t need to learn that; you just need to run the scanner.” My job was web app penetration testing, and it was clear my boss had no idea what I did all day. He seemed to think that manual security testing was unnecessary, and at the time I had no idea how to explain we needed a lot more if we wanted to ensure our apps were very secure. I ended up watching a lot of videos on the internet, playing around, and wasting a ton of time.
When I switched over into Application Security, it got even more difficult, as most of the courses only offered to teach me “the OWASP Top Ten” (which I already knew well), and then the main security controls (authentication, authorization, encryption, identity) and little else. I wanted to know how to do my job, not theory and not basic web app hacking (I already knew that). Plus, they always seemed to go really deep into encryption, but I already knew my teams would never be writing their own encryption, so I didn’t get why they felt the need to always cover it…
I found the entire situation infuriating. I felt like I couldn't win.
If you are asking your boss to take training it must fall into one of two categories:
1. It will help you do your current/new job better
2. It will help you grow within the organization so you can get a promotion someday (also known as career development)
Note: If you are a Rudy developer and you asked your boss to pay for you to take a basket weaving course, this blog article is not going to help you. That said, if you are a Ruby developer and you asked your boss to pay for a secure-coding-in-ruby course or an application security course, then this article can help.
Remember I said in the first article that you needed to read the syllabus and keep track of what’s on the course and how it relates to your job? Now is time to get that info so we can write your justification letter. Just like in the previous article, I am going to use the Application Security Foundations Program from We Hack Purple as the example, but you should be able to use whichever training you have choose.
Right now, our InfoSec team keeps bringing in a PenTester to test our apps right before we release them. They always find 100 things wrong, because none of our devs know anything about security. We always end up with late projects and everyone freaking out, because it’s so last minute. Lots of overtime and stress. I could help teach them what they need to know, so we could create more-secure software. The PenTesters will find a lot less wrong with our apps.
Also, the program comes with a copy of Alice and Bob Learn Application Security. I know the one copy we have is currently constantly being used by my team for reference, so having a second copy would be really great.
If I took this course, I would know how we could do this better. The dev team could do some testing ourselves (carefully), and other stuff to make sure our apps are in better shape by the time the PenTester comes. The program has a secure coding guideline we could adopt, and even an API best practices guide. We currently have no idea how to secure our APIs, and we keep reading on the internet and we’re lost. This course would help me understand so much! And then I could be the ‘security champion’ on our dev team, the one everyone can turn to when they need help. I know you feel this is outside my job description, but someone has to do it. I want that someone to be me.
Up next we will cover Objection 3: There’s no time with your current workload for you to take training.
We’ve all been there. There’s a training you really want to take, but your boss isn’t so sure. This can be because it’s out of budget, they feel it’s too ‘off topic’ from your current job, there’s no time with your current workload, they are afraid they will lose you if you have new skills, or some other reason they won’t tell you. Let’s go through all of these reasons and figure out how YOU can get your training approved.
Note: I run my own training company, We Hack Purple, that specializes in Application Security, Secure Coding and DevSecOps training. While I am definitely hoping this article helps our customers, I’m also hoping it helps everyone else who needs training! For our examples we will use the Application Security Foundations Program from We Hack Purple, and we will try to justify taking it to your boss.
The first thing you need to do is make sure you are selecting the *best* training for your specific job or career development. Don’t take the popular one, or ‘the cool one’ that people are talking about on Twitter. Evaluate very carefully which one will help you level-up in your career and your current job.
Next, read about the content of the training you are taking. Make notes of what’s in there and keep the syllabus handy, as you will likely need to reference it as you write your justification. You also want to have some other links to other courses to compare it to; both to explain why the one you have selected is better and why it’s (hopefully) more cost-effective.
Let’s start creating our defenses for your boss’s potential objections.
Objection 1: We don’t have the budget/it’s too expensive.
This is the one that I personally have received the most often in my career. I have actually had a boss laugh in my face when I suggested one single course (because it would have cost my combined training budget for 5 years). I explained that cyber security courses are quite costly, and all of my bosses continued to reject my requests. I ended up selecting training from several different places that was cheaper, but nowhere near as good as what I had asked for. At the time, I didn’t know how to get around this hurdle.
With a little more industry experience and a chance to see a lot more training, I realized that I needed to explain the value of what we were getting was greater than what we were spending. Let me explain using the We Hack PurpleApplication Security Foundations Program as the example (but this should work with whatever you have chosen, if you have chosen the best training for your situation).
I want to take the Application Security Foundations Program from We Hack Purple for my training this year. I know you feel it’s too expensive and that we might not have the budget, but let me explain how I think it will save us more money than it costs.
We keep hiring consultants to help us with our AppSec Program, and that is very expensive. And we haven’t been getting the results we want, they show up and write one policy or one guideline, then leave. This program will provide some starter policies, standards and guidelines, so we don’t need to pay that consultant anymore. After taking the training I will know what to do and have tools to start with, so I can hit the ground running.
We also keep changing our strategy, because we haven’t been getting the results we want, and the dev teams don’t seem to be ‘on board’ with what we have been doing. This program will not only help me build and plan an entire AppSec program throughout the three courses, in Level 2 of the program there’s an entire module to teach me how to support our culture change (advocacy), how to build a security champions program, AND how to make presentations that aren’t the death-by-PowerPoint that we are used to giving. They even show us how to measure the effectiveness of our program, so we know if the strategy we are using is actually *working*, so we can know when we need to change or stay the course. Right now, we are just guessing at what to do to make sure our software is secure, but with this program, I would know.
I realize that $999 USD is a lot, and we are a small company. But this is the only training I could find like this on the internet, one that will teach me how to build and launch an AppSec program. That’s what the company needs me to do. Please approve this training so I can get started.
Up next we will talk about training that is ‘too off topic'.
The Second Way of DevOps is fast feedback. In security, when we see this we should all be thinking the same thing: Pushing Left. We want to start security at the beginning of the system development life cycle (SDLC) and ensure we are there (providing feedback, support and solutions) the whole way through!
Fast feedback loops means getting important information to the right people, quickly and regularly. One of the main reasons that Waterfall projects failed in the past was the lack of timely feedback; no one wants to find out twelve months after they made a mistake, when it's too late to fix it.
The goal of security activities in a DevOps environment must be to shorten and amplify feedback loops so security flaws (design issues) and bugs (code issues) are fixed as early as possible, when it’s faster, cheaper and easier to do a better job. These DevOps people are really onto something!
Let’s go over several ideas of how to achieve this.
Activities to create fast feedback loops.
Automate as much as humanly possible. Inside or outside the pipeline, automation is key.
Whenever possible integrate your tools with the Dev and Ops team’s tools. For instance, have the issues found by your IAST tool turned into tickets in the developer’s bug tracking system, automagically.
When you have a Pentest done, check all your other apps for the things found in the report, then add create unit tests to look for these things and prevent them from coming back.
Rename insecure functions or libraries as “insecure” with a wrapper, so programmers see immediately that there is an issue.
Add security sprints to your project schedule (to fix all security bugs in backlog).
Asking the Dev and Ops what they are concerned about (in relation to security), so you can fix any problems the security team might be causing them.
Add important security tests that are quick and accurate to the pipeline. For instance, scan for secrets in the code that is being checked in. That is an important test!
If an important security tests fail in the pipeline, the continuous integration server must break the build. Just like quality tests. This is loud feedback.
Create a second pipeline that doesn’t release any code, but runs all the long and slow security tests, then have the security team review the results after and turn the important things into tickets for the Devs.
Tune all security tools as much as possible and validate all results so that the feedback you are giving is *accurate*. There is no point of sending lots of feedback if half of it is wrong.
Work with developers to create negative unit tests (sometimes known as abuse tests). Create copies of regular unit tests, rename them with “Abuse” at the end, then add malicious payloads and ensure that your app fails gracefully and handles bad input well.
Have reports from your security tools automatically send their results to a vulnerability management tool such as Defect Dojo or Thread Fix to keep metrics and use them to improve all of your work. You need feedback too.
Be creative. Any way that you can get feedback faster to other teams is a huge win for your team too!
The first “Way” of DevOps is emphasizing the efficiency of the entire system. Many of us tend to focus only on our part of a giant system, and get bogged down improving only our own contributions to the larger process. It’s rare that we stand back, look at the entire thing, and realize that if we helped another team or changed something small within our part, that it could improve other areas for the better. The first way of DevOps is about looking at the entire system, and making sure the entire thing is as efficient as possible. #speed
When we worked in Waterfall development environments security often acted as a gate. You had to jump through their hoops, then you were let through, and you could push your code to prod. Awesome, right? Not really. It was slow. Security activities took FOREVER. And things got missed. It was rigid and unpleasant and didn’t result in reliably secure software.
It may seem obvious to new developers that security should not slow down the SDLC, but I assure you, this concept is very, very new. When I was a software developer I referred to the security team as “Those who say no”, and I found almost all of my interactions with them left me frustrated and without helpful answers.
When we (security practitioners) think about The First Way, we must figure out how to get our work done, without slowing down all the other teams. They won’t wait for us, and we can’t set up gates. We have to learn to work the way they do; FAST.
Below I will offer suggestions for how we can work together with the dev and ops teams to ensure we get our mandate done, within the DevOps workflows and processes.
First of all, we need to use modern tooling that is made for DevOps pipelines if we are going to put anything into the CI/CD pipeline. Never take an old tool and toss it in there; no DevOps team is going to wait 5 hours for your SAST tool to run. Tune your tools and ensure you select tools that are made for pipelines if that is how you are going to use them. Whenever possible, only run your tools on the ‘delta’ (the code changed in that release, not the entire code base).
When selecting tools, remember that not every tool needs to be put in the pipeline. In fact, having tools that are out-of-band, but located on the ‘left’ of the SDLC, can offer even more value and save time. Examples:
Package management tools that only serve packages that are not known to be insecure (pre-approved by a security research team)
Adding security tests to your unit tests, which are often run before the code arrives in the pipeline (for instance, write input validation tests that ensures your code properly handles input taken from the XSS Filter Evasion Cheat Sheet)
Ddding security tooling to the check-in process, such as secret scans (don’t even let them check it in if it looks like there’s a secret in the code)
Scanning your code repository for known-insecure components. It’s just sitting there, why not use it?
This also means that security bugs should be placed in the same bug tracker or ticketing system that the developers and ops teams are using. They shouldn’t check two systems, that is not efficient.
If at all possible, we should be providing and/or approving tools that assist in finding vulnerabilities in written code (both the code your team wrote, and the code from dependencies) and running code. This could be SAST + SCA + DAST, or it could be SCA + IAST (run during unit testing, QA and in prod). It could also mean manual secure code review plus a PenTest the week before going live (this is the least-efficient of the three options presented here).
Templates and Code Reuse:
If it makes sense, create templates and provide secure code samples, there’s no need to reinvent the wheel. Also, enable the developers and ops teams to scan their own code by providing tools for them (and training on how to use them safely and effectively).
Think Outside The Box
We (security) can no longer be a bottleneck, we must work to enable them to get their jobs done securely, in anyway we can. Examine your processes to ensure they are efficient; create a second asynchronous (which does not release to prod) pipeline to automate your longer tests; write your own tools if you absolutely have to. The sky is the limit.
DevSecOps is the security activities that application security professionals perform, in order to ensure the systems created by DevOps practices are secure. It’s the same thing we (AppSec professionals) have always done, with a new twist. Thanks Imran!
Refresher on The Three Ways:
Emphasize the efficiency of the entire system, not just your part.
Fast feedback loops.
Continuous learning, risk taking and experimentation (failing fast). Taking time to improve your daily work.
Let’s dig in, shall we?
1. Emphasize the efficiency of the entire system, not just one part.
This means that Security CANNOT slow down or stop the entire pipeline (break the build/block a release), unless it’s a true emergency. This means Security learning to sprint, just like Ops and Dev are doing. It means focusing on improving ALL value streams, and sharing how securing the final product offers value to all the other steams. It means fitting security activities into the Dev and Ops processes, and making sure we are fast.
2. Fast feedback loops.
Fast feedback loops = “Pushing Left” (in application security)
Pushing or shifting “left” means starting security earlier in the System Development Life Cycle (SDLC). We want security activities to happen sooner in order to provide feedback earlier, which means this goal is 100% inline with that we want. The goal of security activities must be to shorten and amplify feedback loops so security flaws (design/architecture issues) and bugs (code/implementation issues) are fixed as early as possible, when it’s faster, cheaper and easier to do a better job.
3. Continuous learning, risk taking and experimentation
For most security teams this means serious culture change; my favorite thing! InfoSec really needs some culture change if we are going to do DevOps well. In fact, all of IT does (including Dev and Ops) if we want to make security everybody’s job.
Part of The Third Way:
Allocating time for the improvement of daily work
Creating rituals that reward the team for taking risks: celebrate successes
Introducing faults into the system to increase resilience: red team exercises
We are going to delve deep into each of the three ways over the next several articles, exploring several ways that we can weave security through the DevOps processes to ensure we are creating more secure software, without breaking the flow.
If you are itching for more, but can’t wait until the next post, watch this video by Tanya Janca. She will explain this and much more in her talk ‘Security Learns To Sprint'.
There are many definitions of DevOps, too many, some might say. Some people say it’s “People, Processes, and Products”, and that sounds great, but I don’t know what I’m supposed to do with that. When I did waterfall I also had people, processes, and products, and that was not great. I thought DevOps was supposed to be a huge improvement?
I’ve heard other people say that it’s paying one person to do two jobs (Dev and Ops), which can’t be right… Can it? I’ve also been told once by a CEO that their product was “made out of DevOps”, as though it was a substance. I decided not to work there, but that’s another story. Let’s look at some better sources.
DevOps is a set of practices that combines software development and information-technology operations which aims to shorten the systems development life cycle and provide continuous delivery with high software quality.
But what are the practices? Why are we aiming to shorten the SDLC? Are we making smaller software? What is ‘continuous delivery’?
To get to the bottom of it I decided to read The DevOps Handbook.
Then I knew what DevOps was, and I knew how to do it. And I
discovered that I LOVED DevOps.
According to the DevOps Handbook, DevOps has three goals.
Improved deployment frequency; Shortened lead time between fixes;
Awesome! This means if a security bug is found it can be fixed extremely quickly. I like this.
Lower failure rate of new releases and faster recovery time;
Meaning better availability, which is a key security concern with any application (CIA). Lower failures mean things are available more often (the ‘A’ in CIA), and that’s definitely in the security wheelhouse. So far, so good.
Faster time to market; meaning the business gets what they want.
Sometimes we forget that the entire purpose of every security team is to enable the business to get the job done securely. And if we are doing DevSecOps, getting them products that are more secure, faster, is a win for everyone. Again, a big checkmark for security.
Great! Now I think the DevOps people want the same things that I, as a security person, want. Excellent! Now: How do I *DO* DevOps?
That is where The Three Ways of DevOps comes in.
Emphasize the efficiency of the entire system, not just one part.
Fast feedback loops.
Continuous learning, risk-taking and experimentation (failing fast)
In the next post we will talk more in detail about The 3 Ways (and how security fits in perfectly).
Application Security is every action you take towards ensuring the software that you (or someone else) create is secure.
This can mean a formal secure code review, hiring someone to come in and perform a penetration test, or updating your framework because you heard it has a serious security flaw. It doesn’t need to be extremely formal, it just needs to have the goal of ensuring your systems are more secure.
Now that we know AppSec is, why is it important?
For starters, insecure software is (unfortunately), the #1 cause of data breaches (according to the Verizon Breach Reports, 2016, 2017, 2018 and 2019). This is not a list that anyone wants to be #1 on. According to the reports, insecure software causes 30–40% of breaches, year after year, yet 30–40% of the security budget at most organizations is certainly not being spent on AppSec. This is one part of the problem.
If the problem is that insecure software causes breaches, and one of the causes is that security budgets don’t appear to prioritize software, what are some of the other root causes of this issue?
For starters, universities, colleges, and programming bootcamps are not teaching the students how to ensure that they are creating secure software. Imagine electricians whet to trade school, but they didn’t teach them safety? They twist two cables together and then just push them into the wall, unaware that they need two more layers of safety (electrical tape, and then a marrett). This is what we are doing with our software developers, we teach them from their very first lesson how to make insecure code.
Hello (insecure) World
Lesson #1 for every bootcamp or programming course: Hello World.
Step 1) Output “Hello World” to screen
Step 2) Output “What is your name?” to screen
Step 3) Read the user’s input into a variable (note: we skip teaching input validation)
Step 4) Output the user’s input to the screen with a hello message (note: we skip output encoding)
The above lesson teaches new programmers the best possible recipe for including reflected Cross Site Scripting (XSS) in their application. As far as I know there is not a follow up lesson provided on how ensure the code is secure.
“Hello World” is the most-taught lesson for starting a new programming language, I’m sure you’ve seen it everywhere. More like “Hello Insecure World”.
Although there has been some headway in universities and colleges recently, most of them barely scratch the surface in regards to application security.
So that's reason #2. Let's move onto reason #3.
Another issue that contributes to this problem is that security training for developers is costly. While this is true for all types of professional training, security training is significantly more expensive then other forms of technical training. A single course at the SANS institute (a well-respected training establishment that specializes in all things cyber), could cost an attendee as much as $5000-$7000 USD, for one week of training. There are other less-pricy options, such as taking a course when attending a conference, which usually range from $2000-$5000, however, those are of varying quality, and there is no standardized curriculum, making them a bit of a gamble. I’ve taken several trainings when attending various conferences over the years, and I’d say about 1/2 were good.
There are much cheaper alternatives to the options above (such as the courses from We Hack Purple and other training providers), and they are of very varying quality levels. I’ve seen both good free courses and some where I wish I could have my time back they were so bad. Most of them do not provide a curriculum to follow either, meaning it is often unclear to the student which other courses they should take in order to get the specific job they want. It is very easy to waste quite a bit of time and money; I know, that is how I started my AppSec career… Although I was quite lucky to have two professional mentors guiding me, which made it a lot easier. Not everyone has a mentor.
Another cause (#4) of insecure software is that the security team is usually grossly outnumbered. According to several sources there is usually 100 software developers for every 10 operations employees for every single (1) security professional.
Let me repeat that. There are 100/10/1, Dev/Ops/Sec. With numbers like that you can’t work harder, you have to work smarter. Which is where we are going with this series.
Now we know the problem and several of the causes, what can we do about it? The short answer is DevSecOps, and the long answer is ‘read the rest of the blog series’.
For now though, let’s define DevSecOps, before we dive into what DevOps is, The Three Ways, and so much more, in the next article.
DevSecOps: performing AppSec, adjusted for a DevOps Environment. The same goals, but with different tactics and strategies to get us there. Changing the way we do things, so that we weave ourselves into the DevOps culture and processes.