Four types of security testing and when to use them

When it’s time to ‘security test’ your infrastructure, what should you do? Security testing can mean all kinds of things, and it’s not always obvious what’s the right choice, and when. Here I’ll summarize what I think are the four main types of security testing, when to use each and pitfalls to watch.

1.  Vulnerability scanning

What is vulnerability scanning?

Vulnerability scanning means running automated software that looks for common vulnerabilities in your systems, like a web server that hasn’t been patched or misconfigured cloud storage, exposing customer data. Ideally, you give the software a list of targets, set it in motion and wait to get a report listing vulnerabilities and remediation advice. It’s not vulnerability scanning that improves your security, but acting on the results.

Depending on the scanning software, it may just check your software against lists of known issues, or do something more complex, like a brute force attack: Guessing user credentials and passwords to see if they’re secure.

When should you use it?

Use vulnerability scanning on everything you have that faces the internet (endpoints), like corporate web servers, virtual private network (VPN) endpoints and office internet connections. You can also run vulnerability scans sporadically or scheduled on internal network systems, or as part of your software development lifecycle.

When shouldn’t you use it?

Vulnerability scanning isn’t helpful when you want to know how a human attacker would see your infrastructure.

What are the advantages and disadvantages?

You can run vulnerability scanning yourself, which puts you in control. Or you can have a third party run it for you.

While vulnerability scanning is often seen as the “poor cousin” of security testing, I don’t agree. It quickly highlights problems you might have missed, like a temporary internet-facing website the development team forgot to take down or the internal user account with an easy password. It helps you tidy up low-level issues at little cost.

Companies are most often exploited not by advanced attacks, but by the low-hanging fruit.

Chris Wallis, Founder, vulnerability-scanning company Intruder

2.  Penetration testing

What is penetration testing?

Penetration testing, or pen testing, can mean different things to different people. I define it as combining automated and manual techniques to look for weaknesses in the target’s security posture. A penetration testing team emulates the methods of genuine attackers. How close it is to a real attack depends on the team. They may use the kinds of tools attackers use, or just their techniques, like trying SQL injection on a web interface, when another method might work better.

With help from the system administrators, developers and project team, a penetration testing team can do a more cost-effective and useful audit. They shouldn’t be wasting time evading intrusion detection systems or trying not to get noticed by your Security Operations Center. If that’s the kind of security test you want, see the section below on red team exercises.

When should you use it?

A pen test should have a tight scope around a new installation, project or area of concern. It must concentrate on the most likely source of security issues for your organization.

When shouldn’t you use it?

Don’t use a penetration test to see how your security teams and Security Operations Center would react to a real cyberattack. A penetration test is overt: The system administrators should know it’s happening. See it as an “aggressive technical audit” rather than emulating how attackers think and work.

What are the advantages and disadvantages?

You’ll get an attacker’s mindset to look at your project, program or installation. It’s useful to have someone on your side who sees what you’ve built as a set of weaknesses and targets. And it’s easy to schedule and assign budget for, compared with red team exercises.

Results may not be consistent between one pen test and the next. This isn’t necessarily a bad thing. If you repeat it or have several pen testers, you may find more issues.

Pen testing is poorly defined. There will be penetration testers reading this who define the term differently to me. Do some work before you pen test to make sure there’s a shared understanding.

3.  Red team testing

What is red team testing?

Red teaming, also sometimes called ethical hacking, is a simulation to test how well your people and technology would respond to an adversary’s attack. It’s hard to find the line between penetration testing and red teaming. I define a red team exercise as an engagement with much wider scope than a penetration test. The scope of a red team exercise could be your whole organization.

It’s up to the red team how they attack. Agree strict rules of engagement in advance. Red teamers will stay within the law, but you should also address ethics, staff relations and cultural norms before you start.

When should you use it?

Use a red team exercise when you want to see, as closely as possible, how real attackers would act against you.

It’s more of an adversarial simulation than pen testing. A penetration tester finds as many weaknesses as possible in the time available to help defenders see issues. A red teamer will only find and exploit any vulnerability they need to, to achieve a goal. If they can get into your internal network through a poorly configured VPN but could’ve achieved the same through poor wireless security at your remote offices, you may not discover both issues.

“Purple teaming” can make a red team exercise more useful. It’s when you have an attacking red team and a defending “blue” team.

As a security tester, you’re a sparring partner: You’re not there to win, but to make your opponent better.

When shouldn’t you use it?

Only hold a red team exercise when you’ve done vulnerability scans and penetration tests, and fixed problems found. Otherwise, the attackers’ level of expertise will probably overwhelm the organization’s defenses, giving no useful insight.

What are the advantages and disadvantages?

A significant part of a red team exercise is testing defenders’ detection abilities. Attackers must be covert, careful and evasive, like a genuine attacker. There must also be clear paths for contact and escalation. You don’t want the Security Operations Center concentrating on the red teamers, but missing a genuine compromise.

If you need to show stakeholders the breadth of issues to get resources to fix them, a red team exercise tells a compelling story. It will build a picture of how real attackers could work their way through your infrastructure.

A red team exercise is not the same as adversarial analysis: Examining a company’s way of working or project plan for issues without attack. For a deeper look at adversarial analysis, see the work of Mark Mateski of Red team journal or Bryce Hoffman of Red team thinking.

A downside of a red team exercise is cost. The people involved are specialists with skills and tools that need constant maintenance. They use methodology that encourages slow, careful work. All this affects the day rate.

4.  Bug bounty programs

What are bug bounty programs?

Bug bounty programs see companies offering a reward to those who report specific vulnerabilities in parts of its infrastructure within a given scope. The programs may be by invitation only or open to anyone.

When should you use it?

I would only run a bug bounty program after running and responding to the results of at least two types of tests already mentioned. Get an expert to determine the scope and rewards, and how to run it.

When shouldn’t you use it?

A bug bounty program might distract staff and give you more reports than you can use if your organization is still working out how to deal with penetration tests and red team exercises.

What are the advantages and disadvantages?

All other testing methods are paid on effort rather than results.

Your organization may be mature enough that other options aren’t what you need. Bug hunters follow some of the same steps as an attacker, so their judgment on what’s worth attacking and how may be close to a real attack.

Running a bug bounty program makes it clear to anyone who discovers a security issue with your organization that you’re approachable on security issues.

As bug bounty hunters rarely share details of successful attacks until they’ve got their bounty, your company must do the work of filtering out duplicate or irrelevant reports.

Now when you’re asked about security testing and what your organization should be doing, you can start the conversation informed. Stay aware of the flexible and overlapping definitions out there – always ask what someone means by a term they’re using before you devote budget or schedule testing.