Independent testing of EDR solutions

The best way to prove the effectiveness of a security solution is to test it in conditions that are as real-world as possible, using typical tactics and techniques of targeted attacks. Kaspersky regularly participates in such tests and sits pretty at the top of the ratings.

The results of a recent test — Enterprise Advanced Security (EDR): Enterprise 2022 Q2 – DETECTION — were revealed in an SE Labs‘ report. The British company has been putting the security solutions of major vendors through their paces for several years now. In this latest test, our business product Kaspersky Endpoint Detection and Response Expert achieved an absolute 100% score in targeted attack detection and was awarded the highest possible rating – AAA.

This is not SE Labs’ first analysis of our products for protecting corporate infrastructure against sophisticated threats. The company previously ran its Breach Response Test (which we took part in in 2019). In 2021, our product was tested in their Advanced Security Test (EDR). Since then, the testing methodology has been tweaked, and the test itself has been divided into two parts: Detection and Protection. This time, SE Labs studied how effective security solutions are at detecting malicious activity. Besides Kaspersky EDR Expert, four other products took part in the test: Broadcom Symantec, CrowdStrike, BlackBerry, and another, anonymous, solution.

Grading system

The testing was made up of several checks, but to get a feel for the results, it will suffice to look at the Total Accuracy Ratings. This basically shows how well each solution detected attacks at different stages, and whether it pestered the user with false positives. For even greater visual clarity, the participating solutions were assigned an award: from AAA (for products with a high Total Accuracy Rating) to D (for the least effective solutions). As mentioned, our solution got a 100% result and an AAA rating.

The Total Accuracy Ratings consist of scores in two categories:

• Detection Accuracy: this takes into account the success of detecting each significant stage of an attack.
• Legitimate Software Rating: the fewer the false positives generated by the product, the higher the score.

There’s one other key indicator: Attacks Detected. This is the percentage of attacks detected by the solution during at least one of the stages, giving the infosec team a chance to respond to the incident.

How we were tested

Ideally, testing should reveal how the solution would behave during a real attack. With that in mind, SE Labs tried to make the test environment as life-like as possible. First, it wasn’t the developers who configured the security solutions for the test, but SE Labs’ own testers, who received instructions from the vendor – as clients’ infosec teams usually do. Second, the tests were carried out across the entire attack chain – from first contact to data theft or some other outcome. Third, the tests were based on the attack methods of four real and active APT groups:

• Wizard Spider, which targets corporations, banks and even hospitals. Among its tools is the banking Trojan Trickbot.
• Sandworm, which primarily targets government agencies and is infamous for its NotPetya malware, which masqueraded as ransomware, but in fact destroyed victims’ data beyond recovery.
• Lazarus, which became widely known after the large-scale attack on Sony Pictures in November 2014. Having previously focused on the banking sector, the group has recently set its sights on crypto-exchanges.
• Operation Wocao, which targets government agencies, service providers, energy and tech companies, and the healthcare sector.

Threat detection tests

In the Detection Accuracy test, SE Labs studied how effectively security solutions detect threats. This involved carrying out 17 complex attacks based on four real-world attacks by Wizard Spider, Sandworm, Lazarus Group, and Operation Wocao actors, in which four significant stages were highlighted, each of which consisted of one or more interconnected steps:

• Delivery/Execution
• Action
• Privilege Escalation/Action
• Lateral Movement/Action

The test logic does not require the solution to detect all events at any particular stage of the attack; it is enough to identify at least one of them. For example, if the product failed to notice how the payload got onto the device, but detected an attempt to run it, it successfully passed the first stage.

Delivery/Execution. This stage tested the solution’s capacity to detect an attack in its infancy: at the time of delivery — for example, of a phishing e-mail or malicious link — and execution of the dangerous code. In real conditions, the attack is usually stopped there, since the security solution simply doesn’t allow the malware to go any further. But for the purposes of the test, the attack chain was continued to see how the solution would cope with the next stages.

Action. Here, the researchers studied the solution’s behavior when attackers have already gained access to the endpoint. It was required to detect an illegitimate action by the software.

Privilege Escalation/Action. In a successful attack, the intruder attempts to gain more privileges in the system and cause even more damage. If the security solution monitors such events or the privilege escalation process itself, it’s awarded extra points.

Lateral Movement/Action. Having penetrated the endpoint, the attacker can try to infect other devices on the corporate network. This is known as lateral movement. The testers checked whether the security solutions detected attempts at such movement or any actions made possible as a consequence of it.

Kaspersky EDR Expert scored 100% in this segment; that is, not a single stage of any attack went unnoticed.

Legitimate Software Ratings

Good protection has to not only reliably repel threats, but also not prevent the user from using safe services. For this, the researchers introduced a separate score: the higher it was, the less often the solution mistakenly flagged legitimate websites or programs – especially popular ones – as dangerous.

Once again, Kaspersky EDR Expert got 100%.

Test results

Based on all the test results, Kaspersky Endpoint Detection and Response Expert was awarded the highest available rating: AAA. Three other products earned the same rating: Broadcom Symantec Endpoint Security and Cloud Workload Protection, CrowdStrike Falcon, and the anonymous solution. However, only we and Broadcom Symantec achieved a 100% score in the Total Accuracy Ratings.