Cyber Defender Strategies: What Your Vulnerability Assessment Practices Reveal
I. EXECUTIVE SUMMARY
In this report we analyze real-world end-user vulnerability assessment (VA) behavior using a machine learning (ML)
algorithm to identify four distinct strategies, or “styles.” These are based on five VA key performance indicators (KPIs)
which correlate to VA maturity characteristics.
This study specifically focuses on key performance indicators associated with the Discover and Assess stages of the
five-phase Cyber Exposure Lifecycle. During the first phase – Discover – assets are identified and mapped for visibility
across any computing environment. The second phase – Assess – involves understanding the state of all assets,
including vulnerabilities, misconfigurations, and other health indicators. While these are only two phases of a longer
process, together they decisively determine the scope and pace of subsequent phases, such as prioritization and
remediation.
The actual behavior of each individual enterprise in the data set, in reality, exhibits a mixture of all VA Styles. For the
purposes of this work, enterprises are assigned to the specific style group with which they most closely align. We
provide the global distribution of VA Styles, as well as a distribution across major industry verticals.
FINDINGS
• Enterprises conducting VA fall into four distinct VA Styles, ordered by maturity: Diligent, Investigative,
Surveying and Minimalist.
° The Diligent style represents the highest maturity, yet constitutes only five percent of all
enterprises in the data set.
° The Investigative style represents a medium to high maturity, with 43 percent of enterprises
following this style.
° The Surveying style, with a representation of 19 percent in the data set, corresponds to a low to
medium maturity.
° The Minimalist style represents the lowest maturity and constitutes 33 percent of all enterprises
in the data set.
• The hospitality, transportation, telecommunications, electronics and banking industries had the
highest proportion of the mature Diligent style.
• The utilities, healthcare, education and entertainment industries had the highest proportion of the
low-maturity Minimalist style.
• The utilities industry had the highest proportion of the low-maturity Minimalist style overall.
• The distribution of VA styles by geographical region shows no noteworthy variation.
Cyber Defender Strategies: What Your Vulnerability Assessment Practices Reveal
3
II. INTRODUCTION
RECOMMENDATIONS
The cybersecurity community is heavily focused on what attackers
are doing. While threat intelligence and vulnerability research is
invaluable, it only represents one side of the equation. Far less
research has been dedicated to how defenders are responding.
•
There is a wealth of qualitative data available on what end users are
doing, primarily derived from surveys. The reliability of survey data
is dependent on the knowledge and honesty of participants. Results
can be skewed by cognitive biases and lack of awareness. What
someone believes they are doing is not always the same as what they
are actually doing, especially when practical realities come into play.
Quantitative research based on end-user behavior and telemetry
data provides a more reliable basis for determining the true state of
general VA maturity.
In our last report, “Quantifying the Attacker’s First-Mover Advantage,”
we discovered attackers generally have a median seven-day window
of opportunity during which they have a functional exploit available
to them, before defenders have even determined they are vulnerable.
The resulting seven-day gap is directly related to how enterprises are
conducting VA.
In this study, we analyze real-world VA telemetry data to group end
users into segments and identify four distinct strategies, or “styles,”
of VA. Further analysis focuses on the distribution of these four VA
Styles across industries.
To classify the VA Styles, we applied a machine learning algorithm
called archetypal analysis (AA) to real-world scan telemetry data
from more than 2,100 individual organizations in 66 countries and just
over 300,000 scans during a three-month period from March to May
2018. AA identifies a number of idealized/archetypal VA behaviors
within this data set. Organizations are assigned to groups defined
by the archetype they are most similar to. This does not mean each
organization in a group behaves exactly like the archetype. Rather,
it means that, of the four archetypes, they are most similar to the
archetype which defines that grouping. The scanning behavior styles
described in this report are based on these four archetypes.
Cyber Defender Strategies: What Your Vulnerability Assessment Practices Reveal
Evaluate your own
vulnerability assessment
maturity based on our
five critical VA KPIs:
Scan Frequency, Scan
Intensity, Authentication
Coverage, Asset Coverage
and Vulnerability
Coverage.
• Identify your current
VA Style and compare
yourself to industry
peers.
• Follow the
recommendations for
your style to determine
the KPIs you need
to improve to move
your maturity to the
next level.
4
Please complete the form to gain access to this content
