EPSS and CISA KEV¶
Overview
In this section, we apply interpretation of, and guidance on, EPSS to CISA KEV:
- CISA KEV will be used as the reference (source of truth) for active exploitation.
- CISA KEV is the most known and used catalog of CVEs actively exploited in the wild, and is publicly available.
We start with analysis via plots and interpretation and code
- Analyze the Data Sources relative to CISA KEV
- Analyze EPSS relative to CISA KEV (and these Data Sources)
Then Jay Jacobs (EPSS) presents analysis and interpretation based on data internal to the EPSS model.
User Story
As a user, I want to know what CVEs
- are actively exploited or likely to be
- are not actively exploited or not likely to be
So I can focus on what to remediate first in my environment.
CISA KEV contains a subset of known exploited CVEs
- All CVEs in CISA KEV are actively exploited (see criteria for inclusion in CISA KEV)
- Only ~~5% of CVEs are known exploited
- CISA KEV contains a subset of known exploited CVEs
What does EPSS look like for CISA KEV?¶
TODO: redo colors for other
What does EPSS look like for CISA KEV CyberSecurity Advisories (CSA)?¶
CISA Cybersecurity Advisories (CSA) represent the Top Routinely Exploited Vulnerabilities from the CISA KEV Catalog
CISA (Cybersecurity and Infrastructure Security Agency) co authors (with several international cybersecurity agencies) separate Cybersecurity Advisories (CSA) on the Top Routinely Exploited Vulnerabilities from the CISA KEV Catalog e.g.
- AA23-215A Joint CSA 2022 Top Routinely Exploited Vulnerabilities August 2023
- AA21-209A Joint CSA Top Routinely Exploited Vulnerabilities July 2021
- AA22-279A 2022 covering CVEs from 2022, 2021
- AA22-117A 2022 covering CVEs from 2021
- AA20-133A 2020 covering CVEs from 2016 to 2019
All CISA KEV CyberSecurity Advisories (CSA) Top Routinely Exploited Vulnerabilities¶
Observations
- Approximately half of CISA KEV CVEs have an EPSS score near 0.
- For CyberSecurity Advisories (CSA) Top Routinely Exploited Vulnerabilities CVEs:
- Most, but not all, of the CVEs have EPSS scores near 1.0, but some have an EPSS score near 0.
- 13 CVEs of 98 CVEs have EPSS scores <= 0.1
- Most of the CVEs are weaponized i.e. in metasploit and/or nuclei.
EPSS Observed Exploitation Activity Compared to CISA KEV¶
EPSS probabilities are the direct result of the automated data we
receive and collect about a vulnerability. KEV is based on (to the
best of our knowledge) a list of human curated CVEs with historical
exploitation activity. And so given the different data generating
processes, it is entirely reasonable that a given CVE would appear on
KEV, while also receiving a low EPSS score. Or alternatively, that a CVE
would have a high EPSS score, but not appear on KEV.
The EPSS probability is the result of a predictive model trained on
previous daily exploitation activity. A higher score indicates the
vulnerability is presenting itself (in data) similar to previously and
routinely exploited vulnerabilities and conversely a lower score
indicates the vulnerability is presenting itself like the
vulnerabilities where we, or our data partners, observed less or no
exploitation activity. We can see this in the plots below:
In order to understand what we said above, "A higher score indicates the vulnerability is presenting itself (in data) similar to previously and routinely exploited vulnerabilities and conversely a lower score indicates the vulnerability is presenting itself like the vulnerabilities where we, or our data partners, observed less or no exploitation activity." I filtered CVEs down to those just on the KEV and split them into 5 categories based on EPSS score (shown here with the counts in each category):
epss_range count
1 [0.00042,0.00328] 206
2 (0.00328,0.0279] 203
3 (0.0279,0.896] 205
4 (0.896,0.972] 204
5 (0.972,0.976] 205
We can then look at the proportion of CVEs in each category with specific attributes EPSS derived from data about those vulnerabilities:
Comparison of Prioritizing Remediation of CISA KEV CVEs using EPSS and CVSS¶
Now let's assume that KEV is perfect. Only the CVEs on the KEV List will be exploited this month and those are the only ones you want to prioritize. How would EPSS and CVSS compare in that case? If you start at the top of each scoring system and remediate everything going down the scoring range, how many do you have to prioritize before you close all or even "most" of the CVEs?
Left to right represents the amount of effort (percent of all published CVEs) that is prioritized as you move down the scoring system.
Bottom to top represents the amount of "coverage" for the KEV list.
As we put in more effort (prioritize more CVEs) we will cover more and more of the KEV CVEs, so as the lines move from left to right, we (the defenders) are prioritizing more and more CVEs and getting better coverage of CVEs on the KEV. The dashed lines represent 50% effort (the vertical dashed line) and 50% KEV coverage (horizontal dashed line).
We can create the same type of plot as the last one with some other variables to see what kind of separation EPSS offers over a straight CVSS base score.
Observations
- Closest to the top left is best i.e. the CVEs on the List/Tool are prioritized more than all published CVEs.
- e.g Using EPSS, Metasploit CVEs (weaponized exploits) are prioritized much better than all published CVEs (much better than using CVSS).
Takeaways
- EPSS does better at prioritizing exploited CVEs (those on CISA KEV) than CVSS, which is what we'd expect as the CVSS Base Score is not a good predictor of exploitation per the Understanding Risk chapter.
- Some CVEs on CISA KEV have a low EPSS score.
- Some CVEs with a high EPSS score are not on CISA KEV.
Quote
If there is evidence that a vulnerability is being exploited, then that information should supersede anything EPSS has to say, because again, EPSS is pre-threat intel. If there is an absence of exploitation evidence, then EPSS can be used to estimate the probability it will be exploited." https://www.first.org/epss/faq