certcc / ssvc Goto Github PK

Version 2 will need different context, different signposting, and in general a different introduction than v1.
Some items to address include:

• v1 and v1.1 have different kinds of intros and context. v1 has more decision tree basics. v1.1 has more background on operational and academic related work. Decide which of these to keep; maybe merge and keep both?
• Add the following to related work: Wang, B., Li, X., de Aguiar, L. P., Menasche, D. S., & Shafiq, Z. (2017). Characterizing and modeling patching practices of industrial control systems. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 1(1), 1-23.
• Focus less on CVSS, any discussion of it moves to #30
• reflect new section structure (from #95)
• Focus on what SSVC v2 provides. Do not focus on what SSVC v1 did, or what changes were made. Those go in a separate section, towards the end of the document handled by #122.

The javascript tool has a different version of the tree than either of the two that appear in the WEIS paper. That's fine for showing how one might adapt SSVC for a coordination role, but it'd be helpful too if the js tool also had the "canonical" trees from the paper available too.

Late in the v1 document, there is a "vector string"-like format for expressing decision values compactly. Refine this format. Once we're comfortably with it, mention and signpost to it earlier in the document.
Sub-part of #24
Also related to / might get done when rewriting the intro #34

Merge situated safety impact and mission impact in the applier tree, so that the result of the table or whatever is the max of the two decision values.

E.g., map LL, LM, LH, ML, MM, MH, HL, HM, HH (for two low/med/high dimensions) onto a single L/M/H (input could be more than three possible values, output should be relatively small (3-4 values)

How many rows in the CSV file that exhaustively defines a tree are too many? When does the human decision-maker (who has to approve the risk posture of the tree #20 ) lose track of the context? Discuss aspects of how many distinct combinations of decision values are too many.

Define "public safety impact" on the same axes (physical, financial, etc.). It should be more simple, more broad, acknowledge lack of information readily accessible for many products.

Public safety impact should have at most 3 possible values.

Extends #1

225 is too many possible combinations for the Applier tree, have fewer.
Directly a result of #21

From @j--- in #43 :

Do we have a definition of "risk" that we agree with?
I'd like to avoid the simple (impact x likelihood) formulation of risk, not that it's wrong per se, but because it leaves out distribution of harms, correlated risks, and the uncertainty distribution on the likelihood.

Formally include utility in the applier decision process, even if it just formally limits mission impact choices (as the footnote recommends a user might do).
This is related to feedback, @secursive: https://blog.secursive.com/posts/critical-look-stakeholder-specific-vulnerability-categorization-ssvc/

This has several sub-parts:

• #25
• #26
• #27
• #28
• #29

This issue is cross-referenced with #11 , because coordinators are one important stakeholder who may want to communicate SSVC info.

Some organizations are both developers and appliers. Most software developers use libraries, and so when a library has a vul, that change needs to be applied to the vulnerable product, and then the new version of the product needs to be distributed. SSVC v1 handles this as two separable decisions, one as an applier, the second as a developer. Is this the best way to handle this? How should SSVC facilitate communicating if a library is used by the vulnerable behavior / call is not exposed?

Make clear what is being automated by whom in the definition of virulence (or whatever it is called in v2, #9 )

• Enumerate outcomes for coordinators
• Define decision points for coordinators; share with existing decision points as much as plausible
• Define decision values for any new decision points that need to be created
• Assign an outcome to each combination of decision values

Per conversation on 2020-11-11, add some words to future work about balancing vulnerability risk (which is what SSVC is helping you reason about) with change risk (which it currently does not).

We talked about the potential for change risk profiles (startups might tolerate a lot of change risk, whereas regulated manufacturers might have very low tolerance for change risk) and how that might impact how one decides what to do.

SSVC is telling you how urgently you need to think about your risk, not necessarily what you're going to do about it, you can still choose to accept the risk and take no action. But at least you explicitly thought about it rather than inaction just being the default.

Extends #38

Refinement of #4

Consider merging "none" and "minor" OR "minor" and "major" for the situated safety impact

If we stick with 5 values, then the difference between minor and major would be major directly can cause a safety failure while minor could contribute to a safety failure if other circumstances contributed.

Since we borrowed the categories from established sources, it'd be good if we could maintain the same "cut lines" – i.e., clean merging of categories rather than trying to draw a line somewhere in the middle of an existing one.

This will need to be done after most of the revisions are made to the csv files that actually determine the trees.

We should analyze and report on:

• Gini impurity
• Information gain

in the trees. It may be that these analyses hint at improvements to the tree, such as branch ordering or redundancies. I did this for the v1 trees, and although it didn't result in any major changes it was a useful check that we weren't missing something important.

We were assuming that for there to be a vulnerability, there has to be a technical impact. Make sure this is clear.
There is space to talk about how/when vul mgmt should document things that are not vuls, or some pointer to that discussion. The intended scope for SSVC is "so you've decided there is a vul, what are you going to do about it." But that isn't clear up front, so connect this aspect of the choice about technical impact to some discussion earlier in the document / intro.

Explain the situations where SSVC is complementary to other vul mgmt systems, and where there is overlap.

• CVSS
• EPSS
• Tenable VPR
• https://github.com/varchashva/vPrioritizer

All these systems have seen updates since SSVC v1, make sure the discussion is up to date.

Section 4 of v1.1 says:

There are at least three aspects of asset management that may be important but are out of scope for SSVC. First and most obvious is the transaction cost of conducting the mitigation or fix. System administrators are paid to develop or apply any fixes or mitigations, and there may be other transactional costs such as downtime for updates. Second is the risk of the fix or mitigation introducing a new error or vulnerability. Regression testing is part of managing this type of risk. Finally, there may be an operational cost of applying a fix or mitigation, representing an ongoing change of functionality or increased overhead.

These three costs (transaction, error introduction, and operational change) are listed as out of scope in v1.1 under the assumption that these costs should not change the priority with which a vul ought to be patched. This assumption likely over-simplifies the vulnerability management decision. Version 2 should enumerate the situations in which these costs may legitimately delay the priority with which a vulnerability should be handled. If these situations are both realistic and justified (which seems likely), then how and why these considerations influence the vul handling decision should be integrated into SSVC.

This is important for #14, for example.

Goal is to address the insecure by design issue in areas like ICS. If I can upload new firmware without authentication or code signing verification, do I really care about any patches? There is a huge percentage of assets in ICS where access = complete compromise regardless of vulns. Possible decision values are trivial / minor / major or trivial / partial / total.
(Note: I pasted this from an internal list of suggested changes. Although I'm the opener of this issue, it's not my idea.)

In subsection Guidance on Communicating Results (Section 4) determine what we should do about full format result formatting. It may be too early to do this well, so it may not be appropriate for v2.

This is closely related to #24 and more specifically #25 #26 and #29.

Exposure as written reads like the value is set to what the device should be (SMB or modbus may get set to small) instead of what the device is actually exposed as. "Unavoidable" probably is confusing here.

If someone is communicating information about an SSVC decision point, they may know the value is not one item, but can't identify which of the remaining values is true. Eliminating options is still information, and we should enable this to be communicated efficiently.
The vector-string-like notation (#25) must accommodate this.
Subpart of #24

For example, immediate maybe should be reachable with Exploit = none? If not the default recommendation, explain why and under what conditions it might make sense for someone to adopt this risk posture.

In the Scope section, document the question about "who's security policies do we care about" sort of thing. The user may not be violating their own (implicit) security policy if they jailbreak their own phone. But the kernel vul they use certainly violates the kernel's security policy. Recommend avoiding anything where we need to have evidence of whether the person doing so has an "attacker-like" state of mind, if for no other reason than it's impossible to gather evidence for. So in this case, the jailbreak method is, or at least contains and uses, a vul.

• Where the appropriate decision is "mitigate" use the connection between types of mitigation actions and the SSVC decision points to recommend heuristics about what mitigations may lead to the most efficient risk reduction.

(Associated with Issue #46 )

How could SSVC scoring be automated?
What would automation look like?

Talk about:

• Connection to asset management
• What information feeds that are currently available could be ingested
• Can stock parts of a CVE or CVSS entry be used?
• What feeds could readily be designed to be fit for purpose
• Connection to advice for evidence collection in general #27

This is a one issue because future work and limitations sections are closely related. Future work is problems we can fix. Limitations is problems we know we can't fix because we have made a choice to prioritize something else.

Both of these sections will need to be updated to make sure that all items identified as future work in the rest of the document are collated. And in general, the v2 updates will change what needs to be done in the future and may adjust the limitations. In that sense, a final pass on these sections needs to be one of the last things done in v2.

• Collate items identified as future work in the document to make sure nothing flagged as future work in prior sections is left out of the future work section. The main section headings in the Future Work section should still be used to emphasize the most important items.
• After most v2 work is done, edit / rewrite the future work section
• After most v2 work is done, edit / rewrite the limitations section

Current safety impact text can probably be re-branded as "situated safety impact" or whatever is decided the term should be.

Ref: #1

• For the defined mitigation actions, link them to the SSVC decision point that they can influence.

(Associated with issue #46 )

Describe how SSVC decisions can be used recursively as the situation changes. Changes outside the control of the stakeholder are included in #29. So this issue is to discuss changes to the situation based on actions by the stakeholder. There are situations where, for example, you apply new firewall rules and that will change Exposure. Such a mitigation may not be a complete fix, but if it changes the priority with which you need to act from out-of-cycle to defer, then it was still in some sense an end to the current need to attend to the vul.

Describe difference between 'fix" (remediate) and "mitigate" actions related to recursing decisions.

(formerly part of action #46 )

Be clear about how Industrial Control System and Operations Technology stakeholders are handled. In many cases they may use the usual SSVC v1 decision points, but with a different risk tolerance or suggested tree. Consider demonstrating how such stakeholders might have a different tree. For any changes to v2 are to accommodate these stakeholders and give them appropriate flexibility, document those clearly in one place.

(This was actually from @j--- I think, I just copied & pasted the text into the issue, so be careful when dereferencing the pronoun "I" in the below)
"Safety Impact" probably needs to be split up into one for the vendor and one for the applier. I think the vendor one could be re-used by a coordinator. I'd call it "public safety impact" or some such. The Applier one would be "situated safety impact" or some such.

"Work Item":

• Review the various places the phrase is used
• implement consistently
• a work item may be something besides a 'patch'. It could be a mitigation, accepting risk, or verifying product is at end of life.
• Update usage of "Work Item" to align with the above.

For both of "Patch Applier" and "Patch Developer," the following considerations apply:

• actor description implies only binary actions. Implementing a mitigation, accepting risk or acknowledging product is EOL.
• Suggest and discuss new ROLE identifiers. Eg., system owner, consumer, defender, deployer, supplier, provider, or distributor.
• Substitute the new ROLE identifier throughout the document.
• Review the changes for awkward, or unclear language.
• change "Patch Applier" to "Supplier"
• change "Patch Developer" to "Deployer"

"Worst credible"

• substitute for "worst plausible"

"Out-of-Cycle"

• substitute for "out-of-band" in document
• substitute for "out-of-band" in trees and code that produces trees (@j--- )

This issue is going to be broken down into several sub-parts across different aspects of SSVC.

The "fix" actions, once completed, should close the vulnerability management case and not invite recursion. This makes applying a patch qualitatively different from completing a remediation. It's unclear the best way to explain or notate this.

• Include decision for what type of remediation to apply in the appropriate tree(s).

(Formerly part of issue #46 )

Document importance of what can and cannot change (definitions of decision points and decision values should be fixed, how they are arranged into decisions and what the outcomes are can change).
sub-part of #24

The scope section is clear that "we assign the vulnerability to the nearest, relevant—yet more abstract—discrete component." However, we are generally scoring "units of work." How are these two related? Are we moving away from scoring individual issues and move to scoring the overarching system, be that a library, module, OS or other system? Either way, document the reasoning behind the answer.

Clarify that EOL software is out of scope, because those vendors won't be using the tree or any vul mgmt. Appliers who own EOL'd software should still do vul mgmt, their "work item" just can no longer be "apply a patch."

https://docs.github.com/en/free-pro-team@latest/github/building-a-strong-community/setting-guidelines-for-repository-contributors

040_treesForVulMgmt.md is a big file and is also where a lot of the edits are occurring. It would be better if this file were split into multiple smaller files (perhaps as follows:

Decision Trees for Vulnerability Management
Enumerating Stakeholders
Enumerating Decisions
Developing Patches
Applying Patches
Coordinating Patches
Scope
Boundaries of the Affected System
Reasoning Steps Forward
----cut here----
Likely Decision Points and Relevant Data
Exploitation (Developer, Applier)
Technical Impact (Developer)
Utility (Developer, Applier)
Automatability
Value Density
----cut here----
Safety Impact (Developer, Applier)
Advice for Gathering Information to Answer the Safety Impact Question
Mission Impact (Applier)
Advice for Gathering Information to Answer the Mission Impact Question
System Exposure (Applier)
----cut here----
Relationship to asset management
Patch Developer Tree
Patch Applier Tree
Evidence Gathering Guidance
Development Methodology

CSV to latex to png/pdf.
@j--- has a script for this, needs to clean up and upload to repo.
I tried extensively to figure out how to do this in R or python with graphviz or the data.tree package. They're all optimized for decision trees in an ML context. That's different enough from our operations-research and human decision making aspect that they aren't suitable.

Add vul chaining considerations to automatability, formerly known as virulence #9

Consider renaming Virulence to something more evocative / descriptive, like "wormable." If Virulence remains the title, consider listing synonyms in the heading or foregrounding the common-sense intuition of what it means.

The "fix" actions, once completed, should close the vulnerability management case and not invite recursion. This makes applying a patch qualitatively different from completing a remediation. It's unclear the best way to explain or notate this.

• Decide on most accessible method of incorporating "fix" vs "mitigate" into the SSVC decision process.

(Related to issue #46)

For each decision point, explain a bit better how information to reach a value for that decision point can / should be gathered.
If people follow this guidance, what confidence can be shared when sharing SSVC information?
sub-part of #24

How are timestamps handled? How do users managing changing information? How is changed or changeable information documented?
sub-part of #24

#5 clarifies that the action resulting from SSVC decisions is not just patch. It is any kind of remediation. (Terms follow https://vuls.cert.org/confluence/display/CVD/4.4+Remediation.) The first set of actions is just to lay out this situation clearly and define some possible actions:

• add text to explain that SSVC provides a priority with which to act, and not the action to take. (fixed earlier)
• Define remediation actions for "fix" (probably apply a patch OR change/upgrade the system) (Cite the DoD Instruction 8531.01 and make changes there to align to SSVC as necessary) (fixed earlier)
• Define "mitigation" (may include change configs, change network configs, disable the service, auditing and detection) (Cite the [DoD Instruction 8531.01]https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/853101p.pdf?ver=2020-09-15-143058-347) and make changes there to align to SSVC as necessary) (fixed earlier)
• File any issues related to the CVD guide definitions here
  -- completed by ADH?

The remediation action applied influences what further actions the vulnerability manager should do. That is, there can be a feedback look between actions taken and SSVC decision points. This is not clear in v1.

• Describe how SSVC decisions can be used recursively as the situation changes. Changes outside the control of the stakeholder are covered by #29. So this issue is just to talk about changes to the situation based on actions by the stakeholder. There are situations where, for example, you apply new firewall rules and that will change Exposure. Such a mitigation may not be a complete fix, but if it changes the priority with which you need to act from out-of-cycle to defer, then it was still in some sense an end to the current need to attend to the vul. (Moved to issue #74)
• For the defined remediation actions, link them to the SSVC decision point that they can influence. (Moved to issue #79)
• Seek broad feedback on this section, it's potentially confusing and will be hard to write well. (request comments on drafts related to this issue)

The "fix" actions, once completed, should close the vulnerability management case and not invite recursion. This makes applying a patch qualitatively different from completing a remediation. It's unclear the best way to explain or notate this.

• Describe difference between "fix" and "mitigate" actions related to recursing decisions. (Moved to issue #74)
• Decide on most accessible method of incorporating "fix" vs "mitigate" into the SSVC decision process. (Moved to issue #76)
• Include decision for what type of remediation to apply in the appropriate tree(s). (Moved to issue #77)
• Where the appropriate decision is "mitigate" use the connection between types of mitigation actions and the SSVC decision points to recommend heuristics about what mitigations may lead to the most efficient risk reduction. (Moved to issue #78)

Add a paragraph that explains the difference between the Utility decision point and the Exploitation decision point.

When a vulnerability is published in a central place such as NVD, would it be assigned multiple public impacts based on industry, letting the vendor choose which industries they support up to one step knowledge? (Not sure how these questions should be answered, but maybe it can be handled with NVD issuing partial information and then ISAC's potentially issuing further guidance for their constituencies. )

Extends #1 , #2