[uss_qualifier] Add acceptable_findings to tested_requirements

[BenjaminPelletier]

Summary

Some users expressed concern regarding the lack of clarity in the nuances of when the tested requirements artifact summarizes as "Pass" versus "Pass with findings", and lack of control of that behavior. This PR attempts to address that concern by introducing the concept of "acceptable findings" in the configuration for tested_requirements artifacts. acceptable_findings is a list of checks that the test designer indicates should be ignored when determining tested_requirements outcomes. Specifically, if the test designer specifies a check in acceptable_findings, that check may fail and affected requirements will still be marked as Pass (or other outcome, when ignoring whether that check passes or fails).

Background

One specific situation this intends to address is the Low-severity finding regarding SCD0005. The requirement is that the USS assign a particular (low) priority to an unactivated op intent managed by a down USS. InterUSS tests this by presenting an unactivated op intent managed by a down USS and asking the USS under test to plan an intersecting flight. If the USS plans successfully, compliance to the requirement has been positively verified. If the USS crashes, the USS is non-compliant. However, it is unclear what should happen if the USS rejects the conflicting flight. On one hand, a USS is not required to plan all flights and may choose to be more conservative and reject the conflicting flight even though it is allowed to plan -- in that case, the USS is compliant with the requirement, but InterUSS can't directly observe that compliance. On the other hand, a USS that fails to comply with the requirement and (for instance) assigns a high priority to the flight will also reject the conflicting flight because its prioritization logic is incorrect. Therefore, a USS's rejection of the flight severely limits the conclusions InterUSS can draw regarding compliance to SCD0005, and therefore InterUSS currently produces a Low-severity finding. This indicates that no non-compliance has been positively detected, but the most informative/valuable check InterUSS wanted to do could not be performed.

The result of this situation is that a conservative USS has its tested_requirements artifact outcome reported as "Pass with findings" shown in yellow rather than "Pass" in green. This status is hard for a regulator to interpret because they are unsure whether the findings are important or not. The test designer would like the ability to acknowledge and discard this particular finding so that conservative USSs obtain a clean "Pass" result, even though that means the strength of automated SCD0005 verification may be weaker than it would be otherwise.

Solution

This PR fulfills the above desire by allowing the test designer to specify this particular Low-severity check in acceptable_findings, which will result in a clean "Pass" rather than "Pass with findings" when the USS rejects the conflicting flight.

In general, this approach places more control over the behavior of tested_requirements artifacts in the hands of test designers. I believe this is desirable because ultimately test designers are the ones who are responsible for defining their compliance matrices and how InterUSS automated testing tools fit in that compliance strategy -- InterUSS cannot and should not choose single right answers in terms of test success and compliance mappings. One could imagine, for instance, a future change taking this further by adding required_findings where a test designer requires certain checks to pass in order to obtain an overall passing result because the test designer believes those checks are necessary to achieve "sufficient" test coverage.

Impacts

tested_requirements generation

This change affects tested_requirements generation by adding a number of additional edge cases and therefore increasing the complexity of the generation logic. This PR attempts to mitigate this by rewriting the user-facing documentation in a more straightforward manner and then updating the code logic to more closely match that documentation. The new acceptable_findings information needs to be passed around everywhere when computing the breakdown in this generation process, but the only major change in the breakdown/summary itself is a new flag on TestedCheck indicating whether a finding for that check is acceptable (to be ignored).

The logic that bubbles up TestedChecks to TestedRequirements and the overall status is updated/streamlined to match the documentation and hopefully be more obvious and less prone to edge-case problems. Failed checks are still shown in the requirements list, but they are highlighted purple rather than red and do not affect the requirement rollup (and therefore the overall rollup).

Acceptable CI failures

Currently, we have a TestExclusions resource that changes which actions are performed during a test run. We do this because we don't want to count certain failures against the CI because it's not feasible to pass those checks with the CI. It seems like a better way to accomplish this would be to define which checks it's ok to fail during CI. This PR does this for f3548_self_contained -- the TestExclusions is removed, and instead the check that fails is specified in acceptable_findings. This results in the check failing, but the test still continues, the failure is indicated in both the sequence view and tested requirements artifacts, and the tested requirements artifact rolls DSS0300 up to success despite the failed check. I also tested behavior after disabling the other check for DSS0300 and seeing that the roll up disposition of DSS0300 is "not tested" in that case (as expected).

stop_fast

To enable the CI behavior above, this PR also adds a configuration option to modify the stop_fast behavior -- instead of stopping on any failure, the option allows stopping on any failure that isn't in acceptable_findings. Presumably failed checks are expected for anything in acceptable_findings, so a test run should not treat these expected failures as a reason to shut down early.