feat: add not supported status code

[dabrain34]

In case of 69 exit code from an app, the test should be reported as not supported and be considered as non-error status.

[dabrain34]

KhronosGroup/Vulkan-Video-Samples@0d69850

[dabrain34]

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/10088#note_3191662

[mdimopoulos]

Waiting for https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/10088#note_3191662 to conclude before proceeding with a proper review here

[ylatuya]

@dabrain34 I have opened #323 with a new runner that can be customized to handle errors more easily.

[dabrain34]

That's sound great, let me give it a try

[rsanchez87]

Do you agree to close this PR, @dabrain34 ? Have you seen the changes in this PR?

[dabrain34]

why do you want to close this MR as the idea would be to treat not supported differently from fail and not exit on error?

[rsanchez87]

why do you want to close this MR as the idea would be to treat not supported differently from fail and not exit on error?

You're right, sorry for the confusion. These two PRs are complementary, looking more in detail, this one handles the reporting side, #323 handles the runner side. We'll just need to coordinate the exit codes between them...

[rsanchez87]

Looks good to me, the full path from exit code 69 through to JUnit/CSV/markdown reports is consistent and well wired up.

[rsanchez87]

@dabrain34 , there are merge conflicts with master, looks like _generate_json_summary and _calculate_timeout_adjustment were added in the same area. NOT_SUPPORTED addition to the result_map in _generate_csv_summary plus the new methods from master.

Can you rebase?

[ylatuya]

We should support this in an agnostic way through the NotSupportedExpeception without binding it to the 69 exit code since each decoder implementation might signal this in different ways. As a matter of fact, the GStreamer decoders will not use 69 exit code but instead the runner should raise that exception so it can be handled in a generic way by the test case.

[rsanchez87]

#319 (comment)

[dabrain34]

We should support this in an agnostic way through the NotSupportedExpeception without binding it to the 69 exit code since each decoder implementation might signal this in different ways. As a matter of fact, the GStreamer decoders will not use 69 exit code but instead the runner should raise that exception so it can be handled in a generic way by the test case.

sounds good to me

[dabrain34]

@rsanchez87 rebased and modified to support NotSupported exception in Vulkan Video Samples decoder class