diff --git a/docs/RFCs/011-Soft-Assertions-Nullability-Design.md b/docs/RFCs/011-Soft-Assertions-Nullability-Design.md new file mode 100644 index 0000000000..e16a4d81d3 --- /dev/null +++ b/docs/RFCs/011-Soft-Assertions-Nullability-Design.md @@ -0,0 +1,263 @@ +# RFC 011 - Soft Assertions and Nullability Annotation Design + +- [x] Approved in principle +- [x] Under discussion +- [x] Implementation +- [ ] Shipped + +## Summary + +`Assert.Scope()` introduces soft assertions — assertion failures are collected and reported together when the scope is disposed, rather than throwing immediately. This fundamentally conflicts with C# nullability annotations (`[DoesNotReturn]`, `[DoesNotReturnIf]`, `[NotNull]`) that rely on the assumption that assertion failure always means throwing an exception. This RFC documents the problem, the options considered, and the chosen design. + +## Motivation + +Today, every MSTest assertion throws `AssertFailedException` on failure. With `Assert.Scope()`, we want to allow multiple assertion failures to be collected and reported at once: + +```csharp +using (Assert.Scope()) +{ + Assert.AreEqual(1, actual.X); // failure collected, execution continues + Assert.AreEqual(2, actual.Y); // failure collected, execution continues + Assert.IsTrue(actual.IsValid); // failure collected, execution continues +} +// Dispose() throws AggregateException-like AssertFailedException with all 3 failures +``` + +## Technical challenges + +Soft assertions create a fundamental tension with C# nullability annotations and, more broadly, with all assertion postconditions. + +Before soft assertions, the helper method responsible for reporting assertion failures (for example, `ThrowAssertFailed`) was annotated with `[DoesNotReturn]`, which let the compiler prove post-condition contracts. For example: + +```csharp +public static void IsNotNull([NotNull] object? value, ...) +{ + if (value is null) + { + ThrowAssertIsNotNullFailed(...); // [DoesNotReturn] — compiler trusts value is not null after this + } + // value is known non-null here +} +``` + +To support soft assertions, `ReportAssertFailed` was changed so it no longer always throws — within a scope, it adds the failure to a queue and *returns*. This means: + +1. `[DoesNotReturn]` can no longer be applied to the general failure path. +2. `[DoesNotReturnIf(false)]` on `IsTrue` / `[DoesNotReturnIf(true)]` on `IsFalse` become lies — the method can return even when the condition is not met. +3. `[NotNull]` on parameters like `IsNotNull(object? value)` becomes a lie — the method can return even when `value` is null. + +If we lie about these annotations, **downstream code after the assertion will get wrong nullability analysis**, potentially causing `NullReferenceException` at runtime with no compiler warning. + +After discussion with the Roslyn team, a key insight emerged: **this is a general problem with postconditions, not specific to nullability**. `Assert.Scope()` means assertions are no longer enforcing *any* postconditions. Consider: + +```csharp +using (Assert.Scope()) +{ + Assert.AreEqual("blah", item.Prop); + MyTestHelper(item.Prop); // may explode if Prop doesn't have expected form +} +``` + +`Assert.AreEqual` in scoped mode already does not enforce its postcondition (that the values are equal). Code after the assertion may use `item.Prop` assuming it has a particular value, and that assumption may be wrong. The nullability case (`IsNotNull` not guaranteeing non-null) is conceptually identical — it's just another postcondition that isn't enforced within a scope. + +## Options Considered + +### Option 1: Remove all nullability annotations + +Remove `[DoesNotReturn]`, `[DoesNotReturnIf]`, and `[NotNull]` from all assertions. + +**Pros:** Honest to the compiler. No CS8777 warnings. +**Cons:** Massive regression in developer experience. Users who write `Assert.IsNotNull(obj); obj.Method();` would now get a nullable warning on every call after the assertion. This would be a major breaking change to the user experience of the framework. + +**Verdict:** Rejected. Too disruptive for all users, including those who never use `Assert.Scope()`. + +### Option 2: Pragmatic tier split + +Categorize assertions into tiers based on whether their post-conditions narrow types, and handle each tier differently. + +**Tier 1 — Always throw (hard assertions):** Assertions whose annotations change the type state of a variable for subsequent code. These must always throw, even within a scope, because continuing execution with a wrong type assumption would cause immediate downstream errors unrelated to the assertion. + +- `IsNotNull` — annotated `[NotNull]` on the value parameter +- `IsInstanceOfType` — annotated `[NotNull]` on the value parameter +- `IsExactInstanceOfType` — annotated `[NotNull]` on the value parameter +- `Fail` — semantically means "unconditional failure"; annotated `[DoesNotReturn]` on public API +- `Inconclusive` — semantically means "unconditional inconclusive"; annotated `[DoesNotReturn]` on public API and throws `AssertInconclusiveException` (not `AssertFailedException`) +- `ContainsSingle` — returns the matched element; returning `default` in soft mode would give callers a bogus `null`/`default(T)` causing downstream errors + +**Tier 2 — Soft, but annotations removed:** Assertions that had conditional `[DoesNotReturnIf]` annotations. The annotation is removed so the compiler no longer assumes the condition is guaranteed. The assertions become soft (collected within a scope). + +- `IsTrue` — `[DoesNotReturnIf(false)]` removed +- `IsFalse` — `[DoesNotReturnIf(true)]` removed + +**Tier 3 — Soft, no annotation impact:** All other assertions that don't carry type-narrowing annotations. These become fully soft within a scope. + +**Pros:** Type-narrowing contracts are always truthful. Soft assertions work for the vast majority of assertions. +**Cons:** `IsNotNull` / `IsInstanceOfType` / `IsExactInstanceOfType` won't participate in soft assertion collection — they still throw immediately within a scope. This significantly reduces the value of `Assert.Scope()` for common test patterns like null-checking multiple properties. Users lose `[DoesNotReturnIf]` narrowing on `IsTrue`/`IsFalse` even outside scopes. + +**Verdict:** Rejected. Carving out exceptions makes the scoping feature less useful, and the safety benefit is questionable given that *all* postconditions (not just nullability ones) are already unenforced in scoped mode. + +### Option 3: Keep all annotations, suppress compiler warnings (chosen) + +Keep `[DoesNotReturn]`, `[DoesNotReturnIf]`, `[NotNull]` on all assertion methods. Make all assertions soft within a scope (except `Assert.Fail()`, `Assert.Inconclusive()`, and `CheckParameterNotNull`). Suppress `#pragma warning disable CS8777` / `CS8763` where the compiler objects. + +This is the approach recommended by the Roslyn team: leave all nullable attributes on, but do not actually ensure any of the postconditions when in `Assert.Scope()` context. This is consistent with what we are already doing for all postconditions unrelated to nullability — `Assert.AreEqual` doesn't guarantee equality in scoped mode, `Assert.IsTrue` doesn't guarantee the condition was true, and so on. The nullability annotations are no different. + +**Pros:** + +- **No user-facing annotation changes.** Users outside `Assert.Scope()` get the exact same experience — `Assert.IsNotNull(obj); obj.Method()` has no nullable warning, `Assert.IsTrue(b)` narrows `bool?` to `bool`. Zero regression. +- **All assertions participate in soft collection.** `IsNotNull`, `IsInstanceOfType`, `IsExactInstanceOfType`, `ContainsSingle`, `IsTrue`, `IsFalse` are all soft within a scope. This maximizes the value of `Assert.Scope()`. +- **Consistent mental model.** The rule is simple: within `Assert.Scope()`, assertion failures are collected and postconditions are not enforced. This applies uniformly to all assertions (except `Assert.Fail()` and `Assert.Inconclusive()`), whether the postcondition is about nullability, type narrowing, equality, or anything else. + +**Cons:** + +- **The annotations are lies inside a scope.** `Assert.IsNotNull(obj)` inside a scope won't throw when `obj` is null, meaning `obj` could still be null on the next line, but the compiler thinks it's non-null. This can cause `NullReferenceException` at runtime with no compiler warning. +- **Requires `#pragma warning disable` to suppress CS8777/CS8763.** The compiler correctly identifies that our implementation doesn't fulfill the annotation promises in all code paths. + +The runtime risk is acceptable for the same reason that non-nullability postconditions being unenforced is acceptable: the assertion *will* be reported as failed when the scope disposes. The user will see the failure. If downstream code crashes due to a violated postcondition (whether it's a `NullReferenceException` from a null value, or some other error from an unexpected value), that crash is a secondary symptom of the already-reported assertion failure — not a silent, hidden bug. + +Users who need a postcondition to be enforced for subsequent code to work correctly can use `Assert.Fail()` or `Assert.Inconclusive()` (which always throw) or restructure their test to not depend on the postcondition after the assertion within a scope. + +**Verdict:** Chosen. This approach gives the best user experience both inside and outside `Assert.Scope()`, and is consistent with how all other postconditions already behave in scoped mode. + +## Detailed Design + +This section describes the implementation of Option 3 (keep all annotations, suppress compiler warnings), which centers on a single `ReportAssertFailed` method that switches behavior based on whether an `AssertScope` is active. + +### `ReportAssertFailed` + +```csharp +[StackTraceHidden] +internal static void ReportAssertFailed(string assertionName, string? message) +``` + +- Within an `AssertScope`: adds failure to the scope's queue and **returns**. +- Outside a scope: **throws** `AssertFailedException` (preserves existing behavior). + +### `AssertScope.Dispose()` + +When an `AssertScope` is disposed and it contains collected failures: + +- **Single failure:** Throws the original `AssertFailedException` and triggers the debugger. +- **Multiple failures:** Throws a new `AssertFailedException` wrapping all collected failures into an `AggregateException` as the inner exception. + +This design ensures the debugger breaks at the point where the scope is disposed, giving the developer visibility into all collected failures. + +### Nullable annotations: kept but unenforced in scoped mode + +All nullable annotations (`[NotNull]`, `[DoesNotReturnIf]`) are kept on their respective assertion methods. Within a scope, these postconditions are not enforced — the method may return without the postcondition being true. Compiler warnings (CS8777, CS8763) arising from this are suppressed with `#pragma warning disable`. + +This is the same approach the Roslyn team recommended. As Rikki from the Roslyn team noted: + +> It feels like this is an issue with postconditions in general... Assert.Scoped() means assertions are no longer enforcing postconditions. [...] I would honestly start by just trying leaving all the nullable attributes on, but not actually ensuring any of the postconditions, when in Assert.Scoped() context. Since that is essentially what you are doing already with all postconditions unrelated to nullability. See how that works in practice, and, if the usability feels bad, you could consider introducing certain assertions that throw regardless of whether you're in scoped context or not. + +### `Assert.Fail()` and `Assert.Inconclusive()` — hard by design + +`Assert.Fail()` and `Assert.Inconclusive()` are the only assertions that always throw, even within a scope. They bypass `ReportAssertFailed` for these reasons: + +1. **Semantics:** `Fail()` means "this test has unconditionally failed" and `Inconclusive()` means "this test cannot determine its result." There is no meaningful scenario where you'd want to collect these and keep executing — the developer explicitly declared the test outcome. +2. **Exception types:** `Assert.Fail()` throws `AssertFailedException` while `Assert.Inconclusive()` throws `AssertInconclusiveException`. The test runner relies on the exception type to distinguish between failed and inconclusive outcomes. Routing `Inconclusive()` through `ReportAssertFailed` would also lose this distinction, since the scope's collection queue only holds `AssertFailedException`. +3. **Public API contract:** Both are annotated `[DoesNotReturn]`, and users rely on this for control flow: + +```csharp +var result = condition switch +{ + Case.A => HandleA(), + Case.B => HandleB(), + _ => Assert.Fail("Unexpected case") // compiler requires [DoesNotReturn] or it's CS0161 +}; +``` + +Making `Fail()` and `Inconclusive()` hard keeps the `[DoesNotReturn]` annotation truthful with no pragma suppression needed. + +## Impact on Users + +### No `Assert.Scope()` — no change + +Users who don't use `Assert.Scope()` experience **zero behavioral change**. All assertions throw exactly as before. All nullable annotations remain in place. There is no regression. + +### Within `Assert.Scope()` + +All assertions participate in soft failure collection, with the following exceptions: + +- **`Assert.Fail()`** and **`Assert.Inconclusive()`** are the only assertion APIs that do not respect soft failure mode. They always throw immediately, even within a scope — `Fail()` because it semantically means "this test has unconditionally failed" and `Inconclusive()` because it means "this test cannot determine its result." Additionally, `Inconclusive()` throws `AssertInconclusiveException` (not `AssertFailedException`), so it cannot be collected in the scope's failure queue. +- **Null precondition checks** inside Assert APIs (e.g., validating that a `Type` argument passed to `IsInstanceOfType` is not null) also throw directly rather than collecting. These are internal parameter validation checks (`CheckParameterNotNull`), not assertions on the value under test. Note that `Assert.IsNotNull` / `Assert.IsNull` are *not* precondition checks — they are assertions on test values and participate in soft collection normally. + +### Dealing with postcondition-dependent code in scoped mode + +When using `Assert.Scope()`, code after an assertion should not depend on the assertion's postcondition. This applies to all postconditions, whether nullability-related or not: + +```csharp +using (Assert.Scope()) +{ + Assert.IsNotNull(item); + // item might still be null here — the assertion failure was collected, not thrown. + // If you need item to be non-null for the rest of the test, use Assert.Fail() + // or restructure the test. + + Assert.AreEqual("blah", item.Prop); + MyTestHelper(item.Prop); + // item.Prop might not be "blah" — same issue, different postcondition. +} +``` + +If a test helper depends on a postcondition being true, the user has several options: + +1. **Use `Assert.Fail()` or `Assert.Inconclusive()` for critical preconditions** — they always throw, even in scoped mode. +2. **Restructure the test** to not depend on postconditions within the scope. +3. **Accept the secondary failure** — the primary assertion failure will be reported, and any downstream crash is a secondary symptom. + +This is simply part of the adoption/onboarding cost of using `Assert.Scope()`. The scoping feature trades strict postcondition enforcement for the ability to see multiple failures at once. + +## Design Decisions + +### Why lying to the compiler is acceptable + +The decision to keep annotations that are not enforced in scoped mode is justified by: + +1. **Consistency.** All assertion postconditions are already unenforced in scoped mode. Making nullability postconditions the exception adds complexity without meaningful safety improvement. +2. **User experience.** Removing annotations would regress the experience for all users, including those who never use `Assert.Scope()`. +3. **Practicality.** The Roslyn team confirmed this approach is reasonable. Tests that use `Assert.Scope()` are inherently opting into a mode where postconditions are deferred, and users should expect that downstream code may encounter unexpected state. +4. **Observable failures.** The violated postcondition doesn't cause silent bugs — the assertion failure *is* reported when the scope disposes. Any secondary crash is additional evidence of the already-reported failure. +5. **Experimental API.** The `Assert.Scope()` API is currently marked as experimental, which allows us to gather concrete usages and feedback from users before committing to a stable release. Real-world usage patterns will inform whether any adjustments to the annotation strategy or scoping behavior are needed, and we can iterate on the design without breaking stable API contracts. + +### Why nested scopes are not supported + +Nested `Assert.Scope()` calls are currently not allowed. We do not see a compelling usage scenario that justifies the added complexity of defining nested scope semantics (e.g., should inner scope failures propagate to the parent scope or throw immediately?). This decision can be revisited based on customer feedback if concrete use cases emerge. + +## Future Improvements + +### Explicit hard-assertion opt-in within a scope + +There may be cases where a user wants a specific assertion to throw immediately within a scope, even though it would normally be soft. A possible API could be: + +```csharp +using (Assert.Scope()) +{ + Assert.AreEqual(1, actual.Count); // soft + Assert.Hard.AreEqual("expected", actual); // hard — throws immediately + Assert.AreEqual(2, actual.Other); // soft +} +``` + +The exact shape of this API is not yet designed. As the Roslyn team suggested, users may want certain assertions to always throw so they can enforce postconditions that subsequent code depends on, even within a scope. This would be part of the natural evolution of the feature based on real-world usage feedback. + +### Extensibility for custom assertion authors + +Third-party libraries and users who author custom assertions (via `Assert.That` extension methods or standalone assertion classes) currently have no public API to participate in soft assertion collection. They can only call `Assert.Fail()` (which is hard) or throw `AssertFailedException` directly (which bypasses the scope). + +A future improvement could expose a public API for custom assertion authors to report soft failures, e.g.: + +```csharp +public static class MyCustomAssertions +{ + public static void HasProperty(this Assert assert, object obj, string propertyName) + { + if (obj.GetType().GetProperty(propertyName) is null) + { + Assert.ReportFailure("MyAssert.HasProperty", $"Expected property '{propertyName}' not found."); + } + } +} +``` + +This would require promoting `ReportAssertFailed` (or a new public variant) from `internal` to `public`, with careful API design to avoid exposing implementation details. diff --git a/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs b/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs index 5cd7fe9939..c7f7a27d70 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -49,7 +49,7 @@ internal void ComputeAssertion(string expectedExpression, string actualExpressio if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionTwoParametersMessage, "expected", expectedExpression, "actual", actualExpression) + " "); - ThrowAssertAreEqualFailed(_expected, _actual, _builder.ToString()); + ReportAssertAreEqualFailed(_expected, _actual, _builder.ToString()); } } @@ -116,7 +116,7 @@ internal void ComputeAssertion(string notExpectedExpression, string actualExpres if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionTwoParametersMessage, "notExpected", notExpectedExpression, "actual", actualExpression) + " "); - ThrowAssertAreNotEqualFailed(_notExpected, _actual, _builder.ToString()); + ReportAssertAreNotEqualFailed(_notExpected, _actual, _builder.ToString()); } } @@ -167,7 +167,7 @@ public AssertNonGenericAreEqualInterpolatedStringHandler(int literalLength, int if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -177,7 +177,7 @@ public AssertNonGenericAreEqualInterpolatedStringHandler(int literalLength, int if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -187,7 +187,7 @@ public AssertNonGenericAreEqualInterpolatedStringHandler(int literalLength, int if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -197,7 +197,7 @@ public AssertNonGenericAreEqualInterpolatedStringHandler(int literalLength, int if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -213,7 +213,7 @@ public AssertNonGenericAreEqualInterpolatedStringHandler(int literalLength, int if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreEqualFailed(expected, actual, ignoreCase, culture, userMessage); + _failAction = userMessage => ReportAssertAreEqualFailed(expected, actual, ignoreCase, culture, userMessage); } } @@ -273,7 +273,7 @@ public AssertNonGenericAreNotEqualInterpolatedStringHandler(int literalLength, i if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -283,7 +283,7 @@ public AssertNonGenericAreNotEqualInterpolatedStringHandler(int literalLength, i if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -293,7 +293,7 @@ public AssertNonGenericAreNotEqualInterpolatedStringHandler(int literalLength, i if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -303,7 +303,7 @@ public AssertNonGenericAreNotEqualInterpolatedStringHandler(int literalLength, i if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + _failAction = userMessage => ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -319,7 +319,7 @@ public AssertNonGenericAreNotEqualInterpolatedStringHandler(int literalLength, i if (shouldAppend) { _builder = new StringBuilder(literalLength + formattedCount); - _failAction = userMessage => ThrowAssertAreNotEqualFailed(notExpected, actual, userMessage); + _failAction = userMessage => ReportAssertAreNotEqualFailed(notExpected, actual, userMessage); } } @@ -489,7 +489,7 @@ public static void AreEqual(T? expected, T? actual, IEqualityComparer comp } string userMessage = BuildUserMessageForExpectedExpressionAndActualExpression(message, expectedExpression, actualExpression); - ThrowAssertAreEqualFailed(expected, actual, userMessage); + ReportAssertAreEqualFailed(expected, actual, userMessage); } private static bool AreEqualFailing(T? expected, T? actual, IEqualityComparer? comparer) @@ -643,7 +643,7 @@ private static string FormatStringDifferenceMessage(string expected, string actu } [DoesNotReturn] - private static void ThrowAssertAreEqualFailed(object? expected, object? actual, string userMessage) + private static void ReportAssertAreEqualFailed(object? expected, object? actual, string userMessage) { string finalMessage = actual != null && expected != null && !actual.GetType().Equals(expected.GetType()) ? string.Format( @@ -662,11 +662,11 @@ private static void ThrowAssertAreEqualFailed(object? expected, object? actual, userMessage, ReplaceNulls(expected), ReplaceNulls(actual)); - ThrowAssertFailed("Assert.AreEqual", finalMessage); + ReportAssertFailed("Assert.AreEqual", finalMessage); } [DoesNotReturn] - private static void ThrowAssertAreEqualFailed(T expected, T actual, T delta, string userMessage) + private static void ReportAssertAreEqualFailed(T expected, T actual, T delta, string userMessage) where T : struct, IConvertible { string finalMessage = string.Format( @@ -676,11 +676,11 @@ private static void ThrowAssertAreEqualFailed(T expected, T actual, T delta, expected.ToString(CultureInfo.CurrentCulture.NumberFormat), actual.ToString(CultureInfo.CurrentCulture.NumberFormat), delta.ToString(CultureInfo.CurrentCulture.NumberFormat)); - ThrowAssertFailed("Assert.AreEqual", finalMessage); + ReportAssertFailed("Assert.AreEqual", finalMessage); } [DoesNotReturn] - private static void ThrowAssertAreEqualFailed(string? expected, string? actual, bool ignoreCase, CultureInfo culture, string userMessage) + private static void ReportAssertAreEqualFailed(string? expected, string? actual, bool ignoreCase, CultureInfo culture, string userMessage) { string finalMessage; @@ -700,7 +700,7 @@ private static void ThrowAssertAreEqualFailed(string? expected, string? actual, finalMessage = FormatStringComparisonMessage(expected, actual, userMessage); } - ThrowAssertFailed("Assert.AreEqual", finalMessage); + ReportAssertFailed("Assert.AreEqual", finalMessage); } /// @@ -792,7 +792,7 @@ public static void AreNotEqual(T? notExpected, T? actual, IEqualityComparer @@ -838,7 +838,7 @@ public static void AreEqual(float expected, float actual, float delta, string? m if (AreEqualFailing(expected, actual, delta)) { string userMessage = BuildUserMessageForExpectedExpressionAndActualExpression(message, expectedExpression, actualExpression); - ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -885,7 +885,7 @@ public static void AreNotEqual(float notExpected, float actual, float delta, str if (AreNotEqualFailing(notExpected, actual, delta)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndActualExpression(message, notExpectedExpression, actualExpression); - ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -953,7 +953,7 @@ public static void AreEqual(decimal expected, decimal actual, decimal delta, str if (AreEqualFailing(expected, actual, delta)) { string userMessage = BuildUserMessageForExpectedExpressionAndActualExpression(message, expectedExpression, actualExpression); - ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -1000,7 +1000,7 @@ public static void AreNotEqual(decimal notExpected, decimal actual, decimal delt if (AreNotEqualFailing(notExpected, actual, delta)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndActualExpression(message, notExpectedExpression, actualExpression); - ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -1050,7 +1050,7 @@ public static void AreEqual(long expected, long actual, long delta, string? mess if (AreEqualFailing(expected, actual, delta)) { string userMessage = BuildUserMessageForExpectedExpressionAndActualExpression(message, expectedExpression, actualExpression); - ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -1097,7 +1097,7 @@ public static void AreNotEqual(long notExpected, long actual, long delta, string if (AreNotEqualFailing(notExpected, actual, delta)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndActualExpression(message, notExpectedExpression, actualExpression); - ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -1146,7 +1146,7 @@ public static void AreEqual(double expected, double actual, double delta, string if (AreEqualFailing(expected, actual, delta)) { string userMessage = BuildUserMessageForExpectedExpressionAndActualExpression(message, expectedExpression, actualExpression); - ThrowAssertAreEqualFailed(expected, actual, delta, userMessage); + ReportAssertAreEqualFailed(expected, actual, delta, userMessage); } } @@ -1193,7 +1193,7 @@ public static void AreNotEqual(double notExpected, double actual, double delta, if (AreNotEqualFailing(notExpected, actual, delta)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndActualExpression(message, notExpectedExpression, actualExpression); - ThrowAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); + ReportAssertAreNotEqualFailed(notExpected, actual, delta, userMessage); } } @@ -1219,7 +1219,7 @@ private static bool AreNotEqualFailing(double notExpected, double actual, double } [DoesNotReturn] - private static void ThrowAssertAreNotEqualFailed(T notExpected, T actual, T delta, string userMessage) + private static void ReportAssertAreNotEqualFailed(T notExpected, T actual, T delta, string userMessage) where T : struct, IConvertible { string finalMessage = string.Format( @@ -1229,7 +1229,7 @@ private static void ThrowAssertAreNotEqualFailed(T notExpected, T actual, T d notExpected.ToString(CultureInfo.CurrentCulture.NumberFormat), actual.ToString(CultureInfo.CurrentCulture.NumberFormat), delta.ToString(CultureInfo.CurrentCulture.NumberFormat)); - ThrowAssertFailed("Assert.AreNotEqual", finalMessage); + ReportAssertFailed("Assert.AreNotEqual", finalMessage); } /// @@ -1323,7 +1323,7 @@ public static void AreEqual(string? expected, string? actual, bool ignoreCase, C } string userMessage = BuildUserMessageForExpectedExpressionAndActualExpression(message, expectedExpression, actualExpression); - ThrowAssertAreEqualFailed(expected, actual, ignoreCase, culture, userMessage); + ReportAssertAreEqualFailed(expected, actual, ignoreCase, culture, userMessage); } /// @@ -1419,7 +1419,7 @@ public static void AreNotEqual(string? notExpected, string? actual, bool ignoreC } string userMessage = BuildUserMessageForNotExpectedExpressionAndActualExpression(message, notExpectedExpression, actualExpression); - ThrowAssertAreNotEqualFailed(notExpected, actual, userMessage); + ReportAssertAreNotEqualFailed(notExpected, actual, userMessage); } private static bool AreNotEqualFailing(string? notExpected, string? actual, bool ignoreCase, CultureInfo culture) @@ -1429,7 +1429,7 @@ private static bool AreNotEqualFailing(T? notExpected, T? actual, IEqualityCo => (comparer ?? EqualityComparer.Default).Equals(notExpected!, actual!); [DoesNotReturn] - private static void ThrowAssertAreNotEqualFailed(object? notExpected, object? actual, string userMessage) + private static void ReportAssertAreNotEqualFailed(object? notExpected, object? actual, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, @@ -1437,7 +1437,7 @@ private static void ThrowAssertAreNotEqualFailed(object? notExpected, object? ac userMessage, ReplaceNulls(notExpected), ReplaceNulls(actual)); - ThrowAssertFailed("Assert.AreNotEqual", finalMessage); + ReportAssertFailed("Assert.AreNotEqual", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.AreSame.cs b/src/TestFramework/TestFramework/Assertions/Assert.AreSame.cs index 93be91c7fd..2960d31e69 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.AreSame.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.AreSame.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -39,7 +39,7 @@ internal void ComputeAssertion(string expectedExpression, string actualExpressio if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionTwoParametersMessage, "expected", expectedExpression, "actual", actualExpression) + " "); - ThrowAssertAreSameFailed(_expected, _actual, _builder.ToString()); + ReportAssertAreSameFailed(_expected, _actual, _builder.ToString()); } } @@ -95,7 +95,7 @@ internal void ComputeAssertion(string notExpectedExpression, string actualExpres if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionTwoParametersMessage, "notExpected", notExpectedExpression, "actual", actualExpression) + " "); - ThrowAssertAreNotSameFailed(_builder.ToString()); + ReportAssertAreNotSameFailed(_builder.ToString()); } } @@ -178,14 +178,14 @@ public static void AreSame(T? expected, T? actual, string? message = "", [Cal } string userMessage = BuildUserMessageForExpectedExpressionAndActualExpression(message, expectedExpression, actualExpression); - ThrowAssertAreSameFailed(expected, actual, userMessage); + ReportAssertAreSameFailed(expected, actual, userMessage); } private static bool IsAreSameFailing(T? expected, T? actual) => !object.ReferenceEquals(expected, actual); [DoesNotReturn] - private static void ThrowAssertAreSameFailed(T? expected, T? actual, string userMessage) + private static void ReportAssertAreSameFailed(T? expected, T? actual, string userMessage) { string finalMessage = userMessage; if (expected is ValueType && actual is ValueType) @@ -196,7 +196,7 @@ private static void ThrowAssertAreSameFailed(T? expected, T? actual, string u userMessage); } - ThrowAssertFailed("Assert.AreSame", finalMessage); + ReportAssertFailed("Assert.AreSame", finalMessage); } /// @@ -240,7 +240,7 @@ public static void AreNotSame(T? notExpected, T? actual, string? message = "" { if (IsAreNotSameFailing(notExpected, actual)) { - ThrowAssertAreNotSameFailed(BuildUserMessageForNotExpectedExpressionAndActualExpression(message, notExpectedExpression, actualExpression)); + ReportAssertAreNotSameFailed(BuildUserMessageForNotExpectedExpressionAndActualExpression(message, notExpectedExpression, actualExpression)); } } @@ -248,6 +248,6 @@ private static bool IsAreNotSameFailing(T? notExpected, T? actual) => object.ReferenceEquals(notExpected, actual); [DoesNotReturn] - private static void ThrowAssertAreNotSameFailed(string userMessage) - => ThrowAssertFailed("Assert.AreNotSame", userMessage); + private static void ReportAssertAreNotSameFailed(string userMessage) + => ReportAssertFailed("Assert.AreNotSame", userMessage); } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.Contains.cs b/src/TestFramework/TestFramework/Assertions/Assert.Contains.cs index 819452d1f4..cb7b074f8c 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.Contains.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.Contains.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -40,7 +40,7 @@ internal TItem ComputeAssertion(string collectionExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "collection", collectionExpression) + " "); - ThrowAssertContainsSingleFailed(_actualCount, _builder.ToString()); + ReportAssertContainsSingleFailed(_actualCount, _builder.ToString()); } return _item!; @@ -165,12 +165,12 @@ public static T ContainsSingle(Func predicate, IEnumerable collec if (string.IsNullOrEmpty(predicateExpression)) { string userMessage = BuildUserMessageForCollectionExpression(message, collectionExpression); - ThrowAssertContainsSingleFailed(actualCount, userMessage); + ReportAssertContainsSingleFailed(actualCount, userMessage); } else { string userMessage = BuildUserMessageForPredicateExpressionAndCollectionExpression(message, predicateExpression, collectionExpression); - ThrowAssertSingleMatchFailed(actualCount, userMessage); + ReportAssertSingleMatchFailed(actualCount, userMessage); } // Unreachable code but compiler cannot work it out @@ -226,12 +226,12 @@ public static T ContainsSingle(Func predicate, IEnumerable collec if (string.IsNullOrEmpty(predicateExpression)) { string userMessage = BuildUserMessageForCollectionExpression(message, collectionExpression); - ThrowAssertContainsSingleFailed(matchCount, userMessage); + ReportAssertContainsSingleFailed(matchCount, userMessage); } else { string userMessage = BuildUserMessageForPredicateExpressionAndCollectionExpression(message, predicateExpression, collectionExpression); - ThrowAssertSingleMatchFailed(matchCount, userMessage); + ReportAssertSingleMatchFailed(matchCount, userMessage); } return default; @@ -261,7 +261,7 @@ public static void Contains(T expected, IEnumerable collection, string? me if (!collection.Contains(expected)) { string userMessage = BuildUserMessageForExpectedExpressionAndCollectionExpression(message, expectedExpression, collectionExpression); - ThrowAssertContainsItemFailed(userMessage); + ReportAssertContainsItemFailed(userMessage); } } @@ -292,7 +292,7 @@ public static void Contains(object? expected, IEnumerable collection, string? me } string userMessage = BuildUserMessageForExpectedExpressionAndCollectionExpression(message, expectedExpression, collectionExpression); - ThrowAssertContainsItemFailed(userMessage); + ReportAssertContainsItemFailed(userMessage); } /// @@ -316,7 +316,7 @@ public static void Contains(T expected, IEnumerable collection, IEqualityC if (!collection.Contains(expected, comparer)) { string userMessage = BuildUserMessageForExpectedExpressionAndCollectionExpression(message, expectedExpression, collectionExpression); - ThrowAssertContainsItemFailed(userMessage); + ReportAssertContainsItemFailed(userMessage); } } @@ -349,7 +349,7 @@ public static void Contains(object? expected, IEnumerable collection, IEqualityC } string userMessage = BuildUserMessageForExpectedExpressionAndCollectionExpression(message, expectedExpression, collectionExpression); - ThrowAssertContainsItemFailed(userMessage); + ReportAssertContainsItemFailed(userMessage); } /// @@ -372,7 +372,7 @@ public static void Contains(Func predicate, IEnumerable collectio if (!collection.Any(predicate)) { string userMessage = BuildUserMessageForPredicateExpressionAndCollectionExpression(message, predicateExpression, collectionExpression); - ThrowAssertContainsPredicateFailed(userMessage); + ReportAssertContainsPredicateFailed(userMessage); } } @@ -404,7 +404,7 @@ public static void Contains(Func predicate, IEnumerable collectio } string userMessage = BuildUserMessageForPredicateExpressionAndCollectionExpression(message, predicateExpression, collectionExpression); - ThrowAssertContainsPredicateFailed(userMessage); + ReportAssertContainsPredicateFailed(userMessage); } /// @@ -478,7 +478,7 @@ public static void Contains(string substring, string value, StringComparison com { string userMessage = BuildUserMessageForSubstringExpressionAndValueExpression(message, substringExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.ContainsFail, value, substring, userMessage); - ThrowAssertFailed("Assert.Contains", finalMessage); + ReportAssertFailed("Assert.Contains", finalMessage); } } @@ -506,7 +506,7 @@ public static void DoesNotContain(T notExpected, IEnumerable collection, s if (collection.Contains(notExpected)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndCollectionExpression(message, notExpectedExpression, collectionExpression); - ThrowAssertDoesNotContainItemFailed(userMessage); + ReportAssertDoesNotContainItemFailed(userMessage); } } @@ -533,7 +533,7 @@ public static void DoesNotContain(object? notExpected, IEnumerable collection, s if (object.Equals(notExpected, item)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndCollectionExpression(message, notExpectedExpression, collectionExpression); - ThrowAssertDoesNotContainItemFailed(userMessage); + ReportAssertDoesNotContainItemFailed(userMessage); } } } @@ -559,7 +559,7 @@ public static void DoesNotContain(T notExpected, IEnumerable collection, I if (collection.Contains(notExpected, comparer)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndCollectionExpression(message, notExpectedExpression, collectionExpression); - ThrowAssertDoesNotContainItemFailed(userMessage); + ReportAssertDoesNotContainItemFailed(userMessage); } } @@ -588,7 +588,7 @@ public static void DoesNotContain(object? notExpected, IEnumerable collection, I if (comparer.Equals(item, notExpected)) { string userMessage = BuildUserMessageForNotExpectedExpressionAndCollectionExpression(message, notExpectedExpression, collectionExpression); - ThrowAssertDoesNotContainItemFailed(userMessage); + ReportAssertDoesNotContainItemFailed(userMessage); } } } @@ -613,7 +613,7 @@ public static void DoesNotContain(Func predicate, IEnumerable col if (collection.Any(predicate)) { string userMessage = BuildUserMessageForPredicateExpressionAndCollectionExpression(message, predicateExpression, collectionExpression); - ThrowAssertDoesNotContainPredicateFailed(userMessage); + ReportAssertDoesNotContainPredicateFailed(userMessage); } } @@ -641,7 +641,7 @@ public static void DoesNotContain(Func predicate, IEnumerable col if (predicate(item)) { string userMessage = BuildUserMessageForPredicateExpressionAndCollectionExpression(message, predicateExpression, collectionExpression); - ThrowAssertDoesNotContainPredicateFailed(userMessage); + ReportAssertDoesNotContainPredicateFailed(userMessage); } } } @@ -717,7 +717,7 @@ public static void DoesNotContain(string substring, string value, StringComparis { string userMessage = BuildUserMessageForSubstringExpressionAndValueExpression(message, substringExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.DoesNotContainFail, value, substring, userMessage); - ThrowAssertFailed("Assert.DoesNotContain", finalMessage); + ReportAssertFailed("Assert.DoesNotContain", finalMessage); } } @@ -758,71 +758,71 @@ public static void IsInRange(T minValue, T maxValue, T value, string? message { string userMessage = BuildUserMessageForMinValueExpressionAndMaxValueExpressionAndValueExpression(message, minValueExpression, maxValueExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.IsInRangeFail, value, minValue, maxValue, userMessage); - ThrowAssertFailed("IsInRange", finalMessage); + ReportAssertFailed("IsInRange", finalMessage); } } #endregion // IsInRange [DoesNotReturn] - private static void ThrowAssertSingleMatchFailed(int actualCount, string userMessage) + private static void ReportAssertSingleMatchFailed(int actualCount, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.ContainsSingleMatchFailMsg, userMessage, actualCount); - ThrowAssertFailed("Assert.ContainsSingle", finalMessage); + ReportAssertFailed("Assert.ContainsSingle", finalMessage); } [DoesNotReturn] - private static void ThrowAssertContainsSingleFailed(int actualCount, string userMessage) + private static void ReportAssertContainsSingleFailed(int actualCount, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.ContainsSingleFailMsg, userMessage, actualCount); - ThrowAssertFailed("Assert.ContainsSingle", finalMessage); + ReportAssertFailed("Assert.ContainsSingle", finalMessage); } [DoesNotReturn] - private static void ThrowAssertContainsItemFailed(string userMessage) + private static void ReportAssertContainsItemFailed(string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.ContainsItemFailMsg, userMessage); - ThrowAssertFailed("Assert.Contains", finalMessage); + ReportAssertFailed("Assert.Contains", finalMessage); } [DoesNotReturn] - private static void ThrowAssertContainsPredicateFailed(string userMessage) + private static void ReportAssertContainsPredicateFailed(string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.ContainsPredicateFailMsg, userMessage); - ThrowAssertFailed("Assert.Contains", finalMessage); + ReportAssertFailed("Assert.Contains", finalMessage); } [DoesNotReturn] - private static void ThrowAssertDoesNotContainItemFailed(string userMessage) + private static void ReportAssertDoesNotContainItemFailed(string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.DoesNotContainItemFailMsg, userMessage); - ThrowAssertFailed("Assert.DoesNotContain", finalMessage); + ReportAssertFailed("Assert.DoesNotContain", finalMessage); } [DoesNotReturn] - private static void ThrowAssertDoesNotContainPredicateFailed(string userMessage) + private static void ReportAssertDoesNotContainPredicateFailed(string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.DoesNotContainPredicateFailMsg, userMessage); - ThrowAssertFailed("Assert.DoesNotContain", finalMessage); + ReportAssertFailed("Assert.DoesNotContain", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.Count.cs b/src/TestFramework/TestFramework/Assertions/Assert.Count.cs index 555d341686..4ad00d8d02 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.Count.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.Count.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -48,7 +48,7 @@ internal void ComputeAssertion(string assertionName, string collectionExpression if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "collection", collectionExpression) + " "); - ThrowAssertCountFailed(assertionName, _expectedCount, _actualCount, _builder.ToString()); + ReportAssertCountFailed(assertionName, _expectedCount, _actualCount, _builder.ToString()); } } @@ -116,7 +116,7 @@ internal void ComputeAssertion(string collectionExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "collection", collectionExpression) + " "); - ThrowAssertIsNotEmptyFailed(_builder.ToString()); + ReportAssertIsNotEmptyFailed(_builder.ToString()); } } @@ -203,7 +203,7 @@ public static void IsNotEmpty(IEnumerable collection, string? message = "" } string userMessage = BuildUserMessageForCollectionExpression(message, collectionExpression); - ThrowAssertIsNotEmptyFailed(userMessage); + ReportAssertIsNotEmptyFailed(userMessage); } /// @@ -223,7 +223,7 @@ public static void IsNotEmpty(IEnumerable collection, string? message = "", [Cal } string userMessage = BuildUserMessageForCollectionExpression(message, collectionExpression); - ThrowAssertIsNotEmptyFailed(userMessage); + ReportAssertIsNotEmptyFailed(userMessage); } #endregion // IsNotEmpty @@ -327,14 +327,14 @@ private static void HasCount(string assertionName, int expected, IEnumerable< } string userMessage = BuildUserMessageForCollectionExpression(message, collectionExpression); - ThrowAssertCountFailed(assertionName, expected, actualCount, userMessage); + ReportAssertCountFailed(assertionName, expected, actualCount, userMessage); } private static void HasCount(string assertionName, int expected, IEnumerable collection, string? message, string collectionExpression) => HasCount(assertionName, expected, collection.Cast(), message, collectionExpression); [DoesNotReturn] - private static void ThrowAssertCountFailed(string assertionName, int expectedCount, int actualCount, string userMessage) + private static void ReportAssertCountFailed(string assertionName, int expectedCount, int actualCount, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, @@ -342,16 +342,16 @@ private static void ThrowAssertCountFailed(string assertionName, int expectedCou userMessage, expectedCount, actualCount); - ThrowAssertFailed($"Assert.{assertionName}", finalMessage); + ReportAssertFailed($"Assert.{assertionName}", finalMessage); } [DoesNotReturn] - private static void ThrowAssertIsNotEmptyFailed(string userMessage) + private static void ReportAssertIsNotEmptyFailed(string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.IsNotEmptyFailMsg, userMessage); - ThrowAssertFailed("Assert.IsNotEmpty", finalMessage); + ReportAssertFailed("Assert.IsNotEmpty", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.EndsWith.cs b/src/TestFramework/TestFramework/Assertions/Assert.EndsWith.cs index 772c1d1e48..9ebe0a817c 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.EndsWith.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.EndsWith.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. namespace Microsoft.VisualStudio.TestTools.UnitTesting; @@ -78,7 +78,7 @@ public static void EndsWith([NotNull] string? expectedSuffix, [NotNull] string? { string userMessage = BuildUserMessageForExpectedSuffixExpressionAndValueExpression(message, expectedSuffixExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.EndsWithFail, value, expectedSuffix, userMessage); - ThrowAssertFailed("Assert.EndsWith", finalMessage); + ReportAssertFailed("Assert.EndsWith", finalMessage); } } @@ -152,7 +152,7 @@ public static void DoesNotEndWith([NotNull] string? notExpectedSuffix, [NotNull] { string userMessage = BuildUserMessageForNotExpectedSuffixExpressionAndValueExpression(message, notExpectedSuffixExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.DoesNotEndWithFail, value, notExpectedSuffix, userMessage); - ThrowAssertFailed("Assert.DoesNotEndWith", finalMessage); + ReportAssertFailed("Assert.DoesNotEndWith", finalMessage); } } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.Fail.cs b/src/TestFramework/TestFramework/Assertions/Assert.Fail.cs index 3a3a4ebd24..cbe187dc13 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.Fail.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.Fail.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. namespace Microsoft.VisualStudio.TestTools.UnitTesting; diff --git a/src/TestFramework/TestFramework/Assertions/Assert.IComparable.cs b/src/TestFramework/TestFramework/Assertions/Assert.IComparable.cs index a953f5cd33..ba7f415a54 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.IComparable.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.IComparable.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. namespace Microsoft.VisualStudio.TestTools.UnitTesting; @@ -50,7 +50,7 @@ public static void IsGreaterThan(T lowerBound, T value, string? message = "", } string userMessage = BuildUserMessageForLowerBoundExpressionAndValueExpression(message, lowerBoundExpression, valueExpression); - ThrowAssertIsGreaterThanFailed(lowerBound, value, userMessage); + ReportAssertIsGreaterThanFailed(lowerBound, value, userMessage); } #endregion // IsGreaterThan @@ -95,7 +95,7 @@ public static void IsGreaterThanOrEqualTo(T lowerBound, T value, string? mess } string userMessage = BuildUserMessageForLowerBoundExpressionAndValueExpression(message, lowerBoundExpression, valueExpression); - ThrowAssertIsGreaterThanOrEqualToFailed(lowerBound, value, userMessage); + ReportAssertIsGreaterThanOrEqualToFailed(lowerBound, value, userMessage); } #endregion // IsGreaterThanOrEqualTo @@ -140,7 +140,7 @@ public static void IsLessThan(T upperBound, T value, string? message = "", [C } string userMessage = BuildUserMessageForUpperBoundExpressionAndValueExpression(message, upperBoundExpression, valueExpression); - ThrowAssertIsLessThanFailed(upperBound, value, userMessage); + ReportAssertIsLessThanFailed(upperBound, value, userMessage); } #endregion // IsLessThan @@ -185,7 +185,7 @@ public static void IsLessThanOrEqualTo(T upperBound, T value, string? message } string userMessage = BuildUserMessageForUpperBoundExpressionAndValueExpression(message, upperBoundExpression, valueExpression); - ThrowAssertIsLessThanOrEqualToFailed(upperBound, value, userMessage); + ReportAssertIsLessThanOrEqualToFailed(upperBound, value, userMessage); } #endregion // IsLessThanOrEqualTo @@ -222,14 +222,14 @@ public static void IsPositive(T value, string? message = "", [CallerArgumentE if (value is float.NaN) { string userMessage = BuildUserMessageForValueExpression(message, valueExpression); - ThrowAssertIsPositiveFailed(value, userMessage); + ReportAssertIsPositiveFailed(value, userMessage); return; } if (value is double.NaN) { string userMessage = BuildUserMessageForValueExpression(message, valueExpression); - ThrowAssertIsPositiveFailed(value, userMessage); + ReportAssertIsPositiveFailed(value, userMessage); return; } @@ -239,7 +239,7 @@ public static void IsPositive(T value, string? message = "", [CallerArgumentE } string userMessage2 = BuildUserMessageForValueExpression(message, valueExpression); - ThrowAssertIsPositiveFailed(value, userMessage2); + ReportAssertIsPositiveFailed(value, userMessage2); } #endregion // IsPositive @@ -276,14 +276,14 @@ public static void IsNegative(T value, string? message = "", [CallerArgumentE if (value is float.NaN) { string userMessage = BuildUserMessageForValueExpression(message, valueExpression); - ThrowAssertIsNegativeFailed(value, userMessage); + ReportAssertIsNegativeFailed(value, userMessage); return; } if (value is double.NaN) { string userMessage = BuildUserMessageForValueExpression(message, valueExpression); - ThrowAssertIsNegativeFailed(value, userMessage); + ReportAssertIsNegativeFailed(value, userMessage); return; } @@ -293,13 +293,13 @@ public static void IsNegative(T value, string? message = "", [CallerArgumentE } string userMessage2 = BuildUserMessageForValueExpression(message, valueExpression); - ThrowAssertIsNegativeFailed(value, userMessage2); + ReportAssertIsNegativeFailed(value, userMessage2); } #endregion // IsNegative [DoesNotReturn] - private static void ThrowAssertIsGreaterThanFailed(T lowerBound, T value, string userMessage) + private static void ReportAssertIsGreaterThanFailed(T lowerBound, T value, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, @@ -307,11 +307,11 @@ private static void ThrowAssertIsGreaterThanFailed(T lowerBound, T value, str userMessage, ReplaceNulls(lowerBound), ReplaceNulls(value)); - ThrowAssertFailed("Assert.IsGreaterThan", finalMessage); + ReportAssertFailed("Assert.IsGreaterThan", finalMessage); } [DoesNotReturn] - private static void ThrowAssertIsGreaterThanOrEqualToFailed(T lowerBound, T value, string userMessage) + private static void ReportAssertIsGreaterThanOrEqualToFailed(T lowerBound, T value, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, @@ -319,11 +319,11 @@ private static void ThrowAssertIsGreaterThanOrEqualToFailed(T lowerBound, T v userMessage, ReplaceNulls(lowerBound), ReplaceNulls(value)); - ThrowAssertFailed("Assert.IsGreaterThanOrEqualTo", finalMessage); + ReportAssertFailed("Assert.IsGreaterThanOrEqualTo", finalMessage); } [DoesNotReturn] - private static void ThrowAssertIsLessThanFailed(T upperBound, T value, string userMessage) + private static void ReportAssertIsLessThanFailed(T upperBound, T value, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, @@ -331,11 +331,11 @@ private static void ThrowAssertIsLessThanFailed(T upperBound, T value, string userMessage, ReplaceNulls(upperBound), ReplaceNulls(value)); - ThrowAssertFailed("Assert.IsLessThan", finalMessage); + ReportAssertFailed("Assert.IsLessThan", finalMessage); } [DoesNotReturn] - private static void ThrowAssertIsLessThanOrEqualToFailed(T upperBound, T value, string userMessage) + private static void ReportAssertIsLessThanOrEqualToFailed(T upperBound, T value, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, @@ -343,28 +343,28 @@ private static void ThrowAssertIsLessThanOrEqualToFailed(T upperBound, T valu userMessage, ReplaceNulls(upperBound), ReplaceNulls(value)); - ThrowAssertFailed("Assert.IsLessThanOrEqualTo", finalMessage); + ReportAssertFailed("Assert.IsLessThanOrEqualTo", finalMessage); } [DoesNotReturn] - private static void ThrowAssertIsPositiveFailed(T value, string userMessage) + private static void ReportAssertIsPositiveFailed(T value, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.IsPositiveFailMsg, userMessage, ReplaceNulls(value)); - ThrowAssertFailed("Assert.IsPositive", finalMessage); + ReportAssertFailed("Assert.IsPositive", finalMessage); } [DoesNotReturn] - private static void ThrowAssertIsNegativeFailed(T value, string userMessage) + private static void ReportAssertIsNegativeFailed(T value, string userMessage) { string finalMessage = string.Format( CultureInfo.CurrentCulture, FrameworkMessages.IsNegativeFailMsg, userMessage, ReplaceNulls(value)); - ThrowAssertFailed("Assert.IsNegative", finalMessage); + ReportAssertFailed("Assert.IsNegative", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.Inconclusive.cs b/src/TestFramework/TestFramework/Assertions/Assert.Inconclusive.cs index cca665dbb8..6923b71fbf 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.Inconclusive.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.Inconclusive.cs @@ -25,6 +25,6 @@ public static void Inconclusive(string message = "") { string userMessage = BuildUserMessage(message); throw new AssertInconclusiveException( - string.Format(CultureInfo.CurrentCulture, FrameworkMessages.AssertionFailed, "Assert.Inconclusive", userMessage)); + FormatAssertionFailed("Assert.Inconclusive", userMessage)); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.IsExactInstanceOfType.cs b/src/TestFramework/TestFramework/Assertions/Assert.IsExactInstanceOfType.cs index 144f17cdbe..68cca35077 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.IsExactInstanceOfType.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.IsExactInstanceOfType.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -39,7 +39,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsExactInstanceOfTypeFailed(_value, _expectedType, _builder.ToString()); + ReportAssertIsExactInstanceOfTypeFailed(_value, _expectedType, _builder.ToString()); } } @@ -99,7 +99,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsExactInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); + ReportAssertIsExactInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); } } @@ -161,7 +161,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsNotExactInstanceOfTypeFailed(_value, _wrongType, _builder.ToString()); + ReportAssertIsNotExactInstanceOfTypeFailed(_value, _wrongType, _builder.ToString()); } } @@ -221,7 +221,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsNotExactInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); + ReportAssertIsNotExactInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); } } @@ -291,7 +291,7 @@ public static void IsExactInstanceOfType([NotNull] object? value, [NotNull] Type { if (IsExactInstanceOfTypeFailing(value, expectedType)) { - ThrowAssertIsExactInstanceOfTypeFailed(value, expectedType, BuildUserMessageForValueExpression(message, valueExpression)); + ReportAssertIsExactInstanceOfTypeFailed(value, expectedType, BuildUserMessageForValueExpression(message, valueExpression)); } } @@ -299,7 +299,7 @@ public static void IsExactInstanceOfType([NotNull] object? value, [NotNull] Type #pragma warning disable IDE0060 // Remove unused parameter - https://github.com/dotnet/roslyn/issues/76578 public static void IsExactInstanceOfType([NotNull] object? value, [NotNull] Type? expectedType, [InterpolatedStringHandlerArgument(nameof(value), nameof(expectedType))] ref AssertIsExactInstanceOfTypeInterpolatedStringHandler message, [CallerArgumentExpression(nameof(value))] string valueExpression = "") #pragma warning restore IDE0060 // Remove unused parameter -#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Not sure how to express the semantics to the compiler, but the implementation guarantees that. +#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Deliberately keeping [NotNull] annotation while using soft assertions. Within an AssertScope, the postcondition is not enforced (same as all other assertion postconditions in scoped mode). => message.ComputeAssertion(valueExpression); #pragma warning restore CS8777 // Parameter must have a non-null value when exiting. @@ -318,7 +318,7 @@ public static T IsExactInstanceOfType([NotNull] object? value, string? messag #pragma warning disable IDE0060 // Remove unused parameter - https://github.com/dotnet/roslyn/issues/76578 public static T IsExactInstanceOfType([NotNull] object? value, [InterpolatedStringHandlerArgument(nameof(value))] ref AssertGenericIsExactInstanceOfTypeInterpolatedStringHandler message, [CallerArgumentExpression(nameof(value))] string valueExpression = "") #pragma warning restore IDE0060 // Remove unused parameter -#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Not sure how to express the semantics to the compiler, but the implementation guarantees that. +#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Deliberately keeping [NotNull] annotation while using soft assertions. Within an AssertScope, the postcondition is not enforced (same as all other assertion postconditions in scoped mode). { message.ComputeAssertion(valueExpression); return (T)value!; @@ -329,7 +329,7 @@ private static bool IsExactInstanceOfTypeFailing([NotNullWhen(false)] object? va => expectedType is null || value is null || value.GetType() != expectedType; [DoesNotReturn] - private static void ThrowAssertIsExactInstanceOfTypeFailed(object? value, Type? expectedType, string userMessage) + private static void ReportAssertIsExactInstanceOfTypeFailed(object? value, Type? expectedType, string userMessage) { string finalMessage = userMessage; if (expectedType is not null && value is not null) @@ -342,7 +342,7 @@ private static void ThrowAssertIsExactInstanceOfTypeFailed(object? value, Type? value.GetType().ToString()); } - ThrowAssertFailed("Assert.IsExactInstanceOfType", finalMessage); + ReportAssertFailed("Assert.IsExactInstanceOfType", finalMessage); } /// @@ -373,7 +373,7 @@ public static void IsNotExactInstanceOfType(object? value, [NotNull] Type? wrong { if (IsNotExactInstanceOfTypeFailing(value, wrongType)) { - ThrowAssertIsNotExactInstanceOfTypeFailed(value, wrongType, BuildUserMessageForValueExpression(message, valueExpression)); + ReportAssertIsNotExactInstanceOfTypeFailed(value, wrongType, BuildUserMessageForValueExpression(message, valueExpression)); } } @@ -405,7 +405,7 @@ private static bool IsNotExactInstanceOfTypeFailing(object? value, [NotNullWhen( (value is not null && value.GetType() == wrongType); [DoesNotReturn] - private static void ThrowAssertIsNotExactInstanceOfTypeFailed(object? value, Type? wrongType, string userMessage) + private static void ReportAssertIsNotExactInstanceOfTypeFailed(object? value, Type? wrongType, string userMessage) { string finalMessage = userMessage; if (wrongType is not null) @@ -418,6 +418,6 @@ private static void ThrowAssertIsNotExactInstanceOfTypeFailed(object? value, Typ value!.GetType().ToString()); } - ThrowAssertFailed("Assert.IsNotExactInstanceOfType", finalMessage); + ReportAssertFailed("Assert.IsNotExactInstanceOfType", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.IsInstanceOfType.cs b/src/TestFramework/TestFramework/Assertions/Assert.IsInstanceOfType.cs index bdd68f1607..553a808132 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.IsInstanceOfType.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.IsInstanceOfType.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -39,7 +39,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsInstanceOfTypeFailed(_value, _expectedType, _builder.ToString()); + ReportAssertIsInstanceOfTypeFailed(_value, _expectedType, _builder.ToString()); } } @@ -99,7 +99,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); + ReportAssertIsInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); } } @@ -161,7 +161,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsNotInstanceOfTypeFailed(_value, _wrongType, _builder.ToString()); + ReportAssertIsNotInstanceOfTypeFailed(_value, _wrongType, _builder.ToString()); } } @@ -221,7 +221,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsNotInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); + ReportAssertIsNotInstanceOfTypeFailed(_value, typeof(TArg), _builder.ToString()); } } @@ -292,7 +292,7 @@ public static void IsInstanceOfType([NotNull] object? value, [NotNull] Type? exp { if (IsInstanceOfTypeFailing(value, expectedType)) { - ThrowAssertIsInstanceOfTypeFailed(value, expectedType, BuildUserMessageForValueExpression(message, valueExpression)); + ReportAssertIsInstanceOfTypeFailed(value, expectedType, BuildUserMessageForValueExpression(message, valueExpression)); } } @@ -300,7 +300,7 @@ public static void IsInstanceOfType([NotNull] object? value, [NotNull] Type? exp #pragma warning disable IDE0060 // Remove unused parameter - https://github.com/dotnet/roslyn/issues/76578 public static void IsInstanceOfType([NotNull] object? value, [NotNull] Type? expectedType, [InterpolatedStringHandlerArgument(nameof(value), nameof(expectedType))] ref AssertIsInstanceOfTypeInterpolatedStringHandler message, [CallerArgumentExpression(nameof(value))] string valueExpression = "") #pragma warning restore IDE0060 // Remove unused parameter -#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Not sure how to express the semantics to the compiler, but the implementation guarantees that. +#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Deliberately keeping [NotNull] annotation while using soft assertions. Within an AssertScope, the postcondition is not enforced (same as all other assertion postconditions in scoped mode). => message.ComputeAssertion(valueExpression); #pragma warning restore CS8777 // Parameter must have a non-null value when exiting. @@ -320,7 +320,7 @@ public static T IsInstanceOfType([NotNull] object? value, string? message = " #pragma warning disable IDE0060 // Remove unused parameter - https://github.com/dotnet/roslyn/issues/76578 public static T IsInstanceOfType([NotNull] object? value, [InterpolatedStringHandlerArgument(nameof(value))] ref AssertGenericIsInstanceOfTypeInterpolatedStringHandler message, [CallerArgumentExpression(nameof(value))] string valueExpression = "") #pragma warning restore IDE0060 // Remove unused parameter -#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Not sure how to express the semantics to the compiler, but the implementation guarantees that. +#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Deliberately keeping [NotNull] annotation while using soft assertions. Within an AssertScope, the postcondition is not enforced (same as all other assertion postconditions in scoped mode). { message.ComputeAssertion(valueExpression); return (T)value!; @@ -331,7 +331,7 @@ private static bool IsInstanceOfTypeFailing([NotNullWhen(false)] object? value, => expectedType == null || value == null || !expectedType.IsInstanceOfType(value); [DoesNotReturn] - private static void ThrowAssertIsInstanceOfTypeFailed(object? value, Type? expectedType, string userMessage) + private static void ReportAssertIsInstanceOfTypeFailed(object? value, Type? expectedType, string userMessage) { string finalMessage = userMessage; if (expectedType is not null && value is not null) @@ -344,7 +344,7 @@ private static void ThrowAssertIsInstanceOfTypeFailed(object? value, Type? expec value.GetType().ToString()); } - ThrowAssertFailed("Assert.IsInstanceOfType", finalMessage); + ReportAssertFailed("Assert.IsInstanceOfType", finalMessage); } /// @@ -376,7 +376,7 @@ public static void IsNotInstanceOfType(object? value, [NotNull] Type? wrongType, { if (IsNotInstanceOfTypeFailing(value, wrongType)) { - ThrowAssertIsNotInstanceOfTypeFailed(value, wrongType, BuildUserMessageForValueExpression(message, valueExpression)); + ReportAssertIsNotInstanceOfTypeFailed(value, wrongType, BuildUserMessageForValueExpression(message, valueExpression)); } } @@ -409,7 +409,7 @@ private static bool IsNotInstanceOfTypeFailing(object? value, [NotNullWhen(false (value is not null && wrongType.IsInstanceOfType(value)); [DoesNotReturn] - private static void ThrowAssertIsNotInstanceOfTypeFailed(object? value, Type? wrongType, string userMessage) + private static void ReportAssertIsNotInstanceOfTypeFailed(object? value, Type? wrongType, string userMessage) { string finalMessage = userMessage; if (wrongType is not null) @@ -422,6 +422,6 @@ private static void ThrowAssertIsNotInstanceOfTypeFailed(object? value, Type? wr value!.GetType().ToString()); } - ThrowAssertFailed("Assert.IsNotInstanceOfType", finalMessage); + ReportAssertFailed("Assert.IsNotInstanceOfType", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.IsNull.cs b/src/TestFramework/TestFramework/Assertions/Assert.IsNull.cs index fd6db5b3d4..00b3e1eec6 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.IsNull.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.IsNull.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -37,7 +37,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsNullFailed(_builder.ToString()); + ReportAssertIsNullFailed(_builder.ToString()); } } @@ -91,7 +91,7 @@ internal void ComputeAssertion(string valueExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "value", valueExpression) + " "); - ThrowAssertIsNotNullFailed(_builder.ToString()); + ReportAssertIsNotNullFailed(_builder.ToString()); } } @@ -152,20 +152,20 @@ public static void IsNull(object? value, string? message = "", [CallerArgumentEx { if (IsNullFailing(value)) { - ThrowAssertIsNullFailed(BuildUserMessageForValueExpression(message, valueExpression)); + ReportAssertIsNullFailed(BuildUserMessageForValueExpression(message, valueExpression)); } } private static bool IsNullFailing(object? value) => value is not null; - private static void ThrowAssertIsNullFailed(string? message) - => ThrowAssertFailed("Assert.IsNull", message); + private static void ReportAssertIsNullFailed(string? message) + => ReportAssertFailed("Assert.IsNull", message); /// #pragma warning disable IDE0060 // Remove unused parameter - https://github.com/dotnet/roslyn/issues/76578 public static void IsNotNull([NotNull] object? value, [InterpolatedStringHandlerArgument(nameof(value))] ref AssertIsNotNullInterpolatedStringHandler message, [CallerArgumentExpression(nameof(value))] string valueExpression = "") #pragma warning restore IDE0060 // Remove unused parameter -#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Not sure how to express the semantics to the compiler, but the implementation guarantees that. +#pragma warning disable CS8777 // Parameter must have a non-null value when exiting. - Deliberately keeping [NotNull] annotation while using soft assertions. Within an AssertScope, the postcondition is not enforced (same as all other assertion postconditions in scoped mode). => message.ComputeAssertion(valueExpression); #pragma warning restore CS8777 // Parameter must have a non-null value when exiting. @@ -191,13 +191,13 @@ public static void IsNotNull([NotNull] object? value, string? message = "", [Cal { if (IsNotNullFailing(value)) { - ThrowAssertIsNotNullFailed(BuildUserMessageForValueExpression(message, valueExpression)); + ReportAssertIsNotNullFailed(BuildUserMessageForValueExpression(message, valueExpression)); } } private static bool IsNotNullFailing([NotNullWhen(false)] object? value) => value is null; [DoesNotReturn] - private static void ThrowAssertIsNotNullFailed(string? message) - => ThrowAssertFailed("Assert.IsNotNull", message); + private static void ReportAssertIsNotNullFailed(string? message) + => ReportAssertFailed("Assert.IsNotNull", message); } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs b/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs index 1c31f3add1..d1480fe701 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -37,7 +37,7 @@ internal void ComputeAssertion(string conditionExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "condition", conditionExpression) + " "); - ThrowAssertIsTrueFailed(_builder.ToString()); + ReportAssertIsTrueFailed(_builder.ToString()); } } @@ -89,7 +89,7 @@ internal void ComputeAssertion(string conditionExpression) if (_builder is not null) { _builder.Insert(0, string.Format(CultureInfo.CurrentCulture, FrameworkMessages.CallerArgumentExpressionSingleParameterMessage, "condition", conditionExpression) + " "); - ThrowAssertIsFalseFailed(_builder.ToString()); + ReportAssertIsFalseFailed(_builder.ToString()); } } @@ -150,15 +150,15 @@ public static void IsTrue([DoesNotReturnIf(false)] bool? condition, string? mess { if (IsTrueFailing(condition)) { - ThrowAssertIsTrueFailed(BuildUserMessageForConditionExpression(message, conditionExpression)); + ReportAssertIsTrueFailed(BuildUserMessageForConditionExpression(message, conditionExpression)); } } private static bool IsTrueFailing(bool? condition) => condition is false or null; - private static void ThrowAssertIsTrueFailed(string? message) - => ThrowAssertFailed("Assert.IsTrue", message); + private static void ReportAssertIsTrueFailed(string? message) + => ReportAssertFailed("Assert.IsTrue", message); /// #pragma warning disable IDE0060 // Remove unused parameter - https://github.com/dotnet/roslyn/issues/76578 @@ -188,7 +188,7 @@ public static void IsFalse([DoesNotReturnIf(true)] bool? condition, string? mess { if (IsFalseFailing(condition)) { - ThrowAssertIsFalseFailed(BuildUserMessageForConditionExpression(message, conditionExpression)); + ReportAssertIsFalseFailed(BuildUserMessageForConditionExpression(message, conditionExpression)); } } @@ -196,6 +196,6 @@ private static bool IsFalseFailing(bool? condition) => condition is true or null; [DoesNotReturn] - private static void ThrowAssertIsFalseFailed(string userMessage) - => ThrowAssertFailed("Assert.IsFalse", userMessage); + private static void ReportAssertIsFalseFailed(string userMessage) + => ReportAssertFailed("Assert.IsFalse", userMessage); } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.Matches.cs b/src/TestFramework/TestFramework/Assertions/Assert.Matches.cs index b204970b6d..23b69859c7 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.Matches.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.Matches.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. namespace Microsoft.VisualStudio.TestTools.UnitTesting; @@ -46,7 +46,7 @@ public static void MatchesRegex([NotNull] Regex? pattern, [NotNull] string? valu { string userMessage = BuildUserMessageForPatternExpressionAndValueExpression(message, patternExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.IsMatchFail, value, pattern, userMessage); - ThrowAssertFailed("Assert.MatchesRegex", finalMessage); + ReportAssertFailed("Assert.MatchesRegex", finalMessage); } } @@ -122,7 +122,7 @@ public static void DoesNotMatchRegex([NotNull] Regex? pattern, [NotNull] string? { string userMessage = BuildUserMessageForPatternExpressionAndValueExpression(message, patternExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.IsNotMatchFail, value, pattern, userMessage); - ThrowAssertFailed("Assert.DoesNotMatchRegex", finalMessage); + ReportAssertFailed("Assert.DoesNotMatchRegex", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.Scope.cs b/src/TestFramework/TestFramework/Assertions/Assert.Scope.cs new file mode 100644 index 0000000000..a46fad8ad3 --- /dev/null +++ b/src/TestFramework/TestFramework/Assertions/Assert.Scope.cs @@ -0,0 +1,30 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +namespace Microsoft.VisualStudio.TestTools.UnitTesting; + +/// +/// A collection of helper classes to test various conditions within +/// unit tests. If the condition being tested is not met, an exception +/// is thrown. +/// +public sealed partial class Assert +{ + /// + /// Creates a new assertion scope that collects assertion failures instead of throwing them immediately. + /// When the returned scope is disposed, all collected failures are thrown as a single . + /// + /// An representing the assertion scope. + /// + /// + /// using (Assert.Scope()) + /// { + /// Assert.AreEqual(1, 2); // collected, not thrown + /// Assert.IsTrue(false); // collected, not thrown + /// } + /// // AssertFailedException is thrown here with all collected failures. + /// + /// + [Experimental("MSTESTEXP", UrlFormat = "https://aka.ms/mstest/diagnostics#{0}")] + public static IDisposable Scope() => new AssertScope(); +} diff --git a/src/TestFramework/TestFramework/Assertions/Assert.StartsWith.cs b/src/TestFramework/TestFramework/Assertions/Assert.StartsWith.cs index 6f5c728535..caf58c2a10 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.StartsWith.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.StartsWith.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. namespace Microsoft.VisualStudio.TestTools.UnitTesting; @@ -78,7 +78,7 @@ public static void StartsWith([NotNull] string? expectedPrefix, [NotNull] string { string userMessage = BuildUserMessageForExpectedPrefixExpressionAndValueExpression(message, expectedPrefixExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.StartsWithFail, value, expectedPrefix, userMessage); - ThrowAssertFailed("Assert.StartsWith", finalMessage); + ReportAssertFailed("Assert.StartsWith", finalMessage); } } @@ -150,7 +150,7 @@ public static void DoesNotStartWith([NotNull] string? notExpectedPrefix, [NotNul { string userMessage = BuildUserMessageForNotExpectedPrefixExpressionAndValueExpression(message, notExpectedPrefixExpression, valueExpression); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.DoesNotStartWithFail, value, notExpectedPrefix, userMessage); - ThrowAssertFailed("Assert.DoesNotStartWith", finalMessage); + ReportAssertFailed("Assert.DoesNotStartWith", finalMessage); } } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.That.cs b/src/TestFramework/TestFramework/Assertions/Assert.That.cs index e13c803967..0399e6f27f 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.That.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.That.cs @@ -39,20 +39,25 @@ public static void That(Expression> condition, string? message = null var sb = new StringBuilder(); string expressionText = conditionExpression ?? throw new ArgumentNullException(nameof(conditionExpression)); - sb.AppendLine(string.Format(CultureInfo.InvariantCulture, FrameworkMessages.AssertThatFailedFormat, expressionText)); if (!string.IsNullOrWhiteSpace(message)) { + sb.AppendLine(); sb.AppendLine(string.Format(CultureInfo.InvariantCulture, FrameworkMessages.AssertThatMessageFormat, message)); } string details = ExtractDetails(condition.Body); if (!string.IsNullOrWhiteSpace(details)) { + if (sb.Length == 0) + { + sb.AppendLine(); + } + sb.AppendLine(FrameworkMessages.AssertThatDetailsPrefix); sb.AppendLine(details); } - throw new AssertFailedException(sb.ToString().TrimEnd()); + Assert.ReportAssertFailed($"Assert.That({expressionText})", sb.ToString().TrimEnd()); } } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.ThrowsException.cs b/src/TestFramework/TestFramework/Assertions/Assert.ThrowsException.cs index b5478a28a6..5ef86cd9ad 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.ThrowsException.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.ThrowsException.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. using System.ComponentModel; @@ -538,7 +538,7 @@ private static async Task IsThrowsAsyncFailingAsync IsThrowsAsyncFailingAsync(Action action, b userMessage, typeof(TException), ex.GetType()); - ThrowAssertFailed("Assert." + assertMethodName, finalMessage); + ReportAssertFailed("Assert." + assertMethodName, finalMessage); }, ex); } @@ -590,7 +590,7 @@ private static ThrowsExceptionState IsThrowsFailing(Action action, b FrameworkMessages.NoExceptionThrown, userMessage, typeof(TException)); - ThrowAssertFailed("Assert." + assertMethodName, finalMessage); + ReportAssertFailed("Assert." + assertMethodName, finalMessage); }, null); } diff --git a/src/TestFramework/TestFramework/Assertions/Assert.cs b/src/TestFramework/TestFramework/Assertions/Assert.cs index 4a832186af..0703444ae6 100644 --- a/src/TestFramework/TestFramework/Assertions/Assert.cs +++ b/src/TestFramework/TestFramework/Assertions/Assert.cs @@ -26,7 +26,7 @@ private Assert() public static Assert That { get; } = new(); /// - /// Helper function that creates and throws an AssertionFailedException. + /// Reports an assertion failure and always throws, even within an . /// /// /// name of the assertion throwing an exception. @@ -37,6 +37,50 @@ private Assert() [DoesNotReturn] [StackTraceHidden] internal static void ThrowAssertFailed(string assertionName, string? message) + { + LaunchDebuggerIfNeeded(); + throw CreateAssertFailedException(assertionName, message); + } + + /// + /// Reports an assertion failure. Within an , the failure is collected + /// and execution continues. Outside a scope, the failure is thrown immediately. + /// + /// + /// name of the assertion throwing an exception. + /// + /// + /// The assertion failure message. + /// +#pragma warning disable CS8763 // A method marked [DoesNotReturn] should not return - Deliberately keeping [DoesNotReturn] annotation while using soft assertions. Within an AssertScope, the postcondition is not enforced (same as all other assertion postconditions in scoped mode). + [DoesNotReturn] + [StackTraceHidden] + internal static void ReportAssertFailed(string assertionName, string? message) + { + LaunchDebuggerIfNeeded(); + AssertFailedException assertionFailedException = CreateAssertFailedException(assertionName, message); + if (AssertScope.Current is { } scope) + { + // Throw and catch to capture the stack trace at the point of failure, + // so the exception has a meaningful stack trace when reported from the scope. + try + { + throw assertionFailedException; + } + catch (AssertFailedException ex) + { + assertionFailedException = ex; + } + + scope.AddError(assertionFailedException); + return; + } + + throw assertionFailedException; + } +#pragma warning restore CS8763 // A method marked [DoesNotReturn] should not return + + private static void LaunchDebuggerIfNeeded() { if (ShouldLaunchDebugger()) { @@ -50,17 +94,28 @@ internal static void ThrowAssertFailed(string assertionName, string? message) } } - throw new AssertFailedException( - string.Format(CultureInfo.CurrentCulture, FrameworkMessages.AssertionFailed, assertionName, message)); + // Local functions + static bool ShouldLaunchDebugger() + => AssertionFailureSettings.LaunchDebuggerOnAssertionFailure switch + { + DebuggerLaunchMode.Enabled => true, + DebuggerLaunchMode.EnabledExcludingCI => !CIEnvironmentDetector.Instance.IsCIEnvironment(), + _ => false, + }; } - private static bool ShouldLaunchDebugger() - => AssertionFailureSettings.LaunchDebuggerOnAssertionFailure switch - { - DebuggerLaunchMode.Enabled => true, - DebuggerLaunchMode.EnabledExcludingCI => !CIEnvironmentDetector.Instance.IsCIEnvironment(), - _ => false, - }; + private static AssertFailedException CreateAssertFailedException(string assertionName, string? message) + => new(FormatAssertionFailed(assertionName, message)); + + private static string FormatAssertionFailed(string assertionName, string? message) + { + string failedMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.AssertionFailed, assertionName); + return string.IsNullOrWhiteSpace(message) + ? failedMessage + : message![0] is '\n' or '\r' + ? string.Concat(failedMessage, message) + : $"{failedMessage} {message}"; + } /// /// Builds the formatted message using the given user format message and parameters. @@ -184,10 +239,10 @@ private static string BuildUserMessageForMinValueExpressionAndMaxValueExpression /// internal static void CheckParameterNotNull([NotNull] object? param, string assertionName, string parameterName) { - if (param == null) + if (param is null) { string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.NullParameterToAssert, parameterName); - ThrowAssertFailed(assertionName, finalMessage); + throw CreateAssertFailedException(assertionName, finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/AssertScope.cs b/src/TestFramework/TestFramework/Assertions/AssertScope.cs new file mode 100644 index 0000000000..cef4dbd0a0 --- /dev/null +++ b/src/TestFramework/TestFramework/Assertions/AssertScope.cs @@ -0,0 +1,82 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Runtime.ExceptionServices; + +namespace Microsoft.VisualStudio.TestTools.UnitTesting; + +/// +/// Represents a scope in which assertion failures are collected instead of thrown immediately. +/// When the scope is disposed, all collected failures are thrown as a single . +/// +internal sealed class AssertScope : IDisposable +{ + private static readonly AsyncLocal CurrentScope = new(); + + private readonly ConcurrentQueue _errors = new(); + private bool _disposed; + + internal AssertScope() + { + if (CurrentScope.Value is not null) + { + throw new InvalidOperationException(FrameworkMessages.AssertScopeNestedNotAllowed); + } + + CurrentScope.Value = this; + } + + /// + /// Gets the current active , or if no scope is active. + /// + internal static AssertScope? Current => CurrentScope.Value; + + /// + /// Adds an assertion failure message to the current scope. + /// + /// The assertion failure message. + internal void AddError(AssertFailedException error) + { +#pragma warning disable CA1513 // Use ObjectDisposedException throw helper - ThrowIf is not available on all target frameworks + if (_disposed) + { + throw new ObjectDisposedException(nameof(AssertScope)); + } +#pragma warning restore CA1513 // Use ObjectDisposedException throw helper + + _errors.Enqueue(error); + } + + /// + public void Dispose() + { + if (_disposed) + { + return; + } + + _disposed = true; + CurrentScope.Value = null; + + // We throw the collected exceptions directly instead of going through assertion failure + // helpers (e.g. ThrowAssertFailed) because the debugger was already launched when each + // error was collected. + // Snapshot the ConcurrentQueue into an array to avoid multiple O(n) enumerations + // (ConcurrentQueue.Count is O(n)) and to get a consistent view for branching, + // message formatting, and building the AggregateException. + AssertFailedException[] errorsSnapshot = _errors.ToArray(); + if (errorsSnapshot.Length == 1) + { + // Use ExceptionDispatchInfo to preserve the original stack trace captured at the + // assertion call site, rather than resetting it to point at Dispose. + ExceptionDispatchInfo.Capture(errorsSnapshot[0]).Throw(); + } + + if (errorsSnapshot.Length > 0) + { + throw new AssertFailedException( + string.Format(CultureInfo.CurrentCulture, FrameworkMessages.AssertScopeFailure, errorsSnapshot.Length), + new AggregateException(errorsSnapshot)); + } + } +} diff --git a/src/TestFramework/TestFramework/Assertions/CollectionAssert.cs b/src/TestFramework/TestFramework/Assertions/CollectionAssert.cs index 6be196fa94..11b92190ca 100644 --- a/src/TestFramework/TestFramework/Assertions/CollectionAssert.cs +++ b/src/TestFramework/TestFramework/Assertions/CollectionAssert.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. namespace Microsoft.VisualStudio.TestTools.UnitTesting; @@ -79,7 +79,7 @@ public static void Contains([NotNull] ICollection? collection, object? element, } } - Assert.ThrowAssertFailed("CollectionAssert.Contains", Assert.BuildUserMessage(message)); + Assert.ReportAssertFailed("CollectionAssert.Contains", Assert.BuildUserMessage(message)); } /// @@ -126,7 +126,7 @@ public static void DoesNotContain([NotNull] ICollection? collection, object? ele { if (object.Equals(current, element)) { - Assert.ThrowAssertFailed("CollectionAssert.DoesNotContain", Assert.BuildUserMessage(message)); + Assert.ReportAssertFailed("CollectionAssert.DoesNotContain", Assert.BuildUserMessage(message)); } } } @@ -165,7 +165,7 @@ public static void AllItemsAreNotNull([NotNull] ICollection? collection, string? { if (current == null) { - Assert.ThrowAssertFailed("CollectionAssert.AllItemsAreNotNull", Assert.BuildUserMessage(message)); + Assert.ReportAssertFailed("CollectionAssert.AllItemsAreNotNull", Assert.BuildUserMessage(message)); } } } @@ -226,7 +226,7 @@ public static void AllItemsAreUnique([NotNull] ICollection? collection, string? userMessage, FrameworkMessages.Common_NullInMessages); - Assert.ThrowAssertFailed("CollectionAssert.AllItemsAreUnique", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AllItemsAreUnique", finalMessage); } } else @@ -240,7 +240,7 @@ public static void AllItemsAreUnique([NotNull] ICollection? collection, string? userMessage, Assert.ReplaceNulls(current)); - Assert.ThrowAssertFailed("CollectionAssert.AllItemsAreUnique", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AllItemsAreUnique", finalMessage); } } } @@ -303,11 +303,11 @@ public static void IsSubsetOf([NotNull] ICollection? subset, [NotNull] ICollecti string userMessage = Assert.BuildUserMessage(message); if (string.IsNullOrEmpty(userMessage)) { - Assert.ThrowAssertFailed("CollectionAssert.IsSubsetOf", returnedSubsetValueMessage); + Assert.ReportAssertFailed("CollectionAssert.IsSubsetOf", returnedSubsetValueMessage); } else { - Assert.ThrowAssertFailed("CollectionAssert.IsSubsetOf", $"{returnedSubsetValueMessage} {userMessage}"); + Assert.ReportAssertFailed("CollectionAssert.IsSubsetOf", $"{returnedSubsetValueMessage} {userMessage}"); } } } @@ -357,7 +357,7 @@ public static void IsNotSubsetOf([NotNull] ICollection? subset, [NotNull] IColle Tuple> isSubsetValue = IsSubsetOfHelper(subset, superset); if (isSubsetValue.Item1) { - Assert.ThrowAssertFailed("CollectionAssert.IsNotSubsetOf", Assert.BuildUserMessage(message)); + Assert.ReportAssertFailed("CollectionAssert.IsNotSubsetOf", Assert.BuildUserMessage(message)); } } @@ -476,7 +476,7 @@ public static void AreEquivalent( // Check whether one is null while the other is not. if (expected == null != (actual == null)) { - Assert.ThrowAssertFailed("CollectionAssert.AreEquivalent", Assert.BuildUserMessage(message)); + Assert.ReportAssertFailed("CollectionAssert.AreEquivalent", Assert.BuildUserMessage(message)); } // If the references are the same or both collections are null, they are equivalent. @@ -500,7 +500,7 @@ public static void AreEquivalent( userMessage, expectedCollectionCount, actualCollectionCount); - Assert.ThrowAssertFailed("CollectionAssert.AreEquivalent", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreEquivalent", finalMessage); } // If both collections are empty, they are equivalent. @@ -520,7 +520,7 @@ public static void AreEquivalent( expectedCount.ToString(CultureInfo.CurrentCulture.NumberFormat), Assert.ReplaceNulls(mismatchedElement), actualCount.ToString(CultureInfo.CurrentCulture.NumberFormat)); - Assert.ThrowAssertFailed("CollectionAssert.AreEquivalent", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreEquivalent", finalMessage); } // All the elements and counts matched. @@ -654,7 +654,7 @@ public static void AreNotEquivalent( CultureInfo.CurrentCulture, FrameworkMessages.BothCollectionsSameReference, userMessage); - Assert.ThrowAssertFailed("CollectionAssert.AreNotEquivalent", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreNotEquivalent", finalMessage); } DebugEx.Assert(actual is not null, "actual is not null here"); @@ -674,7 +674,7 @@ public static void AreNotEquivalent( CultureInfo.CurrentCulture, FrameworkMessages.BothCollectionsEmpty, userMessage); - Assert.ThrowAssertFailed("CollectionAssert.AreNotEquivalent", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreNotEquivalent", finalMessage); } // Search for a mismatched element. @@ -685,7 +685,7 @@ public static void AreNotEquivalent( CultureInfo.CurrentCulture, FrameworkMessages.BothSameElements, userMessage); - Assert.ThrowAssertFailed("CollectionAssert.AreNotEquivalent", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreNotEquivalent", finalMessage); } } @@ -755,7 +755,7 @@ public static void AllItemsAreInstancesOfType( i, expectedType.ToString(), element.GetType().ToString()); - Assert.ThrowAssertFailed("CollectionAssert.AllItemsAreInstancesOfType", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AllItemsAreInstancesOfType", finalMessage); } i++; @@ -816,7 +816,7 @@ public static void AreEqual(ICollection? expected, ICollection? actual, string? if (!AreCollectionsEqual(expected, actual, new ObjectComparer(), ref reason)) { string finalMessage = ConstructFinalMessage(reason, message); - Assert.ThrowAssertFailed("CollectionAssert.AreEqual", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreEqual", finalMessage); } } @@ -870,7 +870,7 @@ public static void AreNotEqual(ICollection? notExpected, ICollection? actual, st if (AreCollectionsEqual(notExpected, actual, new ObjectComparer(), ref reason)) { string finalMessage = ConstructFinalMessage(reason, message); - Assert.ThrowAssertFailed("CollectionAssert.AreNotEqual", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreNotEqual", finalMessage); } } @@ -928,7 +928,7 @@ public static void AreEqual(ICollection? expected, ICollection? actual, [NotNull if (!AreCollectionsEqual(expected, actual, comparer, ref reason)) { string finalMessage = ConstructFinalMessage(reason, message); - Assert.ThrowAssertFailed("CollectionAssert.AreEqual", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreEqual", finalMessage); } } @@ -986,7 +986,7 @@ public static void AreNotEqual(ICollection? notExpected, ICollection? actual, [N if (AreCollectionsEqual(notExpected, actual, comparer, ref reason)) { string finalMessage = ConstructFinalMessage(reason, message); - Assert.ThrowAssertFailed("CollectionAssert.AreNotEqual", finalMessage); + Assert.ReportAssertFailed("CollectionAssert.AreNotEqual", finalMessage); } } diff --git a/src/TestFramework/TestFramework/Assertions/StringAssert.cs b/src/TestFramework/TestFramework/Assertions/StringAssert.cs index b2e36625ec..3c65a1ed62 100644 --- a/src/TestFramework/TestFramework/Assertions/StringAssert.cs +++ b/src/TestFramework/TestFramework/Assertions/StringAssert.cs @@ -1,4 +1,4 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. +// Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. namespace Microsoft.VisualStudio.TestTools.UnitTesting; @@ -122,7 +122,7 @@ public static void Contains([NotNull] string? value, [NotNull] string? substring { string userMessage = Assert.BuildUserMessage(message); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.ContainsFail, value, substring, userMessage); - Assert.ThrowAssertFailed("StringAssert.Contains", finalMessage); + Assert.ReportAssertFailed("StringAssert.Contains", finalMessage); } } @@ -219,7 +219,7 @@ public static void StartsWith([NotNull] string? value, [NotNull] string? substri { string userMessage = Assert.BuildUserMessage(message); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.StartsWithFail, value, substring, userMessage); - Assert.ThrowAssertFailed("StringAssert.StartsWith", finalMessage); + Assert.ReportAssertFailed("StringAssert.StartsWith", finalMessage); } } @@ -316,7 +316,7 @@ public static void EndsWith([NotNull] string? value, [NotNull] string? substring { string userMessage = Assert.BuildUserMessage(message); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.EndsWithFail, value, substring, userMessage); - Assert.ThrowAssertFailed("StringAssert.EndsWith", finalMessage); + Assert.ReportAssertFailed("StringAssert.EndsWith", finalMessage); } } @@ -371,7 +371,7 @@ public static void Matches([NotNull] string? value, [NotNull] Regex? pattern, st { string userMessage = Assert.BuildUserMessage(message); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.IsMatchFail, value, pattern, userMessage); - Assert.ThrowAssertFailed("StringAssert.Matches", finalMessage); + Assert.ReportAssertFailed("StringAssert.Matches", finalMessage); } } @@ -422,7 +422,7 @@ public static void DoesNotMatch([NotNull] string? value, [NotNull] Regex? patter { string userMessage = Assert.BuildUserMessage(message); string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.IsNotMatchFail, value, pattern, userMessage); - Assert.ThrowAssertFailed("StringAssert.DoesNotMatch", finalMessage); + Assert.ReportAssertFailed("StringAssert.DoesNotMatch", finalMessage); } } diff --git a/src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt b/src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt index 7dc5c58110..ee81f202d3 100644 --- a/src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt +++ b/src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt @@ -1 +1,2 @@ #nullable enable +[MSTESTEXP]static Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Scope() -> System.IDisposable! diff --git a/src/TestFramework/TestFramework/Resources/FrameworkMessages.resx b/src/TestFramework/TestFramework/Resources/FrameworkMessages.resx index 537b9331a7..261db6e409 100644 --- a/src/TestFramework/TestFramework/Resources/FrameworkMessages.resx +++ b/src/TestFramework/TestFramework/Resources/FrameworkMessages.resx @@ -201,7 +201,7 @@ Actual: {2} String '{0}' does not end with string '{1}'. {2} - {0} failed. {1} + {0} failed. {0} Expected type:<{1}>. Actual type:<{2}>. @@ -368,10 +368,6 @@ Actual: {2} '{0}' expression: '{1}'. Example: "'value' expression: 'new object()'", where 'value' is the parameter name of an assertion method, and 'new object()' is the expression the user passed to the assert method. - - Assert.That({0}) failed. - {0} is the user code expression - Message: {0} {0} user provided message @@ -393,4 +389,11 @@ Actual: {2} The maximum value must be greater than or equal to the minimum value. + + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + \ No newline at end of file diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.cs.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.cs.xlf index 02a8e5a3bc..93ec92327d 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.cs.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.cs.xlf @@ -83,16 +83,21 @@ Nevkládejte hodnotu typů do AreSame(). Hodnoty převedené do typu Object už nebudou nikdy stejné. Zvažte použití AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Podrobnosti: - - Assert.That({0}) failed. - Assert.That({0}) se nezdařilo. - {0} is the user code expression - Message: {0} Zpráva: {0} @@ -233,9 +238,9 @@ Skutečnost: {2} - {0} failed. {1} - {0} selhalo. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.de.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.de.xlf index b240c09df6..32fd458027 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.de.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.de.xlf @@ -83,16 +83,21 @@ Übergeben Sie keine Werttypen an AreSame(). In ein Objekt konvertierte Werte sind niemals identisch. Verwenden Sie stattdessen AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Details: - - Assert.That({0}) failed. - Assert.That({0}) fehlgeschlagen. - {0} is the user code expression - Message: {0} Meldung: {0} @@ -233,9 +238,9 @@ Tatsächlich: {2} - {0} failed. {1} - Fehler bei "{0}". {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.es.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.es.xlf index 427b030537..2353d57da4 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.es.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.es.xlf @@ -83,16 +83,21 @@ No pase tipos de valor a AreSame(). Los valores convertidos a Object no serán nunca iguales. Considere el uso de AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Detalles: - - Assert.That({0}) failed. - Error de Assert.That({0}). - {0} is the user code expression - Message: {0} Mensaje: {0} @@ -233,9 +238,9 @@ Real: {2} - {0} failed. {1} - Error de {0}. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.fr.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.fr.xlf index 0e46d1985a..9f1e566fef 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.fr.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.fr.xlf @@ -83,16 +83,21 @@ Ne passez pas de types valeur à AreSame(). Les valeurs converties en Object ne seront plus jamais les mêmes. Si possible, utilisez AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Détails : - - Assert.That({0}) failed. - Désolé, échec de Assert.That({0}). - {0} is the user code expression - Message: {0} Message : {0} @@ -233,9 +238,9 @@ Réel : {2} - {0} failed. {1} - Échec de {0}. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.it.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.it.xlf index bfdc4c7ee6..3b50a35bc0 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.it.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.it.xlf @@ -83,16 +83,21 @@ Non passare tipi valore a AreSame(). I valori convertiti in Object non saranno mai uguali. Usare AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Dettagli: - - Assert.That({0}) failed. - Assert.That({0}) non riuscito. - {0} is the user code expression - Message: {0} Messaggio: {0} @@ -233,9 +238,9 @@ Effettivo: {2} - {0} failed. {1} - {0} non riuscita. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ja.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ja.xlf index bdced96434..bea427f7db 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ja.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ja.xlf @@ -83,16 +83,21 @@ AreSame() には値型を渡すことはできません。オブジェクトに変換された値が同じにはなりません。AreEqual() を使用することを検討してください。{0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: 詳細: - - Assert.That({0}) failed. - Assert.That({0}) に失敗しました。 - {0} is the user code expression - Message: {0} メッセージ: {0} @@ -233,9 +238,9 @@ Actual: {2} - {0} failed. {1} - {0} に失敗しました。{1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ko.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ko.xlf index 6d9292a56e..80527611b9 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ko.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ko.xlf @@ -83,16 +83,21 @@ AreSame()에 값 형식을 전달하면 안 됩니다. Object로 변환된 값은 동일한 값으로 간주되지 않습니다. AreEqual()을 사용해 보세요. {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: 세부 정보: - - Assert.That({0}) failed. - Assert.That({0})이(가) 실패했습니다. - {0} is the user code expression - Message: {0} 메시지: {0} @@ -233,9 +238,9 @@ Actual: {2} - {0} failed. {1} - {0}이(가) 실패했습니다. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pl.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pl.xlf index c50ff5897a..331ae17a30 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pl.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pl.xlf @@ -83,16 +83,21 @@ Nie przekazuj typów wartości do metody AreSame(). Wartości przekonwertowane na typ Object nigdy nie będą takie same. Rozważ użycie metody AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Szczegóły: - - Assert.That({0}) failed. - Operacja Assert.That({0}) nie powiodła się. - {0} is the user code expression - Message: {0} Komunikat: {0} @@ -233,9 +238,9 @@ Rzeczywiste: {2} - {0} failed. {1} - {0} — niepowodzenie. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pt-BR.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pt-BR.xlf index faf04cc886..202af7d4fb 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pt-BR.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pt-BR.xlf @@ -83,16 +83,21 @@ Não passe tipos de valores para AreSame(). Os valores convertidos para Object nunca serão os mesmos. Considere usar AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Detalhes: - - Assert.That({0}) failed. - Assert.That({0}) falhou. - {0} is the user code expression - Message: {0} Mensagem: {0} @@ -233,9 +238,9 @@ Real: {2} - {0} failed. {1} - {0} falhou. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ru.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ru.xlf index e39af70016..489ca58a9c 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ru.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ru.xlf @@ -83,16 +83,21 @@ Не передавайте типы значений в функцию AreSame(). Значения, преобразованные в Object, никогда не будут одинаковыми. Попробуйте использовать AreEqual(). {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Подробности: - - Assert.That({0}) failed. - Сбой Assert.That({0}). - {0} is the user code expression - Message: {0} Сообщение: {0} @@ -233,9 +238,9 @@ Actual: {2} - {0} failed. {1} - Сбой {0}. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.tr.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.tr.xlf index 3a24012694..d556b3df15 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.tr.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.tr.xlf @@ -83,16 +83,21 @@ AreSame()'e değer türleri geçirmeyin. Object olarak dönüştürülen değerler asla aynı olamayacak. AreEqual() kullanmayı düşün. {0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: Ayrıntılar: - - Assert.That({0}) failed. - Assert.That({0}) başarısız oldu. - {0} is the user code expression - Message: {0} İleti: {0} @@ -233,9 +238,9 @@ Gerçekte olan: {2} - {0} failed. {1} - {0} başarısız. {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hans.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hans.xlf index 94100054a3..433dea62e4 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hans.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hans.xlf @@ -83,16 +83,21 @@ 不要向 AreSame() 传递值类型。转换为 Object 的值将永远不会相等。请考虑使用 AreEqual()。{0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: 详细信息: - - Assert.That({0}) failed. - Assert.That({0}) 失败。 - {0} is the user code expression - Message: {0} 消息: {0} @@ -233,9 +238,9 @@ Actual: {2} - {0} failed. {1} - {0} 失败。{1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hant.xlf b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hant.xlf index 3d758f945a..0ca92f80f1 100644 --- a/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hant.xlf +++ b/src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hant.xlf @@ -83,16 +83,21 @@ 不要將實值型別傳遞給 AreSame()。轉換成 Object 的值從此不再一樣。請考慮使用 AreEqual()。{0} + + {0} assertion(s) failed within the assert scope. + {0} assertion(s) failed within the assert scope. + {0} is the number of assertion failures collected in the scope. + + + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + Nested assert scopes are not allowed. Dispose the current scope before creating a new one. + + Details: 詳細資料: - - Assert.That({0}) failed. - Assert.That({0}) 失敗。 - {0} is the user code expression - Message: {0} 訊息: {0} @@ -233,9 +238,9 @@ Actual: {2} - {0} failed. {1} - {0} 失敗。 {1} - + {0} failed. + {0} failed. + Expected collection of size {1}. Actual: {2}. {0} diff --git a/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs b/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs new file mode 100644 index 0000000000..594dc653d1 --- /dev/null +++ b/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs @@ -0,0 +1,204 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Microsoft.Testing.Platform.Acceptance.IntegrationTests; +using Microsoft.Testing.Platform.Acceptance.IntegrationTests.Helpers; +using Microsoft.Testing.Platform.Helpers; + +namespace MSTest.Acceptance.IntegrationTests; + +[TestClass] +public sealed class SoftAssertionTests : AcceptanceTestBase +{ + [TestMethod] + public async Task ScopeWithNoFailures_TestPasses() + { + var testHost = TestHost.LocateFrom(AssetFixture.ProjectPath, TestAssetFixture.ProjectName, TargetFrameworks.NetCurrent); + TestHostResult testHostResult = await testHost.ExecuteAsync("--filter ScopeWithNoFailures", cancellationToken: TestContext.CancellationToken); + + testHostResult.AssertExitCodeIs(ExitCodes.Success); + testHostResult.AssertOutputContainsSummary(failed: 0, passed: 1, skipped: 0); + } + + [TestMethod] + public async Task ScopeWithSingleFailure_TestFails() + { + var testHost = TestHost.LocateFrom(AssetFixture.ProjectPath, TestAssetFixture.ProjectName, TargetFrameworks.NetCurrent); + TestHostResult testHostResult = await testHost.ExecuteAsync("--filter ScopeWithSingleFailure", cancellationToken: TestContext.CancellationToken); + + testHostResult.AssertExitCodeIs(ExitCodes.AtLeastOneTestFailed); + testHostResult.AssertOutputMatchesRegex( + """failed ScopeWithSingleFailure \(\d+ms\)[\s\S]+Assert\.AreEqual failed\. Expected:<1>\. Actual:<2>\.[\s\S]+at UnitTest1\.ScopeWithSingleFailure\(\)"""); + } + + [TestMethod] + public async Task ScopeWithMultipleFailures_TestFailsWithAggregatedMessage() + { + var testHost = TestHost.LocateFrom(AssetFixture.ProjectPath, TestAssetFixture.ProjectName, TargetFrameworks.NetCurrent); + TestHostResult testHostResult = await testHost.ExecuteAsync("--filter ScopeWithMultipleFailures", cancellationToken: TestContext.CancellationToken); + + testHostResult.AssertExitCodeIs(ExitCodes.AtLeastOneTestFailed); + // Validate the output includes the aggregate message and that inner exception stack traces + // point to the test method (assertion call site). + testHostResult.AssertOutputMatchesRegex( + """failed ScopeWithMultipleFailures \(\d+ms\)[\s\S]+2 assertion\(s\) failed within the assert scope\.[\s\S]+Assert\.AreEqual failed\. Expected:<1>\. Actual:<2>\.[\s\S]+at UnitTest1\.ScopeWithMultipleFailures\(\)[\s\S]+Assert\.IsTrue failed\.[\s\S]+at UnitTest1\.ScopeWithMultipleFailures\(\)"""); + } + + [TestMethod] + public async Task AssertFailIsHardFailure_ThrowsImmediately() + { + var testHost = TestHost.LocateFrom(AssetFixture.ProjectPath, TestAssetFixture.ProjectName, TargetFrameworks.NetCurrent); + TestHostResult testHostResult = await testHost.ExecuteAsync("--filter AssertFailIsHardFailure", cancellationToken: TestContext.CancellationToken); + + testHostResult.AssertExitCodeIs(ExitCodes.AtLeastOneTestFailed); + // Assert.Fail is a hard assertion — it throws immediately, even within a scope. + // The second Assert.Fail should not be reached. + testHostResult.AssertOutputMatchesRegex( + """failed AssertFailIsHardFailure \(\d+ms\)[\s\S]+Assert\.Fail failed\. hard failure"""); + testHostResult.AssertOutputDoesNotContain("second failure"); + } + + [TestMethod] + public async Task ScopeWithSoftFailureFollowedByException_CollectsBoth() + { + var testHost = TestHost.LocateFrom(AssetFixture.ProjectPath, TestAssetFixture.ProjectName, TargetFrameworks.NetCurrent); + TestHostResult testHostResult = await testHost.ExecuteAsync("--filter SoftFailureFollowedByException", cancellationToken: TestContext.CancellationToken); + + testHostResult.AssertExitCodeIs(ExitCodes.AtLeastOneTestFailed); + testHostResult.AssertOutputMatchesRegex( + """failed SoftFailureFollowedByException \(\d+ms\)[\s\S]+at UnitTest1\.SoftFailureFollowedByException\(\)"""); + } + + [TestMethod] + public async Task ScopeWithIsNotNullSoftFailure_CollectsFailure() + { + var testHost = TestHost.LocateFrom(AssetFixture.ProjectPath, TestAssetFixture.ProjectName, TargetFrameworks.NetCurrent); + TestHostResult testHostResult = await testHost.ExecuteAsync("--filter ScopeWithIsNotNullSoftFailure", cancellationToken: TestContext.CancellationToken); + + testHostResult.AssertExitCodeIs(ExitCodes.AtLeastOneTestFailed); + testHostResult.AssertOutputMatchesRegex( + """failed ScopeWithIsNotNullSoftFailure \(\d+ms\)[\s\S]+Assert\.IsNotNull failed\.[\s\S]+at UnitTest1\.ScopeWithIsNotNullSoftFailure\(\)"""); + } + + [TestMethod] + public async Task ScopeAssertionsAreIndependentBetweenTests_SecondTestPasses() + { + var testHost = TestHost.LocateFrom(AssetFixture.ProjectPath, TestAssetFixture.ProjectName, TargetFrameworks.NetCurrent); + TestHostResult testHostResult = await testHost.ExecuteAsync("--filter IndependentTest", cancellationToken: TestContext.CancellationToken); + + testHostResult.AssertExitCodeIs(ExitCodes.Success); + testHostResult.AssertOutputContainsSummary(failed: 0, passed: 1, skipped: 0); + } + + public sealed class TestAssetFixture() : TestAssetFixtureBase(AcceptanceFixture.NuGetGlobalPackagesFolder) + { + public const string ProjectName = "SoftAssertionTests"; + + public string ProjectPath => GetAssetPath(ProjectName); + + public override IEnumerable<(string ID, string Name, string Code)> GetAssetsToGenerate() + { + yield return (ProjectName, ProjectName, + SourceCode + .PatchTargetFrameworks(TargetFrameworks.NetCurrent) + .PatchCodeWithReplace("$MSTestVersion$", MSTestVersion)); + } + + private const string SourceCode = """ +#file SoftAssertionTests.csproj + + + + Exe + true + $TargetFrameworks$ + $(NoWarn);MSTESTEXP + + + + + + + + + +#file UnitTest1.cs +using System; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +[TestClass] +public class UnitTest1 +{ + [TestMethod] + public void ScopeWithNoFailures() + { + using (Assert.Scope()) + { + Assert.IsTrue(true); + Assert.AreEqual(1, 1); + } + } + + [TestMethod] + public void ScopeWithSingleFailure() + { + using (Assert.Scope()) + { + Assert.AreEqual(1, 2); + } + } + + [TestMethod] + public void ScopeWithMultipleFailures() + { + using (Assert.Scope()) + { + Assert.AreEqual(1, 2); + Assert.IsTrue(false); + } + } + + [TestMethod] + public void AssertFailIsHardFailure() + { + using (Assert.Scope()) + { + Assert.Fail("hard failure"); + Assert.Fail("second failure"); + } + } + + [TestMethod] + public void SoftFailureFollowedByException() + { + string x = null; + using (Assert.Scope()) + { + Assert.IsNotNull(x); + Assert.AreEqual(1, x.Length); // throws NullReferenceException + } + } + + [TestMethod] + public void ScopeWithIsNotNullSoftFailure() + { + object value = null; + using (Assert.Scope()) + { + Assert.IsNotNull(value); + Assert.AreEqual(1, 1); + } + } + + [TestMethod] + public void IndependentTest() + { + // Verify that a scope from a previous test does not leak into this test. + Assert.IsTrue(true); + } +} +"""; + } + + public TestContext TestContext { get; set; } +} diff --git a/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs b/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs new file mode 100644 index 0000000000..096dbb9905 --- /dev/null +++ b/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs @@ -0,0 +1,212 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using AwesomeAssertions; + +namespace Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests; + +public partial class AssertTests +{ + public void Scope_NoFailures_DoesNotThrow() + { + Action action = () => + { + using (Assert.Scope()) + { + Assert.IsTrue(true); + Assert.AreEqual(1, 1); + } + }; + + action.Should().NotThrow(); + } + + public void Scope_SingleFailure_ThrowsOnDispose() + { + IDisposable scope = Assert.Scope(); + Assert.AreEqual(1, 2); + Action action = () => scope.Dispose(); + + action.Should().Throw() + .WithMessage("Assert.AreEqual failed. Expected:<1>. Actual:<2>. 'expected' expression: '1', 'actual' expression: '2'."); + } + + public void Scope_MultipleFailures_CollectsAllErrors() + { + Action action = () => + { + using (Assert.Scope()) + { + Assert.AreEqual(1, 2); + Assert.IsTrue(false); + } + }; + + AggregateException innerException = action.Should().Throw() + .WithMessage("2 assertion(s) failed within the assert scope.") + .WithInnerException() + .Which; + + innerException.InnerExceptions.Should().HaveCount(2); + innerException.InnerExceptions[0].Message.Should().Be("Assert.AreEqual failed. Expected:<1>. Actual:<2>. 'expected' expression: '1', 'actual' expression: '2'."); + innerException.InnerExceptions[1].Message.Should().Be("Assert.IsTrue failed. 'condition' expression: 'false'."); + } + + public void Scope_AfterDispose_AssertionsThrowNormally() + { + // Completing a scope should restore normal behavior. + try + { + using (Assert.Scope()) + { + // intentionally empty scope + } + } + catch + { + // ignore + } + + Action action = () => Assert.IsTrue(false); + action.Should().Throw(); + } + + public void Scope_NestedScope_ThrowsInvalidOperationException() + { + Action action = () => + { + using (Assert.Scope()) + { + using (Assert.Scope()) + { + // Should never reach here + } + } + }; + + action.Should().Throw() + .WithMessage("Nested assert scopes are not allowed. Dispose the current scope before creating a new one."); + } + + public void Scope_DoubleDispose_DoesNotThrowTwice() + { + IDisposable scope = Assert.Scope(); + Assert.AreEqual(1, 2); + + Action firstDispose = () => scope.Dispose(); + firstDispose.Should().Throw() + .WithMessage("Assert.AreEqual failed. Expected:<1>. Actual:<2>. 'expected' expression: '1', 'actual' expression: '2'."); + + // Second dispose should be a no-op + Action secondDispose = () => scope.Dispose(); + secondDispose.Should().NotThrow(); + } + + public void Scope_AssertFail_IsHardFailure() + { + Action action = () => + { + using (Assert.Scope()) + { + Assert.Fail("first failure"); + Assert.Fail("second failure"); + } + }; + + // Assert.Fail is a hard assertion — it throws immediately, even within a scope. + action.Should().Throw() + .WithMessage("Assert.Fail failed. first failure"); + } + + public void Scope_AssertIsNotNull_IsSoftFailure() + { + object? value = null; + Action action = () => + { + using (Assert.Scope()) + { + Assert.IsNotNull(value); + Assert.AreEqual(1, 2); + } + }; + + // Assert.IsNotNull is a soft assertion — failure is collected within a scope. + AggregateException innerException = action.Should().Throw() + .WithMessage("2 assertion(s) failed within the assert scope.") + .WithInnerException() + .Which; + + innerException.InnerExceptions.Should().HaveCount(2); + innerException.InnerExceptions[0].Message.Should().Be("Assert.IsNotNull failed. 'value' expression: 'value'."); + innerException.InnerExceptions[1].Message.Should().Contain("Assert.AreEqual failed."); + } + + public void Scope_AssertIsInstanceOfType_IsSoftFailure() + { + object value = "hello"; + Action action = () => + { + using (Assert.Scope()) + { + Assert.IsInstanceOfType(value, typeof(int)); + Assert.AreEqual(1, 2); + } + }; + + // Assert.IsInstanceOfType is a soft assertion — failure is collected within a scope. + AggregateException innerException = action.Should().Throw() + .WithMessage("2 assertion(s) failed within the assert scope.") + .WithInnerException() + .Which; + + innerException.InnerExceptions.Should().HaveCount(2); + innerException.InnerExceptions[0].Message.Should().Be("Assert.IsInstanceOfType failed. 'value' expression: 'value'. Expected type:. Actual type:."); + innerException.InnerExceptions[1].Message.Should().Contain("Assert.AreEqual failed."); + } + + public void Scope_AssertIsExactInstanceOfType_IsSoftFailure() + { + object value = "hello"; + Action action = () => + { + using (Assert.Scope()) + { + Assert.IsExactInstanceOfType(value, typeof(object)); + Assert.AreEqual(1, 2); + } + }; + + // Assert.IsExactInstanceOfType is a soft assertion — failure is collected within a scope. + AggregateException innerException = action.Should().Throw() + .WithMessage("2 assertion(s) failed within the assert scope.") + .WithInnerException() + .Which; + + innerException.InnerExceptions.Should().HaveCount(2); + innerException.InnerExceptions[0].Message.Should().Be("Assert.IsExactInstanceOfType failed. 'value' expression: 'value'. Expected exact type:. Actual type:."); + innerException.InnerExceptions[1].Message.Should().Contain("Assert.AreEqual failed."); + } + + public void Scope_AssertContainsSingle_IsSoftFailure() + { + int[] items = [1, 2, 3]; + Action action = () => + { + using (Assert.Scope()) + { + Assert.ContainsSingle(items); + Assert.AreEqual(1, 2); + } + }; + + // Assert.ContainsSingle is a soft assertion — failure is collected within a scope. + AggregateException innerException = action.Should().Throw() + .WithMessage("2 assertion(s) failed within the assert scope.") + .WithInnerException() + .Which; + + innerException.InnerExceptions.Should().HaveCount(2); + innerException.InnerExceptions[0].Message.Should().Be("Assert.ContainsSingle failed. Expected collection to contain exactly one element but found 3 element(s). 'collection' expression: 'items'."); + innerException.InnerExceptions[1].Message.Should().Contain("Assert.AreEqual failed."); + } +} diff --git a/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.That.cs b/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.That.cs index 62297d11df..c090c24e4c 100644 --- a/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.That.cs +++ b/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.That.cs @@ -31,7 +31,7 @@ public void That_BooleanCondition_FailsAsExpected() act.Should().Throw() .WithMessage( """ - Assert.That(() => False) failed. + Assert.That(() => false) failed. Message: Boolean condition failed """); } @@ -763,7 +763,7 @@ public void That_WithBooleanLiterals_SkipsLiteralInDetails() act.Should().Throw() .WithMessage(""" - Assert.That(() => condition == True) failed. + Assert.That(() => condition == true) failed. Details: condition = False """); @@ -861,7 +861,7 @@ public void That_WithComplexConstantExpression_HandlesCorrectly() act.Should().Throw() .WithMessage(""" - Assert.That(() => dynamicValue.Length == ConstValue && flag == True) failed. + Assert.That(() => dynamicValue.Length == ConstValue && flag == true) failed. Details: dynamicValue.Length = 7 flag = False @@ -982,7 +982,7 @@ public void That_WithMixedLiteralsAndVariables_FiltersCorrectly() act.Should().Throw() .WithMessage(""" - Assert.That(() => name == "Admin" && age == 30 && isActive == False && grade == 'A') failed. + Assert.That(() => name == "Admin" && age == 30 && isActive == false && grade == 'A') failed. Details: age = 25 grade = B diff --git a/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ThrowsExceptionTests.cs b/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ThrowsExceptionTests.cs index c41eba3302..e773d3ad92 100644 --- a/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ThrowsExceptionTests.cs +++ b/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ThrowsExceptionTests.cs @@ -7,11 +7,11 @@ namespace Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests; public partial class AssertTests { - #region ThrowAssertFailed tests + #region ReportAssertFailed tests // See https://github.com/dotnet/sdk/issues/25373 - public void ThrowAssertFailedDoesNotThrowIfMessageContainsInvalidStringFormatComposite() + public void ReportAssertFailedDoesNotThrowIfMessageContainsInvalidStringFormatComposite() { - Action action = () => Assert.ThrowAssertFailed("name", "{"); + Action action = () => Assert.ReportAssertFailed("name", "{"); action.Should().Throw() .WithMessage("*name failed. {*"); }