Two AwesomeAssertions-based unit test projects lack BannedSymbols.txt enforcement (MSTest.SourceGeneration.UnitTests, TestFramew [Content truncated due to length]

#10645Opengithub-actions[bot] 创建于 4 天前

type/tech-debttype/automation

### 🎯 Repository Quality Improvement Report — Assertion-Style Enforcement Gaps **Analysis Date**: 2026-08-18 **Focus Area**: `banned-symbols-project-wiring-completeness` (custom) **Strategy Type**: Custom ### Executive Summary The repository enforces a per-project assertion style via `BannedSymbols.txt` + `<AdditionalFiles Include="BannedSymbols.txt" />`, banning MSTest `Assert`/`StringAssert`/`CollectionAssert` in projects that use `AwesomeAssertions`, and vice versa (per repo instructions and 29 existing `BannedSymbols.txt` files across `src/` and `test/UnitTests/`). Two unit test projects that consume `AwesomeAssertions` as a `PackageReference` do **not** have a `BannedSymbols.txt` at all, and therefore have no compiler-enforced guard against MSTest `Assert.*` calls creeping back in: - `test/UnitTests/MSTest.SourceGeneration.UnitTests/` (4 `.cs` files, 4,556 lines) - `test/UnitTests/TestFramework.UnitTests/` (6 `.cs` files, 591 lines) Every other `AwesomeAssertions`-consuming test project in the repo (`MSTestAdapter.UnitTests`, `MSTestAdapter.PlatformServices.UnitTests`, `Microsoft.Testing.Extensions.VSTestBridge.UnitTests`) has both the `BannedSymbols.txt` file and the matching `<AdditionalFiles Include="BannedSymbols.txt" />` csproj wiring. This is a mechanical/automation gap rather than a design decision — `TestFramework.UnitTests` currently only avoids MSTest `Assert.*` by convention (grep shows zero live `Assert.*` calls, only comments referencing `Assert.Scope`/`Assert.Fail`), which means the safety net is purely social, not enforced. <details> <summary>Full Analysis Report</summary> ### Focus Area: BannedSymbols.txt Wiring Completeness ### Current State Assessment **Metrics Collected:** | Metric | Value | Status | |--------|-------|--------| | Total `BannedSymbols.txt` files in repo | 29 | ✅ | | `test/UnitTests/*` projects using `AwesomeAssertions` | 6 | ⚠️ | | Of those, missing `BannedSymbols.txt` | 2 (`MSTest.SourceGeneration.UnitTests`, `TestFramework.UnitTests`) | ❌ | | Live `Assert.*` calls currently in the 2 unguarded projects | 0 (grep shows only comment references) | ✅ (for now) | ### Findings #### Strengths - The `BannedSymbols.txt` convention is well-established and consistently applied across 27 of 29 relevant projects (`src/Adapter`, all `src/Platform/*` extensions, and 4 of 6 `AwesomeAssertions`-based `test/UnitTests` projects). - `MSTest.Analyzers.UnitTests`, `MSTest.SelfRealExamples.UnitTests`, and all MTP unit-test projects correctly ban `AwesomeAssertions` and enforce MSTest `Assert`. #### Areas for Improvement - **[Medium]** `test/UnitTests/TestFramework.UnitTests/TestFramework.UnitTests.csproj` references `AwesomeAssertions` but has no `BannedSymbols.txt`, so nothing in the build prevents a future PR from reintroducing MSTest `Assert.*` calls (which would be semantically confusing since this project also project-references `TestFramework.csproj`, the very assembly that defines `Assert`). - **[Medium]** `test/UnitTests/MSTest.SourceGeneration.UnitTests/MSTest.SourceGeneration.UnitTests.csproj` has the same gap, across 4,556 lines of test code — the largest of the two by far. - Both projects' `.csproj` files also lack the `<AdditionalFiles Include="BannedSymbols.txt" />` wiring line, so simply dropping a file in without updating the csproj would silently do nothing (a common human mistake given the wiring is manual, not automatic via `Directory.Build.props`). </details> --- ### 🤖 Suggested Improvement Tasks #### Task 1: Add `BannedSymbols.txt` to `test/UnitTests/TestFramework.UnitTests/` **Priority**: Medium **Estimated Effort**: Small Create `test/UnitTests/TestFramework.UnitTests/BannedSymbols.txt` mirroring `test/UnitTests/MSTestAdapter.UnitTests/BannedSymbols.txt`: ``` T:Microsoft.VisualStudio.TestTools.UnitTesting.Assert; Use AwesomeAssertions for testing in this project. T:Microsoft.VisualStudio.TestTools.UnitTesting.StringAssert; Use AwesomeAssertions for testing in this project. T:Microsoft.VisualStudio.TestTools.UnitTesting.CollectionAssert; Use AwesomeAssertions for testing in this project. ``` Then add `<AdditionalFiles Include="BannedSymbols.txt" />` to `TestFramework.UnitTests.csproj`'s `<ItemGroup>`, and build the project to confirm the `BannedApiAnalyzer` (Microsoft.CodeAnalysis.BannedApiAnalyzers, already used elsewhere in the repo for this pattern) picks it up with no violations. --- #### Task 2: Add `BannedSymbols.txt` to `test/UnitTests/MSTest.SourceGeneration.UnitTests/` **Priority**: Medium **Estimated Effort**: Small Same pattern as Task 1: create `BannedSymbols.txt` with the three banned MSTest assertion types, add the `<AdditionalFiles Include="BannedSymbols.txt" />` line to `MSTest.SourceGeneration.UnitTests.csproj`, and rebuild to verify no existing test file trips the new banned-symbol diagnostic (none currently use `Assert.*`, per this analysis, so it should build clean). --- #### Task 3: Add a CI/dev-time consistency check for the `BannedSymbols.txt` ↔ `AwesomeAssertions`/`Assert` convention **Priority**: Low **Estimated Effort**: Medium To prevent this class of gap from recurring, add a lightweight repo script or analyzer check (e.g. a small PowerShell/Python script invoked from an existing CI job) that flags any `test/UnitTests/*` or `src/*` project referencing `AwesomeAssertions` without a `BannedSymbols.txt` + matching `<AdditionalFiles>` entry (and vice versa for projects that reference MSTest.TestFramework as a test dependency without banning `AwesomeAssertions`). This turns today's manual convention into an enforced invariant. --- #### Task 4: Audit remaining `test/UnitTests/*` projects for the same drift pattern **Priority**: Low **Estimated Effort**: Small Beyond the two identified here, periodically re-run the detection query (`grep -rl "PackageReference Include=\"AwesomeAssertions\"" test/UnitTests/*/*.csproj` cross-referenced against `find test/UnitTests -name BannedSymbols.txt`) whenever new unit test projects are added, since new projects created by copy-pasting an existing `.csproj` without also copying the `BannedSymbols.txt` wiring is the likely root cause of this gap. --- ### 📊 Historical Context <details> <summary>Previous Focus Areas</summary> | Date | Focus Area | Type | |------|------------|------| | 2026-08-17 | executable-condition-attribute-parity-gap | Custom | | 2026-08-04 | mtp-exit-code-documentation-gap | Custom | | 2026-07-30 | assert-interpolated-string-handler-coverage-gap | Custom | | 2026-07-29 | warning-suppression-technical-debt | Custom | | 2026-07-23 | parameterized-test-display-name-ux | Custom | | 2026-06-18 | banned-symbols-assertion-enforcement | Custom | </details> --- ### 🎯 Recommendations #### Immediate Actions (This Week) 1. Add `BannedSymbols.txt` + csproj wiring to `TestFramework.UnitTests` — Priority: Medium 2. Add `BannedSymbols.txt` + csproj wiring to `MSTest.SourceGeneration.UnitTests` — Priority: Medium #### Short-term Actions (This Month) 1. Build a lightweight consistency-check script for the assertion-style convention — Priority: Low 2. Periodically re-audit new unit test projects for the same gap — Priority: Low *Next analysis: next scheduled run — Focus area selected based on diversity algorithm* > 🤖 **Automated content by GitHub Copilot.** Generated by the [Repository Quality Improver](https://github.com/microsoft/testfx/actions/runs/32191907273/agentic_workflow) workflow. · auto · 121.1 AIC · ⌖ 3.49 AIC · ⊞ 15.2K · [◷]( · [◷](https://github.com/search?q=repo%3Amicrosoft%2Ftestfx+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2Ftestfx%2Frepository-quality-improver%22&type=issues)) > <details> <summary>Add this agentic workflow to your repo</summary> To install this agentic workflow, run ``` gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main ``` </details> > - [x] expires  on Aug 20, 2026, 10:26 PM UTC

0 条评论