# CI Pipeline Monitor — Test Report Template

This is the template for the test report generated by `ci-pipeline-monitor`.
The test report is generated on request (e.g., weekly report). The debug
log is a separate file (see `log-template.md`).

Output file name: `test-report-<YYYY-MM-DD-HHmmss>.md`

**Rule:** `<table.column>` in this template means use the exact value from
that SQL column verbatim — no rephrasing, no substitution.

**Rule:** All data in SQL is already in final print-ready form. The script reads
and prints as-is — no transformations (e.g., `.WorkItemExecution` is already
stripped by the LLM before inserting).

---

```
================================================================================
CI Pipeline Monitor — Test Report
Date:       <YYYY-MM-DD>
Org:        dnceng-public
Project:    public
Branch:     refs/heads/main
Pipelines:  <N> total (<M> monitored, <K> skipped)
================================================================================

================================================================================
Pipeline Summary
================================================================================

Result: <P> PASS, <F> FAIL, <I> INCONCLUSIVE (of <M> monitored)

  ✅ <pipelines.name>                            <pipelines.build_number>
     https://dev.azure.com/dnceng-public/public/_build/results?buildId=<pipelines.build_id>

  ❌ <pipelines.name>                            <pipelines.build_number>
     https://dev.azure.com/dnceng-public/public/_build/results?buildId=<pipelines.build_id>
     - [<N>] <failures.test_name> (#<N>, https://github.com/dotnet/runtime/issues/<N>)
     - [<N>] <failures.test_name> (#<N>, https://github.com/dotnet/runtime/issues/<N>)

  ❌ <pipelines.name>                            <pipelines.build_number>
     https://dev.azure.com/dnceng-public/public/_build/results?buildId=<pipelines.build_id>
     - [<N>] <failures.test_name> (#<N>, https://github.com/dotnet/runtime/issues/<N>)
     - [<N>] [New] <failures.test_name> (<brief error description>)

  ⚠️ [<pipelines.name> <pipelines.build_number>](https://dev.azure.com/dnceng-public/public/_build/results?buildId=<pipelines.build_id>): INCONCLUSIVE (<reason>)

  ⏭️ <pipelines.name>: SKIPPED (private)

Notes:
- ✅ = all tests passed
- ❌ = one or more test failures
- ⚠️ = inconclusive (build failed but no test failures detected via Test Results API)
- ⏭️ = skipped (private pipeline)
- EVERY pipeline (✅ and ❌) must include the build URL on the line after the name.
- List failing tests per pipeline, deduplicated by test name. Cap at 5 per failure group; show "... and N more" for the rest.
- [N] maps to "FAILURE N" in the Failure Details section. Sorted by failure ID.
- [New] = no matching GitHub issue found — may need a new issue filed.

================================================================================
Failure Details
================================================================================

The following unique test failures were found across all failing pipelines.
Each failure category is listed once with all affected pipelines.

IMPORTANT: Generate EVERY failure in this format — not just the first one.
Every failure must have Summary, Failed in, Failed tests, Error Message,
Stack Trace, and Analysis sections with the "Body (paste as-is)" block.

For failures with a matching GitHub issue, the Body block serves as an
update comment to paste into the existing issue.
For [New] failures, the Body block serves as the new issue body to paste
when filing.

--------------------------------------------------------------------------------
FAILURE <N>: <Short descriptive title> (<arch/platform scope>)
--------------------------------------------------------------------------------
GitHub Issue: #<N> (https://github.com/dotnet/runtime/issues/<N>) — <OPEN/CLOSED>
  (or: GitHub Issue: NEW — needs issue filed)
Assigned to: @<user>
Labels: <label1>, <label2>
Failing since: <YYYY-MM-DD> (build <buildId>)

Work item: <work item name, e.g., HardwareIntrinsics_Arm_ro>

Title: Test Failure: <failures.test_name>
Labels: <area-CodeGen-coreclr>, <os-label>, <arch-label>, <stress-label>, blocking-clean-ci-optional
Milestone: 11.0.0

Body (paste as-is into GitHub issue):
<<<
**Summary:**
  <1-2 sentence description of the failure and its impact>

**Failed in (<count>) pipeline(s):**
- [<pipelines.name> <pipelines.build_number>](https://dev.azure.com/dnceng-public/public/_build/results?buildId=<pipelines.build_id>)
- [<pipelines.name> <pipelines.build_number>](https://dev.azure.com/dnceng-public/public/_build/results?buildId=<pipelines.build_id>)

**Console Log:** [Console Log](<console_log_url>)
**Source:** [<pipeline_name> / <run_name> / <test_name>](https://dev.azure.com/dnceng-public/public/_build/results?buildId=<build_id>&view=ms.vss-test-web.build-test-results-tab)

**Failed tests:**
```
// If failing in 2+ pipelines, group by pipeline name:
<failures.pipeline>
- <failures.run_name>
- <failures.run_name>
<failures.pipeline>
- <failures.run_name>

// If failing in only 1 pipeline, omit pipeline name:
- <failures.run_name>
- <failures.run_name>

  - <failures.test_name>
  - <failures.test_name>
```
List ALL failed test legs. Cap test names at 5; show "... and N more" for the rest.

**Error Message:**
```
<failures.error_message>
```

**Stack Trace:**
```
<failures.stack_trace>
```

**Analysis:**
<1-3 paragraphs:
- What the failure is (timeout, assertion, crash, wrong result, etc.)
- Root cause hypothesis based on the error output
- Which configs/platforms are affected vs unaffected
- Regression info: failing since <date> (build <id>), likely introduced
  by <PR/commit if known>
- Related issues if any
- Suggested priority and next steps>
>>>

(Repeat the above for EVERY failure. Do not abbreviate or skip any.)

================================================================================
GitHub Issue Summary
================================================================================

  #<NNNNN> (https://github.com/dotnet/runtime/issues/<NNNNN>)
    <Issue Title>
    State: OPEN | Assigned: @<user> | Pipelines affected: <N>

  #<NNNNN> (https://github.com/dotnet/runtime/issues/<NNNNN>)
    <Issue Title>
    State: CLOSED | Pipelines affected: <N>

  [New] Test Failure: <failures.test_name>
    No matching GitHub issue found. Affects <N> pipeline(s).
    Suggested labels: <label1>, <label2>

================================================================================
Action Items
================================================================================

### 🆕 Issues to File

- [NEW] <failures.test_name> (<test_count> test(s), <pipe_count> pipeline(s))
  See FAILURE <N> above for pre-formatted issue body.

// Or if none:
None — all failures matched to existing GitHub issues.

### 🔥 High Impact Failures

// Known issues affecting >1 pipeline or >5 test results, ordered by impact:
- <failures.test_name> — #<N> (<test_count> test(s), <pipe_count> pipeline(s))
  https://github.com/dotnet/runtime/issues/<N>

// Omit section if no high-impact failures.

### 👁️ Needs Review

// Only if --validation-warnings flag was passed:
- Unresolved validation warnings — see debug log for details.

// Omit section if validation passed.

================================================================================
End of report. Generated by ci-pipeline-monitor skill.
================================================================================
```
