L1 to L4 design for Cross-Repository CI Relay
## Background
During L1 implementation #7847, we met some confusion about the Cross-Repository CI Relay, which may not be clearly specified in this [RFC](https://github.com/pytorch/pytorch/issues/175022). Therefore, I open this issue to discuss the detailed design of L1 through L4 to improve future L2-L4 implementations. I would also like to listen to different opinions from the community.
## L1
```mermaid
%%{init: {"theme": "base"}}%%
sequenceDiagram
participant U as UpStream Repo
participant W as webhook_handler
participant R as Redis
participant S as result_handler
participant D as DownStream Repo
U->>W: PR/Push event trigger
W->>R: Get Allowlist
W->>D: Passthrough Payload
```
L1 is the most basic event forwarding layer. Its core goal is to allow the upstream PyTorch repository to safely forward PR and push events to onboarded downstream repositories, without writing any status back to the upstream side.
The flow in the diagram is straightforward:
1. An upstream PR event (`opened`/`reopened`/`synchronized`/`closed`) or push event triggers `webhook_handler` through the GitHub App.
2. `webhook_handler` reads the `Allowlist` from a remote source and stores it in `Redis` to reduce follow-up calls, determining which downstream repositories should receive the event.
3. The `payload` is then passed through directly to the downstream repositories.
For the concrete L1 implementation, see #7847.
## L2
```mermaid
%%{init: {"theme": "base"}}%%
sequenceDiagram
participant U as UpStream Repo
participant W as webhook_handler
participant R as Redis
participant S as result_handler
participant D as DownStream Repo
participant H as HUD
U->>W: PR/Push event trigger
W->>R: Get Allowlist
W->>D: Passthrough Payload
rect rgb(240, 240, 240)
Note over S, D: Creating In Progress in HUD
D->>S: In progress call
S->>R: Get Allowlist
S->>H: Show in progress on HUD
end
rect rgb(240, 240, 240)
Note over S, D: Updating Status in HUD
D->>S: Completed workflow run call
S->>R: Get Allowlist
S->>H: Show completed on HUD
end
```
L2 adds the ability to report results back to HUD on top of L1. The first half of the diagram is the same as L1: the upstream event enters `webhook_handler`, the `Allowlist` is read, and the downstream repository is triggered. The new part is the two gray sections in the second half:
- When the downstream workflow run starts running, it sends an `In progress` callback to `result_handler`.
1. `DownStream Repo` actively sends a callback request to `result_handler` from the first job in the workflow.
2. After authenticating the request, `result_handler` reads the `Allowlist` to verify whether the request from this `DownStream Repo` belongs to L2 or above.
3. If the request is valid, the run information triggered by the workflow is written to HUD and marked as `in progress`.
- When the workflow run finishes, it sends a `Completed` callback.
1. `DownStream Repo` actively sends a callback request to `result_handler` from the last job in the workflow.
2. After authenticating the request, `result_handler` reads the `Allowlist` to verify whether the request from this `DownStream Repo` belongs to L2 or above.
3. If the request is valid, the run information triggered by the workflow is written to HUD and marked as `completed`.
## L3
```mermaid
%%{init: {"theme": "base"}}%%
sequenceDiagram
participant U as UpStream Repo
participant W as webhook_handler
participant R as Redis
participant RH as result_handler
participant D as DownStream Repo
participant H as HUD
U->>W: PR/Push event trigger
W->>R: Get Allowlist
W->>D: Passthrough payload
rect rgb(240, 240, 240)
Note over R, D: Scenario 1: label add before workflow run create
U->>W: PR label add
W->>R: Cache PR label info
end
D->>RH: In progress workflow run call
RH->>R: Get Allowlist<br>Cache workflow run info
RH->>H: Show in progress workflow run on HUD
rect rgb(240, 240, 240)
Note over R, D: Scenario 1
RH->>R: Find PR label info record
RH->>U: Create PR in_progress check run
end
rect rgb(240, 240, 240)
Note over R, D: Scenario 2: label add during workflow run execute
U->>W: PR label add
W->>R: Find workflow run info
W->>U: Create PR in_progress check run
end
D->>RH: Completed workflow run call
RH->>R: Get Allowlist<br>Update workflow run info
RH->>H: Show completed workflow run on HUD
rect rgb(240, 240, 240)
Note over R, D: Scenario 1 & 2
RH->>U: Update PR completed check run
end
rect rgb(240, 240, 240)
Note over R, D: Scenario 3: label add after run complete
U->>W: PR label add
W->>R: Find workflow run info
W->>U: Create PR completed check run
end
```
L3 keeps the HUD display capability from L2 and further introduces on-demand upstream PR check runs. Consistent with the `label_only` design in the [RFC](https://github.com/pytorch/pytorch/issues/175022), this layer does not attach downstream results to every PR by default. Instead, the status of the corresponding backend is shown as a non-blocking upstream check only after a label is explicitly added to the PR. The key of L3 is whether the label event or the downstream workflow run status arrives first. Because of that, both sides of the information need to be temporarily stored in Redis, and the check run is created or updated when the timing is right.
L3 has the following three scenarios:
- Scenario 1 means the label arrives before the workflow run:
1. `webhook_handler` first caches the label information in `Redis`.
2. The downstream workflow run starts and calls back to `result_handler`. After finding the matching label record in the cache, `result_handler` immediately creates an `in_progress` check run on the upstream PR.
3. After the workflow run completes and `DownStream Repo` sends the completed callback to `result_handler`, `result_handler` updates both the workflow run status in `Redis` and the check run status on the PR.
- Scenario 2 means the workflow run is already executing and the label arrives later:
1. When `result_handler` receives the `in progress` callback from `DownStream Repo`, it first caches the workflow run information.
2. After the user adds a label to the PR, `webhook_handler` looks up that workflow run record in reverse and backfills an `in_progress` check run.
3. After the workflow run completes and `DownStream Repo` sends the completed callback to `result_handler`, `result_handler` updates both the workflow run status in `Redis` and the check run status on the PR.
- Scenario 3 is the later case where the downstream workflow run has already completed before the user adds the label:
1. In this case, `webhook_handler` directly creates a `completed` check run based on the workflow run result already stored in Redis, without re-triggering execution.
2. If the record for that workflow run has already been removed from `Redis`, the check run will not be created.
**Note**:
The Redis cache TTL is tentatively set to 3 hours to align with the workflow integration requirements, so Redis data will not grow indefinitely.
## L4
The L4 scenario is the same as L3 Scenario 1. The difference is that L3 requires a label to trigger, while L4 triggers by default without requiring a label.
## Check run
```mermaid
%%{init: {"theme": "base"}}%%
sequenceDiagram
participant U as UpStream Repo
participant W as webhook_handler
participant R as Redis
participant RH as result_handler
participant D as DownStream Repo
participant H as HUD
U->>W: Re-run workflow run trigger
W->>R: Get Allowlist
W->>D: Re-run dispatch
D->>RH: In progress workflow run call
RH->>R: Get Allowlist
RH->>U: Update in progress check run
RH->>H: Update in progress workflow run in HUD
D->>RH: Completed workflow run call
RH->>R: Get Allowlist
RH->>U: Update completed check run
RH->>H: Update completed workflow run on HUD
```
This diagram focuses on the re-run scenario:
1. After the upstream side triggers a workflow run re-run request from a check run, `webhook_handler` first reads the `Allowlist` and then dispatches the re-run request to the downstream side.
2. After the downstream side re-runs the workflow run, it uses `result_handler` to synchronize the `in progress` and `completed` states back to the upstream check run and HUD, which is almost the same as L2 and L3.
**Note**:
When a check run is created, the workflow run's `run_id` is stored in the payload's `external_id`. When a re-run is triggered, the corresponding workflow run can be found by looking up the `external_id` in the check run payload.
cc @fffrog @KarhouTam @ZainRizvi
3 条评论