Support checking exception order in `RaisesGroup`
#### What's the problem this feature will solve?
Some tests need to assert that an exception group was raised with multiple exceptions in a particular order. Currently, making such an assertion about the group must be done somewhat manually, e.g. like
```python
def assert_matches(
exception: BaseException, raises: _pytest.raises.AbstractRaises[BaseException]
) -> None:
assert raises.matches(exception), raises.fail_reason
def check_group(exc_grp: ExceptionGroup[Exception], /) -> bool:
assert len(exc_grp.exceptions) == 2
assert_matches(
exc_grp.exceptions[0],
pytest.RaisesExc(RuntimeError, check=check_body_exc),
)
assert_matches(
exc_grp.exceptions[1],
pytest.RaisesExc(RuntimeError, check=check_aclose_exc),
)
return True
with pytest.raises(ExceptionGroup, check=check_group):
...
```
or similarly via `exc_info`:
```python
with pytest.raises(ExceptionGroup) as exc_info:
...
assert many_properties_of(exc_info.value) == ...
```
#### Describe the solution you'd like
`raises` and `RaisesGroup` have made asserting expected exceptions much more convenient and less mistake-prone than the old
```python
try:
...
except ExpectedExceptionType as exc:
assert some_property(exc) == ...
else:
assert False
```
and
```python
with pytest.raises(ExceptionGroup) as exc_info:
...
assert enough_properties_about_every_exc_in(exc_info.value) == ...
```
It would be convenient if `RaisesGroup` also supported cases where the group has an expected order.
In more detail, the proposal is: add an `ordered: bool = True` parameter to `RaisesGroup`. Setting `ordered=False` would cause `RaisesGroup` to use a simpler matching algorithm that does not attempt to reorder exceptions and retry `AbstractRaises.matches` on other children when a `.matches` on a child fails.
For example, two real-world use cases:
* We test a class that builds a `BaseExceptionGroup` containing any exception from the body of its `async with` and an exception raised during a cleanup operation in `__aexit__`. This assertion was being written as
```python
with pytest.RaisesGroup(
pytest.RaisesExc(RuntimeError, check=check_body_exc),
pytest.RaisesExc(RuntimeError, check=check_aclose_exc),
):
...
```
until we realized that this wasn't asserting the order and we needed to assert the order explicitly.
* We test various error, exception, and shutdown semantics of a de/multiplexer class that has a task group containing two [service tasks](https://tricycle.readthedocs.io/en/latest/reference.html#tricycle.open_service_nursery), can propagate an exception from the user (the body of the object's `async with`), can raise socket, TLS, and protocol state machine exceptions in the `_serve_outgoing` and `_serve_incoming` service tasks, and can raise another exception during a final shutdown operation in `__aexit__` after the service task group exits earlier in `multiplexer.__aexit__`. There are various test cases that assert that `async with multiplexer: stuff()` raises the correct exceptions in the correct order for that case.
#### Alternative solutions
It's perfectly possible to make the necessary assertions manually without using `RaisesGroup`, similar to how expected `ExceptionGroup`s were more often asserted before Trio added `RaisesGroup`, e.g.
```python
with pytest.raises(BaseExceptionGroup) as exc_info:
...
assert len(exc_info.value.exceptions) == 3
assert isinstance(exc_info.value.exceptions[0], BrokenResourceError)
assert isinstance(exc_info.value.exceptions[0].__cause__, ...)
assert isinstance(exc_info.value.exceptions[1], RuntimeError)
assert property_a(exc_info.value.exceptions[1]) == ...
assert isinstance(exc_info.value.exceptions[2], RuntimeError)
assert property_b(exc_info.value.exceptions[2]) == ...
```
`assert_matches(exc_info.value.exceptions[i], pytest.RaisesExc(...))` can sometimes simplify this a bit, but it's still a bit tedious and too easy to forget to assert the length and/or to forget to make assertions about one or more of the `exc_info.value.exceptions`, allowing them to slip through. It's a somewhat similar issue to `group_contains`: https://github.com/pytest-dev/pytest/issues/11538#issuecomment-1914533421.
Aside: while it is true that when using `with AbstractRaises`, using a shorter body of the `with` is preferrable (see: flake8-bugbear B908), in cases where the exception group gets raised out of a class's `__aexit__` there isn't much of another choice in order to test the behavior. flake8-bugbear won't even yell about `with big_AbstractRaises: async with thing: a_bunch_of_stuff()`, because there _is_ technically only one top-level statement within `big_AbstractRaises`'s context.
0 条评论