# Accessibility in DVUI

## What is accessibility?

Most modern operating system provide an accessibility API to assist users who cannot use standard computer I/O devices to interact with applications.  See the "Accessibility" section of [README](README.md) for additional accessibility concerns.

These users often interact via 'screen reader' that can read text and other accessibility labels from the application. Some readers will also provide the user with a way to perform actions on an application, such as clicking a button, or setting the text in a text box. Others will rely on the keyboard navigation features provided by the application.

DVUI implements this part of accessibility through the [AccessKit](https://github.com/AccessKit/accesskit) toolkit, which provides a common interface to the operating system-specific accessibility APIs.

While DVUI tries to make this easy to integrate, some effort at the application level is required:
* adding accessibility labels to images and icons
* adding accessiblity labels to entry widgets such as textboxes and groups of widgets. See [Labeling](#labeling) for more details.
* ensuring visible labels are read for the correct content
* testing tab ordering and keyboard navigation
* general testing with screen readers (see below)

## Benefits and Downsides

The primary benefit is making your appliation useable by a wider array of users.

Incorporating accessibility makes your UI application scriptable. Each widget that has a node in the accessibility tree publishes its current value along with the set of actions it can perform. By using your platform's accessibility API, you can write automated tests as well as allowing your users to script common tasks.

Outside of the additional executable size from including the library, enabling AccessKit support has very little impact on runtime performance or memory requirements. 

The overhead of creating AccessKit nodes is only incurred when the operating system's accessibility API is activated by a screen reader or similar application.

## Enabling AccessKit in DVUI

AccessKit can be included as either a static or dynamic link library. Use `-Daccesskit=static` or `-Daccesskit=shared` to enable AccessKit support. 

Not all combinations of backends and operating systems are currently supported. 
- Linux - Supports all backends in for shared and static libraries. SDL2, SDL3 and Raylib.
- Windows - Support for all backends, except raylib static due to symbol clashes.
- MacOS - Supports all backends in for shared and static libraries. SDL2, SDL3 and Raylib.
- Web - Not currently supported by AccessKit but it is planned. 

When you compile you application with AccessKit enabled, most of the work is taken care of for you and your users will gain most of the benefits of accessibility support. However, DVUI doesn't know your application's intent or any semantic details about the UX, so it is not possible to automate everything. 

As examples, any images should be given an accessible label by passing the .label option. When placing radio button in a radio group it is also helpful to supply a label to the group.

There are other AccessKit APIs that let you provide more details to the accessibility API. See [Widget users](#widget-users) for more details on how to use the AccessKit API.

Example of setting a label for an image:
```
    const image_source: dvui.ImageSource = .{ .imageFile = .{ .bytes = zig_favicon, .name = "zig favicon" } };
    const imgsize = dvui.imageSize(image_source) catch dvui.Size.all(50);
    _ = dvui.image(@src(), .{ .source = image_source }, .{
        .gravity_y = 0.5,
        .min_size_content = .{ .w = imgsize.w + icon_image_size_extra, .h = imgsize.h + icon_image_size_extra },
        .rotation = icon_image_rotation,
        .label = .{ .text = image_source.imageFile.name },
    });
```

## Testing for accessibility

As all accessibility APIs are operating system specific, different tools are required to test on each platform.

### Testing accessibility on Windows

The basic test you should do is to open the Narrator using `ctrl-win-enter` (The same keyboard shortcut closes it). Perform the following steps:
1) Make sure the reader highlights a widget when you first click on the window. 
    - If no widget is focused and the whole window stays highlighted, you need to default the focus to a widget whenever the window is first opened using `dvui.focusWidget()` or similar.
2) Press `caps-r` to read the screen. The reader should read all controls and their displayed content.
    - If the reader is reading from controls that should be skipped (e.g. the down triangle in a custom dropdown), set .role = .none in the widget creation options for the image.
    - If the reader is skipping a widget that should be added, set .role to an appropriate value for that widget. 
    - Check that fields are labelled correctly (e.g. associating a label for a text box). DVUI has some heuristics for this but use the .label = .for or .by in the widget options to properly associate labels if needed.
3) Tab through each widget. 
    - Make sure each focusable widget has an appropriate `.tab_index` (either manually assigned or generated) and has a sensible position in the tab order.
    - Make sure the reader highlights each widget and reads out the correct value.
4) Pay special attention to images and icons. Make sure to add a `.label` option to give descriptive labels to these widgets.

The above steps only test reading from the screen. Actions can also be tested.
1) Download, install and run [Accessibility Insights](https://accessibilityinsights.io/)
2) Click on each widget
3) Look in the bottom right pane for a list of values and actions applicable to each widget.
    - If you don't see an action for an actionable widget and it is a DVUI widget, please file an issue.
    - If it is your widget, use `AccessKit.nodeAddAction()` and other relevant API to add the appropriate actions.
    - The best way to understand what actions are available for which roles is to look at the AccessKit source code.

See [AccessKit - Tips for application developers](https://github.com/AccessKit/accesskit/blob/main/README-APPLICATION-DEVELOPERS.md)

### Testing accessibility on MacOS

See [AccessKit - Tips for application developers](https://github.com/AccessKit/accesskit/blob/main/README-APPLICATION-DEVELOPERS.md)

### Testing accessibility on Linux

See [AccessKit - Tips for application developers](https://github.com/AccessKit/accesskit/blob/main/README-APPLICATION-DEVELOPERS.md)

## DVUI and AccessKit Details

AccessKit is an accessibility library written in Rust that provides a unified API to the underlying operating system specific accessibility APIs.

AccessKit requires:
1) A tree of "node ids" representing the parent / child relationships and the roles of each node.
2) A set of updates to those nodes, which contain any data that was changed since the previous update.

As DVUI does not keep state for all widget values, a full set of node id's and a full set of updates is sent at the end of each frame.

The DVUI WidgetId is used as the AccessKit node id. When a widget is created, if 1) Accessibility is active, 2) The Widget has a role and it is not .none, 3) The widget is at least partially visible or the widget is the focused widget, then a new AccessKit node will be created via `dvui.accesskit.nodeCreate()`.  Widgets will then set any values or actions against the created node.

The visibility check should be removed in future and the nodeSetClipsChildren API should be used for anything that clips contained child widgets. The focused node is always added to the tree, even if it is not visible, meaning it can have the wrong parent. But this stops the screen reader's focus from shifting to the main window when the focused widget scrolls off the screen.

Note: AccessKit is still a relatively new and evolving library. Not all platform accessibility features are supported and support for some features may be partially implemented.  Please file any issues!

### Multithreading requirements

There is the potential for any callback from AccessKit to be called on the non-gui thread. This includes the initialTreeUpdate, the frameTreeUpdate and the actionHandler. Which thread these are called on varies by operating system, so you should assume they will be called on a separate thread.

Any direct access to the `currentWindow().accesskit` variables should be protected by `currentWindow().accesskit.mutex`. 
`accesskit.nodes` is safe for `.get` access if used from the gui thread.

### Actions

Actions are input events from the OS accessibility API to perform some function in the GUI, such as focusing a widget, clicking a button or setting a value for a widget. See `AccessKit.Actions` for a full list.

Actions are collected for the current frame in the `action_requests` ArrayList. 
At the start of the next frame, these action requests are converted to the relevant DVUI events for the requested widget.
Values are set via the 'text' event with the `replace` field set to true to indicate that the value should replace any existing value held by that widget.

Currently supported Actions are:
* click
* set_value (text and numeric types are supported)
* focus

It is planned to support scrolling actions once the read-only issue with AccessKit scrollbars is resolved.

### Widget creators

Each widget's `install()` should check if an accessibility node was created by using `self.data().accesskit_node()` or equivalent. If this returns non-null, AccessKit functions can be used on the returned node to set any values and events for that widget. Common functions are:
- `nodeSetValue` / `nodeSetNumericValue` - Used to set the value to be read out by the reader. See AccessKit source code for details on what values can be set for each role.
- `nodesetToggled` - For checkboxes, radio buttons etc.
- `nodeAddAction` - Common actions are .click, .focus, .toggle. See AccessKit source code for the actions available for each role.

If your widget can have a value set, then make sure to handle the `text` event to allow that value to be set via a set_value action.

It is safe to use the `accesskit_node()` for a widget that has been deinitialized. e.g. the following code will work. 
```
var label_wd: WidgetData = undefined;
dvui.labelNoFmt(@src(), message, .{}, .{ .background = true, .corner_radius = dvui.Rect.all(1000), .padding = .{ .x = 16, .y = 8, .w = 16, .h = 8 }, .data_out = &label_wd });
if (label_wd.accesskit_node()) |ak_node| {     
    AccessKit.nodeSetLive(ak_node, AccessKit.Live.polite);
}
```

For cases where you only have access to the widget id. the AccessKit node can be obtained from the `nodes` collection of the `AccessKit` struct. e.g.
```
var label_id: Id = ...;
{
    if (currentWindow().accesskit.nodes.get(label_id)) |ak_node| {
        AccessKit.nodeSetLive(ak_node, AccessKit.Live.polite);
    }
}
```

### Widget users

The AccessKit APIs are also available for widget users to customize accessibility information. As above, the AccessKit node can be accessed either from `accesskit_node()` on widget data, or via the `nodes` collection through `currentWindow().accesskit`

An example of using a text box as a number entry box and adding minimum and maximum values. (Note that dvui.textEntryNumber already does this for you)

```
const te = dvui.textEntry(@src(), .{}, .{ .role = .number_input});
if (te.data().accesskit_node()) | ak_node | {
    // Remove the text value already set by textEntry
    AccessKit.nodeClearValue(ak_node);  
    AccessKit.nodeSetMinNumericValue(ak_node, 0);
    AccessKit.nodeSetMaxNumericValue(ak_node, 128);
    // Set the numeric value.
    AccessKit.nodeSetNumericValue(ak_node, std.fmt.parseInt(te.getText()) catch 0);
}
```

#### Labeling
Labellng is one of the most important things you can do to make your appliction more accessible. Widgets can be labelled by setting the `.label` via options. Labels should give the screen reader enough context to determine that widget's function.

For example, if a user moves focus to a text entry, the text box should be labeled with its purpose. Typically, this will the contents of the label widget preceding the text entry.

DVUI offers the following labeling options:
   * text - set label directly
   * by_id - Pass the id of the label widget containing the label
   * for_id - This label widget labels another widget.
   * label_widget = .prev - This widget is labeled by the last created label widget.
   * label_widget  = .next - This widget is labeled by the next created label widget.

Alternatively, you can also use the `AccessKit.nodeSetLabel` and `AccessKit.nodeSetLabeledBy` functions.

#### Text
Text created through the TextLayoutWidget will make the text accessibile to the screen reader through a text_run role. This is true for any widget that uses TextLayoutWidget to create text, e.g. TextEntryWidget.

Each call to add_text creates a new text_run for each line or partial line of text added. 

To make your text compatible with text_runs, you should:
1) If there are no font/style changes, use addText() to add one or more full lines, including any trailing newline.
2) If there are fonts/styles changes, use a different addText() for each font/style change.
3) Do not call addText() just to add a newline, unless it is for a blank line, only containing a newline.

Ignoring these rules will not break anything, but will make it harder for accessiblilty users to correctly navigate such text in their screen reader.

## Current State of Accessibility in DVUI

* A role of `null` means the widget will not be added to the accessibility tree, unless the user passes a `role` for that widget via Options.

| Widget | Role | Read Support | Action Support | TODO? | Details |
|--------|------|--------------|----------------|-------|---------|
| AnimateWidget| null | Basic | N/A | N | User can pass .role and .label via options |
| BoxWidget | null | Basic | N/A | N | User can pass .role and .label via options |
| ButtonWidget | button | Yes | Yes | N | Focus and Click actions supported |
| ColorPickerWidget | slider x 4 | Partial | Partial | Y | AccessKit does not currently support 2d-sliders. Other sliders are supported. |
| DropDownWidget | combo_box / list_item | N | Y | Y | Supported. Add expand/collapse when supported by accesskit |
| FlexBoxWidget | null | Basic | N/A | N | User can pass .role and .label via options |
| FloatingMenuWidget | none | N/A | N/A | N | Accessibility Handled by Menu / MenuItem |
| FloatingToolTipWidget | tooltip | Y* | N | N | Tooltips are added only when shown. No support for showing / hiding tooltips |
| FloatingWidget | N/A | N/A | N/A | N | |
| FloatingWindowWidget | dialog | Y | Y | N | Defaults to .dialog. User should override with .window if more appropriate. |
| GridWidget | grid, header, row, cell | Y | N/A | N? | Needs more real-world testing.  |
| IconWidget | image |  Y | N/A | N | |
| LabelWidget | label |  Y | N/A | N | | 
| MenuItemWidget | menu_item | Y | Y | Y | Add keyboard shortcuts when supported by dvui | 
| MenuWidget | menu |  Y | Y | N | |
| OverlayWidget | null | Basic | N/A | N | User can pass .role and .label via options |
| PanedWidget | pane |Y | N* | Y | Accesskit currently sets the control as read-only. If this is fixed, implement .text event to set the splitter position. |
| PlotWidget | .group, .image | N | N/A? | N | Labels image with title of plot. Labels plot widget as a plot. |
| ReorderWidget | null | N | N | Y | Not currently supported. Unlikely that interaction will work? | 
| ScaleWidget | null | N/A | N/A | N | Not required |
| ScrollAreaWidget | scroll_view | Y | N | N | Appears as a pane |
| ScrollBarWidget | scroll_bar | Y| N* | Y | Due to bug in AccessKit, is set to read-only and cannot interact.  | 
| SuggestionsWidget | suggestion, list_item | Y | Y | N | 
| TabsWidget | .tab_panel, .tab | Y | Y | N | Sets active tabs and allows tab selection |
| TextEntryWidget | text_input, multiline_text_input | Y* | N* | Y |  Allows selection and navigation. Cached layouts needs more investigation. Needs to implement the text_replace action. Scrolling for off-screen text requires improvement. |
| TextLayoutWidget | label | Y* | N/A | Y | Most features supported. TextRun support implemented for text. Scrolling for off-screen text requires improvement. Needs to split lines of more than 255 chars into multiple text runs | 
| TreeWidget | tree, tree_item | Y* | Y | Y | Adds tree and nodes. Implement expand / collapse when supported by AccessKit | 
| windowHeader | label, button | Y | Y | N | |
| dialogs | window | Y | Y | N | Same as floating window |
| toasts | label | Y | N/A | N | These are set as polite annoucements. |
| comboBox | combo_box | Y | Y | Y | Displays as combo box and shows list items when dropped. Add expand / collapse when supported by accesskit |
| expander | group | Y* | Y* | Y | AccessKit does not currently support expand / collapse |
| context | ? | ? | ? | ? | Will need to implement "show context menu" action. Which will be mapped to right-click. |
| gridHeadingSortable | button | Y* | Y | Y | Sets sort state of asc/desc if sorted, but does not come through in accessibility insights. requires accesskit support. |
| gridHeadingCheckbox | button | Y | Y | N| Labels checkbox as select all / select none. |
| image | image | Y* | N/A | N | Requires user to label the image with .label |
| slider | slider | Y | Y | N | Fully supported |
| progress | progress_indicator | Y | N/A  | N | Fully supported |
| checkbox | check_box | Y | Y | N | Fully supported |
| radio | radio_button | Y* | Y | N | Best practice: Create a radio group with a surrounding radio group widget|
| textEntryNumber | number_input | Y | Y | N | Fully supported. Supports min, max and valid/invalid. |

