<!-- Guide to writing tests for p5.js source code.  -->

# Unit Testing

Unit testing is an essential part of how a large codebase can stay relatively bug-free. Unit tests are small pieces of code that aim to test individual components of a larger code base for correctness. For example, making sure a function is doing what it should be doing or a class is creating its object instances correctly.

p5.js uses unit tests to ensure the correctness of its various functions. Unit tests also ensure that any new changes to the library’s code do not introduce bugs or unexpected behavior elsewhere, called regression. Tests for p5.js are run whenever we use the `npm test` command in the terminal.

This guide will walk you through the process of writing unit tests for p5.js. If you are implementing a new function, adding a new feature to an existing function, or changing the behavior of a function, you should implement the relevant unit tests.


## Prerequisites

- p5.js foundation
- Contributor guidelines with local development setup
- Looking inside p5.js


## Files and folders

All test-related files for p5.js are located in the `test` folder. We won’t go through everything in this folder for this guide; we will instead focus on the `unit` subfolder, which contains all the unit tests for p5.js.

As you can see, the subfolders in the `unit` subfolder roughly correspond to the subfolders in the `src` folder containing the source code of p5.js. At the same time, the files in each subfolder also have a counterpart in the subfolders in the `src` folder. p5.js is composed of many different modules with many different functions in each module. The aim is to have relevant unit tests for every public function that p5.js provides.


## Testing frameworks

p5.js 2.0 runs its tests with [Vitest](https://vitest.dev), which provides Mocha-compatible globals (`suite`, `test`, etc.). It is responsible for running the test code as well as providing a solid framework for reporting test results (i.e., the very long output you see in the terminal when you run the tests!)

However, Vitest still needs an assertion layer, and it bundles Chai for that. An assertion library is a collection of handy functions that lets us test various properties of our code, such as whether two values are equal, two values are of the same type, whether a function throws an error, and many more. p5.js uses [Chai's `assert` (and `expect`)](https://www.chaijs.com/api/assert/) to write individual statements about how the code should behave. You can also import `assert` and `expect` straight from Vitest (Vitest bundles Chai internally):

 ```js
 import { assert, expect } from 'vitest'
 ```


## Writing unit tests

To start writing unit tests, first pick a unit. A unit can be a function or a variable in p5.js. Let’s use `p5.prototype.keyIsPressed` as an example. Before beginning to write tests, we need to understand the expected behavior of this variable. The *expected behavior* is that the `keyIsPressed` variable should be `true` if any key is pressed and `false` if no keys are pressed. Now, you can think of various tests against this expected behavior. Possible test cases could be:

- the variable is a boolean
- it should be `true` if a key is pressed
- it should be `true` if any key is pressed - alphabet keys, number keys, special keys, etc.
- it should be `true` if multiple keys are pressed
- it should be `false` if no keys are pressed

In the example below, `suite()` and `test()` are both built-in functions provided by Vitest’s Mocha-compatible API. If you look into the test file (eg. `test/unit/events/keyboard.js`) you may find additional built-in functions such as `setup()` and `teardown()` as well. Each `suite()` describes a unit of p5.js that you are writing a test for (a function, a variable, etc). Each `test()` in a `suite()` is an individual test case that checks a single feature/behavior of the unit being tested. The first argument passed to `suite()` and `test()` is a description of the suite/test, and its purpose is to give clear output in the terminal for the test case.

- `p5.prototype.keyIsPressed` is the unit being tested in this suite.
- There are three tests in this suite:
  - The first test checks if `keyIsPressed` is a boolean value.
  - The second test checks if `keyIsPressed` is `true` on key press.
  - The third test checks if `keyIsPressed` is `false` when no keys are pressed

```js
suite('p5.prototype.keyIsPressed', function() {
  test('keyIsPressed is a boolean', function() {
    //write test here
  });

  test('keyIsPressed is true on key press', function() {
    //write test here
  });

  test('keyIsPressed is false when no keys are pressed', function() {
    //write test here
  });
});
```

We have structured our tests above but we haven't written the tests yet. We will be using Chai's `assert` for that. In the test file (`test/unit/events/keyboard.js`), you should see a variable called `myp5` defined near the top, and the `setup()` section contains a callback function that creates a new instance mode sketch.

```js
let myp5;

setup(function(done) {
  new p5(function(p) {
    p.setup = function() {
      myp5 = p;
      done();
    };
  });
});
```

The `p` parameter, which is used in instance mode to access various p5.js variables and functions, is then assigned to `myp5`. This setup allows you to use `myp5` to access p5.js variables and functions anywhere in the testing code to test their functionalities.

Remember, Vitest supplies the runner; Chai (bundled by Vitest) supplies the assertions. Consider the following:

```js
test('keyIsPressed is a boolean', function() {
  //Asserts that value is a boolean.
  assert.isBoolean(myp5.keyIsPressed);
});
```

Chai provides many different assertion functions that you can use to create the actual test code, assertion here in practice means functions that checks for a condition, similar to an if statement, and if the condition is `false`, it will throw an error. In the example above, we use Chai’s `assert.isBoolean()` function to check that the type of the value of `myp5.keyIsPressed` is indeed a boolean value. You can look at Chai’s [documentation](https://www.chaijs.com/api/assert/) for all the available assertion functions that you can use to test all kinds of things.

Now that you have written the tests, run them and see if the method behaves as expected. If not, consider whether the tested source code is correct or the test code is not testing the behavior of the code correctly.


## Adding unit tests

If you want to add more unit tests, look and see if there's already a test file for the component you want to add tests for. Generally, tests for a given file in `src` are at the same path under `test/unit`. For example, the tests for `src/color/p5.Color.js` are in `test/unit/color/p5.Color.js`.

If you have added a new source code file in the `src` folder and would like to add the corresponding test file in `test/unit`, you can make a copy of an existing test file and then rename it to match the file name of your source file. In the new test file, you should delete the old test code before inserting your own, keeping the test setup and teardown code.

```js
suite('module_name', function() {
  let myp5;
  let myID = 'myCanvasID';

  setup(function(done) {
    new p5(function(p) {
      p.setup = function() {
        let cnv = p.createCanvas(100, 100);
        cnv.id(myID);
        myp5 = p;
        done();
      };
    });
  });

  teardown(function() {
    myp5.remove();
  });
});
```

After adding a test file for a module to `test/unit`, you'll also need to add the module under test to the `spec` object in `test/unit/spec.js`. This will make sure the necessary modules are loaded for your test to run.

```js
// test/unit/spec.js
var spec = {
  // ...
  typography: ['attributes', 'loadFont', 'p5.Font', 'yourModule'],
  // ...
};
```

To add actual tests, which are grouped into what is called a “suite” (representing a function/variable that is being tested), we expand on the test code with the following:

```js
suite('module_name', function() {
  let myp5;
  let myID = 'myCanvasID';

  setup(function(done) {
    new p5(function(p) {
      p.setup = function() {
        let cnv = p.createCanvas(100, 100);
        cnv.id(myID);
        myp5 = p;
        done();
      };
    });
  });

  teardown(function() {
    myp5.remove();
  });

  suite('p5.prototype.yourFunction', function() {
    test('should [test something]', function() {
      // Your test code and Chai assertions
    });
  });

  // More test suites as needed
});
```

This is also how you will add tests for existing modules for which you didn’t need to create a new test file.


## Conventions

Here are the conventions and best practices that p5.js uses for unit tests which you should follow when writing your own tests:

- Use a single `suite` for each p5.js function/variable being tested.
- Each `suite` representing a function/variable can contain as many `test`s as needed.
- Each `test` should be self-contained and not rely on any other modules in p5.js.
- Test code should be as minimal as possible and only test one thing at a time. Do not test if a value is a function while also testing if it accepts the correct arguments in a single `test`; use two `test`s instead.
- Prefer Chai `assert` when possible instead of `expect`.


## Running tests

The most straightforward way to run the tests is by using the `npm test` command in your terminal. However, `npm test` usually takes a long time to run simply because of the large number of test cases p5.js has. We don’t need to run `npm test` over and over after each change. As soon as we save our code, we can see in the browser or terminal whether the tests pass or not.

This screenshot shows how the test executes on the website and in the terminal.

![Web test](../images/unit-visual-test.png) | ![Terminal test](../images/unit-visual-test-terminal.png)


- You can also mark certain test suites to be skipped or be the only suite that is run with the `.skip` and `.only` syntax.

  ```js
  // This test suite will not run
  suite.skip('p5.prototype.yourFunction', function() {

  });

  // Only this test suite will be run, all other suites will be ignored
  suite.only('p5.prototype.yourFunction', function() {

  });
  ```

- When you run `npm test`, Vitest launches two browser windows. The first window (usually at http://localhost:63315) renders and executes all unit and visual tests; it provides a built‐in interface where you can click on filters (for failed, skipped, passed tests, etc.) to narrow down the results. The layout in this window is self-explanatory. Each test suite and spec is listed with its status, and you can easily expand or collapse sections to see details, rerun individual tests, or inspect error messages. 

- The second window hosts Vitest’s DevTools/BiDi control channel, where you’ll see raw JSON frames exchanged between Vitest and the browser. That second window isn’t an extra test suite, it simply facilitates communication and result reporting, so you can safely ignore it unless you’re debugging Vitest’s runner itself.




## Visual tests

Visual tests are a way to make sure sketches do not unexpectedly change when we change the implementation of p5.js features. Each visual test file lives in the `test/unit/visual/cases` folder. Inside each file there are multiple visual test cases. Each case creates a sample sketch, and then calls `screenshot()` to check how the sketch looks.

```js
visualTest('2D objects maintain correct size', function(p5, screenshot) {
  p5.createCanvas(50, 50, p5.WEBGL);
  p5.noStroke();
  p5.fill('red');
  p5.rectMode(p5.CENTER);
  p5.rect(0, 0, p5.width/2, p5.height/2);
  screenshot();
});
```

To add a new test file, place it into `test/unit/visual/cases`. This will be auto-discovered by Vitest - no manual registration needed.

When you add a new test, running `npm test` will generate new screenshots for any visual tests that do not yet have them. Those screenshots will then be used as a reference the next time tests run to make sure the sketch looks the same. If a test intentionally needs to look different, you can delete the folder matching the test name in the `test/unit/visual/screenshots` folder, and then re-run `npm test` to generate a new one.

Running `npm test` launches Vitest’s interactive UI at http://localhost:63315, where you can view the list of all test cases including visual tests as well. 
In a continuous integration (CI) environment, optimizing test speed is essential. It is advantageous to keep the code concise, avoid unnecessary frames, minimize canvas size, and load assets only when essential for the specific functionality under test.
To address scenarios involving operations like asynchronous 3D model rendering, consider returning a promise that resolves upon completing all the necessary tests, ensuring efficiency in your visual testing approach. Here's an example of how you can asynchronous 3D model rendering in your visual tests:

```js
visualSuite('3D Model rendering', function() {
  visualTest('OBJ model is displayed correctly', function(p5, screenshot) {
    // Return a Promise to ensure the test runner waits for the asynchronous operation to complete
    return new Promise(resolve => {
      p5.createCanvas(50, 50, p5.WEBGL);
      // Load the model asynchronously
      p5.loadModel('unit/assets/teapot.obj', model => {
        p5.background(200);
        p5.rotateX(10 * 0.01);
        p5.rotateY(10 * 0.01);
        p5.model(model);
        // Take a screenshot for visual comparison
        screenshot();
        // Resolve the Promise to indicate completion
        resolve();
      });
    });
  });
});
```

## Visual tests in p5.js 2.0

_Both p5.js 1.x and p5.js 2.0 include visual tests. In p5.js 2.0, there are more tests, and the visual testing system uses a new diffing algorithm that is more robust. This section is this p5.js 2.0 specific system._


Different operating systems and browsers render graphics with subtle variations. These differences are normal and shouldn't cause tests to fail.
Common acceptable differences include:

* Single-pixel shifts in line positions
* Slight variations in anti-aliasing
* Text rendering differences (especially font weight and kerning)
* Minor differences in curve smoothness

For example, text rendered on macOS might appear slightly different from text rendered in a Linux CI environment. The same applies to thin lines, curves, and other graphical elements with anti-aliasing.
An example of this can be the below image which earlier caused tests to fail in CI because of different rendering environments.

![Example](./images/pixelmatch2.png)

The p5.js visual testing system uses a sophisticated algorithm to distinguish between acceptable rendering variations and actual bugs:

* Initial comparison - Compares pixels with a moderate threshold (0.5) to identify differences using [pixelmatch](https://github.com/mapbox/pixelmatch) library for pixel to pixel comparison.
* Cluster identification - Groups connected difference pixels using a Breadth-First Search (BFS) algorithm
* Pattern recognition - The algorithm specifically identifies:

  - "Line shift" clusters - differences that likely represent the same visual element shifted by 1px
  - Isolated pixel differences (noise)


* Smart failure criteria - Applies different thresholds:

  - Ignores clusters smaller than 4 pixels
  - Allows up to 40 total significant difference pixels
  - Permits minor line shifts that are typical across platforms

The below is the example of the tests that should fail:

![Example](./images/pixelmatch.png)



This approach balances sensitivity to real bugs while tolerating platform-specific rendering variations. The algorithm uses these key parameters:
```js
const MIN_CLUSTER_SIZE = 4;       // Minimum significant cluster size
const MAX_TOTAL_DIFF_PIXELS = 40; // Maximum allowed significant differences
```
The algorithm identifies line shifts by analyzing the neighborhood of each difference pixel. If more than 80% of pixels in a cluster have ≤2 neighbors, it's classified as a line shift rather than a structural difference.
This intelligent comparison ensures tests don't fail due to minor rendering differences while still catching actual visual bugs.

It's important to note that the improved algorithm described above allows tests with acceptable platform-specific variations to pass correctly. Tests that previously failed due to minor rendering differences (like anti-aliasing variations or subtle text rendering differences) will now pass as they should, while still detecting actual rendering bugs.
For example, a test showing text rendering that previously failed on CI (despite looking correct visually) will now pass with the improved algorithm, as it can distinguish between meaningful differences and acceptable platform-specific rendering variations. This makes the test suite more reliable and reduces false failures that require manual investigation.

### Some best practices for writing visual tests

When creating visual tests for p5.js, following these practices will help ensure reliable and efficient tests:

* Keep canvas sizes small - Use dimensions close to 50x50 pixels whenever possible. The test system resizes images for efficiency before comparison, and smaller canvases result in faster tests, especially on CI environments.
* Focus on visible details - At small canvas sizes, intricate details may be hard to distinguish. Design your test sketches to clearly demonstrate the feature being tested with elements that are visible at the reduced size.
* Use multiple screenshots per test - Instead of cramming many variants into a single screenshot, call screenshot() multiple times within a test:
  ```js
  visualTest('stroke weight variations', function(p5, screenshot) {
    p5.createCanvas(50, 50);
    
    // Test thin stroke
    p5.background(200);
    p5.stroke(0);
    p5.strokeWeight(1);
    p5.line(10, 25, 40, 25);
    screenshot(); // Screenshot with thin lines
    
    // Test thick stroke
    p5.background(200);
    p5.strokeWeight(5);
    p5.line(10, 25, 40, 25);
    screenshot(); // Screenshot with thick lines
  });
  ```   
