Adding tests to existing specs

In this tutorial we'll help you add tests to an existing spec.

After creating a spec and building the code, you can build out a test suite by introducing @test links into your spec.

This can be a good workflow when you're keen to get some working code to play with as quickly as possible, and then you want to firm it up after to make it easier to make changes in future without regressing the behavior.

In this example we start with a spec for a spaced repetition algorithm. There are no tests, just detailed requirements that describe the expected behaviour:

Spec without test links, for spaced repetition algorithm

# Spaced Repetition System (SM2)

Implements the SM2 (SuperMemo 2) spaced repetition algorithm for calculating optimal review intervals for learning items.

## Target

[@generate](./srs.py)

## Capabilities

### Calculate SM2 algorithm updates

Takes an item with current SRS values and a grade, returns updated values according to SM2 algorithm.

- Validates grade is between 0 and 5 (inclusive)
- For grades 3-5 (correct responses): progresses repetition count and calculates new interval
- For grades 0-2 (incorrect responses): resets repetition to 0 and interval to 1
- Updates easiness factor based on grade performance
- Calculates next review date by adding interval days to current date

### Handle repetition progression

For successful reviews (grade >= 3):

- First repetition (repetition == 0): sets interval to 1 day
- Second repetition (repetition == 1): sets interval to 6 days
- Subsequent repetitions: interval = round(previous_interval \* easiness_factor)
- Increments repetition count

### Calculate easiness factor

Updates easiness factor using SM2 formula: max(1.3, easiness + (0.1 - (5 - grade) _ (0.08 + (5 - grade) _ 0.02)))

- Ensures minimum easiness of 1.3
- Rounds result to 2 decimal places

### Calculate next review date

Adds interval days to current date to determine when item should be reviewed next.

## API

```python { .api }
from datetime import date
from typing import Dict, Any, Union

def sm2(item: Dict[str, Any], grade: int) -> Dict[str, Union[int, float, date]]:
    """
    Calculate SM2 spaced repetition values for a learning item.

    Args:
        item: Dictionary containing current SRS values with keys:
            - repetition (int, optional): Current repetition count, defaults to 0
            - easiness (float, optional): Current easiness factor, defaults to 2.5
            - interval (int, optional): Current interval in days, defaults to 1
        grade: Integer between 0-5 representing quality of response
            - 0-2: Incorrect response (resets progress)
            - 3-5: Correct response (advances progress)

    Returns:
        Dictionary with updated SRS values:
            - repetition (int): Updated repetition count
            - easiness (float): Updated easiness factor (rounded to 2 decimal places)
            - interval (int): Updated interval in days
            - next_date (date): Next review date

    Raises:
        ValueError: If grade is not between 0 and 5
    """
```

Add test links to the spec

But now we decide we want to add in test cases. To do this we will simply edit the spec and prompt to include test links. In this example we'll ask this in an open ended way, but we could extend the prompt to ask only for certain behaviors to be tested - for example, we may not want to trigger tests that will need very complex mocking to verify.

Ask your agent to add test links.

Lets add test links to the spec

This should trigger the agent to use the edit spec MCP tool, and ask for test links to be added. The agent may choose to make a detailed prompt or it may just ask for test links.

The updated spec now includes @test links next to requirements. These point to the files that will contain the tests once generated.

Spec with test links included

````markdown # Spaced Repetition System (SM2)

Implements the SM2 (SuperMemo 2) spaced repetition algorithm for calculating optimal review intervals for learning items.

Target

@generate

Capabilities

Calculate SM2 algorithm updates

Takes an item with current SRS values and a grade, returns updated values according to SM2 algorithm.

Validates grade is between 0 and 5 (inclusive) @test
For grades 3-5 (correct responses): progresses repetition count and calculates new interval @test
For grades 0-2 (incorrect responses): resets repetition to 0 and interval to 1 @test
Updates easiness factor based on grade performance @test
Calculates next review date by adding interval days to current date @test

Handle repetition progression

For successful reviews (grade >= 3):

First repetition (repetition == 0): sets interval to 1 day @test
Second repetition (repetition == 1): sets interval to 6 days @test
Subsequent repetitions: interval = round(previous_interval * easiness_factor) @test
Increments repetition count @test

Calculate easiness factor

Updates easiness factor using SM2 formula: max(1.3, easiness + (0.1 - (5 - grade) _ (0.08 + (5 - grade) _ 0.02)))

Ensures minimum easiness of 1.3 @test
Rounds result to 2 decimal places @test

Calculate next review date

Adds interval days to current date to determine when item should be reviewed next. @test

API

from datetime import date
from typing import Dict, Any, Union

def sm2(item: Dict[str, Any], grade: int) -> Dict[str, Union[int, float, date]]:
    """
    Calculate SM2 spaced repetition values for a learning item.

    Args:
        item: Dictionary containing current SRS values with keys:
            - repetition (int, optional): Current repetition count, defaults to 0
            - easiness (float, optional): Current easiness factor, defaults to 2.5
            - interval (int, optional): Current interval in days, defaults to 1
        grade: Integer between 0-5 representing quality of response
            - 0-2: Incorrect response (resets progress)
            - 3-5: Correct response (advances progress)

    Returns:
        Dictionary with updated SRS values:
            - repetition (int): Updated repetition count
            - easiness (float): Updated easiness factor (rounded to 2 decimal places)
            - interval (int): Updated interval in days
            - next_date (date): Next review date

    Raises:
        ValueError: If grade is not between 0 and 5
    """

{% endcode %}

</details>

## Generating and running tests

We can now ask our agent to build the tests and run them. &#x20;

`

``
Great! Now build the tests and run them
```

This should cause the agent to call the build-tests tool on the spec and to pass the test command.  This will create the new test files and run the test command.  We'll be able to see if the tests are passing.

<figure><img src="../.gitbook/assets/Screenshot 2025-09-15 at 19.18.14 (1).png" alt=""><figcaption></figcaption></figure>

## Getting tests to pass

Work with your agent to iterate until you have working code and a working test suite.&#x20;

This may take multiple iterations, especially if you're first setting up a new project and the set up needs to be refined.

```
Help me get the tests to pass
```

You may see the agent reach to `build` the code or tests again.  If the `build` or `build-tests` command is called with both `testCommand` and `maxIterations` , then the tools will iterate on the code or tests against the test results until they work.

If your tests timeout, either due to the entire test process taking too long, or the tests hanging, you can re-run the `test` command and configure how long you want to wait. Use `timeoutSecs` to configure the overall timeout, and/or `inactivityTimeoutSecs` to configure how long to wait before timing out due to inactivity.

## Lock tests that are passing

Once you have a set of tests that are passing and that you think are useful, you can apply locked tags to them to signal to agents that you do not want these behaviours to change as you extend functionality.

Ask the agent to 'lock' the tests, or to 'add locked tags' to the tests.

```
Lock the tests in the spec
```

The agent should call the `test` tool with `lockTests` as true.

<figure><img src="../.gitbook/assets/Screenshot 2025-09-15 at 19.40.34.png" alt=""><figcaption></figcaption></figure>

The spec with locked tests will look like this:

<details>

<summary>Locked tests in the spec</summary>

<div data-gb-custom-block data-tag="code" data-overflow='wrap'>

```markdown
### Calculate SM2 algorithm updates

Takes an item with current SRS values and a grade, returns updated values according to SM2 algorithm.

- Validates grade is between 0 and 5 (inclusive) [@test](../tests/test_sm2_validation.py) { .locked }
- For grades 3-5 (correct responses): progresses repetition count and calculates new interval [@test](../tests/test_sm2_correct.py) { .locked }
- For grades 0-2 (incorrect responses): resets repetition to 0 and interval to 1 [@test](../tests/test_sm2_incorrect.py) { .locked }
- Updates easiness factor based on grade performance [@test](../tests/test_sm2_factor.py) { .locked }
- Calculates next review date by adding interval days to current date [@test](../tests/test_sm2_date.py) { .locked }
```

</div>

</details>

The test links that are locked will have a `{ .locked }` tag.



You are now ready to continue adding functionality to your app, with the confidence that your agent will keep the behaviour of the tests you have locked.

PreviousBuilding a multi-spec project NextIntegrating Tessl into Test-Driven Development workflows

Last updated 1 month ago