> For the complete documentation index, see [llms.txt](https://docs.tessl.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tessl.io/improving-your-skills/troubleshooting-evals.md). # Troubleshooting evals ## Permissions In order to run an evaluation, your plugin.json must reference a workspace you have access to, and has *publisher*, *manager* or *owner* permissions for. For example if the plugin specifies `"name": "local/"`, change *local* to a workspace you have the correct permissions to. It does not need to be published, however if you do publish, note that specifies to what [workspace](/introduction-to-tessl/organizations-workspaces-and-roles-for-users.md) to publish to. ## Tips when using Agent or CLI Folder structure and references are important! Check that your ./evals is in root of your plugin folder. ``` / ├── .tessl-plugin/ │ └── plugin.json └── skills//SKILL.md ├── evals/ │ └── scenario-1/ │ ├── task.md │ ├── criteria.json │ └── scenario.json # optional: fixtures, includes, setup scripts │ └── scenario-2/ ``` Check the *`.tessl-plugin/plugin.json`* and confirm that `"skills": []` has a reference to your skill's directory, in relation to the plugin folder root. For example, * If SKILL.md is in *`skills/codereviewer`*, you would have `"skills": [ "./skills/codereviewer" ]` * If SKILL.md is in the plugin root, you would have `"skills": [ "." ]` , this often occurs when an `tessl skill import` is used on an existing skill. The ./tessl-plugin folder and plugin.json are placed in the same folder. When you downloaded the Evals folder, did you download it at your plugin root, where the `.tessl-plugin` folder is located? If not, move it. ### Scenario: No "with context displayed" in your evaluation When executing `tessl eval run .` , run from plugin root folder, and be sure both the evals, .tessl-plugins folders are in the root of the plugin folder. Check the reference to the skill location in .tessl-plugin/plugin.json->"skills":\[] reference,
## Tips using the web interface ### Review the scenarios themselves You can do this locally if you generated the evals, or the using the webinterface, you can review the scenarios being tested, by clicking the **Evals** tab and click **Details**.\ \ Review the scenarios to determine if they are correct by clicking the arrows to scroll through them. If the scenarios are incorrect, update or rewrite them, and then republish.\ ![](/files/c3uqHSTjbs9J9RyPmIXe) ### Review what happened during the evaluation You can review the outputs of the agent. Navigate to the **Evals** tab on the web interface and click **Download**. You can review the outputs. 1. Do you see the expected behavior in the "with context" example? 2. If not, recheck the scenario. If the scenario looks right, check activation by checking Quality tab for activation issues in the review, or run `tessl skill review` locally. 3. Consider using verifiers and using [agent session analysis.](https://tessl.io/blog/audit-log-evaluations/)\ ### Scenario: "With context" has unexpected result
#### Premise The context provided should have performed an action that could only work with the skill present, yet you see 0% shown. This can occur when the scenario may not have activated or perhaps the test didn't result in the expected behaviour, review/edit your scenario, review the output! #### Steps to resolve 1. Review scenario, does it have the intention of what you expected? 2. Confirm with Review tab that activation isn't an issue 3. Download context and review if it's there. If not, see [Agents and CLI tips](#tips-when-using-agent-or-cli) above, it's likely the skill and evals were not where expected. Is the plugin.json location correct? 4. Adjust scenario(s) as needed --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.tessl.io/improving-your-skills/troubleshooting-evals.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.