docs: add testing guide (how to add/run OpenAF tests)

[Copilot]

Adds docs/testing.md — a comprehensive, contributor-focused guide on how to write, add, and run tests for OpenAF functions and modules.

What's included

• Purpose — overview of what the doc covers
• Where tests live — tests/ directory structure table covering YAML orchestrators, JS test files, results JSON, JUnit XML, Markdown report, and SVG badges
• Test framework basics — ow.test API reference (assert, profiling, counters) with a corrected minimal example; clarifies that ow.loadTest() is called once by the orchestrator framework and must not be repeated inside individual test functions
• Naming conventions — file and export naming patterns, how tests are discovered via explicit YAML job entries
• Orchestrator YAML example — mirrors the structure of tests/autoTestAll.ZIP.yaml; shows how to register a new area in autoTestAll.allJobs.yaml
• Run tests locally — build steps, ojob/java -jar invocations, and sh oafTest.sh usage
• CI and Maven integration — pom.xml exec-maven-plugin snippet, how oafTest.sh integrates with mvn test
• Step-by-step guide — five steps from creating a JS file to reviewing results and updating autoTestAll.md
• Copy-paste template — minimal JS test file + companion YAML ready to adapt
• Troubleshooting — missing results, permissions, network-dependent tests, single-file debugging
• Links and references — table linking all relevant repo files

Testing

• ✅ Code review passed — no issues found
• ✅ CodeQL check passed — no issues found

Original prompt

Create a new documentation page under docs/ that explains how to write, add, and run tests for OpenAF functions and modules. The new doc should be clear, actionable, and include copy-paste examples and templates so contributors can easily add tests that integrate with the repo's existing test framework and CI. The PR should add a single file at docs/testing.md (or docs/howto-testing.md — choose docs/testing.md) with the following sections and content:

Title: Testing OpenAF: how to add and run tests

Sections and required content:

1. Purpose

• One-paragraph explanation: this doc shows how to create JS tests and orchestrator YAML to test OpenAF functions, and how to run them locally and in CI.

2. Where tests live

• Explain tests/ directory holds test JS files and autoTestAll.*.yaml orchestrators.
• Mention tests/autoTestAll.md shows results and tests/*.svg are badges.
• Reference oafTest.sh and autoTestAll.yaml as orchestrator used by CI and local runs.

3. Test framework basics

• Brief explanation of ow.test utilities (ow.test.assert, ow.test.setOutput, counters) as used in docs/openaf.md Testing Framework section.
• Show minimal test example file (copy-paste) with header and a simple test function using ow.test.assert and export naming convention (exports.test = function() { ... } or show pattern used in repo such as exports.testZIP = function() { ... } ). Use an example for testing a hypothetical function myFunc in module X.

4. Naming conventions and structure

• Filename suggestions: autoTestAll..js for grouped tests; smaller per-feature files OK.
• Export function names pattern and how tests are discovered/run by the orchestrator (explain that orchestrator YAML calls the test JS files or that ojob autoTestAll.yaml loads files — be explicit about how existing tests are structured: they define exports.* and the orchestrator runs them). If exact discovery mechanism is unclear, state contributors should follow the patterns in existing tests (link to tests/autoTestAll.js and area-specific files).

5. Orchestrator YAML example

• Provide a sample minimal autoTestAll..yaml (or example snippet) showing how to run the JS tests, produce autoTestAll.results.json, and generate badges/SVG.
• Reference how to hook into ojob (commands used in BUILD.md and AGENTS.md): ojob autoTestAll.yaml and java -jar ../openaf.jar --ojob -e autoTestAll.yaml.

6. Run tests locally

• Steps to run: build (ojob build.yaml), then cd tests && ojob autoTestAll.yaml OR cd tests && java -jar ../openaf.jar --ojob -e autoTestAll.yaml. Also show sh oafTest.sh usage.
• Explain that the script installs the built JAR into _oaf, runs tests, and checks autoTestAll.results.json and exits non-zero on failures.

7. CI and Maven integration

• Note pom.xml has an exec-maven-plugin execution id oaf-test configured to run oafTest.sh at test phase; explain how tests can be run via mvn test if appropriate or how CI calls the helper script. Provide a brief note that CI integrations should run oafTest.sh or the ojob command.

8. How to add a focused test for a specific OpenAF function (step-by-step)

• Step 1: identify area and create file tests/autoTestAll..js (or add to existing file).
• Step 2: require or use the function inside the test, call ow.test.assert to validate behavior, use io.rm or cleanup to avoid artifacts.
• Step 3: run tests locally and update goldens / results if intentional.
• Step 4: include instructions for adding/updating badges (SVG generation is done by test orchestrator) and updating tests/autoTestAll.md if needed.

9. Example minimal test template (copy-paste ready):

• Provide a small JS file template with header comments, exports.testMyFunc = function() { ow.loadTest(); /* optional load modules */ var res = myModule.myFunc(args); ow.test.assert(res, expected, "message"); } and cleanup.

10. Troubleshooting / common pitfalls

• Permissions, port/timeouts for tests that need network, how to view autoTestAll.results.json, and how to run only one test file for debugging.

11. Links and references

• Link to tests/ directory, README.md testing instructions, BUILD.md, AGENTS.md, docs/openaf.md testing section, oafTest.sh, pom.xml, and autoTestAll.md.

Requirements for the PR:

• Add a single new file docs/testing.md with the content above.
• Ensure the new file is Markdown, uses repo-relative links to the files referenced (e.g., ../oafTest.sh or /oafTest.sh? use paths like ./oafTest.sh or repository root links like https://github.com/OpenAF/openaf/blob/master/oafTest.sh).
• Add a concise PR title: "docs: add testing guide (how to add/run OpenAF tests)" and a descriptive PR body summarizing the file and where to find examples.
• Do not modify existing files except to add the new docs file.

If anything is ambiguous (branch to target, or exact filename preference), create the PR against the default repository branch (master/main) — leave base_ref empty so maintainers can review and adjust. If the repositor...

This pull request was created from Copilot chat.

---

⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.

[nmaguiar]

@copilot try again

[Copilot]

@copilot try again

Done — docs/testing.md is now complete (commit 80d70f2). I also fixed an inaccuracy in the examples: individual test functions should not call ow.loadTest() (the orchestrator framework calls it once before running tests, as seen in tests/autoTestAll.js). Code review and CodeQL both passed cleanly.