test discovery 2.0

.github/instructions/testing-workflow.instructions.md

@@ -578,3 +578,4 @@ envConfig.inspect
 -   When mocking `testController.createTestItem()` in unit tests, use `typemoq.It.isAny()` for parameters when testing handler behavior (not ID/label generation logic), but consider using specific matchers (e.g., `It.is((id: string) => id.startsWith('_error_'))`) when the actual values being passed are important for correctness - this balances test precision with maintainability (2)
 -   Remove unused variables from test code immediately - leftover tracking variables like `validationCallCount` that aren't referenced indicate dead code that should be simplified (1)
 -   Use `Uri.file(path).fsPath` for both sides of path comparisons in tests to ensure cross-platform compatibility - Windows converts forward slashes to backslashes automatically (1)
+-   When tests fail with "Cannot stub non-existent property", the method likely moved to a different class during refactoring - find the class that owns the method and test that class directly instead of stubbing on the original class (1)

.github/instructions/testing_feature_area.instructions.md

@@ -26,6 +26,10 @@ This document maps the testing support in the extension: discovery, execution (r
     -   `src/client/testing/serviceRegistry.ts` — DI/wiring for testing services.
 -   Workspace orchestration
     -   `src/client/testing/testController/workspaceTestAdapter.ts` — `WorkspaceTestAdapter` (provider-agnostic entry used by controller).
+-   **Project-based testing (multi-project workspaces)**
+    -   `src/client/testing/testController/common/testProjectRegistry.ts` — `TestProjectRegistry` (manages project lifecycle, discovery, and nested project handling).
+    -   `src/client/testing/testController/common/projectAdapter.ts` — `ProjectAdapter` interface (represents a single Python project with its own test infrastructure).
+    -   `src/client/testing/testController/common/projectUtils.ts` — utilities for project ID generation, display names, and shared adapter creation.
 -   Provider adapters
     -   Unittest
         -   `src/client/testing/testController/unittest/testDiscoveryAdapter.ts`
@@ -151,6 +155,51 @@ The adapters in the extension don't implement test discovery/run logic themselve
     -   Settings are consumed by `src/client/testing/common/testConfigurationManager.ts`, `src/client/testing/configurationFactory.ts`, and adapters under `src/client/testing/testController/*` which read settings to build CLI args and env for subprocesses.
     -   The setting definitions and descriptions are in `package.json` and localized strings in `package.nls.json`.
 
+## Project-based testing (multi-project workspaces)
+
+Project-based testing enables multi-project workspace support where each Python project gets its own test tree root with its own Python environment.
+
+> **⚠️ Note: unittest support for project-based testing is NOT yet implemented.** Project-based testing currently only works with pytest. unittest support will be added in a future PR.
+
+### Architecture
+
+-   **TestProjectRegistry** (`testProjectRegistry.ts`): Central registry that:
+
+    -   Discovers Python projects via the Python Environments API
+    -   Creates and manages `ProjectAdapter` instances per workspace
+    -   Computes nested project relationships and configures ignore lists
+    -   Falls back to "legacy" single-adapter mode when API unavailable
+
+-   **ProjectAdapter** (`projectAdapter.ts`): Interface representing a single project with:
+    -   Project identity (ID, name, URI from Python Environments API)
+    -   Python environment with execution details
+    -   Test framework adapters (discovery/execution)
+    -   Nested project ignore paths (for parent projects)
+
+### How it works
+
+1. **Activation**: When the extension activates, `PythonTestController` checks if the Python Environments API is available.
+2. **Project discovery**: `TestProjectRegistry.discoverAndRegisterProjects()` queries the API for all Python projects in each workspace.
+3. **Nested handling**: `configureNestedProjectIgnores()` identifies child projects and adds their paths to parent projects' ignore lists.
+4. **Test discovery**: For each project, the controller calls `project.discoveryAdapter.discoverTests()` with the project's URI. The adapter sets `PROJECT_ROOT_PATH` environment variable for the Python runner.
+5. **Python side**: `get_test_root_path()` in `vscode_pytest/__init__.py` returns `PROJECT_ROOT_PATH` (if set) or falls back to `cwd`.
+6. **Test tree**: Each project gets its own root node in the Test Explorer, with test IDs scoped by project ID using the `||` separator.
+
+### Logging prefix
+
+All project-based testing logs use the `[test-by-project]` prefix for easy filtering in the output channel.
+
+### Key files
+
+-   Python side: `python_files/vscode_pytest/__init__.py` — `get_test_root_path()` function and `PROJECT_ROOT_PATH` environment variable.
+-   TypeScript: `testProjectRegistry.ts`, `projectAdapter.ts`, `projectUtils.ts`, and the discovery adapters.
+
+### Tests
+
+-   `src/test/testing/testController/common/testProjectRegistry.unit.test.ts` — TestProjectRegistry tests
+-   `src/test/testing/testController/common/projectUtils.unit.test.ts` — Project utility function tests
+-   `python_files/tests/pytestadapter/test_get_test_root_path.py` — Python-side get_test_root_path() tests
+
 ## Coverage support (how it works)
 
 -   Coverage is supported by running the Python helper scripts with coverage enabled and then collecting a coverage payload from the runner.

python_files/tests/pytestadapter/expected_discovery_test_output.py

@@ -1870,3 +1870,201 @@
     ],
     "id_": TEST_DATA_PATH_STR,
 }
+
+# =====================================================================================
+# PROJECT_ROOT_PATH environment variable tests
+# These test the project-based testing feature where PROJECT_ROOT_PATH changes
+# the test tree root from cwd to the specified project path.
+# =====================================================================================
+
+# This is the expected output for unittest_folder when PROJECT_ROOT_PATH is set to unittest_folder.
+# The root of the tree is unittest_folder (not .data), simulating project-based testing.
+# └── unittest_folder (ROOT - set via PROJECT_ROOT_PATH)
+#    ├── test_add.py
+#    │   └── TestAddFunction
+#    │       ├── test_add_negative_numbers
+#    │       └── test_add_positive_numbers
+#    │   └── TestDuplicateFunction
+#    │       └── test_dup_a
+#    └── test_subtract.py
+#        └── TestSubtractFunction
+#            ├── test_subtract_negative_numbers
+#            └── test_subtract_positive_numbers
+#        └── TestDuplicateFunction
+#            └── test_dup_s
+#
+# Note: This reuses the unittest_folder paths defined earlier in this file.
+project_root_unittest_folder_expected_output = {
+    "name": "unittest_folder",
+    "path": os.fspath(unittest_folder_path),
+    "type_": "folder",
+    "children": [
+        {
+            "name": "test_add.py",
+            "path": os.fspath(test_add_path),
+            "type_": "file",
+            "id_": os.fspath(test_add_path),
+            "children": [
+                {
+                    "name": "TestAddFunction",
+                    "path": os.fspath(test_add_path),
+                    "type_": "class",
+                    "children": [
+                        {
+                            "name": "test_add_negative_numbers",
+                            "path": os.fspath(test_add_path),
+                            "lineno": find_test_line_number(
+                                "test_add_negative_numbers",
+                                os.fspath(test_add_path),
+                            ),
+                            "type_": "test",
+                            "id_": get_absolute_test_id(
+                                "test_add.py::TestAddFunction::test_add_negative_numbers",
+                                test_add_path,
+                            ),
+                            "runID": get_absolute_test_id(
+                                "test_add.py::TestAddFunction::test_add_negative_numbers",
+                                test_add_path,
+                            ),
+                        },
+                        {
+                            "name": "test_add_positive_numbers",
+                            "path": os.fspath(test_add_path),
+                            "lineno": find_test_line_number(
+                                "test_add_positive_numbers",
+                                os.fspath(test_add_path),
+                            ),
+                            "type_": "test",
+                            "id_": get_absolute_test_id(
+                                "test_add.py::TestAddFunction::test_add_positive_numbers",
+                                test_add_path,
+                            ),
+                            "runID": get_absolute_test_id(
+                                "test_add.py::TestAddFunction::test_add_positive_numbers",
+                                test_add_path,
+                            ),
+                        },
+                    ],
+                    "id_": get_absolute_test_id(
+                        "test_add.py::TestAddFunction",
+                        test_add_path,
+                    ),
+                    "lineno": find_class_line_number("TestAddFunction", test_add_path),
+                },
+                {
+                    "name": "TestDuplicateFunction",
+                    "path": os.fspath(test_add_path),
+                    "type_": "class",
+                    "children": [
+                        {
+                            "name": "test_dup_a",
+                            "path": os.fspath(test_add_path),
+                            "lineno": find_test_line_number(
+                                "test_dup_a",
+                                os.fspath(test_add_path),
+                            ),
+                            "type_": "test",
+                            "id_": get_absolute_test_id(
+                                "test_add.py::TestDuplicateFunction::test_dup_a",
+                                test_add_path,
+                            ),
+                            "runID": get_absolute_test_id(
+                                "test_add.py::TestDuplicateFunction::test_dup_a",
+                                test_add_path,
+                            ),
+                        },
+                    ],
+                    "id_": get_absolute_test_id(
+                        "test_add.py::TestDuplicateFunction",
+                        test_add_path,
+                    ),
+                    "lineno": find_class_line_number("TestDuplicateFunction", test_add_path),
+                },
+            ],
+        },
+        {
+            "name": "test_subtract.py",
+            "path": os.fspath(test_subtract_path),
+            "type_": "file",
+            "id_": os.fspath(test_subtract_path),
+            "children": [
+                {
+                    "name": "TestSubtractFunction",
+                    "path": os.fspath(test_subtract_path),
+                    "type_": "class",
+                    "children": [
+                        {
+                            "name": "test_subtract_negative_numbers",
+                            "path": os.fspath(test_subtract_path),
+                            "lineno": find_test_line_number(
+                                "test_subtract_negative_numbers",
+                                os.fspath(test_subtract_path),
+                            ),
+                            "type_": "test",
+                            "id_": get_absolute_test_id(
+                                "test_subtract.py::TestSubtractFunction::test_subtract_negative_numbers",
+                                test_subtract_path,
+                            ),
+                            "runID": get_absolute_test_id(
+                                "test_subtract.py::TestSubtractFunction::test_subtract_negative_numbers",
+                                test_subtract_path,
+                            ),
+                        },
+                        {
+                            "name": "test_subtract_positive_numbers",
+                            "path": os.fspath(test_subtract_path),
+                            "lineno": find_test_line_number(
+                                "test_subtract_positive_numbers",
+                                os.fspath(test_subtract_path),
+                            ),
+                            "type_": "test",
+                            "id_": get_absolute_test_id(
+                                "test_subtract.py::TestSubtractFunction::test_subtract_positive_numbers",
+                                test_subtract_path,
+                            ),
+                            "runID": get_absolute_test_id(
+                                "test_subtract.py::TestSubtractFunction::test_subtract_positive_numbers",
+                                test_subtract_path,
+                            ),
+                        },
+                    ],
+                    "id_": get_absolute_test_id(
+                        "test_subtract.py::TestSubtractFunction",
+                        test_subtract_path,
+                    ),
+                    "lineno": find_class_line_number("TestSubtractFunction", test_subtract_path),
+                },
+                {
+                    "name": "TestDuplicateFunction",
+                    "path": os.fspath(test_subtract_path),
+                    "type_": "class",
+                    "children": [
+                        {
+                            "name": "test_dup_s",
+                            "path": os.fspath(test_subtract_path),
+                            "lineno": find_test_line_number(
+                                "test_dup_s",
+                                os.fspath(test_subtract_path),
+                            ),
+                            "type_": "test",
+                            "id_": get_absolute_test_id(
+                                "test_subtract.py::TestDuplicateFunction::test_dup_s",
+                                test_subtract_path,
+                            ),
+                            "runID": get_absolute_test_id(
+                                "test_subtract.py::TestDuplicateFunction::test_dup_s",
+                                test_subtract_path,
+                            ),
+                        },
+                    ],
+                    "id_": get_absolute_test_id(
+                        "test_subtract.py::TestDuplicateFunction",
+                        test_subtract_path,
+                    ),
+                    "lineno": find_class_line_number("TestDuplicateFunction", test_subtract_path),
+                },
+            ],
+        },
+    ],
+    "id_": os.fspath(unittest_folder_path),
+}

python_files/tests/pytestadapter/test_discovery.py

@@ -386,3 +386,45 @@ def test_plugin_collect(file, expected_const, extra_arg):
         ), (
             f"Tests tree does not match expected value. \n Expected: {json.dumps(expected_const, indent=4)}. \n Actual: {json.dumps(actual_item.get('tests'), indent=4)}"
         )
+
+
+def test_project_root_path_env_var():
+    """Test pytest discovery with PROJECT_ROOT_PATH environment variable set.
+
+    This simulates project-based testing where the test tree root should be
+    the project root (PROJECT_ROOT_PATH) rather than the workspace cwd.
+
+    When PROJECT_ROOT_PATH is set:
+    - The test tree root (name, path, id_) should match PROJECT_ROOT_PATH
+    - The cwd in the response should match PROJECT_ROOT_PATH
+    - Test files should be direct children of the root (not nested under a subfolder)
+    """
+    # Use unittest_folder as our "project" subdirectory
+    project_path = helpers.TEST_DATA_PATH / "unittest_folder"
+
+    actual = helpers.runner_with_cwd_env(
+        [os.fspath(project_path), "--collect-only"],
+        helpers.TEST_DATA_PATH,  # cwd is parent of project
+        {"PROJECT_ROOT_PATH": os.fspath(project_path)},  # Set project root
+    )
+
+    assert actual
+    actual_list: List[Dict[str, Any]] = actual
+    if actual_list is not None:
+        actual_item = actual_list.pop(0)
+
+        assert all(item in actual_item for item in ("status", "cwd", "error"))
+        assert actual_item.get("status") == "success", (
+            f"Status is not 'success', error is: {actual_item.get('error')}"
+        )
+        # cwd in response should be PROJECT_ROOT_PATH
+        assert actual_item.get("cwd") == os.fspath(project_path), (
+            f"Expected cwd '{os.fspath(project_path)}', got '{actual_item.get('cwd')}'"
+        )
+        assert is_same_tree(
+            actual_item.get("tests"),
+            expected_discovery_test_output.project_root_unittest_folder_expected_output,
+            ["id_", "lineno", "name", "runID"],
+        ), (
+            f"Tests tree does not match expected value. \n Expected: {json.dumps(expected_discovery_test_output.project_root_unittest_folder_expected_output, indent=4)}. \n Actual: {json.dumps(actual_item.get('tests'), indent=4)}"
+        )