Bazel cache corruption when generating documentation

[4og]

Description

Documentation generation often fails with not being able to find files in $HOME/.cache/bazel folder, and this can only be resolved with bazel clean --expunge. This strongly suggests Bazel cache corruption.

The issue affects the docs-as-code module and all other modules that rely on it for documentation generation.

The error looks like the following (exact missing files can differ: those marked with WARNING: Ignored unreadable document):

$ bazel run //:docs                                    
INFO: Analyzed target //:docs (347 packages loaded, 23883 targets configured).
ERROR: /home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/external/score_process+/BUILD:39:5: Sphinx building needs for @@score_process+//:needs_json failed: (Exit 1): sphinx_build failed: error executing SphinxBuildDocs command (from target @@score_process+//:needs_json) bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/score_process+/sphinx_build @bazel-out/k8-fastbuild/bin/external/score_process+/needs_json/_build/needs-0.params
Sphinx main() returned failure:   exit code: 1
========== STDOUT EMPTY ==========
========== STDERR START ==========
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/folder_templates/platform/safety_analysis/platform_dfa.rst: WARNING: Ignored unreadable document 'folder_templates/platform/safety_analysis/platform_dfa.rst'.
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/folder_templates/platform/safety_planning/index.rst: WARNING: Ignored unreadable document 'folder_templates/platform/safety_planning/index.rst'.
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/folder_templates/platform/safety_planning/platform_safety_analysis_fdr.rst: WARNING: Ignored unreadable document 'folder_templates/platform/safety_planning/platform_safety_analysis_fdr.rst'.
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/process_areas/security_management/guidance/security_management_checklist_security_package.rst: WARNING: Ignored unreadable document 'process_areas/security_management/guidance/security_management_checklist_security_package.rst'.
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/process_areas/security_management/guidance/security_management_checklist_security_plan.rst: WARNING: Ignored unreadable document 'process_areas/security_management/guidance/security_management_checklist_security_plan.rst'.
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/process_areas/security_management/guidance/security_management_feature_security_wp_template.rst: WARNING: Ignored unreadable document 'process_areas/security_management/guidance/security_management_feature_security_wp_template.rst'.
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/process_areas/security_management/guidance/security_management_module_security_plan_template.rst: WARNING: Ignored unreadable document 'process_areas/security_management/guidance/security_management_module_security_plan_template.rst'.
/home/user/.cache/bazel/_bazel_user/37e77bf952c33149402719c850ed0b53/execroot/_main/bazel-out/k8-fastbuild/bin/external/score_process+/_needs_json/score_process+/process/process_areas/security_management/guidance/security_management_security_manual_template.rst: WARNING: Ignored unreadable document 'process_areas/security_management/guidance/security_management_security_manual_template.rst'.
========== STDERR END ==========
Target //:docs failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 10.658s, Critical Path: 6.79s
INFO: 121 processes: 351 action cache hit, 121 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

In CI, it also occurs occasionally across different repositories. A few examples:
https://github.com/eclipse-score/baselibs/actions/runs/23487268081/job/68345969171
https://github.com/eclipse-score/docs-as-code/actions/runs/23481648438/job/68326696917
In those cases the issue can only be resolved by manually deleting cache via GitHub UI.

Analysis results

No response

Solution

No response

Error Occurrence Rate

Reproducible

How to reproduce

This can be easily reproduced by changing dependency on score_process:

1.  git checkout 4f39de2b0a2002cc5be8a5ef1d6621490380512c # Latest main at issue creation
    bazel clean --expunge
    bazel run //:docs

   Documentation build succeeds
2. Change score_process in MODULE.bazel from 1.4.3 to version 1.4.4.
3. bazel run //:docs - Build fails with an error
4. Change score_process back to 1.4.3.
5. bazel run //:docs
   Build fails with the same error

6.  bazel clean --expunge
    bazel run //:docs

   Documentation build succeeds

The same way the issue can be reproduced in any other module that uses docs-as-code for documentation.

Supporting Information

No response

Classification

Major

First Affected Release

not released (main)

Last Affected Release

not released (main)

Expected Fixed Release

before release (main)

Category

• Safety Relevant
• Security Relevant