Skip to content

Add phase field to file_status JSON output (start/end lifecycle events)#9532

Open
YK-03 wants to merge 2 commits intoborgbackup:masterfrom
YK-03:feature/file-status-phase
Open

Add phase field to file_status JSON output (start/end lifecycle events)#9532
YK-03 wants to merge 2 commits intoborgbackup:masterfrom
YK-03:feature/file-status-phase

Conversation

@YK-03
Copy link

@YK-03 YK-03 commented Mar 22, 2026

Description

This PR adds an optional "phase" field to "type": "file_status" JSON output to indicate lifecycle stages of file processing.

  • "phase": "start" is emitted when processing of a file begins
  • "phase": "end" is emitted for final file status events (default if not explicitly provided)

Implementation details

  • JSON construction remains centralized in Archiver.print_file_status
  • START events are emitted in CreateMixIn._process_any before dispatching to fso.process_*
  • START emission is guarded with retry == 0 to avoid duplicate events
  • START is only emitted for valid, processable file types (no sockets, doors, ports, or unknown types)
  • END events reuse existing call sites, with "phase" defaulting to "end"

Behavior

JSON mode (--log-json --list)

  • Emits paired start/end events for each processed file
  • No duplicate or orphan events

Non-JSON mode (--list)

  • Behavior unchanged
  • Logging guarded to avoid None path output

Compatibility

  • Fully backward compatible
  • Existing fields (type, status, path) unchanged
  • "phase" is additive and optional

Checklist

  • PR is against master
  • New code has tests and docs where appropriate (not included in this PR)
  • Tests pass (not executed locally)
  • Commit messages are clean and focused

Copy link
Member

@ThomasWaldmann ThomasWaldmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a quick look: this does not look correct:

  • for file types where there is basically nothing to do (because they have no content data), it emits start and end.
  • for some file types and processing modes (--read-special) where there is a lot to do (processing chunks of content data), it does not emit start/end.

@codecov
Copy link

codecov bot commented Mar 22, 2026

Codecov Report

❌ Patch coverage is 47.50000% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.33%. Comparing base (2ac51d7) to head (a9c76f7).
⚠️ Report is 2 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/borg/archiver/create_cmd.py 42.85% 9 Missing and 7 partials ⚠️
src/borg/archiver/__init__.py 58.33% 2 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9532      +/-   ##
==========================================
- Coverage   83.36%   83.33%   -0.04%     
==========================================
  Files          87       87              
  Lines       15447    15496      +49     
  Branches     2311     2332      +21     
==========================================
+ Hits        12878    12913      +35     
- Misses       1818     1823       +5     
- Partials      751      760       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@YK-03
Copy link
Author

YK-03 commented Mar 22, 2026

I’ll update the implementation to ensure start and end events are emitted consistently for all files, including --read-special, by wrapping the entire processing lifecycle (including chunk-based processing).

I’ll also add tests to cover these scenarios and ensure consistent behavior.

@YK-03 YK-03 force-pushed the feature/file-status-phase branch from bc12bcf to d80724b Compare March 23, 2026 18:18
@YK-03 YK-03 force-pushed the feature/file-status-phase branch from 519a682 to e6d6ab6 Compare March 24, 2026 06:32
@YK-03
Copy link
Author

YK-03 commented Mar 24, 2026

Hi @ThomasWaldmann,

I've updated the implementation to ensure lifecycle events are emitted only around actual file processing.

  • Lifecycle emission is now placed in process_file and process_pipe, wrapping the chunk processing logic.
  • This ensures correct coverage for --read-special and other chunk-based paths.
  • Lifecycle events are not emitted for trivial or cached files (e.g., when chunks are reused).
  • _process_any is no longer involved in lifecycle emission.

Start/end pairing is guaranteed via try/finally, and existing status event behavior remains unchanged.

Please let me know if anything else needs adjustment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants