Skip to content

feat(server): add threads count and disk space to sysinfo stats#2917

Open
seokjin0414 wants to merge 6 commits intoapache:masterfrom
seokjin0414:2732-add-threads-and-disk-space-to-sysinfo
Open

feat(server): add threads count and disk space to sysinfo stats#2917
seokjin0414 wants to merge 6 commits intoapache:masterfrom
seokjin0414:2732-add-threads-and-disk-space-to-sysinfo

Conversation

@seokjin0414
Copy link
Contributor

@seokjin0414 seokjin0414 commented Mar 11, 2026

Summary

Closes #2732.

  • Add threads_count, free_disk_space, total_disk_space fields to Stats struct
  • Collect thread count via sysinfo Process::tasks() API (Linux only, 0 on macOS)
  • Collect disk space via fs2 crate (statvfs syscall) with warning log on failure
  • Add IggyUsage (messages_size_bytes), Disk (free/total), Threads (conditional) to sysinfo log output
  • Update binary protocol encoding/decoding with backward-compatible field appending
  • Add CLI Table/List output rows for new fields
  • Add integration tests: field presence in all 4 output formats + JSON value verification with message size check

Design

  • Thread count: process.tasks().map(|t| t.len()).unwrap_or(0) — uses existing refresh_processes() call, no additional sysinfo refresh needed. Conditional log output (skipped when 0, matching OpenFDs pattern)
  • Disk space: fs2::available_space() / fs2::total_space() — single statvfs syscall per metric (~10μs), already used in logger.rs. Logs warning on failure instead of silently defaulting to 0
  • IggyUsage: existing messages_size_bytes field (iggy metadata only, no sysinfo) — added to sysinfo log output
  • Binary protocol: new fields appended after cache_metrics, decoded with current_position + N <= payload.len() guards for backward compatibility

@codecov
Copy link

codecov bot commented Mar 11, 2026

Codecov Report

❌ Patch coverage is 74.50980% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.26%. Comparing base (6a6f155) to head (28d5840).

Files with missing lines Patch % Lines
core/binary_protocol/src/utils/mapper.rs 80.64% 3 Missing and 3 partials ⚠️
core/server/src/shard/system/stats.rs 45.45% 6 Missing ⚠️
...server/src/shard/tasks/periodic/sysinfo_printer.rs 66.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2917      +/-   ##
============================================
- Coverage     70.27%   70.26%   -0.01%     
  Complexity      862      862              
============================================
  Files          1029     1029              
  Lines         85283    85332      +49     
  Branches      62653    62715      +62     
============================================
+ Hits          59932    59958      +26     
- Misses        22839    22850      +11     
- Partials       2512     2524      +12     
Flag Coverage Δ
csharp 67.43% <ø> (-0.23%) ⬇️
go 36.36% <ø> (ø)
java 59.87% <ø> (ø)
node 91.37% <ø> (-0.07%) ⬇️
python 81.43% <ø> (ø)
rust 70.64% <74.50%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ore/binary_protocol/src/cli/binary_system/stats.rs 95.00% <ø> (ø)
core/common/src/types/stats/mod.rs 69.86% <100.00%> (+1.29%) ⬆️
core/server/src/binary/mapper.rs 94.41% <100.00%> (+0.04%) ⬆️
...server/src/shard/tasks/periodic/sysinfo_printer.rs 78.26% <66.66%> (-0.81%) ⬇️
core/binary_protocol/src/utils/mapper.rs 79.34% <80.64%> (+0.05%) ⬆️
core/server/src/shard/system/stats.rs 91.20% <45.45%> (-6.36%) ⬇️

... and 12 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@hubcio hubcio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in PR description you say Disks, yet in code I don't see this being used. also please fix my comment in stats.rs. other than that LGTM.

.unwrap_or(0)
.into();
stats.total_disk_space = fs2::total_space(&self.config.system.path)
.unwrap_or(0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add warning print when it fails

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review~
Fixed both — added tracing::warn! on fs2 failure in stats.rs (28d5840) and updated PR description to reflect fs2 instead of Disks

Add threads_count, free_disk_space, and total_disk_space fields to
the Stats struct. Collect thread count via sysinfo Process::tasks()
API and disk space via sysinfo Disks API with longest-prefix mount
point matching. Update binary protocol serialization/deserialization
with backward-compatible decoding, CLI table/list output, sysinfo
printer log format, and integration tests including message size
verification.

Closes apache#2732

Signed-off-by: seokjin0414 <sars21@hanmail.net>
Signed-off-by: shin <sars21@hanmail.net>
Use numeric IDs from server responses instead of string identifiers
for send_messages call to avoid partition_not_found race condition
in multi-shard architecture.

Signed-off-by: shin <sars21@hanmail.net>
…rics

Replace sysinfo::Disks::new_with_refreshed_list() with fs2::available_space()
and fs2::total_space() for significantly lower overhead. fs2 performs a single
statvfs syscall per metric (~10μs) vs sysinfo scanning all mount points (~0.5ms).
fs2 is already used in the server crate (logger.rs).

Signed-off-by: shin <sars21@hanmail.net>
Use Partitioning::default() instead of partition_id(1) to avoid
partition_not_found errors in multi-shard CI environments where
partition metadata may not have propagated to all shards yet.

Signed-off-by: shin <sars21@hanmail.net>
Signed-off-by: shin <sars21@hanmail.net>
Signed-off-by: seokjin0414 <sars21@hanmail.net>
Signed-off-by: shin <sars21@hanmail.net>
@seokjin0414 seokjin0414 force-pushed the 2732-add-threads-and-disk-space-to-sysinfo branch from 7293835 to 28d5840 Compare March 15, 2026 15:12
@seokjin0414 seokjin0414 requested a review from hubcio March 15, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add threads and free space to sysinfo print

2 participants