Skip to content

feat(kvp): add store trait and Hyper-V KVP storage crate#288

Open
peytonr18 wants to merge 8 commits intoAzure:mainfrom
peytonr18:probertson/kvp-store-trait
Open

feat(kvp): add store trait and Hyper-V KVP storage crate#288
peytonr18 wants to merge 8 commits intoAzure:mainfrom
peytonr18:probertson/kvp-store-trait

Conversation

@peytonr18
Copy link
Contributor

@peytonr18 peytonr18 commented Mar 9, 2026

Summary

  • Adds workspace member libazureinit-kvp in root Cargo.toml
  • Adds libazureinit-kvp/Cargo.toml
  • Adds libazureinit-kvp/src/lib.rs and libazureinit-kvp/src/kvp_pool.rs

Crate Design

  • One trait: KvpStore
  • One implementation: KvpPoolStore

KvpStore splits each operation into a backend_* method (raw I/O,
provided by the implementor) and a public method (write, read,
clear) that validates inputs then delegates to the backend.

Public API:

  • write, read (validate then delegate to backend_write/backend_read)
  • entries, entries_raw
  • delete, clear
  • is_stale

Validation is centralized in trait default methods and policy-aware via:

  • max_key_size(&self)
  • max_value_size(&self)
    KvpPoolStore is file-backed using Hyper-V KVP wire format
    (fixed-size 512-byte key + 2048-byte value records), with lock-based concurrency.

Policy and Limits

Constructor:

  • new(path: Option<PathBuf>, mode: PoolMode, truncate_on_stale: bool)
    PoolMode:
  • Restricted (default): key <= 254 bytes, value <= 1022 bytes
  • Full: key <= 512 bytes, value <= 2048 bytes

Behavior:

  • Default path when None: /var/lib/hyperv/.kvp_pool_1
  • Unique key cap: 1024
  • new key beyond cap is rejected
  • overwrite of existing key at cap is allowed
  • clear() truncates the store
  • truncate_on_stale keeps truncation caller-controlled

Errors

KvpError includes explicit variants:

  • EmptyKey
  • KeyContainsNull
  • KeyTooLarge { max, actual }
  • ValueTooLarge { max, actual }
  • MaxUniqueKeysExceeded { max }
  • Io(io::Error)

Testing

17 tests covering:

  • restricted/full key/value boundary checks
  • default and explicit path behavior
  • mode getter
  • unique-key cap behavior (including overwrite-at-cap and add-after-delete)
  • entries last-write-wins and entries_raw duplicate preservation
  • delete, clear, and stale checks
  • key validation (empty and null-byte)

@peytonr18 peytonr18 force-pushed the probertson/kvp-store-trait branch from b134a98 to f435f70 Compare March 9, 2026 19:19
pub const HYPERV_MAX_VALUE_BYTES: usize = 2048;
/// Azure key limit in bytes (policy/default preset).
pub const AZURE_MAX_KEY_BYTES: usize = 512;
/// Azure value limit in bytes (UTF-16: 511 characters + null terminator).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a reference for the null termination of this? I opened up that link in the previous file, but did not see a reference to null termination

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was basing this off of what we have in our existing kvp.rs --

const HV_KVP_EXCHANGE_MAX_KEY_SIZE: usize = 512;
const HV_KVP_EXCHANGE_MAX_VALUE_SIZE: usize = 2048;
const HV_KVP_AZURE_MAX_VALUE_SIZE: usize = 1022;

and my old notes from that KVP process, but I'll look for the official documentation where it says that!

@cadejacobson cadejacobson self-requested a review March 9, 2026 20:42
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new standalone workspace crate (libazureinit-kvp) that defines a storage abstraction (KvpStore) and a production Hyper-V pool-file implementation (HyperVKvpStore), along with updated technical reference documentation.

Changes:

  • Introduces libazureinit-kvp crate with KvpStore and KvpLimits (Hyper-V/Azure presets).
  • Implements Hyper-V binary pool-file backend with flock-based concurrency and stale-file truncation support.
  • Rewrites doc/libazurekvp.md as a technical reference for the new crate/API.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
Cargo.toml Adds libazureinit-kvp to the workspace members.
libazureinit-kvp/Cargo.toml Defines the new crate and its dependencies (fs2, sysinfo).
libazureinit-kvp/src/lib.rs Defines KvpStore, KvpLimits, and exports HyperVKvpStore plus size constants.
libazureinit-kvp/src/hyperv.rs Implements Hyper-V pool-file encoding/decoding and the KvpStore backend, plus unit tests.
doc/libazurekvp.md Updates documentation to describe the new crate’s record format, semantics, truncation behavior, and limits.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

peytonr18 and others added 3 commits March 19, 2026 11:56
…ics and JSON dump

Overhaul the KvpStore trait into a pure customer-facing interface with
no default implementations or backend_* methods. All logic (validation,
file I/O, locking) now lives in the KvpPoolStore implementation.

Key changes:
- Replace append-based write with upsert (insert-or-update) so each key
  has exactly one record in the file. Eliminates entries_raw and
  last-write-wins deduplication.
- Move validate_key/validate_value from trait defaults to private
  KvpPoolStore methods.
- Decouple read validation from PoolMode: reads accept keys up to the
  full wire-format max (512 bytes) regardless of mode; only writes are
  restricted by PoolMode.
- Add len(), is_empty(), and dump() (JSON via serde_json) to the trait.
- Add 4 multi-threaded concurrency tests covering concurrent upserts to
  different keys, same key, mixed readers/writers, and key cap boundary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants