Skip to content

Feat/snapshot download#165

Draft
bburda wants to merge 16 commits intomainfrom
feat/snapshot-download
Draft

Feat/snapshot download#165
bburda wants to merge 16 commits intomainfrom
feat/snapshot-download

Conversation

@bburda
Copy link
Collaborator

@bburda bburda commented Feb 5, 2026

Pull Request

Summary

Briefly describe what changed and why.


Issue

Link the related issue (required):

  • closes #

Type

  • Bug fix
  • New feature or tests
  • Breaking change
  • Documentation only

Testing

How was this tested / how should reviewers verify it?


Checklist

  • Breaking changes are clearly described (and announced in docs / changelog if needed)
  • Tests were added or updated if needed
  • Docs were updated if behavior or public API changed

bburda added 16 commits February 5, 2026 07:21
…rEntity definitions

Add new ROS2 message definitions for SOVD-compliant fault environment data:
- ExtendedDataRecords.msg: first/last occurrence timestamps
- Snapshot.msg: unified freeze frame and rosbag snapshot type
- EnvironmentData.msg: container for snapshots in fault response
- GetFault.srv: NEW service for single fault lookup with environment_data
- ListFaultsForEntity.srv: service to query faults by entity
Add UUID generation for rosbag files and index for UUID-based lookup:
- FaultStorage::generate_uuid() for UUID v4 generation
- RosbagFileInfo::bulk_data_id field for globally unique identifier
- FaultStorage::get_rosbag_by_id() for lookup by UUID
- FaultStorage::get_rosbag_path() for file path retrieval
- FaultStorage::get_rosbags_for_entity() for entity-filtered queries
- rosbag_by_id_ index for O(1) UUID lookups in InMemoryFaultStorage
- SQLite schema migration for existing databases

Required for SOVD bulk-data download pattern.

@verifies REQ_INTEROP_073
Implement GetFault service handler that returns:
- Fault data with basic info and status
- ExtendedDataRecords with first/last occurrence timestamps
- Snapshots including freeze frames and rosbag metadata

Add unit tests for GetFault service responses.
Implement ListFaultsForEntity service for querying faults by entity:
- Add get_all_faults() method to FaultStorage interface and implementations
- Add matches_entity() helper for FQN suffix matching
- Add list_faults_for_entity service to FaultManagerNode

Add unit tests for new functionality including:
- GetAllFaults storage tests
- ListFaultsForEntity service tests
- matches_entity helper tests
Add utility functions for parsing entity paths from HTTP request URLs:
- EntityPathInfo struct with type, entity_id, resource_path, parent_id
- parse_entity_path() extracts entity info from /api/v1/{type}/{id}/... URLs
- extract_bulk_data_category() gets bulk-data category from path
- extract_bulk_data_id() gets bulk-data item ID from path

Supports nested entities (subareas, subcomponents) with parent tracking.
Includes 20 unit tests covering all parsing scenarios.
Add build_sovd_fault_response() for SOVD-compliant responses:
- 'item' wrapper with fault details and status object
- 'environment_data' with extended_data_records and snapshots
- 'x-medkit' extensions with occurrence_count, severity_label
- Rosbag snapshots include bulk_data_uri (absolute path)
- Primary value extraction for common ROS message types

Add FaultManager::get_fault_with_env() for retrieving fault with
environment data via GetFault service. Update handle_get_fault
to use entity path from URL for bulk_data_uri generation.

Includes 12 unit tests covering SOVD response building.

@verifies REQ_INTEROP_013
Add BulkDataHandlers class with:
- handle_list_categories: GET {entity}/bulk-data
- handle_list_descriptors: GET {entity}/bulk-data/{category}
- handle_download: GET {entity}/bulk-data/{category}/{id}

Supports all entity path patterns:
- /apps, /components, /areas, /functions
- /areas/{id}/subareas, /components/{id}/subcomponents

Security: Validates rosbag belongs to requested entity.
CORS: Exposes Content-Disposition header for frontend.

@verifies REQ_INTEROP_071
@verifies REQ_INTEROP_072
@verifies REQ_INTEROP_073
REQ_INTEROP_072, REQ_INTEROP_073

- Remove /faults/{code}/snapshots (system-wide)
- Remove /faults/{code}/snapshots/bag (rosbag download)
- Remove entity-scoped snapshot endpoints for components, apps, areas, functions
- Remove handler declarations from fault_handlers.hpp
- Remove handler implementations from fault_handlers.cpp
- Remove unused helper functions (sanitize_filename, generate_timestamp, validate_fault_code)
- Remove unused includes (filesystem, fstream, iomanip)

Snapshots are now inline in fault responses (environment_data).
Rosbag downloads use bulk-data endpoint pattern (GET /{entity}/bulk-data/rosbags/{fault_code}).
Add comprehensive integration tests:
- Bulk-data category listing for all entity types
- BulkDataDescriptor structure validation
- Rosbag download with headers verification
- Security check for cross-entity access
- Nested entity path support
- SOVD-compliant fault response structure
- Legacy endpoints removal verification

@verifies REQ_INTEROP_071
@verifies REQ_INTEROP_072
@verifies REQ_INTEROP_073
@verifies REQ_INTEROP_013
- Add resolve_rosbag_file_path() to handle rosbag2 directory structure
- Fix MIME type from application/gzip to application/x-sqlite3 for db3 files
- Update unit tests for correct MIME type expectations
- Update integration tests for SQLite format validation
Update documentation for SOVD bulk-data feature:
- Add Bulk Data Endpoints section to REST API reference
- Update fault response with environment_data and SOVD status
- Rewrite snapshots tutorial for new API pattern
- Update changelog with breaking changes
- Mark REQ_INTEROP_071-073 as verified
- Update REQ_INTEROP_013 for fault environment data
- Remove REQ_INTEROP_088 (legacy snapshots endpoint)

Documents breaking changes and migration path.
Fix spelling typo propagated through the entire stack:
- ExtendedDataRecords.msg: first_occurence_ns → first_occurrence_ns
- fault_manager_node.cpp: field access updated
- fault_handlers.cpp: JSON keys updated
- All test files updated to match

This is a breaking change in the ROS message API, fixed now before
the bulk-data feature is merged to main.
P0: Replace memory-loading file transfer with chunked content provider
using httplib::set_content_provider() with 64KB chunks. Rosbag files
can be hundreds of MB to multiple GB.

P0: Add GetRosbags batch service to eliminate N+1 service calls in
handle_list_descriptors(). Uses existing get_rosbags_for_entity()
storage method via a new ROS 2 service.

P1: Add Content-Disposition and Content-Length to global CORS
Access-Control-Expose-Headers in both RESTServer and HandlerContext.

P0: Document mutex locking contract on FaultManager::service_mutex_
to prevent future deadlock risks in get_fault() delegation.
Consolidate duplicate timestamp formatting (to_iso8601_ns in
fault_handlers.cpp and format_timestamp_ns in bulkdata_handlers.cpp)
into a shared inline function in http_utils.hpp.

Add null check for gmtime_r return value — previously undefined
behavior on negative or invalid timestamps.

Add unit tests for the shared utility covering valid timestamps,
epoch, milliseconds, and negative timestamp fallback.
Apply consistent naming convention for ROS 2 service definitions:
- Get* = retrieve single item by key (GetFault, GetRosbag)
- List* = query collection with filters (ListFaults, ListRosbags,
  ListFaultsForEntity)

Renames .srv files, all C++ types/includes/variables/handlers,
Python imports, service endpoint names (~/list_faults, ~/list_rosbags),
storage method names, and documentation references.

BREAKING: Service endpoint names changed:
  ~/get_faults  → ~/list_faults
  ~/get_rosbags → ~/list_rosbags
@bburda bburda self-assigned this Feb 5, 2026
@bburda bburda added the enhancement New feature or request label Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant