Skip to content

Feat/c python timeseries metadata#767

Open
hongzhi-gao wants to merge 3 commits intodevelopfrom
feat/c-python-timeseries-metadata
Open

Feat/c python timeseries metadata#767
hongzhi-gao wants to merge 3 commits intodevelopfrom
feat/c-python-timeseries-metadata

Conversation

@hongzhi-gao
Copy link
Copy Markdown
Contributor

Summary

Adds device enumeration and per-device timeseries metadata to the C wrapper, with matching Python bindings on TsFileReader. A follow-up extends TimeseriesStatistic (numeric min/max/first/last, boolean first/last, string stats)


C API (tsfile_cwrapper.h)

Types: DeviceID, TimeseriesStatistic, TimeseriesMetadata, DeviceTimeseriesMetadataEntry, DeviceTimeseriesMetadataMap.
Sentinel: tsfile_c_metadata_empty_device_list_marker — use with length == 0 when you need a non-null device_ids pointer for an empty query.

Functions

  • tsfile_reader_get_all_devicesout_devices / out_length; free with tsfile_free_device_id_array.
  • tsfile_reader_get_timeseries_metadataout_map; free with tsfile_free_device_timeseries_metadata_map.

Query semantics (tsfile_reader_get_timeseries_metadata)

device_ids length Result
NULL (ignored) All devices in the file
non-NULL 0 Empty map, E_OK (IDs not read)
non-NULL > 0 Subset; only devices that exist appear

Errors: If length > 0 and any device_ids[i].path == NULLE_INVALID_ARG / RET_INVALID_ARG.

Memory: All heap data in out_map (including TimeseriesStatistic.str_*) is owned by the API; only release via tsfile_free_device_timeseries_metadata_map. Do not free str_* individually.

TimeseriesStatistic (which flags matter)

  • int_range_valid + *int64INT32, DATE, INT64, TIMESTAMP
  • float_range_valid + *float64FLOAT, DOUBLE
  • bool_ext_valid + first_bool / last_boolBOOLEAN (plus existing sum_valid / sum)
  • str_ext_valid + str_*STRING (lexicographic min/max, time-ordered first/last); TEXTfirst/last only; do not rely on min/max for TEXT

Python API

Schema: DeviceID, TimeseriesStatistic (extended; string fields Optional[str]), TimeseriesMetadata.

TsFileReader

  • get_all_devices() -> list[DeviceID]
  • get_timeseries_metadata(device_ids=None) -> dict[str, list[TimeseriesMetadata]]
    • None → all devices; []{}; non-empty list → filter (DeviceID or str paths).

Native memory is freed inside the extension; use the returned dict/objects only.

Example

reader = TsFileReader("file.tsfile")
devices = reader.get_all_devices()
meta = reader.get_timeseries_metadata(None)
sub = reader.get_timeseries_metadata([DeviceID("root.sg.d1"), "root.sg.d2"])
reader.close()

Tests

  • C++: cpp/test/cwrapper/cwrapper_metadata_test.cc
  • Python: python/tests/test_reader_metadata.py

Note: Order of devices/series follows the native C++ map/container iteration unless you document a stronger guarantee later.

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 51.32743% with 110 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.68%. Comparing base (b6249d9) to head (135a2c0).

Files with missing lines Patch % Lines
cpp/src/cwrapper/tsfile_cwrapper.cc 51.32% 85 Missing and 25 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #767      +/-   ##
===========================================
- Coverage    62.73%   62.68%   -0.06%     
===========================================
  Files          705      705              
  Lines        42165    42393     +228     
  Branches      6223     6268      +45     
===========================================
+ Hits         26453    26574     +121     
- Misses       14781    14863      +82     
- Partials       931      956      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants