feat(v4): add component-level generic_data support

[janickm]

Summary

• Add set_generic_data(data, meta_data) method to ComponentWriter for storing named numpy arrays and additional metadata at the component level
• Defer component .zattrs writing from register_component_writer() to finalize() to support itar stores (which reject duplicate key writes)
• Add backwards-compatible reader methods (has_generic_data, get_generic_data_names, get_generic_data) to ComponentReader

Motivation

Converters sometimes need to attach auxiliary time-series data to components without defining a custom component type. For example, a KITTI converter needs to store raw GPS/IMU measurements (30 float values per frame at 10Hz) alongside the derived ego poses. This feature provides that extensibility at the component level, mirroring the existing per-frame generic_data mechanism on sensor components.

API

Writer

poses_writer.set_generic_data(
    data={"oxts_data": oxts_array, "oxts_timestamps_us": timestamps},
    meta_data={"oxts_field_names": ["lat", "lon", "alt", ...]},
)

• meta_data keys overwrite init-time generic_meta_data keys
• Arrays stored with LZ4+bitshuffle compression in a generic_data/ subgroup

Reader (backwards-compatible)

reader.has_generic_data("oxts_data")        # -> bool (False for old stores)
reader.get_generic_data_names()             # -> List[str] ([] for old stores)
reader.get_generic_data("oxts_data")        # -> np.ndarray

Changes

File	Description
ncore/impl/data/v4/components.py	Core implementation: deferred .zattrs, set_generic_data, reader methods, super().finalize() in all subclasses
ncore/impl/data/v4/components_test.py	4 new tests: roundtrip, backwards-compat, overwrite semantics, meta-only
docs/data/formats.rst	Documentation for the new component-level generic_data section

Testing

All existing tests pass unchanged. New tests cover:

• Full write/read roundtrip (both itar and directory store types)
• Backwards compatibility (stores without generic_data/ subgroup)
• Metadata overwrite semantics (set_generic_data meta overwrites init-time keys)
• Meta-only usage (empty data dict, metadata-only addition)

No new dependencies required.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[janickm]

/ok to test cf53d3d