Matten library adds dynamic feature flag to handle messy, mixed-type tensor data
The Rust library matten has introduced a 'dynamic' feature flag designed to ingest heterogeneous real-world data from sources such as JSON APIs and CSV files. When enabled, the feature allows tensors to hold mixed cell types including floats, integers, booleans, text, and null values, each stored as a typed Element variant. Developers can inspect missing values and type distributions using tools like none_mask() and schema_summary() before attempting conversion. Conversion to a standard numeric tensor is deliberately explicit: try_numeric() rejects any non-numeric cells, while try_numeric_with() accepts a NumericPolicy that defines how nulls, booleans, and text should be handled. Arithmetic operations, reshaping, and serialization are intentionally unsupported on dynamic tensors, keeping the feature focused solely on data ingestion and cleaning.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in