Apache Sourcelume Adds Machine-Readable Provenance Records for Datasets and Models
Apache Sourcelume is a versioned JSON-LD specification and registry, built on Apache Atlas, for recording dataset origin, custody chain, and licensing in a checkable form. It does not certify licensing claims but aims to make such claims publishable as stable, comparable records. A new TrainingProvenanceRecord type lets model producers link their models to the datasets used in training, including token counts and shares, and to state whether disclosure is complete, partial, or undisclosed. A demonstration repository trains a small character-level model on a public-domain text and writes its provenance record during the same run. The project's sample code is available on GitHub under the apache organization.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in