Converting LangSmith Eval Datasets to RAGAS Without Silent Field Loss
A developer moving an evaluation set from LangSmith to RAGAS found that the two tools represent datasets differently, with RAGAS using columnar parallel arrays and LangSmith using free-form examples. The conversion is essentially a reshape that maps four fields, while fields such as id, metadata, and timestamps have no equivalent in RAGAS's SingleTurnSample and are silently dropped. Mismatched key names, such as inputs.query instead of inputs.question, can also produce empty questions that RAGAS loads without error, skewing downstream metrics. The article recommends renaming keys before or after conversion and checking that every user_input is non-empty. It also provides a sample conversion script that reads examples through the LangSmith client and builds RAGAS samples from the expected keys.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in