DataWeave Benchmark: Filter Twice Outperforms Partition Once for Email Validation

A performance comparison in DataWeave 2.0 tested two approaches for splitting bulk records into valid and invalid email lists: the partition function and a double filter method. The partition approach traverses the array once but requires importing the dw::core::Arrays module, while the filter approach traverses the array twice but needs no import. Benchmarks run via DataWeave CLI 2.12.2 showed that at both 5 and 50,000 records, the filter-twice method posted lower median execution times, with non-overlapping ranges confirming a consistent difference. At 50,000 records, filter twice recorded a median of 326.5 ms compared to 383 ms for partition once. Based on these results, the author recommends the filter-twice approach for its lower latency despite the additional traversal.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in