How to run email subject-line A/B tests using an agent mailbox without an ESP

Developers can conduct email subject-line A/B testing directly through an agent-owned mailbox, eliminating the need for a third-party email service provider (ESP). The approach works by splitting recipients into two groups, sending each a different subject line, and tracking which variant generates more replies. Using the Nylas API, the agent sends messages via POST requests and monitors responses through webhook events on the same mailbox it already operates. Unlike traditional ESP split tests that measure opens or clicks, this method uses reply rate as its signal, which is considered a stronger indicator for outreach or sales use cases. All experiment logic — including random assignment, sample sizing, and winner determination — remains in the developer's own codebase, keeping recipient data within their system.