How Audio Tagging Models Classify Multiple Sounds Simultaneously

Unlike single-label audio classifiers that assume only one sound is present at a time, audio tagging models assign independent probability scores to multiple sound classes simultaneously using per-class sigmoid outputs and binary cross-entropy loss. Google's YAMNet, a well-documented example, processes 16 kHz mono audio into 0.96-second mel spectrogram patches and scores them across 521 AudioSet sound categories, requiring a minimum of 975 ms of input to produce any output. Because AudioSet training labels only confirm a sound occurs somewhere within a ten-second clip rather than pinpointing when, these models learn under weak supervision and are more reliable at detecting whether a sound is present than at locating its precise onset or offset. The choice of pooling strategy significantly affects results: max-pooling across patches captures brief events like a single dog bark, while mean-pooling favors sustained sounds like continuous rain. This architectural distinction makes audio tagging a fundamentally different and more complex task than conventional single-label sound classification.