Audio Embeddings vs. Metadata: Why Music Recommendation Design Choices Matter

Music recommendation systems broadly follow two approaches: collaborative filtering, which models user-track interactions, and content-based filtering, which derives recommendations directly from audio embeddings. Collaborative filtering excels at capturing cultural context — such as shared scenes or playlists — but structurally fails for new or obscure tracks that lack interaction data, a problem no amount of additional users can fix. Content-based systems solve the cold-start problem by generating embeddings the moment a track is uploaded, making new releases immediately recommendable. However, audio embeddings primarily capture timbral and rhythmic features like tempo and instrumentation, which do not always reflect how listeners actually group or relate music. The two methods fail in opposite directions, and understanding which failure applies to a given system is central to making sound architectural decisions.