Mean Pooling vs CLS: How Transformers.js Turns a Sentence Into an Embedding in Node

A developer guide walks through generating text embeddings in Node using Transformers.js, version 4.3.1, which runs ONNX models via ONNX Runtime. The default feature-extraction settings return one vector per token rather than a single sentence vector, since pooling defaults to none and normalization is off. The example uses the all-MiniLM-L6-v2 model, which produces 384-dimensional embeddings and truncates input beyond 256 word pieces. The article explains how the tokenizer's WordPiece splitting, special tokens like [CLS], and attention masks affect the output. Mismatched pooling choices between indexing and querying can silently degrade retrieval quality without raising errors.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in