Direct box regression head speeds up visual grounding and raises accuracy
Current visual grounding systems generate box coordinates token by token using autoregressive decoders, which adds latency to what is essentially a localization task. The Jag model replaces this step with a regression head that reads the fused image-language representation and outputs all four box corners at once. In single-request inference it reports a mean end-to-end latency of 65.97 ms, and it improves accuracy over its autoregressive predecessor on every test split by 9.63 to 17.51 points. The reported gains depend on jointly fine-tuning the full pretrained checkpoint, since freezing components erases the accuracy advantage. Whether the approach scales to very large multimodal models without excessive memory cost remains untested.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in