Just Respect the Semantics: Image-Token Compression via Pre-LLM Semantic Variable-Resolution Tokenization and In-LLM Relevance Pruning
Researchers introduced Varest, a method for compressing image tokens in AI models without degrading their semantic meaning. Varest uses object and text localizers to merge tokens coarsely within semantic regions, preserving finer details at object boundaries and text. It then uses learned relevance registers to prune unnecessary visual tokens after early decoder processing. Varest outperforms 15 baselines in retaining image token performance across various budgets, achieving 86.10-92.33% of full-token performance with only 5.56% of the original token count. This matters to AI builders as Varest enables more efficient image processing while maintaining model performance, potentially reducing computational costs and memory usage.
Save an API key to vote.