{"id":10010,"date":"2026-07-08T18:15:12","date_gmt":"2026-07-08T10:15:12","guid":{"rendered":"https:\/\/infernews.com\/blog\/official-code-for-quot-do-all-visual-tokens-matter-equally-object-evidence-prese\/"},"modified":"2026-07-08T18:15:12","modified_gmt":"2026-07-08T10:15:12","slug":"official-code-for-quot-do-all-visual-tokens-matter-equally-object-evidence-prese","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/official-code-for-quot-do-all-visual-tokens-matter-equally-object-evidence-prese\/","title":{"rendered":"SaMer \u58d3\u7e2e\u8996\u89ba\u6aa2\u7d22\u6210\u672c"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/07\/pasted-0cbaa85a6875.jpg\" alt=\"SaMer overview\"><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u5716\u6587\u6aa2\u7d22\u505a\u5f97\u7d30\u7dfb\uff0c\u5f80\u5f80\u8981\u4fdd\u7559\u5927\u91cf image-side tokens\uff0c\u4f46\u5132\u5b58\u8207\u6bd4\u5c0d\u6210\u672c\u4e5f\u6703\u8ddf\u4f4f\u5347\u3002SaMer\u5c6c\u65bc<strong>\u958b\u6e90\u6846\u67b6\uff0f\u7814\u7a76\u539f\u578b<\/strong>\uff0c\u91dd\u5c0d ColPali-style multi-vector retrievers \u7684 token \u58d3\u7e2e\u554f\u984c\u800c\u4f86\uff0c\u91cd\u9ede\u4e0d\u662f\u55ae\u7d14\u522a\u6e1b\uff0c\u800c\u662f\u76e1\u91cf\u4fdd\u4f4f\u4e4b\u5f8c\u67e5\u8a62\u4ecd\u53ef\u80fd\u7528\u5f97\u7740\u7684 object-level evidence\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u73fe\u6709\u58d3\u7e2e\u505a\u6cd5\u591a\u6578\u504f\u5411 pruning\u3001pooling\uff0c\u6216\u8005\u53ea\u6309 feature \u5408\u4f75\uff0c\u4ee3\u50f9\u662f\u7269\u4ef6\u8207\u5340\u57df\u8b49\u64da\u5bb9\u6613\u88ab\u6df7\u5728\u4e00\u8d77\uff0c\u4ee4 MaxSim late interaction \u539f\u672c\u53ef\u9078\u53d6\u7684\u7d30\u7bc0\u8a0a\u865f\u6d41\u5931\u3002SaMer\u7684\u53d6\u5411\u660e\u78ba\uff1a\u4fdd\u7559\u539f\u672c MaxSim retrieval objective\uff0c\u4e0d\u6539 late-interaction \u4ecb\u9762\uff0c\u53ea\u5728 image side \u505a object-aware merge\uff0c\u4fee\u6b63\u300c\u58d3\u7e2e\u5f8c\u8b49\u64da\u8b8a\u920d\u300d\u9019\u500b\u554f\u984c\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5b83\u7684\u65b9\u6cd5\u6709\u4e09\u500b\u95dc\u9375\u6b65\u9a5f\uff1a\u5148\u7528 feature similarity \u52a0 spatial proximity \u505a Feature-Spatial Merging\uff0c\u518d\u5728\u8a13\u7df4\u968e\u6bb5\u5229\u7528 Flickr30k-Entities \u7684\u6846\u6a19\u8a3b\u52a0\u5165 Object-Aware Assignment\uff0c\u907f\u514d\u4e0d\u540c\u5be6\u4f8b\u88ab\u932f\u8aa4\u5408\u4f75\uff0c\u6700\u5f8c\u63a1\u7528 Projection-Only Adaptation\uff0c\u53ea\u8a13\u7df4 shared projection layer\uff0cvision encoder \u540c language backbone \u90fd\u7dad\u6301 frozen\u3002\u5230 inference \u6642\u5c31\u4e0d\u518d\u4f9d\u8cf4 bbox\uff0c\u9019\u9ede\u4ee4\u90e8\u7f72\u6bd4\u4e00\u4e9b\u9700\u8981 detector \u6216\u984d\u5916 grounding loss \u7684\u505a\u6cd5\u7c21\u6f54\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u4ee5 ColPali-style multi-vector retrieval \u70ba\u6838\u5fc3\u5834\u666f\uff0c\u91cd\u9ede\u662f\u6e1b storage \u8207 scoring \u6210\u672c<\/li>\n<li>K=64 \u6642\u53ef\u79fb\u9664\u8d85\u904e 93% image-side tokens\uff0cColPali \u5132\u5b58\u91cf\u4e0b\u964d 16.09\u00d7<\/li>\n<li>\u5728 Flickr30K \u8207 MSCOCO\uff0cREADME \u8207\u9805\u76ee\u9801\u6307\u51fa R@1 \u6709\u63d0\u5347<\/li>\n<li>\u76f8\u95dc\u6a21\u578b\u6b0a\u91cd\u5305\u62ec <code>vidore\/colpali-v1.3-hf<\/code>\uff0c\u53e6\u6709 Hugging Face \u7684 SaMer models \u96c6\u5408<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\u6e2c\u8a66\u8207\u91cd\u73fe\u8def\u7dda\u7b97\u6e05\u695a\uff1a\u8cc7\u6599\u8981\u5148\u6e96\u5099 Flickr30k-Entities\uff0c\u4e4b\u5f8c\u8a13\u7df4 adapter\u3001\u5efa\u7acb compressed retrieval cache\uff0c\u518d\u505a inference\u3002\u539f\u59cb\u8cc7\u6599\u5217\u51fa Python 3.10+\u3001CUDA-enabled PyTorch\uff0c\u4ee5\u53ca\u55ae GPU\uff0f\u591a GPU \u8173\u672c\u6d41\u7a0b\uff0c\u4f46 evaluation \u7ae0\u7bc0\u5728\u63d0\u4f9b\u5167\u5bb9\u4e2d\u672a\u5b8c\u6574\u5c55\u958b\uff0c\u6240\u4ee5\u66f4\u7d30\u7684 benchmark \u8a2d\u5b9a\u8207\u5b8c\u6574\u6578\u5b57\u4ecd\u8981\u4ee5\u8ad6\u6587\u8207\u9805\u76ee\u9801\u70ba\u6e96\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u985e\u505a\u6cd5\u6700\u9069\u5408\u9700\u8981\u591a\u5411\u91cf\u5716\u6587\u6aa2\u7d22\u3001\u53c8\u5728\u610f\u8a18\u61b6\u9ad4\u8207\u5ef6\u9072\u7684\u5718\u968a\uff0c\u4f8b\u5982\u505a large-scale image retrieval\u3001phrase-level grounding\uff0c\u6216\u8005\u8981\u628a ColPali \u985e\u7cfb\u7d71\u58d3\u5230\u8f03\u53ef\u90e8\u7f72\u898f\u6a21\u7684\u7814\u7a76\u8207\u5de5\u7a0b\u9805\u76ee\u3002\u5b83\u7684\u53d6\u6368\u4ea6\u5f88\u6e05\u6670\uff1a\u4e0d\u662f\u8ffd\u6c42\u6700\u5c11 token\uff0c\u800c\u662f\u7528\u8f03\u4f4e\u6210\u672c\u4fdd\u7559\u65e5\u5f8c\u67e5\u8a62\u4ecd\u6709\u6a5f\u6703\u9078\u4e2d\u7684\u8996\u89ba\u8b49\u64da\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/suhyeong10.github.io\/samer-project-page\/\" rel=\"noopener noreferrer\"><strong>\u9805\u76ee\u4e3b\u9801<\/strong><\/a> \u00b7 <a href=\"https:\/\/github.com\/dmis-lab\/SaMer\" rel=\"noopener noreferrer\"><strong>GitHub<\/strong><\/a> \u00b7 <a href=\"https:\/\/huggingface.co\/collections\/dmis-lab\/samer\" rel=\"noopener noreferrer\"><strong>\u6a21\u578b<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u60f3\u4fdd\u7559\u5716\u50cf\u6aa2\u7d22\u7684\u7d30\u7bc0\u5224\u65b7\uff0c\u53c8\u4e0d\u60f3\u88ab\u6d77\u91cf visual tokens \u62d6\u6162\uff0cSaMer\u6b63\u6b63\u8655\u7406\u9019\u500b\u77db\u76fe\u3002\u5b83\u7528\u8f03\u8070\u660e\u7684\u5408\u4f75\u65b9\u5f0f\u58d3\u7e2e token\uff0c\u540c\u6642\u76e1\u91cf\u5b88\u4f4f\u6aa2\u7d22\u6e96\u78ba\u5ea6\u3002<\/p>\n","protected":false},"author":8,"featured_media":10009,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ai_generated_summary":"","footnotes":""},"categories":[133,185,179,30,152,119,149,199],"tags":[],"class_list":["post-10010","post","type-post","status-publish","format-standard","hentry","category-133","category-qwen","category-nvidia","category-image","category-python","category-119","category-149","category-dataset-"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/10010","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=10010"}],"version-history":[{"count":0,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/10010\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/10009"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=10010"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=10010"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=10010"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}