{"id":9651,"date":"2026-06-26T05:19:07","date_gmt":"2026-06-25T21:19:07","guid":{"rendered":"https:\/\/infernews.com\/blog\/dream\/"},"modified":"2026-06-26T07:22:59","modified_gmt":"2026-06-25T23:22:59","slug":"dream","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/dream\/","title":{"rendered":"DREAM\uff1a\u7528\u8a9e\u8a00\u6a21\u578b\u53cd\u5411\u6559\u6aa2\u7d22"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/06\/dream-banner.jpg\" alt=\"DREAM banner\"\/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">DREAM \u662f\u4e00\u500b<strong>\u7a20\u5bc6\u6aa2\u7d22\u5d4c\u5165\u8a13\u7df4\u65b9\u6cd5\uff0f\u7814\u7a76\u539f\u578b<\/strong>\uff0c\u6838\u5fc3\u662f\u628a autoregressive language model \u7684\u9810\u6e2c\u8a0a\u865f\u62ff\u4f86\u8a13\u7df4 dense retriever\u3002\u5b83\u8981\u89e3\u6c7a\u7684\u554f\u984c\u5f88\u660e\u78ba\uff1a\u50b3\u7d71 dense retrieval \u591a\u6578\u4f9d\u8cf4 contrastive objectives\uff0c\u9700\u8981\u6b63\u8ca0\u6587\u4ef6\u914d\u5c0d\u8207\u6a19\u8a3b\uff0c\u4f46\u9019\u985e\u8cc7\u6599\u6602\u8cb4\uff0chard negatives \u4e5f\u4e0d\u7a69\u5b9a\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u73fe\u6709\u505a\u6cd5\u901a\u5e38\u662f\u66ff query \u914d positive documents \u8207 sampled negatives\uff0c\u518d\u62c9\u8fd1\u6216\u62c9\u9060 embedding \u8ddd\u96e2\uff1b\u4f5c\u8005\u8a8d\u70ba\u9019\u7a2e\u7bc4\u5f0f\u904e\u5ea6\u4f9d\u8cf4\u4eba\u5de5\u6216\u984d\u5916\u6316\u6398\u6d41\u7a0b\uff0c\u672a\u5fc5\u771f\u6b63\u53cd\u6620\u54ea\u4e9b\u6587\u4ef6\u80fd\u5e6b\u52a9\u6a21\u578b\u5b8c\u6210\u751f\u6210\u3002DREAM \u7684\u505a\u6cd5\u662f\u628a query-document \u76f8\u4f3c\u5ea6\u9001\u5165\u6307\u5b9a\u7684 Query-Focused Retrieval Heads\uff08QRHeads\uff09\uff0c\u8b93 frozen LLM \u5728\u9810\u6e2c target \u6642\uff0c\u76f4\u63a5\u7528 next-token prediction loss \u56de\u50b3\u8a0a\u865f\uff0c\u544a\u8a34 retriever \u54ea\u4e9b\u6587\u4ef6\u771f\u7684\u6709\u7528\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u500b\u53d6\u5411\u6700\u503c\u5f97\u7559\u610f\u7684\u5730\u65b9\uff0c\u5728\u65bc\u5b83\u4e0d\u662f\u55ae\u7d14\u6539 loss\uff0c\u800c\u662f\u628a\u6aa2\u7d22\u5206\u6578\u63a5\u9032 attention heads\uff0c\u4ee4\u751f\u6210\u6a21\u578b\u7684\u9810\u6e2c\u96e3\u5ea6\u6210\u70ba\u76e3\u7763\u4f86\u6e90\u3002\u4ee3\u50f9\u4e5f\u5f88\u660e\u986f\uff1a\u6d41\u7a0b\u6bd4\u4e00\u822c embedding fine-tuning \u66f4\u8907\u96dc\uff0c\u8981\u5148\u505a QRHead detection\uff0c\u518d\u8dd1 DREAM adapter \u8a13\u7df4\uff1b\u5132\u5b58\u5eab\u4ea6\u672a\u9644\u5b8c\u6574 training data\u3001checkpoints \u8207 evaluation outputs\uff0c\u8f03\u63a5\u8fd1\u7814\u7a76\u5fa9\u73fe\u8def\u7dda\uff0c\u800c\u4e0d\u662f\u5373\u88dd\u5373\u7528\u5de5\u5177\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5b89\u88dd\u8207\u7406\u89e3\u65b9\u5f0f\u7b97\u6e05\u6670\uff0c\u5132\u5b58\u5eab\u5206\u6210 <code>qrhead_repo\/<\/code>\u3001<code>dream_routing\/<\/code> \u8207 <code>data\/sample\/<\/code> \u4e09\u90e8\u5206\uff1a\u524d\u8005\u8ca0\u8cac\u627e\u51fa QRHeads\uff0c\u5f8c\u8005\u8ca0\u8cac\u8a13\u7df4 adapter\uff0c\u6a23\u672c\u8cc7\u6599\u5247\u7528 JSONL \u63d0\u4f9b <code>query<\/code>\u3001<code>docs<\/code>\u3001<code>target<\/code> \u7d50\u69cb\u3002\u90e8\u7f72\u91cd\u9ede\u4e0d\u662f\u76f4\u63a5\u4e0a\u7dda\u670d\u52d9\uff0c\u800c\u662f\u5148\u6e96\u5099\u81ea\u5df1\u7684 Hugging Face dataset \u6216\u672c\u5730 JSONL\uff0c\u4f9d\u5e8f\u5b8c\u6210 head \u6aa2\u6e2c\u8207\u8a13\u7df4\uff1b\u63a8\u8ad6\u90e8\u5206\u5247\u4e3b\u8981\u4f9d\u8cf4 Hugging Face \u4e0a\u5df2\u91cb\u51fa\u7684 adapters\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5df2\u63d0\u4f9b\u9810\u8a13\u7df4\u6a21\u578b\uff1a<code>DREAM-0.5B<\/code>\u3001<code>DREAM-1B<\/code>\u3001<code>DREAM-3B<\/code><\/li>\n\n\n\n<li>\u5c0d\u61c9\u5e95\u5ea7\u6a21\u578b\uff1a<code>Qwen2.5-0.5B<\/code>\u3001<code>Llama-3.2-1B<\/code>\u3001<code>Llama-3.2-3B<\/code><\/li>\n\n\n\n<li>\u8a55\u6e2c\u6307\u5411 <code>BEIR<\/code> \u8207 <code>RTEB<\/code>\uff0c\u8ad6\u6587\u7a31\u5728\u4e0d\u540c\u6a21\u578b\u5c3a\u5bf8\u4e0a\u90fd\u512a\u65bc\u65e2\u6709 baselines<\/li>\n\n\n\n<li>\u9069\u5408\u7814\u7a76\u6aa2\u7d22\u8a13\u7df4\u3001RAG\u3001embedding \u8a2d\u8a08\u8207 LLM-retriever \u5354\u540c\u512a\u5316\u7684\u5718\u968a<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\u53d7\u76ca\u6700\u5927\u7684\u4e00\u985e\u4eba\uff0c\u4e0d\u662f\u53ea\u60f3\u4e0b\u8f09 embedding \u5373\u7528\u7684\u4f7f\u7528\u8005\uff0c\u800c\u662f\u8981\u7814\u7a76 retriever \u5982\u4f55\u914d\u5408\u751f\u6210\u6a21\u578b\u5de5\u4f5c\u7684\u5718\u968a\u3002\u5c0d\u505a RAG\u3001\u77e5\u8b58\u6aa2\u7d22\u3001\u4ee3\u7406\u5f0f\u641c\u5c0b\u7684\u4eba\u4f86\u8aaa\uff0cDREAM \u63d0\u4f9b\u4e86\u4e00\u689d\u4e0d\u540c\u65bc contrastive training \u7684\u8def\uff1b\u5c0d\u8cc7\u6e90\u6709\u9650\u7684\u5c0f\u5718\u968a\u800c\u8a00\uff0c\u8a13\u7df4\u93c8\u8f03\u9577\u3001\u91cd\u73fe\u9580\u6abb\u8f03\u9ad8\uff0c\u8f03\u9069\u5408\u4f5c\u70ba\u65b9\u6cd5\u53c3\u8003\u6216\u5be6\u9a57\u57fa\u7dda\uff0c\u800c\u975e\u73fe\u6210\u7522\u54c1\u5143\u4ef6\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>GitHub\uff1a<\/strong> <a href=\"https:\/\/github.com\/yixuantt\/DREAM\" rel=\"noopener noreferrer\" target=\"_blank\">https:\/\/github.com\/yixuantt\/DREAM<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Model\uff1a<\/strong> <a href=\"https:\/\/huggingface.co\/collections\/yixuantt\/dream\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/huggingface.co\/collections\/yixuantt\/dream<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>DREAM \u4e0d\u662f\u4e00\u822c\u5c0d\u6bd4\u5f0f\u6aa2\u7d22\u8a13\u7df4\uff0c\u800c\u662f\u501f\u7528\u81ea\u56de\u6b78\u8a9e\u8a00\u6a21\u578b\u7684\u9810\u6e2c\u8aa4\u5dee\u4f86\u6559 retriever\u3002\u5b83\u7784\u6e96\u6a19\u8a3b\u6210\u672c\u9ad8\u3001\u8ca0\u6a23\u672c\u96e3\u6316\u7684\u8001\u554f\u984c\u3002<\/p>\n","protected":false},"author":8,"featured_media":9650,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ai_generated_summary":"","footnotes":""},"categories":[133,185,163,165,38,131,40,152,109,150,76,127,188,199],"tags":[],"class_list":["post-9651","post","type-post","status-publish","format-standard","hentry","category-133","category-qwen","category-163","category-165","category-38","category-embedding","category-llama","category-python","category-rag","category-150","category-76","category-127","category-meta","category-dataset-"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/9651","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=9651"}],"version-history":[{"count":2,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/9651\/revisions"}],"predecessor-version":[{"id":9654,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/9651\/revisions\/9654"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/9650"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=9651"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=9651"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=9651"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}