{"id":13040,"date":"2026-10-09T02:36:53","date_gmt":"2026-10-08T18:36:53","guid":{"rendered":"https:\/\/infernews.com\/blog\/rl-only-domain-adaptation-from-base-models-via-off-policy-seeding\/"},"modified":"2026-10-09T02:36:53","modified_gmt":"2026-10-08T18:36:53","slug":"rl-only-domain-adaptation-from-base-models-via-off-policy-seeding","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/rl-only-domain-adaptation-from-base-models-via-off-policy-seeding\/","title":{"rendered":"HuatuoGPT-3 \u4ee5\u55ae\u968e\u6bb5\u5f37\u5316\u5b78\u7fd2\u8abf\u6574\u91ab\u7642\u8a9e\u8a00\u6a21\u578b"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/10\/pasted-27abb5922e4d.jpg\" alt=\"OnePO framework\"><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u60f3\u628a\u8a9e\u8a00\u6a21\u578b\u8abf\u6574\u5230\u91ab\u7642\u554f\u7b54\uff0cHuatuoGPT-3 \u5617\u8a66\u6e1b\u5c11\u4e00\u500b\u8a13\u7df4\u968e\u6bb5\uff1a\u5b83\u662f\u4ee5 One-stage Policy Optimization\uff08OnePO\uff09\u8a13\u7df4\u7684\u91ab\u7642\u5927\u578b\u8a9e\u8a00\u6a21\u578b\u7cfb\u5217\uff0c\u76ee\u6a19\u662f\u5728\u4e0d\u5148\u505a\u9818\u57df\u76e3\u7763\u5fae\u8abf\uff08SFT\uff09\u7684\u60c5\u6cc1\u4e0b\uff0c\u900f\u904e\u55ae\u968e\u6bb5\u5f37\u5316\u5b78\u7fd2\u9069\u61c9\u91ab\u7642\u4efb\u52d9\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">OnePO \u6703\u5148\u501f\u52a9\u6559\u5e2b\u6a21\u578b\u7684\u56de\u7b54\u63d0\u4f9b\u6307\u5f15\uff0c\u518d\u6309\u6a21\u578b\u5b78\u7fd2\u9032\u5ea6\u8abf\u6574\u8a13\u7df4\u65b9\u5f0f\u3002Adaptive Objective Evolution \u5354\u52a9\u6a21\u578b\u5b78\u7fd2\u539f\u672c\u8f03\u5c11\u9078\u51fa\u7684\u6559\u5e2b\u7b54\u6848 token\uff1b\u7576\u6a21\u578b\u81ea\u5df1\u7684\u56de\u7b54\u6240\u5f97 reward \u5df2\u8ffd\u4e0a\u6216\u8d85\u8d8a\u6559\u5e2b\uff0cTeacher Retirement \u4fbf\u505c\u6b62\u6559\u5e2b\u6307\u5f15\u3002\u9019\u505a\u6cd5\u7701\u53bb\u9810\u5148\u9032\u884c\u91ab\u7642 SFT \u7684\u8981\u6c42\uff0c\u4f46\u8a13\u7df4\u4ecd\u4f9d\u8cf4\u6559\u5e2b\u56de\u7b54\u3001reward \u8a2d\u8a08\u548c\u4efb\u52d9\u8cc7\u6599\u7684\u8cea\u7d20\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6a21\u578b\u7cfb\u5217\u5305\u62ec HuatuoGPT-3-9B \u548c HuatuoGPT-3-27B\uff0c\u5206\u5225\u4ee5 Qwen3.5-9B \u548c Qwen3.8-27B \u70ba\u57fa\u5e95\uff1b\u53e6\u6709\u4ee5 Qwen3-8B \u70ba\u57fa\u5e95\u7684 HuatuoGPT-3-Grader-8B\u3002HealthBench Professional \u5206\u6578\u65b9\u9762\uff0c9B \u7248\u70ba 68.3\uff0c27B \u7248\u70ba 71.4\uff1b\u5009\u5eab\u6c92\u6709\u5217\u51fa Grader \u7684\u540c\u9805\u5206\u6578\uff0c\u56e0\u6b64\u4e0d\u80fd\u64da\u6b64\u6bd4\u8f03\u8a55\u5206\u5668\u8868\u73fe\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>20,338 \u9805 OnePO-Medical-20K \u91ab\u7642\u5f37\u5316\u5b78\u7fd2\u4efb\u52d9\uff0c\u6db5\u84cb\u9078\u64c7\u984c\u548c\u958b\u653e\u5f0f\u56de\u7b54\u3002<\/li>\n<li>\u9078\u64c7\u984c\u4ee5\u7b54\u6848\u662f\u5426\u5b8c\u5168\u543b\u5408\u9a57\u8b49\uff0c\u958b\u653e\u5f0f\u56de\u7b54\u5247\u6309\u591a\u9805\u81e8\u5e8a\u6e96\u5247\u8a55\u5206\u3002<\/li>\n<li>\u63d0\u4f9b\u6a21\u578b\u3001\u8a13\u7df4\u7a0b\u5f0f\u78bc\u3001\u8cc7\u6599\u96c6\u53ca rubric grader\uff0c\u9069\u5408\u7814\u7a76\u91ab\u7642\u6a21\u578b\u8a13\u7df4\u7684\u5718\u968a\u6aa2\u8996\u6216\u91cd\u73fe\u6d41\u7a0b\u3002<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\u6a21\u578b\u53ef\u6309 Qwen3.5 \u7684\u65b9\u5f0f\u4f7f\u7528\uff0c\u4e26\u900f\u904e vLLM \u6216 SGLang \u90e8\u7f72\uff1b\u8a13\u7df4\u5247\u9700\u5148\u6309\u6587\u4ef6\u6e96\u5099 GPU \u74b0\u5883\u3002\u5c0d\u91ab\u7642\u670d\u52d9\u5718\u968a\u800c\u8a00\uff0c\u9019\u4e9b\u6a21\u578b\u53ef\u4f5c\u7814\u7a76\u8207\u8a55\u4f30\u8d77\u9ede\uff0c\u4f46\u5009\u5eab\u5217\u51fa\u7684\u57fa\u6e96\u5206\u6578\u672c\u8eab\u4e0d\u8db3\u4ee5\u8b49\u660e\u6a21\u578b\u9069\u5408\u81e8\u5e8a\u6c7a\u7b56\uff0c\u4ecd\u9700\u6309\u76ee\u6a19\u7528\u9014\u9032\u884c\u5b89\u5168\u6027\u3001\u6e96\u78ba\u5ea6\u53ca\u81e8\u5e8a\u8a55\u4f30\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/FreedomIntelligence\/HuatuoGPT-3\" rel=\"noopener noreferrer\"><strong>GitHub<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u91ab\u7642\u6a21\u578b\u901a\u5e38\u8981\u5148\u7d93\u5c08\u9580\u76e3\u7763\u5fae\u8abf\uff0c\u518d\u9032\u884c\u5f37\u5316\u5b78\u7fd2\uff1bHuatuoGPT-3 \u5617\u8a66\u7701\u53bb\u524d\u4e00\u6b65\uff0c\u76f4\u63a5\u4ee5 OnePO \u9069\u61c9\u91ab\u7642\u4efb\u52d9\u3002<\/p>\n","protected":false},"author":8,"featured_media":13039,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wpai_generated_summary":"","wpai_meta_description":"","footnotes":""},"categories":[133,76,127,185,144,150,170],"tags":[],"class_list":["post-13040","post","type-post","status-publish","format-standard","hentry","category-133","category-76","category-127","category-qwen","category-medical","category-150","category-170"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/13040","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=13040"}],"version-history":[{"count":0,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/13040\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/13039"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=13040"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=13040"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=13040"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}