{"id":11451,"date":"2026-08-26T14:11:22","date_gmt":"2026-08-26T06:11:22","guid":{"rendered":"https:\/\/infernews.com\/blog\/on-policy-self-distillation-in-diffusion-models\/"},"modified":"2026-08-26T14:15:23","modified_gmt":"2026-08-26T06:15:23","slug":"on-policy-self-distillation-in-diffusion-models","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/on-policy-self-distillation-in-diffusion-models\/","title":{"rendered":"DiffusionOPSD\uff1a\u5c07\u81ea\u6211\u84b8\u993e\u5f71\u50cf\u734e\u52f5\u8f49\u70ba\u53ef\u66f4\u65b0\u8a13\u7df4\u76ee\u6a19"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/08\/qualitative_gallery.png\" alt=\"DiffusionOPSD qualitative gallery\"\/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">DiffusionOPSD(ByteDance Seed) \u4e3b\u653b\u7684\u662f diffusion \u6a21\u578b\u5f8c\u8a13\u7df4\u6642\uff0c\u53ea\u6709\u6700\u7d42\u5716\u7247\u5206\u6578\u3001\u6b20\u7f3a\u4e2d\u9593\u6b65\u9a5f\u6307\u5f15\u7684\u554f\u984c\u3002\u5b83\u5c6c\u65bc\u4e00\u5957\u7528\u65bc\u6a21\u578b\u5f8c\u8a13\u7df4\u7684\u958b\u6e90\u65b9\u6cd5\uff0c\u900f\u904e on-policy self-distillation\uff0c\u628a\u5f71\u50cf\u7d1a reward \u8f49\u6210\u53ef\u76f4\u63a5\u5b78\u7fd2\u7684\u4e2d\u9593\u76ee\u6a19\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5b83\u7684\u505a\u6cd5\u4e0d\u662f\u55ae\u7d14\u8ffd\u5206\uff0c\u800c\u662f\u5148\u7528\u51cd\u7d50\u7684 behavior policy \u6536\u96c6\u4f4e\u566a\u8072\u67e5\u8a62\u72c0\u614b\uff0c\u518d\u7528 reward gradient \u5efa\u51fa\u6b63\u8ca0\u5169\u7a2e bounded target\uff0c\u6700\u5f8c\u7531\u53ef\u8a13\u7df4 policy \u53bb\u64ec\u5408\u9019\u4e9b\u5207\u65b7\u8a08\u7b97\u5716\u7684\u76ee\u6a19\u3002\u9019\u7a2e\u5b89\u6392\u628a\u76ee\u6a19\u54c1\u8cea\u548c\u6a21\u578b\u66f4\u65b0\u5206\u958b\u8655\u7406\uff0c\u4ea6\u4ee4 supervision \u53ef\u4ee5\u96a8\u4f4f\u6a21\u578b\u6f14\u5316\u6301\u7e8c\u5237\u65b0\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5be6\u4f5c\u5c64\u9762\uff0c\u516c\u958b\u7248\u672c\u652f\u63f4 SD3.5-M 512\u00b2\uff0c\u540c\u6642\u8986\u84cb\u539f\u751f few-step \u7684 Z-Image-Turbo 1024\u00b2 \u8a2d\u5b9a\uff1b\u6587\u4ef6\u4ea6\u63d0\u5230\u53ef\u505a\u55ae\u4e00 reward \u6216\u591a\u500b reward \u7684\u52a0\u6b0a\u8a13\u7df4\u3002\u539f\u6587\u672a\u63d0\u4f9b\u5b8c\u6574\u5b89\u88dd\u6d41\u7a0b\u6216\u9010\u6b65\u4f7f\u7528\u6307\u5f15\uff0c\u53ea\u80fd\u78ba\u8a8d\u5b83\u6709\u9805\u76ee\u9801\u8207\u7a0b\u5f0f\u78bc\u91cb\u51fa\uff0c\u9069\u5408\u81ea\u884c\u62c9\u53d6\u5f8c\u4f9d\u7814\u7a76\u74b0\u5883\u914d\u7f6e\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5b83\u548c\u540c\u985e diffusion reward optimization \u7684\u4e3b\u8981\u5206\u5225\uff0c\u5728\u65bc\u7528 on-policy \u67e5\u8a62\u914d\u5408 EMA \u6301\u7e8c\u91cd\u5efa supervision\uff0c\u800c\u4e0d\u662f\u53ea\u4f9d\u8cf4\u96e2\u7dda\u6216\u56fa\u5b9a\u566a\u8072\u66ff\u4ee3\u72c0\u614b\u3002\u5b98\u65b9\u7d50\u679c\u8072\u7a31\u5728\u591a\u500b evaluator \u8207\u5169\u7a2e backbone \u4e0a\u90fd\u6709\u8f03\u5f37\u7684\u6700\u7d42 held-out \u8868\u73fe\uff0c\u4ea6\u4ee5\u8f03\u5c11 GPU-hours \u5b8c\u6210\u8a13\u7df4\uff1b\u9019\u4ee4\u5b83\u7279\u5225\u9069\u5408\u505a\u751f\u6210\u6a21\u578b\u5f8c\u8a13\u7df4\u3001\u734e\u52f5\u5c0d\u9f4a\u8207\u65b9\u6cd5\u6bd4\u8f03\u7684\u7814\u7a76\u5718\u968a\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5c07 image-level reward \u8f49\u6210\u53ef\u5b78\u7fd2\u7684\u4e2d\u9593\u76ee\u6a19<\/li>\n\n\n\n<li>\u7528 EMA \u53cd\u8986\u5237\u65b0 trajectory\u3001anchor \u8207 target<\/li>\n\n\n\n<li>\u652f\u63f4 SD3.5-M \u8207 Z-Image-Turbo \u5169\u7a2e backbone<\/li>\n\n\n\n<li>\u53ef\u505a\u55ae\u4e00 reward\uff0c\u4ea6\u53ef\u505a\u591a reward \u52a0\u6b0a<\/li>\n\n\n\n<li>\u9069\u5408\u7814\u7a76 diffusion \u5f8c\u8a13\u7df4\u540c reward alignment \u7684\u5718\u968a<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/diffusionopsd.github.io\/\" rel=\"noopener noreferrer\"><strong>\u9805\u76ee\u4e3b\u9801<\/strong><\/a> \u00b7 <a href=\"https:\/\/github.com\/worldbench\/DiffusionOPSD\" rel=\"noopener noreferrer\"><strong>GitHub<\/strong><\/a> \u00b7 <a href=\"https:\/\/huggingface.co\/WeiChow\/DiffusionOPSD\" rel=\"noopener noreferrer\"><strong>\u6a21\u578b<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>DiffusionOPSD \u5c07\u5f71\u50cf\u7d1a\u734e\u52f5\u8f49\u6210\u53ef\u53cd\u8986\u66f4\u65b0\u7684\u4e2d\u9593\u76ee\u6a19\uff0c\u88dc\u4e0a diffusion \u8a13\u7df4\u4e2d\u9593\u6b65\u9a5f\u7684\u76e3\u7763\u7a7a\u767d\u3002\u5b83\u9069\u5408\u60f3\u5728\u6709\u9650\u8a13\u7df4\u9810\u7b97\u4e0b\uff0c\u6539\u826f\u751f\u6210\u8cea\u7d20\u540c\u53ef\u8a3a\u65b7\u6027\u7684\u5718\u968a\u3002<\/p>\n","protected":false},"author":8,"featured_media":11450,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wpai_generated_summary":"","footnotes":""},"categories":[133,165,176,127,30,120,129,197],"tags":[],"class_list":["post-11451","post","type-post","status-publish","format-standard","hentry","category-133","category-165","category-176","category-127","category-image","category-120","category-txt2img","category-framework"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/11451","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=11451"}],"version-history":[{"count":1,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/11451\/revisions"}],"predecessor-version":[{"id":11453,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/11451\/revisions\/11453"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/11450"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=11451"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=11451"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=11451"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}