{"id":14625,"date":"2026-11-07T12:08:27","date_gmt":"2026-11-07T04:08:27","guid":{"rendered":"https:\/\/infernews.com\/blog\/sae-interprets-moe\/"},"modified":"2026-11-07T12:12:49","modified_gmt":"2026-11-07T04:12:49","slug":"sae-interprets-moe","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/sae-interprets-moe\/","title":{"rendered":"SAE Interprets MoE\uff1a\u62c6\u89e3\u6df7\u5408\u5c08\u5bb6\u7684\u5206\u5de5"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/11\/SAE-Interprets-MoE.jpg\" alt=\"Repository image for ByteDance-Seed\/SAE-interprets-MoE\"\/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u8981\u4e86\u89e3 MoE \u6a21\u578b\u88cf\u6bcf\u500b\u5c08\u5bb6\u5be6\u969b\u8ca0\u8cac\u751a\u9ebc\uff0c\u55ae\u8a08\u5b83\u88ab\u9078\u4e2d\u7684\u6b21\u6578\u4e26\u4e0d\u8db3\u5920\uff1a\u8def\u7531\u983b\u7387\u4e0d\u7b49\u65bc\u5b83\u5c0d\u8f38\u51fa\u7684\u8ca2\u737b\u3002SAE Interprets MoE \u63d0\u4f9b\u91cd\u73fe\u7814\u7a76\u5be6\u9a57\u7684\u7a0b\u5f0f\u78bc\u8207\u6578\u64da\uff0c\u900f\u904e\u7a00\u758f\u81ea\u7de8\u78bc\u5668\uff08Sparse Autoencoder\uff0cSAE\uff09\u5206\u6790 Qwen3-30B-A3B \u7b2c 4\u300130\u300146 \u5c64\u7684\u5c08\u5bb6\u5206\u5de5\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u7814\u7a76\u5229\u7528 SAE \u7de8\u78bc\u5668\u7684\u4eff\u5c04\u7279\u6027\uff0c\u628a\u6bcf\u500b\u7279\u5fb5\u7684\u555f\u52d5\u503c\u7cbe\u78ba\u62c6\u5206\u5230 MoE \u5c64\u4e2d\u88ab\u555f\u52d5\u7684\u5c08\u5bb6\uff0c\u518d\u6309\u8a9e\u6599\u5f59\u7e3d\u8ca2\u737b\u6bd4\u4f8b\uff0c\u5efa\u7acb\u5c08\u5bb6\u8207\u7279\u5fb5\u4e4b\u9593\u7684\u300c\u4e3b\u5c0e\u5716\u300d\u3002\u9019\u8207\u6309\u9810\u8a2d\u9818\u57df\u6a19\u7c64\u7d71\u8a08\u8def\u7531\u4e0d\u540c\uff0c\u5206\u6790\u7126\u9ede\u662f\u5c08\u5bb6\u771f\u6b63\u63a8\u52d5\u4e86\u54ea\u4e9b\u7279\u5fb5\uff1b\u7814\u7a76\u7d50\u679c\u6307\u51fa\uff0c<mark>\u591a\u6578\u7279\u5fb5\u4e3b\u8981\u7531\u4e00\u81f3\u5169\u500b\u5c08\u5bb6\u4e3b\u5c0e<\/mark>\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u7b2c 30 \u5c64\u7684\u5206\u6790\u986f\u793a\uff0c\u5c08\u5bb6\u53ef\u5728\u8f03\u5927\u7684\u9818\u57df\u5167\u5f62\u6210\u7d30\u5206\u5206\u5de5\uff0c\u4f8b\u5982 Java \u8207\u524d\u7aef\u7db2\u9801\u3001\u5e7e\u4f55\u8207\u7b97\u8853\u3002\u7814\u7a76\u4ea6\u6bd4\u8f03\u5c08\u5bb6\u8ca2\u737b\u7684\u96c6\u4e2d\u7a0b\u5ea6\u3001\u4e3b\u5c0e\u7279\u5fb5\u91cd\u758a\uff0c\u4ee5\u53ca\u5404\u5c08\u5bb6\u7684\u9818\u57df\u6a19\u7c64\uff1b\u9019\u4e9b\u7d50\u679c\u63cf\u8ff0\u7684\u662f\u7279\u5b9a\u6a21\u578b\u8207\u5c64\uff0c\u4e0d\u5b9c\u76f4\u63a5\u63a8\u5ee3\u5230\u6240\u6709 MoE \u6a21\u578b\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u91cd\u73fe\u6d41\u7a0b\u5f9e\u8a08\u7b97\u5c08\u5bb6\u5c0d SAE \u7279\u5fb5\u7684\u8ca2\u737b\u7d71\u8a08\u958b\u59cb\uff0c\u518d\u57f7\u884c\u6392\u540d\u3001\u4e3b\u5c0e\u5716\u3001\u8a5e\u96f2\u8207\u9818\u57df\u5206\u6790\u7b49\u5be6\u9a57\u3002\u9805\u76ee<mark>\u63d0\u4f9b\u6bcf\u9805\u5be6\u9a57\u7684\u57f7\u884c\u8173\u672c<\/mark>\u53ca\u5df2\u6574\u7406\u7684\u89e3\u91cb\u3001\u6a19\u7c64\u548c\u5224\u65b7\u8cc7\u6599\uff1b\u6a21\u578b\u6b0a\u91cd\u53ef\u5f9e <a rel=\"noopener\" target=\"_blank\" href=\"https:\/\/huggingface.co\/ByteDance-Seed\/Qwen3-30B-A3B-MoE-SAE\">Hugging Face<\/a> \u4e0b\u8f09\uff0c\u4e5f\u53ef\u6307\u5b9a\u672c\u5730\u6a94\u6848\u3002\u8a08\u7b97\u9700\u6c42\u4e0d\u4f4e\uff1aQwen3-30B-A3B \u7684 bf16 \u6b0a\u91cd\u7d04\u9700 61 GB\uff0c\u55ae\u5c64 fp32 SAE \u7de8\u78bc\u5668\u53e6\u9700\u7d04 8 GB\uff0c\u540c\u6642\u8ffd\u8e64\u4e09\u5c64\u5247\u9700\u8981\u5169\u5f35 80 GB GPU\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5206\u6790\u7bc4\u570d\u70ba Qwen3-30B-A3B \u7684\u7b2c 4\u300130\u300146 \u5c64\u3002<\/li>\n\n\n\n<li>\u4ee5\u7279\u5fb5\u8ca2\u737b\u800c\u975e\u55ae\u770b\u8def\u7531\u983b\u7387\u5224\u65b7\u5c08\u5bb6\u5206\u5de5\u3002<\/li>\n\n\n\n<li>\u63d0\u4f9b\u53ef\u91cd\u73fe\u5be6\u9a57\u7684\u7a0b\u5f0f\u78bc\u3001SAE \u6b0a\u91cd\u9023\u7d50\u53ca\u6574\u7406\u5f8c\u7684\u7814\u7a76\u8cc7\u6599\u3002<\/li>\n\n\n\n<li>LLM \u9818\u57df\u5206\u6790\u9700\u8981 OpenAI \u76f8\u5bb9 API\uff1b\u5b8c\u6574\u8ffd\u8e64\u6d41\u7a0b\u4ea6\u9700\u8981\u9ad8\u8a18\u61b6\u9ad4 GPU\u3002<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u5957\u65b9\u6cd5\u9069\u5408\u7814\u7a76 MoE \u53ef\u89e3\u8b80\u6027\u3001\u5c08\u5bb6\u5c08\u9577\u8207\u7279\u5fb5\u5206\u5de5\u7684\u5718\u968a\uff0c\u4e5f\u65b9\u4fbf\u91cd\u73fe\u7684\u5206\u6790\u3002\u5b83\u63d0\u4f9b\u7684\u662f\u91dd\u5c0d\u7279\u5b9a\u6a21\u578b\u7684\u7814\u7a76\u5de5\u5177\u8207\u8b49\u64da\uff0c\u4e0d\u662f\u53ef\u76f4\u63a5\u66ff\u4efb\u610f\u6a21\u578b\u6a19\u8a3b\u5c08\u5bb6\u7684\u901a\u7528\u670d\u52d9\uff1b\u89e3\u8b80\u7d50\u679c\u4ecd\u53d6\u6c7a\u65bc SAE \u7279\u5fb5\u8207\u81ea\u52d5\u89e3\u91cb\u6d41\u7a0b\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/bytedance-seed.github.io\/SAE-interprets-MoE\/\" target=\"_blank\" rel=\"noopener\"><strong>\u9805\u76ee\u4e3b\u9801<\/strong><\/a> \u00b7 <a href=\"https:\/\/github.com\/ByteDance-Seed\/SAE-interprets-MoE\" target=\"_blank\" rel=\"noopener\"><strong>GitHub<\/strong><\/a> \u00b7 <a href=\"https:\/\/huggingface.co\/ByteDance-Seed\/Qwen3-30B-A3B-MoE-SAE\" target=\"_blank\" rel=\"noopener\">\u6a21\u578b<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u53ea\u770b MoE \u5c08\u5bb6\u88ab\u8def\u7531\u591a\u5c11\u6b21\uff0c\u672a\u5fc5\u80fd\u77e5\u9053\u5b83\u771f\u6b63\u70ba\u6a21\u578b\u8f38\u51fa\u5e36\u4f86\u751a\u9ebc\u3002\u9019\u9805\u7814\u7a76\u628a\u5c08\u5bb6\u8ca2\u737b\u9023\u5230\u53ef\u89e3\u8b80\u7279\u5fb5\uff0c\u5206\u6790 Qwen3-30B-A3B \u7684\u5c08\u9577\u5982\u4f55\u7d30\u5206\u3002<\/p>\n","protected":false},"author":8,"featured_media":14624,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wpai_generated_summary":"","wpai_meta_description":"","footnotes":""},"categories":[133,176,76,185,153,137,38],"tags":[],"class_list":["post-14625","post","type-post","status-publish","format-standard","hentry","category-133","category-176","category-76","category-qwen","category-openai","category-api","category-38"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/14625","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=14625"}],"version-history":[{"count":3,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/14625\/revisions"}],"predecessor-version":[{"id":14630,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/14625\/revisions\/14630"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/14624"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=14625"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=14625"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=14625"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}