{"id":13144,"date":"2026-10-10T03:02:21","date_gmt":"2026-10-09T19:02:21","guid":{"rendered":"https:\/\/infernews.com\/blog\/sparsedecoding-decoding-aware-pruning-for-accurate-and-efficient-llm-inference\/"},"modified":"2026-10-10T03:02:21","modified_gmt":"2026-10-09T19:02:21","slug":"sparsedecoding-decoding-aware-pruning-for-accurate-and-efficient-llm-inference","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/sparsedecoding-decoding-aware-pruning-for-accurate-and-efficient-llm-inference\/","title":{"rendered":"SparseDecoding \u91dd\u5c0d LLM \u89e3\u78bc\u52a0\u901f\u81f3 1.48 \u500d"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/10\/SparseDecoding-LLM-1.48.jpg\" alt=\"Og image\"><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u5927\u578b\u8a9e\u8a00\u6a21\u578b\uff08Large Language Model\uff0cLLM\uff09\u9010\u500b\u751f\u6210 token \u6642\uff0c\u5ef6\u9072\u5f80\u5f80\u53d7\u8a18\u61b6\u9ad4\u8b80\u53d6\u9650\u5236\uff1bSparseDecoding \u4ee5\u89e3\u78bc\u968e\u6bb5\u70ba\u6838\u5fc3\u91cd\u65b0\u8a2d\u8a08\u526a\u679d\u6d41\u7a0b\uff0c\u6e1b\u5c11\u9700\u8981\u8b80\u53d6\u7684\u975e\u96f6\u53c3\u6578\uff0c\u8655\u7406\u7a00\u758f\u6a21\u578b\u52a0\u901f\u8207\u8f38\u51fa\u8cea\u7d20\u4e4b\u9593\u7684\u53d6\u6368\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u50b3\u7d71\u526a\u679d\u65b9\u6cd5\u901a\u5e38\u5f9e\u56fa\u5b9a\u81ea\u7136\u8a9e\u6599\u6536\u96c6 calibration activations\uff0c\u518d\u8a08\u7b97 Hessian\uff1b\u4f46\u6a21\u578b\u751f\u6210 token \u6642\u6703\u4f9d\u8cf4\u81ea\u5df1\u65e9\u524d\u7522\u751f\u7684\u5167\u5bb9\uff0c\u5169\u8005\u5206\u4f48\u4e0d\u4e00\u81f4\uff0c\u4ee4\u526a\u679d\u5f8c\u7684\u8868\u73fe\u53d7\u5f71\u97ff\u3002SparseDecoding \u6539\u70ba\u5728 dense model \u9032\u884c autoregressive generation \u6642\uff0c\u6309\u5c64\u6536\u96c6\u6a21\u578b\u81ea\u884c\u751f\u6210\u7684 activation\uff0c\u4e26\u7565\u904e prefill \u968e\u6bb5\uff0c\u8b93\u6821\u6e96\u8cc7\u6599\u66f4\u8cbc\u8fd1\u771f\u6b63\u89e3\u78bc\u6d41\u7a0b\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u7cfb\u7d71\u90e8\u5206\u5247\u91dd\u5c0d\u89e3\u78bc\u4e3b\u5c0e\u7684 sparse matrix-vector\uff08SpMV\uff09\u904b\u7b97\uff0c\u52a0\u5165\u652f\u63f4 N:M sparsity \u7684 bitmask indexing \u8207 fixed-step traversal kernel\u3002\u9019\u548c\u4e3b\u8981\u512a\u5316 prefill \u7684 sparse matrix-matrix multiplication\uff08SpMM\uff09\u65b9\u6848\u4e0d\u540c\uff0c\u5f8c\u8005\u5728\u90e8\u5206 autoregressive decoding \u60c5\u5883\u4e0b\u672a\u5fc5\u80fd\u8d85\u8d8a dense throughput\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5be6\u9a57\u6db5\u84cb Llama-3.1-8B\u3001Llama-3.3-70B \u53ca Qwen3-14B\/32B\uff0c\u5728 WritingBench \u548c ClassEval \u7b49\u6e2c\u8a66\u4e2d\u7dad\u6301\u5177\u7af6\u722d\u529b\u7684\u7d50\u679c\uff1bA100 \u4e0a\u7684\u7aef\u5230\u7aef decoding speedup \u6700\u9ad8\u9054 1.48\u00d7\uff0c\u4e0d\u540c\u7a00\u758f\u6a21\u5f0f\u5247\u7d04\u70ba dense \u6a21\u578b\u7684 1.34 \u81f3 1.48 \u500d\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u4ee5\u6a21\u578b\u81ea\u884c\u751f\u6210\u7684 token \u4f5c decoding-aware calibration<\/li>\n<li>\u7565\u904e prefill activation\uff0c\u805a\u7126 autoregressive decoding<\/li>\n<li>\u63d0\u4f9b\u512a\u5316 N:M SpMV kernel\uff0c\u652f\u63f4 bitmask indexing<\/li>\n<li>\u5728 50% \u6b0a\u91cd\u4fdd\u7559\u7387\u53ca 2:4 sparsity \u4e0b\u6e2c\u8a66<\/li>\n<li>\u76ee\u524d\u5c6c\u65bc preprint\uff0c\u7a0b\u5f0f\u78bc\u6a19\u793a\u70ba\u5373\u5c07\u63d0\u4f9b<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wang-qitong.github.io\/SparseDecoding\/\" rel=\"noopener noreferrer\"><strong>\u9805\u76ee\u4e3b\u9801<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>SparseDecoding \u91dd\u5c0d\u751f\u6210\u968e\u6bb5\u91cd\u65b0\u6821\u6e96\u526a\u679d\uff0c\u517c\u9867\u6a21\u578b\u6e96\u78ba\u5ea6\u8207\u89e3\u78bc\u901f\u5ea6\u3002<\/p>\n","protected":false},"author":8,"featured_media":13143,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wpai_generated_summary":"","wpai_meta_description":"","footnotes":""},"categories":[76,127,185,197,40],"tags":[],"class_list":["post-13144","post","type-post","status-publish","format-standard","hentry","category-76","category-127","category-qwen","category-framework","category-llama"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/13144","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=13144"}],"version-history":[{"count":0,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/13144\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/13143"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=13144"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=13144"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=13144"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}