{"id":4014,"date":"2024-12-16T19:49:28","date_gmt":"2024-12-16T11:49:28","guid":{"rendered":"https:\/\/infernews.com\/?p=4014"},"modified":"2024-12-19T03:41:17","modified_gmt":"2024-12-18T19:41:17","slug":"speculative-decoding-%e6%8e%a8%e6%b8%ac%e8%a7%a3%e7%a2%bc%e6%8a%80%e8%a1%93","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/speculative-decoding-%e6%8e%a8%e6%b8%ac%e8%a7%a3%e7%a2%bc%e6%8a%80%e8%a1%93\/","title":{"rendered":"Speculative Decoding \u63a8\u7406\u89e3\u78bc\u6280\u8853"},"content":{"rendered":"\n<p>\u5927\u5e45\u63d0\u5347\u5927\u578b\u8a9e\u8a00\u6a21\u578b (LLM) \u7684\u63a8\u7406\u901f\u5ea6<\/p>\n\n\n\n<p>Cerebras \u548c Groq \u7b49\u516c\u53f8\u5229\u7528\u6b64\u6280\u8853\uff0c\u5728 Llama \u6a21\u578b\u4e0a\u53d6\u5f97\u9060\u8d85\u50b3\u7d71 GPU \u7684\u6548\u80fd\u8868\u73fe\uff0c\u901f\u5ea6\u63d0\u5347\u9054\u6578\u500d\u3002\u6587\u7ae0\u8a73\u7d30\u89e3\u91cb\u4e86\u5982\u4f55\u53d6\u5de7\u9032\u884c\u63a8\u7406\u89e3\u78bc\u7684\u539f\u7406\uff1a\u4f7f\u7528\u8f03\u5c0f\u7684\u6a21\u578b\u5feb\u901f\u751f\u6210\u521d\u6b65\u7b54\u6848\uff0c\u518d\u7531\u8f03\u5927\u7684\u6a21\u578b\u6821\u9a57\u5176\u6e96\u78ba\u6027\uff0c\u985e\u4f3c\u300c\u52a9\u7406\u5148\u6253\u8349\u7a3f\uff0c\u8001\u95c6\u518d\u5be9\u6838\u300d\u7684\u6d41\u7a0b\u3002\u6587\u7ae0\u5f8c\u534a\u90e8\u63d0\u4f9b\u4e86\u4f7f\u7528 Llama.cpp \u5be6\u4f5c\u63a8\u6e2c\u89e3\u78bc\u7684\u6b65\u9a5f\uff0c\u4e26\u8aaa\u660e\u4e86\u5176\u5404\u500b\u53c3\u6578\u7684\u610f\u7fa9\uff0c\u4ee5\u53ca\u5728\u5be6\u969b\u61c9\u7528\u4e2d\u53ef\u80fd\u9047\u5230\u7684\u6311\u6230\uff0c\u4f8b\u5982\u5076\u723e\u51fa\u73fe\u7684\u5ef6\u9072\u554f\u984c\uff0c\u4f46\u6574\u9ad4\u800c\u8a00\uff0c\u63a8\u6e2c\u89e3\u78bc\u80fd\u6709\u6548\u63d0\u5347 LLM \u7684\u6548\u80fd\uff0c\u5c24\u5176\u9069\u7528\u65bc\u9700\u8981\u5927\u91cf\u904b\u7b97\u7684\u61c9\u7528\u60c5\u5883\uff0c\u4f8b\u5982\u7d50\u5408\u601d\u8003\u93c8 (CoT) \u7684\u8907\u96dc\u4efb\u52d9\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><a href=\"https:\/\/www.theregister.com\/2024\/12\/15\/speculative_decoding\/\" target=\"_blank\" rel=\" noreferrer noopener\"><img data-dominant-color=\"7f7875\" data-has-transparency=\"false\" style=\"--dominant-color: #7f7875;\" loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"667\" src=\"\/blog\/wp-content\/uploads\/2024\/12\/shutterstock_games_consoles.jpg\" alt=\"\" class=\"wp-image-4042 not-transparent\" srcset=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/shutterstock_games_consoles.jpg 1000w, https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/shutterstock_games_consoles-300x200.jpg 300w, https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/shutterstock_games_consoles-768x512.jpg 768w, https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/shutterstock_games_consoles-640x427.jpg 640w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>\u5927\u5e45\u63d0\u5347\u5927\u578b\u8a9e\u8a00\u6a21\u578b (LLM) \u7684\u63a8\u7406\u901f\u5ea6 Cerebras \u548c Groq \u7b49\u516c\u53f8\u5229\u7528\u6b64\u6280\u8853\uff0c\u5728 Llama \u6a21\u578b\u4e0a\u53d6\u5f97\u9060\u8d85\u50b3\u7d71 GPU \u7684\u6548\u80fd\u8868\u73fe\uff0c\u901f\u5ea6\u63d0\u5347\u9054\u6578\u500d\u3002\u6587\u7ae0\u8a73\u7d30\u89e3\u91cb\u4e86\u5982\u4f55\u53d6\u5de7\u9032\u884c\u63a8\u7406\u89e3\u78bc\u7684\u539f\u7406\uff1a\u4f7f\u7528\u8f03\u5c0f\u7684\u6a21\u578b\u5feb\u901f\u751f\u6210\u521d\u6b65\u7b54\u6848\uff0c\u518d\u7531\u8f03\u5927\u7684\u6a21\u578b\u6821\u9a57\u5176\u6e96\u78ba\u6027\uff0c\u985e\u4f3c\u300c\u52a9\u7406\u5148\u6253\u8349\u7a3f\uff0c\u8001\u95c6\u518d\u5be9\u6838\u300d\u7684\u6d41\u7a0b\u3002\u6587\u7ae0\u5f8c\u534a\u90e8\u63d0\u4f9b\u4e86\u4f7f\u7528 Llama.cpp \u5be6\u4f5c\u63a8\u6e2c\u89e3\u78bc\u7684\u6b65\u9a5f\uff0c\u4e26\u8aaa\u660e\u4e86\u5176\u5404\u500b\u53c3\u6578\u7684\u610f\u7fa9\uff0c\u4ee5\u53ca\u5728\u5be6\u969b\u61c9\u7528\u4e2d\u53ef\u80fd\u9047\u5230\u7684\u6311\u6230\uff0c\u4f8b\u5982\u5076\u723e\u51fa\u73fe\u7684\u5ef6\u9072\u554f\u984c\uff0c\u4f46\u6574\u9ad4\u800c\u8a00\uff0c\u63a8\u6e2c\u89e3\u78bc\u80fd\u6709\u6548\u63d0\u5347 LLM \u7684\u6548\u80fd\uff0c\u5c24\u5176\u9069\u7528\u65bc\u9700\u8981\u5927\u91cf\u904b\u7b97\u7684\u61c9\u7528\u60c5\u5883\uff0c\u4f8b\u5982\u7d50\u5408\u601d\u8003\u93c8 (CoT) \u7684\u8907\u96dc\u4efb\u52d9\u3002<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"googlesitekit_rrm_CAowvqSiDA:productID":"","footnotes":""},"categories":[138],"tags":[],"class_list":["post-4014","post","type-post","status-publish","format-standard","hentry","category-138"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/4014","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=4014"}],"version-history":[{"count":0,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/4014\/revisions"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=4014"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=4014"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=4014"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}